IT Certification and Training

June 9, 2011

Job Changes

Filed under: Uncategorized — Jim Henderson @ 12:29

Another few months have passed, and things have changed pretty dramatically for me.  Novell was acquired by Attachmate Group in late April, and in early May, I was one of the people who was made redundant.

So I’m now in the market for a new job.  I’ve spent the last month taking some online classes and looking at different opportunities.  I’ve also been thinking back on my time at Novell and the things I learned about certification, testing, and training.  Turns out I’ve learned a lot of new things, and have  an appreciation for certification that I didn’t have when I started.

Nothing brought that to the forefront more than an “Ask Slashdot” posting about which certifications are best.

My answers to this question are buried in the comments, so I’ll reproduce them here.  The first is in response to this comment suggesting that one should create their own certification.

— snip —

The parent here is perhaps meant to be funny, but there is a nugget of truth in what he says.

Actually creating a certification takes a lot of work – I spent the past 5 years working as part of the team that worked on IT certification programs and exams at Novell. But to understand what certifications hold value in the industry, it does help to understand the process by which a program is created, because if a program isn’t built around sound principles, then the certification will be worthless as anything other than a wall decoration.

First, you have to certify based on something people actually do. Certifications that have real value start with a job task analysis (JTA) and the program is built around what people actually do for a living. It doesn’t do you any good to certify based on criteria that don’t map to a specific job function.

Second, the testing methodology needs to be sound. People laugh about paper certifications, but paper certs are a real problem in the industry. This can happen because a question pool is leaked and a ‘braindump’ is created. Dealing with braindump sites is like playing whack-a-mole. So the testing methodology should resist braindumps, either through adaptive testing or through the use of performance based testing (sometimes called ‘practical testing’ or some variation of that). Practical testing tends to be more resistant to braindumps because that type of resource gives you the answer – but in a practical exam, you have to demonstrate the application of the answer. So if the braindump tells you “do x, y, and z”, those are the steps you need to do to complete the tasks.

If a certification is ISO 17024 compliant, then it has increased value as well. That ISO standard specifies a number of things (which are adopted by other organisations, like ANSI) about how a certification is built. Vendor-specific certifications tend to not be ISO 17024 compliant (there are a few exceptions) sometimes because of cost or resource requirements. As I understand it, there are pieces of the standard that specify, for example, that the people who create the exam and the people who create the course materials cannot talk with each other about the content. The JTA information can (I think, it might be required or recommended) be shared between the two groups, but they must derive their own information from the pool of information about the topic. The purpose for this is that it’s the knowledge that’s needed, rather than the specific course materials created by the certifying body. In some cases, the certifying body just publishes the objectives and leaves it to others to create the courses around those objectives.

I’m also of the opinion that the value is higher if rather than relying on recall for answers, the exam requires cognitive skills. Exams like this tend to be much more labor intensive to create and evaluate properly to ensure they’re fair, but that value is significant as well because then the certification shows that the candidate knows more than just the answer to the questions on the exam, but how to apply their knowledge in a useful way. Performance-based tests are really the best way to do this in my opinion.

The exams also must have gone through some form of psychometric analysis in order to be legally defensible. If a program uses multiple exam forms (which is generally the case), then the psychometric analysis is used to ensure the forms are fairly balanced and if a candidate can pass the exam on form 1, that they would most likely pass it on the other forms as well.

Thirdly, a properly built certification program is going to have continuing certification requirements. Some organizations (like CompTIA) used to certify “once and forever”, but certifications like that really don’t have that much value over the long term. I hold an LPIC-1 certification that I got in 2003, but that doesn’t really tell anyone what I know about modern Linux distributions.

Certifications are helpful if you’re going through the ‘front door’ trying to find a job – applying through a website or through a company’s HR department. Certifications (like other things you might list on a CV or resume) are keywords that will get your qualifications looked at. More and more companies are using automated systems to sort through submitted CVs/resumes, so it helps to know what the company is looking for. But as others have said, it’s better to work through an ‘insider’ (ie, someone you know at a company) to get a leg up on a job. That said, positions that I’ve been looking at have had an emphasis on project management, so PMI’s PMP or CAPM certifications would be considered valuable. In the IT field, PMI’s certs are useful, Six Sigma certification might be valuable depending on the industry, or looking at something like Novell’s CLP/CLE, RedHat’s RHCT/RHCE, and some of the Microsoft certifications are sought after (though from MS probably not the MCSE as much any more; they have some programs that are ISO 17024 compliant and those would be seen as being more valuable). LPI’s LPIC-* certs are also valuable, and the more current CompTIA certifications also have value, though myself I’d look at the more advanced certifications both organisations offer.

— snip —

I’d been thinking in terms of Linux and non-IT-specific certifications, and clearly missed out on mentioning the Cisco and Citrix certifications.

My second response was to this comment about the overall tenor of the comments being “certification is worthless”:

— snip —

I think [the people who said certification is worthless] posted based on their own experiences in the field of IT. I also have a certain amount of cynicism about the value certifications; I didn’t earn any until after I started with Novell 8 years ago.

Of the ones that I took, the exams that I found to be the most challenging (and rewarding to pass) were the performance-based exams associated with the Novell Certified Directory Engineer (which was definitely a cert worth earning when it was available) and with the Novell Certified Linux Professional. RedHat’s exams are similar, and I know that their focus is on doing only performance-based testing (to my knowledge, they don’t have any exams that are traditional forms-based exams).

There are a lot of people who create certifications who really don’t understand what the goal is, and they don’t go through the process of a proper psychometric evaluation of their exams, which means the tests aren’t fair or an accurate representation of what a person knows. The fact that there are braindumps available for so many exams also devalues the certification because people rely on short-term memorisation of an answer key rather than actually learning what is needed to pass the exams, and eventually the assumption becomes that anyone who earned the certification likely earned it not by learning the actual content but rather by taking the ‘easy way’ and just learning what was on the exam.

The goal of any properly created certification (or exam) is to demonstrate competence for the minimally-qualified candidate. It sets a bar, but particularly at the entry level, it is only for the candidate at the lowest knowledge or skill. Experience counts more for a lot of employers (and I depended on that for years), but there are some (and I interviewed with one 15 years ago myself) who get hung up on the candidate not holding a certification, so not having it can be a barrier (or for me, not having it was a qualifier for a prospective employer – if the cert was more important to them than my experience, I didn’t want to work for them).

Clearly, though, there are fields where certification is mandatory. To practice medicine in the US, for example, you need to be board certified. To work on cars, you generally need your ASE. To be an accountant, you need to be a CPA. To practice law, you have to pass the bar exam (which is a license, but licensing and certification are closely related).

The original purpose of IT certification (which Novell started in the industry) was to provide technical support resources outside Novell’s support organization who were competent to support NetWare – the sales growth Novell saw at the time meant they couldn’t build a large enough support staff to adequately provide technical support for the product. The CNE was created largely for partner organizations to prove to customers that their staff understood NetWare well enough to support it, and at that time (back in the mid- to late- 80’s) it was required for people to provide support on the product.

— snip —

I think these two comments sum up a lot of what I’ve talked about on this blog to date.

As for my future, we’ll see what it brings.  If you know of someone who would benefit from my experience, please drop me a line.


Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create a free website or blog at

%d bloggers like this: