Home / Comment permalink

A skeptic plans a certification program

One of my responsibilities at [Acquia](http://acquia.com) is overseeing the development of our [certification program](http://acquia.com/wiki/projects/yellow-jersey). We're creating this program to establish baseline credentials for working with our commercially supported distribution of [Drupal](http://drupal.org) (codenamed [Carbon](http://acquia.com/projects/wiki/carbon)\) and our [Spokes](http://acquia.com/projects/wiki/spokes) and [Caliper](http://acquia.com/projects/wiki/caliper) network services. Since Carbon is a distribution of Drupal core and a selection of modules, Acquia certification will judge facility to work with Drupal independent of our distribution.

I was not surprised that the Drupal community is [suggesting caution](http://groups.drupal.org/node/10007) when it comes to our effort and the value of certification in general. I've been a certification skeptic since I first started working with Certified™ technical staff. There are too many "paper" certifications in the IT community, and I'm not interested in adding another to the pile. To tell the truth, I'm a bit of a skeptic when it comes to technical training as well--I remember someone close to me telling me so few people failed his classes because "their boss didn't send them to fail."

I've had to come to grips with my skepticism over the past few years as I've had managers ask me to get Certified™ and I've developed and delivered training programs. What I've realized is that it's not really about the coursework, the instructor, or the exam. It's about the student. There are a lot of different approaches certification, and different attitudes to take with you to a training class. There's also a lot of ways to think about the value of being Certified™.

In any given field, the available talent tends to follow a [bell curve](http://en.wikipedia.org/wiki/Normal_distribution). Way over at the right of the curve, certification probably doesn't add much value. People at this level will either be well enough known, or have a strong enough network, that they won't need much in the way of outside credentials to establish themselves. Way over at the left, certification probably doesn't add much value for the candidate (I'm assuming they'd fail), but it does serve to weed them out.

In the middle of the curve is where it becomes more interesting. Here, certifications help distinguish one from the other on an objective scale. The scale is probably not the most important one for selecting an employee or vendor, but it does help to quickly narrow the focus (and thus the effort) for vetting soft skills and other intangibles. The key is designing the scale to minimize the false positives and false negatives.

[Robert Douglass](http://robshouse.net), who owns the certification effort on my team, shared [his success with Java certification](http://acquia.com/blog/certification-success-story) in the [Acquia team blog](http://www.acquia.com/blog). He used the [Sun Certified Java Developer](http://www.sun.com/training/certification/java/scjd.xml) program to organize his efforts to learn Java. Essentially, he moved himself along the bell curve with perspective employers by getting certified. The certification helped two ways--it provided him with a path for his learning and established to the Java community that he had the basis for becoming a successful developer.

How do we design the scale for our certification so that it doesn't become a "paper certification" (too many false positives)? The key challenge is in identifying what we're testing for an how to evaluate those skills. As an example, we could test "book knowledge" with questions like:

> What is the purpose of `hook_enable`?
> a. install the current version of the database schema
> b. perform module setup tasks
> c. called before a node is shown on a form
> d. defines a set of access restrictions

and with sufficient cramming a certification candidate could pass a test like this with no understanding of what a hook is, or why they'd care. This is a great way to devalue the differentiator by flooding the market with "paper" certifications. This is not the certification we want when we build [Yellow Jersey](http://acquia.com/projects/wiki/yellow-jersey).

I want a certification that tests more than just facts and memorization; an exam that tests the knowledge and wisdom that only comes from practical experience. I have a few ideas about this, but I'm not sure of the best way to go about it. We've met with some experts, and I've heard a lot of advice about emulating [Cisco](http://www.cisco.com/web/learning/le3/learning_career_certifications_and...) or [Sun](http://www.sun.com/training/catalog/courses/PK-CSPJ-W02.xml). I haven't studied ether, but I understand that they involve not only multiple choice tests but also a practical component. I think this is necessary, but I'm worried that the cost may be prohibitive in the short term.

I think there's also a lot we can do in terms of developing the multiple choice questions to probe a deeper level of understanding. Questions on performance, for example, that require the test taker to evaluate a set of requirements against a set of deployment and caching options. Or questions that ask for an evaluation of a segment of code to explain what it's doing and why.

Keep an eye on the [Yellow Jersey](http://acquia.com/projects/wiki/yellow-jersey) to follow how it works out.



Posted on by Josiah (not verified).

I'm looking forward to how you answer this question. I'm glad to know you don't want to see paper certs, but rather certs that are evidence of meaty understanding of drupal. I'll be looking to get in line for mine. :-)

Posted on by Joseph Bachana.

Thanks for this post, Chuck. At my company everyone has to hold an industry certification, so we've got experience with a whole lot of tests. My personal one is the PMP (Project management professional) exam and recurring continuing education credits. Staff hold anything from MCSD/MCAD from Microsoft (multiple tests), ZEND, Java, and even a few staffers have Quark and InDesign certifications.

What we all seem to want out of a certification program is more of a 'way of life' -- some place we can go to keep getting continuing education and to re-enforce what we know. Having tests or quizzes to go along with those milestones are great for us since they are kind of like guideposts.

I admit that we have hired a few rare programmers that were certified in something but somehow not competent. Usually they do not have good judgement or common sense about problem solving. However, those issues should be identified during a job interview/screening. If we've done a great job with our hiring efforts, then continuing certification is just one way for all of us to measure our progress.

One suggestion: Perhaps you send out some questionairre to all the partners asking what has worked and what has not in certification exams that they have taken?

Another suggestion: You should have an opt-out of certification for 'master practitioners' that have developed modules already. I think it will annoy the great Drupal ninjas out there to have to take a certification exam, but you can't just grandfather people in with qualitative decisions. A fair way is that if the person has created something within Drupal that has met with approval by the community, they should have some 'black belt' status, or whatever you end up calling it.

And finally, I don't think there can be just ONE certification for Drupal since there are a number of things you can specialize in -- so you might consider breaking this into smaller cert exams.

Good luck Chuck, you're on the right track!

Posted on by Dan Frydman.

Thanks for the info on this. It would be great to have a page on the site that has an overview of what your hoping to include in the certification - rather than just a discussion. What are the modules (no pun intended) that you'll be covering?

  • How will the test be set up?
  • What's the cost of the materials and training?
  • Will it all be available online?
  • Will there be different levels of accreditation?
  • I guess the discussion is all that you can show at the moment, though it makes things hard to plan (for us project management / strategy types) when it comes to what might be useful for staff training requirements.

    I'm all for the training being all about the student and not about the company, but a trained employee who knows that they've done the course / training for their benefit and for the wider benefit of the company will feel that it's then worth contributing back.

    Dan Frydman
    Inigo Media
    Edinburgh, UK