5 Steps to Build a Great Drupal Team: Step 5: Evaluating Applicants
Entire books have been written about hiring practices, and methods for evaluating candidates. At the end of the day, there are some core steps key to building a great team. This post focuses on the the fifth and final step: evaluating the Drupal-specific aspects of an applicant's qualifications.
- Step 1: Understand typical teams, roles and job titles
- Step 2: Define your requirements
- Step 3: Widen your net with your job description
- Step 4: Where and how to post jobs
- Step 5: Evaluating applicants
Step 5: Evaluating applicants
In the earlier steps in this series, we have itemized the kinds of skills as they relate to roles, and ways you can define your requirements. Those will all become useful components in your evaluation. Most importantly, when looking for qualified candidates you need to look at the big picture. There is no single data point that is going to clearly say: Hire me!
Thanks to my colleagues: Eric Gaffen, Global Manager, Talent Acquisition; Ian Shaw, engagement manager here at Acquia; and Meagen Williams, Program Manager at Acquia, for their contributions to this article.
Use a variety of methods
There are a variety of data points available for evaluating potential hires:
- Code review
- Portfolio review
- Community involvement
Taken individually, each method for evaluation has a weakness. Using references might be a good source of information, but it does have weak points. “It’s human nature to be nice when talking about someone else,” says Ian Shaw, engagement manager who also acts as a hiring manager here at Acquia.
Instead, employ a variety of techniques in your evaluation process. Ian says, “If you think about breadth and depth; I prefer to go with breadth… Interviews are so far removed from everyday life,” says Ian, “I’m in favor of getting as close to the real thing as possible.” Giving people practical problems to solves lets you see how they approach a problem.
Interview techniques to bring the best out in people
Meagen Williams is a program manager here at Acquia, and also a hiring manager, responsible for vetting technical candidates. Meagen advocates taking “an inquisitive approach, rather than an inquisition.”
You could ask questions about a particular feature of a site in their portfolio and ask questions which give you insight into how they approach projects. Meagen suggests asking questions such as:
- "Explain to me why you made this choice as opposed to that choice.”
- “What were the other options?”
These open ended questions give you a chance to gain insight into their breadth of knowledge or decision-making process.
In this way, you can follow a pattern of digging deeper and deeper into a topic, without resorting to yes/no or right/wrong answers. You begin to see at what point their thinking breaks down, or they reach some technical dead end. If they are very experienced, you probably won't find an end. This lowers tension in the interview.
“One of the most skilled interviewers on my team is a cheerleader. Even when it's clear you're not a good fit for a role, he tells them in a straightforward way. You feel like, ‘Hey I might not be a good fit, but he likes me, I have promise’.”
This method gives people ways to succeed, “This is a way to get the best out of people,” during interviews she says.
Provide case studies and problems to solve
Another approach during interviews is to prepare case studies to have interviewees give feedback on. At Acquia, when we hire technical consultants for our Professional Services team, we use a detailed scenario-based interview. We use scenarios based on our own experiences, for example on change control, “Imagine you’ve got one week left on a project,” and then introduce your problem. This helps a hiring manager see how candidates articulate themselves under pressure and how pragmatic or diplomatic they are, in addition to technical knowledge.
The more you tailor your scenarios to ones you face day-to-day, the easier it will be for you to evaluate responses.
Understanding community involvement
Involvement with the community and contributions are good indicators of not only passion, but also knowledge of best practices. So make sure you ask applicants to give you their Drupal.org profiles. However, don’t let that limit your choices, many great developers haven’t been able to develop their profiles.
When you review candidate profiles you can look at data available.
- Click on the “Posts” tab of their profile which shows how much interaction they’ve had on the site. You might be able to see if they posted only a few questions, or if they’ve helped others.
- Review their list of projects. If you’re hiring for a senior developer or team lead you might be able to to see some projects here.
- Under History, check Documentation: This shows the number of edits they have contributed to documentation.
- Look at their attendance at events, are they actively participating? You can ask if they were volunteering or speaking at various events.
Having a robust profile does make it easier to review. With that said, even at Acquia we’ve hired many really great people who didn’t have robust Drupal.org profiles when they applied.
There are many cases where people don’t have the opportunity to contribute either due to restrictions with former employers or for lack of personal time. If you limit yourself to only people who have high levels of contribution, you could be excluding many great candidates. Researcher Ashe Dryden, who focuses on diversity in the workplace, explained why we see so little diversity in open source communities, “Marginalized people in tech... have less free time for a few major reasons: dependent care, domestic work and errands, and pay inequity.” (Ethics of unpaid labor in the OSS community, 13 Nov 2013.)
As with most things, you want to look at the whole picture. And of course, when you do bring on new staff, give them time to develop their expertise through contribution. Overall, it will raise their profile and your organization’s profile in turn. This can make future hires much easier, as we explained in the previous step.
Grit over experience
It’s understandable that you need to bring in experts who have specific knowledge and experience to bring to your project, particularly if you’re hiring for more senior roles. As I wrote in Step 2: Define your requirements, make sure you’re not scaring people away with your extensive requirements.
Impostor Syndrome is rife in the field of web development. This affects very skilled people who feel that at any point they’ll be “found out” as being frauds. There is simply too much too know, and the technology is changing constantly. It’s difficult for even the most talented web developer to say with confidence “I’m an expert” in anything. It’s something we also run into at Acquia. "We look for the best of the best, and that scares people… they have some modesty,” says Eric Gaffen, Global Manager of Talent Acquisition for Acquia.
Eric points out that Malcolm Gladwell's book Outliers posits that it would take about 10,000 hours, or 7-10 years, to master a skill. "When's the last time a technology lasted that long?" Eric asked.
Instead, Eric said it’s worth looking at someone with two years of experience, “Consider where are they on the continuum of becoming a great technologist?” He points out there are other benefits with choosing someone with less experience, “Sometimes giving an opportunity to someone with less experience can allow you time to help them adapt to your own culture.”
Eric believes the single biggest factor in success is one’s internal motivation and desire and drive. He looks for these qualities during the initial interviews:
- Strong drive
- Energy and enthusiasm
During the interview, dig for examples of when they have faced failure because they learned from that failure. Eric said, “I want to hear persistence, who isn't going to give up, and focused.” He looks for people who are willing “to try different things and be adaptable, “ he said.
Tools for evaluating candidates
One method for evaluating technical skill is to do a review of existing work; for example by looking at given code. As well, there are numerous sites where you can review candidate’s contributions and activity, such as Github, or social media profiles. If you don’t already have someone expert on your tea, and limited time, these options pose a challenge.
You could use Acquia’s “Instant Insight” check on an applicant’s site, and you don’t need to be logged in. This will give you a high level overview and raise any obvious alarm bells.
Gild.com offers a tool which collates relevant information about a candidate’s technical skills. There is a powerful search tool for subscribers to identify candidates. "The traditional markers people use for hiring can be wrong, profoundly wrong," says Vivienne Ming, the chief scientist at Gild. She thinks “that talented people are ignored, misjudged or fall through the cracks all the time. NY Times, April 2013.
Gild is also aggregating data and helping employers “cast aside preconceived and wide spread biases,” for example about what universities people went to, and increase the chances of talented people finding their dream job. (Gild best practices, 13 Nov 2013).
Evaluating technical expertise
If you do have someone technical on your team, you have a few more options for evaluation, but there are ways non-technical hiring managers can evaluate technical expertise.
Program manager, Meagen Williams likes to ask technical candidates questions such as ‘What does beautiful code look like?’ From this she can get a sense that “this is somebody who is thinking about the code they create,” Meagen said. “I'm a relatively non-technical person, and they should be able to explain it to me... I'm not shy about saying 'I don't know.' ”
An alternative approach is to give applicants a short code sample and ask them to review it, for example to spot patterns, best practices or errors. Then you can mark how many of the elements they pick up. However, on-the-spot coding tasks or “whiteboard coding” are notorious for not yielding an accurate representation of how developers code, let alone solve problems.
Instead, you could pair the candidate up with someone on your team. Using pair-programming techniques they can work together on a problem. You can then see what kinds of questions they ask, and how they approach the problem. Ashe Dryden, researcher in more equitable hiring practices, suggests that this “Pair as an interview” approach also helps create a more level playing field for individuals who have been otherwise unable to provide code samples due to NDAs or otherwise.
Looking at the big picture
Experience is a good indicator of quality and dedication. With that said, you should also consider the possibility that hiring someone with less experience but a strong willingness to learn is going to widen your pool of candidates, and offset the costs of on-boarding or training. Going back to our earlier points about good communication and willingness to collaborate, those are qualities which are difficult to develop.
Technical skills are only one aspect of an employee’s performance on a Drupal team. “Drupal is about open source, community and collaboration. This lends itself to an agile way of doing things: expect change,” says Ian Shaw, Engagement Manager here at Acquia. “Communication is important when there are so many things going on at the same time.”
“I worked on a project where the client preferred working with a junior developer who was more likely to ask questions, even if it took him a little longer; rather than a senior developer who they felt was too arrogant to ask questions,” Ian said. As it turns out, having the bravery to simply say “This might be a stupid question… but…” can save money down the line, and that comes down to communication. Personal qualities like that are difficult to develop, whereas technical skills can be improved.
We hope this series has helped you get a better idea of how to build your Drupal team. We’ve reviewed the roles you might be looking for; associated skills; best places to advertise your job and finally, how to evaluate your candidates.
First we focused on outlining the typical teams, roles and job titles common among companies using Drupal. Next, we emphasized clearly defining realistic requirements. After that you can work on your job description for your listing. Finally you can starting posting your jobs where they can be seen by active, experienced developers. Hopefully this brings you excellent candidates, and you'll be working hard on evaluation.
Throughout this series we've emphasized widening your net and making opportunities for people with less experience. We've learned plenty at Acquia through our own mistakes, and realizing that a good personal fit is even more important than technical expertise. Instead you can invest in professional development and give your staff time to develop their technical mastery. We'd love to help you develop your team. If you'd like to talk with us about training or mentoring for your team, please get in contact.
If you’d like to know more and talk with one of our experts here at Acquia, please sign up to our online session with guest speakers, Summer Swigart, Practice Manager, and Meagen Williams, Program Manager.
Sign up to our training newsletter
Sign up to our training newsletter to be notified about more tutorials for you. Please also check out the Drupal Training events calendar