I'm seeing a lot of comments about hiring on my Twitter feed that read like this:
The answer is no, and in the context of an interview, it's a bad question. So how can you and I fix this? How do we, as developers, get companies to stop doing this?
The fix is for you to, after you get hired (fingers crossed), join the system, become part of the machine, and fix it from the inside. (It's probably easier than you think.) Every company will have a team of engineers whose job it is to set "hiring policies and procedures" (or PnP) specific to developers. This isn't HR type stuff, like don't ask questions about religion. This group figures out what topics to cover, what questions to use to cover those topics, the flow of the interview, screener questions, take-home tests (sigh), and all that.
Your job (should you choose to accept it) particularly if you are a person of color or a woman in tech, is to get on that team. And it won't be hard. First, nobody really likes being on those committees. And second, companies are looking to get more diverse representation on their teams so having you on the team will be a "big win". Even if the meetings are boring AF, believe me, you are part of the fix, so tough it out.
Doing Away With Binary Tree Questions
In the meetings you'll talk about the questions, and you'll get to the binary tree question and the exchange will go like this:
You: So, whose question is this?
Joe Dev: Oh, that's mine.
You: Cool, what are you getting out of this binary tree question?
Joe Dev: Well, I've been asking it for years, so I know the right answers and I know how to evaluate folks based on it. And it's never been wrong.
You: Ok, but what technical info do you actually get out of it, because we don't write binary trees.
Joe Dev: Well, I get to see how someone thinks.
Now all of that may be true. And given unlimited time maybe that could be a reasonable question. But time is limited, and could we make better use it because these types of questions have a number of meaningful failings:
- They don't scale - If we make it a standard question that every team has to use, but only one person really knows how to ask that question the right way, how is that gonna work?
- They don't provide any real world insights - At the end of the question we still have no more data on whether the candidate can actually do the job we are hiring them for since we didn't ask them a job skills related question.
- They are unnecessarily stressful - Interviewing is tough and stressful already. And asking a question that the candidate wouldn't likely prepare for just adds to that.
- They are biased towards recent CS graduates - This kind of question is going to mess up code school graduates mostly. Since code schools, oddly, emphasize practical real world skills.
- They are a form of take home test - Prepping for these types of questions requires after hours study, which is exclusionary to anyone with time constraints outside of work.
I could go on and on, but right there you have a really decent set of arguments against "bad" questions, and you also have a pretty good litmus set for evaluating other questions. And that's really what you want to do, you want to set up a bunch or criteria to evaluate questions; what do we learn from it? can anyone run this question? etc. Because people will always want to use "binary tree" questions because they did that "at the last job" and they are "comfortable with it". So having a systematic way of evaluating interview questions is good.
It's also good to take on "comfortable with it" or "always done it this way" reasoning. Often the point of re-evaluating hiring PnP is that the company lacks diversity, probably acutely in tech, and doing things the "same old way" will simply result in more of the same.
Don't Stop There
There are lots of other bad types of questions. For example, in my experience managers/directors/VPs love to ask "Car Talk Puzzlers". It's basically just a non-tech version of the binary tree question, and it's just as worthless. The solve there is to just re-engage with the purpose of the committee; "Hey, you established this committee to help make our hiring more diverse. Your question is covered by other questions we ask, maybe you should take the candidate to lunch and do soft skills questions that we discussed and you yourself championed as critical..."
A few more things to fix:
- Take home tests - We should also take a hard look at "take home tests" since they are exclusionary as well since the bias against candidates that have time constraints outside of work.
- Evaluating their Github commit graph - Just like take home tests , it strongly biases towards people that have a lot of free time and who choose to use it to contribute to open source.
- Requiring Google-free live coding - This one is just... Sigh. Everyone knows Stack Overflow going down is the tech equivalent of a snow day. Being able to find answers on Google is a skill, not a deficit.
- API trivia night - How many arguments to Array.splice take and what are they? Really? C'mon.
- Cultural Fit - Thar be dragons with cultural fit questions. Interviewers can easily fall into mini-me bias. If you have "know it when you see it" that's a big red flag.
The Question We Should Ask But Mostly Don't
I'm a huge believer in the idea that, at some point during the interview, everyone should have a chance to show their stuff and to shine. For me that comes down to having one segment where we go over a project that the candidate was proud of, be it personal or professional, and we walk through the architecture together.
Then we change one of the pillars of the architecture (e.g. "This game is single player, how would you make it multi-player?") and work through the changes required to make that happen.
Yes, it's an open ended question, but it's home field advantage for the candidate and that puts them at ease. As a result, you will see them at their best. Most people answer with what they've been working on most recently, so they have it top of mind, and they know all the little details and ins and outs.
Just Get On The Committee
Listen, I'd gladly hand you my seat on a hiring PnP team if you wanted to bring more sanity and practicality to the hiring. I didn't go to college. I don't have a degree in CS. And I don't want to spend my home time re-learning basic algorithms for the 3,000th time just for interviews so that I can forget them again a week later.
So get on the damn committee and change this stuff. We could do so much better. And we can. With your help. Decisions are made by those who show up.
I hear this a lot; "I ask this question, because we asked this question at Google and Google hiring was great." First off Google changed its hiring policy forever ago. But if you are into folksy anecdotes here is a good on for just this type of thing.
Mom is looking at Grandma's family recipe for Thanksgiving ham and it says; "Cut the ends off before putting in oven." She's got this beautiful ham with two good ends, and she wonders why she has to do this. She calls Grandma, who says, "I cut the ends because my oven was too small."
Point is; question everything. And question reasoning that "what worked at company X will work here" because we don't know all the thinking that went into their process. So let's think about what we want out of ours and actively create a better process that works for us.