Imagine you have to hire someone for your company but you cannot use interviews as a form of selection. You have a pool of four finalists. Each meet the minimum requirements. Each person is entirely unknown to you. And it’s a key role. Your performance in the company will be dependent on the performance of this individual. Hire well and you will see some real gains. Hire poorly and you might find yourself demoted.
Again, you cannot use interviews. You cannot speak with the person, meet them, see them, or hear their voice. No video, no audio.
What would you do? How would you select the best person?
I, for one, would throw a battery of tests at the candidates. I would have them complete a work assignment, take a cognitive test, a personality test, and provide written answers to structured questions. The first set of structured questions would be in essay form. The second set, as a final round, would be an IM exchange.
This would not all happen at once, mind you. I’d have the candidate do this over a span of three rounds. I think the pacing is important.
And I would do the tests myself. I’d write my own answers before ever issuing it to others so that I could compare myself to them.
You know, actually, I’d have a whole selection committee do this. It would be comprised of another colleague of mine, two teammates of the potential candidate, and a subordinate that would work for them. That’s me and four other people total.
We’d all develop our answers, compare notes, find a shared paradigm, and review candidates for their ability to either provide better answers than ours or, at minimum, give answers that strengthened our general paradigm.
All judgements would be recorded with a numerical scorecard. Compiled and analyzed later. A weighted average of all final scores would lead to a final number for each candidate. I’d then select the highest scorer. Sight unseen.
To review, this means I would have five tests, three rounds of review, a committee of five to help with selection, and a set of ideal answers to compare and judge against. A scorecard would be used for assessment and a final selection would be based on a weighted average.
I feel pretty good about that. It’s not much more rigorous than a traditional interview process.
And besides, could it be any worse than what already happens?
Interviews Are Ineffective
I’m not the first to say it but interviews are a wildly flawed enterprise. Seldom has something been so important and yet so terribly performed. Harvard Business Review has reported 40-60% of new management hires fail within 18 months. Other studies show the same rate of failure even among hourly employee hires. In 2014, Gallup reported something even more astonishing:
Companies fail to choose the candidate with the right talent for the job 82% of the time.
I don’t understand this. How are companies staying afloat amidst such absurd fail rates? If a surgeon had this sort of track record, we’d revoke their license and probably sue them, too.
Is this bad track record due to thin talent pools? I’m sure that’s true in some cases.
Is it due to an employers impossible expectations? In some cases, yes.
That said, my own anecdata shows that the interview process is mostly to blame. The interview process is arbitrary, reactive, and unnecessarily instinctive. It is also as boring as The Postman.
That last line is funny but it’s also frightening. Think about it next time you interview somewhere. Look around the table. Aside from the hiring manager, chances are very high that no one on the panel wants to be there.
And why not? According to Gallup, we are worse than a coin flip. So why bother? Screen applications for a few broad factors and flip the coin. Don’t do interviews. No awkward one-way conversations. Flip a coin. Make a hire.
That might sound crazy but it might be the most fair.
Interviews Are Unfair
Frank Bernieri, a professor of Psychology at Oregon State University, has done fantastic work for many years to show the consistent power of the first impression. First featured in a Malcolm Gladwell article, his team’s discoveries began in 2000 when they took footage of job applicants entering a room, shaking hands with an interviewer, sitting down, and being welcomed to the session. This video footage was about fifteen seconds in total length for each applicant.
Less time than it took you to read that paragraph.
Fifteen seconds of a regular activity (meeting someone) that we all do.
The research team then showed this footage to a series of strangers. These strangers were asked to assess each interviewee on eleven traits that actual interviewers had used to assess candidates for a particular job. In other words, these perfect strangers had fifteen seconds to do what an interviewer had an hour to do: assess a candidate’s viability for a new job.
The result? As quoted from Gladwell’s New Yorker piece:
“On nine out of the eleven traits the applicants were being judged on, the observers significantly predicted the outcome of the interview,” Bernieri says. “The strength of the correlations was extraordinary.”
That was in 2000. Bernieri’s work since has further strengthened the general findings. Rank strangers are 81% as effective as your interviewer in determining your capabilities for a job.
All because of the way you shake, or don’t shake, someone’s hand.
Will Rogers is still correct: you never get a second chance to make a first impression. I’m sad that this is the case.
That first impression, when combined with our inclination towards confirmation bias, makes the interview a charade. But don’t take it from me. Here’s Laszlo Bock from his fantastic book Work Rules!
In other words, most interviews are a waste of time because 99.4 percent of the time is spent trying to confirm whatever impression the interview formed in the first ten seconds.
Laszlo claims it’s ten seconds instead of Bernieri’s fifteen. The window is getting even narrower.
Can We Improve?
When I wrote about the fabulous book Thinking In Systems, I wrote an article titled “A Bad System Beats A Good Person Every Time.” I love that line. It’s one of my few pseudo-original phrases (co-opted from W. E. Deming) and history strengthens its truth.
The reverse is true, too. A good system beats a bad person every time.
Interviewers aren’t bad people. Not when they give me a job, anyway. But really: none of us are “bad” at this work. We are simply flawed. As yesterday’s article explained, we have a narrow local maximum that we can’t easily escape without help.
Laszlo Bock and Google provide that help. They developed a beautiful, rigorous, self-skeptical practice to improve their interview process. His book shows a fabulous set of insights that rhyme with my initial ideas shared in that scenario I wrote above. In a brass tacks fashion, here are a few lessons that I think are deeply valuable:
Finding #1: Based on their deep trove of data from past efforts (a lesson in itself), Google discovered that four interviews were enough to predict whether or not a candidate should be hired.
Finding #2: Screening practices best serve Google as a means to eliminate false positives. As Bock writes,
… we would rather have missed hiring two great performers if it meant we would also avoid hiring a lousy one.
Finding #3: Work sample tests are the best predictor of how someone will perform in a job. Cognitive ability tests are the next-best predictor. As Bock writes, these tests are designed to assess the candidates “capacity to learn”.
Finding #4: Structured interviews are as effective as cognitive tests. What’s a structured interview? Bock defines it as an experience where
… candidates are asked a consistent set of questions with clear criteria to assess the quality of responses.
Finding #5: Rely on the wisdom of the crowds. This is my personal favorite of the findings. In their beautiful research, Bock reports that Google discovered the following:
… no single interviewer’s assessment was by itself that helpful.
So what do we do with this? Well, for one, I can’t do justice to the broader, richer findings from Bock’s book so I recommend reading it first. Second, I offer my humble scenario written at the top of this article for real consideration. Imagine what you would do if you couldn’t rely on interviews (i.e., first impressions). Imagine if you had to make a system of unique, deliberate methods to choose your next hire.
Imagine a methodology, a proverbial machine, that could generate all the data you need. Would you let it?
If you did, could we then get beyond “thin-slicing” and snap judgements and confirmation bias?
Yes, we could.
But would we? Would we allow this better system to do so? Probably not. As beautiful work from the NBER shows, we seldom want to give up our discretion, no matter how flawed it may be.
Maybe we should. I’d argue more on that point but I’ll save you the time. After all, I suspect we all made up our minds about this some 8 minutes and 45 seconds ago.
We can improve this. We owe it to ourselves if not to others.
For more specifics on Google’s practices, check out this great article by Derek Thompson.