Tuesday, November 22, 2005

Why are most job interviews so bad?

We have all been to job interviews, but has it struck you how remarkably random the whole process is? Some companies put in effort in but I recall interviews where the person interviewing me was clearly bored, had been asked to fill in for someone else, didn't really know what the job was about etc. If it is a big company they might have a session about the company e.g. I recall as a graduate going to an interview at Plessey and hearing about the pension plan for an hour; just what every 21 year old is dying to listen to. On the other side of the fence, most of us will have interviewed some surreal people. I had one guy who was clearly on drugs, and one CFO candidate who considered that answering my accounting questions was "beneath him". My colleague had one candidate for a technical author who, when asked about his prior work experience, jumped up, grabbed and opened the large suitcase he was carrying, revealing a cloud of dust, a horrible musty smell and two maintenance manuals for the ejector seat of an aircraft, which he proceeded to read out loud.

Does it really have to be this way?

I have been studying this in some depth recently, and was pleased to find that there is at least some science around. If you look at various selection techniques, it turns out that "unstructured interviews" i.e. the ones most of us are used to, are actually a pretty dismal way to select people. A major 2001 study looked into various selection techniques and tracked back performance at selection e.g. how good their interview was v how well the candidates were performing in the job a few years later. It turns out that unstructured interviews manage just a 0.15 correlation between interview results and job success i.e. only a bit better than random (correlation of 1 is perfect, zero is randomly, while -1 is perfect inverse correlation). Highly structured interviews, based on job competencies and evidence-based questions (which can be trained) manage 0.45 correlation. Ability tests on their own manage a correlation of 0.4, and if combined with structured interviews take the number up to 0.65, which although still not perfect was the best score that had been achieved.

Ability tests take various forms. The best of all are ones that are directly related to the job in hand e.g. actually getting someone to do a sales pitch if they are a salesman, or to write some code if they are a programmer. For more on one of these see an earlier blog. The most creative of these I heard about was an interviewer for a sales position who asked each candidate to sell him the glass of water in front of him on his desk. One candidate want to the waste-paper bin by the desk, pulled out a match, set fire to the paper inside and then said "how much do you want for the water now?". Generally less creative approaches are adequate, and at Kalido we use a software design test for all our software developers which enables us to screen out a lot of less gifted candidates, saving time both for us and the candidates.

General intelligence tests also score well as, all other things being equal, bright people do better in a job than those less bright; studies show that this applies across all job disciplines (yes, yes, you can always think of some individual exception you know, but we are talking averages here). The 0.4 correlation with job success that these tests provide is a lot better than the 0.15 which most interviewing manages. Personality profiles can be used to supplement these, as for some types of job research has been done which shows certainly personality types will find it more comfortable than others. For example a salesman who was hated rejection, didn't enjoy negotiating, disliked working on his own and was pessimistic might still be a good salesman, but would probably not be a very happy one. You don't have to invent such profiles and tests: there are several commercially available ones, such as the ones we use at Kalido from SHL.

The cost/benefit case for employing proper interview training and such tests is an easy one to make: the cost of a bad job hire is huge just in terms of recruitment fees, never mind the cost of management time in sorting it out, the opportunity costs of the wasted time etc. Yet still most software companies don't employ these inexpensive techniques. Perhaps we all like to think our judgment of people is so great that other tools are irrelevant, yet remember that 0.15 correlation score. There may be a few great interviewers out there, but most people are not, and by supplementing interviews by other tools like job-related tests and good interview training we can improve the odds of hiring the best people. I used to work at Shell, who did a superb job of structured interview training, and I recall being trained for several days, including video playback of test interviews, on how to do a half-hour graduate interview. This may sound like a lot of effort, but it is trivial compared to the cost of a bad hire.

Many software companies seem to be missing a trick here. When I applied for jobs as a graduate I recall virtually every large multi-national had an extensive selection process including ability tests, yet in the software industry, where almost all the assets are its people, such things seem rare. I was amused to hear a recruitment agency whining at me for our use of screening tests at Kalido: "but only software companies like Microsoft and Google do tests like that". I rest my case.