Appropriate Interviewing For a Worldwide Search? 440
jellomizer writes 'I am a manager of a small Software Development department, looking to hire some more developers. By edict of the CEO, the search must be made globally, so we are dealing with different cultures and different ideas of truth and embellishment, etc. To try to counteract this, we give the potential employees tests where I watch what they do, to see if they actually know what they say they know. However, it seems a lot of applicants drop out when I mention that this test is mandatory. Is this a sign that we caught them in a lie, or are we weeding out good people where we shouldn't be? Would you be willing to take a test as part of an interview? If so, is there any type of heads up you would like to know beforehand to make the decision of whether to take the test easier?' What other difficulties have people seen while trying to hire from many different cultures?
A good test (Score:5, Insightful)
I'm starting to think that our interviews here should literally be: give them a day's work and see how they do.
Good developers dont have time to take many tests (Score:2, Insightful)
Re:I don't take test as a matter of priniciple (Score:2, Insightful)
No, I wouldn't be willing (Score:5, Insightful)
No, I wouldn't be willing to take a test, and I actually flat walked out on an interview in 2003 when I showed up and was told - by surprised - that I was going to be taking an exam. I was also then informed that the open position was for a junior position. When I expressed surprise at this, the HR flack's response was "Oh, didn't I mention that in my email?" She hadn't. Either of those would be sufficient for me to end the interview process, which I did.
Why would I refuse to take a test? Simple: if you're giving me a test, the usual reason is that I'm being interviewed by someone who does not possess the ability to discern whether I know what I'm doing/talking about or not. If that person is the hiring manager, then I certainly don't want to work there. Working for people who cannot identify competence or incompetence is not pleasant. If that person is not the hiring manager, I still don't want to work there: it shows they would waste my time by having me interview with such a person rather than with the hiring manager or any other person who can tell if I'm competent or not.
Re:A good test (Score:5, Insightful)
it is a good point about tests.. there are 3 kinds of people in the world when it comes to puzzles, those who havn't a clue how to solve it, those who figure it out slowly by thinking it through, and those who read the answer while they were surfing the web when they should have been working.
Trouble is CEOs tend to hire the latter ones. The reality is that the former group can be as good, if not better workers that the other 2 groups, when they're not asked stupid arbitrary tests, and apply themselves to real world problems.
I went to an interview a few years ago, the boss gave the test, on a flipchart. It was a 'write code' type test, unfortunately he asked relatively open ended questions ("how would you implement a stream class") but the answer had to be the one he expected or wanted, and as I did it differently, I was obviously a useless candidate.
So the problem is how to make your tests applicable to real people, in the real world. I would suggest you give them a large problem to solve, one that has no 'right' or 'wrong' answer, and make sure they explain what they're doing and why. The why matters more than anything.
If you're still thinking about this, why not post your test to a slashdot comment. Then I guarantee you'll get a load more candidates who pass your test with flying colours, I'm sure none of them will be the kind who surf the web during their work hours....
Yes, you will filter out good people (Score:1, Insightful)
Re:I don't take test as a matter of priniciple (Score:4, Insightful)
If you can't read a resume why did I write it?
Because people embellish resumes and creatively bullshit through interviews who have no experience anywhere near the task at hand that they are supposed to do. An open ended test weeds out a lot of that. Even those that have been doing it for years may not be versed in how the rest of your group is working and could not have the kinds of experience you really need.
Re:Good developers dont have time to take many tes (Score:4, Insightful)
I have been put in such situations and I just politely end the interview process.
I don't work for free, and anyone that would expect me too is not going to be an employer I want to do business with.
If a serious offer has been made and you want me to write on a whiteboard, great. If you want me to email you a finished project that you get to keep, you are going to have to pay.
Re:Good developers dont have time to take many tes (Score:4, Insightful)
One thing employers should do more often is talk about salary up-front, so prospective employees don't waste a lot of time with testing and interviewing just to get an insulting low-ball offer. This has happened to me several times, and it's annoying. This doesn't mean testing is bad; I had some simple tests in the interview for my current position, and they gave me a very generous offer unlike other firms in the area.
I've found too many companies are tight-lipped when it comes to salary; they want to waste a LOT of your time in interviewing and such (and their own employees' time too), but then when the hiring manager says "he's great! hire him!", the HR person wants to give a lowball offer, and refuses to bring it up to anything reasonable even when you have a much better offer in-hand. If these companies would just state up front what their salary range is, we wouldn't waste all this time and effort.
Meanwhile, these tests do NOT need to be difficult or long. I had to help phone-interview some contractors at one job, and I came up with some very simple questions to ask them. One was, "In C++, can you tell me what a 'class' is?". Several contractors who stated "expert-level C++ knowledge" on their resumes couldn't answer that basic question. So you don't need to give them a 2-hour test; just some simple questions, maybe some short code samples with errors in them, etc. to see if they're completely lying about their credentials or not. So many people lie on their resumes (and this is definitely cultural; I saw this mainly in people from India), it's really important that you screen out the liars from the people who can actually do the job.
Re:Good developers dont have time to take many tes (Score:4, Insightful)
You've got to be kidding me. If I have to come to a location multiple times for a job interview, I'm probably wasting my time. The only way I'd bother is if we start discussing salary right away, so I don't find out at the very end that this employer is a low-baller.
In my 11 years of experience and many, many job interviews, I've never had to come back to a place for a second interview. If the employer can't tell if I'm a good fit in one visit, they're doing something very wrong.
Yay for tests! (Score:4, Insightful)
I've worked several places that had some sort of practical skills test for programmers.
I consider them a strong success.
There's a surprising number of people that can talk a really good game, but fall apart when asked the most basic actual questions. And, conversely, people who aren't extroverts or for whatever reason aren't stars in the verbal part of the interview, but clearly know their stuff when given a written test. It's a really good way to fairly judge technical skills across candidates, and weed out the fakers before you have to fire them later.
And as an employee, I like working with smart, competent people. I know how much, from experience, a bad hire wastes my time and ruins morale, so I'm happy to work for places that put out effort one way or another to not enter into this trap.
Anyone who'd refuse on principal, I'd worry is either a) faking it, or b) too arrogant to work well with others. A good candidate is happy to prove that they're a good candidate, and won't have to work with idiots.
A test isn't the _only_ way to do this. Any sort of nice, concrete technical grilling will do. But for a programmer, it _must_ involve actually writing code of a non-trivial nature. You can't believe resumes - even if people aren't lying, a Senior Programmer at one shop may only be barely competent at another, and not even realize that the bar is set differently.
Of course, the quality of a test can vary just like the quality of an interview question, but the goal is good for the company _and_ the employees, and in my experience it works better than most techniques.
Now, if your shop isn't terribly compelling based on product, and you're desperate to not turn people away... well, you probably should be looking for places that feel like they need a test to screen out unqualified candidates, so you can stop hating your job, and not worry about fixing this particular problem. :)
You have to lie to get the job interview (Score:2, Insightful)
Re:Good developers dont have time to take many tes (Score:3, Insightful)
Even though aptitude tests are pretty annoying, I really see them becoming more important since it's getting harder and harder to judge a person by their resume.
Re:I don't take test as a matter of priniciple (Score:2, Insightful)
Re:I don't take test as a matter of priniciple (Score:4, Insightful)
References are mostly useless unless you also know the reference person. Anyone can find some lackey to put in a good word for them. And lots of people can bullshit about a subject and sound knowledgeable while not being really qualified to work in it.
Re:Good developers dont have time to take many tes (Score:2, Insightful)
I do a software test for my candidates.
I hand them a small (40 line) program that is broken. Since most of my work involves code maintenance it's real world enough.
I tell them up front that the object is not to fix the code per-se, but rather to walk me through them debugging it. I've hired guys who couldn't solve the problem because they showed enough promise that I wasn't worried that I could train them, and it was obvious that they could think well.
We have a total of 4 hands on tests. Three on hardware, one on software (R&D lab). Worked out well for us so far.
-nB
Re:Good developers dont have time to take many tes (Score:4, Insightful)
Copying and pasting code amounts to 90% of even the best programmers' jobs. Other than some aspects of GUI design and input validation (which even then just means "tweak the conditionals of what you have", not "write an input handler from scratch"), if you sit down at a blank editor on anything but a toy project, you either work in academia or need a much tighter deadline.
And there, I have my biggest objection to "testing" applicant, particularly in a vacuum. In the real world, I program with access to huge libraries of functions and at least half the time, a web browser open to look random things up as I need them. Yeah, I could rewrite Windows from scratch if you give me 500 man-years to do it, but do you want to see if I can really do the job, or do you want to see if I happen to know the prototype for some particularly obscure network calls off the top of my head?
People pay me to solve problems efficiently, not because I've memorized the latest edition of K&R.
(and for the record - Yeah, I pretty much do have the prototypes for the entire standard ANSI C library memorized, but would prove just as valuable if you wanted me to program in any language, including one I've never used before)
All my interviews included tests... (Score:4, Insightful)
I'm now retired, but all the way back to my first tech job interview in the '60s, the interviews have included tests of my ability to perform as needed.
If one cannot test an applicant one is seriously handicapped in making valid hiring decisions.
--
Tomas
University Place, WA
Choose your failure mode (Score:3, Insightful)
You will lose some good people who do not want to take the test. But without the test, you might hire an incompetent employee. Which is worse?
Personally I'd take the former failure mode. In that case your job search just takes longer. I'd rather that, than hire a bad employee.
Yes! TEST! (Score:5, Insightful)
If more employers were able to test their employees adequately during screening, there would be a lot fewer bullshitters out there trying to claim they can do what they can't. Burn in hell you cert-chasers who don't know what you are doing!
Every employer who gave me such a test, I always got an offer -- usually a good offer. Some of us are actually good at working through IT related problems and solving them. Some of use are not and just want to coast through on bullshit.
There are very few people who feel that they are "above the test" and the eager ones actually say "bring it on!" Those are the ones you want.
As for setting up a "practical" part of a test? Absolutely! If you have the skills to build a nice one, do it.
Nothing gets under my skin more than HR people screening resumes of really good people because the right set of words didn't appear on a page. Every successful hire I have ever had was when another IT person was involved in the screening and interviewing.
Dropouts (Score:3, Insightful)
However, it seems a lot of applicants drop out when I mention that this test is mandatory. Is this a sign that we caught them in a lie, or are we weeding out good people where we shouldn't be?
I won't venture a guess as to whether you've caught them in a lie but unless your test is seriously overbearing, you're not weeding out anybody you shouldn't be. Competent developers serious about finding a job won't balk at an interview merely because you ask for a demonstration.
I will say this though: be honest, be open and be brief. Tell them that the point of the test is to eliminate folks who talk the talk without walking the walk. Encourage them to take a copy of the test with them so they can use it to improve on any issues they weren't prepared for. And don't ask 100 boring questions. Something as simple as, "write a function in C to reverse the characters in a string," can be surprisingly revealing.
The question (Score:3, Insightful)
The answer to this question is extremely useful in assessing the candidate's knowledge of and maturity in software development. For example:
And so on. Obviously there are no right answers, and it's critical not to be biased in favour of one's own pet methodologies. What you're looking for is someone who takes intense interest in not just coding, but in the entire process of developing quality software.
Re:Good developers dont have time to take many tes (Score:1, Insightful)
You've got to be kidding me. If I have to come to a location multiple times for a job interview, I'm probably wasting my time....
No, you're probably wasting ours.
In my 11 years of experience and many, many job interviews, I've never had to come back to a place for a second interview.
Then you've probably never held a senior position at a LARGE company. Multiple interviews are the norm. There's the preliminary interview with HR, then you'll probably get interviewed by the country/regional director, one of the VP's, or someone on the board, then you'll probably get interviewed by the leader of your team/group and other members of the group. There's at least 3 interviews.
Of course you won't even be CONSIDERED for the position by HR if you haven't taken a number of tests by outside consulting firms that point out your strengths and weaknesses. In fact, continual testing is a part of any serious career.
But then again, I guess we're not talking about the same salary range either.
Re:A good test (Score:4, Insightful)
so somebody who is actually learning outside of their everyday requirements, somebody who is raising their qualification and is able to apply this knowledge when it is needed... is the worst candidate for you ?
I've hired one of them - he did brilliantly at interview. Unfortunately, he could recite the words from the web, but had difficulty applying it. Also, he continued to 'raise his qualifications' during work time. We had to let him go.
I've known others like him, people who always want to apply a new or different technology to work problems, always because the boring work of work is not fun enough or too difficult and the 'grass always seems greener'. They don't *do* anything but chase the latest technology craze, I suppose continually dropping things when they get difficult is a nice alternative to working, wish I could do it!
That's the trouble, when hiring you need someone who has done the stuff you want, not a theoretician. If he knew how to apply his knowledge, by doing some of it, he'd be significantly more desirable as a candidate. Web surfing is not a qualification for a job.
Re:Good developers dont have time to take many tes (Score:3, Insightful)
Re:Good developers dont have time to take many tes (Score:2, Insightful)
Re:Good developers dont have time to take many tes (Score:3, Insightful)
In the real world though, most people have encountered people that sounded great on paper and who had great recommendations, but were lousy in practice. People will lie and embellish, and prior bosses will lie because often company policy is to never geed bad references. This is especially true I think in a "global" search. There are people who will say yes if you ask them if they can do brain surgery if the need arises.
The issue isn't to get you to do free work, but to evaluate you. I have never seen a "free labor" test. There is not enough time in an afternoon to get free labor out of a candidate.
Re:Good developers dont have time to take many tes (Score:4, Insightful)
Test Yes, Code Test No (Score:3, Insightful)
Stay away from pure code tests. Asking a person how they would solve a given problem or a puzzle is a great idea. But have them explain to you in their own words how they would solve it and don't focus on a particular programming language. The reasons are:
A) Programming languages can be learned. You don't want to rule out people who are great problem solvers just because they don't know your favorite language yet. And don't let them pick their favorite language because you may not know it well enough to judge their effort.
B) You want them to demonstrate good communication skills. In the real world good programmers have to deal with people (Customers, Mathematicians, Human Factors Engineers, Electrical Engineers, Safety Engineers, etc...) who aren't coders.
Re:give them a day's work (Score:1, Insightful)
If firing is difficult (depending on the place and circumstance, much of the time it is), then it is less costly to have you go work for the competition. Hiring without some reasonably estimated basis for competency is hiring a risk.
I've interviewed people who couldn't find their way out of a paper bag, and the resume was, none the less, surprisingly impressive. I ask questions about things claimed on the resume because my experience has been that most resumes are filled with exaggeration and sometimes outright lies. Yes. Most of them.
I'd rather turn away a good person than hire a crappy one (the firing, as I've said before, is likely to be expensive).
Re:No, I wouldn't be willing (Score:4, Insightful)
Speaking as someone who interviewed 30+ people over the last year: I can ask you a whole bunch of questions and listen to your answers, or I can sit you in a room and have you write down the answers and spend a 1/4 of the time going over those answers to see if they are right. Given that, in my estimation, 3/4s of the people I've interviewed were almost entirely incompetent at the technical skills claimed on their resume, I've been increasingly tempted to take the later route to save myself the time.
Given the number of people claiming to be ubergurus who know jack shit, some sort of test, either in person in an interview session or with some kind of test is a requirement. The way to identify competence is to ask questions taht require it to be displayed. This can be done verbally in an interview or with pen and paper.
Personally, I prefer interviews that include tests, because it means that the company is more interested in knowledge and skill than ability to bullshit in an interview. I've met plenty of people in interviews who could talk a mile a minute about any buzzword you could name but got the deer in the headlights look if you asked them to solve the simplest problem with code.
Re:Good developers dont have time to take many tes (Score:4, Insightful)
But what if they want you to meet multiple people? This happens a lot when scheduling conflicts make it impossible to do one all-day interview.
Personally I go out of my way for a potential company if I like what I hear throughout the interview process. I can come back more than once. In fact I did two phone interviews with the same company and I'll be flying down there Monday to do an on-site interview. I get paid a very good living and I understand that my employers expect the best so I'm not too put out by it. A great job can turn into a lifetime position so one might as well put as much effort as needed.
Re:A good test (Score:1, Insightful)
I have had a couple of companies offer me contract to hire and turned them down.
Re:Good developers dont have time to take many tes (Score:5, Insightful)
Not having an answer proves they don't know how to use Google. It took me 5 seconds to find a good overview of what a c++ class is. If you were talking to me, I'd tell you straight up; I'm not good at regurgitating formal definitions, but I would tell you where to find the same article I am looking at, then riff on various real world projects and problems.
If you can't tell me, in very general terms, what a C++ class is, then you're not qualified to claim you're an expert on C++. It's that simple.
Yes, I do a lot of Googling around when programming to learn more, to find other answers to problems, etc. I also don't have the C++ operator precedence memorized, and refer to a table for that. But some things you need to know or else you simply cannot claim that you're "proficient" in the language. When the job req says "needs to be proficient in C++", then certain basic things like that you need to know.
I'm sure I could Google around, read a book, or whatever and put together something in Python pretty quickly even though I've never programmed in it before. But I wouldn't tell an employer that I'm proficient in it.
Really, who do you want to hire? someone who can tell you what a class is, or someone who can tell you about problems they have solved.
Are you trying to claim that it's OK to lie on a resume and claim knowledge of programming languages you've never worked with?
Don't forget, this particular job was a contracting job. Long-term employment was not to be assumed here; the company wanted someone who could sit down and start contributing right away.
If you are in a smaller organization, it is very likely that there will be periods where other skills are needed - such as interacting with customers (pre-sales/post-sales), design, documentation, etc. etc. So be careful that your are not screening for one-trick pony (coders) if there is any possibility that you will need someone with other skills as well.
It seems to me that if the job entails those responsibilities, that should all be stated up front instead of assumed. I wouldn't take a job that required a lot of customer contact; my specialty is software engineering, not schmoozing clients. Time wasted with customers is time that could have been spent productively. Find a different employee to do the customer-contact stuff; this is why humans invented the concept of "specialization of labor".
Re:No, I wouldn't be willing (Score:2, Insightful)
I've also interviewed 20 or 30 people in the last $timeframe and I could take either of those aproaches, too. While I haven't found 3/4 of the people to be incompetent or anything near it, most don't get hired, either. Our target candidates are basically the top 10% in the Silicon Valley area; maybe we're just a lot better at screening out the poor candidates well before the interview stage. I wouldn't dream of sticking somebody in a room with a test; it's not a good use of their time or mine, and it keeps both sides from using that time to size up the other.
Would I ask technical questions about the subject matter during an interview? Definitely. Answers can be done on the whiteboard, on my laptop, whatever they feel comfortable with. If people don't have what we want, it'll come out pretty quickly. It doesn't take a lot of time to evaluate technical competence. I spend most of the interview time evaluating the *person* and that person's fit with my team. Decent technical people aren't really *that* hard to find; finding people with the right personality fit takes a bit more work and is more important. We can weed out the bottom half just through resume readings and telephone screens, it's not hard. If you get to the interview stage, you should only be talking to serious candidates.
Re:Good developers dont have time to take many tes (Score:2, Insightful)
You appear to like what you describe. To me, it sounds like hell. Enjoy your incarceration.