Despite what many assume, I do not think the so-called "TopCoder-style" software developer interviews are perfect. In fact, they're very flawed (more on this another time -- and a bit in here too). However, companies have a variety of reasons for doing this -- some good, some bad.

This answer originally appeared on Quora, in response to "Why are programmers in the software engineering job interviews tested on skills similar to a Topcoder contest irrespective of the fact that the skills required in the industry are entirely different?"

1. Because smart people with a strong foundation in computer science fundamentals are believed to make good software developers.

Questions along the lines of "design an algorithm to ____" test two things:

Intelligence / problem-solving skills.
Computer science fundamentals.

Intelligence / Problem-Solving Skills A candidate tells me: "I can't believe Google asked me to design an algorithm to compress a string. That's ridiculous. First of all, I don't know anything about string compression and I shouldn't have to know this. Second, even if I needed to know this at some point on the job, I would just look it up online."

Yeah, so?

When the interviewer asks a question like this, she doesn't care if you know how to do string compression. I repeat: when an interviewer asks you to design an algorithm to do X, she is not trying to test your ability to do X.

What she's trying to test is your ability to solve new, difficult problems that you haven't seen before. This question to "design an algorithm to _____" is a way of estimating that.

This is just like how school works. Your teacher never cared if you knew the derivative of 3x^2 + e^x. He cared if you knew how to take derivatives of equations you've never seen before.

Computer Science Fundamentals This is actually where TopCoder and interview questions differ (or should differ). Whereas many TopCoder questions actually involve advanced algorithm knowledge, interview questions actually don't.

The data structures and algorithms you need to know are mostly covered during the first year or two of a CS program. It's basic stuff, for most graduate with a CS degrees: binary trees, linked lists, hash table (zomg yes hash tables!), etc.

CS fundamentals are tested for a variety of reasons:

They can be useful. Okay, so you won't use a binary tree or a linked list a lot in the real world, but you will almost certainly need to create your own data structure from scratch. Understanding how "standard" ones are implemented can be useful to guide you.
They're proxies for how well you generally understand CS. If you're really comfortable with linked lists and binary trees, you're probably pretty comfortable with related topics.
They're not that hard, so it's not a big "cost" to expect this. If you have a CS degree, you should know this stuff.
It's an accident! One of my favorite questions is to modify a binary tree to support something. When I started asking this question, I didn't think, "Gee, on the 1% chance that this candidate will need to use a binary tree at some point in the real world, I'm going to make sure that they know it." It's just, well, I came up with an interview question that is relatively challenging. The fact that it requires knowledge of a binary trees is unintentional.

That said, some interviewers really screw this up. They want to pick a "challenging" interview question, but they make it challenging on the knowledgeside rather than the problem solving side. As a result, the right percent of candidates are passing (perhaps) but not the right type. If you're asking a question that requires candidates to know (for example) red/black trees, you're doing it wrong.

2. Because the cool kids are doing it. To some extent, companies look at companies like Google, Facebook, and Microsoft and figure, "well they must know what they're doing, so we'll just copy them."

And to some extent, that's true. These companies do spend time thinking about their interview process.

However:

This doesn't mean that they're doing it right. Google, for example, requires interns to go through "conversion interviews" in order to get a full-time offer. Seriously? You just had someone working with you for three months and then, to test if they'd be a good employee, you're going to give them interviewswhich your own internal studies have already shown don't have much predictive power.
What they're hiring for is not necessarily what you're hiring for. Google, Microsoft, Amazon -- these aren't startups. They have different resources and have different requirements. They can put a lower bar on, say, technology knowledge because they have the time and resources to mentor someone and because there's so much internal infrastructure.

3. Because all other interviewing is also broken. So you have a problem with these "design an algorithm to ___" interviews. Welcome to the club. You'll find, oh, most of the industry in here.

If you reject these interviews solely because of this, you might be a bit shortsighted.

Algorithm-style interviews are broken. There's a lot of important things they don't test (e.g., work ethic), and there are a lot of great candidates who flunk these sorts of interviews (due to nervousness, if nothing else).

But whatcha going to do?

I'll test people on relevant knowledge.

Okay, you do that. But, see, a brilliant inexperienced person might know less at first, but they'll learn quickly. Smart people tend to make good decisions and can adjust quickly to changes. A dumb person can be dead weight when the industry or project evolves. They can also make very stupid decisions that can result in a lot of expense.

I'll ask people about their experience. Past performance is the best predictor of success.

Hey, if you know that someone was considered a great programmer at their last job, then that's great. Go ahead and hire them.

However, if you're interviewing them to assess if they were successful, then you're really evaluating their ability to sound like they were successful.

I'll give people a homework problem.

Okay. Good idea. Just one thing: how do you know that it's their work? Even if it is their work, is it really representative of their work? Or did they spend a lot of time thinking about the design and bugs and never would in the real world?

It's also not necessarily a great thing for candidates. Homework-style interviews are ripe for an abusive situation (when companies give candidates homework when they're not really that interested in them).

Companies who give candidates homework assignments: knock it off!

Many candidates are turned off by homework-style interviews, which is a bad thing for you as a company.

I'll have people build a real project in my office. This way I can see what they're like in the real world.

Great. But, if problem-solving ability is something you really care about, how will you test that? Will your real-life project have a sufficient test of problem solving? Maybe you'll be able to throw in one tricky algorithm, but now you're making an assessment of someone's problem-solving skills based on a sample size of 1.

Also, let's be honest: a lot of coding isn't hard or interesting. A lot of it is making this button pull in some file, process the data in an obvious way, and yadda yadda yadda.

What distinguishes mediocre programmers from great ones isn't their ability to do the 80% that's straightforward. It's their ability to do the 20% (or less) that's hard and interesting. By making your interview so "realistic," you've now spent substantially less time on the stuff that actually distinguishes candidates.

Okay fine. Well then I'll ask them the same sorts of algorithm questions, but at least I'll give them a computer. You can't really expect people to write flawless code on the whiteboard.

Err, well, no, you can't expect people to write flawless code on the whiteboard. So don't expect that of them.

The nice thing about a whiteboard is that it allows people to not get tripped up on syntax. There's no little squiggly lines telling you that your syntax isn't correct, or that you didn't add the right includes. "Close enough" isn't good enough for a compiler, but it is good enough for whiteboard coding.

By using a whiteboard, you put the emphasis on the conceptual aspects of the problem rather than annoying, mostly irrelevant details. Go ahead and write your 2D array as [,] instead of [][], or use .add when you should use .insert. The whiteboard, and me, won't care.

I am not saying that you should never give homework-style interviews, or in-person projects, or test relevant knowledge.

I am not saying that these algorithm-style interviews are perfect. They're not. They have flaws too.

All interviewing is flawed. Please go ahead and pick whatever style works for you and your company, with the recognition that it, too, is flawed. It's best for everyone that we use a diversity of interview processes.