How to Review a Peer Review Code Challenge

(Guest post by Andy Davis)

Anyone who has reviewed a coding challenge will agree with me when I say:

The best way to judge a developer’s skill is to get them to write some code.

But there is one problem…

The code submitted to a coding challenge is NOT the developer’s best code. It is their code that was done within the pressure of the hiring process. We could review the code, see if it works, see if the design matches our idea of the design and then decide based on that if they are any good.

But is that a good approach?

I will suggest to you that there is no right or wrong solution. The code isn’t really code, it is much better than that. It is a collection of clues from solving code review challenges.

These clues can easily be overlooked. I want to show you how to get the most out of them when it comes to carrying out a review.

Points to look at when reviewing code challenges include:

• How to spot traits of a good developer in any code (good or bad)
• How to spot the code smells linked with bad development practices and potentially bad developers
• What to do when faced with very average code (Hint: this is 90% of all the submissions)
• The best way to overcome our personal biases when reviewing (even if our way is better)

But first, let me tell you a little story from a few years ago:

I once worked with a ‘Software Developer in Test’ who said to me that he does something called MDD. I was intrigued; what is MDD? I’m into TDD, BDD, and even ATDD, but what is MDD?

“Mortgage Driven Development” he said. “I write the code in a way that only I can understand. That way I will keep my job as long as I want and I’ll continue to be able to pay my mortgage”.

He laughed. We all laughed. The comment was made in jest. The unfortunate thing is, it turned out to be true. It wasn’t long before he was looking for a new company to pay his mortgage.

If only he’d done a coding challenge, perhaps we could have spotted MDD earlier?
 

Want to test your skills and take a peer reviewed Geektastic code challenge? Register now

The types of code you’ll review

As developers, we are the best people to review coding challenge submissions. So what can we expect?

Let’s consider the three types of submissions we will receive as reviewers:-

1. Good code
2. Bad code
3. Average code

1. Good code

What do we do when we receive really good code? No brainer. Praise all the positives and give it a great review.

On the flip side. It is entirely plausible that a bad ‘real-world’ developer has become good at coding challenges, but it’s fairly rare.

2. Bad code

Complete spaghetti code that doesn’t work and doesn’t make sense. No brainer.

This reminds me of two of my Geektastic reviewing experiences. One submission left three of the four methods completely unimplemented (I later worked out why; no data was stored in the first method so it would be impossible to retrieve it with the other three). The other strange submission was a full-blown Windows application when the challenge just asked to implement four small methods.

Neither of those solutions had tests, and neither of them worked. So both were nil points.

Tip: When you get code like this pick out a few of the significant negatives, give some constructive feedback and move on.

3. Average code

These are the hardest to mark and unfortunately, the most common.

We all know that hiring a bad developer can be destructive to the team, code and company, so it is good to be cautious. However, missing out on a great developer is potentially an even greater risk, especially at “screen-out stage”. Accidentally screening someone out could mean tens or hundreds more people who need to apply and go through the process before finding someone that good again.

It is easy to dismiss average code as lousy code because it isn’t as easy to read and follow as the ‘good code’. And it isn’t a disaster zone like the ‘bad code’.

To do it justice, we must overlook the comparison to a perfect solution. What we are interested in is spotting little gems of the good stuff and some smells of the bad stuff (see Martin Fowler’s book for an introduction to Code Smells).

Hiring based on the right kind of code

Hiring costs money, so if we can find a diamond in the rough - a productive developer - this will have a huge saving for the company hiring, and this value will be repaid over and over.

Productive developers are lurking within average code submissions as well as good code submissions; we just have to find them.

Steve Jobs once said:

“The difference between the best worker on computer hardware and the average maybe 2 to 1 if you’re lucky. With automobiles, maybe 2 to 1. But in software, it’s at least 25 to 1. The difference between the average programmer and a great one is at least that“

This tells us that we need to look harder for signs of a good developer. This will mitigate the risk of accidentally screening out that 25x developer. We owe it to the hirer to looks for clues that they might have 25x (or even 5x) potential.

Luckily we’re human, so we are able to do this.
 

In the next post, we’ll be looking at how to find those diamonds in the rough, and what clues can help you find the right developer when carrying out your code review.

You can read it here.

Want to license a peer review code challenge to REALLY analyse the skills of your candidates? Register now