Coding Platform for Interviews
- Reduce time to hire by 50%
- Assessments processed in 24hrs
- Amazing candidate experience
Trusted by these brands
Bulb hired 70 developers in 12 months using Geektastic’s assessments, saving 58 days of engineering resource
- 70Engineers hired in the last 12 months
- 44Hours per week of engingeering time saved
- 50%Reduction in time to hire
- 408Technical assessments outsourced
Where our expert review team work by day
Are you an experienced developer and interested in joining our review team in your spare time?
….make the take-home code test for interview as close to the real world as you can.
If your business is e-commerce - build a take-home code challenge where the task is to build a shopping basket or checkout (we have one of those), if your business is booking flights ask the candidate to build a flight booking page - even better have them integrate with your API, serve up some products to add to the basket or some dummy flight data to parse and pull into the webpage.
Having spoken to developers this is the single biggest factor that has kept software engineers interested in various challenges over the years. There is nothing more frustrating that receiving a code test do do for an interview, and seeing it is another boring algorithm like checking a string for parenthesis counts, or sorting a list without using inbuilt functions.
Doing multiple code tests can be time consuming and tedious if you are applying for multiple jobs. When the challenge is closer to the real role it starts to get the candidate engaged with the brand, they feel like they are actually starting working there - psychologically this is a huge win. Candidates who aren’t that interested in the challenge will naturally not do their best - therefore it kills multiple birds with one stone.
One of the best ways of making your take home code challenge close to real world is to make sure the challenge poses a problem within the general business domain of the company.
IMPORTANT NOTE: If you take this to too far and make this too domain specific, you’ll end up alienating people and could end up with a coding assessment that only lets you hire people who already work for you - so remember to strike a balance.
Code tests should all fulfil the following
- Open for creative interpretation,
- Marked by a human (if you have your coding challenges marked by a machine - please stop. Machine marking is a binary solution to a non-binary problem. People are not binary (not matter how geeky we talk). For more on this topic please read our post here.
- Used for open discussion later
- Production quality code requested (Clean Code).
- More than input/output of a single method.Eg, a light UI, an object model, an API call.
- Simple and clear - can be summarised in a single sentence.
- Time suggestion, not limit
Several BIG Topics came up in the process of our discussions:
- Timed challenges.
- IDE (Code Editor).
- Realisticness of it being like the actual job.
Geektastic programming languages and frameworks covered by our Code Challenges
- JavaScript
- Core JavaScript
- React
- Angular
- Python
- PHP
- C Sharp
- C++
- Ruby
- Scala
- Java
- CSS/HTML
- iOS
- Android
So let’s look at each of those points in detail.
Want to try our coding platform for interviews:
Timed challenges within Coding Interviews
There are two schools of thought on this - We asked Andy Davis (who helped write this page his thoughts) who also asked Zeno Foltin, a London-based (polyglot) software developer here’s what he said (hint they’re not big fans of times challenges)
Challenges with a hard cut off time. It seems obvious why companies might impose a time limits:
Create a level playing field for comparison - if they have the exact same amount of time and no prior knowledge, we can accurately compare candidate solutions.Candidates know upfront that a fixed amount of of time is needed not a snow ball into hours/days and therefore are more likely to accept doing the challenge
Zeno and I put our heads together on this one and we think not enough thought is put into the downsides of that time limit. For example:It doesn’t actually measure how well they will do on the job as the time pressures in the job are not the same as examination pressures.Faster developers are NOT better developers.A candidate who is slightly slower but can model the domain well, TDD and produce something infinitely better than speedy gonzales (who only writes procedural code) may get unfairly compared as a worse developer if they couldn’t get that across in the timeframe.
Zeno said:
“I don’t think timed challenges provide any value in measuring how well the candidate will do on the job. If it’s about trying to find out how someone handles pressure, do it in a pair-programming session maybe. I think it’s silly to think in our creative industry that putting such pressure on people will bring the best out of them.”
So whilst we don’t think timed challenges are completely useless, it just might not be the type of comparison they think they are making - possibly just a superficial comparison.The compromise here is to have a time limit but make it generous enough that the candidate has time to try out a few ideas. The last thing you want to do is cut everyone off mid flow - this would remove the benefit of having a test in the first place.
IDE
Believe it or not, some coding challenge systems HackerRank, Codility actually force you to write and compile code in the browser - it seems to have become a bit of a fad recently. It’s pretty nifty but it only really works for simple code. It seems to have benefits of scale for the companies using them as the computer tests the inputs and outputs for correctness - but this shows very little respect to the candidate and actually makes their life much harder. It is unable to provide rich intellisense or access to commonly used frameworks or testing tools.
Developers like their tools and we are no different. Writing code in the browser is not part of our weaponry and will not be something required on the job. Browser based code editors are great for tutorials and quickly trying something out you’ve just been taught but as a candidate testing tool, this should seriously be reconsidered.
Likeness to the actual job
We know that is it impossible to condense the real nature of the job down into a coding challenge. But there really doesn’t seem to be any point in asking a developer to complete something like an in-memory algorithm to return them the correct change from a vending machine. Sure, there are some algorithms in there and order of complexity to think about. However, if the job isn’t about solving little puzzles, it really doesn’t appear relevant.
Andy was once sent a zip file from an online fashion retailer and told to refactor it. Whilst this sort of request is a little bit lazy - at least this is probably closer to the real job. Refactoring legacy code is part of almost every job. Furthermore it was actually in their domain. So, big points to them for this challenge, it was enjoyable. The only downside of that is that he never heard anything back (after spending 2 hours on the test too) - this is one of the biggest bugbears we have as developers.
The bottom line around getting challenges close to the real world is:
- Avoid algorithms unless that really is what you are hiring for.
- Avoid tests that were clearly designed for a different industry
- Be more honest with your coding challenge, if you are hiring a bug fixer, how about a challenge where you need to fix a couple of bugs
Coding Interview Case Study: JUST EAT
Zeno worked with them for a year and Andy went through the process last year, so he had also taken the same test.
We both had a great experience with it and rated it the number 1 test we’d done. You can see their test here:- https://github.com/justeat/JustEat.RecruitmentTest
Zeno said:
“I really liked the JUST EAT recruitment test because it is simple - list the takeaways for a given postcode using the API. It requires you write a simple UI to present the information, but it doesn’t tell you much more on how to go about it. It is directly to do with their business, it covers UI and logic (I don’t believe in back-end or front-end only roles), and it requires more effort than a couple of minutes figuring out or googling an algorithm. A well designed challenge, can give an opportunity for the candidate to showcase the best quality work that they can do without hard time constraints.”
The JUST EAT recruitment test is also a very enjoyable challenge. So let’s take a deeper look at it:
- It is a very clear and simple requirement. Use API to search for takeaways by postcode and display results
- It covers the full stack of API through to UI
- If you are a mobile developer, you do a mobile client. Web developers do a Web UI.
- It is about their domain - online takeaways. It uses their public API, so the takeaways you get back for a postcode search are real. This immediately hooks us into actually enjoying the challenge a bit more because it doesn’t feel contrived.
Open source - JUST EAT decide to list their challenge on their github account - This alone sparked Andy’s curiosity and got him looking at other public projects they had on there. With this, he found himself inside their published code and started to get a feel for the type of code they are already writing at JUST EAT. JustBehave, JustFakeIt and JustSaying were 3 frameworks right there in their account that he could look at and see some of the real code their teams produce - granted this is only the view of the code they want to be seen, but this isn’t code for show, it is code they actually use.
It’s great that even at the coding challenge stage, there is a bit of two way flow of information, just by being open to open-sourcing. There aren’t many places where you get to see some of the code before you go there. It is often a bait and switch of great stuff spoken about at interview followed by a shocking reality of legacy code once you start.
What can we learn from JUST EAT even if you don’t have a public API?
You might be fooled into thinking JUST EAT have unfair advantage of their domain being highly public facing. Even if you don’t “need a balti”, you still know some of the takeaways in your area that they service and when these familiar venues are returned by the API for your postcode. Not everyone can do this as their domain is not so publicly known - but I bet with a little thought, you can be creative enough and throw the candidates a little bone about the real job rather than have them implement a vending machine, phone bill or shopping cart. There is nothing worse than seeing another challenge that isn’t even related to the business sector that the company is in.
Another lesson that can be learned from JUST EAT is not be too strict about the solution. If you can allow the candidate to be creative, you will get a better picture than from an algorithm implementation with a right or wrong answer. Sure, it takes more effort to have a developer mark the solution, but that time will be infinitely better spent if you discover better candidates from it.
Here are some ideas for you, even if you aren’t as publicly glamourous as JUST EAT:
If you are Compare The Market - provide 3 insurer APIs and request the developer calls all 3 and display results in order of best quote.
If you are Shazam, provide an API that accepts a music snippet and identifies it?
If you are Uber, can you have an API showing closest car to a set of coordinates and have the developer write a little client to request a ride?
If you are Tesco, yes yes ok, you can do an online shopping cart
If you Buzz Fizzes, you can… well… nevermind.
It is a continual learning process
In agile software development, we are told to embrace change. Things will change, so build change into the process and not make it an afterthought.
Like anything else, accept that you won’t your challenge right the first time.
Developers WILL complain.
Some good candidates will slip away - refusing to do it.
Some good developers will ‘fail’ it.
Some bad candidates, will find their way past the challenge and waste precious time that you could be spending with Ninjas Just a joke, there’s no such thing as Ninjas…
…and even if there is you don’t want them
…and even if you think you do, you just alienated them and half the developer community by using the word Ninja, whoops.
You should plan to get feedback from candidates taking your test and LISTEN to this feedback.Warning: FEEDBACK WILL BE EMOTIONAL, especially if they ‘fail’.
Let’s conclude by reminding ourselves of some of the important points that we’ve explored
- Let candidates use a comfortable environment - do you really want them to write code in the browser or is their own IDE a better choice?
- If you impose a time limit, think carefully about the implication of that limit, are you going to cut off a good candidate mid flow. Is it really a fair comparison just because time is equal?
- Can you make it relevant to the job? We all now know that Google’s silly questions about painting jumbo jets and shrinking people into blenders didn’t find them any better candidates than traditional questioning - does the same apply to coding challenges?
- Can the coding challenge cover a little front-end and back-endBe very specific in the requirement. It is ok to say, you may present this on a simple web app. But requirements like “tests are optional” is a huge dilemma for a candidate - this is unfair and risky especially if you have in mind what you want (ie tests).
- If you value performance over clarity, tell the candidate this. Algorithm tests lend themselves to right or wrong (plus performance) - this makes it easier to mark, but making it easier to mark is NOT your goal. Your goal is to filter out the time-wasters, yet algorithms are the easiest to plagiarise
- Iterate, you won’t get it right first time. Keep tweaking it.
- Finally - Make it relevant to your business sector if you can. The best developers aren’t just coders, they care about solving real business problems. Make it clear you understand this by making your test relevant to your business.
Want to try out our platform for free?
About Andy Davis
Andy is a software developer. He’s been writing software for over 20 years and helps businesses solve interesting business problems, by helping them create the right software. He also helps teams with their developer hiring strategy as he believes that hiring the right people has a multiplier effect on awesome software teams.
Ever wondered how Geektastic is different to other assessment platforms like Codility and HackerRank - we wrote a comparison piece.