Creating an accessible interview process for software engineers

One of the original motivations around Hatchways was to build an interview process that was more accessible. We are built on the premise that an interview that best represents and emulates on-the-job skills would be the most accessible, as you can see candidates' skills in action as opposed to using proxies for their skills.

Regrettably, the prevailing practices in hiring engineers are significantly misaligned with this ideal, leading to an inherently inaccessible interviewing environment. Providing alternative options for candidates does not entirely solve the problem, as it introduces difficulty in comparing candidates who have undergone different processes.

In this article, we will explore the components of an accessible interview process specifically for software engineers.

The Current Reality: Live Coding Interviews

For software engineers, the most common interview style to assess technical skills is the traditional live coding interview. There are many issues with this interview style, especially for individuals with disabilities. t fails to accurately mirror real-world tasks and often induces intense anxiety, essentially measuring one’s ability to manage stress rather than aptitude in software engineering. A 2020 study conducted by North Carolina State University and Microsoft concluded that such technical interviews primarily assess a candidate’s performance anxiety level instead of their coding competence.

Below are some specific challenges encountered in the prevalent live coding interviews.

Time Pressure

These interviews usually are timeboxed to less than 1 hour and sometimes the coding part of the interview is only 15 minutes. This time constraint generates significant anxiety and is not reflective of real work conditions; engineers seldom have to resolve issues in such a restricted time frame, especially under scrutiny. This pressured environment poses specific difficulties for those with conditions like anxiety disorders, autism spectrum disorders, ADHD, panic disorders, and dyslexia.

The Stress of Being Observed

Another hurdle in live interviews is the stress induced by being observed while coding. This can be particularly challenging for those who employ non-traditional problem-solving approaches, potentially leading to premature judgments by interviewers before solutions are reached. While companies emphasize understanding a candidate’s thought process, the current approach unrealistically assesses it, disadvantaging certain individuals. Our general suggestion around assessing someone’s thought process is to instead ask candidates after their have completed a task (either asynchronously or on a call) about the tradeoffs they made and why they made them. This better represents the real-world and also provides signal into how candidates arrive at a solution.

Hypothetical Unrealistic Tasks

Due to the time-constrained and live nature of these evaluations, interviewers often pose hypothetical, ambiguous problems. They also do not provide clear enough instructions, as it would take too much time in a time-constrained environment. One suggestion for more inclusive interviews is to avoid such open and hypothetical queries.

Discomfort and Unfamiliar Environment

The most common complaint is that live coding interviews pull individuals out of their comfort zones. Candidates are often compelled to use unfamiliar online IDEs and are not allowed to utilize their preferred tools or environments. Furthermore, candidates typically lack prior knowledge of the interview format or environment. To enhance accessibility, it is crucial to inform candidates about the interview beforehand, allow time for acclimation, and offer a comfortable and realistic working environment.

Our Vision of the Future of Technical Interviews

We founded Hatchways with the vision that the future of technical interviews should resemble actual work scenarios. While Randstad recommends work trials as the most effective and accessible form of assessment, we believe creating a “work-simulated” experience is the next best approach. Hence, our assessments are designed to mirror real-world tasks.

Mimicking Real-world Scenarios

We craft our assessments to reflect real job responsibilities, avoiding hypothetical questions. For example:

• Candidates receive a GitHub repository containing the initial codebase of a project.
• They are tasked with addressing a ticket or issue on the project or reviewing a pull request.
• Completion involves creating or reviewing a pull request, as they would in actual job scenarios.

Remove the Time Pressure

Our platform doesn’t impose rigid time constraints. Companies employing our platform usually set soft time limits, often allotting more time than needed to accomplish the task.

Employing Familiar Environments

Instead of asking candidates to work on an online IDE that they are not familiar with, candidates clone the project to their computer and use whatever tools they are comfortable with. This is not only closer to the job experience, it is also the most accessible way for someone to code.

This also creates the added benefit that you don’t watch a person code in our platform. Companies generally use our platform to provide take-home assessments, which have been identified as one of the formats that increases accessibility.

Provide Clear Instructions and Leverage Rubrics

Our assessments, structured like real-life tasks, come with detailed instructions, accommodating individuals, like those on the Autism Spectrum, who prefer clear directions. Companies using our platform employ structured rubrics, ensuring assessments focus on relevant skills.

Create a Space for Questions

We acknowledge the importance of queries during assessments, a feature often lacking in conventional take-home interviews. Thus, our platform includes a feature allowing candidates to ask questions during assessments.

Some companies extend interaction by inviting candidates to Slack or Microsoft Teams channel, enabling them to connect with multiple team members.

Case studies

With this innovative interview process, we've made strides in fostering accessibility, assisting candidates with disabilities on their employment journey. You can listen to some of their inspiring stories on our “Dear Hiring Manager” podcast. Here are a few notable ones:

• Applying for jobs with ADHD and OCD:
• Evan has a combination of challenges including growing up with tourettes syndrome, having ADHD, and being diagnosed with OCD. The only thing about these barriers that impact his work is his ability to focus on more mundane tasks, and in fact, the advantage of having ADHD is his ability to hyper-focus on really challenging and stimulating problems for extended periods of time. But, the current process of applying for jobs is one that makes it difficult for someone like him, who needs to spend hours on end on applications which spurs a lot of doubt and frustration.
• My Stutter is a Strength not a Weakness:
• Sam, who is a talented software developer working in Ottawa, Canada developed a stutter when he was a child. His stutter does not affect his ability to problem solve or be a software developer, but it is heightened when he is put in high anxiety environments, such as an interview.
• Phone Interviews with Hearing Impairment
• Aecio is a software developer working in Canada who is an immigrant from Brazil and in addition to being new to a country with no network or connections when he was first in search for a job, he also faces the barrier of fairly severe hearing loss. He shares his experience about how he developed hearing loss, how it impacted his education and job prospects, and how the current hiring process is inaccessible to people with hearing impairment.

For all of these examples above, our platform leveraged our unique interview style to help identify these candidates as strong software engineers, while the traditional process failed them.

Conclusion

At Hatchways, we have been focused on improving the accessibility of our platform and creating a process that works for all types of applicants. We are seeing a trend towards more practical assessments, which not only better represents on-the-job skills but also is a more accessible process.

‍