How to avoid AI bias in recruiting

The Artificial Intelligence (AI) industry is fast becoming a behemoth, with the global AI recruitment market expected to nudge US$1 trillion by 2029. Yet it’s important to remember that it's still early days for the technology. AI-assisted video interviewing software, for example, was pioneered less than 10 years ago. The term “child-like” is often employed by programmers to help explain AI’s learning curve as it absorbs information, with most experts agreeing that modern-day AI is about as smart as a four-year-old. And just like a human child, AI can make mistakes. 

Companies use AI-enabled recruitment tech to write effective job posts, automatically match candidates to jobs, automate candidate communication with natural language-enabled chatbots, filter resumés through keyword scanning and even analyse one-way video interviews. 

Commonly quoted benefits include improving quality-of-hire, boosting the candidate experience, speeding up time-to-hire and – importantly – minimising or eliminating bias in hiring by discounting demographic information such as gender and age. But AI is only as good as its inputs, which means it can also unintentionally introduce hiring bias. 

How AI introduces hiring bias

While rules-based AI such as keyword-scanning software is straightforward, AI Machine Learning (ML) is infinitely more complex and prone to bias. Programmers teach the algorithm by feeding it massive data sets such as the organisation’s recruitment data for the past decade. The AI/ML tools may be instructed to search for patterns among successful candidates’ resumés or performance reviews to discover “what good looks like”, then apply those learnings to future candidate screening. But this is where the problem lies.

We’ve come a long way towards addressing gender inequality in hiring, but the fact remains that a decades-long recruitment data set in the average organisation would show, for example, that more men were hired than women. The AI would spot this pattern and begin favouring male candidates in its screening process unless this issue had been accounted for by the programmers. 

Even recruitment marketing AI can be biased, with Facebook accused of deploying a job targeting algorithm that propagated gender biases. Nursing jobs, for example, were seen almost exclusively by women while adverts for mechanics were seen mainly by men. Ongoing Berkeley Haas study of bias in artificial intelligence found that 44% of AI systems demonstrate gender bias and 26% additionally display race bias. 

Other bias-laden patterns that AI can learn from resumés include a preference for university-trained applicants (discriminating against those who didn’t have the opportunity to pursue a tertiary degree) or a trend where people are typically hired from wealthier postcodes. A report by Wired lists plenty more examples: penalising applicants for having a Black-sounding name, discriminating against people who stutter in video interviews, or (perplexingly) assigning a lower score to people wearing spectacles. 

A German study of video-hiring AI by Bavarian Broadcasting revealed how odd examples of bias can unintentionally creep in. Candidates were rewarded for apparently arbitrary reasons such as being in a brighter room or sitting in front of a bookshelf but penalised for wearing a hat or headscarf. In this instance the AI is not learning from human bias in the data set but is simply recognising and reacting to what should be an irrelevant pattern; that is, many “good” candidates in the data set did not wear something on their head. 

A 2020 Stanford study found that Black speakers are more likely to be misunderstood (and therefore penalised in an AI hiring situation) by automated speech recognition software with a 16% higher word error rate compared with white speakers. 

Regulators in the US and elsewhere are belatedly taking action to address bias in AI recruitment, including Illinois’ Artificial Intelligence Video Interview Act, new rules for AI in the EU that flags recruitment AI as high-risk, and a total ban on AI candidate screening pending a bias audit by the New York City Council. Australia launched an Artificial Intelligence Ethics Framework in 2019 but lags behind other nations in terms of regulation.  

Three ways to defend against recruitment AI bias

While there are clearly some issues with AI-supported recruitment, for some organisations, the advantages still outweigh the risks. There are also specific actions these organisations can take to minimise the risk of hiring bias when using AI technologies.

1. Conduct ongoing audits. The nature of ML is that it continuously learns and evolves, meaning a one-off audit will be ineffective. Use an external party to regularly test the model for known biases and gather feedback from users including candidates and the talent acquisition team. 
2. Clean the data. Less is more. Rather than piling in more data, only feed the ML model data that it makes sense to learn from. In the Bavarian Broadcasting example mentioned above, the AI learnt from what should have been irrelevant patterns such as the candidates’ spectacles or background furniture. 
3. Ensure the AI’s hiring recommendations are explainable. As PwC notes, the need to know how AI arrives at its decisions has become business critical as it grows in complexity and sophistication, especially given what we know about the algorithms’ high level of susceptibility to human bias through data sets. In practice, this should mean a hiring manager should be able to easily find out why the AI screened in or out an individual candidate without having to engage an IT specialist. 

What to consider when purchasing hiring AI

For every benefit involved in hiring AI, there’s a caveat lurking around the corner. AI can automate resumé screening but unintentionally screen out great talent due to a missing keyword. It has the potential to vastly improve the candidate experience in high-volume situations but may also dehumanise the process. And while AI is often heralded as the silver bullet for eliminating bias, it has the potential to introduce new bias unless managed with care.

If your organisation is considering AI-supported recruitment, the first step is to look internally and define what that solution should accomplish. Why do we need recruitment AI? What exactly are we seeking to improve? What would success look like?

Once we have some basic parameters in place, here are a few questions to ask a prospective solution provider.

• What standards were applied to the model’s development? 
• What known biases have been removed from the data set? 
• Will the AI be able to explain its recommendations in terms I can understand? 
• Is the ML training data set relevant to my organisation? 
• Will the model continue to learn post-deployment? 
• How can the model be tested for introduced biases? How often should this happen?