Australia can learn from New York City’s new rules that offer a glimpse into the future of AI regulation in HR.
Key Takeaways
- There is currently no generally applicable law regulating the use of AI in Australia, however Voluntary AI Ethics Principles exist at a Federal Level and NSW State Government is leading the pack by implementing an AI Assurance Framework.
- There are a range of consultations, reform proposals on the table by Australian Government bodies that are looking at a narrower approach to regulating the use of AI by focusing on high risk cases.
- Australia can learn from New York City’s law requiring independent audits of AI tools used in hiring decisions that went into force in July 2023.
- Experts say the current situation offers insight into how future regulation might evolve and what employers can do now to prepare for responsible AI use and potential future regulatory requirements.
In July 2023, New York City began enforcing the world’s first law regulating the use of AI software for hiring and promotion decisions – also known as automated employment decision tools (AEDTs). Though Local Law 144* is only applicable to companies that operate or hire in New York City, it’s considered a bellwether for regulation around the use of AI in HR*. Employers throughout the United States (and beyond) have looked to the law to help them anticipate future regulations. In Australia, whilst we don't have specific AI legislation, there are various laws that affect the development or use of AI in Australia. A non-exhaustive list of these laws include:
- The Online Safety Act 2021 which includes mechanisms to address online safety issues, extending to AI generated material
- The Australian Consumer Law, which was applied to algorithmic decision making in a Federal Court case which ordered Trivago to pay $44.7 million in penalties for misleading consumers about room rates in the recommendations made by its algorithm
- The Privacy Act 1988
- The Corporations Act 2001
- Intellectual property laws may affect several aspects of AI development and use
- Anti-discrimination laws, for example, where an individual is a victim of a discriminatory outcome resulting from an AI-driven process
The Australian Government has acknowledged that existing laws will likely need to be strengthened to address harms posed by AI. To that end, the Australian Government launched the interim response consultation to safe and responsible AI and is currently developing new laws that will provide the Australian Communications and Media Authority ("ACMA") with regulatory powers to combat online misinformation and disinformation, extending to content on digital platforms that are generated by AI.
So what can Australia learn from the US?
Unfortunately the first year the AI hiring law has been in effect in the US hasn’t had much of an impact*.
Local Law 144 mandates that employers using Automated Employment Decision Tools (AEDT) conduct independent annual audits for racial and gender bias and disclose the results on their websites. Employers also need to include a notice for job seekers about their use of AI tools and allow applicants to request alternative screening methods.
Researchers from the Citizens and Technology (CAT) Lab at Cornell University, the Data & Society Research Institute and Consumer Reports surveyed 391 companies with operations in New York City. They found that just 18 companies have published audit reports, and 13 posted transparency notices for job seekers.
In most cases, the researchers concluded, it isn’t necessarily a case of wilful non-compliance, but rather confusion about the rules and challenges with the requirements.
Why is this happening? We asked experts to identify key challenges and how these might influence future legislation, and what employers can do now to get ahead of evolving regulations.
A narrow definition of AEDTs limits the AI hiring law’s reach
Local Law 144 gives employers discretion to determine whether their AEDTs fall under the legislation’s scope. Though the law’s original language encompassed a wide range of AEDTs, including resume-scanning and applicant-ranking tools, the final guidance that was published when the law went into force* was much narrower in scope: The New York Department of Consumer and Worker Protection (DCWP) defines an AEDT as something that 'substantially assists or replaces discretionary decision-making'.
This definition suggests that AEDTs only need to be audited if they are the single most important factor in the decision-making process or if they replace human decision-making altogether – which, in practice, is quite rare. “I think that, outside of gig work, there’s just not a lot of fully automated employment decisions being made,” says Trey Causey, Head of Responsible AI and Senior Director of Data Science at Indeed.
For many employers, therefore, the AI hiring tools they use don’t fall under the scope of the law. Matt Scherer, Senior Policy Counsel for Workers’ Rights and Technology at the Center for Democracy and Technology, points out that companies aren’t required to conduct audits if they’ve established a process of human review, or if a human has the final say in an automated decision process – “even if that human, in practice, acts as a rubber stamp.”
How it might affect future regulation: Lawmakers continue to debate what the definition of an AEDT should be. And in future legislation, a broader range of tools may be subject to audit requirements.
How employers should prepare: Any AI tool used in hiring can exhibit bias. Even if a tool doesn’t completely replace human decision-making, employers should proactively audit their AI models to avoid reputational risk and ensure fair hiring practices. With good audit practices in place, employers will be well prepared for any regulation on the horizon.
Employers lack the historical data the law demands
Making a world of better work isn’t for the faint of heart. But it’s worth it.
If this is real, if this is how you want to do business, then now is the time to be loud about your ESG commitments. Ideally, you wouldn’t have to be. But the reality is that, when you stay quiet, it at best encourages other leaders and organisations to do the same and, at worst, may discourage them from pursuing these commitments in the first place.
You may get both internal and external backlash, but that can also happen if you don't take a stand. You have to decide as a business leader at this moment: What side of history do I want to be on?
AEDT audit reporting may expose companies to risk
Local Law 144 requires employers to publish reports that show bias against protected groups* based on applicants’ race/ethnicity, sex and any intersection of these characteristics. Employers must show the rate at which applicants in each group are selected for jobs and compare that to the rate at which applicants in all other categories are selected. They must then calculate an 'impact ratio' that shows whether any group is disparately affected by the employer’s use of the AEDT.
“But the law doesn’t say, ‘This is the threshold for how much bias is acceptable or unacceptable,’” Appel says. There are also no guidelines for how to remediate discriminatory AI.
Publishing an audit report that shows AEDTs have a disparate impact could potentially open employers to a discrimination suit, which could be a bigger burden than the law’s relatively low penalty fees. The first violation of Local Law 144 incurs a maximum fine of 746.26 AUD, with additional violations penalised between 746.26 AUD and 2238.79 AUD. “A discrimination claim brought by a single worker can easily exceed that amount, to say nothing of class actions,” Scherer says.
How it might affect future regulation: Future laws need clearer structures with stronger incentives to comply, Scherer says. “That includes clear definitions of what sorts of tools are subject to the law and stronger enforcement provisions so companies don’t feel that the cost-benefit analysis weighs in favour of not telling people that they’re using an AI tool,” he says. Causey recommends that lawmakers create a 'cure period' during which companies can conduct audits and remediate issues without fear of penalties.
How employers should prepare: Companies might wish to define their own thresholds for how much bias is acceptable and define a process for responding to audit results. “What are the accountability mechanisms?” Causey asks. “If an audit reveals something, what is the recourse?” This is about more than complying with regulations – establishing principles around your organisation’s use of AI is an important best practice.
What’s next for AI laws in Australia?
Whilst we don't yet have a formal legislation, we all know that the regulatory environments and demands will evolve as AI does.
While the government shapes Australia's laws, defining their application across various industries like HR, it's crucial to consider the US experience. As the world's inaugural global AI hiring law, it could significantly influence the direction of other countries' regulations.
Responsible AI advocates say that, in addition to addressing the issues above, future laws should require audits to include more protected groups. “Very broadly speaking, race and gender are two of the areas of bias that people are often worried about when it comes to algorithmic systems, but that doesn’t mean they’re the only things worth worrying about,” Appel says.
For example, certain AI applications may be discriminatory against those with disabilities. “If you’re talking about a video interview or an online digital assessment, what are you doing to accommodate candidates who are blind or have fine motor impairments?” Scherer says. “You’re using a screening test that may be not really about measuring somebody’s ability to perform the job but their ability to take the assessment.”
Understanding how regulators are grappling with the use of AI in hiring isn’t just important for anticipating future laws. Exploring these issues can help employers define what responsible AI means for their organisation and implement processes to ensure their tools aren’t just compliant but effective in creating better, more efficient and more equitable hiring practices.
“There is a shared responsibility in new technology,” Causey says. Regardless of what laws are enforced in future, “there’s an onus on employers to think about their processes being fair and as unbiased as possible.”
Here are our top 3 recommendations for Australian employers
- Consider the transparency of use of AI tools during applicant screening.
- Consider how your organisation can safely and securely collect data on the race and gender of job applicants to more accurately evaluate bias in AI models
- Establish clear principles around your organisation's use of AI as a best practice model. Using the AI Ethics Principles and AI Assurance Framework.
*Hyperlinked webpages lead to English-language content
*The values in this article reflect the market at the time of writing and may not reflect the current market conditions due to fluctuations.
