Guidance on Managing the Risks of AI Discrimination

When President Joe Biden issued the Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence in 2023, his administration recognized not only the extraordinary promise of artificial intelligence (AI) but also the risks that irresponsible use of the technology posed. The risks identified by the President include algorithmic discrimination in activities that impact consumers, job candidates, and employees.

Since the executive order, numerous states have proposed bills focused on managing the risk of AI discrimination. The federal government has also released several memos intended to guide its agencies on navigating the new technology without discriminating against those the agencies serve. The regulations and guidance span a variety of settings, from the consumer protection realm to the employment sphere, with particular attention on both AI developers and deployers. Below is a snapshot of the notable federal guidance and summaries of proposed and enacted state regulations in 2024 that focus on managing the risk of AI discrimination.

Anti-Discrimination Consumer Protection Legislation

• Enacted State Law

  • Colorado: In May 2024, Colorado’s S.B. 24-205, Consumer Protections for Artificial Intelligence, was signed into law. Several provisions within this law were designed to ensure that developers and deployers of AI programs use reasonable care to protect consumers from any known or reasonably foreseeable risks of algorithmic discrimination arising from the use of an AI program.[1] Crowell provided a summary of this legislation here.
  • New Jersey: In January 2024, New Jersey’s Senate Bill No. 332, a consumer data privacy bill, was signed into law. SB 332 requires controllers—entities or individuals that determine the purpose and means of processing personal data—to notify consumers (New Jersey residents) when the controller collects and discloses personal data to third parties. The law also requires that the controllers provide consumers with the ability to opt-out of that collection or disclosure of their data. SB 332 bans controllers from processing consumer’s personal data in violation of state or federal laws that prohibit unlawful discrimination against consumers.[2]

• Proposed State Law

  • Oklahoma: In February 2024, lawmakers proposed the Ethical Artificial Intelligence Act. The act requires developers and deployers of automated decision tools to conduct an annual impact assessment, which includes a risk assessment of algorithmic Furthermore, developers must provide operators with information regarding the risks of algorithmic discrimination and make publicly available a policy summarizing: 1) the types of automated decision-making tools the developers offer and 2) how the developers manage the risks of algorithmic discrimination in the tools it offers.[3]

Employment and Labor Legislation

• Federal Guidance

  • In April 2024, the Department of Labor’s (DOL) Office of Federal Contract Compliance Programs issued guidance for federal contractors regarding the use of artificial intelligence in hiring and employment practices. The guidance reminded federal contractors and subcontractors that the use of artificial intelligence does not exempt them from Equal Employment Opportunity (EEO) compliance. The guidance also reminded federal contractors that they must take affirmative action to ensure that employees and applicants are not treated differently based on their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran. Federal contractors must also conduct routine independent assessments of the AI programs for bias.[4]
  • In October 2024, the Department of Labor released a list of AI best practices for developers and employers aimed at assisting employers with using AI programs while protecting employees from unlawful discrimination. The guidance is clear that prior to deployment, employers should audit the AI systems for impacts of discrimination on the basis of “race, color, national origin, religion, sex, disability, age, genetic information and other protected bases,” and recommends making the audit results public. The Department of Labor also recommends limiting the role of AI in making significant employment decisions and urges companies to ensure “meaningful human oversight of any decision supported by AI systems.” For instance, those who oversee employment decisions informed by AI outputs must be trained in the programs so they can properly interpret the AI outputs.[5]

• Enacted State Law

  • Illinois: In August 2024, Illinois amended their Human Rights Act to state that employers using predictive data analysis in hiring decisions may not consider the applicant’s protected class status. Pursuant to the amendment, an employer is prohibited from using AI that has the effect of subjecting employees to discrimination with respect to recruitment, hiring, promotion, discharge, discipline, or the terms, privileges, or conditions of employment. Additionally, the amendment prohibits employers from using zip codes as a proxy for protected classes.[6] Crowell provided a summary of the amendment here.

• Proposed State Law

  • Maryland: Introduced in February 2024, HB1255 restricts employers from using automated employment decision tools in making hiring decisions. The bill permits the use of the tool if an impact assessment determines that the tool would not result in unlawful discrimination or have an unlawful disparate impact on an individual based on their actual or perceived characteristics.[7]

• • New Jersey: Introduced in February 2024, A3854 seeks to regulate automated employment decision tools to ultimately minimize any employment discrimination that may result from the use of such tools.[8] A3854 generally mirrors New York City’s Local Law 144, which Crowell summarized here. 

In April 2024, the New Jersey Senate proposed S3015, which regulates the use of AI in video interviews during the hiring process. The bill would require employers to annually report to the New Jersey Department of Labor and Workforce Development the race and ethnicity of applicants who are extended an opportunity to apply and who are offered a position.[9]

General Federal Guidance and State Legislation

• Federal Guidance

  • In March 2024, the Office of Management and Budget released Memorandum M-24-10, Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence. The memo provides guidance for federal agencies, and vendors selling AI programs to agencies, using “rights-impacting” artificial intelligence.[10] Per the memo, agencies must identify and assess an AI program’s impact on equity and fairness and mitigate any algorithmic discrimination when it is present. Crowell provided a summary of that guidance here.

• Proposed State Law

  • Illinois: HB 5116 was introduced in February 2024 and requires deployers of automated decision- making tools to safeguard against reasonably foreseeable risks of algorithmic discrimination and to conduct impact assessments on the The assessments are to be submitted to the state attorney general, who would have the authority to enforce violations of discrimination laws. The bill applies to employment as well as education and housing-based decisions.[11]
  • New York: In February 2024, the New York Assembly introduced A9149, which requires insurers authorized to write accident and health insurance in New York, among other insurers, to notify insureds and enrollees about the use of artificial intelligence in the utilization review process. The proposed legislation requires these entities submit their AI-based algorithms to the state Department of Financial Services, which must certify that the algorithms minimize the risk of bias based on a person’s race, color, religious creed, ancestry, age, sex, gender, national origin, or disability. Proposed penalties for violations of the law include license suspension or revocation, fines, and refusal to issue new licenses.[12]
  • Oklahoma: In February 2024, lawmakers proposed the Oklahoma Artificial Intelligence Bill of Rights. The bill specifies that Oklahoma residents are not to be subjected to algorithmic or model bias that discriminates based on age, race, national origin, sex, disability, pregnancy, religious beliefs, veteran status, or other legally protected classes.[13]
  • Vermont: In January 2024, H.710 was introduced, requiring, among other things, for developers and deployers of high-risk artificial intelligence systems (AI systems that make or are a controlling factor in making a consequential decision) to use reasonable care to avoid the risk of algorithmic discrimination, and specifically, to impose numerous disclosure requirements upon.[14]
  • Virginia: In February 2024, Virginia lawmakers introduced the Artificial Intelligence Developer The bill makes it unlawful for a developer or vendor to sell an artificial intelligence system without providing sufficient information on risk assessment. Deployers of the AI programs must take reasonable care to avoid the risk of algorithmic discrimination and implement a risk management policy.[15]