BlogLine

AI for hire: The well-intentioned bias 

7/17/24

AI

By: Christopher J. Fulmer

As companies seek to maximize productivity and efficiency, some employers are attempting to harness the innovative power of Artificial Intelligence (AI) to find the “best” job applicants without having to spend valuable manhours sorting through applications and resumes. A word of caution, however, is warranted for employers who seek to streamline the hiring and interviewing process through the implementation of AI. While it may seem like a sure-fire way to speed up the hiring process and provide a filter for siphoning the best candidates from the applicant pool, AI brings with it the inherent danger of discriminatory bias, no matter how well-intentioned the protocols. The good news is that the Equal Employment Opportunity Commission (EEOC) and the Department of Labor (DOL) are quickly developing guidance to help employers navigate the use of AI in the hiring process. 

AI systems can unknowingly favor certain classes of applicants over others because AI systems learn from historical data. Thus, if an employer inputs data which carries some inherent bias, the AI system will adopt the biases and perpetuate the systemic prejudices of the data.   

For example, Amazon developed an experimental automated recruitment system, which was intended to evaluate job applicants based on talent. Amazon’s AI models observed patterns in resumes submitted to the company over a 10-year period, most of which were submitted by male applicants. Essentially, the AI system concluded that Amazon preferred male applicants and penalized resumes that referenced females, such as participation in women’s sports in the education section of a resume.  When the company discovered the discriminatory effect, it scrapped the program.1 

Amazon is not the only company to witness AI’s potential for hiring bias. Last year, the Eastern District of New York saw the first settlement of an AI discrimination lawsuit in EEOC v. iTutorgroup Inc. et al.2 In that case, the Equal Employment Opportunity Commission (EEOC) alleged the company’s hiring software discriminated against older applicants in violation of the Age Discrimination in Employment Act (ADEA). The company paid $365,000 to settle the case after a female applicant submitted two applications: one with the woman’s actual date of birth, and one with a much younger date of birth. The applicant was contacted for an interview after she submitted the second application with a younger birthday. In addition to the settlement, the defendant company agreed to a litany of injunctive relief including being enjoined from: screening applicants based on age, requesting dates of birth before making a job offer, providing EEOC-approved training to all supervisors and managers focusing on the ADEA, revising the company’s anti-discrimination policies, implementation of a complaint process for employees, and EEOC monitoring. 

Another case from the Northern District of California involving claims of discrimination on the basis of race, age, and disability has received support from the EEOC as amicus curiae.3 The Plaintiff, an over-40 African American man with depression and anxiety, alleged that the defendant, Workday, relied on an algorithm-based AI system to screen out African Americans, applicants over forty, and applicants with disabilities due to data input created by humans who have built-in motivations, both conscious and unconscious, to discriminate.   

Mobley is still in litigation, with Workday’s motion to dismiss the first amended complaint pending as of the date of this writing. However, Mobley and iTutorgroup indicate the EEOC is prioritizing AI hiring bias in the field of employment discrimination. To that end, the EEOC and the Department of Labor (DOL) have both published important guidance to assist employers who seek to use AI systems to assist in the recruitment process.4    

Employers should carefully evaluate the EEOC and DOL guidance to determine their own compliance in the use of AI systems. To get started, some measures that should be taken before using AI include: 

  • Evaluate AI data and algorithms for inherent biases prior to implementation; 
  • Develop a bias mitigation plan and continuously monitor for discriminatory effects; 
  • Communicate and inform AI developers about potential biases while the AI system is being built and define clear nondiscriminatory guidelines for screening applicants. 

Simply put, companies should not be asking about potential areas for discriminatory data to become an issue. Employers must determine whether the use of AI disadvantages individuals based on a protected trait. Employment screening should not take into account or inquire into an applicant’s age, sex, race, national origin, religion, or disability status.   

For more information, please contact Christopher J. Fulmer at Christopher.fulmer@fmglaw.com, or your local FMG attorney. 

  1. See https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G/ ↩︎
  2. JVR No. 2310200016, 2023 WL 6998296 (E.D.N.Y. Sept. 8, 2023) ↩︎
  3. Mobley v. Workday, Inc., No. 23-cv-00770-RFL, WL 2813608 (N.D.Cal. April 23, 2024). ↩︎
  4. U.S. Equal Emp. Opportunity Comm’n, The Americans with Disabilities Act and the Use of Software, Algorithms, and Artificial Intelligence to Assess Job Applicants and Employees, (2024)  https://www.eeoc.gov/newsroom/eeoc-releases-new-resource-artificial-intelligence-and-title-vii; U.S. Dep’t of Labor, Department of Labor’s Artificial Intelligence and Worker Well-being: Principles for Developers and Employers,  https://www.dol.gov/general/AI-Principles.  ↩︎

Share

Save Print