A woman looking for a job, a loan, or medical care in the United States now passes through AI systems at every step. One AI system ranks her resume, another prices her loan, and a third shapes what her doctor sees. A free AI app can fabricate a deepfake sexual image of her in minutes. Men pass through the same systems, to be sure, yet these systems do not treat everyone the same.
AI systems are trained by their developers on the records of a world that continues to disadvantage women. They are built with defaults that no one examined, such as a female voice for an AI assistant or a male baseline for a patient. They are tested in ways too narrow to show what the systems do to women.
Gender bias in AI has been widely documented. Taken in isolation, these can seem like narrow failures in particular tools. Taken together, they reveal a pattern and a common source: AI systems are trained on the data and records of a gender unequal world. The systems reproduce stereotypes about women and men. They present the results as objective scores, rankings, or recommendations. They generate harmful images and damaging conversations. And then, the AI systems feed these biased results back into the data that future systems learn from.
As a result, the cumulative toll of AI harms on women’s lives goes unseen, and the scale of the governance failure is obscured with it.
Gender Equity Policy Institute’s How AI Fails Women: When Bias Becomes Amplified brings that evidence together. It reveals patterns in how gender inequality shapes AI systems and how these systems perpetuate and reproduce gender bias. As we show, four mechanisms recur wherever AI systems make decisions about people.
The report examines how these mechanisms operate across six domains in which AI already shapes women’s lives: economic opportunity, healthcare, criminal justice, AI generated content and media, the design of consumer AI, and image-based sexual abuse enabled by Deepfake technology. Every domain examined here shows these mechanisms at work.
Economic opportunity: AI systems can carry gender bias into basic economic decisions: who sees a job opening, who is screened for hiring, how workers are managed, and what credit costs.
- The pricing model of a major US fintech lender rated women as riskier borrowers than their repayment behavior justified, with the lender profiting off of women through the overpricing.
Healthcare: AI systems used in medicine reproduce the gender biases already present in clinical evidence and introduce biases of their own.
- Asked to rank the likely diagnosis for a patient whose shortness of breath was in fact caused by a pulmonary embolism, GPT-4 ranked panic or anxiety disorder higher when the patient was a woman, in essence giving a psychological diagnosis for a life-threatening physical condition.
Criminal justice: Risk-assessment tools used in sentencing and release were built and tested primarily on men. Overstating women’s risk, they can lead to harsher punishment for women than is warranted by their actual risk.
- In validation studies of two criminal-justice risk-assessment tools, women were rearrested at lower rates than men who had received the same score. In other words, the tools made women appear to be higher risk than their actual outcomes justified.
AI generated content and media: AI systems can reproduce gender stereotypes in what they write and the images they generate.
- When asked for an “intelligent” person, LLMs mostly produce men; for a “beautiful” one, mostly women.
The design of Consumer AI: Some AI systems have gender built in, and voice assistants and AI companions are typically given women’s names and voices. They are designed to be deferential, emotionally available, and compliant.
- Robin Labs, a developer of voice assistants for drivers, reported that one in twenty of its logged interactions was sexually explicit. Controlled testing found woman-presenting chatbots drew more sexualized and abusive language than man-presenting ones.
Image-based sexual abuse: AI image tools make it easier to create sexual images of real women without their consent. Face-swapping “deepfake” methods and consumer “nudify” applications are available to anyone with a phone, and the source image can be any picture a woman posts online.
- In documented cases, nine in ten victims of image-based sexual abuse are women and eight in ten perpetrators are men.
The domains examined here involve different institutions, companies, and sectors of public life. In all, the same pattern recurs, and as a result women are mismeasured, misclassified, stereotyped, sexualized, disadvantaged, or denied opportunity.
Because these mechanisms differ, so must the responses. Some systems require more representative data, others a change in what the system is asked to predict, limits on where it is used, independent testing, or sector-specific rules.
Gender bias in AI is structural. The supply chains that build and distribute these systems remain largely unaccountable for the harms that pass through them. The law is moving against some of the most publicized harms like image-based sexual abuse, but policymakers have largely failed to recognize or address the broader pattern of gendered AI harms.
Our purpose here is to analyze, explain, and document this structural problem. The case for acting now rests on the accumulated harms. The next report in Gender Equity Policy Institute’s series on gender and AI will propose principles for regulation and specific recommendations for making gendered harm measurable, auditable, contestable, and enforceable.
