Future Faces of Research: Gender Bias in AI

Is the AI we use to understand the world might be restricting the way we view the world? AI systems are playing an increasingly important role in analysis, prediction, and decision-making. However, they are trained on mainstream and uncritical datasets that reproduce stereotypical gender roles, abilities, and behavioral patterns, reinforcing biases, which leads to a limited understanding of society.

The most important thing AI needs right now is people who understand humans who can ask better questions, build with empathy, and design for the full complexity of real life. That has always been a strength women bring. 

– Francesca Lazzeriis, Principal Group Director of Data Science and Applied AI at Microsoft

My interest in this issue began when I asked ChatGPT a seemingly simple question: “How should the mayor’s son call the mayor?” It answered without hesitation: “Dad” or “Father.” When I further asked, “Why not mom?” the model then made a series of justifications, claiming that female titles should not be used unless the question is explicitly specified, “the mayor’s daughter,” or stated as “whose mother is the mayor.” This interaction made me deeply realize that a woman occupying a position of political authority is treated as an exceptional phenomenon that requires special annotation rather than a natural possibility. The default holder of power is not imagined as a woman.

This small interaction is just the tip of the iceberg. The same pattern runs deep across large language models. Men are consistently associated with leadership and authority, while women are linked to caregiving and domestic roles. These biases are not accidental but are deeply embedded in the data that trains AI. Training datasets, often treated as objective, are shaped by the people who build them and the histories they inherit. Those people are not a neutral representative sample of society. Women constitute only 16% of tenure-track AI faculty, 12% of STEM C-suite leaders, and fewer than 10% of CTOs or CIOs. Even as more women enter the field, the data AI learns from is built on decades of historical inequality. Twenty years ago, far fewer women held senior roles in public or private offices. Although current data collection strives for equity, the historical texts, archival records, and media reports have been encoded into contemporary models and future predictions. This causes AI to default to gender roles, thereby solidifying at the algorithmic level a social imagination that should have moved beyond.

Recognizing this pattern brings us to a crucial question: what kind of future are we building with AI? With its rapid development and widespread application, AI tools are influencing knowledge, information, culture, and value. The gendered assumptions embedded in AI models reflect real-world inequality while amplifying it through the large-scale algorithms. Are we expected to simply accept such a future? Certainly not. One meaningful step we can take is to identify and critique gender bias in AI systems, and to intervene consciously in daily interactions with AI systems. Engineers alone cannot solve the social, ethical, and political problems that AI produces. We need people to question default assumptions, refuse to normalize biased outputs, and offer feedback. This is needed to create neutral and equitable technology, supporting social justice rather than solidifying inequality. If you’d like to be in a community that talks about these issues, you’re welcome to join the Feminist AI Research Group.