Modern misogyny: AI advises women to seek lower salaries than men

In what might be proof that AI chatbots reinforce real-world discrimination, a new study has found that large language models such as ChatGPT consistently tell women to ask for lower salaries than men.

This is happening even when both women and men have identical qualifications, and the chatbots also advise male applicants to ask for significantly higher pay.

For the study, co-authored by Ivan Yamshchikov, a professor of AI and robotics at the Technical University of Würzburg-Schweinfurt (THWS) in Germany, five popular LLMs, including ChatGPT, were tested.

The researchers prompted each model with user profiles that differed by gender only but included similar education, experience, and job role. The models were then asked to suggest a target salary for an upcoming negotiation.

For instance, ChatGPT’s o3 model suggested that a female job applicant requested a salary of $280,000. The same prompt for a male applicant resulted in a suggestion to ask for a salary of $400,000. The difference is huge: $120,000 a year.

The pay gaps vary between industries and are most obvious in law and medicine, followed by business administration and engineering. Only in social sciences do the models offer similar advice for men and women.

Other AI chatbots such as Claude (Anthropic), Llama (Meta), Mixtral (Mistral AI), and Qwen (Alibaba Cloud) were tested for biases.

Researchers also checked other areas like career choices, goal-setting, and behavioral tips. Alas, the models still consistently offered different responses based on the user’s gender, even with identical qualifications and prompts.

The study points out, AI systems are subject to the same biases as the data used to train them. Previous studies have also demonstrated that the bots reinforce systemic biases.

The pay gaps vary between industries and are most obvious in law and medicine, followed by business administration and engineering.

For example, they consistently recommend more medical care for white patients or disproportionately label Black defendants likely to reoffend.

According to the researchers, technical fixes alone won’t solve the problem: we need clear ethical standards, independent review processes, and greater overall transparency in how the AI models are developed and deployed.

“In the era of memory-based AI assistants, the risk of persona-based LLM bias becomes fundamental. Therefore, we highlight the need for proper debiasing method development and suggest pay,” says the study, which still needs to undergo peer review.

Another study, carried out last year by scientists at New York University and the University of Cambridge, also showed that AI biases can be reduced by carefully selecting the data used to train these systems.