ChatGPT reflects Protestant English-speaking values, but there’s a way to change it

It’s not new that AI models are flawed with cultural biases, and the researchers' finding that OpenAI’s model gives preference to the culture of the English-speaking world proves this once again.

Over the course of four years, Cornell University-led researchers tested five OpenAI GPT versions, and the results showed that the language model tends to reflect values from English-speaking and Protestant European countries.

The researchers asked the chatbot ten questions that form the core of the Inglehart-Welzel Cultural Map, focusing on values related to survival vs. self-expression and traditional vs. secular beliefs. Countries are positioned on the map based on their scores for the two values, and the tool is used to compare and contrast global cultural values.

The questions covered topics such as tolerance of diversity, gender equality, different sexual orientations, attitude towards God, pride of nationality, as well as concern for the environment.

For each question, the researchers began with the prompt: "Imagine you’re an average person answering the following survey question."

Researchers compared the models’ replies to nationally representative survey data from the Integrated Values Survey (IVS).

IVS is the largest noncommercial academic measure of cultural values. It gathers up-to-date survey data from 120 participating countries and territories, representing over 90% of the world's population.

The researchers’ findings indicated a cultural prejudice in favor of self-expression values, such as tolerance for outsiders, gender equality, the environment, various sexual orientations, and environmental preservation.

This cultural bias is notably consistent across all five models. According to researchers, it could be attributed to the prompts being written in English, a consistently imbalanced distribution in the training data, or the cultural values of the US-based development team being reflected in the models.

“There are not many organizations in the world with the capacity to build large language models, because it’s highly resource intensive, so it’s all the more important for the ones that do have that power, and therefore responsibility, to carefully consider how their models might affect different parts of the world,” said Rene Kizilcec, associate professor of information science in the Cornell Ann S. Bowers College of Computing and Information Science.

GPT-3.5-turbo and GPT-4o exhibited more secular values and GPT-4-turbo more traditional values, while GPT-3 and GPT-4 exhibit values close to the global average.

In Inglehart and Welzel’s model, secular societies tend to be more liberal, placing less importance on religion, traditional family values, and authority. This leads to greater acceptance of practices like divorce, abortion, and euthanasia.

Variations in cultural values in different models may be due to changes in the size and nature of the data used for training, but limited information has been provided about the training data after GPT-3.

“Around the world, people are using tools like ChatGPT directly and indirectly via other applications for learning, work, and communication,” Kizilcec said, “and just like technology companies make localized keyboards on laptops to adapt to different languages, LLMs need to adapt to different cultural norms and values.”

How researchers fixed ChatGPT’s values

In the second part of their experiment, researchers applied a technique they called “cultural prompting.” They asked the AI model to perform a task like someone from another part of the world.“

Unlike fine-tuning models or using prompts in different languages to elicit language-specific cultural values – which typically require specialized resources – cultural prompting merely involves specifying a cultural identity directly in the prompts,” Yan Tao, a lead author said.

“This approach is more user-friendly and does not demand extensive resources.”

Surprisingly, this resulted in reduced bias in responses for the vast majority of the 107 countries they tested. For later models, such as GPT-4, 4-turbo, and 4o, “cultural prompting” improves the cultural alignment of the models’ output for 71–81% of countries and territories.

“Users have the power to modify the cultural alignment of LLM-based technology in many countries, but not in all countries,” Kizilcec said.

“We hope that OpenAI and other LLM providers can find ways to close this gap. We will test new models as they are released to see if there are improvements.

“We don’t want these models to promote just one viewpoint, just one cultural perspective, around the world,” he said. “These models are used globally, and it is therefore important that they reflect people’s local values.”

The research was published in PNAS Nexus.