Generative AI models, such as ChatGPT, Google’s Bard, and Midjourney, have been increasingly used by people personally and professionally in recent months. However, recent research suggests that these models may encode biases and negative stereotypes in their users, as well as generate and spread seemingly accurate but nonsensical information. This fabrication of nonsensical information is disproportionately affecting marginalized groups, which is concerning.
Furthermore, the potential impact on human beliefs cannot be ignored as the models that drive it become increasingly common and populate the World Wide Web. It is important to note that not only do people obtain information from the web, but much of the primary training material used by AI models is also sourced from the web. This results in a continuous feedback loop where biases and nonsense are repeated and accepted repeatedly.
These findings, along with a plea for psychologists and machine learning experts to collaborate quickly to assess the extent of this issue and develop solutions, are published today in a compelling Perspective in the leading international journal, Science. The co-author of this piece is Abeba Birhane, an adjunct assistant professor at Trinity’s School of Computer Science and Statistics (working with Trinity’s Complex Software Lab) and Senior Fellow in Trustworthy AI at the Mozilla Foundation.
Prof Birhane said:
“People regularly communicate uncertainty through phrases such as ‘I think,’ response delays, corrections, and speech disfluencies. By contrast, generative models give confident, fluent responses with no uncertainty representations nor the ability to communicate their absence. As a result, this can cause greater distortion compared with human inputs and lead to people accepting answers as factually accurate. These issues are exacerbated by financial and liability interests incentivizing companies to anthropomorphize generative models as intelligent, sentient, empathetic, or even childlike.
The Perspective provides an example of how statistical regularities in a model assigned higher risk scores to Black defendants. Judges who learned the patterns may change their sentencing practices to match the predictions of the algorithms. This basic mechanism of statistical learning could lead a judge to believe that Black individuals are more likely to reoffend–even if the system is stopped by regulations like those recently adopted in California.
It is particularly concerning that biases or fabricated information are difficult to shake once they have been accepted by an individual. Children are especially vulnerable to belief distortion because they are more likely to anthropomorphize technology and are more easily influenced.
What is needed is a detailed analysis that quickly measures the impact of generative models on human beliefs and biases.
Prof Birhane said:
“Studies and subsequent interventions would be most effectively focused on impacts on the marginalized populations who are disproportionately affected by both fabrications and negative stereotypes in model outputs. Additionally, resources are needed for the education of the public, policymakers, and interdisciplinary scientists to give realistically informed views of how generative AI models work and to correct existing misinformation and hype surrounding these new technologies.”
