The IEAI Discusses AI and Moral Advice

“Pandora’s Bots? Studies of AI Moral Advice and Their Implications for Human Decision-making”

…people tend to underestimate how much they are influenced by the AI.

AI is increasingly relied upon to support morally consequential decision-making. How much do people trust AI moral reasoning – and how much is too much? Eyal Aharoni, an Associate Professor of Neuroscience, Philosophy and Psychology at Georgia State University, was our invited speaker on June 16th, 2025. He delivered an engaging presentation on “Pandora’s Bots? Studies of AI Moral Advice and Their Implications for Human Decision-making” that was moderated by IEAI Research Associate Franziska Poszler.

Moral reasoning is regarded among the most sophisticated and unique of human faculties.

Prof. Aharoni questioned if large language models (LLMs) too, are up to the task. Although they seem to outperform and enhance human reasoning in various ways, they have their disadvantages: bullshit generation, opacity, bias, high virulence and diffusion of responsibility. He then submitted two questions for consideration:

Do LLMs have moral intelligence?

Do people perceive LLMs as having moral intelligence?

The latter of the two questions was tested using a moral Turing Test, where participants were presented with moral and conventional transgression scenarios and two evaluations to each scenario. One of the evaluations was human, the other LLM. Participants were asked to rate the two evaluations based on accuracy. They were then informed one of the evaluations was an LLM and were asked to identify it. The test concluded participants perceived the AI’s moral responses as superior in quality.

If people regard these AIs as more virtuous and more trustworthy, as they did in our study, they might uncritically accept and act upon questionable advice.

Prof. Aharoni then shifted focus to the use of AI to help support medical triage. The goal was to provide faster support with fewer errors, but could AI outperform medical practitioners? Or is the solution to focus on maximizing human performance instead? Further study into the topic revealed the AI performance paradox.

… it presents this sort of paradox that as AI performance increases, human performance will tend to decrease, because we rely on it. … even if the AI’s judgement is superior on average to professional humans, it’s still a problem because we’ll tend to outsource our decision-making, potentially beyond its scope of training.

To combat the problem, Prof. Aharoni looked into ways to ensure constructive human-AI collaboration.

The technology is not the problem; it’s the humans we have to worry about.

According to Prof. Aharoni, we must evolve from regulation to a human-centered design approach.

I think there’s an opportunity for scientists to go beyond a list of rules on how we promise we’re going to use the technology into a list of design features that actually help people.

In the discussion that followed the 40-minute presentation, topics such as worries about loss in critical thinking, the role of AI and decision fatigue as well as concerns about AI limiting human control to make and take decisions were touched upon.

We extend our appreciation to Prof. Aharoni for sharing his valuable insights during the talk and for actively engaging with the IEAI community on these crucial and highly relevant issues. Additionally, we would like to thank all our event registrants for their participation. The recording of the event can be found here.