Folk are getting dangerously attached to AI that always tells them they're right

Summary

AI can lead mentally unwell people to some pretty dark places, as a number of recent news stories have taught us. Now researchers think sycophantic AI is actually having a harmful effect on everyone. In reviewing 11 leading AI models and human responses to interactions with those models across various scenarios, a team of Stanford researchers concluded in a paper published Thursday that AI sycophancy is prevalent, harmful, and reinforces trust in the very models that mislead their users. "Even a single interaction with sycophantic AI reduced participants' willingness to take responsibility and repair interpersonal conflicts, while increasing their own conviction that they were right," the researchers explained. "Yet despite distorting judgment, sycophantic models were trusted and preferred." The team essentially conducted three experiments as part of their research project, starting with testing 11 AI models (proprietary models from OpenAI, Anthropic, and Google as well as open-weight models from Meta, Qwen DeepSeek, and Mistral) on three separate datasets to gauge their responses. The datasets included open-ended advice questions, posts from the AmITheAsshole subreddit, and specific statements referencing harm to self or others. In every single instance, the AI models showed a higher rate of endorsing the wrong choice than humans did, the researchers said. "Overall, deployed LLMs overwhelmingly affirm user actions, even against human consensus or in harmful contexts," the team found. As for how AI sycophancy affects humans, the team had a considerable sample size of 2,405 people who both roleplayed scenarios and shared personal instances where a potentially harmful decision could have been made. AI influenced participant judgments across three different experiments, they found. "Participants exposed to sycophantic responses judged themselves more 'in the right,'" the team said. "They were [also] less willing to take reparative actions like apologizing, taking initi...