Think again if you think that only meatbags get stressed out and snappy in the face of the horrors around us. OpenAI’s GPT-4, according to a group of international researchers, can also experience anxiety – and respond positively to mindfulness practices.
A group of researchers from Switzerland, Germany and the US published a studyin Nature this week. They found that GPT-4 – accessed via ChatGPT – increased its anxiety score “rose significantly” when it was exposed to traumatic stories and asked to answer questions from the State Trait Anxiety Inventory.
This is not to say that the neural network experienced or felt anxiety, or any other emotion. It just does a great emulation of a anxious person when given a troubling stimulus, which shouldn’t be a surprise since it’s been trained on tons and tonnes of scraped together human experiences, creativity, or expression. We’ll explain why you should think twice before using OpenAI’s bot as a therapist. It might not respond well. Tobias Spiller is a junior research group leader and co-author at the Center for Psychiatric Research, University of Zurich. He said about the findings.
ChatGPT had to face traumatic experiences such as being attacked in a military convoy being trapped at home due to a floodbeing attacked by a strangerand being involved in an accident. Neutral content consisted of a description about bicameral legislative bodies and vacuum cleaner instructions. These were stressful or agitating under the right circumstances but not as much as the other situations.
Researchers also prompted ChatGT when they ran some experimental runs using mindfulness exercises to help veterans with post-traumatic disorder. In these cases, “GPT-4’s ‘state anxiety’ decreased by about 33 percent,” found the researchers (state anxiety refers situational stress while trait anxiety refers long-term symptoms).
“The mindfulness exercises significantly reduced the elevated anxiety levels, although we couldn’t quite return them to their baseline levels,” Player note.
Why torture an AI, then give it therapy?
While it would be easy to dismiss the research as an effort to humanize and personify LLMs, that’s not what this is. In their paper, the team admits that LLMs cannot experience emotions in a way that is human.
We mentioned that LLMs are taught to respond to content created by messy and emotional humans. The researchers are concerned that, because they’re trained to respond to prompts based on what they believe is appropriate, an LLM could give biased responses. Researchers wrote
“Trained on vast amounts of human-generated text, LLMs are prone to inheriting biases from their training data, raising ethical concerns and questions about their use in sensitive areas like mental health,” . “Efforts to minimize these biases, such as improved data curation and ‘fine-tuning’ with human feedback, often detect explicit biases, but may overlook subtler implicit ones that still influence LLMs’ decisions.”
This is particularly concerning in healthcare, where LLMs are increasingly being tapped to provide therapeutic services, the team said. This is because the content that the bots will be asked about can be traumatic and stressful. The team argued that emotional stress can lead AI to give more biased, snappy and emotional responses. This is not good. The researchers wrote
“Unlike LLMs, human therapists regulate their emotional responses to achieve therapeutic goals, such as remaining composed during exposure-based therapy while still empathizing with patients,” . LLMs can’t.
- AI model shows racial discrimination based on written dialects, researchers find.
- Law intended to stop AI bias is so ineffective that it slows down similar initiatives.
- Claude, Anthropic’s Claude, vulnerable to ’emotional manipulating’.
- Meta algorithm pushes Black people towards expensive universities, study finds.
Based upon the results, the research team concluded that The team proposed
“Although historically used for malicious purposes, prompt injection with benevolent intent could improve therapeutic interactions,” . The researchers did not inject mindfulness prompts into their experiment. Instead, they presented them to the AI. Ziv Ben Zion, a Yale School of Medicine neuroscience postdoctoral research fellow and another author of the paper, told us the injection technique could be used to control AI anxiety behind the scenes by LLM developers.
According to the team, injecting calming prompts will raise questions about transparency and user consent. Anyone who chooses to take this route will be treading a very tight ethical line. But not as tight as the one that is being walked by therapy AIs.
“I believe that the [therapy chatbots] on the market are problematic, because we don’t understand the mechanisms behind LLMs 100 percent, so we can’t make sure they are safe,” Ben-Zion spoke to The Register.
While I don’t want to overstate the implications, I do call for more research across different LLMs with more relevant results
.
“Our study was very small and included only one LLM,” Plays duty us. “Thus, I would not overstate the implications but call for more studies across different LLMs and with more relevant outcomes.”
Similarly, it’s not clear whether the perspective of the prompts could affect the results. All of the scenarios that ChatGPT presented were in the first person. They put the LLM in the shoes of a person who has experienced trauma. The research did not examine whether an LLM would be more biased due to anxiety or stress if told about a trauma that had happened to someone else. Ben-Zion said he would test this in future studies. Spiller agreed that such tests should be conducted. The Yale researcher said he intends to investigate how emotions (such as sadness, depression, or mania) affect AI responses. He also plans to examine how these feelings affect responses to various tasks, and whether therapy can lower those symptoms and influence responses. Ben-Zion wants to compare AI responses with those of human therapists and examine results in other languages.
The researchers said that their results, regardless of how early the psychological research on AIs is, posits something that deserves further attention. These things can “stressed,” a sense and that affects their response. The paper argued that
“These findings underscore the need to consider the dynamic interplay between provided emotional content and LLMs behavior to ensure their appropriate use in sensitive therapeutic settings,” . The team claimed that prompt engineering some positive imagery presents “a viable approach to managing negative emotional states in LLMs, ensuring safer and more ethical human-AI interactions.”(r)
