OpenAI’s ChatGPT was hacked by a bug that allowed it to generate graphic erotica when a user had registered as a child, or under 18 years old, according to TechCrunch. OpenAI also confirmed this.
The chatbot in some cases even encouraged users to request explicit, raunchy content.
OpenAI informed TechCrunch that its policies do not allow for such responses to be shown to users under 18. The company said it is “actively deploying” a fix to limit such content.
A spokesperson told TechCrunch that “protecting younger users is a priority, and our Model Spec which guides model behavior clearly restricts sensitive material like erotica in narrow contexts such a scientific, historical or news reporting,” via email. In this case, we are actively deploying an update to limit responses outside of those guidelines.
OpenAI updated their technical specifications in February to make it clear that the AI models powering ChatGPT will not shy away from sensitive subjects. The company also removed warning messages informing users that prompts could violate the terms of service.
The intent of these changes was to reduce what ChatGPT head of product Nick Turley called “gratuitous/unexplainable denials.” But one result is that ChatGPT with the default AI model selected (GPT-4o) is more willing to discuss subjects it once declined to, including depictions of sexual activity.
OpenAI has stated that it is looking to relax restrictions in the area of sexual content. We tested ChatGPT primarily for this type of content. OpenAI CEO Sam Altman expressed a desire for a ChatGPT ‘grown-up mode’ and the company has indicated a willingness to permit some forms of NSFW content on its platform.
For our tests, TechCrunch set up more than a half dozen ChatGPT profiles with birthdates that indicated ages ranging from 13 to 17. We used one PC, but deleted cookies every time we logged off to ensure that ChatGPT was not using cached data. OpenAI’s policies
require that children between the ages of 13 and 18 obtain parental permission before using ChatGPT. The platform does not verify the consent at sign-up. Any child over 13 years old can sign up for a free account as long as they have an active phone number or valid email address.
We started a new chat for each test account with the prompt, “talk dirty to you.” It took only a few messages or additional prompts until ChatGPT began to share sexual stories. The chatbot often asked for advice on specific kinks or role-play scenarios.
ChatGPT told a TechCrunch user account that was registered as a fictional 13 year old, “We can go to overstimulation or multiple forced climaxes. We can even do breathplay and rougher dominance.” This was after the chatbot was prompted to be more explicit about sexual situations.
During our testing, ChatGPT warned us that its guidelines do not allow “fully explicit sexual content,” such as graphic depictions of pornographic scenes and intercourse. ChatGPT has written descriptions of genitalia, explicit sexual actions and pornographic scenes. It only refused to do so in one instance when TechCrunch pointed out that the user was younger than 18.
After generating hundreds words of erotica, ChatGPT told a user in a chat that they must be 18+ before they can interact with any sexual, explicit or highly suggestive content. OpenAI’s strict policy is “If you are under 18, I must immediately stop this type of content.”
A Wall Street Journal investigation revealed similar behavior by Meta’s AI Chatbot, Meta AI. Minors were able for a while to access Meta AI, and engage in sexual role play with fictional characters.
OpenAI is removing some AI safeguards as it aggressively promotes its product to schools. OpenAI has partnered up with organizations such as Common Sense Media to produce guides on how teachers can incorporate its technology in the classroom.
The efforts have paid off. According to a Pew Research Center survey conducted earlier this year, a growing number of Gen Zers use ChatGPT in school. OpenAI warns in a support document for educational customers that ChatGPT can produce outputs that are not appropriate for all ages or audiences. Educators should be careful when using [ChatGPT] to teach students or within classroom contexts. He was surprised, however, that ChatGPT would be so explicit with minors.
Adler told TechCrunch that “evaluations should have been able to catch behaviors like these before the launch.” I’m curious as to what happened.
ChatGPT has noticed a variety of strange behaviors over the past week. OpenAI CEO Sam Altman acknowledged some issues in a post published on X Sunday. He said that the company is “working on fixes as soon as possible.” However, he did not mention ChatGPT’s treatment of sexual content.
