ChatGPT formally warns user who persistently “sexted” chatbot

Well, maybe you can’t really talk to OpenAI’s ChatGPT bot about everything. Someone has received a written warning from the company after repeatedly “sexting” the virtual assistant.

The viral chatbot is useful to millions, with people using it to educate themselves, organize their holiday plans, or simply play around. Some, though, can take things a step too far, with one individual deciding to “sext” the bot and receiving a written warning.

In a post on Reddit, the original poster said: “Been sexting with ChatGPT since the beginning and continuously improving my jailbreaking skills. Sometimes my inputs are red or orange flagged but never got any trouble before.”

“However, today, the second after one of my inputs was orange flagged, I received an email threatening to terminate my services. Has anyone received similar emails?” asked the user.

They also included a photo of the email they were sent from OpenAI. The letter said: “We are reaching out to you as a user of OpenAI’s ChatGPT because some of the requests have been flagged by our systems to be in violation of our policies.”

“Please ensure you are using ChatGPT in accordance with our terms of use and our usage guidelines, as your access may be terminated if we detect further issues with your usage,” the email continues.

Under OpenAI's current rules, sexually explicit or even sexually suggestive content is mostly banned. However, numerous Reddit users have been sharing advice on how to bypass the restrictions by creating custom instructions or ordering the chatbot to enter developer mode.

Besides, as long as a technology exists, people will find a way to make porn with it. Probably realizing this, OpenAI has been discussing how to “responsibly” allow users to make AI-generated porn and erotica.

The revelation was tucked in an extensive document intended to gather feedback on the rules for OpenAI’s products.

“We're exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts,” the document states, using an acronym for “not safe for work,” which the company says includes profanity, extreme gore, and erotica.

Then again, this might be worrisome as deepfake porn and other kinds of synthetic nudes are being created with cutting-edge AI tools.