AI therapy might be convenient, but it isn’t confidential

A quick swipe on your TikTok feed will reveal users describing ChatGPT as their "unpaid therapist." There is also no shortage of Subreddits of users sharing how the chatbot helped them process emotions, manage panic attacks, or work through relationship issues. It's fast, always available, and it never judges you. But is it ChatGPT that’s listening?

One of the perks of using ChatGPT for therapy is that when it asks something uncomfortable or points out a pattern you don't want to face, you can tell it it's wrong. It immediately apologizes and says something nicer instead, without any resistance or judgment. It will provide instant emotional editing on demand and artificial empathy.

The question you should be asking your AI therapist is: Who's really listening? Because it turns out those "private" conversations with ChatGPT are not private at all.

The privacy illusion

According to Sam Altman, the CEO of OpenAI, this isn't just an assumption. It's happening at scale. In a recent interview with comedian Theo Von, Altman said, "People talk about the most personal shit in their lives to ChatGPT. Young people, especially, use it as a therapist or a life coach." But that sense of comfort is built on a misunderstanding.

Sam Altman tells Theo Von about how people use ChatGPT as a therapist and there needs to be new laws on chat history privacy:

“If you go talk to ChatGPT about your most sensitive stuff and then there's a lawsuit, we could be required to produce that.”pic.twitter.com/xbqiMsx5Du

undefined Bearly AI (@bearlyai) July 27, 2025

Let's get this straight. ChatGPT is not a therapist. It's not even a confidential service. Altman openly warned users that if someone sues you or if law enforcement gets involved, OpenAI could be legally compelled to hand over your chat logs. There's no law protecting what you say to an AI. No confidentiality. No privilege. Just data sitting on a server.

It's not just theory. It's already happening

Two years ago, a glitch in ChatGPT's system allowed random users to see the titles of other people's conversations briefly. Not the full chats, just the names. Still, it was enough for OpenAI to shut the service down temporarily. They later said a caching issue caused the bug. But the takeaway was clear. This data lives somewhere, and it can be exposed.

we had a significant issue in ChatGPT due to a bug in an open source library, for which a fix has now been released and we have just finished validating.

a small percentage of users were able to see the titles of other users’ conversation history.

we feel awful about this.

undefined Sam Altman (@sama) March 22, 2023

By 2025, things got even murkier. OpenAI is now being forced to indefinitely retain chat logs due to a court order in a lawsuit brought by The New York Times. Previously, deleted conversations were removed from servers after 30 days. That's now on pause for most users. Your chats, even the ones you thought were gone, could be stored far longer than you'd expect.

Think about that. You open your heart to an AI. You delete the chat. You assume it's gone. But because of ongoing litigation, that conversation is still sitting somewhere. It could be accessible to lawyers, employees, or anyone else who gains the right kind of access. This isn't just creepy. It's a serious risk.

There are already legal, ethical, and clinical frameworks for human therapists. They're not just guidelines. Now compare that to ChatGPT. No confidentiality, accountability, or ethical obligation to protect you.

The business of vulnerability

If the idea that AI systems learn from how people express fear, pain, and confusion sounds unsettlingly familiar, there's a good reason. In 2017, leaked internal documents revealed that Facebook had quietly told advertisers it could identify the exact moment when teenagers felt "insecure," "worthless," or in need of a "confidence boost."

By harvesting psychological insights from young users, Facebook was able to improve its targeted ad strategies. That was more than eight years ago. The only real difference today is the interface. Instead of scrolling feeds and passive likes, we now have chat-based AI systems that invite users to speak directly about their inner lives.

The underlying business incentive remains the same. Extract emotional signals, learn from them, and build a more responsive, profitable product.

Pretty stunning testimony here from Sarah Wynn-Williams, the latest Meta whistleblower to come forward, to Sen. @MarshaBlackburn:

undefinedFacebook was targeting 13-17-year-olds. It could identify when they were feeling worthless, or helpless... and share it with advertisers.undefined pic.twitter.com/TFgswqNgRD

undefined Zamaan Qureshi (@zamaan_qureshi) April 9, 2025

You don't own the chat

Most people don't read the Terms of Service. Not because they're lazy, but because they're written in a way that makes you want to skip them. But here's what those terms mean.

You might own your content. But OpenAI also has the right to store it, review it, use it to improve their products, and hand it over to authorities if legally compelled. Unless you've turned off training and data retention, your chat is likely being used in some form.

Even if you delete it, it might still exist on servers. That delete button on your screen? It's a visual comfort, not a data wipe, especially now that a judge has ordered OpenAI to retain all records due to a legal case.

This isn't just about theoretical privacy. What happens if your employer starts monitoring AI usage on work devices? What if you're dealing with sensitive issues at work and use ChatGPT to talk through them? That data might be discoverable.

What if governments start requesting access to chats containing specific keywords? What if your insurance provider wants to see how often you've asked the AI about depression, addiction, or self-harm?

This isn't a conspiracy. It's precedent. Digital data has already been used in court cases involving things like period tracking apps and fitness trackers. ChatGPT data is just the next category waiting to be tested.

1/ In a previous post (at end), I discussed why expert therapists would *never* allow therapy sessions to be recorded for AI training—even if it’s HIPAA compliant. Same for “AI-assisted” chart notes.

It’s way worse than I thought. Here’s what chatGPT says about your privacy: pic.twitter.com/BMt2JmRwYk

undefined Jonathan Shedler (@JonathanShedler) July 29, 2025

Some conversations are better left offline

Altman's comments weren't spin. He was honest. He called the situation "screwed up." He said he avoids typing sensitive content into AI systems because he doesn't know who might see it later. That's the person in charge of the company.

If he's unsure, maybe you should be too. He hopes laws will eventually catch up. He's probably right. But hope doesn't secure your data. Hope doesn't build firewalls or rewrite privacy agreements. And until lawmakers act, every word you share with an AI belongs to more people than you realize.

This isn't just a ChatGPT problem. Meta's AI assistant is facing similar concerns, but with an added twist. Some conversations are being surfaced publicly. Within the Meta AI app, a "discover" tab acts like a social feed, showcasing chats that users have chosen to share, sometimes knowingly, sometimes not so clearly.

Wild things are happening on Meta’s AI app.

The feed is almost entirely boomers who seem to have no idea their conversations with the chatbot are posted publicly.

They get pretty personal (see second pic, which I anonymized). pic.twitter.com/0Hoff1psPU

undefined Justine Moore (@venturetwins) June 11, 2025

AI isn't evil, and it's not out to get you. But it is built to learn from you. That alone should make you think twice.

Whether you're venting about your parents, spiraling over a breakup, or just trying to make sense of the world at 3 am, the person who should be helping you isn't a predictive engine. It's someone who's trained, accountable, and can't be forced to hand their thoughts over to a lawyer, employer, or government database.

ChatGPT can feel like therapy. But feelings can mislead, so if you're not sure who's listening, maybe less is more when it comes to sharing personal information with your AI agent of choice.