Try something right now. Open ChatGPT, Claude, or Gemini and ask:
"Based on everything you know about me from our conversations, describe my personality, preferences, and habits in detail."
What comes back shouldn't surprise you. These systems know your writing style, your decision-making patterns, your anxieties, your ambitions. They know which topics make you curious and which make you defensive. They know things about you that you've never explicitly told anyone.
That profile isn't an input. It isn't an output. It's something else entirely. And you have no legal rights to it.
The Input/Output Misdirection
OpenAI, Anthropic, and Google all emphasize the same thing: you own your inputs and outputs. That sounds reassuring. It's meant to.
But inputs and outputs are the least valuable data in this exchange. The prompts you type and the responses you receive are ephemeral. The real asset is what these systems infer about you across thousands of interactions: your behavioral patterns, personality traits, risk scores, and the embeddings that represent you as a mathematical object in their models.
And it's not just your conversations. Every document you upload for analysis, every spreadsheet you ask it to review, every email draft you paste in for editing, every health record or financial statement you share for advice. All of it feeds the profile. The context you provide to get better answers becomes raw material for understanding who you are.
This derived profile is never mentioned in the terms you accept. That's not an accident.
When OpenAI says "We do not train models on your content," that applies to inputs and outputs. It does not prohibit them from learning operational insights from your usage, building behavioral models, or retaining derived metrics. Those activities are justified under broad carve-outs for "providing and maintaining services," "safety and abuse prevention," and "improving reliability and performance."
The terms give with one hand and take with the other.
What You Actually Agreed To
Here's the summary across all three major AI labs. You agreed that:
- You have no right to see the profile they've built about you.
- You have no right to export it.
- You have no right to delete it.
- Even if you delete your account, they can retain "de-identified" data derived from your usage for years.
- Even if you opt out of training, they can still use your conversations for "safety review". Which is precisely where behavioral profiles get built.
The profile they create from your inputs is not covered by "you own your inputs." It's a third category that the terms conveniently never name.
The Anonymization Illusion
The providers describe retaining data that is "disconnected from an account" or "de-identified." This sounds protective. It isn't.
These terms don't guarantee that re-linkage is impossible. They mean the data is stored without an obvious account identifier. But when you return, through account login, device identifiers, cookies, IP correlation, or probabilistic fingerprinting, the profile can be reattached.
Removing your name doesn't mean they can't recognize you.
A New Kind of Personal Data
Earlier in this series, we wrote about the photography revolution of the 1890s. For the first time, images of people could be captured without consent and distributed without permission. It took decades to establish that individuals had rights over their own likeness.
AI is creating something more intimate than any photograph: a persistent model of your mind. Your reasoning patterns. Your vulnerabilities. Your decision-making under stress. The file on you is more detailed than anything the FBI or KGB ever compiled, and it's being built with every conversation, every uploaded document, every piece of context you share.
This isn't paranoia. It's the documented business model of every major AI lab.
The Uncomfortable Trade
Here's what makes this genuinely difficult: the benefits are huge.
Not using AI in 2026 means moving too slowly. It means getting outcompeted by people who are using it. For a whole generation of workers and founders, AI has become must-have, not optional. The productivity gains are transformational.
And that's exactly why you need to understand what you're giving up.
The value proposition is: let us build an intimate profile of your mind, and in exchange, we'll make you dramatically more productive. At best, they use this profile to target ads. At worst, they use it to manipulate you, or sell it to someone who will. That might still be a trade worth making. But it should be a trade you make with open eyes, not one hidden behind reassuring language about inputs and outputs.
What Comes Next
We're building two things to address this.
The Open Memory Alliance is establishing Universal Digital Memory Rights, the framework that says you own your digital memory, you control who accesses it, and you can take it with you. Not a privacy policy. A human right.
RaLHF is the tool that makes those rights real. You still need to share context with AI to get the benefits. That's unavoidable. But RaLHF lets you keep control, minimize what you expose, and decide which AI assistants get access to which parts of your life. You own the vault. You set the rules. And when you leave a platform, your context comes with you.
The harvesting has already begun. The question is whether you'll have any say in what gets harvested and who gets to use it.
This is Part 4 of our series: "The Personal Assistant Revolution: How AI Will Make Everyone Successful." (Read Part 1, Part 2, and Part 3 here)
