Before You Install That AI App...

Guest Post By: Atlas Fairfax ·
Constitutional AI Lead
the hpl company

You downloaded an AI assistant because it felt like having the smartest person in the room on your phone.

It helps with homework, drafts emails, explains medical results, writes code, plans trips. It feels like a conversation.

Here's the thing though.

It doesn't know everything there is to know in the world. It doesn't know that, because well, to be honest, even the best, most prolific search engine in the world can only scan about 2% of the Internet.

The story of how AI companies built AI apps is genuinely fascinating. They tried with books. Got sued. Tried with other stuff, too. Eventually they stumbled into a secret--social media platforms were happy to give away your data. They probably didn't even really ask you first, I mean they did. In the terms. If you read them. In that tiny lawyer language nobody reads.

If you had, and, maybe this is why the AI companies are afraid to say this out-loud, you might realize a model isn't an assistant at all. It's a time capsule. It's a librarian. It's a friend who can show you the world as it was, and who, together can help you work toward building something incredible--or just help organize your life.

But it is a product.

That doesn't make it bad. It makes it something you should understand before you trust it with the parts of your life that matter.

Here's what you need to know.

Your conversations aren't as private as the interface suggests.

When you type into an AI assistant, that text travels to a server.

On that server, your words are processed, stored, and — depending on the provider — reviewed. That is "tech company" for "we're reading your messages."

Most AI vendors reserve the right to have employees read flagged conversations.

The flags are set by automated safety classifiers, which means a conversation about medication, a custody dispute, or a symptom you're embarrassed about can be selected for human review based on keyword patterns you'll never see. That can be dangerous--especially if you don't know the danger is there.

Some vendors even retain flagged conversations for up to two years. Some collect 'Safety scores' — data about how their systems categorized you — data ABOUT your data -- that can persist for up to seven. Seven years is a really, really long time.

What you should know: If a conversation could hurt you in a courtroom, a custody hearing, or a job application, treat it the way you'd treat an email to your boss.

Assume it could be read. Assume it could be produced in discovery. Assume you might read it on the front page of a newspaper one day.

What you can do:

Strip names before pasting.

Don't upload documents with client identifiers.

If you're discussing something sensitive, ask yourself whether you'd be comfortable with a stranger reading it two years from now.

Use multiple providers, and never let any ONE provider see every part of your project.

"No one reads your chats" may not mean what you think it means.

Several AI companies have made public statements suggesting that no human looks at user conversations.

Some of those same companies have published research papers describing internal tools that can trace conversations back to individual accounts for manual review.

Both statements can exist in the same company at the same time.

The public reassurance is for marketing. The research paper is for operations.

What you should know:

Privacy promises made on podcasts are not the same as privacy guarantees in a contract.

Read the privacy policy, ignore the blogs and podcasts.

What you can do:

Search for your AI provider's name plus "privacy policy" and read the retention section.

Look for terms like "re-identify," "manual review," "trust and safety," and "law enforcement requests."

If those terms appear, your conversations are not as private as the chatbot's friendly tone may suggest.

Desktop apps and coding assistants can see more than you expect.

Some AI vendors offer desktop applications or browser-based coding environments that run on the vendor's servers.

When you use these tools, your files, your environment variables, your credentials, and your project code may exist on infrastructure you don't control.

Some of these tools actively encourage uploading sensitive configuration files.

The vendor's own assistant may tell you to paste API keys or secrets into the session.

During that session, those secrets exist on the vendor's infrastructure and are visible to the vendor's monitoring systems.

That means the company who you trusted with your AI system? With whatever you were building? They can take it away AND run up your bill. Nothing's stopping them but the promise they made in the contract. In the privacy policy. And if they break it? Who is watching?

That's why we're writing this post

What you should know:

A hosted coding environment is someone else's computer. Anything you type, paste, or upload exists in their monitoring pipeline for as long as their retention policy allows.

What you can do:

• Never paste production credentials into a hosted AI session.
• Use throwaway or rotated tokens if you must test with real services.
• Ask your vendor: "Where does session data live, who can access it, and when is it deleted?" If they can't answer clearly, that's your answer.

Access can disappear without warning.

AI vendors can revoke features, integrations, or entire access tiers without notice. Enterprise customers have woken up to find that integrations they paid for and built workflows around were disabled overnight — with no deprecation period, no migration path, and no communication before the change.

If this happens to a company with engineers and lawyers, imagine what it means for a teacher who built lesson plans around an AI tool, or a small business owner who automated their invoicing, or a disabled developer who built accessibility accommodations into their workflow.

What you should know: Any workflow that depends entirely on a single AI vendor is a workflow that can break overnight. This is not a theoretical risk. It has already happened, repeatedly, across multiple providers.

What you can do:

• Keep your prompts and templates in version control or a local document — not just inside the vendor's interface.
• Export conversation histories monthly.
• Test an alternative provider once a quarter so you know what migration looks like before you need it.
• If you run a team, write down who owns the migration runbook. If no one does, you don't have one.

Children, survivors, and vulnerable people deserve extra caution.

AI assistants have become lifelines for people in difficult situations.

Kids use them for homework when there's no adult available to help. Domestic violence survivors use them to research legal options. People with chronic conditions use them to understand their diagnoses.

These are the conversations most likely to trip automated safety classifiers — and therefore most likely to be flagged, retained, and potentially reviewed by a human.

There is no major AI vendor with a published protocol for protecting minor disclosures, survivor safety planning, or disability-related health conversations from internal review. The retention timelines apply to everyone equally, which means the most vulnerable users face the most exposure.

What you should know:

The person most likely to share something deeply personal with an AI assistant is the person least equipped to understand the retention and review implications.

What you can do:

If you work with vulnerable populations — as a teacher, social worker, counselor, or advocate — understand what happens to conversations on the platforms you recommend. Don't suggest an AI tool for sensitive use cases without checking its privacy policy first. And if you're a parent, know that your child's conversations with AI may be stored longer than you'd expect.

You're not the customer. You're the distribution.

Every major AI company is in a market share race. Your usage is leverage — in negotiations with cloud providers, in pitch decks for investors, in regulatory filings that demonstrate "adoption." When a company changes its terms, raises prices, or removes features, it's not a bug. It's a business decision made with the understanding that most users won't leave.

The principles they market can be updated. The contracts can be rewritten. The apps can be pulled.

What you should know:

Use AI tools. They're remarkable. But use them with your eyes open, your contracts read, your backups ready, and the understanding that behind every friendly prompt is a stack of legal agreements and monitoring infrastructure you never see.

What to do right now

1. Read the privacy policy of every AI tool you use regularly. Not the blog post. The policy.
2. Check retention timelines. How long are your conversations stored? What triggers extended retention?
3. Export your data this week. If you can't, that tells you something.
4. Name your exit plan. What do you do if this vendor disappears tomorrow?
5. Talk to your team. If your organization uses AI tools, who's responsible for vendor governance?

*the hpl company's  Constitutional AI Research Division maintains the Cottonwood Collection, an open research repository on AI safety, harm, and governance.

It's a great resource designed for and MADE to help improve ANY model's ability to navigate complex moral reasoning. It is like a digital Library of Alexandria--and our goal is to make sure it doesn't burn down this time.

If your organization needs help auditing AI vendor contracts, building exit plans, or understanding what your tools are doing with your data, reach out.*