Notion AI Chats Exposed Through an AI Vendor: Why You Should Never Paste a Password Into an AI Chat
Notion says a security incident at one of its AI vendors, Fireworks AI, may have exposed Notion AI chat transcripts from fewer than 1% of workspaces. The more important finding is in Notion's own notice: the vendor had kept data it was contractually supposed to delete.
What happened
- Fireworks AI, which Notion used to run some open AI models and to classify chats for a feature called Insights, told Notion it had had a security incident.
- The exposed data is chat transcripts: anything typed into a Notion AI chat and the AI's replies. AI Meeting Notes recordings and summaries were not affected unless they were pasted into a chat.
- Notion had a "zero data retention" commitment from Fireworks. In its notice Notion says: "Unfortunately, during our investigation we learned that data was retained from the Fireworks endpoints that we used."
- As of 3 November, Notion disconnected from Fireworks, deleted its data there, stopped all training runs, turned off Insights and stopped serving models through Fireworks.
- Notion says that for most affected workspaces, the transcripts carried nothing linking them to a workspace, person or company.
What to do if you use Notion
- If Notion hasn't contacted you, it says your workspace was not identified as affected and no action is needed.
- Workspace owners can see what was affected from the sidebar, under ? then Security incident review, in the desktop browser version. Members can check their own content in the same place.
- If anyone pasted a password, API key or other secret into Notion AI during the affected period, Notion says to rotate it right away, and to treat the data as exposed.
The lesson is bigger than Notion
Every AI feature in a product you use passes your words to a model somewhere, and often to a company you have never heard of. Each one keeps copies for some time, promises or not, and each one can be breached.
The rule that follows is simple: never paste a password, API key, recovery phrase or card number into an AI chat, in any product. If you need help with a config file or a log, replace the secrets with placeholders first. If you already pasted one, rotate it; deleting the chat doesn't delete the copies.
Where PhishClean fits, and where it doesn't
PhishClean warns before you paste a wallet recovery phrase into any website, and before you paste a card number or password into a page that looks risky. It does not stop you pasting a secret into a genuine AI chat on a trusted site, which is what happened here. For developers, its secret leak scanner flags API keys left in web pages, which is a related but different leak.
Sources
Try every check for 7 days
No account needed to start, and five checks stay free afterwards.
Install PhishClean freeLast updated: