Attackers who breached the AI inference provider Fireworks AI took customer billing records, datasets and partial model weights, the company said in a bulletin updated on November 9. One affected customer, Notion, has confirmed that Notion AI chat transcripts from fewer than 1 percent of its workspaces were among the stolen data. The case matters because Notion says the transcripts were kept on Fireworks systems even though the vendor had contractually promised not to retain them.
What Fireworks says happened
Fireworks runs open models for other companies. It first disclosed an incident on October 9. At that time it said someone had used internal Fireworks credentials to read environment variables, including API keys stored as secrets, in evaluation jobs for a limited number of accounts.
The November 9 update widens that picture. Fireworks now says account and billing information was affected, and that for a limited set of accounts the intruder also reached secrets, datasets and incomplete model weights. Billing data includes addresses, invoices and payment history. Fireworks says it does not store payment card numbers.
There was also a second wave. A third-party cloud provider that hosted Fireworks research machines had its own security incident on October 23. A day later the intruder used that access to reach Fireworks development machines and run an unauthorized workload, which exposed more customer data. Fireworks says it detected this the same day and has seen no further successful access since October 25.
The company brought in Palo Alto Networks Unit 42 for forensics and notified law enforcement. It says it has not detected any impact on its inference service, Training SDK or Nexus products, which do not retain data by default. Self-serve dataset uploads and downloads are disabled while the investigation continues.
What Notion customers lost
Notion used Fireworks in two ways, according to its help-center notice: as a general provider of open-weight models inside its product, and to run classification jobs for a feature called Insights, which shows administrators how their members use Notion AI.
The exposed data consists of chat transcripts. Notion says a transcript includes any input to or output from the model that was written to the chat. Meeting notes, audio and summaries were not affected unless someone pasted them into a chat. For most workspaces the transcripts carried no metadata tying them to a user or company, Notion says, but the text itself may contain sensitive material.
On November 3 Notion disconnected from Fireworks, deleted its data there and stopped all training runs. It has disabled Insights. Affected workspace owners can download the exact exposed text from a review page inside Notion, with a file that flags possible credentials. Notion advises treating the data as exposed and rotating any password or API key that was pasted into Notion AI.
The motive: reselling model access
Fireworks says the evidence points to a financially motivated criminal group that sells tokens obtained through unauthorized access to language models, a practice called LLMjacking. Google Threat Intelligence Group and Mandiant described this market in a report on November 2. We explain it in detail in our deep dive on LLMjacking and the resale market for stolen AI access.
That motive fits the first wave, which targeted stored API keys. It does not make the stolen transcripts harmless. Data taken as a by-product can still be sold or leaked.
What this means for companies that buy AI through vendors
The lesson for buyers is about how far a zero data retention promise reaches. Notion explains that its own retention commitment covers the API requests it sends to model vendors to answer a prompt. It does not cover features that need stored conversations, such as chat history and Insights. For Insights, Notion sent stored requests and responses to Fireworks for analysis. Fireworks had agreed contractually to zero retention, yet Notion’s investigation found data was retained on the endpoints it used.
Three practical points follow from the two companies’ statements:
- A retention clause is a contract term, not a technical control. Notion now says it will audit critical vendors to verify the practice.
- Analytics and training features often create a second copy of conversations with a different vendor. Ask which features do this and whether they can be switched off.
- Secrets pasted into a chat become part of the transcript. Our guide to least-privilege API keys for AI coding agents covers how to limit the damage when one leaks.
The incident also resembles the OpenCode console breach that exposed API keys in October: in both cases a developer-facing AI service held credentials that opened other systems.
What to watch next
- Further Fireworks bulletins, since the company says its investigation is ongoing.
- Whether other Fireworks customers disclose exposed data.
- The results of Notion’s vendor audit and whether Insights returns with a different provider.
- Regulatory notifications, if the transcripts contain personal data covered by breach laws.
Key takeaways
- Fireworks AI says intruders accessed billing data and, for some accounts, secrets, datasets and partial model weights.
- Notion confirms that chat transcripts from fewer than 1 percent of its workspaces were taken.
- The data sat with Fireworks because of Notion’s Insights analytics feature, not ordinary prompt handling.
- Notion says Fireworks retained data despite a contractual zero-retention commitment.
- Fireworks links the attack to a group that resells stolen access to language models.
Frequently asked questions
Was my Notion workspace affected by the Fireworks breach?
Notion says it has contacted affected customers directly. If neither you nor your workspace owner has heard from Notion, your workspace has not been identified as affected.
What should affected Notion users do?
Review the exposed text in the Security incident review page in the Notion sidebar. Rotate any passwords, API keys or other credentials that were pasted into Notion AI.
What is zero data retention?
It is a commitment by an AI provider not to store the prompts and responses it processes. In this case Notion says its vendor had made that commitment but data was retained anyway.
