On 10 September 2026, Anthropic published a threat intelligence report covering December 2025 to August 2026. It is serious security work, and the headline numbers travelled fast.
More than 151 million exchanges attributed to a single campaign between May and July 2026, peaking at close to three million a day, spread across more than 3,500 accounts. A separate case saw roughly 300,000 requests routed through around 5,000 accounts inside a ten day window. Anthropic calls this illicit distillation: harvesting the outputs of a stronger model in order to train a competing one.
Almost every write up stopped at the numbers. The more interesting question is the one hardly anyone asked.
How would you know?
If a provider does not store and examine what you send it, how does it work out that 3,500 apparently unrelated accounts are running one coordinated campaign? How does it know which model was targeted, or that the traffic was shaped to build reasoning training data rather than to do ordinary work?
Anthropic answers that itself. It built “classifiers and behavioral fingerprinting systems designed to identify distillation attack patterns in API traffic”. It detects “chain-of-thought elicitation used to construct reasoning training data”. It looks for “content that maps directly onto what is most valuable for training an AI model”. Attribution came from “IP address correlation, request metadata, infrastructure indicators, and in some cases corroboration from industry partners”.
Read those again. A classifier that recognises chain of thought elicitation has to read the prompt. A judgement about whether content is valuable as training data is a judgement about content, not about metadata.
The detail that settles it
Attribution of that 151 million exchange campaign to a single effort rested on one observation: the accounts shared a fixed prompt. Anthropic describes seeing “variations of that prompt arrive tens of thousands of times across hundreds of coordinated accounts, all targeting the same narrow capability”.
You cannot detect a shared prompt across thousands of accounts from metadata. You can only detect it by retaining the prompts and comparing what they say.
The retention terms tell the same story. Since 9 June 2026 a 30 day retention period became mandatory for what Anthropic calls Covered Models, explicitly so that it can catch attacks which only become visible across multiple requests. Trust and safety classification scores are kept for seven years. Human review happens through a controlled path when automated systems flag something.
This is not a scandal. It is architecture.
It would be lazy, and wrong, to call this sinister. The same capability that spotted the distillation campaigns is the capability that spotted a cell using coding agents to write guidance software for a guided rocket. Nobody sensible wants that missed. A hosted frontier API cannot meet its safety obligations without inspecting the traffic running through it.
That is exactly the point. The inspection is not a policy preference that a firmer privacy promise could reverse. It is load bearing. Remove it and the provider goes blind to abuse. Every centralised API vendor faces the same trade, whether or not they discuss it on the pricing page.
“We do not train on your data” is not the same sentence as “we do not read your data.”
Both can be true at once. Commercial and enterprise content is genuinely carved out of training. Machines still read it, because they have to. Most people hear the first sentence and assume the second, and that gap is where the risk sits.
This is not a regulated industries problem
If you picture the people asking these questions as banks and hospitals, you have the wrong picture. We answer them every week, and most come from ordinary software companies. Property management platforms. Project management tools. Education software. Tender processing. A group rolling out an internal assistant to 600 staff. Not one of them is supervised by a financial regulator.
They are not unusually cautious. They are in the middle of a chain.
One recent request asked for a DPA amendment because the sender is a processor and we would be their sub-processor. Another put it more plainly still: in order to use you with our clients, we need a signed agreement. Neither was satisfying a regulator. Both were answering a customer who had asked them first, and neither could answer until we answered them.
That is the mechanic most coverage misses. GDPR does not create a special duty for regulated sectors. Article 28 means a processor cannot bring in a sub-processor without authorisation and disclosure. Article 30 means keeping a record of your processing, including transfers. Articles 13 and 14 mean telling people where their data goes. Article 35 means assessing the risk where it is likely to be high. The EU AI Act adds its own documentation and transparency duties on top. None of it is sectoral. If you hold other people’s data and you have put AI in your product, the question arrives at your desk whether you went looking for it or not.
Know, then tell, then assess
There is an order to this, and it matters.
- Know. Trace where inference actually happens, who operates it, what is retained, and what reads it.
- Tell. Put that in your own sub-processor list and privacy notice, in terms a customer can act on.
- Assess. Decide whether it is acceptable for your use, and record why you decided that.
Most organisations never clear step one. Not because they do not care. The companies writing to us plainly care. It is that step one is genuinely hard to finish, because the answers that come back are written to sound like answers without quite being them.
What the usual answers do not cover
Three responses come up again and again. Each is true. None of them finishes the job.
“We have a DPA.” Good, you need one. A DPA is a contract about obligations. It is not a description of the data path, and it will not tell you which systems read the content or how long it is kept.
“We do not train on your data.” Usually true, and not the question. Training is one use of data. Inspection is a different one, and the threat report is a description of inspection working exactly as designed.
“It runs in an EU region.” This is the most misleading of the three precisely because it is real. EU regions and data boundaries exist, they are contractually backed, and they are not marketing fiction. They answer the question where is it stored. They do not answer who can read it, what is retained in order to detect abuse, or which country’s law binds the company that decides. Storage location and legal control are different things, and only one of them appears on the product page.
The only answer that survives the question
There is one architecture where these questions stop being awkward, and it is not a firmer promise. It is removing the third party from the path. Open weight models, running on hardware inside your own jurisdiction, retaining nothing, with no parent company abroad that can be compelled. Nothing to inspect, because nobody else is in the way.
A word on the models named in the report. Qwen, Kimi and DeepSeek are open weight models, and we serve them on our own GPUs in the EU. The report concerns those companies’ own hosted services and their training practices, not the published checkpoints. It is worth being precise about that, and worth noting the difference it makes. When you run the weights yourself there is no silent route from your prompt to somebody else’s API.
Anthropic deserves credit for publishing any of this. Most providers would not. Read it for what it tells you about the model of computing you are buying into, not only about the people it caught.
You do not want a provider who promises not to look. You want a provider who structurally cannot.
