- Model distillation can replicate a frontier model’s valuable behavior by collecting millions of high-quality Q&A outputs via normal-looking API usage, without stealing weights or code.
- Anthropic says operators linked to Alibaba’s Qwen lab ran ~28.8M Claude conversations across ~25,000 fraudulent accounts, aiming to copy Claude rather than use it.
- The most targeted capabilities were agentic reasoning and software engineering—commercially valuable behaviors that are harder to build and evaluate than general knowledge.
- For API-backed AI products, the defensible moat is increasingly the ability to detect and respond to output harvesting, not the underlying model alone.
- Rate limits and per-account thresholds are insufficient against distributed harvesting; effective defense requires cross-account, aggregate behavior detection.
Distillation lets a competitor copy the expensive part of a frontier model without touching the weights, just by asking it millions of good questions and keeping the answers.
The capability worth stealing in 2026 is not raw knowledge, it is agentic and coding behavior, and that is exactly what got targeted.
If you run any API-backed AI product, your moat is no longer the model, it is your ability to notice when someone is harvesting it.
On June 24, Anthropic went public with something it had already put in a letter to the US Senate Banking Committee two weeks earlier. Between April 22 and June 5, operators it links to Alibaba's Qwen lab ran roughly 28.8 million conversations with Claude across about 25,000 fraudulent accounts. The goal was not to use the product. It was to copy it.
The technique is called distillation, and it is worth understanding precisely because it does not look like a breach.
What distillation actually does
You take a strong model and you ask it a very large number of carefully chosen questions. You keep every answer. Then you train a smaller, cheaper model on those question and answer pairs until the cheap model behaves like the expensive one on the cases you care about.
Nobody steals source code. Nobody exfiltrates weights. No credential leaks. From the API's point of view, 25,000 accounts each had a series of completely normal conversations. The product did exactly what it was built to do, 28.8 million times, and the output was the prize.
This is the part that should make any builder uncomfortable. The thing you sell, the high-quality response, is also the most useful training data anyone could want for building a competitor to you. Every good answer is a labeled example. A model that is genuinely useful is, by definition, generating a clean dataset for whoever is collecting.
The target tells you where the value moved
The detail that matters most here is not the headline number. It is what the campaign went after.
Anthropic says the queries concentrated on software engineering and agentic reasoning, the two capabilities it calls the most commercially valuable and the hardest to build. Not trivia. Not summarization. Not writing copy. The part of the model that plans, calls tools, holds a multi-step task together, and writes working code.
That is a quiet confirmation of where the moat sits now. A year ago you might have said the moat was the size of the training run or the breadth of knowledge. Knowledge commoditized fast. What did not commoditize is the agentic layer, the behavior that turns a chatbot into something that can actually do a job. It is expensive to build, it is hard to evaluate, and apparently it is valuable enough that someone spun up 25,000 accounts to copy it.
If you are deciding what to invest in for your own product, that is the signal. The differentiated thing is the behavior under load, not the facts in the weights.
Your outputs are a dataset whether you like it or not
Most teams reading this are not frontier labs. You are building a product on top of an API, or you are running a model that serves answers to users. The Anthropic story still applies to you, just shifted down a layer.
If your product produces good output, someone can harvest that output and train on it. A support agent that resolves tickets well is generating a transcript of resolved tickets. A coding assistant is generating working diffs. A pricing or routing engine is generating decisions paired with inputs. Anyone who can hit your product at scale can collect the pairs.
Rate limits do not stop this. The Alibaba campaign was not one account hammering an endpoint. It was 25,000 accounts each behaving within normal bounds. Spread thin enough, harvesting looks identical to ordinary heavy usage. Any single account passes every threshold you would set.
So the defense is not throttling, it is detection across accounts. Anthropic caught this by looking at behavior in aggregate, the shape of the queries, the coordination between accounts that individually looked fine. That is a different muscle than per-user rate limiting, and most products do not have it.
What you can actually do
You will not out-engineer a determined distiller with a single trick. But there are concrete moves.
Watch behavior across accounts, not just within them. The signal of harvesting is coordination, many accounts probing the same capability in the same systematic way. That pattern is invisible per user and obvious in aggregate. If you only monitor individual rate limits, you are blind to the exact thing that happened to Claude.
Treat your highest-value capability as the likeliest target for copying. The distillers went straight for agentic and coding behavior. Whatever your equivalent is, your most expensive capability that competitors most want to replicate, assume it is the target and instrument it most heavily.
Know what your output exposes. Every response you return is a training example. That is not a reason to degrade your product, it is a reason to be deliberate about who gets bulk programmatic access and on what terms. Terms of service are a legal backstop, not a technical control, but the legal backstop is part of why Anthropic could write to the Senate instead of just absorbing the loss.
Do not assume your model is the moat. If a competitor can approximate your behavior with a cheaper model trained on your own outputs, the weights were never the defensible thing. The defensible thing is the product around the model, the data you have that they do not, the detection that flags abuse, and the speed at which you ship the next capability before the last one gets copied.
The shape of the next few years
Distillation is not new and it is not going away. Anthropic noted three other Chinese labs caught doing smaller versions of this in February. The Alibaba campaign was larger than all of those combined. The trend line points up, because the economics are too good. Why fund a frontier training run when you can buy 28.8 million answers from one and train on those.
The lesson for builders is not paranoia. It is a reframe. The model is increasingly a commodity input that leaks its own capability through normal use. The durable advantages are the boring operational ones, knowing your traffic, owning proprietary data, detecting abuse in aggregate, and shipping faster than your work can be copied. The companies that treat their model as the moat are going to be surprised. The ones that treat it as a leaky asset to be protected and rebuilt continuously will not.
Your best answers are training data. Build like that is true, because for at least one lab, on the order of 28.8 million times, it was.
Assume your best outputs are automatically a training dataset for someone else, and treat large-scale querying as a competitive risk, not just “usage.” Invest in monitoring that can spot coordinated patterns across many accounts—shared prompts, unusually systematic task coverage, repeated evaluation-style queries, and synchronized activity—rather than relying on simple rate limits. If your product’s differentiation is agentic workflows or coding performance, prioritize protecting and instrumenting those paths, because that’s where attackers are most likely to focus.