- Anthropic showed Claude Sonnet 4 jump from 16.9% to 92.8% on the VirBench viral-sequence retrieval benchmark without changing the model, only the retrieval tool underneath it.
- The biggest source of failure wasn’t scientific reasoning but unreliable retrieval: incomplete result sets, inconsistent filtering/metadata interpretation, and high run-to-run variability.
- A deterministic retrieval layer (gget virus) standardized how queries are executed across NCBI APIs, pulled full paginated results, applied filters correctly, and produced auditable logs.
- Once retrieval became reliable, performance across cheap and expensive models converged above 90%, meaning infrastructure quality can matter more than model choice for tool-heavy tasks.
Claude Sonnet 4 scored 16.9 percent on a viral-sequence retrieval benchmark, then 92.8 percent on the same questions, and the only thing that changed was a deterministic tool sitting underneath it.
The failures were never about reasoning. They were about retrieval that came back incomplete, filtered inconsistently, and different every time you ran it.
Once a boring, deterministic lookup layer was in place, the gap between the cheapest model and the most expensive one mostly closed. That is the part worth copying.
The number that should stop you
At its AI for Science event on June 30, Anthropic put out research called "Paving the way for agents in biology." Buried in it is a benchmark named VirBench: 120 viral-sequence retrieval queries across 40 pathogens. The task is the kind of thing a working scientist does constantly. Go find all the SARS-CoV-2 sequences matching these criteria, count them, pull the right accession numbers.
Run the agents cold, with only their own tool use and a web interface, and the results are all over the map. Claude Sonnet 4 landed at 16.9 percent mean accuracy. GPT-5.5 landed at 91.3 percent. Opus, the open-source Biomni stack, and the rest scattered in between. If you were choosing a model on those numbers, you would spend more money to buy your way up the curve, and you would still be wrong a lot.
Then Anthropic gave every agent the same tool, a deterministic retrieval layer called gget virus. Accuracy rose above 90 percent for all of them. GPT-5.5 peaked near 99.7 percent. Claude Sonnet 4, the one that started at 16.9, hit 92.8. Run-to-run variability, the thing that makes a benchmark score a lie, was largely eliminated.
Read that again. The weakest performer in the room, once it stopped doing the lookup itself, was within a few points of the strongest. The model did not get smarter. The infrastructure got reliable.
What was actually breaking
Look at why the agents failed, because that is the whole lesson. They were not failing because they could not reason about virology. They were failing because the plumbing under them was bad.
Anthropic is specific about it. Agents under-counted because they never retrieved the full result set, which hurt most on high-volume pathogens like SARS-CoV-2 and Influenza A where the answer is thousands of records deep. Filtering was inconsistent because the web interface exposed behavior that no single API endpoint reproduced. Metadata fields meant different things in different contexts and the models guessed wrong. And the same prompt, run three times, returned 106 sequences, then 15, then 5.
None of that is a thinking problem. All of it is a "the tool you handed the model is unreliable, so the model's answer is unreliable" problem. When you ask a language model to be the database, the query planner, and the batching logic all at once, it will improvise, and improvisation is not reproducible.
What gget virus actually does
The fix is unglamorous, which is exactly why it works. gget virus is a deterministic tool built with the NCBI team. It coordinates across NCBI's REST, Datasets, and E-utilities APIs, figures out which filters can run at the API level versus which have to run locally, pulls large result sets in full instead of stopping at the first page, fetches supplementary records when a filter needs them, and returns standardized output with a retrieval log you can check.
Notice there is no cleverness in the model layer. The intelligence moved down into a tool that does the same thing the same way every time. The model's job shrank to deciding what to ask for and reading back a result it can trust. That is a much smaller, much more reliable job.
The lesson has nothing to do with biology
You are probably not retrieving viral genomes. It does not matter. The shape of this problem is everywhere in production AI.
You have a support agent that looks up order status. A coding agent that reads your dependency graph. A research assistant that pulls figures from a data warehouse. A pipeline that classifies documents against a taxonomy. In every one of those, there is a moment where the model has to go get ground truth, and you have a choice about how that retrieval happens.
The tempting path is to let the model handle it. Give it API access and some instructions and trust that a good enough model will figure out the pagination, the filtering, the edge cases. The VirBench result is a clean argument against that path. A frontier model will paper over bad retrieval well enough to demo. It will not paper over it well enough to trust, and it will not do it the same way twice.
The other path is to build the boring tool. Wrap the retrieval in something deterministic that always pulls the full set, always filters the same way, and logs what it did so you can verify. Then the model sits on top of a foundation that does not move under it.
Anthropic's own summary is the quiet headline: adding a deterministic retrieval layer made model choice much less important. Think about what that means for a budget. The reliability you have been trying to buy with a bigger model was sitting in the infrastructure the whole time, and it costs a fraction as much.
Where this lands for builders
If you run agents in production, this maps directly onto how you should spend the next sprint.
Find the places where your agent does retrieval or lookup by improvising against a raw API, and count how often it gives you a different answer to the same question. That variance is your 16.9 percent. Then move that logic into a deterministic tool, the way MCP servers are supposed to work, and measure again. You are not prompting your way to reliability. You are engineering it below the model.
This is also the cheaper answer to the model-upgrade treadmill. Every few weeks a new model ships and the reflex is to swap it in and hope the scores climb. Sometimes they do. But if your bottleneck is a flaky lookup, a better model just fails more confidently. Fix the tool and a mid-tier model gets you to 92 percent, which is the number that actually matters when someone downstream is trusting the output.
The set-the-bar, hold-the-bar split holds here too. You set the bar by specifying exactly what the tool must return. The tool holds the bar by returning it the same way every time. The model, freed from doing the part it was bad at, gets to do the part it is good at.
A weak model with a strong tool beat a strong model with a weak one. Build the tool.
If you’re using AI agents for any workflow that depends on looking things up—tickets, orders, codebases, internal docs—treat retrieval as production infrastructure, not something the model should improvise. Replace flaky web scraping and ad-hoc tool calls with deterministic, well-scoped APIs that handle pagination, filtering, and normalization the same way every time, and add logs you can audit. Before paying for a bigger model, measure how much of your “model failure” is actually inconsistent data access, then fix the plumbing and re-benchmark.