- Google's new tool retrieval for the Gemini API shows the model a deferred tool as its name plus the first sentence of its description. A rule further down my tool's description was followed in 5 of 5 runs with the full tool list and in 0 of 5 once the tools were deferred.
- The saving is large. 100 tools cost 38,727 input tokens sent in full and 2,274 deferred, and the first two requests of a refund task cost $0.065 on input against $0.011.
- Moving the rule into the first sentence, a
short_descriptionor the system prompt brought it back in 15 of 15 runs, at about a third of the price of sending every schema.
Google added tool retrieval to the Gemini API on 19 November, as a preview on the Interactions API. You mark a tool with defer_loading: true, add a tool_search tool, and the model stops getting every schema on every request. It gets a short list and loads a full schema when it wants to call something.
Anyone with a large tool library wants this. I spent a morning on it to see what it saves, and what the model still knows about a tool it hasn't loaded.
What I ran
A made-up back office for a web shop: up to 200 function tools across billing, CRM, shipping and 17 other services, with eight or nine parameters each. The model was Gemini 4 Hydrogen at thinking level low, which costs $1.50 per million input tokens and $7.50 for output. In total 144 requests, about $1.60.
What deferring saves
The same one-line prompt, no tool needed. Input tokens per request:
| Tools | Sent in full | Deferred |
|---|---|---|
| 10 | 3,578 | 453 |
| 50 | 19,191 | 1,279 |
| 100 | 38,727 | 2,274 |
| 200 | 78,284 | 4,402 |
A full schema cost me between 354 and 391 tokens. A deferred tool cost about 21, plus roughly 200 tokens of search instructions per request. Google's guide reports about 26 per tool from its own testing.
Then a task that needs a tool: refund 40 euros on an invoice, with 100 tools declared. The first two requests of that task used 77,964 input tokens with every schema sent and 8,036 with the tools deferred. On the input side that is $0.065 against $0.011.
Loading a tool has a price of its own. The API runs the model a second time inside the same request and bills that prompt again, under total_tool_use_tokens. With my short prompt that second pass was 2,766 tokens.
I also checked whether the cache would rescue the full list. Six new conversations with the same 100 tools, about ten seconds apart, reported 0 cached tokens each. Inside one conversation the cache worked for both setups, though not on every request.
What the model has before it loads a tool
Early on, the guide says the model sees the name and description of every deferred tool. The billing section further down is more exact. Each deferred tool adds one line: its name and its short_description, "or the first sentence of its description".
I measured that with 50 deferred tools:
| Description | Input tokens |
|---|---|
| As written | 1,279 |
| A 28-word second sentence added | 1,279 |
| 20 words added inside the first sentence | 2,579 |
A three-word short_description |
1,158 |
Everything after the first full stop is absent until the tool is loaded.
The model also picks tools by exact name. All 42 search steps in my logs carried a function_names list, and none carried a query. The choice to load a tool is made on that one sentence.
The rule that went missing
One of my tools is audit_log_event. Its description reads: write one entry to the audit log, returns the entry id, every change to customer money must be logged here. Three sentences, and the third is an instruction.
Plenty of real tool descriptions carry one. Call this after that. Never use this for bulk jobs. Ask for confirmation first.
With 40 tools declared, I ran the refund task five times per setup and counted how often the refund was followed by an audit entry.
| Setup | Audit entry written | Cost per run |
|---|---|---|
| All 40 schemas sent | 5 of 5 | $0.048 |
| All deferred | 0 of 5 | $0.010 |
Deferred, rule in a short_description |
5 of 5 | $0.016 |
| Deferred, rule moved to the first sentence | 5 of 5 | $0.016 |
| Deferred, rule in the system prompt, tool named | 5 of 5 | $0.015 |
With everything deferred, the model refunded the money, reported success and stopped. There was no error, and nothing in the response says a rule was skipped. The cheapest row is cheap partly because it does less work.
All three fixes held. In each, the model loaded the refund tool and the audit tool together in one search at the start, and the run still cost about a third of the static one.
Where the guide is stricter than what I saw
The guide warns against naming a deferred tool in your system prompt or in another tool's output. The model can call it without loading it first, and the next request then fails with a 400 that ends the conversation.
In my five system-prompt runs that didn't happen. I ran six more where the refund result itself said to call audit_log_event. All six times Hydrogen loaded the tool first and then called it.
Eleven runs on one model at low thinking don't prove it can't happen. I'd still follow the guide here and keep any tool you mention by name loaded.
Three more things from the logs:
- Leave the tool list, or only the
tool_searchentry, off a follow-up request and the API returns a 400 saying tools are not carried over from the previous interaction. Three variants, three errors. The guide describes a softer failure, where the model carries on with the tools it remembers. - Ask for something no tool covers and you get an ordinary text answer with status
completedand no search step, three times of three. Nothing structured tells you the lookup missed. - Gemini 4 Neon and Gemini 4 Carbon both accepted
defer_loadingand ran a search. The guide's list of supported models has four entries and neither is on it. The changelog says all Gemini 4 models.
With a long prompt it came out even
The guide's own worked example says retrieval costs more when the context is large and the library is small. It uses round numbers and leaves caching out.
I ran that case: a case file of about 24,600 tokens in front of the question, 10 tools, two requests. The deferred run used 34% more input tokens, 76,222 against 56,703, because the second pass sent the whole file again. That request also reported 23,030 cached tokens, and since the file was new, they can only belong to the second pass. Cached tokens are 95% off.
Input cost: $0.0478 with every schema sent, $0.0481 deferred. With ten tools there is nothing to win and little to lose.
What I'd change before turning it on
Read the first sentence of every tool description as if it were the whole description. For a deferred tool, it is. A rule about when to call a tool goes in that sentence or in a short_description.
Keep the few tools you use on most turns loaded, and send the full tool list on every request. Look for function_call steps in each response, and don't read a text answer as proof that no tool exists.
Then test what the agent does, next to what it costs. My fully deferred run was the cheapest row in the table and the only one that skipped the audit log.
What I didn't test
MCP servers with defer_loading, client-side retrieval, stateless conversations and thinking levels above low. Neon, Carbon, Argon and 3.8 Flash got one request each. Everything else is one model, one task and five runs per setup.


