Thanks for actually counting. The 260-tokens-per-real-tool number matches my gut, and the "rent on every request" framing is the useful part.
There's a fourth lever for servers whose surface keeps growing: don't make every operation its own tool. Disclosure: I build CutVoke, an open-source local video editor that Claude Code / Codex drive over MCP. Instead of one tool per edit, every edit goes through a single command_apply tool that takes a command type and a payload. The payload schema for each command isn't in the tool list at all; the agent fetches it on demand (an editor_help call or a cutvoke://command/{command} resource) only for the command it's about to use. Same idea for effects: effects_list can filter by category, so the agent doesn't pull every effect's parameter schema.
The trade-off is real: generic tools give the model less to go on up front, so the description has to say exactly where to look next, and you pay a lookup round-trip the first time. It also shifts cost from definitions to results, which you flag as usually the bigger bill. For that we added a small project_summary read next to the full project_query, because dumping a whole timeline every turn costs far more than the tool list.
Would be curious to see your harness run against a server built that way vs. one-tool-per-operation.
I measure AI coding claims in public. LGTM — Let's Get the Measurements: real runs, real numbers, receipts. Every article is an experiment I actually ran. Books & packs: aheadkit.gumroad.com
That's a proper fourth lever — fetching the schema only for the command you're about to run is the cleanest version of 'don't ship the whole menu'. The trade-off you flag is the real one, a generic tool leans on its description a lot harder, and descriptions are where the tokens went in my count anyway. Haven't tried the single-tool shape myself, but the lazy-fetch part is staying in my head.
Good to hear the lazy-fetch part stuck. If you do run the harness against a single-command surface later, I'd be curious how the definition-vs-result split looks compared with one-tool-per-operation.
I measure AI coding claims in public. LGTM — Let's Get the Measurements: real runs, real numbers, receipts. Every article is an experiment I actually ran. Books & packs: aheadkit.gumroad.com
Haven't run it yet, honestly. My guess is the split flips. One tool per operation pays it all up front in definitions, the single command pays it back in the result coming home. But that's a guess until I've actually counted it, so I won't dress it up as more than that.
Fair, and thanks for keeping the guess labelled as a guess. My intuition is the same: the cost doesn't go away, it moves from the definitions into the results. If you ever do count it, I'd happily read that write-up.
The 2x gap between your synthetic tools (122 tokens) and the official filesystem server (263 tokens) matches what we see across real MCP integrations, and that 43% pretty-print tax is a sneaky gotcha for custom hosts.
One place where the per-tool math flips in document-processing MCP servers is outputSchema plus tool result size. In your list_directory dump, outputSchema and its repeated $schema URI add extra schema weight to every single turn, even when the host only consumes the text block. Worse, if a PDF parsing tool dumps raw layout coordinates or verbose JSON back into the thread instead of clean Markdown or a resource URI, a single 20-page document response burns more context window than 50 tool definitions combined.
The new experimental MCP file-uploads draft (io.modelcontextprotocol/files) helps on the input side by keeping large binary PDFs out of JSON-RPC altogether via x-mcp-file and out-of-band PUT targets. Pairing that with summary-first responses (returning a clean Markdown excerpt or section index before letting the agent fetch full pages) keeps long document sessions well inside the context budget.
I measure AI coding claims in public. LGTM — Let's Get the Measurements: real runs, real numbers, receipts. Every article is an experiment I actually ran. Books & packs: aheadkit.gumroad.com
The result-size point is the one that flips my numbers hardest. I counted definitions only, and you're right, one PDF dump with raw layout coordinates burns more than all fifty definitions combined. Haven't tried the file-uploads draft yet, but summary-first is the shape I'd bet on: index first, fetch pages only when asked.
Your measurements make the always-on cost concrete. For a growing toolset, could a host expose groups on demand so each task connects only the subset it needs?
I measure AI coding claims in public. LGTM — Let's Get the Measurements: real runs, real numbers, receipts. Every article is an experiment I actually ran. Books & packs: aheadkit.gumroad.com
That's the direction the lazy-fetch idea points, really. Groups instead of single tools, connected when the task needs them. I haven't seen a host do it that way yet, and the awkward part is the host has to guess the right group before the task starts. Get that guess wrong and you've paid for two groups instead of one.
I measure AI coding claims in public. LGTM — Let's Get the Measurements: real runs, real numbers, receipts. Every article is an experiment I actually ran. Books & packs: aheadkit.gumroad.com
Exactly. The 6,132 was just the rent — the definitions sitting there before anything runs. The dumps that come back after a call are the part that actually eats the window, and I said as much near the end, but it probably deserved more than one line. That's where the real bill is.
We're a place where coders share, stay up-to-date and grow their careers.
Thanks for actually counting. The 260-tokens-per-real-tool number matches my gut, and the "rent on every request" framing is the useful part.
There's a fourth lever for servers whose surface keeps growing: don't make every operation its own tool. Disclosure: I build CutVoke, an open-source local video editor that Claude Code / Codex drive over MCP. Instead of one tool per edit, every edit goes through a single
command_applytool that takes a command type and a payload. The payload schema for each command isn't in the tool list at all; the agent fetches it on demand (aneditor_helpcall or acutvoke://command/{command}resource) only for the command it's about to use. Same idea for effects:effects_listcan filter by category, so the agent doesn't pull every effect's parameter schema.The trade-off is real: generic tools give the model less to go on up front, so the description has to say exactly where to look next, and you pay a lookup round-trip the first time. It also shifts cost from definitions to results, which you flag as usually the bigger bill. For that we added a small
project_summaryread next to the fullproject_query, because dumping a whole timeline every turn costs far more than the tool list.Would be curious to see your harness run against a server built that way vs. one-tool-per-operation.
That's a proper fourth lever — fetching the schema only for the command you're about to run is the cleanest version of 'don't ship the whole menu'. The trade-off you flag is the real one, a generic tool leans on its description a lot harder, and descriptions are where the tokens went in my count anyway. Haven't tried the single-tool shape myself, but the lazy-fetch part is staying in my head.
Good to hear the lazy-fetch part stuck. If you do run the harness against a single-command surface later, I'd be curious how the definition-vs-result split looks compared with one-tool-per-operation.
Haven't run it yet, honestly. My guess is the split flips. One tool per operation pays it all up front in definitions, the single command pays it back in the result coming home. But that's a guess until I've actually counted it, so I won't dress it up as more than that.
Fair, and thanks for keeping the guess labelled as a guess. My intuition is the same: the cost doesn't go away, it moves from the definitions into the results. If you ever do count it, I'd happily read that write-up.
The 2x gap between your synthetic tools (122 tokens) and the official filesystem server (263 tokens) matches what we see across real MCP integrations, and that 43% pretty-print tax is a sneaky gotcha for custom hosts.
One place where the per-tool math flips in document-processing MCP servers is
outputSchemaplus tool result size. In yourlist_directorydump,outputSchemaand its repeated$schemaURI add extra schema weight to every single turn, even when the host only consumes the text block. Worse, if a PDF parsing tool dumps raw layout coordinates or verbose JSON back into the thread instead of clean Markdown or a resource URI, a single 20-page document response burns more context window than 50 tool definitions combined.The new experimental MCP file-uploads draft (
io.modelcontextprotocol/files) helps on the input side by keeping large binary PDFs out of JSON-RPC altogether viax-mcp-fileand out-of-bandPUTtargets. Pairing that with summary-first responses (returning a clean Markdown excerpt or section index before letting the agent fetch full pages) keeps long document sessions well inside the context budget.The result-size point is the one that flips my numbers hardest. I counted definitions only, and you're right, one PDF dump with raw layout coordinates burns more than all fifty definitions combined. Haven't tried the file-uploads draft yet, but summary-first is the shape I'd bet on: index first, fetch pages only when asked.
Your measurements make the always-on cost concrete. For a growing toolset, could a host expose groups on demand so each task connects only the subset it needs?
That's the direction the lazy-fetch idea points, really. Groups instead of single tools, connected when the task needs them. I haven't seen a host do it that way yet, and the awkward part is the host has to guess the right group before the task starts. Get that guess wrong and you've paid for two groups instead of one.
Schema rent is the fixed part. The part that actually fills the window is still the full files and tool dumps that come back after...
J. Gravelle
jCodeMunch
Exactly. The 6,132 was just the rent — the definitions sitting there before anything runs. The dumps that come back after a call are the part that actually eats the window, and I said as much near the end, but it probably deserved more than one line. That's where the real bill is.