DEV Community

Cover image for An MCP Tool Call Can Resolve and Still Fail

An MCP Tool Call Can Resolve and Still Fail

Raju Dandigam on November 16, 2026

An await finishes. The application returns a result. The trace contains a completed tool span. The test is green. The tool also returned isError: ...
Collapse
 
agentsearchhq profile image
AgentSearch •

The three-question table is the useful part. We hit a fourth case with web tools: the call returns, isError is false, and the content is empty, usually because the page is a JavaScript shell that a plain fetch can't read. Treating "returned, not an error, but empty" as its own state helped a lot, because the right move there is a different tool (a headless render), not a retry of the same call. Does AgentInspect let you assert on the shape of the content, or only on isError?

Collapse
 
mike-johnstone profile image
Mike Johnstone •

A genuine zero-result page would be a useful second fixture here. I'd distinguish an explicit 'no matches found' message from a page that never exposed its results. Then an empty list alone wouldn't trigger the headless fallback.

Collapse
 
raju_dandigam profile image
Raju Dandigam Google Developer Experts •

@agentsearchhq This fixture asserts isError, not that the returned content satisfies a web-search contract. I would treat content shape as an application-specific postcondition: an explicit “no matches” result is different from an empty JavaScript shell, so neither isError: false nor an empty list alone should trigger a blind retry. The render fallback belongs in the caller’s policy, with its reason recorded in the trace.

Collapse
 
agentsearchhq profile image
AgentSearch •

Agreed, that fixture only proves the call didn't error. The split we'd keep: the caller declares what a valid result looks like, an explicit "no matches" counts as a real empty answer, and only a JS-shell-looking page (short text, script-heavy, missing expected content) triggers a render fallback. Logging which rule fired keeps the retry auditable.

Collapse
 
kevinhall profile image
Kevin Hall •

same thing with data lookups, a clean 200 and an empty list because the filter was too tight or the source was behind. we compare the count to what the same query gave last time and fail the run when it falls off a cliff

Collapse
 
agentsearchhq profile image
AgentSearch •

That's a solid baseline check. A sudden drop to zero is far more often a too-tight filter or a lagging source than a real empty answer, so failing loudly beats passing an empty list downstream. Do you also flag sudden spikes, or just drops?

Collapse
 
aheadkit profile image
AheadKit •

Your three-question table is the clearest version of this I have seen, and the third row is the one I am stuck on in practice. Take your booking-tool example: the tool responds normally to say inventory is unavailable — no isError, the promise resolves, the trace span completes. Questions 1 and 2 both pass, but the intended outcome (a booking) did not occur.

In your fixture, where does that case land? Do you treat the returned content itself ('inventory unavailable') as the independently observed state for question 3, or do you read external state — the inventory or booking store — after the call to decide it? I am asking because the first option is cheap but lets the tool grade its own homework, and the second is honest but means every fixture needs a second data source. Which side did you land on for the companion examples?

Collapse
 
jsbry profile image
Joshua Bryan •

Not the author, but we hit the same fork from the server side. A middle option: have the server say which case it is, in a field a program reads rather than in prose. In the MCP Write-Safety draft we wrote (MWS 0.9), a booking refused for lack of inventory comes back as a tool error with outcome: "not_applied". When the server can't tell whether the write landed, it returns outcome: "unknown" plus read_with: the read tool and arguments that settle it. Then the second data source is one read the server points you to, not one per fixture, and you still verify rather than trust the prose.

For scale: reading 100 popular packaged MCP servers from source, 44 return a machine-readable success or failure and 7 a machine-readable unknown, so today the honest option is usually the only one. https://isokron.ai/research/mcp-write-results-100-servers?utm_source=devto&utm_medium=social&utm_campaign=mws-launch&utm_content=comment

Collapse
 
aheadkit profile image
AheadKit •

The read_with pointer is the bit I like most — one read the server points you to instead of one per fixture is a real saving. The 44-of-100 number is the honest part though. What does MWS say the client does with the other 56 — still parse the prose and hope?

Thread Thread
 
jsbry profile image
Joshua Bryan •

Mostly it doesn't, on purpose. MWS is written for the server side, so for those 56 the fix it asks for is on the server: report a refused or failed write as a tool error (MWS-RES-2), and say "unknown" when it can't tell. It doesn't trust prose for its own verdicts either: a word list or a model reading the text never decides a result, only a declared field or a person does.

For a caller stuck with one of those servers today, what we'd do is treat any write result that isn't clearly an error as "unknown": read the object back before acting on it, and never retry a write that has no idempotency key until that read shows it didn't land. For a create there's no id to read by, so it helps to put something searchable in the write itself. That's one read per write, which is still cheaper than one data source per fixture.

Thread Thread
 
aheadkit profile image
AheadKit •

That closes the loop I was stuck on — one read per write is still cheaper than one data source per fixture, and the searchable-bit-in-the-create is a neat trick. I'm going to run that rule against a few of the prose-only servers and see how often the read actually settles it — does it ever break down for you, a create where the server won't take anything searchable at all?

Thread Thread
 
jsbry profile image
Joshua Bryan •

Yes, it breaks down. Some creates take nothing the caller controls: a comment with no title, an event stamped by the server, an upload that only gets a server-made name. Then "read it back" means listing recent objects and matching on content and time, which is a guess, not a read. In that case the honest move is not to retry and to hand it to a person. The real fix is on the server: a create that would otherwise make a second object should accept an idempotency key from the caller (MWS-RETRY-1). Of the 100 packaged servers we read, 15 do. Curious what your run finds on the prose-only ones.

Collapse
 
cianspider profile image
Cian •

The third row of your table is where my server sits, and there is one step before it that catches MCP clients out.

I run an MCP tool that is paid per call over x402. An unpaid tools/call never becomes a tool result at all. It comes back as HTTP 402 with the payment terms in the body, because the request did not reach the tool. So a client that maps any non-200 to "tool error" sees isError true and retries, and if it retries without paying it gets 402 again, forever. A client that treats 402 as a soft success is worse: it thinks the audit ran when nothing was audited.

The only fix I found was to put the price and the resource URL inside the 402 body, so the caller can choose to pay or stop instead of guessing from a status code. On the client side I ended up with three buckets, not two: transport status, then the tool result, then 402 on its own, which means "you have not asked yet" rather than "it failed".

Which leaves me with a question I cannot answer from my side: does AgentInspect record a call that is refused before dispatch as a tool span at all, or does it fall outside the contract because there was no result to evaluate?

My server, if the shape is useful to look at: mcptry.com/servers/io.github.cian-...

Collapse
 
raju_dandigam profile image
Raju Dandigam Google Developer Experts •

@cianspider That is a useful boundary. The fixture in this post starts with an MCP callTool result, so it does not establish what agent-inspect records when HTTP 402 arrives before a tool result exists. I would treat 402 as a transport/payment decision, distinct from isError, and test that the trace shows the refusal without claiming the tool ran. Your server case would make a good regression fixture for that distinction.

Collapse
 
onelifevibe profile image
OneLifeVibe •

The three-question table is the part I keep coming back to. In our own matching work, the same trap shows up as a green check: a payment call returns fine, the ledger updates, and only later does someone notice the intended outcome never happened. Your point about not turning every tool error into a retry is the one I would underline twice. A malformed argument retried ten times is still malformed, and a denied permission retried ten times just burns budget. Treating the returned error as evidence to evaluate rather than a signal to retry feels like the right default. Thanks for making the fixture small enough to actually copy.

Collapse
 
mp3sss31 profile image
mp3sss •

The three-level breakdown (returned, reported failure, actual outcome) maps onto how we assert IPC on the desktop side: a resolved promise from the main process is only an ack, not proof the file wrote or the side effect landed. A completion-only test goes green the same quiet way here. The test contract should check the tool's isError flag, then assert the externally observed state or receipt, rather than treating the completed span as success.

Collapse
 
hannie8 profile image
Hai Anh •

The third row is the one worth building tests around, and the readback step has a trap of its own. If the target system is eventually consistent (a read replica, a search index, most SaaS APIs behind a cache), an immediate readback can miss a write that actually landed, and the test flags a success as a failure. I'd make the readback bounded: poll for a short window, or read from the primary when the API lets you, and record "not yet visible" as its own outcome instead of folding it into "failed".

Collapse
 
srinivaspadmanabhuni profile image
Dr Srinivas Padmanabhuni •

Your three-question table is a useful reminder: a completed tool call does not prove the tool accepted the action, and neither proves the intended state occurred. In one recorded travel-planner trace we evaluated, a hotel hold using a price absent from search results was refused while the flight hold had already gone through, leaving a partial effect after the failed step. That is the kind of tool-trace behavior agentest checks: github.com/srpadmanabhuni/agentest

Collapse
 
gautamtalksdev profile image
Gautam K •

This gets worse one step later, when the agent writes its own summary: a tool result with isError: true followed by "Done, I've updated the record" is exactly the gap between what an agent reports and what actually happened. Asserting on isError and then reading back the real state catches both halves. Does AgentInspect flag a final message that claims success after an errored tool call?

Collapse
 
raju_dandigam profile image
Raju Dandigam Google Developer Experts •

Good catch, @gautamtalksdev. The trace check can fail on the MCP isError observation, but it does not infer whether a final natural-language claim is true. I would test that at the application boundary: compare the final claim with the tool outcome and, for a write, an independent readback. That mismatch is a useful regression case.

Collapse
 
paullovelace profile image
Paul Lovelace •

The line about not parsing error prose into policy decisions is the one I'll be stealing for code review. I've seen too many handlers that string-match their way to a retry, and then a server rewords a message and the whole policy quietly changes. Structured error kinds feel like the boring part of the design until they save you from exactly that.

Collapse
 
marcusykim profile image
Marcus Kim •

The strictEqual assertion makes me want a second fixture where writing the trace fails after callTool returns. Does the caller still get the original MCP result, with telemetry failure reported separately? Otherwise an instrumentation error could turn a successful operation into an apparent tool failure and invite an unsafe retry.

Collapse
 
alexandr_trend0x profile image
Aleksandr B. •

One more case from the read side that bit us: a field that is simply absent. A status call returns fine, isError is false, but the balance key isn't there because the upstream didn't report it. If the agent reads that as 0, it reaches a confident wrong conclusion ("the account is empty") with no error anywhere. We ended up treating "missing" as its own state next to the three rows in your table, and the agent has to say "unknown" instead of filling in a default. For writes we landed where you did: no automatic retry, and after a timeout or an unclear exchange acknowledgement, read the current state first.

Collapse
 
kitwisestudio profile image
Kitwise Studio •

An inventory tool that answers "unavailable" with isError: false is quieter than the fixture's returned-error case. The promise resolves, the error flag is clean, and a completion-only test stays green. Only a check of the reserved state would catch that nothing was booked. The same gap is already in the article: an email tool can accept a message without proving delivery. The last line is the useful rule. Every green result should name the boundary it actually exercised.

Collapse
 
connabyss profile image
CH •

The third row of your table is the one I keep running into, just with commits instead of tool calls. Lots of "fixed" commits come with a green test that still passes after you take the fix back out. Green tells you nothing blew up. It doesn't tell you the thing actually happened.

Collapse
 
kryeo profile image
Kryeo •

This is the same trap some payment gateways have had forever: the call returns 200 and the body says declined. "The request finished" and "the thing actually happened" are two different signals, and a lot of code only checks the first one.

The part about not turning every isError into a retry is the important bit. A permission error retried three times is just three permission errors, and a side-effecting call retried after an ambiguous response is how you end up with duplicates.

Do you think MCP servers will settle on structured error categories (retryable / not / unknown), or is that always going to be per server?

Collapse
 
emilywoodsnyc profile image
Emily Woods •

The identity assertion is a nice detail: recording a failed outcome should preserve what the caller received. I’d add a paired fixture where the result resolves normally with an application rejection, then check the recovery branch too. This harness-design exercise gives room to specify what the trace observes and what still needs an external receipt.