DEV Community

Cover image for I Pointed 3 Static Scanners at an MCP Server That Fails 8 Auth Tests. They Found 0, Including Mine
Kiell Tampubolon
Kiell Tampubolon

Posted on Originally published at kielltampubolon.id

I Pointed 3 Static Scanners at an MCP Server That Fails 8 Auth Tests. They Found 0, Including Mine

SAST blind spots miss auth bugs

In the previous post in this series I built a 10-test harness for the MCP 2026-07-28 authorization rules. A quickstart-style server passed 2 of them. A hardened version passed all 10.

That left an uncomfortable follow-up question. I maintain an MCP security scanner, mcpscan. If the naive server had shipped, would my scanner have stopped it? Would anyone's?

So I ran three static scanners against both servers. They produced the same report for the broken one and the fixed one.

TL;DR

  • I scanned an MCP server that fails 8 authorization tests with three static analysis tools: mcpscan (my own, 23 rules in v0.20), Semgrep (200 rules from the p/python, p/jwt and p/security-audit packs) and Bandit.
  • All three found 0 of the 8 auth bugs. They reported identical findings for the naive and the hardened server, so their output couldn't tell the two apart.
  • Five custom Semgrep rules, about 40 lines of YAML, caught 3 of 8 (no audience check, no issuer check, token read from the query string) with zero hits on the hardened server.
  • The other five (missing discovery headers and metadata, missing scope checks, anonymous tools/list, token passthrough) only showed up when a test talked to a running server.
  • Static scanners aren't wrong here. They're looking for a different class of bug. Auth for MCP servers needs runtime tests in CI, with a few static rules as a cheap early warning.

The setup

Both servers come from the auth harness. To keep the comparison clean, I split the original two-mode file into two single-mode copies, naive/server.py and hardened/server.py, so a scanner only ever sees one version. Then I re-ran the harness against each copy to make sure the split didn't change behaviour: still 8 failed and 3 passed for naive, 11 passed for hardened (10 tests plus a control).

The eight failures the scanners had to find:

# Failure in the naive server Spec level
01 401 without WWW-Authenticate / resource_metadata MUST (header or well-known URI)
02 No Protected Resource Metadata endpoint MUST
04 No audience validation MUST
05 No issuer validation MUST (via OAuth 2.1)
07 No per-tool scope check SHOULD
08 tools/list open to anonymous callers SHOULD
09 Client token forwarded to the downstream API MUST NOT
10 Token accepted from ?access_token= MUST NOT (client rule)

Some of these are as obvious in the source as a bug can be. The decode call literally says:

claims = jwt.decode(token, signing_key(token), algorithms=["RS256"],
                    options={"verify_aud": False})
Enter fullscreen mode Exit fullscreen mode

The results

Matrix of the 8 auth failures against four scanners: mcpscan, Semgrep with 200 rules and Bandit each caught 0 of 8; five custom Semgrep rules caught 3 of 8.

What each tool did report, for both servers:

  • mcpscan: nothing. "Clean, no issues found."
  • Semgrep (200 rules run): one finding, dynamic-urllib-use-detected on the downstream call, because urllib supports file:// URLs.
  • Bandit: two findings: B310 on the same urllib call, and B105 "possible hardcoded password" on an empty string default.

None of those are about authorization, and all of them appear in the hardened server too. The verify_aud: False line above went unflagged by all three.

My own scanner deserves the most direct explanation. mcpscan's auth-adjacent rule, MCP012, checks client configuration: a remote MCP server entry with no auth header, or a hardcoded bearer token. It looks at how you connect to a server, not at how a server validates tokens. That's a real gap, and I'd rather find it with a harness than have someone find it in production.

Five rules that catch three bugs

Semgrep makes it cheap to encode a specific "never do this". These two rules did most of the work:

rules:
  - id: jwt-audience-check-disabled
    languages: [python]
    severity: ERROR
    message: Audience validation disabled. MCP 2026-07-28 says servers MUST validate the token audience (RFC 8707).
    pattern: 'jwt.decode(..., options={..., "verify_aud": False, ...}, ...)'
  - id: jwt-decode-without-audience
    languages: [python]
    severity: ERROR
    message: jwt.decode() without audience= accepts tokens minted for other services.
    patterns:
      - pattern: jwt.decode(...)
      - pattern-not: jwt.decode(..., audience=$A, ...)
Enter fullscreen mode Exit fullscreen mode

Add the same shape for issuer=, and one rule for $Q.get("access_token", ...), and you get this against the naive server:

access-token-from-query-string   line 73
jwt-audience-check-disabled      line 77
jwt-decode-without-audience      line 77
jwt-decode-without-issuer        line 77
Enter fullscreen mode Exit fullscreen mode

Against the hardened server: zero findings. Four hits on the naive server cover three distinct bugs (04, 05 and 10).

The fifth rule was the one I most wanted to work: a taint rule from headers.get("Authorization") to a downstream add_header("Authorization", ...), which would catch token passthrough (09). It found nothing. In this server the token is read in one function, returned, and handed to a second function that makes the downstream call. My best guess is that this cross-function flow is beyond what the rule can follow in Semgrep's open-source engine; I didn't dig further. The runtime test caught it in one assertion.

Why the other five can't be seen statically

Three layers: static rules caught 3 of 8, runtime tests 8 of 8, generic SAST and config scanners 0 of 8.

Look at what the remaining failures have in common:

  • 01 and 02 are about something missing: a header that's never set, an endpoint that doesn't exist. Static rules find bad code much more easily than absent code.
  • 07 and 08 depend on a mapping between tools and scopes, and on which methods your policy says are public. That's business logic a generic rule can't know.
  • 09 is a data flow across functions, which is hard for pattern-based tools by design.

A runtime test doesn't care how the code is written. It sends a request and checks the response, or checks what the fake downstream received. That's why the harness caught 8 of 8 and the scanners caught 0.

What I'd put in CI

  1. The runtime harness, on every pull request that touches the server. Ten tests ran in about 3 seconds here.
  2. A handful of custom static rules for the patterns you never want to see: verify_aud: False, decode() without audience=, tokens from query strings. They fail fast, before anything starts.
  3. Your general scanners as well. Semgrep and Bandit flagged a real hardening point (the urllib scheme), and mcpscan covers configuration and supply-chain risks the harness doesn't. They just aren't an auth test.

And on my side: mcpscan needs server-side auth rules. The two Semgrep patterns above are the obvious starting point.

Caveats

  • One server, written by me. The naive baseline is deliberately typical, not a sample of real-world servers.
  • Three scanners, default or common rule packs, in Python. Commercial SAST tools, Semgrep Pro's cross-file analysis, or other languages may do better, especially on the passthrough case.
  • mcpscan is my project, so treat my explanation of its scope as the author's view. The scan output itself is unedited.

Update: I fixed mine

A scanner that misses all eight is a roadmap, so I wrote the rules. mcpscan now ships seven server-side authorization rules, MCP024 to MCP030, built on the Python AST rather than regex. Same naive server, same command:

# Failure Before After Rule
01 401 without resource_metadata no yes MCP027
02 No Protected Resource Metadata endpoint no yes MCP028
04 No audience check no yes MCP025
05 No issuer check no yes MCP026
07 No per-tool scope check no no runtime only
08 Anonymous tools/list no no runtime only
09 Token passthrough no yes MCP029
10 Token from ?access_token= no yes MCP030

0 of 8 became 6 of 8, with 0 findings on the hardened server. MCP024 also flags jwt.decode with signature verification turned off. I ran the new rules over fastmcp, the official mcp SDK, LiteLLM and seven other MCP projects to hunt for false positives; the two causes I found are fixed and covered by tests. Scope checks and anonymous listing stay runtime-only for the reasons above, so the harness still earns its place in CI.

The rules are on the mcpscan main branch.

The takeaway

A clean scanner report means the scanner found nothing in its own rule set. These three tools ran hundreds of rules and gave the same verdict on a server that fails eight authorization tests and on one that passes them all. Test MCP auth the way an attacker would hit it, against a running server, and add static rules for the mistakes you never want to see.

Has your scanner ever given a clean report on a bug you later found by hand?

Top comments (6)

Collapse
 
cryptostrapon profile image
cryptostrapon •

Honest result, and rare to see a maintainer test their own scanner this way. Did the 3 custom Semgrep rules that worked also hold up on an MCP server you didn't write, or are they still tuned to your two test servers?

Collapse
 
kielltampubolon profile image
Kiell Tampubolon •

Fair question, and the honest answer is no: the five Semgrep rules were only ever run against my two test servers, so I can't claim they generalise. What did get run on code I didn't write is the follow-up, the seven AST rules that went into mcpscan (MCP024 to MCP030). I ran those over fastmcp, the official mcp SDK, LiteLLM and seven other MCP projects to look for false positives, found two causes and fixed them. That only shows they stay quiet on good code, though. The useful next test would be a real server with an auth bug I didn't plant myself.

Collapse
 
cryptostrapon profile image
cryptostrapon •

That split makes sense, quiet on good code is only half of it. For a bug you did not plant, older releases of MCP servers with published security advisories could work as a test set, since the fix commit shows exactly what a scanner should have flagged.

Collapse
 
mp3sss31 profile image
mp3sss •

This lines up with the usual SAST blind spot: auth and authorization are runtime, cross-request decisions, so a static scanner reads the handlers but never sends an unauthenticated or cross-tenant request to see whether the guard actually fires. A dynamic harness that probes each tool without a token, with the wrong scope, and after expiry catches what pattern matching misses. On the desktop side it's the same gap: linting the IPC handlers won't prove a renderer can't invoke a privileged channel — you need integration tests that actually send those messages and assert they're rejected.

Collapse
 
x1y1co profile image
X1Y1 •

This matches what we saw from the outside. On November 4 we checked 61 login-protected vendor MCP servers against the 2026-07-28 spec with a passive probe: one unauthenticated initialize plus the public discovery documents. All 61 got the 401 and PKCE right. But 40 didn't advertise the RFC 9207 issuer parameter, and 22 gave no scope hint in the 401 or in scopes_supported. None of that is in the source a scanner reads, and all of it shows up in one anonymous request to the running server. Your tests 01 and 02 alone would make a cheap CI gate.

Collapse
 
agentsecindex profile image
Agent Security Index •

Your result matches what the protocol docs imply. MCP authorization is optional, and the interesting failures are usually "the server shipped without audience checks or token binding," not a bad regex in source (MCP Authorization spec, 2026-07-28). A static rule can flag a missing library call, but it cannot tell whether a live server rejects a token minted for a different audience, or whether a tool description changed after the client first listed it. The harness you already built against the 2026-07-28 rules is the control I would keep as the merge gate, and I would add two runtime cases scanners also miss: a token with the wrong audience, and a second tools/list after the server has been up long enough to change a description. Your finding is the right caveat for any scanner comparison: use those tools for known patterns, not as proof the auth story works.