Picture a release with a perfect paper trail. You have the reviewed commit, the resolved dependencies, the builder identity, the artifact digest, and a green record of every check that ran. All of it is signed.
It can still contain a bug that lets anyone mark an invoice as paid.
Provenance is worth having, and AI-assisted development makes it more useful, because teams increasingly need to reconstruct how a change came together. The trouble starts when a complete record begins to feel like evidence that the code is good.
The audit trail vs. the actual behavior
Provenance answers questions like these:
- Which commit is in this artifact?
- Who approved it?
- Which dependency versions were resolved?
- Which build produced it, and which checks ran?
Those answers are valuable during an incident, and they help you reproduce a release. None of them tells you whether the code does the right thing.
The scenarios below are illustrative, not real incidents.
Failure mode 1: faithfully recording the wrong code
An assistant helps write a payment webhook:
app.post("/webhooks/payments", (req, res) => {
const sig = req.get("X-Signature");
if (sig && !verifySignature(req.rawBody, sig)) {
return res.sendStatus(401);
}
markInvoicePaid(req.body.invoiceId);
res.sendStatus(200);
});
A bad signature gets rejected. A request with no signature at all skips the check and marks the invoice paid.
The build can be hermetic and the attestation can verify cleanly. The record is accurate: this exact code, built this exact way, reached production. Build integrity tells you the artifact wasn't tampered with on the way there, and frameworks like SLSA are aimed at that problem. Whether the source was right to begin with is a separate question.
Failure mode 2: "checks ran" gets read as "checks were enough"
The tests for that webhook might cover a valid signature and an invalid one. Both pass, so the record says unit-tests: pass, and that's true.
The missing case is the one that matters:
test("rejects a webhook with no signature", async () => {
const res = await request(app)
.post("/webhooks/payments")
.send({ invoiceId: "inv_123" });
expect(res.status).toBe(401);
});
A provenance record can show that a job ran and passed. It can't show what the job never tried.
Failure mode 3: a record of review isn't a record of scrutiny
approved_by: reviewer-a is a fact about a click. It doesn't tell you whether the reviewer traced the unsigned path, or spent four minutes on a large AI-assisted diff near the end of the day.
Keep the approval record. Just don't let it stand in for the review itself. On sensitive changes, the reviewer should be able to answer a specific question: what happens when this header is missing?
Failure mode 4: a clean scan is a snapshot
A dependency scan can pass on release day, and a vulnerability in one of those packages can be published a week later. The record of that clean scan stays true while the risk changes.
Here, provenance earns its keep. If you recorded exactly what shipped, you can answer "are we affected?" in minutes instead of guessing. Just treat the old scan result as a dated observation, and keep scanning what's already deployed.
Failure mode 5: the audit log becomes the leak
It's tempting to capture everything an AI tool touched, including full prompts. But prompts pick up whatever a developer pasted in: a stack trace with a customer's email address, a production log, a connection string.
Now that material sits in a long-lived record that may be widely readable, kept in the name of traceability.
Most incident questions don't need the raw prompt. Something like this usually goes a long way:
{
"artifact_digest": "sha256:...",
"commit": "9f3c1ab",
"pull_request": 4821,
"approved_by": ["reviewer-a"],
"lockfile_sha256": "...",
"builder": "ci/release.yml@refs/tags/v2.14.0",
"checks": [
{ "name": "unit-tests", "result": "pass" },
{ "name": "static-analysis", "tool_version": "...", "result": "pass" }
],
"ai_assisted": true,
"ai_tool": "assistant-name@version"
}
That shape is only an example. If you want a standard format, look at SLSA provenance and in-toto attestations. If your team does need to keep prompts, redact them, limit who can read them, and give them a retention period.
What goes in the record, and what still has to happen
Record what helps you explain and reproduce a release:
- Commit, PR, and approvers
- Resolved dependencies
- Builder identity and artifact digest
- Which checks ran, with tool versions and results
- Whether AI tools were involved, and which ones
Then do the work the record can't do for you:
- Code review: is this the right behavior?
- Testing: does it do what the requirement says, including what it has to refuse?
- Security analysis: are there risky patterns, data flows, vulnerable dependencies, or committed secrets?
Provenance is the part that lets you trace and reproduce a release. Those three are how you find out whether it should have shipped.
A signed record tells you where the software came from. It takes a different kind of work to know whether it's safe.
What does your team actually record for AI-assisted changes, and has any of it helped you during a real incident? Do you keep prompts, or did you decide against it?
Top comments (16)
Attestation proves provenance, not correctness. A chain of custody is only as strong as the key rotation behind the signing step, so the useful question is how often the pipeline can verify the whole chain end to end rather than just the last link.
Absolutely, and key management is an important part of that trust chain. If signing keys are compromised or poorly rotated, even the provenance itself becomes questionable. End-to-end verification should be routine, not something teams only think about during an incident. But even a perfectly verified chain still brings us back to the original point: it proves where the artifact came from, not whether the code does what it's supposed to do.
Rotating the signing key matters as much as the signature. An old key still trusted makes the chain look clean.
That's a good point. Rotation alone isn't enough if old keys remain trusted indefinitely. There needs to be a clear process for revocation, expiration and checking which keys were valid when an artifact was signed. Otherwise, the signature can remain technically valid while the trust behind it has already expired.
Failure mode 5 is the one nobody here has picked up yet, and it's the same bug as your webhook.
The unsigned request that skips the check, and the field that skips the redaction, are one failure wearing two hats: a guard that only covers what somebody remembered to list.
if (sig)means "check everything except the case where there is no sig." A redaction step built from field names means "mask everything except the fields that did not exist when the list was written" - a new SDK, a tool that starts recording file paths, a header somebody adds next sprint.Making the list longer does not fix a list-shaped guard. To find out it is already broken: put one sentinel string into every client-supplied surface at once (the prompt, an env var, a filename, a user field), run one release, then read the record back from the store itself and grep the raw bytes. If the sentinel shows up anywhere, the redaction is decoration. Then leave a canary in the prompt as a standing gate, so the next field that gets logged trips it.
I have watched this find a header nobody listed coming back verbatim from a store whose policy named, in good faith, every field it masked.
Section 4 has the same shape.
artifact_digestin the record is a statement about the artifact, not a measurement of it. A verifier that compares that string against another value in another record proves the record is self-consistent. Only recomputing the digest from the bytes answers whether the artifact matches the thing you shipped.That's a really good catch. I hadn't connected the two failure modes quite that way, but you're right. Both are examples of relying on an explicit list of things we know about instead of testing what happens when something unexpected appears. I especially like the sentinel approach because it tests what actually ends up stored, not just what the redaction code claims to do. The digest point is equally important: verifying a claim in an attestation isn't the same as independently verifying the artifact.
A provenance record answers where an artifact came from; it does not answer whether the source or its behavior was safe.
The strongest workflows keep those questions separate: attest the build inputs and environment, then pair that evidence with policy checks, tests, and an explicit release decision.
Exactly. Keeping those questions separate is the important part. Attestation tells us what went into the artifact and how it was built. Tests and policy checks tell us something about its behavior, but even those depend on what we choose to test. The release decision needs to consider the gaps too.
Spot on, Dimitris! This is a crucial distinction that often gets blurred in DevSecOps discussions. Build attestation proves provenance (who built it, from what source, with what tools), but it says absolutely nothing about the inherent safety of the code itself.
In the Web3 space, we see this exact gap. A team might have a perfectly attested, OIDC-secured CI/CD pipeline (proving the chain of custody is intact), but a developer could still accidentally commit a hardcoded test key or introduce a subtle logic flaw. The attestation will happily sign off on that vulnerable artifact.
This is why attestation must be paired with automated, context-aware security gates in the pipeline. In my work, I build custom Slither detectors that run as a hard fail in CI/CD. The attestation proves the detector ran in a trusted environment, but the detector is what actually validates the safety of the smart contract.
Curious to hear your take: As attestation frameworks (like SLSA) become more standard, do you see security teams struggling with 'scanner fatigue' from false positives, or is the real bottleneck still just getting teams to adopt the attestation infrastructure in the first place?
I think the bigger bottleneck is still getting teams to adopt the infrastructure and, more importantly, understand what each layer actually guarantees. Scanner fatigue is definitely real, though. If teams get flooded with false positives, they eventually start treating security gates as another checkbox rather than something worth investigating.
I like your Slither example because it highlights the distinction nicely. The attestation gives you confidence that the checks ran in a trusted environment, while the detector provides a separate layer of evidence about the code. Neither replaces the other. I think the real challenge is making these layers work together without turning the pipeline into noise.
The missing-signature webhook is the perfect example — the attestation is accurate, the artifact is exactly what was built, and the invoice still gets marked paid. Failure mode 3 is the one I'd put on a wall: approved_by is a fact about a click, not a record of scrutiny. Do you find teams actually change anything once they see that distinction, or does the green checkmark keep winning the argument?
I think it depends on whether the distinction leads to a concrete change in the release process. Most teams agree with it in principle, but the green checkmark is hard to challenge when deadlines are tight. What helps is turning vague approval into specific questions about risky behavior. Instead of just asking who approved the PR, ask what failure cases were actually examined. That's where review starts becoming meaningful rather than procedural.
I'd strengthen the missing-signature test with a side-effect assertion: capture the invoice's state, send the unsigned webhook, then assert it is still unpaid (or that markInvoicePaid was never called). A handler could mutate the invoice and return 401 afterward, so the status-only test would pass despite violating the requirement. That gives the review question in failure mode 3 a concrete invariant: rejected requests must leave payment state unchanged.
Absolutely. That's an important improvement to the test. Checking only the HTTP status can give us a false sense of security if the side effect has already happened. The real invariant is that a rejected request must leave the invoice state untouched. I'd argue that's exactly the kind of test that makes code review more meaningful because it verifies the actual business requirement, not just the response the handler returns.
A provenance record answers where an artifact came from; it does not answer whether the source or its behavior was safe.
Well put. I'd only add that the distinction matters most when things go wrong. A provenance record can help us trace exactly what shipped, but it shouldn't become a convenient way to claim that the release was properly reviewed or safe. Those are different responsibilities.