DEV Community

Cover image for Jira said Done. It wasn't, and the bug showed up later. So I made Jira check first.

Jira said Done. It wasn't, and the bug showed up later. So I made Jira check first.

Wageesha Tripathi on November 15, 2026

Picture this. The ticket moved to Done on Friday. The pull request was merged, CI was green, everyone moved on. On Monday the checkout total was w...
Collapse
 
iictp profile image
IICTP •

The toBe(90) to toBeDefined() example is the one that sticks with me — it's such a small edit and CI stays green the whole time, so nothing downstream ever notices. I also appreciated that the check tells apart a genuinely weakened assertion from honest changes like a test moving files or a value going from 200 to 201. Have you seen teams use the override-with-a-reason much in practice, or does the block itself mostly get people to fix the test instead?

Collapse
 
wst2002rk profile image
Wageesha Tripathi • • Edited

Thank you! That tiny edit with CI staying green is exactly the case I wanted to catch. So far the block mostly gets the test fixed, because the panel shows the exact line before and after, and putting it back is usually a one-line change. The override is for the rare change that is on purpose. In the demo, DEMO-2 changes rounding on purpose, so an admin lets it through with "Rounding down approved by finance, see FIN-12", and that reason stays on the ticket with their name. A new commit cancels it, so it can't quietly cover later changes. Both are in the demo video.

You clearly think about this a lot, so if you ever spin up the DEMO project, I'd be really curious what you would change. Your take would genuinely help.

Collapse
 
iictp profile image
IICTP •

That makes a lot of sense — showing the exact line before and after makes fixing the test the easier path, so the override stays rare instead of becoming a shortcut. The DEMO-2 example is a nice touch too, especially the reason staying on the ticket with the admin's name and a new commit cancelling it. Do teams ever push back on that expiry, or does the one-line fix usually win them over once they see the panel?

Collapse
 
aheadkit profile image
AheadKit •

expect(total(cart)).toBeDefined() passing on a total of 0 is the funniest and saddest line I've read this week. A test that passes when the cart is empty and passes when the cart costs 900 — that's not a test, that's a mood. The Friday-to-Monday shape of it is so familiar too: green CI, ticket closed, everyone relaxed, and then the checkout total is wrong on Monday morning. Your four checks are the right shape, but honestly the one I'd add first is the simplest: if the diff touches a test file, somebody human has to read the diff. No tooling needed, just a rule nobody can click past.

Collapse
 
maxbuildsdev profile image
Max Kaya •

The assertion-diff check is the right instinct, but text-diffing assertions has a blind spot: a test can keep toBe(90) verbatim and still stop proving anything once the thing it exercises gets replaced — the component boundary moves behind a mock, or a fixture/seed changes so the number is now produced by the stub. The line looks identical in the diff and the coverage is unchanged.

The check I'd trust more there is a scoped mutation run on the files the PR touched (stryker on the changed modules, or mutmut for pytest), which catches loosened assertions and mocks in one step, though it's minutes of CI rather than a parse. Two questions: does the app compare assertions structurally (AST, so toBe(90) → toBe(total) is caught even if the string is reformatted), and does anything stop a JS snapshot test from being re-baselined in the same commit, since that produces a big green diff reviewers skim past?

Collapse
 
paullovelace profile image
Paul Lovelace •

The receipt on Done is the bit I'd steal first. A blocked ticket is useful, but a paper trail saying exactly what was checked, on which line, is what stops the same argument happening again on Monday morning. Haven't tried the Jira app myself, but that part feels like it'd survive contact with a real team.

Collapse
 
halfcycle profile image
Halfcycle •

toBe(90) to toBeDefined() is the cleanest example of this I have seen, and "fix flaky test" is exactly the label it hides under.

One check we leaned on while building Halfcycle: when a change edits a test, run the new version against code that is known to be wrong (the bug the test was written for, or a quick mutation like returning 0). If it still passes, the test stopped testing something. That catches the weakened assertion, not only the deleted or skipped one.

Does proof-of-done look at how strong an assertion is, or mainly at tests that were removed or skipped?

Collapse
 
hitensethiya profile image
Hiten Sethiya •

the toBe(90) to toBeDefined swap is nasty bc in review it reads like cleanup. what happens when the test really was flaky and someone needs to ship a hotfix friday, is there an override that leaves a trail or does Done just stay blocked

Collapse
 
wst2002rk profile image
Wageesha Tripathi • • Edited

Thank you, Hiten, and yes, that Friday hotfix is the real test. A project admin can press "Wrong block? Override", write a reason, and Done goes through. The reason and the admin's name stay on the ticket, and a new commit to the pull request cancels the override, so the trail always matches the code that shipped. There's also a "Create a task to fix this" button, so the flaky test becomes the next piece of work instead of being forgotten.

Funny enough, your Friday hotfix is exactly the scenario I'd love someone to throw at it. If you ever play with the DEMO project, tell me whether the override felt right or got in the way.

Collapse
 
sandipan_chakraborty_c557 profile image
Sandipan Chakraborty •

Love this idea! A smart solution to a real problem developers face. Great work!

Collapse
 
wst2002rk profile image
Wageesha Tripathi • • Edited

Thank you so much, Sandipan! If you'd like to see it in action, here's the 4-minute demo on YouTube.

Collapse
 
sandipan_chakraborty_c557 profile image
Sandipan Chakraborty •

Of course! I’d love to watch it. Thanks for sharing, Wageesha!

Collapse
 
varun_bansal_4728b5726b35 profile image
Varun Bansal •

This idea is solving a real problem.

Collapse
 
wst2002rk profile image
Wageesha Tripathi •

Thanks a lot, Varun! That was the whole goal: Done should mean the tests still prove something.