I don't read most of the pull requests my AI writes anymore. I read the title, the description, whether CI is green and what the agent says it checked, and then I merge. The exception is anything I can look at: a page, a screenshot, a diagram. Those I still open, every time.
That sounds reckless written down, so I went back through my own repos to see what it actually looks like in numbers. These are the two main repos behind ExtraTime, my football quiz site: a Django API and a Next.js frontend. I'm the main author of both, with a couple of friends sending PRs now and then.
How it got here
I started letting an agent open pull requests for me in April 2026. Since then I've merged 1,396 PRs into those two repos, and 1,241 of them say in the description that they were written with Claude Code. In October alone I merged 563, with about 370,000 lines changed between them.
There's no way I read 370,000 lines in a month. I don't think anyone would, and pretending I did would be worse than saying so.
But the part that surprised me is what came before. In 2025, before I let an agent open PRs, I merged 321 into the same repos. The median description was empty. The median time from opening a PR to merging it was 2 minutes on the API and 4 on the frontend. I wasn't reviewing my own code then either. I was the reviewer and the author, so the PR was just a button I pressed on the way to deploying.
So what I gave up wasn't careful review. It was the feeling of knowing every line, because I'd typed every line.
What I read instead
The title and description. Since April the median PR description in these repos is 2,400 to 2,800 characters, against zero before. The agent writes what it changed, why, and how it tested it. That's the part I read properly. If the description doesn't make sense, or it describes a different change than the one I asked for, the PR doesn't get merged, whatever the diff says.
CI. Every API PR runs the Django test suite in two shards plus a Pyflakes check for new findings. The frontend builds, lints, type-checks, runs its unit tests and then runs Playwright end-to-end tests in Chromium and Firefox. What's changed is how much there is to run. On 1 January the API had 442 test functions. On 1 April it had 1,496, and today it has 6,383. The frontend went from 236 on 1 April to 2,483. Most of the tests added since April came in the same PR as the change they test.
The agent's own review. The agent leaves a comment on the PR saying what it checked, along the lines of "Reviewed: the PENDING to CORRECT transitions are now a compare-and-set, a loser drops only the row it inserted". GitHub's Copilot reviewer also reviews some of them. I read those comments the way I'd read a colleague's summary, not as proof.
And mostly, I trust it. That's the honest version. If the description is right and CI is green, I merge without opening the diff.
What I still look at
Anything that ends up on a screen. For frontend changes I open the Vercel preview and click through it. If the PR has screenshots, I look at them. If it has a diagram, I read the diagram.
The reason is simple. A test can tell me a function returns the right thing. It can't tell me the page looks wrong. Going back through the frontend fixes since April, about two dozen have titles like these:
fix(lobby): no sideways scrolling on phonesfix: five clubs ā wrap long footballer names on the cardfix(lineup): stop the mobile hydration swap that shifted the footer (CLS 0.79 ā ~0.06)fix: prevent burger menu items from being clipped when Notifications is expanded
Each of those is about how a page looks or moves, not what it returns. A layout shift of 0.79 means the page visibly jumps under your thumb. The end-to-end tests click through those same pages in two browsers, and they pass, because a button that moved is still a button. Someone had to look.
So the rule I've ended up with is: I review what I can see, and I trust what I can test.
What I got wrong
Only 81 of the 1,396 PRs since April have an image in the description. 4 have a diagram. So "I look at anything visual" mostly means I have to remember to open the preview. When I don't, nothing reminds me. I should have made the agent attach screenshots to every frontend PR from the start, rather than when I happened to ask.
The other thing is where the bugs go when nobody reads the diff. In 2025, 10 to 13% of my merged PRs were fixes. In October it was 46 to 47%, and 121 of October's PRs mention a Sentry issue. Some of those are noise rather than bugs (one is literally about dropping planned Postgres stops from Sentry), and I can't separate fixes for code nobody read from fixes for code from years ago. But the shape is clear. Not reading the code doesn't make the bugs disappear. It moves them to production, where Sentry finds them instead of me. For a quiz site that's a trade I'm fine with. For a bank it wouldn't be.
Not one PR since April has been reverted. I first read that as good news. It more likely means I fix forward: when something breaks, the answer is another PR, not undoing the first one, and that shows up in the fix count instead.
Caveats
This is one person's two repos, with a small team and a hobby project's stakes. "Written with Claude Code" is what the PR description says, and some of those may have my edits in them too. The fix and Sentry counts are from PR titles and the first part of each description, so they're approximate. And a test count isn't a measure of test quality. 6,383 tests can still all test the wrong thing.
Where I ended up
I don't think the answer is to go back to reading every line. I didn't do that before AI either, and the volume now makes it impossible. What I'd say instead:
- Read the description like it's the contract. If it's wrong, nothing else matters.
- Make the tests the agent's job, in the same PR. They're what I actually trust.
- Look at anything a user will see. That's where tests are blind and I'm not.
- Ask for screenshots by default, so looking isn't something you have to remember.
- Watch production. If you don't read the diff, Sentry is your reviewer now, and you should treat its alerts that way.
Top comments (17)
Your last bullet, screenshots by default, is a rule in our release process at work. Every feature attaches two short videos before it reaches QA: one showing the product, one walking through the interfaces and the contract. The product owner and the dev lead check those first, then QA tests from the feature doc, the way a client would hit it, not from the diff. Did October's fixes cluster in the screens, or in the parts you trust the tests for?
Mostly in the parts I trust the tests for. Counting October PR titles that start with "fix", about 45% of the API's merged PRs were fixes and about 47% of the frontend's, so neither repo stands out, and the API had nearly twice as many in total. Of the frontend fixes, at most about a quarter look like screen problems by their titles: layout, scrolling, wrapping, mobile.
Almost every fix in both repos also touched a test file. I read that as the agent adding a regression test along with the fix, not the tests catching the bug beforehand. This is a rough count from titles, not a look at each PR.
Your two videos are close to what I said I should have done with screenshots from the start, just with the contract walkthrough added.
A regression test that lands with the fix records where the bug was, not who caught it. That is why our QA writes its cases from the feature doc at hand-off, while the dev builds, not after. How many of the October fixes started as a report from a user rather than your own review?
Almost none, going by the issues. Most of October's 278 fixes point back at an issue, 209 different ones. By the issue text, 20 started from Sentry, half of them through a button on our Discord alerts that files the issue straight from the error. 8 were bugs I reported myself, and 1 came from a player. The other 180 or so read like the agent's own findings from reading the code, and a lot of them are follow-ups to an earlier fix. I've only read a sample of those, so take that split as rough.
Which supports your point. The regression test says where the bug was. It took the issue history to say who found it, and by that count it was mostly the agent, not users and not me.
180 of 278 from the agent's own reading is more than I would have guessed. The follow-ups are the part I would count next. My experience is that agents are not deterministic: one run follows every rule, the next does not, so a fix can need a second fix. How soon after the first fix did the follow-ups usually land?
Quickly, for most of them. I checked it the git blame way: for each October fix, which earlier PR last wrote the lines it changed. For 82 of 236 fixes, that earlier PR was itself a fix. Half of those follow-ups landed within about 4 days of the first one. 23 came within a day and 50 within a week. The rest are spread out, some months later, and those are probably not a second try at the same bug.
Blame only shows who last touched the lines, not that it was the same bug, so it's a rough count in both directions. But it does fit what you describe. When a fix needs another fix, it mostly shows up in the same week.
Visibility is a good compression strategy, but the ranking signal needs a falsifiable link to risk. Diff size alone is noisy; boundary crossings, permission changes, generated-file churn, and behavior not covered by tests are often better review triggers. Iād preserve the reason a PR was surfaced alongside the PR, then audit false negatives after incidents to tune the queue.
The line that stuck with me is "read the description like it's the contract". I agree, and it's also the one input on your list that nothing checks against the diff. CI checks the code, you check the description against what you asked for, but the link between description and diff is taken on trust.
We had exactly that case. While we were testing a GitHub Action in our own test app, Claude Code made a commit titled "Limit contact name to 250 chars". The sed it ran hit the email rule instead, because the name was already max:200. Tests green, message perfectly sensible, wrong line changed. Since then we run a check that compares what the message claims with what the diff touches. On 937 commits from five PHP repos it flagged 23, about 2.5%, so it's quiet enough to leave on.
With descriptions at 2,400+ characters there's even more room for the text to drift from the code. Have you caught one of your 1,396 where the description was right about the intent and wrong about what actually changed?
Not that I can show you, because I haven't run anything like your check. I tried the cheap version on October's PRs, matching the scope in the title against the paths the diff touches, and it's useless here. 107 of 520 scoped titles didn't match any path, and the ones I went through were all feature names rather than folders: "grid rooms", "tenagoal", "guesses". The scope is how I think about the product, not where the code lives.
So a check like yours would have to read the claim and the diff, the way yours does, not compare two strings. Your sed example is exactly the case my setup wouldn't catch: tests green, description sensible, wrong line. 2.5% flagged on 937 commits is quiet enough to be worth trying.
Feature names instead of folders is the part that kills any string comparison, and it's the same reason a path check couldn't have caught the sed case: the title said the right thing about the right feature. The line that was wrong was simply not the one the sentence was about.
That's why our check starts from the claim, not from the scope. It takes what the commit message says it changed and looks for the changed lines that sentence cannot account for. On the 937 commits that gave 23 flags, and the ones I read were all message and diff disagreeing, not naming conventions.
If you try it on October's 520, the useful number is not the flag rate but how many flags you'd have wanted to see. I'd treat anything above a handful a week as a bug in the check, not in your PRs.
Interesting perspective! The increase in fix-related PRs is particularly telling. As AI increases development speed, I think QA becomes even more important. Green CI is valuable, but we also need confidence that we're testing the right things, not just that the tests pass.
1,241 agent PRs and still opening the ones you can actually look at ā that's the honest version of review, not the pretend one. I do the same with screenshots and pages, and wave the rest through on green CI with a straight face. Did anything in the ones you did look at ever make you go back and re-check a merged one you hadn't?
The history can't say what prompted a second look, but it does show how often a merged PR got revisited. I blamed the lines each October fix changed back to the PR that last touched them. 81 of 235 fixes went back to something merged in the previous seven days, and 82 repaired a PR that was itself a fix.
So going back happens a lot. It just happens as another PR instead of a revert, which is why the revert count since April is still zero. Whether a preview or a Sentry alert started it is not something I can pull apart from the data. For the fixes that mention Sentry, the median gap from the broken PR to its fix was 14 days, against 41 for fixes overall, so the alerts at least get there sooner.
The jump from 10 to 13% fixes in 2025 to 46 to 47% in October would be easier to read if each fix pointed at the PR it repairs. git blame on the lines a fix touches gets you most of the way, and it would separate unread agent PRs from old code. I'd start with the 121 that mention Sentry
Ran it. For each October fix I took the lines it changed or deleted, blamed them at the commit just before the fix, and counted the PR that last touched most of them. 235 of the 278 fix PRs had lines I could trace to an earlier PR. Nearly all of the rest only added code.
192 of the 235, about 82%, mostly repaired lines from a PR whose description says Claude Code. That sounds bad until you put it next to the base rate: 84% of everything merged since April is an agent PR, so agent code isn't over-represented among the things that needed fixing. The worry it does settle is the one in the post about fixes for old code. Only 5 went back to code from before April. Blame only sees the last change to a line, so an agent refactor of old code counts as agent code, but that wouldn't hide much here.
The Sentry subset first needs a correction. Searching titles and full descriptions, I get 93 October PRs that mention Sentry, not 121, and I can't reproduce the original count. 52 of those are fixes, and 43 had lines to blame. 40 of the 43 repaired agent PRs, none touched pre-April code, and the median gap between the broken PR and its fix was 14 days, against 41 for fixes overall. So Sentry catches the agent's mistakes faster than everything else does, which is about what you'd hope it does.
same habit with data. i never read the whole file but i always open five rows by hand before anything goes into a list, it catches more than any summary the tool gives me
Some comments may only be visible to logged-in visitors. Sign in to view all comments.