Three months ago your team switched on an AI reviewer for every pull request. At first it was impressive. It spotted a missing null check and an unclosed file handle in the first week. Now every pull request gets fifteen or twenty comments from it, most of them about naming, small refactors or things that are already handled elsewhere.
Then you notice something. People have started resolving comments in bulk without reading them. That habit does not stay with the bot. Last week a colleague left one careful comment about a race condition, and it was marked resolved along with twelve others in under a minute.
This is the real cost of noisy feedback. Each comment costs very little, but together they teach people to stop reading, and then the comments that matter get the same treatment as the ones that do not.
So raise it with the team as a problem about attention, not as a complaint about the tool. Bring a few real examples. Count, roughly, how many of the bot's comments led to a change in the last month. The number often settles the discussion on its own.
Then agree on a few rules. Most of these tools can be configured. Turn off the categories that never lead to changes, such as style points your formatter already enforces. Decide what an AI comment requires from the author. Perhaps a reply or a reaction, but not a fix unless the author agrees with it. And decide what a human comment requires, which should be more.
Make human feedback easy to spot. Some teams ask reviewers to start blocking comments with a clear label, or to leave a short summary at the top of the review saying what they checked and what worried them. Anything that separates a person's judgement from the general noise helps.
And keep the human review. The bot can be a useful first pass, but it does not know why the change exists, who depends on it, or what the team tried last year. Approving because the bot found nothing is not a review. It is a guess with a green tick on it.
– Asael Shinder
Top comments (1)
Where the bot's output lands matters as much as how much of it there is. If the team has "Require conversation resolution before merging" turned on, that rule counts twenty bot threads and one colleague's thread the same way, and resolving all twenty-one in a minute is the fastest way to a mergeable pull request. A reviewer that reports findings as annotations on a check run still shows them on the changed lines, but they are not conversations, so nothing about them has to be resolved and a person's comment is the only thread left to answer. Your count of how many bot comments led to a change is a good first number; how many human comments were resolved without a reply in the same month is a good second one.