The Uplevel stat — 41% higher bug rate with cycle time moving under two minutes — is the receipt I would attach to every acceptance-rate dashboard, because it is the cleanest proof I have seen that gaming the metric is the expensive move, not the harmless one. I run a small engineering team where I build backend APIs, and I learned to treat AI-generated code the way I treat external input to an auth system: it does not cross the trust boundary until a human countersigns it, which is your Phase 2 idea wearing a security costume. The metric I would add to your outcome list: time from AI-suggested change to production incident, since acceptance rate hides deferred bugs. How are teams you have seen drawing the line between verify the diff and verify the design — do the evaluation groups review generated architecture too, or only code?
Two great additions. "Time from AI-suggested change to production incident" is the metric that makes deferred bugs visible — acceptance rate measures the speed of a decision and hides whether the decision was any good, so pairing it with time-to-incident turns it from a vanity number into a real one. Adding it.
On your question — mostly the second, and that's the problem. The teams doing it well draw the line before generation, not at review: the trust matrix (Fix 3 in my sibling piece) decides up front which classes of change AI may touch autonomously and which must go through a human designer. Review-time "verify the diff" is what everyone does; it can check the code matches the design, but it rarely reopens whether the design was right — by then the design is a fait accompli and the reviewer's budget is spent on the diff. Teams that verify the design do it as a separate gate: a short design-intent note at the point AI is about to generate something non-trivial, reviewed before generation. Treating generated architecture as external input that doesn't cross the trust boundary until a human countersigns — your auth-system framing — is the right posture; the trick is countersigning the plan, not just the output.
There is a finer point here - typically engineers check code against design, that’s their influence. However, when AI can now derive code from design, an engineer needs to expand role into Product side and question if the design is right in the first place. Does this make resonate?
Yes, that’s where many engineers and engineering leaders are focusing. In other words, at some point engineering software as a discipline or testing will cease to exist and they all become product engineering. There maybe some specialized disciplines as someone needs to build the AI engines in the first place.
We're a place where coders share, stay up-to-date and grow their careers.
The Uplevel stat — 41% higher bug rate with cycle time moving under two minutes — is the receipt I would attach to every acceptance-rate dashboard, because it is the cleanest proof I have seen that gaming the metric is the expensive move, not the harmless one. I run a small engineering team where I build backend APIs, and I learned to treat AI-generated code the way I treat external input to an auth system: it does not cross the trust boundary until a human countersigns it, which is your Phase 2 idea wearing a security costume. The metric I would add to your outcome list: time from AI-suggested change to production incident, since acceptance rate hides deferred bugs. How are teams you have seen drawing the line between verify the diff and verify the design — do the evaluation groups review generated architecture too, or only code?
Two great additions. "Time from AI-suggested change to production incident" is the metric that makes deferred bugs visible — acceptance rate measures the speed of a decision and hides whether the decision was any good, so pairing it with time-to-incident turns it from a vanity number into a real one. Adding it.
On your question — mostly the second, and that's the problem. The teams doing it well draw the line before generation, not at review: the trust matrix (Fix 3 in my sibling piece) decides up front which classes of change AI may touch autonomously and which must go through a human designer. Review-time "verify the diff" is what everyone does; it can check the code matches the design, but it rarely reopens whether the design was right — by then the design is a fait accompli and the reviewer's budget is spent on the diff. Teams that verify the design do it as a separate gate: a short design-intent note at the point AI is about to generate something non-trivial, reviewed before generation. Treating generated architecture as external input that doesn't cross the trust boundary until a human countersigns — your auth-system framing — is the right posture; the trick is countersigning the plan, not just the output.
drawing the line before generation is the part that sticks. review can check code against a design, but it rarely gets to ask if the design was right.
There is a finer point here - typically engineers check code against design, that’s their influence. However, when AI can now derive code from design, an engineer needs to expand role into Product side and question if the design is right in the first place. Does this make resonate?
That Product-side shift is the interesting part. Once the design itself becomes the input, checking it stops being someone else's job.
Yes, that’s where many engineers and engineering leaders are focusing. In other words, at some point engineering software as a discipline or testing will cease to exist and they all become product engineering. There maybe some specialized disciplines as someone needs to build the AI engines in the first place.