AI coding tools increase output but stall progress during code review
Article Summary
A Harvard study of 718 firms utilizing the Jellyfish platform found that while AI coding tools increase code volume, they do not proportionally boost completed work due to downstream bottlenecks. Researchers observed that while AI agents increased resolved Jira issues by 12%, code review times rose by 49% and the share of pull requests requiring changes doubled. These findings indicate that the increased workload in code reviews and complementary tasks limits the productivity gains from generative AI in software development.
AI Analysis
## Core Assessment
The Harvard study confirms what many of us have seen in the trenches: generating code is trivial, but shipping software remains a human-gated bottleneck. While AI agents boosted code volume by up to 30%, the 49% increase in review times and the doubling of pull requests requiring rework demonstrate that we are merely shifting the labor burden from "writing" to "policing." By treating lines of code as the primary unit of productivity, many firms are ignoring the reality that verification—not generation—is the primary constraint in the software lifecycle. Unless the team’s ability to review and maintain that code scales proportionally, AI adoption effectively trades developer speed for increased technical debt and operational drag.
## Impact and Advice
* **Engineering Managers:** Stop measuring productivity by commit volume or lines of code. Shift your KPIs to "Mean Time to Review" and "Change Failure Rate." If your team is buried in PR feedback, you have a tooling or process deficit, not a developer output problem.
* **Senior Developers/Leads:** Update your code review standards. If AI-generated code is increasing review latency, mandate stricter linting, automated testing requirements, or smaller, atomic PRs to reduce the cognitive load on reviewers.
* **CTOs/Architecture Leads:** Evaluate whether your current deployment pipeline can handle a 20-30% increase in code throughput. If your downstream testing and QA are already manual or slow, AI agents will likely collapse your release velocity rather than accelerate it.
## Contrarian View
Some may argue that the 12% increase in resolved Jira issues—while modest compared to the code volume hike—represents a legitimate gain in velocity that will improve as LLM capabilities mature. It is possible that the current "review bottleneck" is merely a temporary transitional phase while engineers learn to trust and refine AI output, rather than a permanent structural constraint.
## Bottom Line
AI tools have successfully automated the act of typing, but they haven't yet mastered the art of being correct; we're just spending the time we saved on coding to clean up the mess we just created.
The Harvard study confirms what many of us have seen in the trenches: generating code is trivial, but shipping software remains a human-gated bottleneck. While AI agents boosted code volume by up to 30%, the 49% increase in review times and the doubling of pull requests requiring rework demonstrate that we are merely shifting the labor burden from "writing" to "policing." By treating lines of code as the primary unit of productivity, many firms are ignoring the reality that verification—not generation—is the primary constraint in the software lifecycle. Unless the team’s ability to review and maintain that code scales proportionally, AI adoption effectively trades developer speed for increased technical debt and operational drag.
## Impact and Advice
* **Engineering Managers:** Stop measuring productivity by commit volume or lines of code. Shift your KPIs to "Mean Time to Review" and "Change Failure Rate." If your team is buried in PR feedback, you have a tooling or process deficit, not a developer output problem.
* **Senior Developers/Leads:** Update your code review standards. If AI-generated code is increasing review latency, mandate stricter linting, automated testing requirements, or smaller, atomic PRs to reduce the cognitive load on reviewers.
* **CTOs/Architecture Leads:** Evaluate whether your current deployment pipeline can handle a 20-30% increase in code throughput. If your downstream testing and QA are already manual or slow, AI agents will likely collapse your release velocity rather than accelerate it.
## Contrarian View
Some may argue that the 12% increase in resolved Jira issues—while modest compared to the code volume hike—represents a legitimate gain in velocity that will improve as LLM capabilities mature. It is possible that the current "review bottleneck" is merely a temporary transitional phase while engineers learn to trust and refine AI output, rather than a permanent structural constraint.
## Bottom Line
AI tools have successfully automated the act of typing, but they haven't yet mastered the art of being correct; we're just spending the time we saved on coding to clean up the mess we just created.