AI Code Generation: How Much Can It Really Speed Up Development?

A Microsoft Research, GitHub, and MIT Sloan randomized controlled trial found developers using GitHub Copilot completed a task 55.8% faster than a control group. A METR randomized controlled trial found experienced developers using AI tools took 19% longer than without them.

Both are rigorous RCTs. Both are correct. The reason they reach opposite conclusions is the single most useful thing to understand about ai code generation, because it tells you exactly when to expect a speedup and when to expect the opposite. This is one piece of a broader picture covered in our generative ai for business guide, but it's the piece engineering leaders get wrong most often.

The headline numbers get quoted constantly without the study conditions attached. Those conditions are the whole story.

One study measured a developer writing new code in unfamiliar territory. The other measured a developer modifying code they already knew intimately. Those are different activities, and AI helps enormously with the first while adding overhead to the second.

Three variables consistently determine whether AI code generation speeds you up or slows you down.

The most uncomfortable finding in the METR study isn't the slowdown itself. Before the trial, developers predicted AI would make them 24% faster. Afterward, having actually done the work, they reported feeling 20% faster. The clock said 19% slower.

That's a 39-percentage-point gap between perception and measured reality, from developers assessing their own work on their own codebases. It matters because nearly every corporate claim about AI coding productivity rests on self-reporting or proxy metrics. If your evidence that AI is working is that your team says it's working, you don't yet have evidence.

A National Bureau of Economic Research working paper studying over 100,000 GitHub developers found that autonomous coding agents drove 180% more commits and 50% more projects, but only 30% more actual releases.

Output volume rose far faster than delivered software. The bottleneck moved rather than disappearing: generating code got dramatically cheaper, while reviewing, integrating, testing, and shipping it did not. GitClear's analysis points in the same direction, reporting a fourfold growth in duplicated code blocks alongside a roughly 60% drop in refactoring activity, both signals of volume outpacing maintenance.

Mapping the research onto real scenarios gives a more useful answer than any single headline percentage.

Field studies at Microsoft and Accenture landed at 12% to 22% improvement, well below the 55% lab figure, which is roughly what you'd predict once real codebases and real review cycles enter the picture. That range is a more honest planning number than either headline. Getting closer to the top of it is largely a matter of who is using the tools and on what, which is why GenAI developer skills matter more than tool selection.