What Matters is How Much Software You Successfully Deliver
If you listen to the conversation around AI in software development, you might think we’re entering a golden age of productivity. Developers are generating code at unprecedented rates. Pull requests are multiplying. Features appear to move from idea to implementation in record time. Yet I’m becoming increasingly convinced that many organizations are measuring the wrong thing.
The metric that matters isn’t how many lines of code AI can generate. It’s how much software gets successfully delivered, adopted by users, and translated into business outcomes. That distinction is becoming harder to ignore.
A recent MIT-led analysis found that AI tools dramatically increased coding activity. In some environments, file edits nearly tripled after AI coding assistants were introduced. Yet much of that increased activity disappeared as projects moved through testing, review, deployment, and adoption. The researchers found that while coding output surged, software releases increased by only about 30 percent. Even more striking, many of those additional releases failed to generate meaningful user engagement or business value.
In other words, AI is accelerating code production faster than it’s accelerating value creation.
That finding aligns with what our team at SONATAFY has seen across the industry. Many leadership teams are celebrating activity metrics because they’re easy to measure. They track commits, pull requests, story points, and development velocity. AI makes all of those numbers look better. A developer who once produced ten units of output can now produce twenty.
Code Generation is Not the Problem
The problem is that software development has never been a code-generation problem. It’s about problem-definition, prioritization, customer-adoption, and ultimately business-value. Code is only one step in a much larger chain.
Imagine a construction company measuring success by the number of bricks delivered to a job site. If the building is never completed, nobody would call that productive. Yet that’s effectively what many organizations are doing with AI-generated code. The danger is that AI creates an illusion of progress. When executives see development activity rising, they naturally assume more value is being created. The dashboards look impressive. The engineering organization appears more productive. Teams are shipping more artifacts into repositories.
Yet customers don’t buy code. Customers buy solutions. A feature that nobody uses has exactly the same business value whether it required ten lines of code or ten thousand.
That’s why some of the latest research on AI productivity is so important. Another study conducted by the research organization METR found that experienced developers working on familiar codebases actually took longer to complete tasks when using AI coding tools. Even more surprising, developers believed they were becoming more productive when the measured results showed the opposite.
That finding highlights a broader challenge. AI can make work feel faster because it reduces the effort required to generate output. However, software engineering involves much more than generation. Developers still need to review code, validate assumptions, test integrations, resolve edge cases, address security concerns, and maintain the resulting systems. As AI-generated code volumes increase, so does the burden of verification.
Organizations that focus solely on code output risk creating what I call synthetic productivity. Activity rises. Repositories grow. Dashboards improve. Yet the underlying business outcomes remain unchanged. The better question isn’t how much code AI generated. The better question is what value that code created.
The companies that benefit most from AI won’t be the ones generating the most code. They’ll be the ones that become disciplined about connecting software development to measurable outcomes. AI has given us the ability to manufacture code at industrial scale. That’s a remarkable achievement. The challenge now is ensuring we don’t confuse production with progress.
History is filled with examples of organizations that optimized the wrong metric. Factories produced inventory nobody wanted. Sales teams generated leads that never converted. Marketing departments celebrated clicks that never became customers. Software development is facing a similar moment.
Steve Taplin
WANT TO LEARN MORE? DOWNLOAD YOUR FREE REPORT.
