AI Productivity vs Business Value: What Leaders Should Measure
When technology makes people more capable, does the organisation become more capable too?
AI is helping people complete work faster. But does that necessarily mean the organisation is becoming more effective?
I've been thinking about this distinction, particularly after observing AI adoption within enterprise engineering teams.
I think there are three things leaders should be measuring:
Productivity: Are people completing comparable work more efficiently?
Delivery: Is that efficiency translating into faster, more reliable end-to-end outcomes?
Business value: Are customers, costs, revenue or risk outcomes actually improving?
They sound closely related, but improving one doesn't necessarily mean we're improving the others.
Understanding the difference may tell us something important about whether AI requires us to redesign our operating models at all.
The stand-up stayed the same
During my work in enterprise banking, I observed AI becoming part of everyday engineering delivery.
Developers were increasingly using AI-assisted tools. Senior engineers continued reviewing code and the organisation was examining how additional productive capacity could be used.
As part of my work across Agile delivery and transformation, I reported on team performance using measures such as throughput, cycle time and predictability.
We could observe improvements in delivery performance. Yet something interested me.
The underlying Agile delivery system remained largely the same.
Teams still held stand-ups, sprint planning sessions, reviews and retrospectives. Portfolio walls continued providing visibility over priorities and dependencies.
What changed was the amount of work engineers could complete and, consequently, what the organisation expected of them.
More backlog work could be taken on. Capacity assumptions were changing. Greater productivity was gradually becoming the new normal.
The fundamental relationship between engineering and the business hadn't changed dramatically.
Which made me wonder: had the organisation transformed, or had it simply become more efficient within an existing operating model?
Perhaps more importantly, did anything actually need to change?
What does workflow redesign really mean?
This question becomes interesting when we look beyond one organisation.
McKinsey's 2026 research on AI found that nearly three-quarters of organisations classified as AI high performers reported fundamentally redesigning workflows, compared with approximately one-quarter of other respondents.
That's an interesting association, although it doesn't establish that redesign caused the better outcomes.
However, what exactly were they redesigning?
Consider a software engineering team.
AI might help developers interpret requirements, generate code, create tests and produce documentation.
The sprint remains the same. The stand-up remains the same.
Yet the activities performed inside that sprint have changed.
This suggests an important distinction.
Changing how individual tasks are performed isn't necessarily the same as changing how work is organised across an enterprise.
Perhaps we've been using the term transformation to describe both.
When faster work doesn't mean faster delivery
Google's DORA research suggests that AI can amplify the existing strengths and weaknesses of an organisation's delivery system.
Imagine two organisations adopting similar AI development tools.
The first has mature engineering practices, automated testing and well-established delivery processes. As developers become faster, the wider system can accommodate the additional work.
The second has manual testing, fragmented approvals and significant dependencies between teams.
Developers become faster there too, but completed work begins accumulating downstream.
Both organisations may report improvements in engineering productivity, yet their overall delivery outcomes could be very different.
How would we recognise the difference?
I'd look beyond individual task efficiency and examine:
Throughput: Is more work actually being completed?
Cycle time: Is work moving faster from commencement to completion?
Predictability: Are teams delivering more reliably?
Deployment frequency and quality: Are valuable changes reaching production without increasing defects or instability?
Work in progress: Is completed development accumulating in testing, integration or approval queues?
Together, these measures can reveal whether AI is improving the delivery system or simply accelerating one part of it.
We should also avoid assuming that AI always makes engineering faster. A 2025 METR study found that experienced developers took longer on certain tasks when using the AI tools available at the time. Later research suggests outcomes may be changing as tools improve, although reliable estimates remain difficult.
The point isn't that AI does or doesn't improve productivity.
It's that the benefits depend on the work, the people performing it and the system surrounding them.
Is the organisation actually creating value?
This is where I think leaders need to look beyond delivery metrics.
An engineering team can increase throughput and reduce cycle time without necessarily improving business performance.
We can become exceptionally efficient at delivering things that ultimately make very little difference.
AI could make that problem worse by allowing us to do it faster.
So I'd separate business value from productivity and delivery.
If the objective is cost reduction, measure actual operating expenditure, including AI licensing, infrastructure and oversight costs.
If it's customer experience, look at resolution times, satisfaction, complaints and service quality.
If it's revenue growth, examine whether AI-enabled improvements contribute to sales, retention or new sources of income.
If the objective is innovation, measure whether new capabilities are being adopted and producing meaningful outcomes.
There's also an important question about capacity.
When engineers report saving hundreds of hours, what happens to those hours?
Are costs actually reduced? Is more valuable work delivered? Or does the organisation simply raise its performance expectations?
Time saved isn't automatically money saved.
Self-reported efficiencies are useful signals, but I'd want to compare them with observed delivery performance and actual business outcomes, accounting for changes in workload, complexity and quality.
This is how we begin connecting productivity improvements to realised enterprise value.
Five questions I'd ask about the operating model
An operating model connects an organisation's strategy with the people, processes, governance and capabilities required to execute it.
AI may change productive capacity without requiring us to redesign every part of that system.
I've been exploring this through five interconnected questions.
1. Strategy and intent: What are we trying to improve?
Efficiency, costs, delivery speed, customer outcomes or entirely new capabilities?
Start with a clear value hypothesis and measurable business outcome. Without that, we may struggle to distinguish successful adoption from simply increased activity.
2. Governance and decision rights: Are decisions keeping pace?
Who determines how AI is used and what risks are acceptable?
Look at decision lead times, approval delays, exceptions and control failures. The objective isn't necessarily fewer controls, but appropriate oversight without unnecessary friction.
3. Delivery systems: Can the organisation absorb the additional capacity?
Follow work through development, testing, integration and production.
If throughput improves but work accumulates elsewhere, the constraint may have moved rather than disappeared.
4. Data and insight: Can we demonstrate the value?
Compare reported productivity improvements with delivery metrics and business outcomes.
If these measures tell different stories, investigate why. The discrepancy may reveal an unseen constraint or a benefit that isn't being captured.
5. People and leadership: How are expectations changing?
Are new capabilities creating opportunities for reinvestment, workforce redesign or different responsibilities?
Look beyond productivity to skills, workload, quality and engagement.
Because greater productivity doesn't necessarily mean a better experience of work.
These five questions aren't a prescription for reorganising every organisation that adopts AI.
They're a way of understanding where change is necessary, where existing practices are sufficient and where potential value might be getting lost.
Perhaps we don't need to change everything
One assumption I'm increasingly questioning is that meaningful AI transformation always requires dramatic organisational redesign.
Sometimes it might.
Perhaps a mature operating model is precisely what allows an organisation to adopt AI effectively without disrupting everything around it.
The stand-up doesn't need to disappear because engineers are using AI.
The portfolio wall doesn't become obsolete because teams can deliver faster.
If those practices still help people coordinate decisions, manage dependencies and deliver meaningful outcomes, why replace them?
If they don't, what exactly needs to change?
I think that's a more useful question than assuming AI requires us to reinvent every established way of working.
AI may increase what people are capable of doing but the organisation still determines what it does with that capability.
Whether that results in better customer outcomes, lower costs, greater innovation or simply higher expectations depends on choices that go well beyond the technology itself.
The research is still evolving and organisations are experiencing very different outcomes.
I'm not convinced there's one answer that applies to everyone.
Following that perhaps the question worth leaving with is this:
When technology makes people more capable, does the organisation become more capable too? Or does it simply become capable of doing more of what it was already doing? Perhaps understanding that difference is where the more interesting work begins.

Comments