AI Transformation: What the Data Actually Says

By 2024, McKinsey's annual State of AI survey reported that more than 70% of large enterprises had deployed at least one GenAI use case — up from under 20% in 2022. On its face, this looks like a transformation wave.

But adoption is not impact. A separate strand of empirical work — most bluntly summarised in work associated with the MIT Initiative on the Digital Economy's 2025 analysis of GenAI in the enterprise, sometimes called the GenAI Divide — finds that roughly 95% of enterprise GenAI pilots show no statistically measurable impact on P&L outcomes over the periods studied. Both statistics are real. They coexist because piloting a chatbot on a SharePoint corpus and transforming firm economics are very different activities. Executives who read the McKinsey adoption headline without the MIT impact caveat are being told half the story.

McKinsey has run its State of AI report annually since 2017. Key facts about what the survey does and does not tell you:

• It surveys self-selected respondents — senior leaders at large firms who have time to answer McKinsey surveys. It is not a random sample of companies.
• The headline metric is adoption ("has your company used AI in at least one business function?"), not impact ("has AI moved a P&L metric?").
• McKinsey's 2024 edition reported around 65–72% adoption (the band shifts by how functions are counted), and noted that companies reporting the most AI use were also more likely to report cost reduction — but the causal arrow is hard to establish from a survey.
• McKinsey does hedge: the same report notes that less than 30% of respondents said AI had generated "significant" revenue lift, and fewer still could attribute it to a bottom-line number.

The survey is a useful directional read. It is not causal evidence. Use it for market context; do not use it to justify a business case.

MIT researchers affiliated with the Initiative on the Digital Economy documented what they called the GenAI Divide: a gap between companies that have scaled AI to measurable business outcomes and the much larger group that has accumulated pilots with no P&L trace.

The central finding, frequently cited as the "95% no P&L impact" figure, refers to the share of enterprise GenAI initiatives that, when tracked through to financial reporting, show no statistically detectable effect on the firm's margin or revenue within the measurement window (typically 6–18 months). The finding aligns with the broader IT productivity paradox literature (Brynjolfsson 1993, revisited): technology diffuses broadly before firms reorganise workflows and incentives enough to capture the value. The lag between deployment and measurable impact in past waves (ERP, internet, cloud) ranged from roughly five to fifteen years at the sector level.

This does not mean AI has no value. It means most firms are still in the reorganisation lag — or burning budget on AI theatre rather than workflow redesign.

The empirical record shows AI value concentrating in narrow, function-specific use cases — not enterprise-wide transformation. The best-replicated findings come from field experiments in specific workflows:

• Customer support: Brynjolfsson, Li, and Raymond's 2023 NBER field study of 5,179 customer-support agents found roughly 14% average productivity improvement, with the largest gains (around 34%) for the least experienced quartile. The task was narrow: answering inbound software support tickets.
• Knowledge work / consulting: BCG and Harvard (Dell'Acqua et al., 2023) ran a randomised experiment with 758 Boston Consulting Group consultants on structured tasks. AI-assisted consultants performed about 12% faster and 25% higher quality on tasks within the model's capability boundary — but worse on tasks outside it (the "jagged frontier").
• Code completion: Cui et al. (2024) showed roughly +26% completed pull requests across 4,867 enterprise developers using GitHub Copilot.

Notice the pattern: every positive result is task-specific, narrow, and measured in a controlled setting. None of the evidence supports "transform the entire enterprise overnight."

The pilot-to-P&L chain and where most enterprises stall.

The following table contrasts commonly heard claims against what the empirical record actually supports:

The discipline is treating every AI claim as a hypothesis requiring a measurement plan — not a forecast requiring a budget.

A pilot succeeds when it demonstrates feasibility in a controlled setting with motivated users and curated data. Production fails when the same system meets adversarial users, messy data, ambiguous edge cases, and legacy integrations that nobody documented.

The gap has predictable failure modes:

1. Data quality decay. Pilot data was cleaned for the demo. Production data wasn't.
2. Scope creep at scale. The use case that worked for 10 analysts needs to handle 10,000 with different workflows.
3. Oversight bottleneck. Human-in-the-loop designs that were feasible at pilot scale become paralysing at production throughput.
4. Incentive misalignment. Teams that ran the pilot got rewarded for shipping. Teams that inherit production maintenance weren't in the room.
5. Benchmark ≠ task. The model scored well on an internal benchmark. The benchmark didn't measure what the job actually requires.

Organisations that understand these failure modes design pilots with production in mind from day one — smaller scope, real data, explicit measurement, and a pre-committed kill criterion.

Erik Brynjolfsson's 1993 paper The Productivity Paradox of Information Technology asked why the PC revolution wasn't showing up in productivity statistics. The answer — worked out over the following decade — was that the gains were real but lagged by years of complementary investment: process redesign, skill development, and organisational restructuring.

The same dynamics appear to be operating with GenAI. MIT's 2025 analysis is consistent with the view that most firms are still in the pre-reorganisation phase — deploying AI on top of existing processes rather than rebuilding processes to take advantage of AI's actual capabilities.

The BCG/Harvard jagged frontier study adds a second layer: even where the technology helps, it doesn't help uniformly. Firms that win will be those that identify which tasks sit inside the frontier (and deploy AI there with measurement) and which tasks sit outside (and apply critical judgment rather than automation). This is a harder management problem than "buy the best model."

The pattern across the credible studies is consistent:

• Value is real but narrow. It concentrates in well-defined, repetitive, measurable tasks — not vague enterprise-wide mandates.
• Speed of adoption is a weak signal. Fast adoption scores in McKinsey surveys do not predict P&L outcomes; quality of workflow redesign does.
• The pilot era is over. Organisations with more than 3 years of AI investment that cannot point to a single measured P&L line are likely in the pilot trap, not the transformation journey.
• Measurement is the unlock. The firms that show up in the positive evidence base — Brynjolfsson's call-centre study, the BCG experiment — all ran controlled measurements. Measurement is not a post-hoc audit; it is the strategy.

The uncomfortable executive question is not "are we investing in AI?" It is "can we name a business outcome that changed, with a counterfactual?"

This is not an argument against AI investment. It is an argument for calibrated ambition.

The evidence supports:

• Short-term (0–18 months): Expect measurable gains in 2–4 specific, narrow workflows where you can run a clean measurement. Target the Brynjolfsson-type tasks: high volume, repetitive, clear quality signal.
• Medium-term (18 months–3 years): Expect the productivity paradox lag. Invest in data quality, change management, and workflow redesign. These are the complementary assets that turn AI deployment into AI value.
• Long-term (3+ years): If the complementary investments land, expect sector-level recomposition — some roles redefined, some cost structures restructured. The BCG jagged frontier will shift, but firms that locked in early measurement habits will see it first.

Organisations that skip directly to "long-term transformation" narratives without locking in short-term measurement are burning capital on hope.