When AI Looks Valuable but Proves Nothing

McKinsey's 2025 survey found 36% of C-suite executives saw no revenue change from AI investment. That number gets cited as evidence that AI underperforms. It isn't. It's evidence that most firms deployed AI into processes where no one was counting anything before the deployment began.

The number that wasn't there before

Invoice automation at Ellby tells the story cleanly. Before deploying generative AI on Amazon Bedrock, 40% of invoices required manual validation. After deployment, the failure rate dropped to under 6%. That result convinced no one by accident. Someone was counting invoices before the system changed. The countable unit — invoices processed without human intervention — existed before the AI arrived. The AI moved a number that was already being tracked.

ImaginAb cut 1,750 accounts payable hours per year. Centerfield Media eliminated 20 weeks of AP work annually. Both of those results read as investor-grade evidence for the same reason: the hours existed on a timesheet before anyone touched the software.

Contrast that with a founder who deploys an AI writing tool across the marketing team to "improve content quality." Quality is not a number. Velocity of content production might be, if someone counted pieces per week before deployment. Engagement rate is a number, if baseline data exists. The tool itself is irrelevant to whether the evidence will hold. The process structure is everything.

Where marketing personalization earns its numbers

Kortical ran an eight-week A/B test for a US hotel chain. The AI system predicted which customers to target, which room type to offer, and what discount depth would maximize expected revenue per booking. The test compared AI-driven campaigns against the chain's existing campaign. Revenue increased 56% versus the control. Marketing ROI reached roughly 1,000%. Those numbers survive scrutiny because the test design produced a clean counterfactual — the same firm, the same period, two different approaches.

Luxury Escapes replaced fixed segmentation rules with an AI system that evaluated ten website event signals per new user. Revenue per user lifted 10% against the rule-based control. Transaction value increased 7%. The rule-based control was the countable baseline. Without it, the 10% lift is just a number floating in the air.

I'll be direct about something: I have no patience for AI vendors who sell "productivity improvements" without specifying the denominator. A 40% productivity gain on what? Measured over what period? Against what baseline? That framing is not a measurement problem the founder created. It's a sales problem the vendor handed them, and founders carry it into lender meetings as if it were evidence.

The diagnostic question you need to answer first

Before you choose an AI tool, ask one question: is there a unit of work in this process that someone counted last month? Not estimated. Counted. Invoices processed. AP hours logged. Bookings per campaign. Email time per week.

The Microsoft 365 Copilot randomized trial across 7,137 workers at 66 firms showed time savings clustering in email and document collaboration specifically. Meetings and drafting volume did not shift meaningfully. The tool did not underperform in meetings. Meetings simply lacked a pre-existing countable overhead unit the way email did. The same tool, same deployment, produced evidence in one channel and nothing auditable in another. Process structure, not tool quality, determined what could be proven.

The OECD's firm-level productivity study found that worker-level AI gains of 10-56% on specific tasks do not automatically translate to firm-level productivity. That gap is not a failure of the technology. It's a failure of aggregation — task-level gains stay invisible at the firm level unless the task was already being counted at the firm level.

What this means for the lender conversation

A lender does not need to believe in AI. A lender needs a number from before and a number from after, with a credible explanation of what changed between them. Tipalti's AP cases show month-end close dropping by up to 60% for some clients, with one firm moving from 20 workdays to 8. That is a before-and-after any credit committee reads without translation.

Pick the process with the existing count. Measure it for two weeks before deployment. Measure it again 30 days after. The AI tool matters less than you think. The baseline is the whole argument.