Why Your AI Pilot Stalled Before It Reached Anyone

The model works. The accuracy numbers look reasonable. The demo ran cleanly. Then the pilot ends and nothing changes in the business. This is the most common shape of AI failure, and it has almost nothing to do with the model.

The accuracy trap

RAND Corporation spent five months in 2023 interviewing 65 experienced AI practitioners across industry and academia about why AI projects fail. The leading root cause they identified was not bad data or weak infrastructure. It was leadership-driven misunderstanding of the problem and the metrics used to measure it. That ranked ahead of data quality, ahead of infrastructure gaps, ahead of every technical explanation founders reach for first.

Sculley and co-authors documented a related pattern in their 2015 NIPS paper on hidden technical debt in machine learning systems. Their finding was that ML systems accumulate failure risk not in model code but in the surrounding system: data dependencies no one declared, feedback loops no one mapped, consumers of model output no one knew existed, boundary erosion between components. None of these failures show up in accuracy metrics. A model scores well on your evaluation set and still delivers nothing because the surrounding system was never designed to carry value from output to decision.

These two sources arrive at the same place from different angles. The failure is structural and organisational. Founders who treat it as technical will spend time on the wrong fix.

The data problem is real — it's just not where the chain breaks

Here is the legitimate objection: for an early-stage company, the data pipeline often genuinely does not work. RAND's practitioners ranked "lack of adequate data" as the second root cause of AI project failure. Sculley et al. document data dependencies, unstable input signals, and entanglement as concrete engineering failures requiring engineering fixes. A founder who sees a model producing garbage outputs and traces it to a broken data pipeline is not wrong.

The problem is what happens after the pipeline gets fixed. Sculley et al.'s most important observation is not about missing data. It is about undeclared data dependencies and hidden feedback loops — situations where a system consumes data from sources no one explicitly chose, or where model outputs feed back into the training signal without anyone noticing. These are not pipeline bugs. They are failures of design and communication. No one mapped what data the system would consume, from where, or what would happen downstream when the model produced an output.

Fix the pipeline and you address one broken link. If no one defined what a working model was supposed to change in the workflow, the pilot stalls at the next link. The RAND practitioners did not rank data problems as primary. They ranked misunderstanding of the problem as primary. That ordering is the finding.

What a causal map actually forces you to do

The exercise is not complicated. Write down the AI task. Write down the input data and where it comes from. Write down the expected output. Then write down the exact decision or action the output is supposed to change, who owns that decision, and what workflow they use to make it.

Most pilots cannot complete that last step. The model produces a score or a prediction or a classification, and no one has specified whose behavior changes as a result, by how much, or how that change connects to a business metric anyone tracks. Sculley et al. call this the undeclared consumer problem: downstream dependencies that were never named, which means they were never designed for, which means they fail silently.

The RAND study frames the same problem from the leadership side. When the people running a pilot do not understand the problem well enough to define a success metric in terms of workflow change, the pilot will never graduate from experiment to production. Not because the model is wrong. Because there is no defined state that counts as right.

The case against cause-and-effect mapping

I find most pre-pilot planning templates useless. They produce documentation that looks thorough and commits to nothing. A six-column spreadsheet listing data sources, owners, and refresh rates does not tell you whether the model output will reach the person who needs to act on it.

The causal map is different from planning documentation because it forces a specific, falsifiable claim: if the model produces output X, person Y will take action Z, and metric W will move. That claim is either true or it is not. You can check it before the pilot runs by asking person Y whether they would act on output X in their current workflow. Often they will tell you they do not own that decision, or the output arrives in a system they do not use, or the metric W is not one their team tracks. The pilot would have stalled there regardless of model performance.

The RAND practitioners who described leadership-driven misunderstanding as the top failure cause were not describing a planning failure. They were describing a failure to ask that falsifiable question before committing resources to a model.

Where this leaves the diagnosis

Sculley et al. published their paper in 2015. The patterns they documented — undeclared consumers, hidden feedback loops, boundary erosion — were present in ML systems then and the RAND interviews from 2023 show they are present now. The failure mode is stable. The fix is not more accurate models.

Start with the output. Ask who acts on it. Ask what system they use to make that decision. Ask whether your model's output reaches that system in a form they can use. If any of those answers are unclear, the pilot will stall at that point regardless of what the accuracy numbers say. That is not a prediction. The RAND practitioners described it as a pattern across real projects. Fix the link, not the model.