More Data Made You Worse At This

Executives who receive more data without learning to question it make worse decisions than those who receive less and know how to evaluate it.

A CFO at a mid-size retailer once showed me a dashboard with forty-seven metrics on it. He was proud of it. Six months later, the company had expanded into a market the data "confirmed" was ready, and the expansion failed. The data was real. The interpretation was not.

The problem is not the data

Davenport, Harris, and Shapiro documented in 2010 that firms using analytics more aggressively tend to outperform those that don't. Opponents of the judgment argument love this finding. It seems to say: get better tools, get better outcomes. But Davenport's own framing includes a caveat that gets dropped in most executive summaries — managerial judgment about what the data means is a separate, required skill from data access itself. The finding describes a correlation between analytics use and performance. It does not tell you which executives were reading the data correctly.

Kale et al. (2014) make the gap more concrete. Whether analytics helps a decision depends on the decision type and the context, not on whether the data is clean or plentiful. A well-built dashboard does not resolve the question of whether the pattern you're looking at applies to the decision you're actually facing.

What dashboards cannot do

Brent Dykes, writing in Forbes in 2024, puts it plainly: curiosity and skepticism must come before interpretation. You have to want to question the metric before you trust it. No infrastructure investment installs that habit.

Laura Huang at Harvard Business School identifies three checks a leader needs to make before acting on any evidence: internal validity (was this measured correctly?), external validity (does this finding hold outside the context where it was gathered?), and decision fit (is this the right evidence for this specific choice?). Automated anomaly detection handles none of these. A flagged outlier tells you something is unusual. It does not tell you whether the underlying finding transfers to your market, your customer base, or your quarter.

Judea Pearl's work on causality establishes why this matters structurally. A dataset, however clean, cannot tell you whether a relationship in it is causal or coincidental. That determination requires deliberate logical reasoning. It is a skill, not a feature.

The strongest case against this

The tooling-first argument is not stupid. LinkedIn and Velosio both argue that most executive decision failures trace back to bad infrastructure: siloed data, inconsistent definitions, no traceability. Fix the plumbing, the argument goes, and the judgment problem shrinks to something manageable. This is true for a specific class of failures. Organizations with genuinely broken data pipelines do fail for infrastructure reasons.

But infrastructure fixes stop working at the point where the data is good and the interpretation is still wrong. Epstein and Buhovac (HBR, 2010) show that deciding which metrics are relevant to a given decision is itself a judgment call, one that happens before any dashboard interaction begins. Clean data fed into a wrong frame produces a confident wrong answer. That is harder to catch than a missing data point.

What to do before you act on any claim

Rousseau's 2006 case for evidence-based management assumed leaders already knew how to evaluate evidence quality. Most don't, which is the gap the whole framework was supposed to close. [Inference] The practical version of her argument is not "trust evidence over instinct" — it is "learn to interrogate evidence before you trust it."

Before acting on any data claim, ask four things. How was this collected, and by whom? Does the relationship in the data reflect cause or coincidence? Did this finding come from a context similar enough to mine that it transfers? Which metrics here actually bear on the decision I'm making, and which are noise?

These are not complicated questions. Executives who ask them consistently make fewer confident mistakes. Those who skip them and rely on dashboard volume are doing something closer to reading a horoscope printed in a very expensive font.