Why Your AI Reports Conflict with Your Spreadsheets

Your CRM, your accounting tool, and your revenue dashboard each hold a different count of the same customers, and none of them agree. You check the underlying data. Nothing is corrupted. No rows are missing. The models are working exactly as designed. The problem is upstream of the model entirely: nobody in your organisation ever agreed on what "active customer" means, so each tool resolved the question on its own terms.

This is not a technical malfunction. It is a predictable outcome of asking a system to compute an answer to a question with no agreed meaning inside the business.

The mechanism is semantic, not computational

Research on AI-based data quality assurance for business intelligence identifies consistency as one of the core criteria for reliable BI output, and links semantic reconciliation directly to whether outputs across sessions and tools produce comparable results. The 2025 study on AI-driven BI pipeline automation reinforces this: upstream pipeline structure determines downstream analytical reliability. A pipeline where term interpretation varies by tool or by query session is structurally inconsistent regardless of how well each individual query executes.

This is worth sitting with. The AI is not guessing. It is computing correctly against the definition it was given, or the definition it inferred. The failure is not in the computation. It is in the absence of a shared definition that all tools inherit.

When asking the AI works just as well as defining the term first

The strongest counterargument comes from research on natural language interfaces to databases. The text-to-SQL ambiguity framework distinguishes "cooperative queries," where user and system jointly specify what a query means, from queries too underspecified to yield a valid result. The NaLIR system demonstrates this in practice: it converts an ambiguous natural language question into a structured interpretation, surfaces that interpretation to the user in plain language, and waits for confirmation before executing. Under this model, a founder who types "show me active customers this quarter" and receives a clarifying question is not in a failure state. The system is doing what it was built to do.

For a single session, this works. The problem is what accumulates across sessions.

NaLIR resolves ambiguity at the moment of query execution. It does not record which interpretation was selected, enforce it on subsequent queries, or make it visible to a different team member running the same question two weeks later. Each session starts fresh. Each clarification is local to that interaction. Two queries run by two people on different days, each receiving a cooperative clarification but different clarifications, produce numbers that look equivalent but measure different things. The AI data quality assurance research identifies this precisely as a consistency failure: semantic drift accumulates across queries, and no subsequent clarification corrects the historical record.

What to define before you deploy

The checklist below addresses the three terms founders most often leave undefined. For each one, the question is not what the industry definition is. The question is what your organisation means, written down, enforced in the data model, and visible to every tool querying it.

Active customer: Decide whether this means a customer who purchased within a specific window, logged in within a specific window, or holds an active subscription. Pick one. Record the window length. Apply it uniformly across your CRM, billing system, and any AI reporting layer sitting above them.

Profit margin: Specify which costs are included. Gross margin excludes operating expenses. Net margin includes them. An AI tool asked about "profit margin" with no further specification will resolve this differently depending on the schema it reads. The resolution needs to happen in your data model, not in the query.

Lead quality: This term is the loosest of the three. Research on lead scoring shows that AI models trained on historical conversion labels handle messy, partially defined metrics through statistical learning, which is a genuine partial exception to the governance requirement. It applies to ML models trained on labelled outcomes. It does not apply to a natural language query asking your reporting tool which leads are worth calling this week. For that query, you need a written definition tied to observable attributes: company size, source channel, engagement score threshold.

The data governance and semantic layer literature is consistent on one point: consistency requires governed definitions embedded in the data model, not policy statements in a shared document. A definition written in Notion and a definition enforced in a semantic layer are not the same thing. One is a note. The other is an instruction the tools actually follow.