Three Data Decisions That Make AI Work At Startup Scale

Your AI tool isn't broken. Your data is. Here's what to fix first — entity model, single source of truth, instrumented collection — no data engineer needed.

I once watched a founder spend three weeks debugging why their churn prediction kept misfiring. The model was GPT-4 fine-tuned on eighteen months of customer data. Turned out their CRM called it user_id, their product database called it account_uuid, and their billing system stored the email address in that field instead. The model had learned a fiction. Three weeks. One schema decision, made late.

The entity model is the actual problem

When researchers at arXiv tested how data quality dimensions affect classification performance, they found that completeness and consistency failures degraded predictive accuracy even when the model architecture stayed identical. The model wasn't the variable. The inputs were.

At startup scale this surfaces in a specific way. You name a field user_id in your auth table. Six months later someone on the team builds a reporting view and calls the same concept customer_ref. Neither person is wrong. Both are following local convention. But now you have a model training on six months of data where the same human being appears as two different entities, and it has learned noise instead of behaviour.

Fixing this doesn't require a data engineer. It requires making one decision early: name your core business objects, assign each a stable identifier, and write down what "active user" means somewhere your schema enforces rather than somewhere a Notion doc quietly rots. That's it. The arXiv data-centric AI paper puts the requirement plainly — high-performing systems depend on entity representations that remain stable over time. Stable. Not perfect. Stable.

One authoritative record per object

Your CRM holds a version of the customer. Your product database holds a version. Your billing system holds a third. When those three versions drift — and they always drift — any model trained on customer behaviour is working from a composite that doesn't match any real person.

Pick one system as the authoritative record for each entity type. Your product database owns the user. Your billing system owns the subscription. When you need a joined view, build the join explicitly. Don't let each system accumulate its own shadow copy of the other's data. Startups that skip this step don't notice until they try to answer "which users churned in Q2" and get three different numbers from three different places. At that point the problem isn't analytics. It's that the training data for any model touching churn is built on a contradiction.

Schema overhead is real — it just doesn't cost what the cleanup costs

A fair objection: designing entity schemas before you know which entities matter is premature. The research scope behind this article acknowledges the gap — evidence on the specific iteration cost of early schema work at startup scale is thin.

What the research does show is that the cleanup alternative isn't equivalent. Retroactive cleanup of historical training data means either retraining on corrected records — discarding the signal you accumulated during the messy period — or accepting that a model trained on inconsistent inputs will behave unpredictably on consistent ones. The DQLabs synthesis describes this as a compounding problem: missing values, mislabeled examples, and noisy features distort training in ways that don't stay contained. The cost of early schema discipline is paid once. The cost of deferred cleanup compounds with every training run.

Replace manual entry with instrumented collection

A ScienceDirect study on data entry methods found that manual entry through complex forms produces higher error rates than structured, guided collection. The mechanism matters: a person who sometimes forgets a field isn't making a random error. They're introducing a systematic gap that a model will later treat as a pattern.

The fix isn't better forms. Instrument your product to emit events automatically. Segment and Rudderstack both capture user actions at the source and route them to your warehouse without a person in the loop. The event schema you define there becomes your data collection practice. Treat schema changes like code changes — versioned, reviewed, deployed deliberately.

Airtable as a primary data source for an AI training pipeline is a choice that ages badly. I've watched founders spend weeks cleaning exports from it before a fine-tuning run. Both said they'd have structured collection differently if they'd known. Neither was wrong to use Airtable for operations. Both were wrong to let it become the record of truth for model training.

The ACM data-centric AI article states it directly: future gains in AI performance depend on better data, not more complex networks. Fix the entity model first.