Clean Two Fields, Not Fifty

Your CRM, your accounting tool, and your inbox each hold a different count of the same customer, and none of them agree. The instinct is to fix all of it. That instinct is the problem.

Why broad cleanup makes predictions worse

Krishnan and colleagues ran controlled experiments across multiple datasets and found something that should make any small business owner pause: cleaning sometimes depresses model accuracy when the cleaning strategy doesn't match the error type. Not neutral. Worse. The researchers weren't testing bad cleaning — they were testing reasonable cleaning applied to the wrong things.

The IJSRD survey of machine learning experiments sharpens this. Removing duplicates and smoothing out cosmetic inconsistencies produced little measurable effect on model accuracy. Fixing missing values and label errors in structurally important fields produced sizable gains. The effort is the same. The outcome is not.

This is the part most cleanup advice skips. It treats a dataset as a uniform surface where every dirty cell costs you equally. It doesn't. The cells your model reads cost you. The rest are noise in a different sense — they're noise in your cleanup effort.

The 25% floor

The IJIRMPS paper on data quality and machine learning performance gives the clearest number in this literature: training on low-quality data in controlled experiments produced accuracy drops exceeding 25%. That's not a rounding error. That's a model that has learned the wrong thing from bad inputs and will keep getting it wrong until the inputs change.

The same paper identifies which fields drive that drop. Accuracy, completeness, and consistency in the columns the model reads — those are the operative dimensions. Not the whole dataset. Not every column with a typo. The columns feeding the prediction.

For a customer contact dataset, those columns are usually email address, phone number, and whatever field you use to segment customers for targeting. For inventory, they're product identifier and stock level. Fix those. Leave the rest.

What free tools can actually do

Google Sheets works for datasets under roughly 10,000 rows. For a customer contact list or an inventory table at small business scale, that covers most cases. The FIND function locates blank cells in a column. Conditional formatting flags format inconsistencies — phone numbers with country codes mixed in with local formats, emails missing the @ symbol. These are not sophisticated operations. They are sufficient for the task.

OpenRefine is free, runs locally on your machine, and does one thing Google Sheets does not: it clusters near-duplicate values and lets you collapse them with a single click. If your product identifier column contains "SKU-1042," "sku1042," and "SKU 1042" as three separate entries, OpenRefine's text clustering will surface all three and let you standardize them in about thirty seconds. That matters because a model treating those three strings as distinct products is learning from a fiction.

A short Python script using pandas adds one capability neither tool handles cleanly: automated format validation across every row. You write the rule once — phone numbers must be ten digits, emails must contain exactly one @ and a domain — and the script flags every violation. You don't need to scan visually. The script does it in seconds for a file of any size Google Sheets would choke on.

The argument against this approach, stated fairly

The real objection to free-tool cleanup isn't about the tools. It's about the operator. A small business owner working in a spreadsheet has no reliable way to know whether their edits align with how the model reads those fields. Standardize a phone number format the wrong way, and you erase a distinction the model was using. Remove values you think are missing, and you delete what the model treated as informative absences. Each of those actions looks like cleaning. Neither of them is.

Krishnan et al. supply the mechanism here: when cleaning strategy misaligns with error type or model behavior, performance degrades. A non-engineer making edits in Google Sheets has no guarantee their strategy aligns.

This is a real concern. It doesn't resolve cleanly. But the relevant comparison is not between free-tool targeted cleaning and professional data engineering. It's between free-tool targeted cleaning and an attempted full overhaul by the same non-engineer with no engineering support. The IJSRD paper shows that the errors dominating a full overhaul attempt — duplicates, cosmetic inconsistencies — have little effect on model accuracy. The ActiveClean VLDB paper shows that prioritizing records most likely to affect the model produces up to 2.5x accuracy improvement per record cleaned compared to uniform cleaning. Restricting scope to two fields doesn't eliminate the risk of a bad edit. It eliminates the additional risk of bad edits in columns that wouldn't have helped anyway.

[Inference: no source in the research directly measures error introduction rates by operator skill level using free tools at small-business scale. The argument that restricted scope reduces compounding risk is logically sound given the evidence but not directly tested.]

Start with one column

The ActiveClean framework, published at VLDB, was built around a specific insight: clean the records most likely to be dirty and most likely to affect the model first. The researchers showed this approach improves accuracy by up to 2.5x for the same volume of records cleaned, compared to cleaning uniformly across the dataset. You don't need the full ActiveClean system to apply the underlying logic. You need to know which column your model reads most heavily, and start there.

Open your dataset. Find the column your AI feature reads first — the one used for segmentation, matching, or prediction. Run a blank-cell count. Check format consistency. Fix those. Export. Retrain or refresh. Check whether predictions shifted.

That sequence, run on one column, will tell you more about your data's actual condition than a full audit of every field ever would.