A measurement that can only come back positive isn’t a measurement. It’s theater.
Every data tool promises your AI will get better. The honest test is whether it can tell you when it didn’t.
Last week a head of data told me her team had run an AI-readiness assessment. The score came back high. Everyone in the room nodded. Three weeks later the model went to production and the numbers fell apart.
She asked me what went wrong with the assessment. Nothing went wrong with it. That was the problem. The assessment was built so that it could only come back looking good.
This is the quiet thing about most readiness scores, data-quality dashboards, and “your AI is ready” reports. They are designed to pass. A check that can only return green is not checking anything. It is performing.
The tell: ask what a bad result would look like
Here is a question worth asking any tool, any vendor, any internal dashboard that tells you your data is ready: what does it look like when the answer is no?
If nobody can describe the failing state, there isn’t one. The output was never a measurement. It was a foregone conclusion dressed as one.
A real measurement has to be able to disappoint you. That is the entire point of measuring instead of asserting. When you step on a scale, it can tell you that you gained weight. That capacity to deliver bad news is what makes the good news mean anything.
Get Minchan Chung’s stories in your inbox
Join Medium for free to get updates from this writer.
Most AI-readiness checks have quietly removed that capacity. They score the things that are easy to score well. They avoid the one test that could fail: take a real model, run it on your data before and after you’ve prepared it, and compare the result on a metric your business actually cares about.
Four outcomes, not one
When you measure honestly — same baseline model, same evaluation, before and after — you don’t get one outcome. You get four:
- It improved. The preparation moved the business metric. This is the one everyone hopes for, and it does happen.
- It improved a little. Real but marginal. Worth knowing, because now you can decide if the effort was worth it.
- It didn’t move. The data wasn’t the bottleneck this time. Painful to hear, useful to know.
- It couldn’t run. The data was missing what the model needed. The honest systems stop here and say so instead of inventing an answer.
A tool that only ever produces the first outcome isn’t optimistic. It’s broken in a way that happens to feel good.
This is where the “AI-ready” argument is actually stuck
Everyone is busy defining “AI-ready data” right now. Clean? Governed? Validated? Safe to move? The definitions multiply because they’re all written to be agreed with, not tested.
Here’s the shift that ends the argument: a definition you can run, that can come back negative, beats any definition you can only nod at. The team that owns this category won’t be the one with the cleanest one-liner. It’ll be the one that made readiness something you check on real data, where the check is allowed to fail.
That last part is the whole game. Readiness theater survives because the claim is never forced to run again, and never allowed to lose. Make it a run that can lose, and the theater has nowhere to hide.
What this changes
If you lead a team putting AI into production, you don’t need a better-looking readiness deck. You need a readiness result you’d actually trust — which means one that sometimes tells you no.
So the next time a score says your data is ready, ask the uncomfortable question. Run the same model before and after. Look at whether the metric your business runs on actually moved. Sometimes it will. Sometimes it won’t. A check worth trusting is the one that can tell you the difference.
The point was never to make readiness look better. It was to make it answerable — by a run, on your data, with the freedom to come back negative.
That is the only kind of “ready” worth putting into production.
