The prototype is not the product: turning AI experiments into tools people trust
By Ashok Volati
Two years ago, the hard part of AI product work in media was getting anyone to try it. Now it’s the other way around. Anyone with a browser and an afternoon can build something that looks like an AI product. In a large media company, that means a steady stream of prototypes every quarter, built by producers, analysts, and engineers who got tired of waiting for a roadmap slot.
I think that’s good. It also means product management has more work to do, not less. Here is what I’ve learned leading AI products where demos are cheap and trust takes a long time to earn.
1. Treat the prototype as research
When a colleague shows me an AI app they built quickly, I don’t start with “how do we scale this?” I ask what it taught us about the problem. A working prototype is the best user research we’ve ever had. It shows which workflow hurts enough that someone built a tool on their own time.
At my company we started an initiative to take these quickly built apps and harden them into secure internal tools. The rebuild matters less than the interview at the start. Who uses it? What decision does it support? What happens when it’s wrong? Many prototypes turn out to solve a slightly different problem than the builder thought. Finding that out early saves months.
2. Define “wrong” before you define “done”
Traditional software has bugs. AI products have behavior. A model that matches finance receipts to production projects will match some of them incorrectly. So the product questions are how often it’s wrong, by how much, and what each mistake costs.
On one operations project, using AI to match procurement receipts to project records, we spent more time designing for failure than on the happy path. We set confidence thresholds, sent low-confidence matches to a human queue, and made every automated match explainable and reversible. We didn’t get full automation. We got a workflow where people review the hard 20 percent instead of keying in all of it. A finance team will adopt that.
My rule is that no AI feature ships without a written answer to “what does the user see when the model is wrong?”
3. Build governance into the product
Media data is sensitive in ways people underestimate: talent contracts, unreleased schedules, rights windows, freelancer personal information. Earlier in my career I delivered a mobile app for about 9,000 freelancers. Single sign-on and access control were the reason that app could exist at all, so we designed them in from the start.
Get Ashok Volati’s stories in your inbox
Join Medium for free to get updates from this writer.
AI raises the stakes. If you treat security and legal review as a final checkpoint, expect your launch to stall. Design the controls into the product instead. Set clear data boundaries, give users audit logs they can see, and match permissions to how the business already works. People use tools more when they can see the controls and understand them.
4. Measure the workflow
Model accuracy is an engineering metric. In operations, I judge a product by hours saved, errors avoided, and how much faster people make decisions. For every AI initiative, I ask the team to name the workflow metric before anyone picks a model. If we can’t describe the before and after in the user’s terms, we aren’t ready to build.
This also keeps teams honest about cost. A slightly less accurate model that runs reliably inside an existing tool usually beats the best model available if using it means switching to another app.
5. Put AI inside the tools people already use
The fastest AI adoption I’ve seen came from features added to systems people already work in, like their CRM, project tracker, or field service app. Years on enterprise platforms taught me that people pick up a new feature faster when it doesn’t need a new login. AI is the same. Before building a standalone assistant, ask whether the same thing could be one button on a screen they already open every day.
6. Keep the builders close
When a prototype becomes a supported product, bring the original builder onto the product team as a design partner. They know the domain, they already want it to succeed, and they’ll tell you when the hardened version lost something important. Hardening should make a tool safer and more reliable without making it slower or harder to use.
What this means for product managers
AI has made building cheap. Being wrong, insecure, or ignored costs as much as it always did. Product managers earn their keep by deciding which experiments should become products, planning for the model being wrong, and measuring the results the business cares about.
A prototype shows that something can work. A product has to work every time. In media, live events start on schedule and audiences notice mistakes, so that difference decides whether a tool gets used.
Ashok Volati leads AI product teams in media and entertainment. Views are his own. Connect on LinkedIn.
