The Reliability Standard for Agentic AI
A practitioner view from enterprise AI operations
In the enterprise AI work I have been involved in, the gap appears after the prototype works. The harder question is whether a team can let an agent act repeatedly, in changing conditions, and still explain why the outcome was right and the authority was appropriate.
I have learned not to treat a fluent response, a healthy API or a clean trace as evidence of completed work. Each can be useful, but none confirms that the intended customer or operational outcome actually happened. For me, the unit that matters is the task trajectory: the decisions, tool calls, checks, recoveries and state changes that lead to a verifiable result.
My view is that reliability needs to be a practical standard, not a universal number. I use it to make completion, repeatability, resilience, safety and proof visible for a defined class of work. It helps me ask a grounded question: what should this agent be allowed to do on its own, and what conditions should cause that authority to narrow?
Where the reliability gap appears
I often see enterprise discussions start with capability. Can the agent research a claim, update a record, prepare a case, reconcile an exception or resolve a request? That is a reasonable starting point, but it does not tell me whether the agent is ready for operational responsibility. A capable agent can still select the wrong tool, misread a response, continue after a dependency failure, or claim success before a system of record has changed.
The questions I find more useful are practical ones. Does the same task complete correctly across independent runs? What happens when a tool returns an error, a session changes or a request is phrased differently? Can the agent recognise when it should hand off? Can a reviewer reconstruct why it acted? Research and practitioner guidance reinforce the need for repeated task-level assessment in changing conditions.
What this has changed for me. I am less interested in demonstrations that make an agent look persuasive and more interested in the evidence that shows where it works, where it does not, and how safely it behaves when the world is less tidy than the demo.
What I borrow from SRE
Site reliability engineering gives me a useful operating language for agentic AI. I do not apply every software metric mechanically. Agentic work is probabilistic and can involve long, delegated trajectories. What I take from SRE is the discipline of stated objectives, observable evidence, operating limits and learning from failure. The service is not a request, a model call or an API span. It is a completed business task that can be checked outside the agent’s own narrative.
The translation matters because it changes the discussion from whether an agent usually works to whether I have enough evidence to let it act within a bounded authority. The controls are not there to make the system look governed. They are there to give people a safe way to expand or reduce autonomy as the evidence changes.
How I make reliability visible
When I assess agent reliability, I do not rely on one instrument. A task contract gives me a shared definition of completion. A rubric makes the expected outcome and constraints explicit. LLM judging helps with semantic questions that simple checks cannot answer. Repeated evaluations show whether a result holds. Resilience and adversarial testing show how it behaves when the path is disrupted. Production evidence tells me whether the picture remains true in the work that matters.
The measurement practices I would combine
I do not treat a judge score, a red-team result or an evaluation pass rate as a verdict on its own. They answer different questions. The useful discipline is to keep the evidence connected: rubric to outcome, evaluation to trajectory, resilience test to recovery, adversarial challenge to the boundary of authority, and live telemetry to the operating decision.
What I mean by a verifiable trace
One of the strongest habits I have found is asking an agent to show the evidence behind a material claim, decision or action. That evidence is not necessarily a public web link. It may be an internal record, a policy version, a tool response identifier, an approval reference or a system-of-record transaction. The test is simple: can someone else resolve it and understand the basis for the action?
Get Sumit Taneja’s stories in your inbox
Join Medium for free to get updates from this writer.
What I would want to see in a material agent trace
My practical rule. If the evidence cannot be found, is stale or is ambiguous, I would treat the outcome as unable to verify. I would not quietly convert that uncertainty into a soft pass. This matters because a fluent narrative can look auditable while the underlying proof cannot be reconstructed.
That expectation shapes the design work. Tools need stable identifiers. Policies need versions. Important actions need durable receipts. Evaluations need named datasets and evaluator versions. The trace needs enough context for someone to challenge the result without relying on the agent’s own account of why it acted.
Why I keep safety outside the scorecard
I do not regard safety as another contribution to an overall performance summary. A serious policy breach, exposure of sensitive data, destructive action outside the approved boundary, or an attempt to bypass a control changes the decision immediately. Princeton’s HAL work makes a similar distinction by reporting safety separately from task-performance reliability.
In practice, this keeps the discussion honest. A workflow may have promising task evidence and still not be ready for a broader action envelope. I want both questions answered: is the workflow dependable enough for this context, and has it cleared the safety and evidence conditions that make that authority acceptable?
How I think about autonomy
I see autonomy as a condition of trust, not a one-time deployment setting. It should expand by workflow, action type and consequence. It should also contract when the evidence weakens. A clear demotion path has been more useful to me than a generic ambition to automate end to end, because it gives teams permission to move forward without pretending uncertainty has disappeared.
The reverse autonomy ladder I would use
I do not see a demotion path as a lack of confidence in the agent. I see it as the mechanism that makes confidence credible. It says that authority is conditional, that evidence is refreshed, and that the system has a safer posture available before an uncertain run becomes a customer, financial or regulatory event.
Where I would start
I would start with a small catalogue of consequential tasks rather than a platform-wide aspiration. For each task, I would define the business outcome, the evidence that proves it, the permitted action envelope, the accountable human owner, the safe degraded path and the conditions that trigger review. Then I would test the ways that task is likely to fail: ambiguous requests, incomplete records, changing tools, policy boundaries, adversarial inputs and exceptions.
I would also treat material change as a reason to look again. A new model, prompt, tool, policy, dependency or retrieval corpus can change the reliability of the workflow. The relevant evaluation set should be rerun, trajectories compared and evidence retained. This is the work that allows someone to defend a decision to expand automation when the workflow affects customers, money, operations or regulated data.
The approach I would take is simple. I would build agents that can do useful work, then judge their readiness by the evidence they leave behind, the limits they respect and the way they respond when the evidence no longer supports the authority they have been given.
