Durable Execution for AI Agents: State, Checkpointing, Retries and Human-in-the-Loop
How to build AI agents that survive failures, resume interrupted tasks, handle approvals, and execute reliably in production.
1. Why Do AI Agents Need Durable Execution?
Most AI applications start with a relatively straightforward execution model.
A user sends a message, the application invokes an LLM, the model may call a tool, and the application returns a response.
For simple interactions, this is usually sufficient.
However, agentic applications introduce a different level of complexity.
An agent may execute several model calls, invoke multiple tools, delegate tasks to other agents, wait for external systems, and require human approval before continuing.
Consider an agent responsible for reviewing and processing a business request:
- The user submits a request.
- The agent analyzes the request.
- It retrieves information from internal systems.
- It delegates an analysis task to a specialist agent.
- It generates a proposed action.
- It requests approval from an authorized user.
- After approval, it executes the operation.
- It records the result and completes the task.
This workflow may take seconds, minutes, or even several hours.
Now imagine the application crashes after step five.
What happens to the work already completed?
Does the agent start over? Are previous model calls executed again? Will the user need to submit the request again? Could an external operation be executed twice?
These are not unusual edge cases. They are fundamental engineering problems in long-running AI systems.
A conventional in-memory agent generally keeps its execution state inside the current process. When that process terminates unexpectedly, the state may be lost.
Saving conversation messages to a database does not solve this problem by itself.
Durable Execution is the ability to preserve workflow progress and resume execution from a known point after failures, interruptions, or external events.
It allows us to build agents that are not dependent on one uninterrupted application process.
The objective is not merely to make an agent remember what happened. It is to make the entire execution reliable.
2. State Persistence vs. Durable Execution
These concepts are related, but they are not interchangeable.
State Persistence
State persistence means storing information so that it can be retrieved later.
For an AI application, this might include:
- Conversation history
- User preferences
- Current task status
- Agent configuration
- Intermediate results
- Retrieved documents
- Tool execution results
For example, we can store conversation messages in PostgreSQL and load them when a user returns to the application.
This provides conversation continuity.
However, storing messages does not automatically tell us where an interrupted workflow should resume.
Durable Execution
Durable execution goes further.
It records the execution progress and the results of completed steps so that the workflow can continue without repeating all previous work.
Consider this example:
Agent Workflow
|
v
Retrieve Context
|
v
LLM Analysis
|
v
Execute Tool
|
v
Generate Report
|
X Application CrashesWithout durable execution, the application may need to restart the entire workflow.
With checkpointing:
Agent Workflow
|
v
Retrieve Context ✓ Checkpoint
|
v
LLM Analysis ✓ Checkpoint
|
v
Execute Tool ✓ Checkpoint
|
v
Generate Report X Failed
|
v
Application Restarts
|
v
Resume from last
completed checkpoint
|
v
Generate Report
|
v
CompleteThe completed results can be reused while the failed operation is retried.
This distinction becomes particularly important when model calls are expensive or when tools perform operations against external systems.
What about Agent Memory?
Memory and durable execution are also different concepts.
ConceptResponsibilityConversation historyWhat the user and agent discussedShort-term memoryInformation needed during the current interactionLong-term memoryInformation retained across interactionsCheckpointingWhere execution stopped and what has completedDurable executionHow the workflow resumes and continues reliably
For example, an agent may remember a user’s preferences through long-term memory while a durable workflow maintains the execution state of an ongoing approval process.
One does not replace the other.
Reference: LangGraph — Persistence
3. Core Concepts Behind Durable Execution
Before choosing a framework, it helps to understand the underlying engineering primitives.
3.1. Checkpointing
A checkpoint is a persisted representation of workflow state at a specific point in execution.
Depending on the runtime, it may include:
- Completed step identifiers
- Step outputs
- Workflow state
- Execution metadata
- Pending operations
- Current task status
- References to external payloads
When a workflow resumes, the runtime uses these checkpoints to determine which operations have already completed.
Checkpointing is particularly useful when a workflow contains expensive or long-running operations.
However, we should avoid storing unnecessarily large objects in every checkpoint.
For example, instead of persisting a 50 MB document inside workflow state, we can store it in object storage and keep only its identifier and metadata in the checkpoint.
3.2. Replay
Replay is the process of reconstructing workflow execution using its previously recorded history.
Some durable execution engines replay workflow code to rebuild the current execution state.
This introduces an important engineering requirement: deterministic execution.
If the same workflow is replayed with the same history, it should make the same workflow decisions.
Consider:
async function workflow() {
const timestamp = Date.now();
// Continue execution...
}If the workflow is replayed later, Date.now() returns a different value.
This could change its execution path.
Likewise, direct network calls, random values, and database queries should not be performed inside deterministic workflow definitions when the runtime expects replay-safe code.
These operations should be isolated into activities or durable steps.
For example:
async function workflow() {
const timestamp = await getCurrentTimeStep();
// Continue using the recorded result...
}The durable runtime records the step result and reuses it during replay.
This is a foundational concept in systems such as Temporal.
3.3. Durable Steps
A durable workflow is usually divided into independent execution units.
For example:
Workflow
|
+-- Step 1: Retrieve documents
|
+-- Step 2: Generate analysis
|
+-- Step 3: Validate output
|
+-- Step 4: Request approval
|
+-- Step 5: Execute operation
|
+-- Step 6: Save final resultEach completed step produces a checkpoint.
If step five fails, the workflow should not need to repeat document retrieval, model analysis, and validation.
This also allows us to apply different retry policies to different operations.
3.4. Durable Waits
A workflow may need to wait for something that is outside the application’s control.
Examples include:
- Human approval
- An external webhook
- A scheduled execution time
- A payment confirmation
- A response from another agent
- Completion of a long-running background operation
A regular asynchronous function can wait for an event, but keeping the process alive indefinitely is not a scalable approach.
Durable execution systems can suspend the workflow, persist its state, and resume it when the expected event arrives.
The application does not need to keep a worker occupied during the waiting period.
3.5. Retries and Recovery
Failures should be handled according to their type.
A network timeout, provider rate limit, validation error, and authorization failure should not necessarily follow the same retry policy.
A retry policy should define:
- Maximum attempts
- Initial delay
- Backoff strategy
- Maximum delay
- Retryable error categories
- Overall execution deadline
- Cancellation behavior
A durable execution engine provides the infrastructure for retries, but the application still needs to define what should be retried and why.
4. The Most Important Concept: Idempotency
This is one of the most important aspects of building reliable AI agents.
Durable execution can resume workflows and retry failed steps. But retries can introduce duplicate side effects.
Consider an agent that generates a report and publishes it to an external system.
The operation succeeds, but the application crashes before the workflow records the result.
When execution resumes, the workflow may attempt to publish the report again.
Without additional protection, we could create two reports.
This is why idempotency matters.
An operation is idempotent when executing it multiple times with the same logical input produces the same intended outcome as executing it once.
For example, consider a report publishing API:
async function publishReport(input: {
reportId: string;
content: string;
operationId: string;
}) {
const response = await fetch(
'https://api.example.com/reports',
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Idempotency-Key': input.operationId,
},
body: JSON.stringify({
reportId: input.reportId,
content: input.content,
}),
},
);
if (!response.ok) {
throw new Error(
`Report publishing failed: ${response.status}`,
);
}
return response.json();
}The important detail is operationId.
It must be generated and persisted by a trusted part of the application before the external operation is attempted. It should not be an arbitrary identifier generated by the LLM.
The receiving service must also implement idempotency-key handling. Merely sending the header does not guarantee duplicate prevention.
Why isn’t exactly-once execution automatic?
This is a common misunderstanding.
A workflow engine can guarantee that a successfully checkpointed step is not unnecessarily executed again during replay.
However, an external side effect and the workflow’s checkpoint cannot always participate in the same atomic transaction.
There is a failure window:
- An external API accepts the request.
- The external operation succeeds.
- The application crashes.
- The workflow has not recorded the successful result.
- The workflow retries.
This means we should not assume that every external operation executes exactly once.
A more accurate approach is to design for:
- At-least-once execution attempts.
- Idempotent external operations.
- Durable operation identifiers.
- Deduplication at the receiving service.
- Transactional outbox patterns where appropriate.
- Reconciliation for operations whose result is uncertain.
For critical business operations, this distinction is non-negotiable.
A durable workflow is not a substitute for transactional business design.
5. Human-in-the-Loop: When an Agent Needs Approval
Not every operation should be executed automatically.
In enterprise applications, an agent may need authorization before performing an operation that has financial, operational, or security implications.
For example:
- Publishing a report
- Updating customer information
- Executing a database modification
- Approving a business transaction
- Changing production configuration
- Triggering an external workflow
A traditional implementation may look like this:
Agent
|
v
Generate Proposed Action
|
v
Save Pending Approval
|
v
Notify User
|
v
Wait for Response
|
v
User Approves
|
v
Continue AgentThe problem is that the application needs to preserve enough state to resume the original execution.
A durable workflow makes this much easier.
It can:
- Execute the agent until an approval boundary.
- Persist the current state.
- Suspend execution.
- Notify the appropriate user.
- Wait for an approval event.
- Resume from the saved state.
- Continue or abort based on the decision.
The approval process may take seconds, hours, or days.
Importantly, the application should verify the identity and authorization of the person responding. The decision should be bound to the original operation, its parameters, and its expiration policy.
An approval should not be a generic boolean that can be reused for another action.
6. Implementing Durable Agents with Vercel AI SDK
For TypeScript-based applications, the Vercel AI SDK ecosystem provides an interesting integration through WorkflowAgent.
The standard ToolLoopAgent runs its execution loop in memory.
This is suitable for short interactions, but its progress can be lost when the underlying process terminates.
WorkflowAgent, provided through @ai-sdk/workflow, runs the agent inside the Vercel Workflow runtime.
It provides:
- Persistent execution progress
- Independent tool retries
- Recovery after process interruptions
- Human approval that can survive long waits
- Workflow-level execution visibility
- Resumable chat streaming through
WorkflowChatTransport
An important distinction: durability comes from the Workflow runtime, not simply from changing the agent class.
Step 1: Install the packages
pnpm add ai @ai-sdk/workflow workflow zodStep 2: Define durable tool operations
A tool operation can be marked as a durable step using the 'use step' directive.
async function searchDocumentsStep(input: {
query: string;
}) {
'use step';
const response = await fetch(
`https://api.example.com/search?q=${encodeURIComponent(
input.query,
)}`,
);
if (!response.ok) {
throw new Error(
`Document search failed: ${response.status}`,
);
}
return response.json();
}The runtime can persist the result of this step and retry it independently when a retryable failure occurs.
This is particularly useful when the operation interacts with an unreliable external service.
Step 3: Create a WorkflowAgent
import {
WorkflowAgent,
type ModelCallStreamPart,
} from '@ai-sdk/workflow';
import {
convertToModelMessages,
tool,
type UIMessage,
} from 'ai';
import { getWritable } from 'workflow';
import { z } from 'zod';
async function searchDocumentsStep(input: {
query: string;
}) {
'use step';
const response = await fetch(
`https://api.example.com/search?q=${encodeURIComponent(
input.query,
)}`,
);
if (!response.ok) {
throw new Error('Document search failed');
}
return response.json();
}
async function publishReportStep(input: {
reportId: string;
}) {
'use step';
const response = await fetch(
'https://api.example.com/reports/publish',
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(input),
},
);
if (!response.ok) {
throw new Error('Report publishing failed');
}
return response.json();
}
export async function runAssistant(
messages: UIMessage[],
) {
'use workflow';
const agent = new WorkflowAgent({
model: 'anthropic/claude-sonnet-4-6',
instructions: `
You are an enterprise research assistant.
Search internal documentation when needed.
Prepare reports based on verified information.
Never publish a report without approval.
`,
tools: {
searchDocuments: tool({
description:
'Search internal documentation.',
inputSchema: z.object({
query: z.string(),
}),
execute: searchDocumentsStep,
}),
publishReport: tool({
description:
'Publish a prepared report.',
inputSchema: z.object({
reportId: z.string(),
}),
needsApproval: true,
execute: publishReportStep,
}),
},
});
const result = await agent.stream({
messages: convertToModelMessages(messages),
writable: getWritable<ModelCallStreamPart>(),
});
return {
messages: result.messages,
};
}This example demonstrates the core pattern:
- The workflow function is marked with
'use workflow'. - Tool implementations that need durable retries are marked with
'use step'. - The report publishing tool requires human approval.
- The workflow can suspend and resume while waiting for the approval decision.
For a real deployment, the publishing operation should also implement server-controlled idempotency and authorization. The simple example focuses on the workflow integration rather than the complete business security layer.
Step 4: Understand the difference between WorkflowAgent and ToolLoopAgent
FeatureToolLoopAgentWorkflowAgentExecutionIn-memoryDurable workflowProcess restart recoveryNo automatic recoverySupported by runtimeTool retriesApplication-managedPer-step retry supportLong approval waitsRequires custom handlingDurable suspensionTool execution visibilityAI SDK telemetryWorkflow step visibilityStreamingDirect stream consumptionWorkflow writable
The recommended approach is not to replace every ToolLoopAgent with WorkflowAgent.
For short, synchronous agent interactions, an in-memory agent is often simpler.
Use durable execution when the workflow needs to outlive its original request, survive process interruptions, wait for external events, or retry individual operations independently.
Step 5: Make Chat Streaming Resumable
One challenge in long-running AI applications is that the backend workflow and the frontend connection have different lifecycles.
A user might:
- Refresh the browser.
- Lose network connectivity.
- Navigate away from a page.
- Reconnect after a function timeout.
A durable workflow may still be running even though the original HTTP connection has disappeared.
WorkflowChatTransport provides a mechanism for resuming the chat stream using the workflow run ID and previously received stream chunks.
A simplified client configuration:
'use client';
import { useChat } from '@ai-sdk/react';
import { WorkflowChatTransport } from '@ai-sdk/workflow';
import { useMemo } from 'react';
export default function Chat() {
const transport = useMemo(
() =>
new WorkflowChatTransport({
api: '/api/chat',
maxConsecutiveErrors: 5,
initialStartIndex: -50,
}),
[],
);
const { messages, sendMessage } = useChat({
transport,
});
// Render conversation and handle user input.
}The server also needs the corresponding workflow stream endpoints.
This is an important architectural detail: a durable backend does not automatically make the browser connection durable. The frontend needs a way to reconnect and retrieve the appropriate part of the workflow stream.
References: Vercel — WorkflowAgent · AI SDK — WorkflowAgent Migration
7. LangGraph: Checkpointing and Interrupts
LangGraph approaches durable agent execution through graph state and persistence.
Its persistence architecture provides two complementary mechanisms:
Checkpointers
Persist graph state for a specific thread.
They support:
- Conversation continuity
- Recovery from interruptions
- Human-in-the-loop workflows
- Time travel
- Fault tolerance
Stores
Persist application-defined information across different threads.
They are useful for:
- User preferences
- Long-term memory
- Shared knowledge
- Cross-conversation information
A simplified configuration:
import {
MemorySaver,
StateGraph,
Annotation,
} from '@langchain/langgraph';
const State = Annotation.Root({
messages: Annotation<string[]>({
reducer: (current, update) => [
...current,
...update,
],
default: () => [],
}),
});
const builder = new StateGraph(State);
// Add nodes and edges...
const checkpointer = new MemorySaver();
const graph = builder.compile({
checkpointer,
});
const result = await graph.invoke(
{
messages: ['Analyze the requested document.'],
},
{
configurable: {
thread_id: 'analysis-123',
},
},
);This example uses an in-memory checkpointer for illustration.
Get Serif Colakel’s stories in your inbox
Join Medium for free to get updates from this writer.
It is not suitable for surviving process restarts. For production, use a persistent implementation such as PostgreSQL or another supported durable checkpointer.
Human-in-the-loop with interrupts
LangGraph provides the interrupt() function to pause a graph and wait for external input.
Conceptually:
import {
interrupt,
Command,
} from '@langchain/langgraph';
async function approvalNode(state: {
proposedAction: string;
}) {
const decision = interrupt({
action: state.proposedAction,
message: 'Approval is required.',
});
if (!decision.approved) {
return {
status: 'rejected',
};
}
return {
status: 'approved',
};
}The graph persists its state through its checkpointer.
Later, the application can resume the same thread using Command with the user's response.
This is useful for workflows where execution must pause for approval or additional input.
One subtle but important implementation detail: when resuming an interrupted graph, the interrupted node may execute again from its beginning. Operations performed before interrupt() therefore need to be designed carefully to avoid duplicate side effects.
References: LangGraph Persistence · LangGraph Interrupts
8. Temporal: Durable Execution as an Infrastructure Primitive
Temporal takes a different approach.
Rather than focusing specifically on agent state or conversation management, Temporal provides a general-purpose durable execution platform.
It is designed for long-running, stateful workflows that need to survive failures.
Two concepts are particularly important:
Workflow
Defines the orchestration logic and execution history.
Activity
Performs external or non-deterministic operations such as API requests, database access, LLM calls, and file operations.
A simplified AI workflow could look like this:
Temporal Workflow
|
v
Retrieve Context Activity
|
v
LLM Analysis Activity
|
v
Validation Activity
|
v
Wait for Approval Signal
|
v
Execute Business Operation
|
v
CompleteTemporal records workflow history and uses deterministic replay to reconstruct execution state.
External interactions belong in Activities so their results can be recorded and reused during replay.
This separation is fundamental.
An LLM invocation should not be performed directly in a deterministic workflow definition. It should be executed as an Activity.
The same applies to database calls and external HTTP requests.
When does Temporal make sense?
Temporal is particularly relevant when:
- Workflows span multiple services.
- Tasks may run for hours or days.
- Strong recovery requirements exist.
- Human approval is part of the process.
- Many independent workers execute tasks.
- Workflow execution is critical to business operations.
It also introduces additional infrastructure and operational responsibilities.
Teams need to understand workflow history, Activity retries, Worker deployment, workflow versioning, and task queue management.
References: Temporal — Durable AI · Temporal TypeScript SDK · Temporal Workflow Determinism
9. Inngest and DBOS: Other Approaches Worth Evaluating
Temporal and LangGraph are not the only options.
For TypeScript-based applications, two other projects are worth considering.
Inngest
Inngest provides durable functions and agent workflows.
Its execution model is built around steps.
For example:
import { inngest } from './client';
export const researchWorkflow = inngest.createFunction(
{
id: 'research-workflow',
},
async ({ event, step }) => {
const context = await step.run(
'retrieve-context',
async () => {
return retrieveContext(event.data.query);
},
);
const analysis = await step.run(
'generate-analysis',
async () => {
return generateAnalysis(context);
},
);
const report = await step.run(
'validate-report',
async () => {
return validateReport(analysis);
},
);
return report;
},
);The runtime checkpoints completed steps and resumes from the failed step when execution recovers.
Inngest also provides step.waitForEvent() for human approvals and other external events.
Its durable agent approach goes further by treating each model call and tool call as a separately checkpointed step.
This is useful when we want to write ordinary TypeScript functions instead of defining an explicit graph.
DBOS
DBOS provides durable execution for TypeScript and other languages, using a database-backed workflow model.
Its TypeScript SDK offers:
- Workflow registration
- Durable steps
- Configurable retries
- Workflow IDs
- Idempotency and deduplication mechanisms
- Background execution
- Recovery after process restarts
A simplified example:
import { DBOS } from '@dbos-inc/dbos-sdk';
async function retrieveDocuments() {
// External API or database operation
return ['document-1', 'document-2'];
}
async function analyzeDocuments() {
// LLM interaction
return {
summary: 'Analysis completed',
};
}
async function researchWorkflow() {
const documents = await DBOS.runStep(
() => retrieveDocuments(),
{
name: 'retrieve-documents',
retriesAllowed: true,
maxAttempts: 3,
},
);
const analysis = await DBOS.runStep(
() => analyzeDocuments(),
{
name: 'analyze-documents',
retriesAllowed: true,
maxAttempts: 3,
},
);
return {
documents,
analysis,
};
}
const workflow = DBOS.registerWorkflow(
researchWorkflow,
);This example shows the basic step model. A production application must initialize DBOS, register the workflow, and configure its database and worker lifecycle.
DBOS also requires deterministic workflow code and appropriate serialization of workflow inputs and outputs.
How do these approaches compare?
TechnologyMain strengthBest suited forLangGraphGraph state and checkpointingStateful agent workflowsTemporalGeneral-purpose durable executionComplex distributed business processesInngestDurable TypeScript functions and agentsEvent-driven agent workflowsDBOSDatabase-backed durable executionTypeScript applications needing durable stepsVercel Workflow + WorkflowAgentNative AI SDK integrationDurable agents in Vercel-based applications
There is no universally correct choice.
The right decision depends on the execution model, deployment environment, workflow duration, infrastructure constraints, and the level of operational control required.
References: Inngest Durable Agents · DBOS TypeScript Workflows
10. Designing a Production-Ready Agent Workflow
Choosing a durable execution framework is only one part of the solution.
We also need to design the workflow itself.
Let’s consider a document analysis and approval workflow.
The system should:
- Receive a document.
- Extract its content.
- Analyze the information.
- Generate a proposed report.
- Wait for human approval.
- Publish the approved report.
- Store the final result.
A possible architecture:
User
|
v
AI Application
|
v
Workflow Runtime
|
+-------+--------+
| |
v v
Extract Step Analysis Step
| |
v v
Storage LLM / Tools
| |
+--------+-------+
|
v
Validation Step
|
v
Approval Required
|
+------+------+
| |
Reject Approve
| |
v v
Stop Publish Step
|
v
Save ResultSeveral design decisions matter here.
Persist the right information
Do not put every document, model response, or binary object into the workflow’s event history.
Use references to external storage for large payloads.
Persist only the data required to recover the next step.
Separate planning from execution
The LLM may decide which operation should happen next, but the runtime should control which operations are actually executed.
The agent should not own unrestricted workflow persistence or authorization decisions.
Define explicit failure boundaries
A document extraction failure should not necessarily require the analysis to restart.
An approval timeout should not cause the document to be processed again.
A publishing failure should not require repeating the original LLM request.
Each operation should have a clear recovery strategy.
Keep business state authoritative
A workflow checkpoint records execution progress.
It should not replace the authoritative transaction state of the business system.
For example, an approval workflow may persist its state in the orchestration engine, but the actual transaction should be committed and recorded in the relevant business system.
Make workflows version-aware
Long-running workflows can remain active across application deployments.
A workflow that started using one version of the code may resume after a newer version has been deployed.
Changing step ordering, workflow logic, or serialized state formats can break recovery if compatibility is not considered.
Workflow versioning, migration strategies, and backward-compatible payloads should be part of the deployment design.
11. Common Mistakes in Durable Agent Implementations
There are a few mistakes I would specifically avoid.
1. Treating conversation history as workflow state
Saving messages does not mean the agent can resume from the exact point where it stopped.
2. Retrying everything
A failed tool should not force all previous model calls and tool executions to run again.
Retry boundaries should be explicit.
3. Assuming exactly-once side effects
A durable workflow cannot automatically guarantee exactly-once behavior across arbitrary external services.
Use idempotency keys, deduplication, and reconciliation.
4. Storing large payloads inside checkpoints
Store references to documents and large artifacts instead of repeatedly serializing them.
5. Ignoring workflow versioning
A long-running workflow may outlive multiple application deployments.
Plan how in-flight executions will behave when code changes.
6. Keeping workers alive while waiting for approval
A durable wait should suspend execution and release the worker rather than holding a process open.
7. Using durable execution for every interaction
A simple question-answering endpoint does not necessarily need a workflow engine.
The added infrastructure should solve an actual reliability problem.
8. Confusing agent reasoning with business authorization
An agent deciding to perform an action does not mean that action is authorized.
Sensitive operations must be validated and approved by trusted application logic.
12. What Can an AI Chapter Lead Contribute?
I believe durable execution is particularly important when multiple teams are building AI-powered features within the same organization.
Without shared patterns, each team may implement its own persistence, retry logic, background execution, and approval mechanisms.
This creates duplicated engineering effort and inconsistent reliability.
There are several practical areas where an AI Chapter Lead can contribute.
1. Establish a Common Durable Execution Pattern
Define when an agent should run in memory and when it should run inside a durable workflow.
For example:
Use caseRecommended approachSimple chat responseIn-memory agentShort tool interactionStandard agent with controlled retriesMulti-step background taskDurable workflowHuman approvalDurable wait and resumeLong-running researchDurable agentCross-service business processWorkflow engineIndependent subagent tasksDurable child workflows or delegated executions
This prevents unnecessary infrastructure while ensuring that critical workflows have appropriate recovery mechanisms.
2. Build a Shared Workflow Abstraction
We can create a reusable internal package that provides:
- Workflow lifecycle management
- Checkpoint persistence
- Standard retry policies
- Idempotency helpers
- Cancellation support
- Human approval mechanisms
- Execution state management
- Observability integration
Teams should not need to implement these capabilities independently.
However, shared abstractions should not hide framework-specific behavior that affects reliability.
3. Define Standard Retry and Recovery Policies
Not every failure should trigger another attempt.
For example:
FailureStrategyTemporary network failureRetry with backoffProvider rate limitRespect retry guidance and budgetInvalid model outputValidate and optionally regenerateAuthorization failureStop and escalateInvalid tool argumentsCorrect or request clarificationBusiness rule violationStop or request human reviewExternal operation timeoutReconcile before retrying
We can define common retry budgets, deadlines, maximum attempts, and error classification rules.
This reduces inconsistent failure handling across applications.
4. Establish Idempotency Standards
Every operation that creates an external side effect should have an explicit idempotency strategy.
For example:
- Database writes
- Report publishing
- Email sending
- External API requests
- Business transaction execution
- Agent-to-agent task submission
This is especially important when workflows involve retries or uncertain execution outcomes.
5. Standardize Human Approval
We can establish a shared approval model that defines:
- Who can approve an action.
- Which operations require approval.
- How approval requests are stored.
- How decisions are associated with workflow executions.
- What happens when approval expires.
- How rejected actions are handled.
- How approval decisions are audited.
This makes human-in-the-loop workflows consistent across teams.
6. Build a Reference Implementation
A practical internal example should demonstrate:
- Durable agent execution.
- Step-level checkpointing.
- Independent retries.
- Human approval.
- Resume after application restart.
- Idempotent external operations.
- Distributed tracing.
- Workflow versioning.
This is more useful than a standalone demo that only shows a successful agent response.
7. Measure Reliability
We should define measurable outcomes.
Useful metrics include:
- Workflow completion rate
- Recovery success rate
- Number of repeated model calls
- Duplicate side-effect attempts
- Approval wait duration
- Retry count per workflow
- Average workflow duration
- Cost per successful workflow
- Human escalation rate
The objective should be to understand whether durable execution actually improves reliability without creating unnecessary complexity.
13. When Should We Adopt Durable Execution?
A durable execution framework is particularly valuable when:
- A task involves multiple dependent operations.
- Execution can exceed HTTP request timeouts.
- A workflow must survive process restarts.
- Expensive model calls should not be repeated unnecessarily.
- A human may need to approve an operation later.
- The agent interacts with external systems that may fail independently.
- Multiple workers or services participate in one business process.
For a short chat response, a regular AI SDK agent may be sufficient.
For a multi-hour research workflow or a business operation that cannot be safely restarted from the beginning, durable execution becomes much more important.
The correct architecture is the simplest one that provides the required failure guarantees.
Conclusion
AI agents are becoming more capable, but greater capability introduces additional operational complexity.
An agent that can call tools, delegate tasks, and interact with external systems must also be able to recover from failures, preserve execution progress, and handle interruptions safely.
Durable execution provides the foundation for this.
Checkpointing preserves progress. Retries recover from transient failures. Idempotency prevents duplicate side effects. Durable waits enable human approval without keeping workers occupied. Workflow versioning allows long-running executions to survive application changes.
For AI engineering teams, the goal should not be to make every agent use a workflow engine.
It should be to identify which operations require durability and apply the right execution model to them.
My recommendation: Start with one multi-step workflow that includes an external API call and a human approval step. Implement it with a durable execution framework, simulate process failures, and verify that completed work is not repeated and external side effects are not duplicated.
A production-ready agent is not one that only works when everything goes right.
It is one that knows how to recover when things go wrong.
References and Further Reading
Vercel AI SDK and Workflow
- Vercel — What is WorkflowAgent? — Durable AI agents, retries, approvals, and resumable execution.
- Vercel — Build AI Agents with AI SDK and AI Gateway — AI SDK and Workflow integration.
- Vercel — Workflow SDK — Durable functions, steps, and execution model.
- Vercel — Migrate from DurableAgent to WorkflowAgent — AI SDK v7 integration changes.
LangGraph
- LangGraph — Persistence — Checkpointers, stores, state, and memory.
- LangGraph — Interrupts — Human-in-the-loop and resumable execution.
- LangGraph — Fault Tolerance — Recovery and execution reliability.
Temporal
- Temporal — Durable AI — Durable execution for AI applications.
- Temporal — TypeScript SDK — Workflows, Activities, Workers, and message passing.
- Temporal — Workflow Determinism — Replay-safe workflow implementation.
- Temporal — AI Cookbook — AI integrations, agent loops, tool calling, and human approval.
Inngest
- Inngest — Durable Execution — Steps, retries, checkpoints, and event handling.
- Inngest — Durable Agents — Agent loops, independent retries, and subagent delegation.
- Inngest — Human-in-the-Loop — Durable approval and resume patterns.
DBOS
- DBOS — TypeScript Workflows — Durable workflows and automatic recovery.
- DBOS — Workflows and Steps — Retry configuration and step execution.
- DBOS — Architecture — Checkpointing, recovery, and deterministic execution.
