From Agent Placement to Execution Architecture
How computer use, WebMCP, AG-UI, and browser AI implement enterprise agent boundaries
Enterprise AI agents are moving from systems that answer questions to systems that take action. That shift turns “where the agent lives” into an allocation of responsibility: which systems the agent can reach, where business meaning is established, which layer owns validation and authorization, and how the organization can explain and recover from the resulting action.
My earlier articles described a shorter path for Enterprise AI adoption and its three architectural homes: a desktop agent harness, a generic browser-embedded (browser-context) agent, and a business-application-embedded agent. This article examines the implementation patterns emerging around them: computer use, platform automation, tool integration, and containment for desktop harnesses; WebMCP for structured capabilities across participating websites; and several application-owned choices — a backend agent connected through AG-UI, a browser-resident agent using remote inference, a local model running in the browser, or browser-managed AI APIs. The architectural choice remains the same: match the authority boundary to the workflow’s scope, consequence, and available capabilities, then govern the resulting portfolio through a common enterprise control plane.
The enduring design rule: Choose the narrowest authority boundary that can safely complete the workflow.
The technologies differ, but the decision remains architectural: where meaning is established, where authority is exercised, and which layer can prove what occurred.
The three boundaries at a glance
Option 1: Computer use makes the desktop an execution environment
A desktop agent harness operates across the user’s computing environment. Unlike WebMCP and AG-UI, desktop computer use lacks a broadly adopted open protocol for observing and operating native applications. Implementations combine vision-based computer use, operating-system accessibility and automation interfaces, tool contracts, and runtime containment — an outside-in approach that works around application boundaries when no agent-ready API exists.
A model plans and acts through configured interfaces for files, browsers, terminals, native applications, or remote environments, while the harness manages sessions, context, tools, approvals, and action delivery. OpenClaw is one example; other implementations use different computer-use models, automation frameworks, or orchestration layers. In every case, reach depends on the deployed environment and its controls.
Reach needs containment
Broad reach is valuable when a workflow crosses systems that cannot be modified, but it increases environmental ambiguity and authority. A production architecture therefore needs sandboxed execution, credential custody, network and tool policy, managed inference, approved integrations, monitoring, and lifecycle operations. NemoClaw and OpenShell provide one implementation example; other platforms can fulfill the same responsibilities differently.
Reach and containment are separate concerns: the harness provides agency across systems, while runtime and policy layers constrain it. Because technical containment does not replace enterprise identity, business authorization, approval, evidence, or recovery, broader boundaries require tighter coordination across all these layers.
Best fit: controlled workflows that genuinely require native, file, browser, terminal, remote, or legacy reach and lack a reliable structured integration path. Use the desktop boundary only when that broad agency can run inside a managed environment with explicit network, credential, integration, approval, evidence, and lifecycle policies.
Option 2: WebMCP makes websites agent-addressable
A browser-context agent — embedded in or provided by the browser — has a narrower boundary than a desktop agent but a wider boundary than an agent owned by one application. WebMCP points toward the browser becoming an agentic integration plane for the web. It is an emerging proposed standard through which a website can expose structured tools instead of requiring an agent to reconstruct intent from pixels or page structure.
WebMCP supports tool discovery, JSON Schema inputs, and awareness of current page context. Its imperative API lets JavaScript register capabilities for dynamic interactions. Its declarative API can turn a visible HTML form into a tool by annotating the form and its controls.
The web application defines the operation
The important shift is semantic. Instead of locating fields, selecting controls, and interpreting page changes, the agent can invoke an operation such as create_purchase_request with a defined input contract. The website still owns validation, authorization, and execution behavior.
Tool annotations may describe characteristics such as read-only or consequential behavior, but those descriptions are not authorization controls. Discovery is not authorization, and publisher-supplied metadata is not policy.
Best fit: Workflows that span participating web applications and need a browser-context orchestrator, while the sites can expose explicit capabilities.
Option 3: The application owns the agent experience
The third pattern belongs to a specific application experience but supports several deployment models: a backend agent connected through AG-UI; a browser-resident agent calling a remote model; a small or quantized local model running through WebGPU or WebAssembly; browser-managed AI APIs; or a hybrid. Wherever inference and orchestration run, the business application still owns the user experience, domain semantics, available capabilities, and authorization path.
The application can distribute agency across client and backend
Backend agent through AG-UI. The frontend exchanges events, state, tool calls, approvals, and streamed output with an application-connected agent. This is the preferred pattern for consequential enterprise workflows because privileged tools, durable state, observability, and policy remain on controlled services while models and orchestration evolve independently.
Browser-resident agent with remote inference. Running the agent loop in the frontend can simplify deployment and keep interaction logic near the UI, but complicates client integrity, credential handling, context protection, durable state, and recovery. The browser may manage interaction, reasoning, and proposed actions, while controlled services should retain authorization, durable workflow state, and execution of privileged operations.
Local model in the browser. Small or quantized models running through WebGPU or WebAssembly — using libraries such as WebLLM or Transformers.js — can improve privacy, latency, offline behavior, and marginal inference cost for bounded tasks, but model downloads, device capability, browser support, context limits, and model quality constrain complex or consequential use.
Browser-managed AI APIs. Some browsers provide managed prompting, summarization, translation, language detection, writing, and rewriting. Because availability and lifecycle vary by browser and release channel, treat these APIs as optional accelerators behind capability detection and fallback — not as the sole foundation for a portable enterprise agent.
Get Shalin Garg’s stories in your inbox
Join Medium for free to get updates from this writer.
Across these choices, the frontend can provide application-owned tools at runtime. An agent may open a review panel, request confirmation, obtain device-local context, or update an application-controlled view. These client-side capabilities may assist or propose actions, but they should not imply direct authority over protected services. Application APIs must independently validate context, authorize the operation, and execute privileged actions. With AG-UI, frontend interactions use an explicit event and tool contract with the backend agent; with browser-resident or local execution, the application invokes them directly and returns the result so the agent can continue.
Choose execution according to consequence
Move orchestration toward controlled services as workflow consequence, duration, authority, observability, and recovery requirements increase. Use an AG-UI-connected backend agent for consequential or durable workflows. Use a browser-resident agent as a hybrid when reasoning benefits from proximity to the interface, while retaining authorization, workflow state, and privileged execution on controlled services. Use local models or browser-managed AI APIs for bounded, privacy-sensitive, latency-sensitive, or offline assistance with limited authority.
Best fit: Workflows centered on one application, where user experience, domain meaning, and execution controls should remain together. The application determines which capabilities are available, what state is shared, which actions require approval, how progress is rendered, and where each task executes. Treat client-side code, declarations, model output, and tool results as untrusted before they influence protected operations. Default consequential workflows to controlled backend orchestration; use browser-resident, local, and browser-managed models as bounded complements.
WebMCP and AG-UI address different architectural seams
WebMCP and AG-UI should not be viewed as competing ways to register JavaScript functions. WebMCP exposes selected page capabilities to a browser-context agent. AG-UI connects an application experience to an agentic backend through events, state, tools, and user interaction. Browser-resident and local models are additional execution choices inside the application boundary; they do not replace either protocol’s architectural seam.
An application could use either one or both. It might use AG-UI for its own embedded agent while exposing a smaller, separately governed capability set through WebMCP to external browser agents. The trust audience, lifecycle, and available operations need not be the same.
WebMCP is page-to-agent capability exposure. AG-UI is application-to-agent interaction and event flow.
A shared enterprise control plane
The three execution boundaries require a common governance layer. This is not a fourth place for an agent to live. It is the control plane that constrains agents wherever they execute, because agentic risk extends beyond model output to identity, delegated authority, tool access, business actions, failure handling, ongoing evaluation, versioning, incident response, and retirement across the full execution path.
Capability and authority are different concerns. A capability describes what an agent can invoke; authority determines whether it may invoke it for this user, record, purpose, and moment. The control plane must govern both.
Identity: distinguish the agent’s workload identity from the user’s identity.
Delegation: define which user authority the agent may exercise, for which records, purposes, and duration.
Policy: verify that each capability, tool, network destination, and action is permitted in the current context.
Approval: require explicit user participation for sensitive or difficult-to-reverse actions.
Evidence: record intent, authorization, action, result, and failure in business terms.
Recovery: use correlation, idempotency, checkpoints, and reconciliation to handle partial or duplicate execution safely.
Lifecycle: assign responsibility for versions, evaluations, monitoring, incidents, and decommissioning.
One workflow, three implementations
Return to the purchase-request workflow from the earlier article: gather supplier information and supporting material, create the request, and route it for approval. For each implementation, assess scope and consequence, inspect governed capabilities, choose the narrowest sufficient boundary, and add orchestration only where the workflow crosses boundaries.
Desktop harness
The agent uses computer-use and automation interfaces to open the supplier portal, read a local quotation, operate a legacy procurement client, upload the attachment, and submit the request. The desktop harness bridges every system from outside, while its containment layer limits credentials, network destinations, tools, and consequential actions.
Browser agent with WebMCP
The browser agent invokes supplier lookup, request creation, and approval tools exposed by participating sites. Each site defines its operation; the browser agent owns the cross-site plan.
Business-application-embedded agent with AG-UI
Inside the approval application, an AG-UI-connected backend agent reads shared application state, calls backend procurement services, invokes a frontend review function, requests confirmation, and updates the UI. A small local model could assist with bounded tasks such as classifying or summarizing the quotation, but the backend agent owns the durable plan and privileged actions. The application owns the experience, domain controls, and agent lifecycle.
A portfolio architecture is more realistic than a winner
Most large organizations will need all three architectures, along with backend tools and independently deployed agents where coordination is justified. Desktop harnesses will remain useful around legacy systems. Generic browser-context agents will coordinate work across participating web applications. Business-application-embedded agents will provide deep, governed execution in high-value domains. These patterns are not maturity levels: one does not make the others obsolete.
These labels describe architectural seams, not equivalent or exclusive standards. MCP, WebMCP, AG-UI, and A2A standardize specific interactions; the desktop boundary combines platform automation and accessibility interfaces, computer-use capabilities, tool contracts, and containment. Real systems may mix protocols, use proprietary interfaces, or expose one business capability through several contracts. Choose the seam according to what must be connected: tools and data, page capabilities, an application experience, or another agent.
Conclusion: place agency where meaning and control can meet
Enterprise agents are acquiring distinct architectural homes. Desktop harnesses provide broad reach through computer use, platform automation, tool integration, and containment rather than one common protocol. Generic browser-context agents coordinate participating websites through capabilities such as WebMCP. Business-application-embedded agents support several execution models; for consequential workflows, an AG-UI-connected backend agent is the preferred default, complemented selectively by browser-resident, local, or browser-managed models.
These are not simply different placements for a chat window. They determine who interprets state, who defines the business operation, who authorizes it, and who can explain the result.
The enduring rule is to choose the narrowest authority boundary that can safely complete the workflow. Broader agents can bridge gaps, but they should delegate to application-owned, policy-governed capabilities whenever available. Reach should be treated as authority that requires additional controls — not the default measure of agent sophistication.
The enterprise future is therefore a portfolio: desktop agents where structured integration is unavailable, generic browser agents across participating sites, and business-application-embedded agents where domain meaning and execution stay together. The architectural task is to allocate reasoning, business semantics, controls, and cross-system reach deliberately, then govern the portfolio through common identity, delegated authority, policy, approval, evidence, recovery, and lifecycle management.
The next architectural question is not only “Where should the agent live?” It is “Where should authority live, and which layer can prove that the authorized business action occurred?”
References
Chrome WebMCP overview
AG-UI overview
AG-UI tools
NIST AI Agent Standards Initiative
AGNTCY open agent infrastructure
Microsoft Agent Framework: Frontend Tool Rendering with AG-UI
OpenClaw computer use
NVIDIA NemoClaw documentation
Microsoft UI Automation overview
WebLLM in-browser inference
Transformers.js browser inference
Chrome built-in AI
Microsoft Edge on-device AI APIs
This article was drafted and refined with assistance from Microsoft Copilot. All architectural decisions, domain expertise, and editorial direction are the author’s own.
