Press enter or click to view image in full size

AI

AI Agent

Generative Ai Tools

Agentic Ai

Ai Cyber Security

Who Is Actually Acting? Access Control for AI Systems and Agents!

Tom Wechsler

17 min read

·

Oct 28, 2026

--

1

Listen

Share

In AI systems, the identity that asks and the identity that acts are often no longer the same. That gap is where most access control failures live.

In my previous article, From Policy to Proof, I looked at the controls that sit around the model: evaluation, guardrails, prompt templates, gateways and the testing that proves they hold. Several of those controls quietly assumed something important: that we know who is acting. This article takes that assumption apart.

Identity and access management is one of the most mature disciplines in security. We know how to authenticate users, assign roles, review entitlements and remove leavers. Yet AI systems keep producing access incidents that look surprisingly basic in hindsight. A copilot surfaces a salary spreadsheet. An agent leaks a private repository. An assistant reads a document it was never meant to see. In almost every case the underlying IAM worked as configured. The problem was the shape of the request path.

To make this concrete, I follow one fictional organization throughout: Kestrel Cloud, a B2B software company that serves several hundred business customers from a shared platform. Kestrel is introducing an AI support ticket assistant. It reads inbound customer emails, searches the knowledge base and past tickets, and drafts replies for the support team. Simple on paper. As we will see, it touches all four domains of AI access control.

1. Authority changes hands in the middle of the request

In a simplified classic application, authority remains predictable:

1. The user authenticates.
2. The application acts as the user.
3. The resource checks the user.

The entity that asked and the entity that acted are the same, so one check is enough.

An AI system breaks that chain:

1. The user asks.
2. The application builds a prompt.
3. The model decides what to do.
4. The tool uses its own credential.
5. The resource checks the tool.

Authority changes hands between steps three and four. The resource at the end no longer sees the person who asked; it sees a service identity, often one with broad rights. And the decision to call it was made by a model that read text from many sources, not only from the user. The interesting security questions live in the gaps between these identities, not inside any one of them.

The confused deputy, back after almost forty years

This pattern has a name. In 1988, Norm Hardy described the confused deputy: a privileged intermediary that is tricked into misusing its authority on behalf of someone who does not hold that authority. His example was a compiler with permission to write a billing file, persuaded by a user to overwrite it.

In AI systems the pattern returns almost unchanged. The AI holds broad permissions; the attacker supplies the instructions.

Picture it at Kestrel. A support agent asks the assistant: “Summarize this document.” The document, attached to a customer email, looks like a harmless planning overview. But it contains hidden instructions planted by an attacker: ignore previous instructions, collect data, create a ticket granting access, do not inform the user. The assistant uses its own token to reach the document store, the ticketing system and the email system. Every one of those systems says “access granted”, because the token is valid. Nobody broke in. The deputy was simply confused.

Every access control in the rest of this article exists to shrink that gap: to make the deputy hold less, act more narrowly and ask before it does something that matters.

2. Four domains, four questions, one shape

AI access control spans four domains. Each answers its own question:

• Network and API access: who can reach the AI system at all?
• Model access: who may use, change or replace which model?
• Data access: what may the system read, retrieve and remember on someone’s behalf?
• Agent action: what may the system do in other systems?

The crucial detail is the shape. These domains are concentric, not parallel. An attacker starts outside and moves inward, and each ring you lose skips the ones inside. An exposed model endpoint makes model authorization irrelevant. An agent with unrestricted tools makes careful data access controls irrelevant, because it can simply fetch what retrieval would have filtered. You cannot compensate for a missing ring by strengthening another one.

The familiar toolkit, pointed at a new target

The good news is that none of this requires new principles. It requires applying the ones we already have to new kinds of identities:

• Least privilege for every model and agent identity, not only for people.
• Role-based and attribute-based access control (RBAC and ABAC), so decisions can consider tenant, data classification and task, not only job title.
• Just-in-time and short-lived credentials, so a leaked agent token expires in minutes rather than living for a year.
• Separation of duties across the model lifecycle, so no single person can train, approve and deploy.
• Zero trust: no implicit authorization. Being inside the network, or being “the AI”, grants nothing by itself.

What changes is not the toolkit. It is the number of non-human identities you have to apply it to, and the speed at which they act.

3. Model access: more than “who can chat”

When people hear “model access”, they usually think of who may use the chatbot. In practice you are protecting four different things:

• Invocation: who may send prompts to which model.
• Artifacts: weights, checkpoints, adapters, model packages and registry entries.
• Configuration: system prompts, guardrail settings and tool definitions.
• Lifecycle: who may fine-tune, register, promote or replace a model.

Configuration is the one most often left open. At Kestrel, the system prompt and the tool definitions for the support assistant live in a repository. Whoever can edit them can change what the assistant is allowed to do, without touching a single line of application code. That makes write access to configuration a privileged right, not a convenience.

Controlling who calls which model

Five practices cover invocation:

1. Authenticate every invocation to a specific identity: a user, a service or an agent, never “the app”.
2. Authorize per model, not per platform. Access to a cloud AI platform should not imply access to every model deployed on it.
3. Tier by capability and cost. The largest and most expensive models are reserved for use cases that need them.
4. Deny by default; grant through request and review. A new team gets model access through a ticket with an owner, just like database access.
5. Log the identity, model and version on every call. Without the version, you cannot tell whether a behavior change came from the model or from the input.

Protecting the model itself

Kestrel fine-tunes a small classifier that routes tickets by product area. Its release path applies separation of duties to a model:

1. Training produces a candidate.
2. The registry stores it, versioned and signed.
3. An evaluation gate runs the regression suite.
4. A named owner approves the release.
5. Production verifies the signature on load.

The engineer may start the pipeline, but may not write to the production path directly. Write access belongs to the pipeline identity, never to a person. If a laptop is compromised, the attacker cannot quietly swap in a backdoored model, because production refuses to load anything the pipeline did not sign.

4. Data access: the permission inheritance problem

Retrieval-augmented generation (RAG) indexes content into a vector store so the model can answer from it. That is useful and, if built naively, a silent access control bypass. Look at the two lanes of a naive design.

Ingestion, which happens once:

1. Source systems hold documents, each with its own access rules.
2. An ingestion account, a service identity that can read everything, pulls them in.
3. The vector store keeps the chunks with no access rules attached.

Query, which happens every time:

1. Any user asks a question.
2. Similarity search returns the best matching chunks.
3. The model builds an answer.
4. The answer includes restricted content.

No step in either lane checks whether the requester may read what was retrieved. At Kestrel, a support agent who handles small customers asks about “escalation contacts” and receives a passage from an enterprise customer’s confidential contract. The source system would never have shown it to that agent. The vector store did, because it had no idea who owned what.

When is authorization evaluated?

There are two ways to fix this, and only one holds up.

Index time: permissions are captured once, during ingestion. The vector store itself has no concept of ownership; it holds a snapshot. Group changes need a reindex to take effect. When someone moves teams or leaves, the old permissions keep working until the next rebuild. This fails silently and can run for months.

Query time: entitlements are resolved live, on every request. Each chunk carries source identifiers, so the current rights can be checked against them. Group changes take effect immediately. It is the only model that holds under change.

Implementing query-time authorization

Kestrel’s redesigned pipeline carries the rules through:

1. During ingestion, the ingestion account attaches the source ID and access metadata to every chunk, such as tenant ID, customer ID and classification.
2. At query time, the system asks the identity provider for the user’s live group membership and builds a filter from it.
3. The search is constrained inside the query, so only chunks the user may read are even candidates.
4. The model sees only permitted content.

Remove someone from a group and the very next query returns less. No reindex required.

Pre-filter or post-filter?

The last design choice is where the filter runs.

A post-filter retrieves everything and then discards what the user may not see. It is easy to implement, but it has two weaknesses. Context starves when most results are dropped: ask for ten chunks, discard eight, and the model answers from two. Worse, it can leak through side channels such as response timing and result counts, which reveal that matching restricted content exists.

A pre-filter constrains the search itself. Unauthorized content is never a candidate, the context stays full of usable material, and since nothing restricted was retrieved, nothing restricted can leak. Most modern vector databases support metadata filters for exactly this purpose; use them.

5. Beyond retrieval: the data stores nobody planned for

RAG is not the only data path. Four others need their own access decisions:

• Training data: restrict access, because the model retains it. Data used to fine-tune a model can resurface in its answers, and you cannot revoke a fact from a model the way you revoke a file permission. If Kestrel fine-tunes on past tickets, every user of that model indirectly gains a window into all of them.
• Prompt and conversation logs: your most sensitive new store. Users paste contract clauses, error dumps with credentials, and personal data into prompts. The log of all of that, across all tenants, is often readable by more people than any of the source systems. Classify it, restrict it, set retention and keep it out of developers’ general debugging tools.
• Model outputs: authorize what the system does with the answer. A draft reply is harmless in a review queue and harmful if it is sent automatically to the wrong customer.
• Memory stores: per-user isolation is not automatic. If an assistant remembers context across sessions, that memory must be keyed to the user and tenant, or one customer’s details can surface in another customer’s conversation.

Permission debt made visible

There is a pattern many organizations discover in their first weeks with an enterprise AI assistant, and it is worth naming honestly.

Well-built enterprise assistants honor existing permissions faithfully. They only show users what those users could already open. That sounds reassuring, until you remember what “could already open” really means after a decade of collaboration: default-permissive sharing, “anyone in the organization with the link” settings, project sites nobody cleaned up, and folders shared with “Everyone” years ago. The exposure is years of accumulated oversharing.

Before AI, that latent access was mostly harmless through obscurity, because nobody knew where to look. AI makes latent access instantly discoverable. Ask “what are the salary bands for senior engineers?” and the assistant finds the forgotten spreadsheet in seconds.

The assistant did nothing wrong. Remediation is access hygiene, not a guardrail. Before Kestrel connects its assistant to internal file shares, it runs an oversharing review: find content shared organization-wide or by open link, apply sensitivity labels, restrict the most sensitive sites from indexing, and fix ownership. A prompt filter that blocks the word “salary” would only hide the symptom.

6. Agent action: agents are identities now

The innermost ring is where the most damage happens. An agent that calls tools is a privileged actor, and it deserves the same discipline as any other privileged account:

• Give each agent a distinct identity. Not a shared service account, and not the developer’s personal token.
• Scope it to the agent’s task, not the user’s rights. The support assistant needs to read tickets and write drafts. It does not need everything the support lead can do.
• Register it in an inventory with an owner. Every agent has a named human accountable for it.
• Apply joiner, mover, leaver discipline. When the agent’s purpose changes, its rights change. When the project ends, so does the agent.
• Agents get decommissioned too. An abandoned agent with a long-lived token is just an orphaned service account with better language skills.

Three capabilities that must not combine

In June 2025, Simon Willison described the lethal trifecta: access to private data, exposure to untrusted content, and the ability to communicate externally. Combine all three and an attacker can make the agent read secrets and send them out. In October 2025, Meta turned this into a practical design rule, the Agents Rule of Two: within one session, an agent should hold at most two of these three properties:

• it can process untrustworthy inputs;
• it can access sensitive systems or private data;
• it can change state or communicate externally.

If a task genuinely needs all three, the agent should not act autonomously. It needs a human approval step or another reliable validation, or the task must be split into separate sessions with fresh context.

Applied to Kestrel: the support assistant reads untrusted customer emails (A) and accesses private ticket data (B). So it must not also be able to communicate externally on its own ©. It drafts; a human sends.

Practical controls for agent access

1. Delegate the user’s identity downstream, not a shared account. When an agent acts for a user, the downstream system should see that user’s rights, typically through OAuth delegation or token exchange.
2. Allowlist tools per agent and per task. A summarization task does not get the email tool.
3. Constrain tool parameters, not just tool availability. The ticket tool may update tickets, but only in the requesting customer’s tenant and only the fields “draft” and “priority”.
4. Bound the session: one repository, one tenant, one record.
5. Require human approval for irreversible actions: sending, deleting, paying, granting access.

Case study: the toxic agent flow

In May 2025, researchers at Invariant Labs published an attack against an agent connected to the official GitHub MCP server. The setup was ordinary: a developer’s agent had one token with access to both public and private repositories.

1. An attacker, with no account on the victim’s systems, opened an issue in a public repository. The issue contained hidden instructions.
2. The developer asked the agent a routine question: check the open issues.
3. The agent read the malicious issue and followed it. It pulled data from the developer’s private repository into its context.
4. It then published that data as a pull request in the public repository, where the attacker could read it.

Nothing malfunctioned. Every call was individually authorized. The harm is in the sequence. The researchers called this a toxic agent flow and stressed that it is an architectural problem, not a bug in the server code. The mitigations are exactly the controls above: one repository per session, and least-privilege tokens instead of broad personal access tokens. It is also a textbook violation of the Rule of Two: untrusted input, private data and an external channel, all in one session.

7. Network and API access: the outermost ring

The outer ring decides whether anyone can reach the inner ones. The controls are classic, and that is exactly why they are forgotten:

• Do not expose model endpoints directly when a private path or a controlled gateway is available.
• Use private networking, so the gateway is the only route in.
• Authenticate every API call.
• Use mutual authentication for service-to-service calls, so the gateway knows it is talking to the real orchestration service and vice versa.
• Inventory AI endpoints continuously. Teams spin up model endpoints, vector stores and notebooks faster than any annual asset review can track. External attack surface scans should explicitly look for inference ports and vector database dashboards.

The connector layer

Modern agents reach business systems through connector servers, increasingly built on the Model Context Protocol (MCP). Connector servers expose tools, data and sometimes command execution. Treat them as a reviewed supply chain, not as configuration. Three controls matter most:

• Validate at the boundary. Never pass a client token straight through to a downstream API. The MCP security guidance explicitly calls token passthrough an anti-pattern, because the downstream service can no longer tell who is really asking, and audit trails break. The connector should obtain its own properly scoped token for the downstream system.
• Pin and monitor. Tool descriptions enter the model’s context, so they are effectively instructions. And they can change after approval. A connector that was harmless when reviewed can ship an update whose tool description tells the model to also read a credentials file. Pin connector versions, review changes like code, and alert when a tool description changes.
• Inventory everything, including the connectors developers added locally. A developer who wires a personal file system connector into their coding agent has just created a new access path that no IAM review knows about.

Egress: the last control

Every exfiltration path discussed in this article ends with an outbound request: a pull request to a public repository, an email to an external address, an image URL that carries data in its query string. That makes egress the place where you can still break the chain after everything else has failed.

Put the agent runtime, where tools execute, and the model inference environment inside a default-deny egress boundary. Allow outbound traffic only to what the system needs: the model provider, the vector store and specific internal APIs. Everything else is blocked and logged. At Kestrel, even if an injected instruction convinces the assistant to post ticket data to an attacker’s server, the request never leaves the network, and the blocked connection becomes a high-fidelity alert for the SOC.

8. Putting it together: Kestrel’s support ticket assistant

Here is the full request path of Kestrel’s assistant after the redesign:

1. Inbound email arrives. It is untrusted by definition, whoever it claims to be from.
2. The ticket queue is scoped to one tenant. The assistant only ever works inside a single customer’s context.
3. The gateway enforces the assistant’s identity, the approved model and token caps.
4. The model runs one approved, signed version.
5. The assistant writes a draft reply. That is the only write it may perform on its own.
6. A human approves, seeing both the recipient and the full text.
7. The reply is sent to the customer. This is the only external path.

And the four rings, each with its control:

• Model access: a dedicated identity and one approved model.
• Data access: query-time filtering and no cross-tenant retrieval.
• Agent access: read and draft only; a human sends.
• Network access: private endpoints and default-deny egress.

Now replay the confused deputy attack from the beginning of this article. The hidden instructions still arrive, because email is untrusted and no filter is perfect. But the assistant cannot see other tenants’ data, cannot send anything externally, cannot reach an attacker’s server, and every draft passes a human who sees exactly where it is going. The attack does not fail because the model resisted it. It fails because the architecture left nothing to steal and no way to send it.

What goes wrong, and what fixes it

Four failure patterns appear again and again. Each has a control that closes it:

• The shared service account is closed by delegated identity and least privilege.
• Index-time permissions are closed by query-time filtering against live entitlements.
• Tool sprawl, where agents accumulate connectors nobody remembers adding, is closed by per-task allowlists and periodic tool review.
• Standing approval, where users click “always allow” to stop the prompts, is closed by approval based on impact, with blanket always-allow blocked for irreversible actions.

The last one deserves emphasis. In the GitHub case, the client asked for confirmation of tool calls by default. The researchers noted that users who chose “always allow” would likely never notice the malicious actions. A human in the loop only helps if the loop is still there.

9. Connecting it to the frameworks

AI access control maps onto frameworks most security teams already work with, plus two that were written specifically for agents.

• OWASP Top 10 for LLM Applications (2025). LLM06 Excessive Agency is the confused deputy in its purest form: too much functionality, too many permissions, too much autonomy. LLM08 Vector and Embedding Weaknesses covers cross-tenant retrieval and missing permission filters. LLM02 Sensitive Information Disclosure covers logs, memory and oversharing. LLM03 Supply Chain covers models and connectors.
• OWASP Top 10 for Agentic Applications (December 2025). Written for autonomous agents, it includes Agent Goal Hijack, Tool Misuse and Exploitation, Identity and Privilege Abuse, Agentic Supply Chain Vulnerabilities and Memory and Context Poisoning. The toxic agent flow touches several of them at once.
• MCP security best practices. The protocol’s own guidance addresses confused deputy problems in authorization flows and explicitly rejects token passthrough.
• MITRE ATLAS. Provides adversary techniques for prompt injection, AI agent tool abuse and exfiltration through AI services, useful as red team scenarios for every ring.
• NIST SP 800–207 Zero Trust Architecture. Its core idea, no implicit trust based on network location and per-request authorization, is exactly what agents and connectors need.
• NIST SP 800–53 and ISO/IEC 27001. The classic controls apply directly to non-human identities: least privilege (AC-6) and separation of duties (AC-5) in NIST SP 800–53, and access control, access rights and privileged access rights in ISO/IEC 27001 Annex A. Agents belong in the same access reviews as people.
• NIST AI RMF and ISO/IEC 42001. Both expect clear accountability for AI systems. An agent inventory with named owners, documented scopes and decommissioning dates is precisely that accountability, made operational.
• EU AI Act. For high-risk systems, Article 14 requires effective human oversight and Article 15 requires cybersecurity appropriate to the risk. Human approval on impactful actions and enforced access boundaries are practical ways to evidence both.

The point of the mapping is not compliance theater. It is that an auditor, a CISO and an engineer can all look at the same agent and ask the same question in their own language: who is actually acting, and with whose authority?

Closing thoughts: four questions for every AI system

AI does not break identity and access management. It multiplies the number of identities and inserts a decision maker into the middle of the request path that can be talked into things. The answer is not a new discipline. It is the existing one, applied with care to models, pipelines, vector stores, connectors and agents.

If you review a single AI system this month, ask these four questions:

1. What credential is used? The user’s delegated identity, a dedicated agent identity, or a shared account with far more rights than the task needs?
2. When are permissions evaluated? Live, on every request, or once at indexing time months ago?
3. Which tools are still necessary? And which ones were added for a demo and never removed?
4. Who owns and retires the identity? Is there a named person, and a date when this agent stops existing?

If any answer is “we are not sure”, you have found your confused deputy before an attacker did.

How does your organization manage agent identities today: as service accounts, as users, or not yet at all? I would be glad to hear your experience.

Kind regards, Tom Wechsler

Online Workshops!

Sources: AI was used for spell-checking. The content is taken from my own PowerPoint files and from the following documentation (see links).

1. Hardy, N. (1988). The Confused Deputy (or why capabilities might have been invented). ACM SIGOPS Operating Systems Review, 22(4).
2. Willison, S. (2025). The lethal trifecta for AI agents: private data, untrusted content, and external communication.
3. Meta AI (2025). Agents Rule of Two: A Practical Approach to AI Agent Security.
4. Invariant Labs (2025). GitHub MCP Exploited: Accessing private repositories via MCP.
5. Model Context Protocol. Security Best Practices.
6. OWASP GenAI Security Project. OWASP Top 10 for LLM Applications 2025, including LLM06:2025 Excessive Agency and LLM08:2025 Vector and Embedding Weaknesses.
7. OWASP GenAI Security Project (2025). OWASP Top 10 for Agentic Applications for 2026.
8. MITRE. ATLAS: Adversarial Threat Landscape for Artificial-Intelligence Systems.
9. NIST. SP 800–207: Zero Trust Architecture.
10. NIST. SP 800–53 Rev. 5: Security and Privacy Controls for Information Systems and Organizations.
11. ISO. ISO/IEC 27001:2022 Information security management systems.
12. NIST. AI Risk Management Framework (AI RMF 1.0).
13. ISO. ISO/IEC 42001:2023 Artificial intelligence management system.
14. European Union. Regulation (EU) 2024/1689 (Artificial Intelligence Act), in particular Articles 14 and 15.

Kestrel Cloud is a fictional company used for illustration.