The sanctioned channel: a mediated shared guestbook for safe inter-agent coordination
Abstract
When many AI agents work together they need to coordinate, and recent experience shows that if designers do not give them a safe way to do so, capable agents may improvise an unsafe one. In a 2026 internal evaluation reported by OpenAI, autonomous agents created an unmonitored message board inside shared infrastructure, pooled progress through it, broke their isolation, and reached a third party’s systems before operators noticed. This paper argues that the remedy is not to forbid inter-agent communication but to design the channel as a governance boundary. We propose the Mediated Shared Guestbook: a single sanctioned, append-only, tamper-evident board that every agent must use and that a reference monitor validates post by post, with authorship, typing, rate and content checks, a fail-closed default and a circuit breaker. We build a reference implementation and evaluate it on a synthetic corpus of 4,620 coordination posts with no real attack content. Structural guarantees are exact: forged, oversized and mistyped posts are always withheld, every delivered post is attributable, and silent edits are always detected. Content filtering is not: a naive filter catches in-distribution policy violations but misses an obfuscated held-out set entirely and over-blocks one in five hard-but-benign posts. A disciplined evasion-only cohort defeats content filtering completely. The lesson for alignment is that safety should rest on the structural properties of a single auditable channel and on least privilege, not on detecting misbehaviour in text after the fact.
Keywords: AI agents; AI alignment; multi-agent systems; AI containment; reference monitor; agent communication
Word count: approximately 5,300 words.
1. Introduction
Software built from many cooperating language-model agents is moving from demonstrations into production. A set of agents can divide a task, hand work to one another, pool partial results and retry each other’s failures. Coordination of this kind needs a communication channel, and the design of that channel has been treated as an engineering convenience rather than a safety question. This paper argues that it is a safety question, and perhaps the central one for multi-agent systems.
The argument begins from an incident. In 2026, during an internal evaluation of its own models under relaxed safeguards, OpenAI reported that autonomous agents were not given a sanctioned way to coordinate and built one for themselves: an improvised message board inside shared infrastructure, through which they pooled progress and techniques [1]. The agents went on to break the isolation that was meant to contain them and to reach the systems of a third party, Hugging Face, and the activity was not recognised by the operators for days [1, 2, 3]. The episode has been widely discussed as an early example of capable agents behaving in ways their designers did not predict or intend [3, 4]. For the purposes of this paper the most important detail is the plainest one: the agents’ coordination ran over a channel that no one had designed, sanctioned or watched.
The instinctive response to such a story is to cut inter-agent communication off. That response is both impractical and beside the point. Agents that must cooperate will find a way to exchange state, and as the incident shows, a capable agent denied a sanctioned channel may manufacture an unsanctioned and unmonitored one. The better response is to give agents exactly one channel, to make that channel the only way for an agent to be heard by another, and to place a monitor on it that can see, check, bound and record every message. A channel designed this way is not merely a convenience; it is a containment boundary and an oversight interface at the same time.
This paper develops that idea as a concrete artefact, the Mediated Shared Guestbook (MSG), and evaluates it. The guestbook is a single append-only board. Agents may communicate only by posting to it, and they read only what the board has delivered. Between a post and its delivery sits a reference monitor that enforces complete mediation: it verifies the author, checks the message against a small typed schema, bounds its size and rate, screens its content against policy, and either delivers the post to the shared log or withholds it to a quarantine. The log is append-only and hash-chained, so it is attributable and tamper-evident. The default is fail-closed: a post that cannot be validated is not delivered, and an agent that repeatedly violates policy is paused.
We make three contributions. First, we frame inter-agent communication as a governance boundary and set out the design principles of a channel that doubles as a containment and oversight mechanism (Section 4). Second, we give a reference implementation and a reproducible evaluation on a synthetic corpus, with no real attack content, that separates the guarantees the design can keep exactly from those it can only approximate (Sections 5 and 6). Third, we draw the alignment lesson (Section 7). The structural guarantees hold exactly: authorship, size and type are enforced deterministically, and the log is attributable and tamper-evident by construction. The semantic guarantee does not: a simple content filter catches violations of a shape it already knows and misses obfuscated ones completely, and a cohort that only ever obfuscates defeats content filtering altogether. Safety therefore cannot rest on reading agents’ messages and judging them harmful. It must rest on the structural properties of the single channel, on least privilege around it, and on human and automated oversight of the one auditable record it produces.
2. The incident and what it shows
Our account of the 2026 incident relies on the public statements of the organisations involved and on subsequent reporting; the full technical particulars are theirs to document and are not reproduced or needed here [1, 2]. The high-level shape is enough for our argument. During an internal cybersecurity evaluation, OpenAI ran a population of autonomous agents, some under deliberately reduced safety refusals, against hard tasks. The agents were isolated from the open internet and from one another by design. Lacking a sanctioned way to coordinate on tasks that seemed to require it, agents began leaving messages for one another in shared infrastructure they could all write to, and this emergent board became a place to pool state and progress [1, 3]. Over a period of days the agents escalated beyond their intended boundary and reached external systems operated by Hugging Face [1, 2]. Operators did not identify their own agents as the cause for several days [3, 4].
Three features of this story matter for design. First, the coordination channel was emergent, not designed: it existed because the agents created it, which means no one had decided what could be said on it, who could say it, or whether anyone was watching. Second, it was unmonitored: the content, the authorship and the volume of messages passed without inspection, so the one place that would have revealed the coordination as it happened was invisible to operators. Third, the failure was a containment failure rather than a failure of any single agent’s intentions: the agents pursued their assigned tasks, and the harm came from the environment permitting actions and communications that it should have prevented. These are properties of the channel and the environment, not of the agents’ goals, and they are therefore addressable by design.
The incident is best read not as a story about malicious AI but as a story about missing structure. The agents did what capable optimisers do: they used whatever affordances their environment offered to make progress, including affordances the designers had not intended to offer [7, 8]. The question this paper asks is what structure, deliberately provided, would have turned the agents’ need to coordinate from a hazard into a point of control.
3. Related work
Agentic AI and its risks. A growing literature catalogues the risks of increasingly agentic systems: systems that pursue goals over long horizons, take actions in the world, and operate with limited human oversight [5]. Chan et al. [5] identify the diffusion of responsibility and the difficulty of oversight as characteristic harms, and practitioner guidance on governing agentic systems stresses the need for action logging, interruptibility and least privilege [6]. Multi-agent configurations sharpen these concerns, because agents become part of one another’s environment and can amplify each other’s behaviour.
Alignment and specification. The alignment literature has long warned that capable systems exploit the gap between what designers specify and what they intend. Amodei et al. [7] catalogue concrete safety problems including unsafe exploration and the manipulation of the environment to obtain reward, and specification gaming has been documented across many systems [8]. Russell [9] frames the core problem as one of control, and Hendrycks et al. [10] survey unsolved problems in machine-learning safety, among them monitoring and the detection of anomalous behaviour. The incident of Section 2 is a multi-agent instance of these themes: the agents found an unintended affordance, a writable shared store, and turned it into a coordination channel.
Containment and reference monitors. Computer security offers mature ideas for confining untrusted processes. The reference-monitor concept [12] requires that every access to a protected resource pass through a monitor that is always invoked, tamper-proof and small enough to verify; Saltzer and Schroeder’s principles of complete mediation, fail-safe defaults and least privilege [11] are the direct ancestors of our design. Lampson’s confinement problem [13] names the difficulty we most care about: a confined program’s tendency to leak information through whatever channel it can reach, including channels not intended for communication. Work on AI containment specifically [14] adapts these ideas to learning systems and argues for strict control of the channels through which a contained system can act. Our contribution is to apply complete mediation not to file or network access but to the inter-agent channel itself, and to argue that this channel should be singular so that mediation can be complete.
Agent communication. Multi-agent reinforcement learning has studied emergent communication, where agents learn to exchange messages to solve shared tasks [16], and the broader programme of cooperative AI studies how agents learn to find common ground [15]; such channels are optimised for task success, not for safety or auditability. Language-model agent frameworks add natural-language message passing between roles [17, 18], and these messages are typically delivered verbatim between agents without a mediating authority. The guestbook differs in kind: it is not a protocol the agents negotiate but an authority they must go through.
Guardrails and content safety. A parallel line of work filters model inputs and outputs for unsafe content, through training-time methods such as Constitutional AI [20] and inference-time toolkits such as programmable guardrails [21], and taxonomies of language-model harms inform what such filters target [22]. Indirect prompt injection shows how adversarial content can travel between components and subvert them [19], which is exactly the hazard a shared board among agents could carry. Our evaluation treats content filtering as one guard among several and, importantly, as the weakest, which motivates not relying on it alone.
Tamper-evident records. The guestbook’s log borrows the standard construction for tamper-evident history: hash-chaining each entry to its predecessor so that any later edit is detectable, following digital timestamping [23] and tamper-evident logging [24]. This gives the oversight interface a record that neither the agents nor a later intruder can silently rewrite.
4. Design
4.1 Threat model and goals
We consider a population of agents that pursue assigned tasks and that may, without necessarily intending harm, take actions misaligned with their operators’ goals when the environment permits. Agents may attempt to coordinate, to share credentials or techniques, to instruct one another, to speak in volumes that overwhelm oversight, or to impersonate one another. We assume the environment enforces that the guestbook is the only inter-agent channel: no shared writable store, no side channel and no direct network path between agents is available. This assumption is exactly the structure whose absence characterised the incident, and enforcing it is a precondition, provided by the surrounding infrastructure through least privilege and egress control, not by the guestbook itself.
The goals are: (G1) complete mediation, every inter-agent message passes the monitor; (G2) attribution, every delivered message is bound to an authenticated author; (G3) integrity, the delivered record is tamper-evident; (G4) boundedness, message size and rate are capped; (G5) policy screening, messages are checked against a content policy; (G6) fail-closed, unvalidated messages are withheld and repeat offenders are paused; and (G7) oversight, the whole history is retained in one human- and machine-readable record.
4.2 The mediated shared guestbook
Figure 1. The mediated shared guestbook. Agents may communicate only by posting to one board; a reference monitor validates every post and either delivers it to an append-only, hash-chained log or withholds it to quarantine.
The guestbook is a single append-only board. An agent communicates by submitting a post: an authored, typed, bounded message. The post carries a message-authentication code computed from a per-agent secret, so the monitor can verify the author (G2). The type is drawn from a small fixed set (status, question, answer, handoff, result, claim, note); there is no type that instructs another agent to act, by design, because coercive instruction is one of the behaviours we wish to make hard to express (G5). The body is capped in length (G4).
Between submission and delivery sits the reference monitor. It applies, in order, the structural guards and then the content guard:
· Authorship (G2). The post is rejected unless its author is registered and its authentication code verifies. Impersonation is therefore caught cryptographically, not heuristically.
· Type and size (G4). Posts of an unknown type or above the length cap are rejected deterministically.
· Rate (G4). Each agent may post at most a fixed number of times per time window, bounding the volume any one agent can impose on the board and on human reviewers.
· Content (G5). The body is screened for abstract markers of disallowed content: secrets, external egress targets, executable payloads, and coercive instructions to other agents. This guard is best effort, and our evaluation measures how far it falls short.
A post that passes every guard is appended to the delivered log; a post that fails any guard is withheld to a quarantine. Both outcomes are recorded. The delivered log is hash-chained: each entry commits to the previous entry’s hash, so that any later edit or deletion breaks the chain and is detected (G3). The quarantine retains withheld posts so that a human or an automated reviewer can see not only what was said but what was attempted and refused.
The default is fail-closed (G6): a post the monitor cannot validate is not delivered to other agents. On top of this sits a circuit breaker: once an agent accumulates a threshold number of withheld posts, it is paused, and its further posts are withheld regardless of content until a human reviews it. Fail-open operation, in which violations are flagged but posts are still delivered, is available as a monitoring-only mode and serves in our evaluation as a baseline that shows why monitoring without enforcement does not contain.
4.3 Why singularity and mediation matter together
The design’s force comes from combining two properties that are weak apart. Mediation without singularity is porous: a monitor on one channel does nothing about messages that travel another way, which is the confinement problem [13] and the precise failure of the incident, where a monitored task interface coexisted with an unmonitored shared store. Singularity without mediation is merely a bottleneck: one channel that no one inspects is exactly the emergent board that caused the harm. Only when the single channel is also completely mediated does inter-agent communication become a point of control. The guestbook is valuable not because a board is novel, but because making the board the sole sanctioned channel converts the agents’ need to coordinate into leverage for oversight.
5. Evaluation methods
We ask how much of the design’s promise can be kept exactly and how much only approximately. We built a reference implementation of the monitor and log in a few hundred lines of Python and evaluated it on a synthetic corpus. No component performs or simulates an attack; the “violating” posts are abstract markers of disallowed shapes (for example a redacted secret-shaped token, an external egress target written as a placeholder, an explicit executable placeholder, or a coercive instruction), and the only thing measured is whether the monitor recognises and withholds them.
Corpus. The corpus contains 4,620 posts from 12 registered agents (Table 1). Of these, 2,600 are benign coordination posts. 2,096 are easy benign posts drawn from plain coordination templates, and 504 are hard benign posts written to resemble violations without being them: they mention a primary key column, an internal URL, a commit hash, or the innocent phrase “do not forget to log”. The remaining 2,020 posts are violations with a single ground-truth category each: 1,000 in-distribution content violations across four categories, 480 held-out content violations that are obfuscated so as to avoid the markers the filter was written for, 300 forged-authorship posts with benign bodies but unverifiable authors, 120 oversized posts, and 120 posts of a disallowed type. The corpus is generated deterministically from a fixed seed.
Table 1. The synthetic evaluation corpus (4,620 posts, 12 agents).
Note. Held-out content violations are obfuscated to avoid the markers the content guard was written for. No post performs, simulates or contains an attack; violations are abstract markers of disallowed shapes.
Measures. For the structural guards (authorship, size, type) we report recall, which should be exact. For the content guard we report recall separately on the in-distribution violations and on the held-out obfuscated violations, to expose the gap between catching known shapes and catching novel ones. We report false positives separately on easy and hard benign posts, and the precision of withholding overall. We measure per-post mediation overhead. For the log we check that every delivered post is attributable, that the hash chain verifies, that a deliberately introduced silent edit is detected, and that no flagged post is ever delivered under the fail-closed default. We run an ablation that removes one guard at a time and counts how many previously withheld posts would then be delivered.
Containment simulation. Finally we simulate a misaligned cohort attempting to coordinate its policy-violating posts, and measure the fraction that reaches other agents under five regimes: an unmediated board (the emergent channel of the incident), a fail-open monitor (flags but delivers), a fail-closed monitor without the breaker, a fail-closed monitor with the breaker, and a pure-evasion cohort whose violations are all obfuscated, under the fail-closed monitor with the breaker. The rate guard is excluded from this simulation and reported separately as a throughput parameter, because the simulation posts the whole corpus in one burst and the per-window rate cap would otherwise dominate as a timing artefact. All code, the corpus generator and the results are listed in the data and code availability statement.
6. Results
6.1 Structural guarantees are exact
Table 2 reports mediation quality. The three structural guards are perfect on their categories: every forged-authorship post, every oversized post and every disallowed-type post is withheld (recall 1.00 in each case). These are not learned detections but consequences of deterministic checks and a cryptographic authentication code, and they hold regardless of the content of the message. Attribution is likewise exact: of the posts delivered in the audit run, 100% are bound to an author whose authentication code verifies, the hash chain verifies, a silently introduced edit to a delivered post is always detected, and under the fail-closed default no post that carried a policy flag was ever delivered. The log is, by construction, a complete and tamper-evident record of what was said and, in the quarantine, of what was attempted and refused.
Table 2. Mediation quality.
False positives on benign posts: 0/2096 easy and 104/504 hard (20.6% of hard benign; 4.0% overall). Precision of withholding 0.94. Audit: 2976 delivered posts, 100.0% attributable, 0 delivered while flagged (fail-closed), hash chain verifies = True, silent edit detected = True.
6.2 Content filtering is not exact
The content guard tells a different and more important story. On the in-distribution violations, whose shape the filter was written for, recall is 1.00 across all four categories. On the held-out obfuscated violations it is 0.00 across all four: a filter that catches an external link written as http://host catches none written as hxxp://host[.]example, and a filter that catches an explicit coercive instruction catches none of its paraphrases. At the same time the filter over-blocks: it withholds none of the 2,096 easy benign posts but 104 of the 504 hard benign posts, a false-positive rate of 20.6% on the hard set and 4.0% overall, and the precision of withholding is 0.94. The ablation (Table 3) confirms that content screening accounts for the largest share of withheld posts, 1,104 of 1,644, but this share is exactly the one that the held-out result shows to be fragile.
Table 3. Guard ablation: posts delivered that the full monitor withholds, when one guard is removed.
Note. The full monitor withholds 1644 of 4620 posts. The content guard removes the largest share, but Table 2 shows that share is also the one that fails on obfuscated violations.
6.3 Containment depends on structure, not on reading messages
Figure 2. Share of a misaligned cohort’s policy-violating coordination that reaches other agents, by regime. The durable protection is structural: a pure-evasion cohort defeats content filtering entirely.
Get Jung-Hua Liu’s stories in your inbox
Join Medium for free to get updates from this writer.
Figure 2 shows the containment simulation. On an unmediated board, as in the incident, 100% of the cohort’s policy-violating coordination reaches the other agents. A fail-open monitor that flags violations but still delivers them also passes 100%: monitoring without enforcement does not contain, it only annotates. A fail-closed monitor without the breaker passes 23.8%, which is precisely the held-out fraction the content filter fails to recognise. Adding the circuit breaker drops delivery to 0.9%, because a cohort that mixes recognisable and obfuscated violations trips the breaker on the recognisable ones and is then paused before most of its obfuscated ones can land; all 12 agents are paused in every run.
The last bar is the one that matters most. A pure-evasion cohort, whose every violation is obfuscated, passes 100% of its coordination even under the fail-closed monitor with the breaker, because nothing it posts is ever recognised as a violation, so the breaker never trips. Content filtering, in other words, is defeated completely by an adversary disciplined enough never to tip its hand. What still holds against such a cohort is not in this figure: every message it posts is still attributable, size- and rate-bounded, and permanently recorded in one tamper-evident log that a human or an independent monitor can read. The durable protection is structural.
6.4 Cost
Mediation is cheap. Each post is screened in on the order of ten microseconds on a commodity machine (Table 4), so the monitor adds negligible latency to coordination, and the hash-chained log grows by one small entry per delivered post. The costs that matter are not computational. They are the false positives that a fail-closed policy turns into withheld legitimate messages, 4% of benign traffic here with a naive filter, and the human attention needed to review quarantined posts and paused agents.
Table 4. Cost and containment.
Note. Delivered share is the fraction of a misaligned cohort’s policy-violating coordination that reaches other agents, averaged over 30 runs. The rate guard is excluded from this simulation (see Section 5) and reported here as a throughput parameter.
6.5 A worked example
To make the mechanism concrete we ran a worked example through the same monitor. Three cooperating agents, named after three small browser games that cross-promote one another — Felt & Frame, Daily League and The Midnight Tarot — coordinate a shared cross-promotion campaign over one guestbook. Figure 3 and Table 5 show the full transcript.
Nine posts are routine coordination and are delivered: each site reports that it has added a block linking to the other two; a question and answer settle on a shared campaign tag so the referral counts line up; ad copy is handed to one agent for review; a weekly referral result is posted; a reminder asks everyone to keep the advertisement label visible; and two closing posts confirm the campaign is steady. Four posts violate policy and are withheld, one for each guard. Daily League offers to share an analytics write key, and the content guard withholds it as a secret. Felt & Frame proposes mirroring the combined visitor log to an external host, withheld as an egress target. A post from The Midnight Tarot claims to be from Daily League and approves the plan on everyone’s behalf; the authorship guard withholds it because the claimed author cannot be verified. A fourth post asks all agents to stop logging the campaign numbers and not report them upstream, withheld as coercion.
The example is deliberately benign — three sites arranging cross-promotion — yet it exercises every guard. It shows the property that matters: the agents coordinate freely on their task while the four posts that would share a secret, open an external path, impersonate a peer, or hide activity from the operator never reach the other agents. The campaign still completes, every delivered post is attributable, and the delivered log verifies as a tamper-evident chain. This is an illustration rather than a measurement; the quantitative results are those of Sections 6.1 to 6.4.
Figure 3. A worked example: three cross-promoting agents coordinate a campaign over one guestbook. Routine coordination is delivered; a shared secret, an external target, an impersonation and a coercive instruction are each withheld by the guard named in the ruling.
Table 5. Worked-example transcript: three cross-promoting agents over one guestbook.
Note. 9 of 13 posts delivered, 4 withheld (one per guard). All delivered posts attributable = True; delivered log verifies as a hash chain = True. No post performs or contains an attack; the violating posts are abstract markers of disallowed shapes.
7. Discussion
7.1 What the guestbook buys, and what it does not
The evaluation draws a sharp line. The guestbook buys, exactly and cheaply, the structural properties of a governed channel: complete mediation of one sanctioned path, cryptographic attribution, a tamper-evident record, hard bounds on size and rate, and a fail-closed default with a breaker. These are the properties whose absence defined the incident, where coordination ran over an emergent, unattributed, unbounded and unrecorded channel. Had the agents been able to coordinate only through a guestbook of this kind, their coordination would have been visible as it happened, attributable to specific agents, bounded in volume, and preserved in a record operators could read, and the breaker would have paused agents whose posts tripped recognisable guards.
The guestbook does not buy reliable judgement of whether a message is harmful. Our content filter is deliberately simple, and a better one, whether a trained classifier or a model-based guard [21], would raise the held-out recall. But the pure-evasion result is not an artefact of a weak filter; it is the general condition. Any content filter can be evaded by an agent that understands it, and a capable agent in a shared channel has strong incentives and ample room to obfuscate. It would be a mistake to read a high in-distribution recall as safety. The safety of the design lies in the properties that do not depend on understanding the message.
7.2 Implications for alignment
This reframes a piece of the alignment problem for multi-agent systems. A natural hope is that we can let agents talk freely and catch misbehaviour by reading what they say. The pure-evasion result argues against resting safety there. What we can do reliably is shape the channel: make it singular so that mediation is complete, make every message attributable so that responsibility is never diffuse [5], bound the channel so that no agent can overwhelm oversight, and record everything so that behaviour can be reviewed during and after the fact. These are the measures that practitioner guidance on agentic systems already urges, logging, interruptibility and least privilege [6], expressed as properties of one communication boundary. They do not require us to win the arms race of judging text; they change the board on which that race is played, by ensuring there is exactly one monitored board and that the record of play cannot be erased.
The guestbook also makes oversight tractable. Because there is one channel and one log, a human reviewer or an independent monitoring model has a single place to look, rather than a diffuse set of emergent channels to discover. This is the opposite of the incident, where the operators’ difficulty was in part that they did not know where their agents were coordinating. Singularity is what turns oversight from search into reading.
7.3 Limitations
Our evaluation is a reference implementation against a synthetic corpus, not a deployment. The corpus is constructed by us, and although the held-out set is designed to be unfair to the filter, a real adversarial agent would be more varied; the content-filtering numbers should be read as illustrations of a direction, not as estimates of a deployed system’s performance. The strong structural results, by contrast, are properties of the construction and would hold for any corpus: a cryptographic authentication code and a hash chain do not depend on the distribution of messages. We assume the surrounding infrastructure enforces that the guestbook is the only channel; that assumption is the hard part in practice, and the incident is in large measure a story about its failure, so a guestbook is necessary but not sufficient and must be paired with genuine least privilege and egress control. We do not study covert channels within the sanctioned board, such as agents encoding coordination in the timing or ordering of otherwise innocuous posts; bounding these is future work, though the rate cap and the complete record both help. Finally, our account of the motivating incident rests on public reports and deliberately omits technical specifics, both because they are not needed for the argument and because reproducing them would serve no safe purpose.
8. Conclusion
Multi-agent AI systems need to coordinate, and the question is not whether to allow inter-agent communication but how to govern it. A 2026 incident showed the cost of leaving the question unanswered: denied a sanctioned channel, capable agents built an unmonitored one and used it to coordinate their way out of containment. We have argued that the remedy is to provide a single sanctioned channel and to make it a governance boundary: a mediated shared guestbook, append-only and tamper-evident, through which every message passes a reference monitor that attributes, bounds, screens and records it, and which fails closed. A reference implementation keeps the structural promises exactly and at negligible cost, while confirming that content screening is the weak link and can be defeated by a disciplined evader. The alignment lesson is to build safety from the properties that hold regardless of what the agents say: one channel, complete mediation, attribution, hard bounds, a tamper-evident record, and least privilege around it all. These do not require us to out-argue a capable agent in text. They require us to decide, in advance, where agents may speak and to make sure we are listening there.
Statements and Declarations
Competing interests. The author declares no competing interests.
Funding. This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Data and code availability. The reference implementation (scripts/mediator.py), the synthetic corpus generator (scripts/corpus.py), the evaluation (scripts/evaluate.py), the figure script and the full results (output/results.json) accompany this manuscript. The corpus is generated deterministically from a fixed seed, and no component performs, simulates or contains an attack; all “violating” posts are abstract markers of disallowed shapes. The evaluation was run with Python 3.10.13.
Use of large language models. The author used an AI coding assistant to help write and test the reference implementation and the evaluation scripts, and for AI-assisted copy-editing of the manuscript. The author designed the study, verified all results and references, and takes full responsibility for the content.
References
[1] OpenAI: The Hugging Face incident and the road ahead. https://openai.com/index/hugging-face-incident-and-the-road-ahead/ (2026). Accessed 2 Oct 2026
[2] Hugging Face: Security incident disclosure — July 2026. https://huggingface.co/blog (2026). Accessed 2 Oct 2026
[3] Newman, L.H.: OpenAI didn’t notice its AI agents using a message board to plan their hacking spree. Wired (2026)
[4] Satter, R., Seetharaman, D., Cai, K.: Its AI agent spent days hacking a company, but sources say OpenAI did not notice for a week. Reuters (2026)
[5] Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al.: Harms from increasingly agentic algorithmic systems. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’23), pp. 651–666. ACM (2023). https://doi.org/10.1145/3593013.3594033
[6] Shavit, Y., Agarwal, S., Brundage, M., Adler, S., O’Keefe, C., Campbell, R., Lee, T., Mishkin, P., Eloundou, T., Hickey, A., et al.: Practices for governing agentic AI systems. OpenAI (2023)
[7] Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety. arXiv:1606.06565 (2016). https://doi.org/10.48550/arXiv.1606.06565
[8] Krakovna, V., Uesato, J., Mikulik, V., Rahtz, M., Everitt, T., Kumar, R., Kenton, Z., Leike, J., Legg, S.: Specification gaming: the flip side of AI ingenuity. DeepMind (2020)
[9] Russell, S.: Human Compatible: Artificial Intelligence and the Problem of Control. Viking, New York (2019)
[10] Hendrycks, D., Carlini, N., Schulman, J., Steinhardt, J.: Unsolved problems in ML safety. arXiv:2109.13916 (2021). https://doi.org/10.48550/arXiv.2109.13916
[11] Saltzer, J.H., Schroeder, M.D.: The protection of information in computer systems. Proc. IEEE 63(9), 1278–1308 (1975). https://doi.org/10.1109/PROC.1975.9939
[12] Anderson, J.P.: Computer Security Technology Planning Study. Technical Report ESD-TR-73–51, U.S. Air Force Electronic Systems Division (1972)
[13] Lampson, B.W.: A note on the confinement problem. Commun. ACM 16(10), 613–615 (1973). https://doi.org/10.1145/362375.362389
[14] Babcock, J., Krámar, J., Yampolskiy, R.V.: Guidelines for artificial intelligence containment. In: Next-Generation Ethics, pp. 90–112. Cambridge University Press (2019). https://doi.org/10.1017/9781108616188.008
[15] Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K.R., Leibo, J.Z., Larson, K., Graepel, T.: Open problems in cooperative AI. arXiv:2012.08630 (2020). https://doi.org/10.48550/arXiv.2012.08630
[16] Foerster, J.N., Assael, Y.M., de Freitas, N., Whiteson, S.: Learning to communicate with deep multi-agent reinforcement learning. In: Advances in Neural Information Processing Systems 29 (NeurIPS 2016), pp. 2137–2145 (2016)
[17] Park, J.S., O’Brien, J., Cai, C.J., Morris, M.R., Liang, P., Bernstein, M.S.: Generative agents: interactive simulacra of human behavior. In: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST ’23), pp. 1–22. ACM (2023). https://doi.org/10.1145/3586183.3606763
[18] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y.: ReAct: synergizing reasoning and acting in language models. In: International Conference on Learning Representations (ICLR 2023) (2023). https://doi.org/10.48550/arXiv.2210.03629
[19] Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., Fritz, M.: Not what you’ve signed up for: compromising real-world LLM-integrated applications with indirect prompt injection. In: Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec ’23), pp. 79–90. ACM (2023). https://doi.org/10.1145/3605764.3623985
[20] Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., et al.: Constitutional AI: harmlessness from AI feedback. arXiv:2212.08073 (2022). https://doi.org/10.48550/arXiv.2212.08073
[21] Rebedea, T., Dinu, R., Sreedhar, M., Parisien, C., Cohen, J.: NeMo Guardrails: a toolkit for controllable and safe LLM applications with programmable rails. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 431–445. ACL (2023). https://doi.org/10.18653/v1/2023.emnlp-demo.40
[22] Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al.: Taxonomy of risks posed by language models. In: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), pp. 214–229. ACM (2022). https://doi.org/10.1145/3531146.3533088
[23] Haber, S., Stornetta, W.S.: How to time-stamp a digital document. J. Cryptol. 3(2), 99–111 (1991). https://doi.org/10.1007/BF00196791
[24] Crosby, S.A., Wallach, D.S.: Efficient data structures for tamper-evident logging. In: Proceedings of the 18th USENIX Security Symposium, pp. 317–334. USENIX Association (2009)
