AI is for Alignment: Execution Contracts
In the last post, I wrote about one of my favorite prompts when I’m working with an AI agent:
The idea is pretty simple. Before I trust the agent to build something, I want to see what it thinks I mean. Walk me through a real scenario. Show me what success looks like. Show me how it fails. Try the weird edge cases.
Basically: don’t tell me you understand. Show me what your understanding predicts.
I’ve been doing something similar with coding agents lately, but one step earlier in the process.
Before I ask them to write the code, I ask them to write a contract.
I didn’t really think of it as a contract when I started doing this. I thought I was just getting more specific about the requirements: tell the agent what the software should do, spell out the acceptance criteria, make sure there aren’t any obvious ambiguities, then let it implement the thing.
But after doing this a few times, I realized I was solving a slightly different problem.
A specification tells you what the software is supposed to do. A contract tells the agent what it’s allowed to do while getting there.
That distinction doesn’t matter much if I’m using AI as autocomplete. In that situation, I’m still making nearly every decision myself. The AI suggests some code, I look at it, maybe I take it, maybe I don’t.
An agent is different. If I give it access to a repository, it can inspect the code, edit files, run tests, chase down failures, create commits, and potentially open a pull request. At that point I’m not really asking for a suggestion anymore. I’m delegating a chunk of engineering work.
And once I’m delegating, I care about more than the final behavior.
A simple bug fix is a good example. Imagine a renderer that expects some nested object to exist. Production data comes back without that object, the renderer throws, and the page breaks.
The software requirement is almost trivial:
If the object is missing, render an empty value instead of throwing.
That’s a perfectly good description of what the software should do. But it says almost nothing about how I want an agent to change the system.
The agent could add optional chaining to one line. It could change the API so the object is always returned. It could wrap the renderer in a try/catch, introduce a generic error-handling abstraction, refactor the data model, or notice three similar accesses nearby and “clean those up” too.
Some of those might even be reasonable engineering decisions. They’re just not necessarily the work I meant to delegate.
That’s the part the contract handles.
For a change like this, I might say that only two files are in scope. The API is explicitly out of scope. Shared rendering infrastructure is out of scope. Don’t refactor neighboring code. Add a failing test before changing the implementation. Preserve a few specific existing behaviors. Run a particular set of commands before calling the work complete.
And then there’s the line I’ve started caring about most:
If reality doesn’t match the assumptions in this contract, stop and report the discrepancy.
Don’t silently reinterpret the assignment.
That probably sounds obvious because it’s something experienced engineers do all the time without being told. If I give another engineer a small bug ticket, I usually don’t need to add, “Please don’t redesign the architecture while you’re in there.” I also don’t need to say, “If the ticket is based on a bad assumption, come talk to me instead of inventing a new requirement.”
Those expectations are just part of working on a team. There’s a social contract around delegation that humans pick up over time.
AI agents don’t really get that for free.
They have an objective, some context, a set of tools, and instructions. If there are five plausible ways to accomplish the objective, something has to tell them which decisions are theirs to make and which ones aren’t.
That’s how I’ve started thinking about execution contracts. They turn a bunch of implicit engineering expectations into explicit boundaries.
But this is where the idea from “Simulate This” comes back in.
Once the contract is written, I don’t necessarily want the agent to start coding yet.
I can say:
Simulate this contract.
Stay read-only. Don’t change any files. Don’t run anything that mutates the repository. Walk me through exactly what you expect to do, in order. Show me which files you’ll touch, which tests you’ll add, what you expect to fail before the fix, what you expect to pass afterward, which commands you’ll run, and where the contract requires you to stop.
That gives me a dry run of the work before the work happens.
I tried this recently with a real bug-fix contract, and this is where things got interesting.
The agent walked through the contract step by step. It checked the preconditions, identified the tests it intended to add, predicted the red phase, showed the one-line implementation change it expected to make, then walked through every verification command and the final delivery report.
That alone was useful. I could see the whole change before it touched anything.
But the simulation also found two bugs in the contract.
One part of the contract said that after adding the new tests, I should expect one test to fail. The simulation walked through the actual data shapes and showed that two tests would fail.
The implementation wasn’t wrong. The contract was.
Then it found a much more important problem.
One of the required verification steps was the project’s normal lint command. Nothing unusual about that, except the script included --fix.
So if the agent followed the contract literally, the verification step itself could modify files across the repository. The same contract that said “only these two files may change” was telling the agent to run a command that could change other files.
That’s exactly the kind of thing I want to discover before execution.
The dry run wasn’t just predicting the code. It was testing whether the instructions I had given the agent actually made sense when followed as a sequence of actions.
That changed how I think about simulation.
In the original version, “simulate this” was mostly about making the agent’s model of the system visible. Show me how you think this feature behaves. Show me what this configuration does. Show me what happens in this edge case.
Get Owen Zanzal’s stories in your inbox
Join Medium for free to get updates from this writer.
When I pair simulation with a contract, I’m making the agent’s model of the delegation visible too.
What do you think I authorized you to change? In what order are you going to act? Which commands are you going to run? What do you think counts as evidence? What happens if one of your assumptions is wrong? Where do you believe your authority ends?
That’s a different kind of simulation.
It’s basically a rehearsal.
And I like that there’s a human approval point between the rehearsal and the real thing. The flow becomes:
Write the contract.
Simulate the contract.
Fix the misunderstandings.
Then execute.
The contract defines the boundaries. The simulation shows me how the agent interprets those boundaries before it exercises any authority.
That matters because capable agents notice things.
They notice the neighboring bug, the inconsistent API, the duplicated code, the abstraction that could probably be cleaner. And because they’re trying to be useful, there’s a natural temptation to fix those things too.
Humans do this, obviously. We call some versions of it scope creep.
The difference with agents is that implementation is cheap. An agent doesn’t get three files into an unnecessary refactor, feel the afternoon disappearing, and decide maybe this isn’t the day to clean up the whole subsystem. If you give it broad permission to solve a problem, it can solve it very broadly.
So I’ve started putting prohibited changes directly into contracts. Don’t change the API. Don’t add a dependency. Don’t refactor neighboring code. Don’t change the test configuration. Don’t weaken an acceptance test to get the suite green.
If you find another legitimate issue, record it as a known gap and leave it alone.
I don’t think that removes engineering judgment. It defines where the judgment belongs.
There’s a big difference between telling an agent exactly which keystrokes to make and telling it, “Choose the implementation, but these architectural boundaries are fixed.”
If I make the contract too prescriptive, I’ve basically written the patch in English. The agent is just translating prose into syntax. But if I make it too loose, I’m relying on the agent’s idea of a reasonable implementation being the same as mine.
The useful space seems to be somewhere in the middle. Define the outcome, the invariants, the scope, and the things that must not change. Define the evidence that will count as success, and define the points where the agent has to hand the decision back to me.
Testing is where this gets especially useful.
I don’t always want a contract to say:
Add a test.
Sometimes I want it to say:
Add the test first. Run it against the current implementation. Show that it fails for the reason we think it should fail. Then change the implementation.
I ran into a case recently where that distinction mattered a lot. There was already a unit test that looked almost exactly like coverage for a production bug. Even the name of the test made it sound like we were covered.
But the mock returned an empty object, while production returned no object at all. Those two states sounded almost interchangeable when you described them in English. In JavaScript, they behaved very differently.
The existing test passed against the broken implementation.
So the red phase became part of the contract. Use the actual production shape. Run the test before fixing anything. Capture the failure. Then change the code.
And this is another place where the dry run helps. Before changing the implementation, the agent can tell me which cases it expects to fail and why. If that prediction doesn’t match reality when it eventually runs the tests, we’ve learned something. Maybe the contract is wrong. Maybe the agent’s model is wrong. Maybe there’s something about the system neither of us understood.
Any of those are useful things to discover before the agent keeps going.
This is why I’ve started thinking of contracts less as detailed requirements documents and more as agreements about evidence.
What does the agent actually have to prove before it can say it’s done? Which tests ran? Which build? Which type checks? Did the diff stay inside the agreed scope? Were there manual checks the agent couldn’t perform? Is anything still uncertain?
I’ve started asking for a delivery report against the contract at the end of the work too. I don’t want the usual “implemented the requested fix and all tests pass.”
I want to know which requirements were satisfied and how, which acceptance scenarios passed, which commands ran, and which files changed. I also want to know what wasn’t tested, what gaps were discovered and deliberately left alone, and whether anything deviated from the original contract.
There’s a nice symmetry to that.
Before implementation, simulate the contract and show me what you think is going to happen.
After implementation, report against the contract and show me what actually happened.
Now I have something to compare.
And there’s another benefit to this that I didn’t expect. Writing and simulating the contract often exposes problems before anyone writes code.
If I can’t clearly describe the required behavior, maybe the requirement isn’t ready. If I can’t write down an acceptance scenario, maybe I don’t actually know what success looks like. If I keep adding exceptions to the allowed scope, maybe this isn’t a local change after all.
And if the dry run keeps hitting places where I don’t know whether the agent should continue or stop, there’s probably a product or engineering decision hiding there that I haven’t made yet.
That’s why these two ideas fit together so well for me.
A contract forces me to make the delegation explicit.
Simulation forces the agent to make its interpretation of that delegation explicit.
One makes my model visible. The other makes its model visible. And we get a chance to compare them before anything expensive happens.
So the distinction I keep coming back to is still:
A specification constrains the software. A contract constrains the agent changing the software.
But now I’d add one more thing.
Before I let the agent operate inside that contract, I want to see the rehearsal.
Write the contract.
Then simulate it.
