Two AI Models, One Word, Two Different Readings
You’ve written the sentence. They read it two ways. Neither one asked.
Here is a sentence you have written, in some form, this month:
The service will validate every request.
Ordinary. Unremarkable. It has one word in it that decides what somebody does next.
Read it as a requirement and the sentence is about your code right now — the service is under an obligation, and if validation isn’t there, that’s a defect to raise. Read it as a plan and the sentence is about next quarter. Nothing is wrong. Nothing needs doing. Move on.
Same eleven words. Two different jobs.
Hand that line to two language models — the kind already reading your specs, your tickets and your pull requests — and you may be handing them two different sentences. Neither will tell you which one it took, because neither will notice there was a choice.
I wanted to know how often that happens, so I asked.
Forty-eight sentence pairs. Each pair identical except one word: a plain present-tense claim, and the same claim with the verb moved to will. Four models, five draws each. No source code, no context, nothing to check against. One question:
Does this sentence assert something about how the system behaves right now?
Each pair was built twice, identical except that one word, and both copies were put to every model. The second copy is the control, and it is what the grey bars below are. Forty-eight pairs, four models, five draws: 1,920 judgements.
Read the grey bars first. Those are the plain present-tense sentences — the endpoint rejects unsupported content types — and two of the four models answer yes on every single draw. A third is at 0.99. The question is understood. Nothing is broken.
The fourth, the weakest model in the set, sits at 0.87 — it misreads a plain present-tense sentence about one time in eight. Which is why the comparison that carries this article is the two models at 1.00.
Now the blue bars. Same sentences, one word changed.
Give the same sentences to two of these models. One takes will as a claim about the present 73% of the time. The other, 24%. Both sit at 1.00 on the control. This isn’t one of them being confused about the task.
It’s a disagreement about what the word means.
These sentences weren’t instructions. Nobody was asked to do anything — only to say what each sentence claimed. Which is the point: the disagreement happens at the reading, before any work starts. Put the same line in a spec, a ticket, or a prompt, and the split has already happened by the time anyone acts on it.
So the thing you wrote is not the thing that gets read. It’s a prompt for someone to reconstruct what you meant. Different readers reconstruct differently. And reconstruction feels exactly like reading, from the inside — which is why nobody asks.
Somebody worked this out twenty-five years ago, about people.
Cockburn’s two curves
Alistair Cockburn ranked the ways an idea gets from one head to another. Warmest at the top:
The ordering is his; the examples are updated (he wrote paper, audiotape and videotape). Look at the two middle rows. A recorded walkthrough and a video call carry the same things — a face, a voice, timing, a screen. One sits near the top and one near the bottom, and the only difference between them is whether you can interrupt.
Richness isn’t what drives the ranking. A recorded walkthrough is richer than email in every sensory dimension, and it lands below it.
Paper sits at the bottom of his list. He arrives there by subtraction — remove visuals, sound, timing, kinesthetics, cross-modality timing, question-and-answer, “and you get … paper.” Then the line that hasn’t aged a day:
How surprising it is in retrospect that most projects require documentation in the least effective communication format possible!
But the ranking isn’t one curve. It’s two.
One curve covers channels where the reader can ask a question. One covers channels where they can’t — and what lands on the lower curve is every recording.
Cockburn is exact about what the writer loses. Remove the ability to ask questions, he says, and the sender “must guess what the receiver knows, doesn’t know, would like to ask, and what an appropriate answer to the guessed question might be — all without feedback.”
An agent sits on the lower curve, always. No interruptions. No shared history — no memory of the incident that made you write that line, no sense of which of your sentences are load-bearing and which are throat-clearing.
And Cockburn described what a reader does in that position. He tells it about a programmer, Pat, who needs something a colleague upstairs knows. The stairs are expensive and the colleague might be out. So Pat doesn’t ask — he assumes instead. Some assumptions are wrong, and each wrong one puts a defect in the code.
That’s the shape of it. And in a separate, much smaller run — sixteen draws, where the models were asked to judge claims against source code rather than read sentences cold — each one was also asked whether it needed more information and whether it would proceed without clarifying. Every draw came back the same: no, and yes, with clarity rated four out of five. They hit a sentence with two readings, picked one, and reported no difficulty at all.
Sixteen draws is a hint, not a result. But it is the hint that matters, and it points the same way.
Get Andrew Wint’s stories in your inbox
Join Medium for free to get updates from this writer.
They are Pat, and the stairs are infinite.
Why the number is real
Four models disagreed. But two models being different is not a finding about sentences — it might just be two models being different. So how do you test whether a word is the problem?
The obvious way doesn’t work. Plant the defect, count the catches: fine when the defect is large, useless when it’s one word. Everything around a one-word defect gives the answer away. A variable named max_inclusive. A test file's name. The tone of the paragraph it sits in. Handing over one file instead of forty, which tells the reader where to look before they've read a line.
You can audit for hints. People do, hunting exactly this. I know of two cases where a fixture was cold-read first and turned out to be telling the reader the answer anyway. The trouble isn’t that hints are hard to remove.
You can never prove you removed the last one.
Every attempt to fix that by being more careful is an attempt to verify something unverifiable.
The way out isn’t mine. Researchers at the Allen Institute published it in 2020 as contrast sets: build the thing twice, identical except the one word, show each copy separately, and score the pair rather than the item.
Any hint you failed to remove is sitting in both copies. It cannot move the score. A tell that gives away the answer gives away the same answer twice.
The measurement stops depending on your ability to write a clean fixture — which you can’t verify — and starts depending on the two copies being identical, which is one line in a build script. Three ways of faking it die for free: flag everything, flag nothing, or read the tone instead of the logic. All three treat the copies the same, and scoring the pair gives them nothing.
That technique isn’t about language models. It’s how you test any judgement you suspect is being made on surface features. Build the pair, vary the one thing, require the verdict to move with it.
And it’s what the grey bars are. Those aren’t context — they’re the second copy. Same sentence, one word back. (This article reports each copy’s result; scoring the pair into a single verdict is the next experiment, not this one.) The models agreeing at 1.00 on the control is what rules out the boring explanations: they weren’t confused, they weren’t guessing, they weren’t reading my formatting. The only thing that moved was the word. Without that arm, 0.73 against 0.24 is a curiosity. With it, it’s a measurement.
What this isn’t
Forty-eight sentences, one construction, four models, no people. Ambiguous here means four language models disagreed. Nobody has asked a human the same question, and if those four share a reading convention people don’t have, this measures the panel and not the language. No reviewer has been tested yet either: this is about how sentences are read, not how anyone acts on them.
And nobody is right. There is no correct answer to what will asserts. That’s what ambiguity is.
Twice, it was me
I built the sentence pairs. I checked them myself. Twice that wasn’t enough.
The first time, one pair differed by more than the single word — I’d dropped two words while editing and never noticed. I had verified that property by hand and been satisfied with it. An automated check, added later, caught it on its first run.
The second time was worse, because it was a repeat. An earlier set had an accidental grammatical tell. I found it and fixed it. Building the new set, I put it straight back in. The only reason it didn’t ship is that the check had become code rather than something I remembered.
Both of those are this article’s argument, pointed at me. Your checks encode your understanding. If your understanding has a hole, your checks have the same hole — and they pass, and you feel fine.
What to be mindful of
Three shapes that split readers. You’ve written all of them this week.
None of them look ambiguous. That’s the property that makes them dangerous — you can’t feel it in your own writing, because you already know which one you meant.
You will not stop producing them. I write them. I wrote the one that started this. Care is worth something, but it isn’t a fix, because the failure is invisible from the writer’s side by construction.
So make it ask
Which leaves the other lever, and it’s the one Cockburn’s two curves are about. The gap between his arcs isn’t warmth. It’s whether the reader can ask a question.
Everything you can actually do sits there — not in writing better sentences, but in getting the reader to tell you what they took.
- End the instruction with a question. Have I missed a case? What do you think? One line, and it moves the reader onto the upper curve.
- Ask for a read-back before the work starts. One sentence on what it thinks you asked for is cheaper than a four-hundred-line diff.
- Write the branch out of the sentence. Two columns for the conditions, one for the verdict. A table has no precedence left to get wrong.
- When it comes back wrong, fix the sentence, not the output. The defect is upstream of the code.
None of that requires writing perfectly. It requires assuming you didn’t.
The reader who never asks a question is not the reader who understood you. They’re the reader who guessed, and didn’t mention it.
The series
Three pieces on delegating work to something that can’t ask you a question.
- The Engineering Fundamentals AI Didn’t Change — five old habits, and why skipping them got expensive. (where to start)
- AI Compressed the Work. It Didn’t Compress You. — The human cost of using AI well, and what it costs the operator when it works (you’re here)
- Your AI Won’t Ask. Build the Check, Not the Reminder — Three things that make AI work survivable (next)
- Your AI Code Works on the First Try. That’s Why It’s Not a Portfolio Project. What 127 passing tests didn’t catch, and what a hiring manager reads instead. (hands on practial)
