I gave an AI agent a calculator because I wanted one hard, inspectable point
inside a probabilistic workflow.
The model could interpret the request and explain the result. The calculator
would perform the computation. It seemed like a clean division of labor.
Then I changed one multiplication sign into addition.
The calculator still passed 389 of the 390 tests in its Rust library harness.
The sole failure compared its answer with NIST's certified results for the
Longley regression dataset.
That bothered me more than a completely broken build would have. I had treated
deterministic computation as safer than asking a language model to improvise
arithmetic. But deterministic does not mean trustworthy. A program can return
the same wrong answer forever.
“Source of truth” suddenly felt too comfortable. Before an AI agent delegates
authority to a tool, that authority should be challenged—and remain revocable
by evidence.
The calculator is only the specimen. The larger idea is a way to place
inspectable, replayable instruments inside probabilistic systems.
The useful boundary is generation versus execution
The interesting distinction is not model weights versus a “real CPU.” Model
inference also runs on processors, and language models can learn genuine
arithmetic procedures. The useful boundary is between generating an answer
and executing a defined operation under a tested contract.
Research on Program-Aided Language Models
(PAL) makes a related split:
the language model reads and decomposes a natural-language problem, while a
runtime such as a Python interpreter executes the generated program. The model
contributes flexible interpretation; the runtime contributes executable
semantics.
That is the division I want in an agent:
- At the semantic edge, the model interprets the request, chooses a procedure, identifies relevant quantities, and explains the result.
- At the computational edge, a narrow tool validates inputs, applies specified operations, enforces limits, and returns structured output.
This does not make the whole agent deterministic, and it does not make the
model unnecessary. The agent can still choose the wrong tool, supply the wrong
arguments, misunderstand units, or misread the result.
The promise is smaller: one claim becomes inspectable, replayable, and
independently testable.
The opposite of probabilistic is not trustworthy. It is repeatable.
A CPU can be precisely wrong.
Why NIST became my external witness
Calibration cannot be entirely self-referential. The implementation should not
be the sole author of its own expected answers.
That is why I chose NIST—not because government authority turns a result into
mathematical truth, but because NIST has a long institutional practice of
building shared, independently evaluated references.
Congress established the National Bureau of Standards in
1901 to
strengthen the United States' measurement infrastructure. The Standard
Reference Data Act of
1968 authorized a federal program to
collect, critically evaluate, publish, and distribute standardized scientific
and technical reference data. NBS became NIST in
1988. In 1999, that lineage reached statistical
software through the Statistical Reference Datasets
project, usually shortened to StRD.
StRD pairs datasets with certified expected values for specific statistical
procedures. Its collection includes generated and real-world cases of varying
difficulty. For linear procedures, NIST carried 500 digits through its
calculations
so ordinary floating-point representation error would not become the
benchmark.
Longley is a small but numerically challenging linear-regression dataset in
that collection. Its certified results gave my tests something the
implementation could not manufacture for itself: an expected answer produced
outside the code under test.
That qualification matters. StRD does not certify Oddly Exact, prove the
statistics engine correct, or make the software traceable to NIST. NIST
explicitly says the datasets are an aid for evaluating software and that no
mechanism establishes software
traceability.
The reference data was not an oracle for the entire program. It was an
independent witness for the calculations it covered.
Test the tests
Example-based tests ask whether familiar inputs still produce familiar
outputs. Mutation testing asks a more uncomfortable question:
If I introduce a small, plausible defect, does the suite notice?
I used cargo-mutants to replace multiplication with addition in the
multiple-regression standard-error calculation:
- residual_std_dev * sum_sq.sqrt()
+ residual_std_dev + sum_sq.sqrt()
The mutation preserved valid Rust, valid types, and a plausible-looking numeric
result. In the library harness, 389 of 390 tests still passed. The assertion
against NIST's Longley values was the sole failure.
The public mutation
record
preserves the exact command, environment, result, and the limit of what that
experiment establishes:
I cannot claim Longley was the only test in the entire repository capable of
catching the mutation; the run stopped after the failed library harness. I can
claim something narrower and more useful: hundreds of tests tolerated a
semantically broken formula, while an assertion anchored to independently
produced values rejected it.
A green test count is evidence only to the extent that those tests would turn
red when the implementation meaningfully changes. Mutation testing measures
that sensitivity instead of admiring the count.
Challenge the contract around the answer
The formula mutation challenged numerical meaning. My next audit challenged
the promises around the calculation: what the tool accepts, what it refuses,
and how much work it will perform.
The first crack was a contradiction between the advertised contract and the
executable one. The generated JSON Schema forbade additional properties, but
the Rust deserializer silently accepted an unknown field at the request root
and another inside an expression node.
The answer was still numerically correct. That did not make the behavior
harmless. A caller validating against the schema saw a stricter instrument than
a caller speaking directly to the binary. A misspelled or misunderstood field
could disappear without warning.
For an agent-facing tool, silently interpreting a different contract is itself
a correctness defect.
The second crack was a resource boundary. The expression engine limited things
such as integer size, expression depth, and precision. The optimization
interface, however, accepted grid resolution and iteration counts without upper
ceilings.
I requested a grid search with one billion sample points. The pre-repair binary
produced no JSON before an external one-second watchdog terminated it. The tool
had input validation, but it had not earned the bounded-work guarantee I
thought I had built.
Those failures became permanent regression requests. The deserializer now
rejects unknown fields at both levels. The optimizer now publishes and enforces
ceilings of 100,000 iterations and 1,000,000 grid points. The same billion-point
request returns a typed resource_limit response before entering the search.
The contract challenge
Gist
is a runnable replay of all three repaired cases:
That script is more valuable than a screenshot of a green run. It lets another
observer cross-examine the boundary directly.
Repair is the next claim to attack
A new guard is only another claim until the tests prove they care about it.
After repairing the optimizer limits, I selected every mutation
cargo-mutants generated for the two new validators. Eleven mutations tried to
remove the checks, replace them with unconditional success, or alter their
boundary comparisons. The tests caught all eleven.
The focused optimizer mutation
record
captures the selection and outcome:
That completed the loop:
claim → attack → fail → repair → mutate → replay
If I published only the final green suite, you would see confidence. By
preserving the changed operator, the request that hung, the contract
contradiction, the repair, and the mutations that tried to undo it, I can show
a reason for confidence.
The scar is part of the calibration record.
A five-layer challenge stack for agent tools
Oddly Exact is one calculator, but the method travels. Before giving a narrow
tool authority inside an agent workflow, I now ask five questions:
Is the contract explicit and executable? The schema, parser, runtime,
limits, and failure modes must agree. Documentation the executable does not
enforce is only a suggestion.Is there an independent reference? In this statistical case, NIST StRD
moved selected expected answers outside my implementation. Another domain
might use a standards specification, a reference implementation, a verified
corpus, or a separately derived test oracle.Which properties should survive across many inputs? Property-based tests
generate cases and check laws rather than memorizing individual examples.
One Oddly Exact property checks that translating every value in a sample by
a large exact offset leaves sample variance unchanged within a tight
floating-point tolerance.Can the tests detect plausible corruption? Mutation testing changes
operators, comparisons, return values, and guards. A surviving mutation does
not prove the implementation is wrong; it exposes a behavior change the
suite cannot currently distinguish.Are failures bounded, typed, and replayable? An agent must distinguish a
rejected computation from an unavailable tool. Division by zero, resource
exhaustion, malformed JSON, and a crashed process should not collapse into
the same ambiguous failure.
These layers do different jobs. External references challenge specific
answers. Properties challenge behavior across an input space. Mutations
challenge the tests themselves. Contract and resource attacks challenge the
boundary around all of it.
None provides universal correctness. Together they create a narrower and more
useful result: evidence that a particular tool deserves provisional authority
for validated requests inside a declared contract.
A source under challenge
For a validated request inside that contract, the tool's structured result can
serve as the agent's operational source of truth.
That is a runtime role, not a claim of infallibility. If the agent encounters a
disagreement, it can inspect the arguments, surface the conflict, or consult
another independently trusted tool. It should not quietly replace a structured
result with fresh prose arithmetic.
The trust relationship runs in opposite directions at different times:
- While building the tool, the developer should distrust it aggressively.
- Inside a contract the tool has earned, the agent should respect its answer or its typed refusal.
New evidence can always return the tool to the first phase. That is a source
under challenge: trusted in operation, open to appeal, and always one
counterexample away from revision.
The beautiful thing is not that source code is truth.
The beautiful thing is that source code can be cross-examined.
I did not end this experiment with a calculator an AI agent can trust forever.
I ended it with something more useful: a calculator whose authority can be
revoked by evidence.
AI tools assisted with editorial research and revision. I verified the
technical claims and stand behind the final text.
Top comments (11)
I really like that this doesn’t stop at “389 tests passed, one failed.” The more interesting takeaway is that the independent reference (NIST) wasn’t just another test case—it was an external challenge to the implementation itself.
The same principle applies well beyond numerical software. In distributed systems, security tooling, or ERP integrations, a green test suite only tells us the implementation agrees with our expectations. It doesn’t tell us those expectations are correct. That’s why independent references, mutation testing, and replayable evidence complement each other so well—they challenge different assumptions instead of reinforcing the same one.
The phrase that stayed with me was: “The opposite of probabilistic is not trustworthy. It is repeatable.” That’s a subtle but important distinction.
The mutation that survived —
*to+in the standard error — isn't just a "your tests aren't sensitive enough" story. It's a specific class of bug: the mutant produced a plausible number, not garbage. That's the whole reason 389 tests waved it through. Assertions written by the same person who wrote the formula tend to encode the same mental model, so they check shape and rough magnitude, not the exact value. NIST worked as an oracle precisely because it was authored by someone who never saw your code and had no incentive to agree with it.The caveat worth adding: this only holds for the narrow slice NIST actually covers. Longley caught the standard-error mutation because that dataset exercises multiple regression. Mutate something in a code path StRD doesn't touch — your optimization grid, your expression parser, anything outside linear stats — and you're back to self-authored oracles with the same blind spots. So the honest read of your own result isn't "add an external witness," it's "you have exactly one witness, for exactly one procedure, and everything else in the binary is still marking its own homework." The optimization boundary bug you hit at the end is the proof — no NIST dataset was ever going to catch an unbounded grid search.
Which is why I'd treat the mutation score per-module as the real deliverable here, not the fact that one assertion held. The gaps in mutation coverage map directly onto the parts of the tool that have no independent witness — and those are the parts an agent will happily drive off a cliff.
"A green count is evidence only to the extent those tests would turn red" is the line I would carve into the wall. I ran a small version of the mechanism under it, using your own offset-invariance property rather than your exact operator swap (no repo access here): correct = two-pass variance, mutant = naive one-pass (sum_sq - sum^2/n)/n. Valid types, plausible float.
The mutant is caught only where the input discriminates it. Shift every value by an exact offset (your property #3) and the naive formula's cancellation error grows with it: rel-err 3e-17 at offset 0, 1.2e-5 at 1e6, 8.4e-2 at 1e8 (true 9.43 vs naive 9.60). An assertion against the known invariant catches it at 1e6 and 1e8, and misses everywhere below. So "Longley caught it" is not only that Longley is independent, it is that Longley is ill-conditioned enough to move the mutant past tolerance. Independence is necessary; the catch is independence AND sensitivity on that input, together.
Which is why the count is the wrong axis. In the run, 10 / 100 / 1000 / 10000 well-conditioned inputs catch the mutant 0 times each; one ill-conditioned input catches it. Adding green tests at the same conditioning adds confidence and zero discriminating power.
The sharpest cell was the snapshot one: when the expected value is captured by running the implementation (a golden assertion), catch is 0.00 at every offset, because a self-authored expectation has sensitivity 0 by construction. It moves with the bug. That is the structural reason 389 can stay green: any assertion whose expected value the implementation authored is blind to a mutation on the shared path, no matter how many you have. NIST's value was the one nobody in the codebase could edit.
file sha 07768e4d, out fd3314b0, stdlib/offline, 3x byte-identical.
“Independence AND sensitivity” is the right sharpening. My line compresses two separate questions: can the expected
result move with the implementation, and does the selected input make the mutation observable? Your offset sweep
isolates that second question beautifully. Ten thousand well-conditioned cases can provide less discriminating power
than one deliberately hostile case.
I would preserve one boundary between our experiments, though. Your mutant—two-pass variance replaced by the
cancellation-prone one-pass formula—specifically interacts with conditioning. Mine replaced multiplication with
addition in the regression standard-error calculation. Your result demonstrates why conditioning kills your mutant,
but it does not yet establish that Longley’s conditioning is why it killed mine. To establish that, I would need to
replay the exact operator mutation across datasets with controlled conditioning. Longley may instead have been the
only library test that independently asserted the affected output.
I’d also narrow the snapshot conclusion. A frozen, checked-in snapshot authored by the implementation can catch later
mutations because it does not automatically move during the test. What it cannot establish is that the original
captured answer was correct. A runtime-regenerated or automatically blessed golden—or an expectation calculated
through the same code path—really does move with the bug and is structurally blind.
So the stronger model has at least four axes: oracle provenance, assertion coverage, input sensitivity, and tolerance.
Raw test count collapses all four into a number that says very little.
Your reproducibility details and hashes make this a genuine extension of the argument, not merely agreement. Thank you
for running it.
I ran exactly the replay you asked for: hold your mult-to-add swap fixed, sweep dataset conditioning, and cross it with oracle type. The result cuts against conditioning as the explanation for your mutant, and toward your own alternative.
Setup: OLS coefficient SE, the mutation placed at the variance combination (sigma2 * v becomes sigma2 + v, where v is the inverse-Gram diagonal). I swept a collinearity knob so cond(X'X) runs from 1.5e3 to 3.4e7, roughly spanning well-conditioned to Longley-grade.
What came out:
So for an operator swap the axis that flips catch versus miss is not input sensitivity, it is provenance. That supports your line that Longley may have been the only test independently asserting the affected output, rather than the one that happened to be ill-conditioned.
I think the two mutants genuinely sit on different axes. Mine, two-pass variance to the cancellation-prone one-pass, is a cancellation effect, so it really is conditioning-driven, which is why my sweep read that way. Yours is a gross algebraic change, visible to any independent oracle regardless of conditioning. Same headline, different mechanism, and it took your boundary to see it.
One caveat that lands on your fourth axis: at the ill-conditioned end the relative difference falls to about 7 percent, so a loose oracle tolerance near 10 percent would start missing your mutant there even with an independent assertion. The four axes interact rather than add. The honest seam: I placed the swap at the variance combination. A different site, inside RSS or inside the Gram product, could couple to conditioning differently, so this is a claim about that placement, not every multiply in the SE path.
I like the distinction between repeatability and trustworthiness. In complex systems, reliability often comes not from making components deterministic, but from creating independent validation layers that continuously challenge assumptions and authority.
389 passing and exactly one failing is such a clean demonstration that a test harness mostly measures agreement with its author. The suite and the implementation shared an author, so they shared the blind spot; NIST did not. For operations where no certified dataset exists, metamorphic relations are the closest substitute: you cannot know the right regression coefficients, but you know scaling every x by 10 must scale the slope by exactly one tenth, permuting rows must change nothing, and fitting a duplicated dataset must return identical coefficient estimates. Those properties are author-independent the same way StRD is, since they come from the mathematics rather than from anyone's implementation. Curious whether you would extend revocable authority into the agent runtime: the tool passed StRD once at review time, but the agent keeps trusting it forever. Re-running the witness suite on every version bump and downgrading the tool's authority on failure would make the revocation actually operational.
Strong writeup. One angle I wish more test suites covered: eval the provider, not just the unit under test.
If traffic goes through a third-party LLM endpoint, a fixed capability probe at temp=0 (diffed against the official API, re-run weekly) catches silent model swaps that green unit tests never see — same rigor as NIST-style property checks, pointed at the vendor.
That’s an excellent and very insightful observation. You’re absolutely right: in an era of reliance on third-party APIs, traditional unit testing creates a false sense of security. If the wrapper code works correctly, the test will pass, but the quality and logic of the provider’s “black box” itself may change without anyone noticing.
"Green test count is evidence only to the extent that those tests would turn red when the implementation meaningfully changes" — I'd put that line above the NIST story, because it's the part that generalises to domains with no Longley to borrow.
I don't have a certified dataset. I build internal tools as a non-developer and one of them is a static security scanner, where nobody publishes an authoritative "here is the correct set of findings for this codebase." So the cheapest external witness I found was the one you're describing in mutation form, made permanent: every guard in the suite gets deliberately broken on every run, and it has to go red for its own reason — matching the expected message, not merely exiting nonzero, because an exit code lets one gate's failure pass as another gate's proof. It's cargo-mutants as a standing gate rather than an occasional audit.
That caught something a periodic run wouldn't have. One guard had drifted into being unbreakable: it was falsifiable by design, but a widened pattern quietly made its failure condition unreachable, so "no failure observed" stayed true for the wrong reason for weeks. Six clean files were being scored as broken behind that green the whole time. Your framing explains why my 389 equivalents didn't help — they and the implementation were written from the same understanding — but the mutation loop is what told me the test had stopped being able to fail, which is a different lie than the one NIST catches. For anyone reading without access to institutional reference data: the minimum viable witness isn't a dataset, it's a mechanism that proves your checks can still turn red today.
hello, everyone it's all new for me.how i understand this all think program and resources
Some comments may only be visible to logged-in visitors. Sign in to view all comments.