Janagaraj Sangamanarayanan·Sep 23Jev vs Laya: What 350 Support Tickets Taught Me About Evaluating AI Decision ModelsA step-by-step account of comparing two typed-decision models on support-ticket triage, and how refining the scoring rule, the question…
Rockfish Data Team·Sep 9Generating synthetic data that actually holds upReal use cases need data with real structure — timing, sequence, referential integrity. Here’s how to generate it by putting the right…
Micheal Lanham·Sep 6Hand the Agent a Gauge, Not an OpinionTips 91 to 100 of 100. The finale. Replace a judgement with something that can be run, and put the judging somewhere the worker cannot…
Micheal Lanham·Sep 1The Answer Was at Rank 11Tips 59 to 64 of 100. A collection and the means of finding things inside it are two separate pieces of engineering.
Micheal Lanham·Aug 27Your Eval Says 97 Percent. Your Users Say Otherwise.Tips 25 to 32 of 100. A judge is not an instrument you bought. It is a model with opinions, and several of them point the same way as the…
Micheal Lanham·Aug 26Knowing What Right Means: Building an Eval Suite That Actually BitesTips 17 to 24 of 100. Underwriters Laboratories did not invent testing. They invented testing against the fires that had already happened.
Micheal Lanham·Jul 9Stop Trusting the Vendor’s Scoreboard. Weigh It Yourself.A launch-day benchmark chart grew taller overnight with no changelog. Here is the afternoon of code that turns your quality bar into a…
Micheal Lanham·Jun 26Your Eval Suite Is Green. Your Agent Still Isn’t Safe.How to turn your eval pieces into a gate that blocks bad changes automatically, and why a passing gate means “no known regressions,” not…
Micheal Lanham·Jun 25Your Agent Got the Right Answer. That Doesn’t Mean It’s Reliable.How to score the path an agent took, not just where it landed, and catch the runs that look healthy but aren’t. With runnable Agent SDK…
Micheal Lanham·Jun 24A Model Can Judge the Quality Code Can’t. Here’s How Not to Get Fooled.How to build an LLM-as-judge for your agent, the predictable ways it lies, and the one check that turns it from a guess into a measurement.
