Your Agent Is Blind.
Most AI agent failures are context failures, and context engineering, not prompt engineering, is how you fix them.
Ask an AI agent “what was churn last quarter?” and it will answer confidently. It will also, quite often, answer wrong. Not because the model is bad at math or SQL. Because nobody told it which of your company’s three definitions of churn is the real one, which timezone “last quarter” ends in, or which rows this particular user is even allowed to see.
When that happens, most teams reach for one of two fixes: a better prompt, or a bigger model. Both miss the actual problem. The model never saw the facts it needed. You can’t prompt your way out of missing information.
This is the difference between prompt engineering and context engineering. A prompt shapes the ask. Context supplies the world the ask lives in: the instructions, the retrieved facts, the tool results, the memory of what happened three steps ago, the permissions that decide what the agent may touch. Prompt engineering was the skill of 2023. Context engineering is the skill that decides whether your agent survives contact with production.
Blind, Not Dumb
Here’s the observation that changed how we build at Upsolve: when we trace a bad answer back to its cause, it’s almost never the model reasoning badly over good inputs. It’s the model reasoning fine over incomplete ones.
The model is a very fast, very literal new hire. On day one, a new analyst doesn’t fail because they’re not smart. They fail because they don’t know that “active user” excludes internal accounts, that the finance team’s numbers are canonical and the ops dashboard is not, that the Q3 board deck used constant currency. Everything we call onboarding is context transfer. Agents need the same thing, except they need it re-transferred on every single request, because they remember nothing you don’t explicitly hand them.
So the job stops being “write a clever instruction” and becomes “design the pipeline that assembles the right slice of reality for each step.” That’s an architecture problem, not a wording problem.
More Is Worse
The tempting shortcut is to skip the design and stuff everything in. Context windows are huge now. Why not paste the whole schema, every doc, all the history?
Get Ka Ling Wu’s stories in your inbox
Join Medium for free to get updates from this writer.
Because models read the way people read: attention is finite, and it degrades with clutter. Bury the one fact that matters under forty tables of irrelevant schema and the model starts pattern-matching on noise. Long sessions accumulate small errors that quietly poison later answers. Two tool outputs contradict each other and the model picks one at random. Practitioners have names for these failure modes now: context rot, context clash, context distraction. They all share one root cause, which is treating the window as a bucket instead of a budget.
The discipline that works is the opposite of stuffing. For each step, assemble the minimum context that lets the model complete that step: the goal, the constraints, the few facts that bear on this decision, the two tools it actually needs. Not the smallest context you can get away with. The smallest one that’s sufficient. Everything past that point isn’t safety margin. It’s interference.
Which raises the real question: if you can only hand the model a small slice of your world, how do you know which slice?
The Slice
You can’t answer that with retrieval alone. Vector search finds text that looks like the question. But “what was churn last quarter, for enterprise accounts, excluding the ones that migrated plans” isn’t a similarity problem. It’s a structure problem. The answer depends on how your business defines things and how those definitions connect.
For analytics agents, that structure mostly lives in what the BI world calls a semantic layer: metric definitions, entity relationships, allowed joins, ownership, freshness. We’ve come to think of the semantic layer as context engineering’s best-kept secret. It’s a machine-readable map of what your data means, built years before anyone knew agents would need one. When an agent’s context includes “churn = logo churn, monthly cohorts, defined in metrics.yml, owned by finance,” the guessing stops. The same structure that makes answers correct also makes them explainable: you can show a skeptical customer exactly which definition and which filters produced the number. Trust in an answer is really trust in its provenance.
Then there’s the context that never appears in any document: state. The customer ID resolved in step one. The date filter the user set two questions ago. The permission boundary that says this tenant sees only this tenant’s rows. In a multi-step agent, this is the context most likely to silently fall on the floor between steps, and every dropped piece becomes a wrong answer with a delay. The fix is unglamorous: pass state forward explicitly, as small structured facts, and make each tool return typed records instead of prose so the next step consumes facts rather than re-parsing paragraphs.
Where the Moat Went
Step back and something bigger comes into view. Every team building agents rents the same intelligence. The models are a few API calls away from anyone, and whatever gap exists between them closes a little every quarter. What can’t be rented is the context: your definitions, your permission model, your accumulated map of what the data means and who’s allowed to ask what of it.
Which means the leverage has moved. Two years ago the differentiating question was “which model are you on?” Now the model is the commodity input, and the differentiating question is the one this essay started with: what does your system show the model, and when? Teams that treat that as a first-class engineering problem ship agents that are reliable, auditable, and boring in the best sense. Teams that don’t will keep tweaking prompts and wondering why the demo never survives the customer.
The models are rented. The context is yours.
Follow me on Medium and LinkedIn for more on analytics agents, context engineering, and the practical infrastructure behind production-grade AI.
