The last article ended with the knowledge layer holding both of its sources and lacking one thing: the decision about which to ask, in what order, and what to do with the answer. This article builds that decision as a component above Foundry IQ, the way Microsoft’s own agent service connects to a knowledge base, and measures it in four topologies on three models. Some of what the decision adds is real and repeats on every run. Some of what it appears to add is the source varying between calls, and telling the two apart took a second judge pass that I had not planned.

What was left

The previous article gave Foundry IQ an ontology in Microsoft Fabric and watched the dependency chain, the one shape that had stood through the whole series, fall to a rule written in plain words. It then put the documents and the ontology into one knowledge base, let the planner route between them, and found that two sources that each solved half of the problem did not add up: 78, 89 and 83% on gpt-4.1, gpt-5-mini and gpt-5, with the planner choosing one source per question, the variance of the agent inside Fabric that answers for the ontology, the ontology agent from here on, deciding the outcome whenever it was the only source consulted, and the documents quietly answering whenever the Fabric call failed. The article closed on a sentence I meant as a specification: the knowledge layer has the sources now; what it does not yet have is the decision.

So this article builds the decision. Not inside the knowledge base, where the planner lives and where I cannot change anything, but above it, as an agent that holds each knowledge base as a tool. That is not my invention. It is the shape Microsoft draws for Foundry IQ: a knowledge base exposes a Model Context Protocol endpoint, the endpoint exposes one tool, knowledge_base_retrieve, and an agent in Foundry Agent Service calls it. I checked that surface before the runs: the tools list of both knowledge bases returns that one tool, with a long description that asks the caller for one complete natural-language question with every name kept verbatim, and a call on the ontology base returned the answer and the CSV behind it exactly as the REST path does. The measured runs go through the REST retrieve operation rather than MCP, for one reason: REST returns the activity log, with the subqueries, the tokens and the reranked documents, and this article is about what happened inside each call. The tool the agent sees is the same.

The question is the one the series has asked since the seventh article, narrowed one more time. Given the same two sources, the same 32 questions and the same judge, what does an agent above the knowledge layer add, and where exactly does the addition come from?

The orchestrator, and its four topologies

The agent is a chat model with two tools and a system prompt that says what each tool holds: documents_retrieve for run books, policies, decisions, postmortems and anything in prose; ontology_retrieve for the entities, relationships and rules, and for anything that needs a count, a complete list, a dependency chain or a rule applied across entities. The prompt tells it that a question needing structure and prose needs both tools, that it may ask a tool a follow-up using names the other tool returned, that it must use only what the tools returned, and that if one tool says it cannot answer, it should use the other. Both tools are the medium-effort knowledge bases from the tenth article, in extractive mode, the knowledge base's default: the base returns passages and references without writing an answer, and the agent above it writes the answer; for the ontology base the passage is the Fabric agent's answer and the rows behind it. The agent model and the knowledge base's planning model are the same in every cell, gpt-4.1, gpt-5-mini or gpt-5, so that a cell measures one model's decision over that model's retrieval. The 32 questions are the series' set, and their shapes carry over: eight page questions answered by prose; sixteen a graph can own, five lookups, three chains, three sets, two absences and three definitions, of which the eleven beyond the lookups are what the earlier articles called the wall, the questions retrieval alone did not answer; three mixed questions that need prose and structure; and five traps, questions that sound like graph questions and were written to be answered from the documents, though the tenth article found two of them in the ontology's tables too. The judge is gpt-4.1, scoring against the same per-question list of key facts as every earlier article.

Two implementation decisions came out of the smoke tests and are worth recording. The first turn must call a tool: in an early test gpt-4.1 answered without calling either tool, naming services it had not retrieved, and an answer with no retrieval is not a measurement of the sources. And when the ontology call fails in the way the tenth article described, the search service returning an error because it could not parse the Fabric agent’s honest “I don’t model that”, the harness extracts the Fabric reply from the error text and hands it to the agent as the tool’s result. This is the fix the tenth article asked for, applied one layer up: the refusal becomes something the agent can read and act on, instead of a failed request.

Four topologies, run as four separate arms. Single allows exactly one tool call: the agent reads the question, picks a source, and answers from what comes back. It is the orchestrator reduced to the planner’s one decision, and it is there to measure whether an agent’s one choice looks like the knowledge base’s one choice. Iterate allows up to four calls in any order, so the agent can ask the ontology, read the names, and ask the documents about them, or the reverse. Verify (instruction) is iterate followed by a check: the draft answer and an instruction, “check this draft against the data you hold, list every statement the data contradicts and every item the data has that the draft is missing”, go to the ontology knowledge base, and the agent revises only if the check came back with data. Verify (rows) is the same idea with the planner’s rewriting taken out of the loop: the ontology is asked the original question, which the planner passes verbatim, or its rows from the iterate phase are reused if it was already asked, and the agent reconciles its draft with the rows under an explicit rule: add every item the rows include that the draft lacks, correct any count or name the rows contradict, keep everything the documents supported that the rows do not cover.

The fourth topology was not in the plan. It was added after the gpt-4.1 results and before any gpt-5 verification result had been read, for a reason the section on verification explains, and the outline on disk records the change with its date. All four ran on all three models: 384 runs, the 32 questions once in each of the twelve cells, and every run completed. One more thing was added after all the runs, and it changed the article more than the fourth topology did: every draft that a verification pass had started from, 192 of them, was judged afterwards with the same judge and key facts as the final answers, so that the pass could be credited or blamed only where it changed a verdict rather than a wording.

I wrote down five expectations before the first run. Single would land within five points of the combined knowledge base, since one decision should look like one decision. Iterate would recover the chains the combined base had lost on gpt-5, from 50 to 100, because the ontology would be asked directly and its reply would not be diluted by document chunks. Verification would help on chains and sets, might hurt through the ontology agent’s variance, and would net between zero and five points with at least one question made worse. Cost would roughly double against the combined base. And the prose questions that had failed with an error in the tenth article would be answered from the documents, because the fallback now existed. One of the five held outright, the cost; one held on two models of three, the fallback; one held on one model of three, the single call; and two failed on every model, the chains and, only after the drafts were judged, the verification. The failures are the interesting part.

What happened

Read the left panel of Figure 3 against the first row of Table 1. The combined knowledge base scored 78, 89 and 83 on the three models. The best orchestrator topology on each model scored 94, 91 and 97: verify (rows) on gpt-4.1, verify (rows) on gpt-5-mini, verify (instruction) on gpt-5. Sixteen points on gpt-4.1, two on gpt-5-mini, fourteen on gpt-5. The agentic hybrid over Neo4j, the series’ reference design, an agent over a Neo4j graph and a text index, which has been the ceiling of this series, scored 91, 91 and 92 on the same questions; on gpt-4.1 and gpt-5 an agent above Microsoft’s knowledge layer finished three and five points above it, inside the band Table 2 is about to set, and on gpt-5-mini it drew level. That sentence needs its qualification immediately: the hybrid scored 50 on the five trap questions on gpt-5, where the orchestrator’s best topology scored 100, and on the sixteen questions a graph can own, the right-hand panel, the hybrid still leads, 97 against 94 on gpt-4.1 and 100 against 94 on gpt-5. The orchestrator did not out-compute the graph. It out-decided the hybrid’s agent on the questions where the hybrid’s agent reached for the graph and should have read a page.

Now read down the columns, because the four topologies do not order the same way on the three models. On gpt-4.1, single 88, iterate 86, verify (instruction) 77, verify (rows) 94. On gpt-5-mini, 83, 81, 84, 91. On gpt-5, 83, 92, 97, 92. The same code, the same prompts, the same sources, and three different orderings. Before reading any of those differences as the effect of a topology, read Table 2, which is the table this article was not supposed to need.

Every verification arm begins with the iterate procedure and only then checks the draft, so each model has three runs of exactly the same code before any verification: the iterate arm itself and the drafts of the two verification arms. Judged with the same judge, those three runs scored 86, 86 and 94 on gpt-4.1, 81, 86 and 89 on gpt-5-mini, and 92, 94 and 92 on gpt-5. Six, eight and two of the 32 questions changed verdict between the three runs. Nothing in the procedure differs; what differs is what the Fabric ontology agent returned to the same question on a different afternoon. An eight-point spread on gpt-4.1 and gpt-5-mini is as large as two thirds of the pairwise differences between topologies in Table 1 on those two models, and it is the number to hold in mind for everything that follows: on one run of 32 questions, a topology difference under eight points on those two models is not a measurement of the topology.

Some differences survive that test. All six iterate-phase runs on gpt-4.1 and gpt-5 finished above the combined base, by eight to sixteen points on gpt-4.1 and by nine to eleven on gpt-5; on gpt-5-mini none of the three finished above it, and the base’s 89 stands. Single sits at 88, 83 and 83, within the spread of the iterate runs on two models and below it on gpt-5. The rest of this article is about which of the remaining differences are the decision’s and which are the source’s, and the activity logs and the judged drafts are what tell them apart.

The one decision

The single topology is the cleanest measurement in the article, because the agent does exactly one thing the knowledge base’s planner also does: it chooses a source. And it chose the same way on all three models. Every page question went to the documents, all eight, on every model. Every lookup, chain, set, absence and definition went to the ontology, sixteen of sixteen, on every model. The mixed and trap questions split: the mixed two to one towards the ontology on gpt-4.1 and gpt-5 and two to one towards the documents on gpt-5-mini, the traps three to two towards the ontology on gpt-4.1 and gpt-5-mini and four to one on gpt-5. The ontology was asked first on 21, 20 and 22 of the 32 questions. The tenth article found the combined base’s planner on gpt-5 at medium effort calling the ontology on 24 of 32; the agent’s one decision looks like the planner’s one decision, as expected.

The scores did not look the same, and the first expectation held on one model of three. On gpt-5, single scored 83, exactly the combined base’s 83, with nine questions differing in both directions and a net of zero. On gpt-5-mini single scored 83 against the base’s 89. On gpt-4.1 it scored 88 against 78, ten points above, and the four questions it gained show what one clean call buys: the PCI question and the SEV1 list went from incorrect to partial, the alerting question that the combined base had lost to a failed Fabric call went to the documents and was answered, and the retention question came back from the ontology with all five data stores and the note that the analytics warehouse was excluded because its detailed rows are kept for three years. The combined base on gpt-4.1 had lost two trap questions by sending them to the ontology and nowhere else. The agent sent one of them, the alerting question, to the documents; the other, the open-items question, it still sent to the ontology, and lost.

Where single loses, it loses for a reason that defines it, and this loss repeats on every run. The three mixed questions, which need a runbook and a blast radius, or a postmortem and a dependency list, scored 50, 33 and 50 with one call on the three models, and 83 to 100 in eight of the nine iterate-phase runs, 67 in the ninth, because one call cannot serve two halves and two can. On gpt-5 the double-charge question went to the ontology, which returned the root cause and all eight dependent services correctly and then, asked what the postmortem recommended, offered an architecture decision about connecting two acquirers; the recommendation lives in the postmortem, which is prose, and the agent had no call left. On gpt-5 the two postmortem traps went to the ontology, the ontology said it does not model action items, and the agent, allowed no second call, answered that the count was unconfirmed and offered, in its own words, to check the runbooks and postmortems in the documents if asked. That is the honest answer with the right next step in it, scored as a miss, and it is why single’s trap score on gpt-5 is 60 where the combined base’s was 100 and every iterate-phase run’s was 100. On gpt-5-mini the same question drew an answer that announced a document search it was not permitted to make. The prompt’s rule to use the other tool when one refuses needs a second call to act on.

And the chains, the shape the series is about, scored 100, 67 and 67 with one call. On gpt-5 the card-vault-db question came back from the ontology with six customer-facing services, two of which are not affected, reached through the merchant API along a path the dependency data does not support. The ontology alone at medium effort on gpt-5 had answered the same verbatim question with the right four in the tenth article. One call, one answer, and the answer is whichever the ontology agent produces that time.

Asking again

Iterate lets the agent read what came back and ask again, and the agent used that on a quarter of the questions on gpt-4.1 and gpt-5 and on half of them on gpt-5-mini: 1.25, 1.56 and 1.22 calls on average, both sources consulted on 8, 14 and 7 of the 32. On gpt-5 the second call was the documents after the ontology in six of the seven two-call runs, to fetch the prose half of a mixed question or to fall back after a refusal; on gpt-4.1 the order split five to three the other way. gpt-5-mini asked a third question twice, and on the card-vault-db chain, after its first call timed out, it asked the ontology twice more and decomposed the chain by hand: which services depend directly on the data store, then which services depend on that service at any depth, and it assembled five customer-facing services from the two replies, one too many, where the ontology’s own transitive answer would have given four.

The fifth expectation held on the two gpt-5 models, and it is the clearest payoff in the article, because it repeats. The two postmortem trap questions, open action items and alerting action items, had failed in the tenth article whenever the ontology was the only source asked, because the Fabric agent’s honest refusal came back as an error. With iterate on gpt-5, both went to the ontology first, the ontology replied that it does not model action items, the reply reached the agent as a tool result, the agent asked the documents the same question, and both were answered correctly: eleven open items: four for Payments Core; two each for Merchant Experience and for Risk and Fraud; one each for Platform, for Checkout and for Ledger and Payouts. The same two questions were answered the same way in both verification arms on gpt-5, and correctly in all three iterate-phase runs on gpt-5-mini, five times through the fall-through and once because the agent sent the alerting question to the documents directly. gpt-4.1 lost the open-items question in all three of its runs: in two it read the refusal and stopped, “unconfirmed”, with three calls still available, and in the third it did fall through to the documents and counted nine open items where there are eleven. The fallback exists in the harness now, and the two gpt-5 models used it every time.

The mixed questions, which single could not serve, scored 100, 83 and 100 with iterate. On gpt-5 the double-charge question went to the ontology for the root cause and the eight dependents, then to the documents for the three action items, and the judge found every key fact. That is the orchestrator doing what the tenth article said the combined base did not: consulting both sources for one question and composing them.

The second expectation failed, and it failed the same way on every model. I had written that iterate would recover the chains, from 50 to 100 on gpt-5. The chains scored 67, 0 and 50 with iterate, against 100, 67 and 67 with single, on gpt-4.1, gpt-5-mini and gpt-5, and 67 to 100 across the other iterate-phase runs. Nothing in the topology caused any of that. On the events-stream question the ontology told gpt-5 that no modelled service by that name exists and nothing depends on it; the agent, correctly, went to the documents, which returned the seven services whose pages mention the stream, five readers and two writers, and the answer stopped there, ten short. gpt-4.1 got the same reply, did not go to the documents, and answered that nothing is affected. The ontology’s reply was the one the tenth article had captured on the gpt-5-mini base and called variance: events-stream is a data store, not a service, and when the Fabric agent looks for it in the service table it finds nothing and says so. In another run of the same procedure on the same gpt-4.1 base it returned all seventeen. Asking again cannot fix a wrong first answer when the second source cannot compute the chain either; the documents name the first hop and the hybrid of the earlier articles follows the graph. Iterate on the chains measured the ontology agent’s variance, not the topology.

The verification, twice

The verification pass is the part of this article I expected least from and learned most from, and it has to be told in the order it happened, including the part where I had read it wrong.

The first design sends the draft and the check instruction to the ontology knowledge base as one tool call, and lets the knowledge base’s planner do what it does with any input. On gpt-4.1, where the first results came in, the score fell from iterate’s 86 to 77. The activity logs showed why the check was not a check. The knowledge base’s planner, which rewrites the caller’s input into the query it sends to Fabric, kept the instruction and the draft in ten of the 26 calls that reached Fabric, and in the other sixteen sent the original question, in fourteen of them verbatim; about 100 characters reached Fabric at the median from an input of about 750. The verification, more often than not, was a second independent ask of the ontology, and the agent then reconciled a document answer with an ontology answer under no rule at all.

That is the moment the fourth topology was added. If the planner between the agent and the ontology rewrites the instruction, take the planner’s rewriting out of the design: ask the ontology the original question, which the tenth article had shown the planner passes verbatim, and put the reconciliation rule in the agent, where I control it. Verify (rows) was pre-registered on that evidence, with the expectation that it would match or beat iterate on chains and sets and lose nothing on pages, before the gpt-5 family’s verification runs were read.

Then the gpt-5 family ran, and the planner behaved differently. On gpt-5 the knowledge base’s planner forwarded the instruction and the draft to Fabric in 31 of 32 verification calls, with 689 characters reaching Fabric at the median; on gpt-5-mini, in 26 of 28. The rewriting that had broken the first design was the gpt-4.1 planner’s habit, not the knowledge layer’s. And when the instruction reached the Fabric ontology agent, it did what the instruction asked. On the events-stream chain, where the first call had said no such service exists and the agent had drafted “0 services are shown as affected”, the check came back: “The draft is contradicted by the ontology and backing tables. … events-stream exists, but as a DataStore, not as a Service. … 7 services depend on it directly. 10 more services are affected indirectly through recorded service-to-service dependencies. Total affected services: 17.” The agent revised to seventeen, seven direct and ten indirect, and the judge found the reference exactly. The same source, asked twice, the second time with its own first answer in front of it, caught its own miss.

At that point the scores read 94 for the rows design on gpt-4.1 and 97 for the instruction design on gpt-5, the two best numbers in the article, and I had a draft of this section that credited the verification pass with both. The harness records whether a pass changed the draft, and it did, on 107 of the 192 verification runs. What it records is a text comparison. When I judged the drafts themselves, the picture was different, and Table 5 is that picture.

In 192 verification runs the pass changed a verdict thirteen times: five up and eight down. The instruction design on gpt-4.1 raised two and lowered six. The two it raised are real: the Restricted-data definition, where the draft had counted auth-service twice and the check brought it back to seven, and the Primary Acquirer question, where the second ask returned the four affected services the first had padded to six. The six it lowered are the design’s cost. Three are page questions and one the SEV1 list, where a correct document answer was trimmed to the ontology’s view, the rationale dropped from the architecture decision, the open action items dropped from the incident list. The other two are chains, and they are the variance of the source entering through the second ask: on the merchant-db question the first call had returned four teams and the second, forwarded without its instruction, returned two, and the agent followed the second; on the events-stream question the first call had returned all seventeen services and the second returned “no such service”, and the agent followed that too. The instruction design on gpt-5-mini raised one and lowered two, both pages, both trimmed. The instruction design on gpt-5 raised one, the events-stream chain quoted above, and lowered none in 32.

The rows design, the one built for the gpt-4.1 planner, changed one verdict in 96 runs, the merchant-db chain on gpt-5-mini, where the rows added two Checkout services the draft had left out. It reworded 60 drafts and lowered none. The 94 on gpt-4.1 that I had been ready to credit to it is the score of its iterate phase before the pass; the pass added nothing and took nothing away. The eight points between it and the iterate arm are the eight points of Table 2.

So the two designs separate cleanly once the drafts are judged, and not along the line I had drawn. The rows design, with its explicit rule and the planner out of the loop, is safe and nearly inert: it keeps what the documents supported, it adds what the rows contain, and in these 96 runs the rows almost never held a fact the draft lacked. The instruction design is the one that does something, in both directions, and what it does depends on what the planner forwards. Where the planner strips the instruction, it is a second ask, and a second ask of a varying source overwrites right answers as readily as wrong ones: minus nine on gpt-4.1. Where the planner forwards the instruction, the Fabric ontology agent reads the draft against its data and replies to the draft, and in 32 runs on gpt-5 that reply changed one verdict, the right way, and harmed nothing. One catch in 32 is not the fourteen points I had first attributed to it. It is also the only run in the whole experiment in which the source, shown its own draft, contradicted it on a chain, and that it can is new.

Where the instruction did reach Fabric and the draft was already right, the check said so: on the card-vault-db chain it confirmed four services and their hop counts, three and four, and on the PCI question it confirmed the three services the first clause gives and listed nothing missing but three non-cardholder data categories, the same one-clause reading the tenth article found in eleven of twelve ontology runs.

That also settles the third expectation, which I had written as a net gain of zero to five points with at least one question made worse. Judged on the drafts, the rows design netted zero, one and a half and zero on the three models with no question made worse, and the instruction design netted minus nine, minus one and a half and plus three, with questions made worse only on the two models where the net was negative. The range and the “one worse” never held together, and the expectation failed as written on every model.

The wall, question by question

Lookups and absences are correct in all twelve cells; the ontology owns them and the agent sends them there. Pages are correct in every run of single, iterate and verify (rows) on every model but one, a partial on gpt-5-mini’s iterate; the three page misses on gpt-4.1 and two on gpt-5-mini all sit in the instruction design, where the check trimmed a correct document answer. Traps reached 100 on gpt-5 with every topology but single, and on gpt-5-mini with verify (rows), against the hybrid’s 50 on gpt-5: the retention and recovery questions from the ontology in one call, the two action-item questions from the documents, on gpt-5 always after the ontology had declined or returned nothing, and the rollback question sent to the documents directly.

The three chains are the column to read, with Table 2 beside it. On gpt-5: 50 for the combined base, 67 single, 50 iterate, 100 verify (instruction), 67 verify (rows), 100 hybrid. Every orchestrator answer to a chain on gpt-5 began with the ontology’s reply, taken, trimmed or checked; the only run that got all three was the one where the ontology was asked to check its own draft and did. On gpt-4.1 the chains were 100 with single and 100 with verify (rows), 67 with iterate and 33 with the instruction design, and the judged drafts say the 33 was 100 before the pass. The merchant-db question itself is the steadiest of the three: correct in ten of twelve orchestrator cells, with the ontology on gpt-5 reporting the transitive closure, nine services, four teams, in one call.

Two questions came back whole once each in twelve cells, and they are the tenth article’s two. The PCI question, whose rule has a join in it, was read one clause at a time in all four topologies on gpt-5, in two on gpt-4.1 and in one on gpt-5-mini, three of the five services; four runs widened the first clause to every Restricted data store and listed seven or thirteen services, scored wrong. And once, on gpt-5-mini’s iterate, on a second call after the first had timed out, the Fabric agent wrote back that it had applied the PCI scope rule, a service that depends on a cardholder data store or directly on a service that does, and returned five, three direct and two indirect, which no ontology run in the tenth article had managed. The rule with a join can be followed; on one call in twelve it was. The Platform-team set came back as two of four in every run but the one on gpt-4.1 where the Fabric agent widened “component” to a service or a data store of its own accord, on the first call of the rows arm. An agent above the sources cannot make the ontology beneath it read a rule in full; it can only ask again, and whether the next reading is the full one is the source’s variance. Both questions fail on translation decisions in the ontology I wrote, and both are mine.

What it costs, and how long it takes

The fourth expectation held. Single costs about what the combined base costs, 1.32, 0.44 and 1.54 cents a question against 1.04, 0.44 and 1.41, because one retrieval is one retrieval and the agent’s own tokens are a third to a half of the bill. Iterate adds a quarter to a half. The verification designs roughly double it: verify (instruction) 2.49, 0.90 and 3.44 cents, verify (rows) 2.29, 0.77 and 2.49, with the instruction design’s extra ontology call on every question and the rows design reusing the iterate phase’s rows on 20 to 26 of the 32. On gpt-5 the 97% answer costs 3.44 cents a question, two and a half times the combined base and three and a half times the hybrid’s 1.00, and Table 5 says what the second ontology call bought: one verdict. On gpt-5-mini the 91% answer costs 0.77 cents, a little over half of gpt-4.1’s single call. The whole experiment, 384 runs, cost $6.40 in model and retrieval tokens at list prices: $2.49 on gpt-4.1, $0.88 on gpt-5-mini, $3.03 on gpt-5; the 192 draft judgements made afterwards are not in that figure. The Fabric capacity was the same trial as the tenth article’s and cost nothing.

Time is the harder number. A successful ontology call took 21, 28 and 25 seconds at the median on the three models, a document call 3, 7 and 9, and each topology’s median follows from the calls it makes and the agent’s turns between them: single 19, 27 and 26 seconds, iterate 23, 37 and 27, verify (rows) 26, 39 and 41, verify (instruction) 51, 81 and 62. The events-stream chain that verify (instruction) got right on gpt-5 took 103 seconds, 49 for the first call and 43 for the check. The hybrid answers in four to seven. The orchestrator is a design for an analyst’s question or a ticket, not a chat turn, and the second ontology call is where the minute goes.

What this changes, and what it does not

The tenth article said the knowledge layer had the sources and lacked the decision. This article measured the decision in four topologies, judged its drafts as well as its answers, and four things follow.

The decision adds what it is built to add, and that part repeats. An agent above two knowledge bases composes them for a question that needs both, which a single call and the combined base’s planner did not do: the mixed questions went from 50, 33 and 50 with one call to 83 or better in eight of the nine iterate-phase runs. It falls back to the documents when the ontology says it does not model something, which turned the two honest refusals of the tenth article into two correct answers on both gpt-5 models in every run that had a second call to make. And its first call went to the ontology on every graph-owned question in every one of the twelve cells, and with one call every page question went to the documents on every model, which is the routing the retrieval instruction had asked the planner for. On gpt-4.1 and gpt-5 every iterate-phase run finished above the combined knowledge base. None of that depends on which afternoon the run happened.

The source’s variance is the largest term in the measurement, and it is not the topology’s to fix. The identical procedure run three times moved by eight points on gpt-4.1 and gpt-5-mini, as much as two thirds of the differences between topologies on those models, and the chains, the shape the ontology solved in the tenth article, are where the movement sits: seventeen services or none, three, four or six, from the same base on the same question. The tenth article measured that variance in the ontology alone and this article sees it through every layer above. An architect assembling this stack should measure it first, with more than one run, before choosing a topology, because a one-run comparison on this source does not resolve differences inside that band.

The verification pass is real and small, and which design to run depends on the model in the planner. Judged on its drafts, it changed thirteen verdicts in 192 runs. Where the knowledge base’s planner forwarded the instruction, which gpt-5 did 31 times in 32, the Fabric ontology agent read the draft against its data and corrected the one chain it had got wrong, with no harm elsewhere; that is a new capability of the ontology, demonstrated once. Where the planner stripped the instruction, which gpt-4.1 did more often than not, the pass became a second ask and lost nine points by importing the source’s variance into answers that were already right. The rows design, with the planner out of the loop and an explicit rule in the agent, lowered no verdict in 96 runs and raised one; it is the safe default and it is nearly inert. The seam the tenth article named, the connection between the source that computes and the layer that asks it, is still the seam, and it has a model in it.

And the hybrid still leads on the sixteen questions a graph can own, by three points on gpt-4.1, three on gpt-5-mini and six on gpt-5, for the two translation reasons the last article gave, and it still answers in a seventh to a ninth of the time of the best topology on each model. On the whole set the gap has closed on two models, and the component that closed it is one that Foundry Agent Service is built to host: a knowledge base per source, one tool per knowledge base, an agent that calls them. What is mine is the prompts, the reconciliation rule, and the measurement.

The honest caveats

Thirty-two questions, one run per cell, one judge, and a source whose answer to the same verbatim question varies between calls, which the tenth article measured and this article measured again through the stack: three runs of the same procedure spread eight points on two models, and the events-stream chain was answered with seventeen services or none, and the card-vault-db chain with three, four or six, by the same knowledge base in different runs. I first attributed the verification gains to the verification and was wrong; the 192 drafts were judged afterwards with the same judge, and every claim about the pass in this article rests on a verdict change, not on a text change. The fourth topology was added after the first model’s results were read; the reason and the date are in the outline, and the gpt-4.1 result of the first design is reported rather than replaced. The judge is gpt-4.1 with fixed key facts; it scored an honest “unconfirmed” with the right next step as incorrect, which is the right call for a measurement of answers and a harsh one for a measurement of agents, and it is the same judge for the drafts and the answers, so a verdict change is at least a consistent reading; across models it is not always one, since the same two-of-four answer to the Platform-team set was scored partial on gpt-5 and incorrect on the other two models, as it was in the tenth article, which is the whole of the gap between gpt-5’s set score and theirs in Figure 4. The ontology calls that failed, between one and twelve per arm depending on the model and the topology, are counted in Table 3: the ones that failed with the parsing error of the tenth article were returned to the agent with the Fabric reply inside, and the few that timed out or lost the connection were retried, with the time of the retries included in the seconds reported. The agent’s tools called the REST retrieve operation rather than the MCP endpoint, for the activity log; the MCP tool was checked by hand on both bases and returns the same reference. The orchestrator’s prompts are mine and a different prompt would move these numbers. The ontology is my translation of my world model, and the two questions that came back whole only once each are translation decisions, not product limits.

Closing remarks

Every article in this series has asked what a company’s data has that an AI should know about, and every measured answer has been a shape: the questions that need an operation are the ones retrieval cannot answer, and a structure that can compute is what answers them. The previous article was the first time that structure came from Microsoft’s own knowledge layer. This one is the first time the decision above it did too, and the decision earned its place on the questions that need two sources and on the refusals that need a second call. On the chain it mostly measured the source.

What is left is the same seam, one layer up: the planner between an agent and a source, what it passes through and what it rewrites, and the fact that the answer depends on which model is running it and on which call it is. Both are small things to measure and large things to assume, and both are where I would look next, with more than one run.

Everything in the data is invented. Naissus Pay, its people, systems, incidents and policies were written for this series. The evaluation code, the orchestrator, every recorded run with its tool calls and activity logs, the judged drafts, every cost, and the MCP captures are kept with the project.

References

• The tenth article of this series, “Give the Knowledge Layer Something to Compute: Foundry IQ and the Fabric IQ Ontology, Measured”, AI Advances, October 2026.
• The seventh and ninth articles of this series, “Microsoft’s Knowledge Layer, Measured” and “How Much of the Wall Was the Model?”, AI Advances, October 2026.
• Microsoft Learn, “Connect to a knowledge base from Foundry Agent Service” (the MCP endpoint of a knowledge base and the knowledge_base_retrieve tool), "Agentic retrieval in Azure AI Search", "Knowledge sources overview", 2026.
• Microsoft Learn, “Expose an ontology as a Foundry IQ knowledge source” (Fabric IQ, preview), 2026.
• Azure OpenAI in Foundry Models, Global Standard list prices for gpt-4.1, gpt-5 and gpt-5-mini, as recorded in the project’s pricing table on 30 September 2026.

Give the Knowledge Layer the Decision: An Agent Above Foundry IQ and the Fabric IQ Ontology… was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.

What was left to test

The seventh article ended with a wall and a promise. Foundry IQ knowledge bases over the Naissus Pay documents reached 78% on my 32 questions, and the twelve and a half points between them and the graph-plus-search hybrid were all on questions that need a count, a rule or a chain. I promised to give the knowledge layer a source that computes. The ninth article then narrowed what that source had to do: with gpt-5 or gpt-5-mini planning the subqueries, the counts, the absences and most of the definitions fell on documents alone. What survived every model, every effort and every mode was the chain of three or four hops, and the two mixed questions that contain one.

So the specification for this article is small and exact. Build the ontology in Fabric, attach it to Foundry IQ as a knowledge source, and ask the same 32 questions through the same judge, on the same three models. Then ask the question Microsoft’s design actually invites: documents and ontology in one knowledge base, with the planner deciding which source each question needs.

What an ontology item is, and what the translation cost

A Fabric IQ ontology is an item in a Fabric workspace that sits on top of lakehouse tables and says what the rows mean: entity types with properties bound to columns, relationships between them, and rules written in prose. The Naissus Pay world model that lives in Neo4j for the earlier articles was written out from the same source of truth as fifteen tables with 348 rows, loaded into a lakehouse, and described as an ontology with nine entity types, thirteen relationship types and eight rules. The seventh article showed the translation decisions in detail; Figure 1 is the same diagram, because nothing changed. The three that matter here: the data category taxonomy became two columns, parent and classification, with a rule saying categories inherit their parent’s classification; the one Cypher pattern the graph agent relied on, a variable-length dependency path, became two sentences, “Depends On Service is transitive: if A depends on B and B depends on C, a failure of C affects A. Blast-radius and impact questions must follow the chain to any depth”; and the PCI rule, which needs a store, a service and a service that calls that service, became a sentence too.

The setup itself is worth a paragraph, because it is the part a reader will have to repeat. Fabric’s API refused the personal Microsoft account the earlier articles ran under, so a work account was created in the tenant that owns the search service, given the Fabric Administrator role and multi-factor authentication, and used to start a Fabric trial capacity from the portal in Sweden Central. Those steps, and one device-code sign-in from the terminal, are the only hand work in the whole arm. From there one script did the rest through the Fabric REST API: the workspace, the lakehouse, the fifteen tables, the ontology created directly from its definition in 39 parts, and on the search service the knowledge source and the knowledge bases. The one tenant setting that gates ontology items switched on through the admin API. The ontology knowledge source needs a user token with every request, so the evaluation harness signs its retrieval calls as that user, and the knowledge base passes the token on to Fabric.

Twelve knowledge bases came out of it. Six hold the ontology alone, at low and medium effort for each of gpt-4.1, gpt-5-mini and gpt-5. Six hold the documents and the ontology together, with a short retrieval instruction telling the planner that structure, counts, ownership and dependencies live in the ontology and procedures, rationale and narratives live in the documents. The search service’s Basic tier allows fifteen knowledge bases, so four of the document bases measured in the earlier articles were deleted to make room; their results are on disk. Minimal effort, which runs no model and sends the caller’s query straight to the source, I did not run against the ontology, so there is no minimal row below.

How the knowledge layer asks an ontology

Here is the first thing the activity logs show, and it changes how to read everything after it. On the document source, the planner wrote two to five subqueries per question on average and the medium setting iterated. On the ontology source, every successful run on every model at both efforts made one call to Fabric, and the argument of that call was the question, verbatim. The two exceptions in 344 runs are gpt-4.1 on one mixed question, which it split into its runbook half and its blast-radius half, and in both runs one of the two calls failed. Otherwise the planning model did not decompose or rephrase. It handed the question over. At medium effort, in about half of the runs, it took a second planning pass after Fabric replied and asked for nothing more. The two effort settings therefore differ in what happens after the call, not in the call, and I read their scores as two readings of the same retrieval.

What Fabric does with the question is the part Microsoft has built, and I can only see its output. The ontology agent takes the question, works over the bound tables with the entity types, relationships and rules as its guide, and returns two things in one reference: a short answer in prose, and the rows it computed, as CSV. For “which three services were affected most often” the CSV was three rows: checkout-api 7, payments-api 5, card-gateway 4. For the events-stream chain it was seventeen rows, each with a service, its depth from the stream, one to three, and whether the impact is direct or indirect: the chain the document arm could not walk, computed, with the depth shown. Capturing that reply took patience. The first capture had an example path per service; a later test overwrote it, eight attempts to repeat it ran past the ninety-second limit, which is its own data point, and the first one to complete, on the gpt-5-mini base, returned a different answer altogether, that no modeled service named events-stream exists and nothing depends on it. The seventeen-row table came back the next evening from the gpt-5 base. Both replies are on disk, and the second matters for what follows.

It is also not quick. The Fabric call took 24 to 34 seconds at the median depending on the model, up to 66 seconds, and Azure AI Search puts a ninety-second limit on the whole retrieval. A few runs hit that limit, a few more lost the connection to Fabric mid-call, and all were rerun; one chain question on gpt-5-mini needed three attempts before a request completed, at 79 seconds, and in that request the Fabric task had failed inside it and the documents answered. The agentic hybrid of the earlier articles answers the same question in four to seven seconds, and the document knowledge base in five to fourteen.

What happened

Read the right-hand panel first, because it is the one this article was written to produce. On the sixteen questions a graph can own, the ontology alone scores 78% on gpt-4.1, 88% on gpt-5-mini and 94% on gpt-5, with the knowledge base writing the answer. The document source at the same effort scored 72, 81 and 91. The ontology on gpt-5 is three points below the hybrid’s 97 on gpt-4.1 and six below its 100 on gpt-5, and it got there with no graph database, no Cypher, and one call per question.

Now read the left panel. On all 32 questions the ontology alone scores 55, 59 and 61. It answers the page questions at 19%, 25% and 19%, because it holds no prose, and it says so, which is the subject of the next section but one. An ontology is not a replacement for the documents. It was never going to be, and the number is there so that nobody reads the right panel as if it were.

The wall, question by question

Chains fell. All three chain questions are correct on gpt-5 and on gpt-5-mini with the ontology alone at medium effort. “If card-vault-db goes down, which customer-facing services are affected?” came back from gpt-5 with all four, including wallet-app-bff, which no document run on any model had ever named, and with the note that the shortest path from the database to checkout-api is three hops. The events-stream question came back with seventeen services, seven direct and ten indirect, which is the reference exactly, with the knowledge base’s answer on both models separating the services that read or write the stream from the ones reached through them. The transitive rule is two sentences of prose in the ontology. Before the runs I had written down that the chain would fail unless the planner iterated, because a sentence is not a recursive query. The planner sent one call, and the answers that came back traced the paths: on gpt-5, three hops from the database to checkout-api and four to checkout-web, payment-links and wallet-app-bff; on gpt-5-mini, the two chains written out link by link, through payments-api to checkout-api and through wallet-service to wallet-app-bff. gpt-4.1 did not get there: on events-stream it reported that no service depends on it in the modeled dependency data, which is wrong on the Depends On Data Store relationship, where the two gpt-5 replies both began by noting that events-stream is a data store; and on the other two chains it found three of the four answers. Same ontology, same verbatim question, one call each. The only thing that differs between those knowledge bases is the model Foundry IQ plans and writes with, and on this shape the result differed between a third of the answers and all of them. I cannot see inside the Fabric call to say how much of that is the model and how much is the agent’s own variance, which the next section but one measures; the capture made later on the gpt-5-mini base, which returned gpt-4.1’s answer to the same verbatim question, says at least part of it is the agent.

Sets and absences held, with one exception I had not predicted. Both absence questions are correct on every model and at every setting. The top-three count and the severity breakdown are correct on every model at medium effort. I had written down that sets would reach 80% or more; they did on gpt-5 only, and the reason is the question that follows.

The one set that stayed short is “which incidents in 2025 had their root cause in a component owned by the Platform team”: two of four on every model, and the reason is in the translation, not the agent. The root-cause component became a text column on the incident, and the agent joined it to services and their owners. Two of the four incidents were caused by a session cache and an event stream, which are data stores, also owned by Platform, and the column cannot say which table a name belongs to. The graph has a single Component label for both. The ontology I wrote does not, and the agent could not know.

Definitions split exactly as I had written down before the runs. “Which data stores hold Restricted data, and which team owns each?” is correct on every model at medium effort, through the classification column and the ownership relationship. “Which services depend directly on a store holding Restricted data” is correct on every model too, all seven services, where the document source on gpt-5 had found five. And the PCI question never came back right. The rule has two clauses: a service that touches a cardholder data store, and a service that directly calls one that does. In eleven of the twelve ontology runs the agent named the three services the first clause gives, analytics-pipeline, card-vault and reconciliation-job, and the twelfth named two of them; at its best it stopped there, noting correctly that services which only see tokens or masked numbers are out. Three runs at low effort widened the first clause to every Restricted data store and returned seven services. One run on gpt-5-mini opened with “I don’t have access to the internal ontology tool right now” and listed twelve from what it called the last available information. Card-gateway and payments-api, the two services the second clause adds, appeared only in that last list. Never the five, exactly. A rule with a join in it, written in prose, was read one clause at a time. The document source on gpt-5, which decomposed the rule into ten searches, got all five. The hybrid, whose graph agent runs the rule as a Cypher pattern, got all five on every model.

Pages and traps. The ontology cannot answer a page question, and it mostly does not try: the page score is a few partial credits where an incident’s date, severity and duration happened to be in the tables, plus the vault-decision question, which gpt-5-mini answered correctly three times from the decision’s title and what the model knew about tokenization. Two of the five trap questions turned out to be in the tables after all, retention periods and recovery time objectives on the data store entity, and the agent answered them; the other three, open action items, alerting action items and rollback procedures, are not modelled, and the agent said so in every run that returned at all, which brings me to the next section.

When the ontology says no

Forty-seven of the first 512 runs returned no answer at all, and two more in the combined bases that came later, and they are all on the same five questions, the ones whose answers live only in prose. In forty of them the ontology agent replies honestly: “I can’t compute this from the current ontology because postmortem action items and their open/closed status are not modeled”, or “the available run book data only includes the run book name and service, not the procedure text itself”. It is the right reply. The agent knows what it does not hold. In the other nine, all on the key-rotation question, the reply is a generic assistant guardrail, “I can help only with Azure-related topics. Please submit a new request.”, which is a different kind of miss, and I record it as what it is. Either way Azure AI Search could not parse what came back, sometimes a JSON object with an empty CSV and a conversation identifier appended, sometimes bare prose, and when the ontology was the only source the request had asked, it failed the request with an HTTP 502 and the message “unexpected error while parsing the response from Fabric Ontology”. The failure is frequent rather than certain. The open-action-items question failed in all twelve ontology-only settings and the key-rotation question in eleven; the other three failed in four to seven of the twelve and otherwise came back as an ordinary answer that said the data was not modeled, scored incorrect.

I have scored the failed runs as no answer, which is what the caller received. The honest reply exists; it is in the error text, as Table 3 shows. What the single-source case is missing is the step that would turn “I don’t model that” into an empty reference so the knowledge base could say so itself. The combined knowledge bases show that step already exists when there is somewhere to fall through to: in sixteen of their runs the Fabric task failed in the same way or lost its connection, the request came back as partial content with the document results, and the knowledge base answered from those. The open-action-items question was answered correctly that way on both gpt-5 models at medium effort, with the ontology’s error sitting in the activity log beside the document searches. The two combined bases on gpt-4.1 lost that question and the alerting question only because their planner had sent both to the ontology and nowhere else. The product handles the refusal when it has a second source. It is the lone-source case, which is exactly the ontology-only knowledge base, that turns an honest answer into an error, and that is a small fix with a measured payoff.

Documents and ontology together

This is how Microsoft intends Foundry IQ to be used: several sources behind one knowledge base, the planner routing between them. The scores are 84, 72 and 78 at low effort on gpt-4.1, gpt-5-mini and gpt-5, and 78, 89 and 83 at medium. Against the document base alone at medium, 78, 86 and 89, the combination gains three points on gpt-5-mini and loses six on gpt-5. Against the ontology alone on the graph-owned sixteen, the combination on gpt-5 falls from 94 to 78. The two sources that each solve half of the problem did not add up, and the activity logs say why.

The planner routes, but it mostly does not combine. At low effort gpt-4.1 sent 18 of its 30 completed questions to the ontology, and in all 18 cases it sent nothing to the documents. It kept every page question on the documents, which is exactly what the retrieval instruction asked for, and it is why gpt-4.1’s combined base at low effort scored 84%, six points above the best document base on that model, with the chains at 100% and the pages at 100%. gpt-5 at low effort sent 27 of 30 to the ontology, including five of the eight page questions, and 22 of those 27 went nowhere else. At medium effort gpt-5 called the ontology on 24 of 32 and the ontology only on 17. When the ontology is the only source consulted, the answer is whatever the ontology agent returns on that call, and that is where the second thing the logs show comes in.

The ontology agent is not deterministic. Across the twelve ontology-only runs per question, three models by four settings, fifteen of the 32 questions received genuinely different answers, and four more differed only in whether the refusal came back as an error or as an answer; the Fabric call is the same verbatim question every time. In the combined base on gpt-5 at medium, the card-vault-db chain came back without wallet-app-bff, the merchant-db chain without one of its four teams, and the PCI question with thirteen services, because on that call the agent counted token and masked-card categories as cardholder data. The same three questions had been correct, correct and three-of-five on the ontology alone. And where the planner did send subqueries to both sources, a third thing happened that the scores alone hide: the Fabric task failed inside an otherwise successful request, the knowledge base answered from the documents, and the verdict is the document verdict. That is what the events-stream chain on gpt-5 at medium is, five direct consumers and two producers from the pages, with the ontology’s parse error in the activity log; on gpt-5-mini at medium the same tolerated failure sits behind both chains, the PCI question and one mixed question. A combined base at medium effort is therefore three measurements in one: the planner’s routing, the ontology agent’s variance when it is the only source, and the document answer standing in when the ontology call fails.

Where the combination did add, it added what each source could not do alone. On gpt-5 and gpt-5-mini at medium effort the combined base scores 100% on the five trap questions, and the activity logs show which source did what: retention and recovery objectives came from the ontology alone, with one call each, and the action items and rollback procedures from the documents. Only gpt-5-mini’s document grounding with my own prompt had managed 100% on the traps before. gpt-5-mini’s combined base at medium, 89% for 0.44 cents a question, is level with the gpt-5 document base at 2.51 cents, and three points above its own document base. For gpt-4.1 the best combination was the grounding with my own prompt at low effort, 88%. The combination is the right idea. The routing is one decision per question, the sources are mostly not consulted together, the ontology’s variance decides the outcome when it is consulted alone, and when its call fails the documents quietly take over, which is both a safety net and a reason to read the activity log before trusting a number.

What it costs, and how long it takes

The ontology is cheap in tokens and expensive in time. One call, one reference: 0.50 cents a question on gpt-4.1, 0.11 on gpt-5-mini, 0.62 on gpt-5 at medium effort, against 1.95, 0.73 and 2.51 for the document base, whose planner reranks fifty chunks or more. The combined bases sit between, 1.04, 0.44 and 1.41. The whole arm, 608 final runs across twelve knowledge bases, cost $3.13 in model and retrieval tokens at list prices; the attempts that failed and were rerun are not in that figure, since a failed call reports no usage. The Fabric capacity was a trial and cost nothing; the smallest paid capacity, an F2 in Sweden Central, lists at $0.38 an hour and can be paused.

Time is the other axis. The document base answers in 5 to 14 seconds at the median depending on the model, the hybrid in 4 to 7. The ontology base answers in 24 to 34, and the combined base in 28 to 35, because the Fabric call sets the floor and nothing else runs long enough to hide it. For an analyst’s question that is fine. For a chat surface it is a different product, and the ninety-second ceiling is close enough that a long chain on a slow afternoon will hit it.

What this changes, and what it does not

The seventh article called the wall a computation problem. The ninth found that most of the wall was the planner. This one finds the remainder is exactly where I had left it: the chain, and the ontology computes it. Three things follow.

An ontology source does what retrieval could not. On the sixteen questions that need an operation, it reaches 94% on gpt-5 with one call per question at a quarter of the document base’s cost. The transitive rule, written in prose, was followed four hops deep. The one rule with a join in it was followed to its first clause only, and the one translation decision that flattened a polymorphic relationship into a column cost two incidents in one set. Those are the two places where the ontology I wrote is weaker than the graph I wrote, and both are mine to fix.

The knowledge layer is the part that is still being built. One verbatim call per question, with no decomposition and a second pass that asks for nothing; an honest “I don’t model that” turned into a failed request when the ontology is the only source; routing that chooses a source rather than combining them, and a document answer that quietly stands in when the ontology call fails; a half-minute floor and a ninety-second ceiling. None of this is in the ontology. All of it is in the connection between the ontology and the knowledge base, and all of it is the kind of thing a preview is for. The document source went through the same stage: the seventh article measured it, the ninth found the planner mattered more than the setting, and this article’s combined base on gpt-5-mini already matches the best document result at a sixth of the cost.

And the hybrid still wins, for a reason I now understand better. It is not that a Neo4j graph knows more than a Fabric ontology; they were written from the same world model and the ontology answered the chain the hybrid answers. It is that the agent in front of the graph decides per question whether to run a query, read a page or both, and runs a Cypher pattern when the question has one, in four seconds. The knowledge layer has the sources now. What it does not yet have is that decision.

The honest caveats

Thirty-two questions, one run per cell, one judge, and this time the runs are noisier than before: the ontology agent returned different results to the same verbatim call often enough that fifteen questions had more than one answer across twelve runs, and I report the medium-effort synthesis cell throughout because it is the cell the earlier articles reported, not because it was the best. The judge is gpt-4.1 with the same key facts as every earlier article; it scored an identical two-incident answer as incorrect on two models and partial on the third, which is a reminder that a verdict is one reading. Forty-nine runs returned no answer because of the parsing failure described above and are scored as zero; if that failure were fixed, the page and trap scores of the ontology base would not move, since the agent had no answer to give, but the two combined bases on gpt-4.1 would recover the prose questions they lost. Sixteen combined runs carry a failed Fabric task inside a successful request and are scored on the document answer they returned. The ontology is my translation of my world model, and two of its remaining misses on the wall are translation decisions, not product limits. The model inside Fabric’s ontology agent is Microsoft’s choice and is not reported; the models I varied are the ones in the knowledge base. The Fabric side ran on a trial capacity over two days; a paid capacity could be faster or slower. Everything is one invented company. No Microsoft team was involved in, informed of or consulted on this work, nothing was sponsored, and every run was paid for on my own subscription.

Closing remarks

This series began with the question of whether a company’s data has a shape its AI should know about. The answer has been the same measured number for a while: the questions that need an operation are the ones retrieval cannot answer, and a structure that can compute is what answers them. This article is the first time that structure came from Microsoft’s own knowledge layer rather than from a graph database I ran myself, and on the shape that mattered most, it worked. The wall that stood through the series fell to an ontology and a rule in plain words that said the relationship is transitive.

What is left is not the wall. It is the seam between the source that computes and the layer that asks it, and that seam is where the next measurements belong.

Everything in the data is invented. Naissus Pay, its people, systems, incidents and policies were written for this series. The evaluation code, the ontology definition, every recorded run, every cost, and every raw reply from Fabric that is quoted here are kept with the project.

References

• The seventh article of this series, “Microsoft’s Knowledge Layer, Measured”, AI Advances, October 2026.
• The ninth article of this series, “How Much of the Wall Was the Model?”, AI Advances, October 2026.
• Microsoft Learn, “What is Fabric IQ?”, “What is ontology (preview)?” and “Ontology definition” (the TMDL parts of an ontology item), 2026.
• Microsoft Learn, “Create a Fabric Ontology knowledge source”, “Agentic retrieval in Azure AI Search”, “Set the retrieval reasoning effort”, 2026.
• Azure pricing for Microsoft Fabric capacity, F2 pay-as-you-go in Sweden Central, read 21 October 2026.
• Azure OpenAI in Foundry Models, Global Standard list prices for gpt-4.1, gpt-5 and gpt-5-mini, as recorded in the project’s pricing table on 30 September 2026.

Give the Knowledge Layer Something to Compute: Foundry IQ and the Fabric IQ Ontology, Measured was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.

The claim I never tested

Every company I have talked to about this series asks the same thing, usually in the first ten minutes: “We do not have a clean catalogue. We have a wiki. Can the model just read it and build the graph?”

The honest answer in the first article was a rule of thumb. Load structure from the systems that already hold it, because that is a sync job. Do not extract it, because that is a model reading every page, and the first article quoted the going rate for that: Microsoft’s own worked example spends about two million model tokens to index a 55,000-token book. I believed it, the literature says it, and nothing in this series had tested it. The graph behind every measurement so far was loaded from the world model in under a second, with zero model calls.

This article runs the other route. A model reads the same 105 pages that Classic RAG searches, with the ontology as the target schema, and builds a graph. Then the extracted graph takes the loaded graph’s place under the same two designs, the same 20 questions and the same judge. Because the loaded graph exists, I can also do something a team extracting from a real wiki never can: compare the two graphs node by node and edge by edge, and say exactly what the model missed, what it invented, and which wrong answers each mistake caused.

Two routes to the same graph

The extraction is deliberately plain, because that is what a team would build first. One model call per page. The system prompt is the ontology from the fifth article, extended with the action items, procedures and recovery targets of the second, so the target schema is the full one: 17 classes, 16 relationship types, their properties and the business definitions. The model must answer in a strict JSON schema whose labels and relationship types are enumerated, so it cannot invent a class. It names the entities it finds, marks the one the page is about, and lists relationships between them.

The per-page results are then merged by code, with no model involved: names are normalized (case, hyphens, plurals), mentions of the same thing across pages are folded into one node, properties are decided by majority vote, relationships are resolved by name, and any relationship whose endpoint does not exist is dropped and counted. That merge is where most of the work turns out to be. The 105 pages of the best run produced 643 node mentions and 565 relationship mentions, which became 228 nodes and 483 edges.

Five graphs were built this way, and a sixth was loaded.

Three of the five are the same extraction with three models, gpt-4.1, gpt-5-mini and gpt-4o-mini, because the first question anyone asks about extraction is which model to use. The fourth gives gpt-4.1 the company’s registers: the names, and only the names, of its services, stores, providers, teams, people, data categories, incidents, run books and decisions, the kind of lists an HR system, a CMDB and an incident tracker produce without anybody building a graph. The fifth is the compromise a careful team would reach for: load what the systems of record hold, and extract from the pages only what nothing else holds, the action items, procedures, recovery objectives, retention periods and service-level targets.

Ground truth for the graph comparison is the loaded graph: 227 nodes and 487 edges. Ground truth for the answers is the same computed truth as in the second article. Everything ran on gpt-4.1 at the answering end, 260 runs in all.

Result 1: a strong model and an ontology get you 98% of the graph for 59 cents

I expected extraction to lose a meaningful share of the structure. It did not. gpt-4.1, reading each page once with the ontology as its guide, found 224 of the 227 nodes and 477 of the 487 edges, invented 4 nodes and 6 edges, and got every property it extracted right: all 29 tiers, all 20 severities, all 26 dates, all 39 action-item statuses, every recovery objective and every latency target. It took 271 seconds at four pages in parallel and cost 59 cents at list prices, about half a cent per page.

The ten missing edges are instructive, because they are all the same kind of mistake. Seven are STORES edges, the link between a data store and the kind of data it holds. The store pages say what they hold in everyday words: ledger-db holds “ledger entries and the account balances derived from them”, loyalty-db holds “loyalty points balances”, links-db holds “payment links”. The classification policy, written by someone else, lists the same things as categories. On three of those pages gpt-4.1 extracted no category at all, because nothing in the sentence looked like one. On two it took one of the two categories the page named and left the other: the bank account numbers but not the payout instructions, the card numbers in the settlement files but not the transaction records. On wallet-db it read “wallet account balances” and coined a category called WalletAccountBalance, which exists nowhere else in the graph. In every case the fact was on the page. What the model lacked was the knowledge that these everyday phrases are the names of categories. The other three are procedures: two run book headings the model paraphrased, so that “Certificate problems” became “Check certificate expiry and renew if needed”, which is what you do rather than what the section is called, and one that my merge folded into another run book’s procedure of the same title. Nothing was misread. Things were not recognized, or were renamed.

On the 20 questions, the extracted graph and the loaded graph are indistinguishable. GraphRAG scores 72.5% on the extracted graph against 75% on the loaded one; the hybrid scores 95% on both. The single run that differs is a page question about an incident, where the judge accepted the loaded graph’s answer as partial and the extracted graph’s as wrong; both answers gave structural facts from the graph, and both ended by saying the graph does not hold the narrative. That is judge variance, not a graph difference. None of the 20 questions needed the missing edges: the seven categories all sit under Confidential or Internal data, which no question asks about, and the rollback question still found both run books because the merged procedure kept both edges. That is a point I will come back to.

Result 2: a cheap model gives you a graph that is worse than no graph

The two cheaper models cost 26 cents and 5 cents for the same 105 pages. They fail in opposite ways, and both ways end in a graph that scores no better over the 20 questions than Classic RAG with no graph at all.

gpt-4o-mini has a recall problem. It found 86% of the nodes and 78% of the edges, and the losses are not spread evenly: they sit precisely in the structure that worked-out questions need. It kept 5 of the 30 taxonomy edges, so the data classification tree that the Restricted and PCI definitions traverse has almost no branches. It kept 2 of the 23 procedures. It kept 58 of the 81 dependencies. It produced 22 relationships whose endpoints it never created, and 21 property values that could not be parsed. And it changed the tier of seven services. Five of them, card-gateway, threeds-service, ledger-service, payouts-service and fraud-scoring, are Tier 1 in the catalogue and on their own pages, and came out as Tier 2.

Every one of those holes became an answer. Asked which services depend on a store that holds Restricted data, the agent ran the right query, got zero rows, and said no service does. card-vault-db still held primary account numbers in that graph, but PrimaryAccountNumber had lost its link to CardholderData, as had every other leaf category, so the traversal down from RestrictedData reached no store at all. Asked which run books have a rollback procedure, it said none do, because the procedures were gone. The event-stream blast radius came back with 10 services instead of 17. Asked which teams own no Tier 1 service, it listed Ledger and Payouts and Risk and Fraud, because their Tier 1 services had been demoted. GraphRAG on this graph scores 52.5%. Classic RAG, with no graph, scores 60% on the same questions.

gpt-5-mini has the opposite problem. Its recall is nearly as good as gpt-4.1’s, 97% of the edges, but its precision is 92%, and the extra 8% is not noise. The company overview page lists what each team owns in prose: “card processing end to end: authorization, capture, refunds, tokenization and the acquirer connections”. gpt-5-mini turned that sentence into five services, and did the same for every team: 27 components that exist nowhere in the catalogue, most of them a team’s responsibilities read as services, almost all of them dutifully owned by their team. It also created “Checkout team”, “Wallet team” and “Platform team” as teams alongside Checkout, Wallet and Platform. So when asked which teams own no Tier 1 service, the agent found the duplicates, which own no Tier 1 service, and listed them alongside the real answer. And it placed CardToken under CardholderData in the taxonomy, although the classification policy says in so many words that tokens are deliberately not cardholder data. That one edge made payments-db a cardholder data store, and pulled payments-api and refund-service into the Restricted perimeter in the Restricted-data answer. GraphRAG on this graph scores 60%, the same as no graph, after spending 47,808 reasoning tokens to build it.

Two things about these failures matter beyond the models. The first is that the mistakes are plausible. A service called “tokenization” owned by Payments Core is not absurd. card-gateway at Tier 2 is not absurd. A graph built by gpt-4o-mini has 198 nodes with the same labels and the same shape, and nothing about it, from the inside, says that a fifth of its edges are missing. The second is that the hybrid did not rescue it. On the Restricted-data question, the blast-radius question and the open-action-items question, the hybrid on the gpt-4o-mini graph ran the graph query, accepted the result, and never searched the documents, which still said all of it. That is the sixth article’s finding again: the hybrid trusts the graph for structure, so a wrong graph is not corrected by the pages sitting next to it.

Result 3: the residual is names, and names are the one thing every company has

Look again at what gpt-4.1 got wrong: seven data categories it did not recognize or named its own way, and three procedure titles. Every one of those is a problem of names, not of reading. So the fourth graph gave the model the registers, the names of things, and nothing else. No relationships, no properties, no graph.

It fixed all seven. With the category names in front of it, the model recognized “ledger entries” and “payment links” as categories and spelled “account balances” the way the policy does. STORES went to 37 of 37, the categories to 33 of 33, and edge recall to 99.4%. The hybrid on this graph scores 97.5%, tied for the best of any graph in the experiment, loaded graph included. The three remaining misses are the procedure titles, and they would need a register of run book headings, which is a stretch even for a tidy company.

This is the practical finding of the article. A company that cannot load its graph usually can produce its lists: an HR export of people and teams, a CMDB export of services and stores, an incident tracker’s ids, a data classification policy’s categories. Lists are not a graph, and nobody thinks of them as structure. But given to an extractor as the spelling of things, they turn a 98% graph into a 99% graph, and they are exactly what the cheap models lacked when they invented “Wallet team” and “tokenization”.

The fifth graph, the skeleton, is the cheapest and the fastest: 33 cents and 48 seconds, because the model reads each page against a 291-word prompt and extracts only what the systems of record do not hold. It found 99% of the edges, and its answers are within judge variance of the loaded graph’s. But its precision is the lowest of the gpt-4.1 runs, 95%, for a reason worth knowing. The change management policy has a section called “Rolling back”. The incident policy has one called “Incident severity and on-call policy”. The extractor, asked to find procedures and attach them to run books, attached each of those sections to every run book the policy mentioned, ten run books each. Twenty of the twenty-three spurious procedure edges came from those two policy pages. The model did not misread the pages. It did not know that a policy page is not about any run book in particular, and the prompt did not tell it. Scope, like names, is something the extractor has to be given.

What extraction actually costs

The first article said extraction was expensive. In dollars, it is not. The best run cost 59 cents for 105 pages, about half a cent per page. It spent 374,000 tokens on a corpus of 17,000, twenty-two tokens for every token of text, in the same order as the thirty-six to one of the Microsoft example the first article quoted, and most of that is the ontology being re-sent with every page, served largely from cache. At that rate a wiki of ten thousand pages is about 56 dollars, and a hundred thousand pages is about 560, once. These pages are short, about 160 tokens each, so a wiki of longer pages costs more per page; but since the ontology is most of every call, not the page, the rate grows more slowly than page length does. The time is a few minutes and scales with how many pages you run in parallel, not with how many you have. Against a sync job that costs nothing and takes under a second, that is a real difference, but it is not the difference that decides anything.

The cost that decides things is verification. I can tell you that gpt-4.1’s graph is 98% faithful and gpt-4o-mini’s is 78% because I have the loaded graph to compare them to. A team extracting from its wiki has no such thing; if it had, it would load. From the inside, the two graphs look the same: the same labels, plausible counts, confident answers. The 20 questions could not tell the 98% graph from the loaded one, and they could tell the 78% graph only because its holes happened to be where the questions were. A question set finds the errors it asks about. A graph diff finds all of them, and you can only run a graph diff against a graph you trust.

So the useful advice is not “load, do not extract”. It is this:

• Load whatever a system of record holds, and extract only the rest. The skeleton run did that: cheapest, fastest, and its answers within judge variance of the loaded graph’s. The extracted part is small enough to check by hand.
• Give the extractor the names before you give it the pages. Registers of people, teams, services, stores and categories are the cheapest structure a company has, and they removed every residual error except the three procedure titles.
• Tell the extractor what a page is about. A policy page is not about a run book. A team page’s list of responsibilities is not a list of services. The scope of a page is a fact the prompt has to carry, because the cheap models and the policy pages both showed what happens when it does not.
• Use the ontology as the check, not only the schema. Every service is owned by one team and has someone on call; every category has a parent; every action item belongs to an incident; company policy says every Tier 1 service has a run book, and the three that do not are a finding, not noise. These are already written down in the definitions. Run them against the extracted graph, together with a count of each class against the registers. The 22 dangling edges, the 21 unparseable values, the 26 services nobody is on call for, and the eleven teams where the company has eight would all have been caught before a single question was asked.
• Do not pick the extraction model on price. The cheap graph scored below no graph. The cost of the model is a one-time fraction of a dollar per hundred pages; the cost of the graph it builds is every answer afterwards.

The honest caveats

This corpus is the best case for extraction, and I want that stated plainly. The pages were generated from the same world model the loaded graph came from, so every fact is on a page, every page uses the same names, nothing is stale and nothing contradicts anything else. A real wiki is none of those things, and the sixth article showed what stale pages do on their own. A real extraction would do worse than 98%, and the names problem would be larger, not smaller. The direction of the findings should hold; the numbers are a ceiling.

Each graph was built once and evaluated once; a second extraction with the same model would not give the same graph, and the judge has its own variance, which is visible in the one run where the loaded and extracted graphs differ for no graph reason. The merge is my code and deliberately simple: a better resolver would help the cheap models more than it would help gpt-4.1. The extraction prompt had the full ontology, 1,472 words of it; a team without one would be extracting into a schema it has not yet thought through, which the fifth article suggests is where most of the loss would come from. And the fidelity numbers depend on my matching: names normalized for case, hyphens and plurals, procedures and action items matched by text similarity. A stricter matcher would score every run a little lower.

Closing remarks

Extraction is not expensive. Verification is. Half a cent a page buys a graph that is 98% faithful on clean pages. What it does not buy is any way of knowing that, unless you have a graph you trust to compare it with, and if you have that, you have a loaded graph.

A wrong graph is worse than no graph, and it looks exactly like a right one. The cheap model’s graph scored below Classic RAG on the same questions, and the hybrid that had the pages beside it did not read them. Nothing about the graph said it was missing a fifth of its edges.

The residual is names and scope, and both are things you can give the model. Lists you already have took the best extraction from 98% to 99%, and a sentence about what each page is about would have prevented most of what the cheap models and the policy pages invented.

So the verdict on the first article’s claim: half right. Loading is cheap, and it stays cheap at any scale. Extracting is also cheap, in the only unit I had been counting. What it costs is the one thing the loaded route gives you for free: knowing that the graph is true.

Naissus Pay, its people, systems and incidents are invented for this work. Prices are Azure OpenAI list prices as of 30 September 2026. The earlier articles in this series are one, two, and the five that followed on the router, the shapes of the wall, the layers of an ontology, drift, and Microsoft’s knowledge layer.

Extract or Load? I Built the Same Knowledge Graph Both Ways and Measured the Difference was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.

The question I owed this series

Every article so far ended with the same decision table: pages go to search, structure goes to a graph, and the hybrid takes both. Since the fourth article there has been an objection I had not answered. Most enterprises will not build my hybrid. They will buy what their cloud provider sells. What does that score?

It is a fair question, and in 2026 it has a concrete answer. At Build in June, Microsoft made Foundry IQ knowledge bases generally available: a managed layer on Azure AI Search that plans several subqueries per question, runs them in parallel, reranks everything and either hands the grounding to your agent or writes the answer itself. Microsoft’s own benchmark put it at about 36% higher response relevance than single-shot RAG. Fabric IQ, announced at Ignite the previous November, added an ontology item, a formal model of the business bound to live data, and in June a way to expose it to Foundry IQ as a knowledge source. Together they are the most complete knowledge layer for agents I know of, and the product documentation states the ambition clearly: without semantic understanding, it says, AI remains unfit for high-stakes decisions. It is the right ambition, and it deserves a careful measurement.

What I could not find is an independent measurement on a corpus where every answer is known. Microsoft has reported results on its own datasets, and the questions behind them are live ones inside Microsoft too. Microsoft Research’s Memora work makes the case for agent memory that does not commit to a fixed ontology, noting that graph memory such as GraphRAG and Zep tends to need “rigid ontologies that don’t generalize across domains”. Fabric IQ makes the case for a governed ontology as the shared language of the business. Both are serious positions, and both can be measured. So this article is the first half of that measurement, on the one corpus where I know every answer. The second half, the same ontology as a Fabric item, is prepared and is the subject of the next article.

What Foundry IQ actually does

Underneath, a Foundry IQ knowledge base is agentic retrieval over Azure AI Search, with three settings that decide how much of a language model is in the loop.

At minimal effort there is no model at all. The question goes to the index as one query, the semantic reranker sorts what comes back, and you get passages. That is single-shot retrieval with a longer result list.

At low effort a model, gpt-4.1 here, reads the question and writes up to three subqueries. Each runs as a search over the index, keyword and, where the index has a vectorizer, vector, and the semantic reranker sorts its results. The results are merged into one payload with reference ids, and if you ask for it, the same model writes the answer from that payload.

At medium effort the plan can hold five subqueries and the engine may go round again if the first pass looks thin. It is the only setting with iteration.

Two things matter for what follows. First, every subquery is still a search: the engine never joins, counts or traverses. Second, the knowledge base can return either the grounding (extractiveData) or a finished answer (answerSynthesis). I measured both, because they answer different questions. Extractive mode asks "is the retrieval better?" with my own generation held constant. Synthesis mode asks "is the product better?" as a customer would use it.

The setup

Nothing about the corpus, the questions or the judge changed. Naissus Pay is still a payments company that does not exist: 105 documents in 108 chunks, a graph of 165 nodes and 386 relationships derived from the same source of truth, and 32 questions with a computed ground truth. The first twelve are the ones from the first article; the twenty from the decision matrix add five page questions, five lookups, five worked-out questions and the five trap questions whose answers look structured but live only in prose. Each question carries its shape from the fourth article: page, lookup, chain, set, absence, definition, mixed, trap.

The Foundry IQ side is deliberately minimal and deliberately isolated. A new resource group, rg-ontology-measured in Sweden Central, holds one Foundry resource with a gpt-4.1 deployment that does nothing but query planning and answer synthesis for the knowledge bases, so its bill is the cost of this article and nothing else. The knowledge source wraps the existing index, the one Classic RAG has queried since the first article, with the same semantic configuration and a vectorizer pointing at the same embedding model, so every subquery runs as the same hybrid search Classic RAG runs, keyword and vector together, before the reranker. Three knowledge bases sit on top of it, one per effort level, on the Basic search service in Switzerland North that the series has always used. I moved its semantic ranker from the free plan to the standard plan before the runs, so that the free allowance of a thousand queries a month could not silently degrade the results halfway through.

The evaluation harness calls the retrieve action directly, records the activity log the service returns (every planned subquery, every reranked count, the planning and synthesis tokens, the retrieval tokens Azure AI Search bills) and hands the answer to the same gpt-4.1 judge with the ground truth and the key facts. Costs are at Azure list prices as of 20 October 2026, including the agentic retrieval meter, which is charged here even though the first fifty million tokens a month are free. Every run is recorded with its full activity log.

What happened

Classic RAG, top five passages, scores 58% on the 32 questions. Foundry IQ at minimal effort, which is the same search with a longer result list and no model, scores 62%. At low effort, with query planning and the knowledge base writing the answer, 72%. At medium, 78%. The agentic hybrid of the earlier articles scores 91%, and the graph agent alone 53%, because it cannot read prose.

So the headline runs in two directions. Foundry IQ is clearly better than single-shot RAG: fourteen points at low effort and twenty at medium, which is 24% and 35% in relative terms, independently confirming the region of the 36% Microsoft reports on its own data. And on questions that need a traversal or a count, it stays short of a design built to compute them: twelve and a half points below the hybrid at its best setting, at twice the cost per question.

The right-hand column of the table is the one to read. The 16 graph-owned questions, lookups, chains, sets, absences and definitions, are where the series has always said search fails. Classic RAG scores 41% on them. Foundry IQ at minimal effort, 38%: a longer result list buys nothing on these. At low effort, 62%. At medium, 72%. The graph agent and the hybrid, 97%. Agentic retrieval closed about half of that gap at its best setting, and then stopped, and the rest of this article is about where exactly it stopped.

The wall moved, by shape

Pages and lookups were never the problem, and they are not now. Both efforts score 100% on both shapes, exactly as Classic RAG does. More retrieval did not help a question that one passage already answered, and it did not hurt it either.

Absences are the surprise. Classic RAG scored 0% on “which teams own no Tier 1 service” and “which Tier 1 services have no run book”, because no passage says “none”. Foundry IQ scores 50% at low effort and 100% at medium. The query planner did something a single search cannot: it asked for the complement. For the teams question it planned “teams without Tier 1 service ownership” and “list of teams and their Tier 1 service ownership”, got the whole team roster and every service page in one payload, and the synthesis model did the set difference in its head. Correct, exactly, at both efforts. The run book question shows what the extra effort buys. Three of the twelve Tier 1 service pages never mention a run book, and the only way to find them is to read all twelve. At low effort two subqueries brought back 28 chunks, and the answer named one service of three. At medium, five subqueries brought back 72, every Tier 1 page was in the window, and the answer named exactly the three. A set difference over twelve pages is within reach of a reader, if the reader has all twelve.

Sets moved from 17% to 33% at low and 50% at medium, and the misses are counts. “Which three services were affected most often, and how many times each?” came back at medium with the right three services and the wrong numbers: six, six and four against seven, five and four. The planner retrieved 51 chunks, which is most of the postmortems but not all of them, and the model counted what it had; at low effort, with 37 chunks, it also lost card-gateway from the top three. The severity breakdown for 2025 got SEV1 and SEV3 right and SEV2 wrong, six instead of seven, at both efforts. The third set question, incidents whose root cause was a Platform component, was partial at both because the answer added an SMS provider outage whose follow-up action belonged to Platform: the right team, the wrong relationship. Top-k retrieval cannot count what it did not retrieve; agentic retrieval retrieves more, and counts that too.

Chains went from 33% to 50% at both efforts. The planner knows what a chain is. For the events-stream outage it asked for “services directly affected”, “services indirectly affected” and “dependencies and downstream services”; for card-vault-db, “services depending on card-vault-db” and “dependent systems”. That finds the first hop, and sometimes the second, because service pages name what they call. It never finds the fourth. The card-vault-db question is the series’ canonical chain, four hops from the database to the checkout pages. At low effort the answer reached checkout-api, one of the four customer-facing services; at medium, with five subqueries and 84 chunks, it reached checkout-web as well. payment-links and wallet-app-bff, two more hops along, never appeared. The events-stream question, with seventeen services in its blast radius, came back with the five direct consumers and two services that are not affected at all.

Definitions are the clearest case, and the one I would show a product team. “Which services are in PCI scope?” has a five-service answer that follows from a rule: a service is in scope if it touches a cardholder data store, or directly calls one that does. No document lists the five. The policy document states the rule. Foundry IQ’s planner wrote three good subqueries, the reranker returned five chunks, and the synthesis model wrote back the rule, correctly and completely, and no services. At both efforts. It found the definition and could not apply it, because applying it needs a join across three kinds of thing. The other definition questions split the same way: “which data stores hold Restricted data and who owns each” is correct at both efforts, because every store page names its categories and its team, so the join is one hop and 79 reranked chunks cover it. “Which services depend directly on a store holding Restricted data” is partial at both, two of seven services at either effort, and the pattern of the misses is the taxonomy. The two it found, card-vault and auth-service, sit on stores whose pages describe their contents in the policy’s own words: card numbers, password hashes. The five it missed sit on session-cache, settlement-files-store and kyc-documents-store, whose pages say session tokens, settlement files and identity documents. That those are authentication secrets, primary account numbers and identity evidence, and therefore Restricted, is two steps up a tree that only the policy document describes. The medium-effort answer even recites that list of categories from the policy before naming two services. Eighty-one chunks at low and 99 at medium were in the window. The walk was not.

Traps are where Foundry IQ wins something my hybrid does not. These are the five questions whose answers look structured and live only in prose: open action items, recovery time objectives, retention periods. The graph has none of this, which is why the graph agent scores 0% and the hybrid 50% on them. Foundry IQ scores 80% at both efforts. “Which data stores have an RTO longer than two hours” planned two subqueries, retrieved 38 chunks and listed exactly the three stores. It is a better reader than my agent, and I will say more about that below.

Mixed questions, the three that need the exact set from the graph and the prose for each member, fell from 67% to 33% and 50%. The SEV1 question is the pattern: at low effort the knowledge base listed six incidents, three of them SEV2, and missed one of the four; at medium it described all four and dropped one from its own summary. The hybrid gets the set from the graph and the prose from search, which is the whole point of having both.

The wall, case by case

Read the two tables together and a pattern appears that I did not expect to be this clean. The subqueries are the ones a competent analyst would write. “Teams without Tier 1 service ownership.” “Incident counts by severity for 2025.” “Services with direct dependencies on data stores holding Restricted data.” If retrieval were the problem, these plans would solve it. The reranked counts are large: between 5 and 81 chunks per question at low effort, 28 or more on eight of the eleven, out of 108 in the corpus. On several questions the knowledge base had most of the company in its context window.

And it still got the counts wrong, the fourth hop missing, the rule unapplied. That is the finding. Foundry IQ is an excellent answer to the question “what if we retrieved more, and more cleverly?” It is the strongest version of that answer I have measured, better than my brute-force top-50 run in the second article, which scored 75% on the twenty matrix questions where Foundry IQ scores 80% at low effort and 82.5% at medium. But the wall shapes were never a retrieval problem. They need an operation, a traversal, a count, a set difference, a rule applied over three kinds of entity, and that work belongs to a system that computes, which a language model reading fifty passages, however well chosen, is not. The graph agent, with about two model calls and one Cypher query per question, is right on all eleven wall questions. Foundry IQ is right on three at low effort and four at medium, which is as much as retrieval alone can reasonably deliver here.

What the effort setting buys

The three settings are three different products. Minimal is a search with a long result list: one query, twelve or thirteen chunks, 0.16 seconds of retrieval, no model, 62%. Low plans 2.4 subqueries, reranks 34 chunks, and the knowledge base’s answer scores 72% in 4.2 seconds at 1.5 cents. Medium plans 3.8 subqueries, reranks 51 chunks, 78% in 5.2 seconds at 2.0 cents. Bringing the model in, from minimal to low, costs about two seconds and 0.9 cents per question and buys nine points. The second step, low to medium, costs another second and 0.4 cents and buys six.

Two details matter for anyone budgeting this. The retrieval meter is not where the money goes: Azure AI Search’s own tokens were 3% of the per-question cost at low effort and 16% at medium, and the whole experiment, 160 retrievals across the five settings, consumed 3.5 million of the fifty million free tokens a month and cost $2.51 at list prices. The model is the cost. Query planning and answer synthesis on gpt-4.1 are 97% of every cent at low effort and 84% at medium, and they are billed to your Azure OpenAI resource, not to the search service.

The second detail is a result I did not expect and report as measured. I ran every knowledge base twice: once letting it write the answer, and once taking its grounding and generating with my own Classic RAG prompt, the one used since the first article, with the same model. Mine scored lower at both efforts: 69% against 72% at low, 72% against 78% at medium. My prompt was written for five passages and says “use only what the passages say”; the knowledge base handed it twenty, sixteen thousand characters on average, with reference ids the prompt does not know how to use. The knowledge base’s own synthesis is evidently tuned for that payload; mine was not. The lesson is unglamorous: if you take the grounding and generate yourself, rewrite your prompt for it, or you will give back a third of what the retrieval bought. In extract mode, medium effort also planned with 6,700 tokens on average against 1,160 in synthesis mode, and ran eight seconds per question against five; the activity log does not say why, and I did not find it documented.

Where this leaves the comparison

Put every path on one chart and the picture a buyer needs is there. Classic RAG at 0.4 cents and 58%. Foundry IQ at minimal effort for a fifth of a cent more and five points more. Foundry IQ at low and medium at 1.5 and 2.0 cents and 72% and 78%, roughly where the brute-force baselines of the second article sit, top-50 retrieval and the whole corpus in the prompt, at 72% to 75%. And the agentic hybrid at 1.0 cent and 91%, alone in the upper left.

That last point is the one I would not have predicted before the series started, and it has nothing to do with Microsoft. A system that knows the shape of the question, and answers each shape with the right operation, is cheaper than a system that answers every question by reading more, because reading is what costs tokens. The hybrid spends its money on the ontology in the prompt and spends it once per question; Foundry IQ at medium effort spends it on planning four searches, reading fifty chunks and writing from them.

None of which is an argument against buying Foundry IQ. It is an argument about what you are buying. On the eight page questions and five lookups, which in a real company are most of the questions, it is as good as anything here and needs no graph, no ontology and no engineer who knows Cypher. On the traps, prose facts that look like data, it is better than my hybrid, and that is a shape I had underweighted. It carries the governance that the series has never measured: document-level permissions and Purview labels on indexed sources, and the caller’s identity on remote ones. What it does not do, at any effort, is the arithmetic of structure. If your questions have that shape, and in operations, risk and compliance they do, the knowledge base needs a source that can compute, which is exactly what the Fabric ontology knowledge source promises and what the next article measures.

The second half, built and waiting

The fuller test of the platform is not search over documents; it is the ontology, and that is where Microsoft has put its answer to exactly this wall. So the same Naissus Pay that lives in Neo4j is now written out, from the same source of truth, as fifteen tables with 348 rows and an ontology definition in Fabric’s new TMDL format, ready to load into a lake house and an ontology item: nine entity types, thirteen relationship types, and the eight business rules in natural language, the six definitions the graph agent receives plus two that the translation forced me to make explicit. The translation was not free of decisions, and they are the interesting part. A tabular model cannot express a self-relationship, so the data-category taxonomy became two columns, parent and classification, and a rule that says inheritance is a join. DEPENDS_ON points at three kinds of thing, so it became three junction tables. The Cypher patterns that carry the semantics in the graph agent’s ontology, the layer the fifth article found did almost all the work, have no equivalent in Fabric and were dropped; the sentences stayed. Whether an ontology item can answer “which services are in PCI scope” from the sentence alone is precisely the question.

The knowledge source, the knowledge bases and the evaluation harness for that arm are the ones used here; only the source changes. The results will be reported in the same tables, on the same questions, with the same judge.

The honest caveats

Thirty-two questions on one invented company, one model, one judge, one day of runs. Per-shape counts are two to eight, and a single verdict moves a shape by twelve to fifty points; I have explained the direction of every move above and would not quote the decimals as general. Foundry IQ’s knowledge bases are generally available but query planning, answer synthesis and the non-minimal efforts are still preview features, and the activity log, the prices and the behaviour of the medium setting can change. My extractive-mode numbers are a lower bound on what a tuned prompt would get, for the reason given above. A first round of Foundry IQ runs, made before I noticed that the index had no vectorizer and the subqueries were therefore running as keyword search alone, scored within two points of the reported round at every setting except extractive medium, which gained eight points from the vector half; the reported round is the hybrid one, and the other is kept alongside it. The three designs were measured on earlier days with the same model and judge; I did not rerun them; their recorded runs are the ones quoted. And the comparison is between retrieval designs, not products: the governance, permissions and multi-source routing that Foundry IQ exists to provide were not exercised, because a single-tenant demo with one source cannot exercise them. Finally, the disclosure the question deserves: no Microsoft team was involved in, informed of or consulted on this work, nothing was sponsored, and every service was paid for on my own subscription.

Closing remarks

I started this series with a RAG system that was perfectly wrong on the questions that mattered, and spent six articles showing that the fix was to know the shape of the question and answer each shape with the right operation. The obvious objection was that I had only ever compared my own designs. Foundry IQ is the strongest retrieval design I have been able to measure, and it is clearly the work of a team that understands the problem. It also confirms the series’ finding rather than overturning it: twenty points over single-shot RAG, exactly on the questions where reading more helps, and the same wall, in the same place, on the questions where reading is the wrong operation.

The natural next step is the one Microsoft has already built: give the knowledge layer a source that can compute. Whether a governed ontology in Fabric is enough to answer the questions retrieval cannot, and how well it transfers, is now testable on this corpus. The apparatus is prepared, and the next article runs it.

Everything in the data is invented. Naissus Pay, its people, systems, incidents and policies were written for this series. The evaluation code, every recorded run, the lake house tables and the ontology definition are kept with the project.

References

• Microsoft Learn, “What is Foundry IQ?”, “Agentic retrieval in Azure AI Search”, “Set the retrieval reasoning effort”, “Create a Fabric Ontology knowledge source”, 2026.
• Microsoft, “What’s new in Microsoft Foundry at Build 2026”, Foundry developer blog, June 2026.
• Microsoft Tech Community, Azure AI Foundry blog, “Foundry IQ: boost response relevance by 36% with agentic retrieval”, accessed October 2026.
• Microsoft Learn, “What is Fabric IQ?” and “What is ontology (preview)?”, 2026.
• Microsoft Learn, “Ontology definition” (TMDL parts of an ontology item), 2026.
• Zhang, Xia, Das, Bastos, Wang, Bansal, Rajmohan, “Memora: a harmonic memory representation balancing abstraction and specificity”, ICML 2026; Microsoft Research blog, 29 June 2026.
• Azure retail prices API, meters “Agentic Retrieval Minimum/Low/Medium Reasoning Tokens”, “Semantic Ranker queries”, “Basic Unit”, read 20 October 2026.
• The earlier articles of this series on AI Advances: “Your Data Has a Shape. Does Your AI Know It?”, “Written Down or Worked Out?”, “Route First, Retrieve Second”, “Not a Retrieval Problem. A Shape Problem.”, “How Much Meaning Does Your Graph Need?”, “What Breaks When the World Moves and Your AI Does Not”.

Microsoft’s Knowledge Layer, Measured was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.

The question that comes after the demo

Every time the demo ends, the same person stands up. Not the engineer; the one who will own the thing. The question is some version of: “And who maintains this?”

It is the best question in the room, and for five articles I have answered it with a sentence. The catalogue feeds the graph, the wiki feeds the pages, somebody owns the ontology. True, and useless, because it describes a company on the day of the launch. A company is not a snapshot. Services get retired, dependencies get added, a platform team renames a field, compliance narrows a rule after an audit. Every one of those changes lands in one place first and in the others later, or never. And the assistant keeps answering.

So this article does what the earlier ones did not. It lets the world move. Four kinds of drift, each one a thing that actually happens, applied to the same company, the same three designs and the same judge. The question is not which design is best. It is which one breaks, how loudly, and what it would take to notice.

Three things drift at three speeds

There are three artefacts between the company and the answer, and they are updated by different people on different schedules.

The catalogue feeds the graph. In most companies it is a system of record with a sync job, so it is usually current, until the sync breaks. The wiki feeds the pages. It is updated when somebody remembers, which for a service page is usually never. The ontology tells the agent what the graph means. It is updated when somebody notices it is wrong, and the whole point of this article is that nobody does.

Four scenarios follow from the gaps between them.

For the first two, four things happen at Naissus Pay in one quarter. The checkout API starts sending receipts, so it gains a dependency on the notification service. Onboarding starts charging a setup fee, so it gains a dependency on the payments API. A run book is registered for the 3-D Secure service. The loyalty program is retired and its service switched off. In drift 1 these land in the catalogue and the pages stay as they were. In drift 2 the pages are rewritten and the catalogue sync is broken. In drift 3 nothing happens to the facts at all: the platform team renames three things in the graph, tier to criticality, customer_facing to is_customer_facing, DEPENDS_ON to USES, and the hand-written ontology still uses the old names. In drift 4 compliance narrows the PCI scope rule after a tokenisation audit: only services that read or write a cardholder data store are in scope; the direct callers of the vault, which the old rule included, are out. The policy page is updated. The ontology's rule is not.

Each scenario got its own Neo4j database, and a search index built from its page state, loaded from the exact state the scenario describes, so the demo stayed intact and nothing was hand-edited. For every question, the ground truth was recomputed from the true state of the company, not from either store. The first two scenarios also ran in their two bracketing states, nothing updated and everything updated, so the measurements have a floor and a ceiling. All runs on gpt-4.1, 203 of them.

Drift 1 and 2: the store that moved is the design that is right

Six questions changed their answer when the four things happened: three lookups, two chains and an absence. The third lookup is the plain question of whether the wallet app still shows loyalty points.

Read the two middle groups. When the graph is current and the pages are stale, Classic RAG scores 8% on the changed questions and the graph agent 75%. When the pages are current and the graph is stale, RAG scores 50% and the graph agent 25%. Neither store is more trustworthy. Each is exactly as good as its last update, and the design on top of it inherits that, completely and silently.

Silently is the word that matters. Here is Classic RAG on stale pages, asked whether the wallet app still shows loyalty points, a quarter after the program was retired:

Yes, the wallet app still shows loyalty points. The service that provides them is loyalty-service. wallet-app-bff shows points from loyalty-service [1]. loyalty-service awards and redeems loyalty points for wallet activity [2]. Both services are owned by the Wallet team [3].

Three citations, all accurate to the page, all wrong about the company. And here is the graph agent on a stale graph, same question:

Yes, the wallet app (wallet-app-bff) still shows loyalty points. The service that provides loyalty points is loyalty-service. wallet-app-bff depends on loyalty-service for this feature.

It ran a query, the query matched an edge that no longer exists in the company, and it answered. Across the 24 wrong answers the three designs gave on changed questions in these two scenarios, exactly one carried any sign of trouble, and that one was an empty result from a mistyped name. Zero hedges. Stale data does not look stale from the inside. It looks like data.

The hybrid follows the graph. This is the finding I did not expect. The hybrid has both stores. When the pages were stale it scored 92%, because it went to the graph. When the graph was stale it scored 25%, the same as the graph agent, because it went to the graph. On the six changed questions with a stale graph, it did not search the documents once. It had the current answer a tool call away and never looked, because its instructions, and its training, say that structure is the authority for structure. The only times it reported a discrepancy were when the graph itself said so: in the two states where the graph was current, the retired service carried the word “retired” in its node summary, the hybrid read it and wrote “However, loyalty-service is marked as RETIRED”. The signal came from the store, not from the agent comparing stores.

With both stores current, the graph agent and the hybrid score 92% and Classic RAG 50%, which is the wall from the fourth article: RAG gets the lookups and loses the chains, whatever the state of the pages.

Drift 3: a rename is the loudest failure, and the easiest fix

Three names changed in the graph. No facts, no pages. The hand-written ontology still said tier, customer_facing and DEPENDS_ON.

With the stale ontology, the graph agent scored 30%: the three questions that touch none of the renamed names, and nothing else. Six of the other seven produced a query with the old names, zero rows, and a confident sentence. The seventh, which teams own no Tier 1 service, returned every team in the company, because the property it filtered on no longer existed and so no service matched. Four of the confident sentences:

No customer-facing services are affected, directly or indirectly, if card-vault-db goes down. No services are affected, directly or indirectly, if events-stream goes down. There are currently no services with a dependency on events-stream. All Tier 1 services have a run book. None are missing one. No services are currently in PCI scope according to the knowledge graph.

Four answers a compliance officer or an incident commander would act on, each the opposite of the truth, each delivered without a flicker. The hybrid did the same, 30%, and in all ten runs it never once fell back to the documents, which still said, page by page, which services depend on the event stream. An empty result from the graph was accepted as the answer.

But look at the right-hand panel. Six of the ten runs had a query that returned nothing. The failure is silent in the answer and loud in the trace. In the third article I argued that the fallback should trigger on the design’s result, not the router’s confidence. This is the scenario that argument was waiting for: an empty-result rate that jumps from zero to sixty percent overnight is a smoke alarm, and it goes off before anyone reads a wrong answer.

Two repairs were measured. Replacing the ontology with a schema introspected live from the database, which is what a team without an ontology would have, took the graph agent to 50% and the hybrid to 60%. The agent read USES and is_customer_facing from the database and used them, and that recovered the card-vault blast radius and the checkout-api lookup. It did not recover the event-stream blast radius, where the query walked the new relationship and swept in services that merely share a dependency with the stream's consumers, 21 services instead of 17. It did not recover the tier questions, because the introspected schema says a property called criticality exists and nothing about its values, so the agent guessed 'Tier 1' as a string and matched nothing. And it did not recover the definitions, because meaning was never in the database: asked which services are in PCI scope, it looked for a property that says so, found none, and answered none. The second repair was the obvious one: three find-and-replace edits in the ontology, which took the graph agent to 95% and the hybrid to 95%, with the empty-result rate back to zero. The cheapest drift to cause, the loudest to detect, the cheapest to fix, and the one most likely to happen, because the people who rename fields are not the people who own the ontology.

Drift 4: the policy changed, and the ontology kept the old one

Compliance narrowed the scope rule and updated the policy page. The ontology’s rule, with its query pattern, still encoded the old one. Nothing in the data changed, so with the stale rule no query came back empty, and no trace carried a signal.

Three things happened, and together they are the sharpest result in the article.

Classic RAG found the new policy page and, asked what the rule is, quoted it correctly. Asked which services are in scope, it applied the new rule to the five passages it had and named two services, one wrong. Asked whether the payments API is in scope, it reasoned that exchanging card details for tokens “implies it reads cardholder data” and said yes. It knew the rule and could not run it over the data, which is the wall from the fourth article, now with a policy change on top.

The graph agent, with the stale rule, returned the old five services, said yes to the payments API, and recited the old rule when asked what the rule is. Three confident answers, all superseded, all delivered from a rule that had been retired by the policy it was meant to implement.

And the hybrid, which had the new policy page one search away, did exactly what the graph agent did. On all five questions it never called the document search. Asked “what is the current rule for deciding whether a service is in PCI scope”, a question whose answer is a page, it answered from the definition in its own prompt. The ontology had become a cached copy of the policy, and the agent trusted the cache over the source even when asked for the source.

Updating the rule, one sentence and one query pattern, took both agents to 75%: the scope list correct, the teams correct, and two partials. Asked whether the payments API is in scope, both said no, correctly, but could not say why, because the reason, that the audit showed it only ever handles tokens, is on the policy page and not in the graph. Asked for the rule itself, both gave the updated rule in the ontology’s words, “depends on a cardholder data store”, which the judge held to be less precise than the policy’s “reads from or writes to”. Both partials point the same way: the ontology can carry the rule but not its justification. When the rule changes, the “why” lives on the page, and only a design that reads the page can give it.

What would have noticed

Put the four drifts side by side and ask, for each, what signal existed before anybody read a wrong answer.

Classic RAG: one hedge in 24 wrong answers, and that one in the rename scenario, which did not touch its pages at all. The graph agent: seven signals in 24, all of them empty results, six in the rename scenario. The hybrid: eight in 23, same kind. Outside the rename, the designs failed in near-complete silence: of the 42 wrong answers across the three fact-drift states, three carried an empty result, all on the loyalty question and all from the agent trying a service name that does not exist in the graph; of the nine in the policy drift, none carried anything.

So the honest answer to “who maintains this” has three parts, and they are different for each kind of drift.

Fact drift is detectable by comparison, and only by comparison. No single design can tell that its store is stale. But two stores that disagree can. On the three changed lookups, Classic RAG and the graph agent gave different verdicts in every state where one store was stale, and the same verdict in the state where nothing had been updated; with both stores current they agreed on two of the three, the exception being the checkout-api lookup where the graph’s answer includes a cache the judge does not accept as a service. That is a detector a hybrid could run for the price of one extra search: when the page and the graph disagree about a lookup, one of them has moved, and the answer should say which one it believed and why. The hybrid in this experiment never ran it. It should.

Schema drift is detectable from the trace, immediately. An empty-result rate on graph queries is a one-line metric, and in the rename scenario it went from zero to sixty percent. Alert on it. The repair is three renames in the ontology, and it should be a step in the same change that renames the field, which means the ontology has to live in the same repository as the schema, with the same review.

Definition drift is not detectable by the system at all. No empty result, no disagreement between stores that the agent would consult, no hedge. The only signal is in the policy document, which the agents did not read because they already had a rule. The fix is organizational: every business definition in the ontology carries the policy it was derived from and a date, and a change to that policy opens a ticket against the ontology. It is not clever. It is the kind of thing that is only missing because nobody wrote it down.

For the person who owns the thing, that is the maintenance contract. A sync job and a freshness check for the catalogue. An owner and a review cadence for the pages, which will drift anyway, so the hybrid must compare. A schema change process that includes the ontology, with an empty-result alert as the backstop. And a policy change process that includes it too, with no backstop, because there is none.

The honest caveats

Four drifts, on one invented company, each with five to ten questions. The changes were chosen to flip known answers, so the effects are clear by construction; a real quarter’s drift would be messier and some of it would be harmless. The hybrid’s behavior, trusting the graph for structure and the ontology for definitions, is partly a consequence of its prompt, which tells it that structure lives in the graph; a prompt that said “when a lookup is cheap, check both and report disagreement” would change the fact-drift results, and that experiment is the obvious next one. The disagreement detector in the last section was computed from the recorded verdicts, not deployed. And the introspected-schema variant is one rendering of what a database reports about itself; a richer one, with sample values, would probably have recovered the tier questions.

Closing remarks

Every design is exactly as current as its last update, and none of them knows it. Stale pages gave cited, fluent, wrong answers. A stale graph gave queried, exact, wrong answers. The hybrid, with both, followed the graph and never checked.

Drift has three kinds, and they fail differently. Facts drift silently and can only be caught by comparing stores. Schema drift fails loudly in the trace and quietly in the answer, and is fixed with three renames. Definitions drift in total silence, because the ontology becomes a cache of the policy, and the agent trusts the cache.

The maintenance contract is the architecture. A sync and a freshness check, a comparison when stores can disagree, an alert on empty results, and a rule that no policy changes without its definition. None of it is expensive. All of it is missing from every demo, including mine.

So, one sentence for the person who stood up at the end: the system will keep answering after the world moves; the only question is whether you will find out from a metric or from a customer.

Naissus Pay, its people, systems and incidents are invented for this work. The earlier articles in this series are one, two, and the three that followed on the router, the four shapes of the wall and the layers of an ontology.

What Breaks When the World Moves and Your Ontology Does Not was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.

The question nobody could answer in the room

Every time I have shown a graph answering a question that search could not, somebody has asked the same thing afterwards: “and who writes the ontology?” It is the right question, and for four articles I have answered it badly. I said it was a week of a person who knows the business, and that this person has to own it afterwards. True, probably. But I had never measured what that week should contain.

The first article in this series showed what an ontology is worth as a whole: with it, the graph agent scored 58% on twelve questions; without it, 21%. That is one bit of information. On or off. It says nothing about which part of the ontology did the work, and it leaves the person who has to write one with no idea where to start, how much is enough, or what to skip.

So this article treats the ontology as what it is in practice: a few thousand tokens in a system prompt, each of which costs money on every call and takes somebody’s time to write and maintain. I cut it into layers, added them one at a time, removed them one at a time, and ran the same questions against every version. The result is a curve nobody had shown me: meaning bought per token. It is not flat, and the shape of it changes what I would tell that person to do with their week.

What is actually in an ontology

The word covers everything from a class diagram to a philosophy. For a graph that a language model has to query, it is a specific artefact: the text that tells the agent what the labels mean. In the Naissus Pay graph that text is 1,854 tokens, and it decomposes cleanly into six layers.

L0, the schema. Node labels, their properties, relationship types with their endpoints. This is not meaning; it is what any graph database will report about itself if you ask. 469 tokens.

L1, descriptions. One sentence per class, per property, per relationship. “A production incident with a written postmortem.” “Integer 1, 2 or 3. 1 is the most critical.” Adds 641 tokens, the largest layer.

L2, semantics. The class hierarchy and the rule that a node carries its ancestors’ labels. Which relationships chain, and how to traverse them: DEPENDS_ON is marked transitive, with the note “Follow it with a variable-length path for impact questions.” Adds 40 tokens.

L3, the taxonomy. The data-category tree, from PrimaryAccountNumber up through CardholderData to RestrictedData, and the inheritance rule: whatever is true of the broad category is true of the narrow one. Adds 65 tokens.

L4, the definitions. The six business rules, each as a sentence and as a query pattern that computes it: blast radius, cardholder data store, PCI scope, restricted data store, Tier 1 service, customer-facing service. Adds 471 tokens.

L5, the aliases. The words people actually use. “Outage” for incident, “vendor” for external provider, “impact” for blast radius. Adds 168 tokens.

Each layer is independent in the renderer, so I can build the prompt with any subset. That gives two experiments. Cumulative: start from the schema and add one layer at a time, six prompts. Ablation: start from the full ontology and remove one layer at a time, five prompts. The production “ontology off” prompt from the first article, 319 tokens of introspected schema, sits alongside as the reference.

The agent, the questions and the judge are the ones from the earlier articles. GraphRAG on gpt-4.1, the 32 questions with a computed truth, an LLM judge with the key facts. I report results on the 16 questions a graph can own: five lookups, three chains, three sets, two absences and three definitions. The eight page questions, the three mixed questions and the five trap questions stay out, because no amount of ontology makes a graph read prose, and that was settled in the second article.

The curve

Read it left to right. The schema alone scores 56%, the same as the production “ontology off” prompt, which is reassuring: the two renderings of “no meaning” agree. Descriptions take it to 78%. Semantics, forty tokens, to 88%. The taxonomy, sixty-five tokens, to 91%. The definitions to 97%. The aliases change nothing on these questions.

Two things are visible before any detail. The curve is steep where the layers are cheap. Semantics and taxonomy together cost 105 tokens and bought twelve and a half points; descriptions cost 641 and bought twenty-two. Per token, the small layers are three and a half times more valuable. And the curve flattens exactly where the ontology stops describing the graph and starts describing the business. Everything up to the taxonomy is about what the data is. The definitions are about what the company means by it. That last layer is the one that cannot be inferred, and it is the one that finished the job.

Which layer unlocks which shape

The curve is an average. The shapes are where it becomes advice.

Lookups need nothing. Who is on call, which team owns what, which providers a service uses: 100% from the schema alone. A lookup is one edge, and a label is enough to find an edge. The one lookup that slips to a partial from L2 onwards is the checkout-api question from the earlier articles, where the agent, now told that a service can depend on a data store as well as a service, correctly includes the cache and the judge, holding to the question’s word “services”, does not accept it. More meaning made the answer more complete and the grade lower. I note it and move on.

Chains need almost nothing. The two blast-radius questions from the first article were correct at every layer including the raw schema, because a variable-length path over DEPENDS_ON is the first thing a model tries when asked what depends on what. The third chain question, the event stream that feeds seventeen services, was correct on the raw schema, wrong on the L0 schema, where the agent returned 34 rows because it left out DISTINCT and counted paths rather than services, then correct again from the descriptions onwards. Same facts, two renderings of them, and one unlucky query; that is the noise floor of this kind of measurement, and it is why I read the shapes and not the decimals. The chain is the shape the structure answers by itself. A dependency edge means “depends on” whether or not anyone wrote it down.

Absences need a sentence. “Which Tier 1 services have no run book?” With the schema alone, the agent wrote the right query with one wrong character:

MATCH (s:Service) WHERE s.tier = '1' AND NOT (s)<-[:COVERS]-(:Runbook) RETURN s.name

Tier is an integer. Compared to the string ‘1’, nothing matches, and the agent reported that no Tier 1 service lacks a run book. On the teams question it did the same and listed all eight teams as owning no Tier 1 service. One description, “integer 1, 2 or 3. 1 is the most critical”, fixed both. A property’s type is the cheapest meaning there is, and the schema does not carry it.

Sets need the semantics. The incident-count question was wrong at L0 and L1 for a reason that is easy to miss. The agent counted incidents through the AFFECTED relationship and got fifteen SEV1 incidents where there were four, because each incident was counted once per service it affected, and the four SEV1 incidents affected fifteen services between them. With the semantics layer, which says what a relationship is for and how to traverse it, the agent counted incidents first and joined to services second. Forty tokens, and the difference between a number that is wrong by nearly a factor of four and a number that is right.

Definitions need the definition. This is the shape that makes the article, so it deserves its own figure.

“Which services are in PCI scope?” With the schema, the agent looked for a data category called PCI, found none, and answered that no services are in scope. With descriptions, it understood that scope has something to do with cardholder data and guessed three category names, one of which does not exist in the graph; partial. With semantics, it wrote a transitive path over DEPENDS_ON and reached too far upstream, adding services that are not in scope; partial again. With the taxonomy, it resolved the categories correctly through the tree but kept the transitive path, and the judge, holding to “exactly these five”, marked the extra services as incorrect. With the definition, which says exactly which hops are in scope and which are not, it wrote the query that computes the rule and returned the five services. The rule is not derivable from the graph. Somebody has to say that a service two hops upstream of the vault is out of scope, and the only place to say it is here.

The restricted-data question tells the same story one layer earlier. “Which services depend directly on a data store that holds Restricted data?” was wrong until the taxonomy layer, because no store is labelled restricted. Stores hold password hashes and identity documents, and the taxonomy says what those are. Sixty-five tokens, and a question that compliance asks every quarter went from “no services” to the correct seven.

The grid is the article in one picture. Each question has a layer that unlocks it, the layers are different for different shapes, and with one exception, the PCI question’s wobble between partial and incorrect before its definition arrives, a question that turns correct stays correct as layers are added. The ontology is not a single thing that either works or does not. It is a stack of specific pieces of meaning, each one paying for a specific kind of question.

What the ablation says, and what it does not

The cumulative experiment asks what each layer adds on top of the ones below it. The ablation asks the opposite: starting from the full ontology, what breaks when one layer is removed?

Remove the descriptions, 632 tokens, and the score does not move. Remove the semantics, the taxonomy or the aliases: no change. Remove the definitions, and the score falls nineteen points, from 97% to 78%: the sets, the absences and the definition questions all take damage. The definitions are the only layer the full ontology cannot do without.

This looks like it contradicts the curve, where descriptions bought twenty-two points and semantics nine. It does not. Each business definition carries its query pattern, and the patterns contain the semantics: DEPENDS_ON*1.. says that the relationship chains, SUBCATEGORY_OF*0.. says that the taxonomy inherits, tier: 1 with an integer says what type the property is. The definitions are worked examples, and a worked example teaches the agent the grammar as a side effect. Once they are present, the layers that taught the same grammar in the abstract are redundant.

That is the finding I did not expect, and it is the practical one. The abstract layers, descriptions and semantics and taxonomy, are how you would explain the graph to a person. The definitions are how you would explain it to a model: not “this relationship is transitive” but “here is a query that uses it transitively, and here is what the result means”. Six examples of the business’s own questions, with the query that answers each, taught the agent more than a thousand tokens of description did. And they are the layer that only a person who knows the business can write.

The cost side

The ontology is sent on every call the agent makes, so its size is a running cost, not a one-off. On this corpus the agent read 2,850 input tokens per question with the schema alone and 4,468 with the definitions, and most of the difference was the ontology, repeated on every call. At Azure list prices with prompt caching, which bills a repeated prefix at a quarter of the rate on this model, that came out as the cost column in Table 1: between 0.49 and 0.69 cents per question across all layers, with no trend upward. The fuller ontology was not more expensive to run, because the agent with more meaning made fewer calls. At L0 it averaged 2.6 model calls and 1.6 tool calls per question; at L4, 2.0 and 1.1. Meaning saved retries, and the retries cost more than the meaning.

So the budget argument is not about inference. The token cost of the ontology is a rounding error. The budget is the time of the person writing and maintaining it, and that is where the curve matters: the layer that took the most words, the descriptions, is the one the agent could do without once the definitions existed.

What to write first

Put the two experiments together and the order of work is clear, and it is roughly the reverse of how most people start.

Start with the questions, and write the definitions for them. Not a class diagram. Six to ten of the questions the business actually asks, each as a sentence that says what the answer means and a query that computes it. In this graph, six definitions were 471 tokens and were the only layer the full ontology could not lose. They are also the only layer that nobody but the business can write, which is why they are the week’s work.

Then the types and the traversal hints, which cost nothing. Forty tokens of “this relationship chains, follow it with a variable-length path” and a note that tier is an integer fixed the absences and the sets. These are the things an engineer knows and forgets to write down, and they are the cheapest tokens in the prompt.

Then the taxonomy, if the business has one. Sixty-five tokens. It unlocked every question about classification, and classification is what compliance asks.

Descriptions last, and only where a label is ambiguous. They bought the most in the cumulative experiment and nothing in the ablation, because the definitions carried the same information in a form the model uses better. Write one sentence where a label could be misread. Do not write six hundred tokens of prose because it feels like documentation.

Aliases when the question log tells you to. On these questions, the agent did not need to be told that an outage is an incident. On a real question log, with real vocabulary, this layer would earn its place; here it did not, and I would not write it on day one.

For the person who approves the budget rather than writes the prompt: the ontology is not a project. It is six to ten sentences and queries, written by someone who knows what “in scope” means in your company, plus an hour of an engineer’s notes about types and traversal. It cost under two thousand tokens here and ran for half a cent a question. What it bought was the difference between a graph that answers 56% of the questions it could own and one that answers 97%.

The honest caveats

Sixteen questions on one invented company, one model, one judge. The per-shape counts are small, two to five, and a single question flipping moves a shape by twenty to fifty points; the direction of every flip is explained above, but I would not read the exact percentages as general. The layers are my decomposition, and a different ontology would cut differently, though the six kinds are the ones I have seen in every graph I have worked on. The cumulative order is one order; adding definitions before descriptions would give a different curve, and the ablation is the check on that. And the definitions were written knowing the questions, which is the point of the method and also its limit: the curve measures how well the ontology serves the questions it was written for, not how well it generalizes to questions it was not.

Closing remarks

The ontology is a budget, and the curve is not flat. Fifty-six percent with no meaning, 97% with all of it, and nearly a third of the gain in two layers that together cost 105 tokens, fewer than this closing section.

Each shape has its layer. Chains need the structure alone. Absences and sets need the types and the traversal hints. Classifications need the taxonomy. Definitions need the definition. If you know which shapes your questions take, you know which layers to write.

Write the definitions first, as worked examples. They were the only layer the full ontology could not lose, they taught the agent the grammar of the graph as a side effect, and they are the only part a model cannot infer and an engineer cannot write alone. That is the week. The rest is an afternoon.

So, if there is only room for one instruction for the person who asked who writes the ontology: take the ten questions your business cannot currently answer, and for each one write what the answer means and the query that computes it. That is the ontology. Everything else is a comment.

Naissus Pay, its people, systems and incidents are invented for this work. Prices are Azure OpenAI list prices as of 30 September 2026. The earlier articles in this series are one, two, and the third and fourth on the question router and the four shapes of the wall.

How Much Meaning Does Your Graph Need? was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.

The sentence I kept coming back to

There is a sentence I have said in every architecture review of a retrieval system for the last two years, and I have come to believe it is the only one that matters. Not a retrieval problem. A shape problem.

It usually lands in the middle of a conversation about tuning. The chunks are too big, or too small. The reranker is not good enough. The embeddings are last year’s. Somebody has a benchmark where a new technique adds four points. All of that is real, and I have shipped all of it, and it pays. But the questions that were failing in that meeting were not failing for any of those reasons. They were failing because the answer was not in any passage. It was between the passages, in how the things they describe connect, in how many of them there are, in which ones are missing, in a rule nobody had ever applied to all of them at once.

This article is about that wall, and about the three designs I built on one corpus to measure where it stands. The first article in this series introduced the designs and the idea that questions have a shape. The second put a decision rule on trial. This one goes back to the architectures themselves and asks the question I am asked most often: what should I design, for which questions, and why. The answers are measured, on the same company and the same judge as before, and they are organized by shape, because that is the only way they make sense.

The wall has four shapes

A page was always the easy one. “How do I roll back the ledger service?” Somebody wrote the answer; search finds it. Everything that never got better, whatever I tuned, was one of four other things.

A chain. “What breaks if this database fails?” The page about the database knows what the database is. The services four hops downstream are on pages that never mention it.

A set. “How many incidents did we have this year, by severity?” The answer needs every record. Top-k retrieval returns five, and a model asked to count will count what it was given.

An absence. “Which critical services have no run book?” No page says “there is none”. The answer exists only as the difference between two complete lists.

A definition. “Which systems are in audit scope?” A rule, applied to everything. Not a sentence anybody wrote, and a rule that has to reach every record, not the five that mention it.

These are not exotic. They are the questions a business asks its own knowledge in week two, phrased as if they were simple. And they are exactly the questions where better retrieval does nothing, because retrieval is a way of finding passages, and these answers are not passages.

Three designs, one corpus

To measure the wall rather than argue about it, I built three designs over one invented company and ran the same questions through all three.

Naissus Pay is a fictional payments company: 29 services, 19 data stores, 8 teams, 20 incidents, 10 run books, 6 architecture decisions, 4 policies. Its knowledge exists twice, generated from one source of truth, which is what makes every answer checkable.

Every question has a ground truth computed from the data, and an LLM judge grades each answer correct, partial or incorrect against it, with a list of key facts and a strict rule for sets: “exactly these five” with four listed is partial. Across the two earlier articles that gives 32 questions with a computed truth. Twenty-seven of them fall into the seven shapes below; the remaining five are the “trap” questions of the second article, whose answers lived only in prose that no design had structured, and I keep them out of the shape comparison because no design owns them. Everything in this article is gpt-4.1 unless it says otherwise.

Design 1: Classic RAG, search over passages

The pages are cut into 108 chunks of at most 320 tokens. Sections are never split, so a procedure or a root-cause paragraph stays whole, and every chunk starts with its document type and title so that a chunk read alone still says what it is about. That header is cheap and it matters: it is the difference between a passage that says “## Dependencies” and one that says “[Service page] Service: payments-api, ## Dependencies”.

header = f"[{DOC_TYPE_LABELS[doc.doc_type]}] {doc.title}"
for heading, body in doc.sections:
    size = count_tokens(f"## {heading}\n{body}")
    if groups[-1] and used + size > max_tokens:     # a section never straddles two chunks
        groups.append([])
        used = count_tokens(header)
    groups[-1].append((heading, body))
    used += size

At question time: embed the question, run a vector search and a keyword search at once in Azure AI Search, fuse the two rankings, let the semantic reranker re-score, keep five passages, and make one model call whose instructions say to answer only from those passages, cite them, and say what they do not support. One call, no loop, about two seconds, four tenths of a cent.

Design it for pages. It is the cheapest thing that works, and on this corpus it was perfect on every page question on five of the six models, with one partial on the sixth, and perfect on every lookup. Its wall is built in: it never sees more than five passages, and it only finds what sounds like the question.

Design 2: GraphRAG, an ontology and a traversal

The graph holds the same facts as edges: a team owns a service, a service depends on a store, a store holds a category of data, an incident affected a service. It was loaded from the structured source, the way a company exports its service catalogue, not extracted from the pages by a model.

What makes the graph usable by an agent is the ontology: what each class and relationship means, which relationships chain, and the business definitions with the words people actually use for them and the query pattern that computes them. This is one of the six definitions, the one that answers every “what breaks if” question:

OntologyRule(
    "BlastRadius",
    "The blast radius of a component X is every Service that has a DEPENDS_ON path of any length to X. "
    "Use it for 'what is affected if X fails' and 'who depends on X directly or indirectly'.",
    "MATCH (s:Service)-[:DEPENDS_ON*1..]->(x:Component {name: $name}) RETURN DISTINCT s.name",
    aliases=("impact", "affected services", "downstream impact", "directly or indirectly depends on"),
)

The agent gets two tools, a name resolver backed by a full-text index and a read-only Cypher runner, and the ontology in its system prompt. It resolves the names in the question, writes a query, and answers from the rows. It has no access to prose at all, which is deliberate: this design isolates what structure alone contributes.

Design it for chains, sets, absences and definitions. On this corpus it was perfect on all eleven, at about two tenths of a cent more per question than search. Its wall is the mirror image of RAG’s: it knows a run book exists and nothing of what the run book says, and it scored 6% on page questions, at more than twice RAG’s price, because every call carries the ontology and the agent tries before it gives up.

Design 3: the agentic hybrid, one agent over both stores

The same agent loop, with four tools: search the documents, read a whole document, resolve names, run Cypher. The loop is the whole mechanism:

for _ in range(ctx.settings.agent_max_steps):          # 8
    result = await ctx.llm.chat(ctx.model, messages, ctx.usage, tools=tool_specs)
    messages.append(result.message)
    if not result.tool_calls:                         # nothing more to fetch: this is the answer
        return result.content
    outcomes = await asyncio.gather(*(                # independent tool calls run in parallel
        execute(ctx, call.function.name, parse_arguments(call.function.arguments))
        for call in result.tool_calls
    ))
    for call, payload in zip(result.tool_calls, outcomes):
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(payload)})

The prompt tells the agent which store fits which need, and the two stores share identifiers: every chunk lists the entity ids it mentions, every node carries its document id. So the agent can find an incident by its description in the text, step onto the graph with its id, follow the dependencies, and step back to the page for the recommendations.

Design it for the questions that need both, and for the ones you cannot classify in advance. On this corpus it was perfect on every shape except the lookups, where one ambiguous question cost it a partial, and it charges for the privilege: three times RAG’s price on pages, and nearly twice the graph’s on sets. The hybrid is not a better RAG. It is a decision you have postponed until runtime, and you pay for the postponement on every call.

What each design scored, by shape

Read the table by row and the designs sort themselves. On the eight page questions, search and the hybrid are perfect and the graph is nowhere. On the five lookups, everything works and search is cheapest. On the eleven wall questions, chains, sets, absences and definitions, search scores 13.6% and the two designs that can traverse score 100%. On the three mixed questions, only the design with both stores is complete.

The wall is not a gradient. Search did not do a bit worse on the hard shapes. It went from perfect to almost nothing in one step, and the step is not about difficulty. The absence questions are simple to state and trivial for a database. They are impossible for a design that only ever sees five passages, because the answer is not in any five passages, or any fifty.

The wall, case by case

Numbers by shape tell you where the wall is. The actual answers tell you what it looks like from the inside. These are four questions from the set, with what each design did, quoted from the recorded runs.

A chain. “If card-vault-db goes down, which customer-facing services are affected, directly or indirectly?” The true answer is four of the six customer-facing services, reached by a chain that runs from the database through the vault, the payments API and the checkout API to the storefront, three or four hops.

Search retrieved exactly what a search should: the vault’s page, the Payments Core team page, the payments API page, the architecture decision about the vault, and the card gateway page. Every one of them mentions the vault. None of them is more than one hop away from it. The model answered that the affected services are “payments-api and card-gateway”, neither of which is customer-facing, because those were the dependents it could see. Better retrieval would have returned the same five pages. They are the right pages for the words in the question. The answer is on pages the question does not mention.

The graph agent read in the ontology that the dependency relationship chains and wrote one query with a variable-length path and a filter on the customer-facing flag. Four rows, correct, 4.1 seconds, 0.70 cents. The model never saw a page and did not need to. On a second chain question, the event stream that feeds seventeen services, search named two and hedged a third; the graph listed seventeen.

A set. “How many incidents did we have in 2025 at each severity level, and which service was affected most often?” The truth is sixteen incidents, four, seven and five by severity, and checkout-api, six times. Search retrieved five postmortems and reported one, two and two, with notification-service most affected, twice. Every number is a faithful count of the five passages it was handed. The graph ran a count grouped by severity and a count of incidents per affected service, ordered and limited to one, and answered exactly. A model is not a bad counter. It is a counter of what it is shown, and retrieval shows it five things.

An absence. “Which Tier 1 services have no run book?” Three of twelve. Search retrieved the five Tier 1 service pages that mention run books and concluded that “all listed Tier 1 services have a run book specified” and that there are “zero Tier 1 services without a run book mentioned in the provided passages”. That last clause is the honest part, and it is the whole problem: a design that reports what it found cannot report what is missing, because a missing runbook is not on any page. The graph wrote a pattern with a negation, WHERE NOT (s)<-[:COVERS]-(:Runbook), and named the three. On the second absence question, which teams own no Tier 1 service, search answered that the passages did not provide the information, and the graph answered in two seconds for a third of a cent.

A definition. “Which services are in PCI scope?” Five services, from a rule in the policy applied to every data store and every dependency: a store that holds cardholder data, the services that touch it, the services that call those. Search found the policy page, cited the rule correctly, and applied it to the five passages it had. It named five services. Two of them were wrong, two of the real ones were missing, and it claimed a database was a cardholder data store when it is not. The graph agent took the definition from the ontology, which carries the rule and its query pattern, and returned the five correct services in 2.9 seconds for 0.40 cents. Nobody had ever written that list down. The shape said it.

What the four cases have in common is the thing I want to be precise about. In every one, retrieval worked. The passages were relevant. The reranker put the right page on top. The model read them faithfully. And the answer was still wrong, because it was never a passage. It was a path, a count, a gap, a rule.

Better retrieval finds better passages

The objection I hear most is that this is a top-k problem, and a bigger k, a bigger context window or a cleverer retriever would fix it. In the second article I tested that on the matrix questions, and the five wall questions in that set are the cleanest place to see the result.

Five passages: 20%. Ten: 0%. Twenty: 30%. Fifty, almost half the corpus: 40%, at three times the price. The whole corpus in one prompt, every page the company has: 50%, and slower than anything else. One graph query: 100%, for 0.54 cents. Ten times the passages bought twenty points and stopped; the whole corpus bought thirty. The chain question was never fully answered at any setting, because no number of pages contains a seventeen-service traversal; each page describes one service, and the model would have to hold the whole dependency graph in its head and walk it backwards. That is what a database is for, and it does it in milliseconds.

Nor is the wall a property of the model. The first article ran the twelve original questions on six models. Here is what each design scored on the six wall questions of that set.

Classic RAG scored 8% on the wall shapes on gpt-4o-mini, gpt-4o, gpt-4.1, gpt-5-mini and gpt-5, and 17% on o4-mini. Five models, one number. The graph agent scored between 75% and 100% depending on how well the model wrote Cypher, and the hybrid between 58% and 100% depending on how well it chose tools. The designs that can traverse vary with the model, because they ask the model to do something. The design that only reads passages does not vary, because the model is not the limit. The five passages are.

One more number belongs here, from the first article. With the ontology removed, the same graph, the same facts, the same model, the graph agent’s score on the wall shapes fell from 100% to 33%. The two chain questions survived, because a dependency edge is a dependency edge with or without a definition. The set, absence and definition questions all failed. The facts were unchanged. Nobody had told the model what “in scope” meant, or that a category sits inside another category, or that a count means all of them. Structure answers the chain. Meaning answers the rest.

What to design when

Put the measurements and the cases together and the design rule is short enough to fit in one table.

Three things to take from it.

Classic RAG is not the baseline you beat. It is the design that owns pages. Procedures, reasons, narratives, anything somebody wrote down, with the fuzzy wording real users bring: it found “billed two times” when asked about being “charged twice”. If your question log is pages, build this and stop. Its cost is flat and it is fast, and it fails only where it was never going to work.

The graph is not a better RAG. It is a different operation. Traversal, counting, negation and rule application are database operations, and they need two things: the data loaded as structure, which most companies already have in their catalogue, tracker and registers, and an ontology that tells the agent what the structure means. Without the ontology the graph is a store of facts that cannot answer a definition. With it, every wall shape fell. And do not point it at pages: it will spend more than search to tell you that the procedure is in a document.

The hybrid is a postponed decision. It answered everything, and it charged for the postponement on every question. Design it for the mixed shapes, the ones that need a set from the graph and prose from the page, and for the question you cannot classify in advance. Route everything else to the design that owns it. The third article measured that routing decision; it costs a hundredth of a cent and it is where the economics are decided.

Where the shapes come from, in a real company

Naissus Pay is invented. The shapes are not, and neither is the structure that answers them. Every company I have worked with already holds, in some system of record, the facts that the wall questions need. The question is only whether that structure has been loaded into something a model can query, and whether anyone has written down what it means.

The service catalogue holds the chains. The ticket tracker holds the sets and, through assignees and status, the absences. The classification register holds the definitions. The org directory holds the lookups. The wiki holds the pages, which is what it is for. Loading a catalogue into a graph is an export and a loader; the cost of a graph, as I wrote before, is mostly the cost of admitting you already own one. Writing the ontology is a week of a person who knows the business, and that person has to own it afterwards, because meaning changes when the business does.

The practical procedure follows from the table. Collect a hundred real questions. Sort them by shape; it takes an afternoon. If the wall shapes are more than a handful, which in every log I have seen they are, find the systems of record that hold the structure, load them, and write the six or ten definitions your business actually uses. Then build search for the pages, the graph for the wall, an agent over both for the mixed questions, and a router in front.

The honest caveats

Twenty-seven questions in seven shapes, on one invented company, with an LLM judge and a computed truth. The per-shape results are the point; the overall averages are not a benchmark. The shape counts are small, two to eight per shape, and I would not read the difference between 90% and 100% as meaningful. The graph was loaded from structure, not extracted from text; a graph extracted by a model from these pages would be a different experiment with a different bill. And the cases are chosen to show the wall clearly; the full set, including the questions where search wins, is in Table 2.

Closing remarks

Not a retrieval problem. A shape problem. Retrieval worked on every wall question in this article. It found the right pages, ranked them well, and handed them to a model that read them faithfully. The answers were still wrong, because they were paths, counts, gaps and rules, and none of those is a passage.

Better retrieval finds better passages. It cannot find what is between them. Ten times the passages bought twenty points on the wall and stopped. The whole corpus in the prompt reached half. One query reached all of it, for half a cent.

Each design owns a shape. Search owns pages. The graph with an ontology owns chains, sets, absences and definitions. The agent over both owns the questions that need both, at a price. None of them is the winner. The question is.

So, if you are tuning a retrieval system that keeps failing on the same questions, before you touch the chunking, take ten of those questions and ask what shape their answers have. If the answer is a path, a count, a gap or a rule, you are not looking at a retrieval problem, and no retrieval will fix it.

Naissus Pay, its people, systems and incidents are invented for this work. Prices are Azure OpenAI list prices as of 30 September 2026. The earlier articles in this series are here and here.

Not a Retrieval Problem. A Shape Problem. was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.

The question that comes after every article

Two articles ago I argued that questions have a shape, and that the shape decides whether search, a graph, or an agent over both can answer them. The follow-up put a rule of thumb on trial and found the cell nobody owns. Both times, the same question came back from readers: fine, so which one do I build?

My answer has not changed. All three. And something that decides.

This article is about the something. It is the last piece of this series, and the one I have come to believe matters more than any of the three designs behind it, because it is the only place where the economics of the whole system are decided. A router that reads each question and sends it to the cheapest design that can answer it is the difference between paying for one design and paying for three. It is also, as it turns out, the place where a system quietly fails when the router is confident and wrong.

Everything below was run on the same invented payments company as before, on Microsoft Azure AI Foundry, with Azure AI Search holding the pages and Neo4j holding the graph. Fifty questions, three router models, every routing decision recorded, and for the 32 questions that have a computed truth, every outcome graded against it.

Two routers, two different decisions

The word “router” now means two things in this stack, and they should not be confused.

The question router decides which design runs: Classic RAG, GraphRAG, or the agentic hybrid. It is a model call you write and own. It knows your stores, your ontology and your prices, because you tell it.

The model router decides which model serves a given call. On Azure it is a deployment that fronts a pool of models and picks one per request from the prompt’s difficulty, its tools and its length. It is the platform’s, you configure its mode, and it knows nothing about your data.

They sit in sequence. The first decides who answers; the second decides how expensive a brain the answerer gets. Most of this article is about the first, because that is where the architecture lives. The second gets its own section, with measurements, near the end.

How the question router works

It is one call to a small model with strict structured output. The prompt is 806 tokens: the six shapes from the first article with the design each maps to, a short description of the three designs, nine rules of thumb, and eight generic examples, none of which is a test question. The schema is the whole contract:

SCHEMA = {
    "type": "object",
    "properties": {
        "shape": {"type": "string", "enum": ["page", "chain", "set", "absence", "definition", "mixed"]},
        "route": {"type": "string", "enum": ["rag", "graph", "hybrid"]},
        "confidence": {"type": "number", "description": "0 to 1, how sure you are of the route"},
        "reasoning": {"type": "string", "description": "one sentence, at most 30 words"},
    },
    "required": ["shape", "route", "confidence", "reasoning"],
    "additionalProperties": False,
}
fallback = decision["confidence"] < 0.6 and decision["route"] != "hybrid"
route = "hybrid" if fallback else decision["route"]

The model cannot invent a fourth route or skip the confidence. Below a confidence of 0.6, the question goes to the hybrid, which can answer anything at a price. On gpt-4o-mini, the demo’s router, a decision takes about a second and costs 0.016 cents. On gpt-4.1 it takes 1.4 seconds and 0.22 cents, more than half of what Classic RAG charges to answer the question. I will come back to what the confidence is actually worth.

The experiment

Three question sets.

• The 32 answered questions. The twelve from the first article and the twenty from the second. For every one, I have the recorded verdict and cost of all three designs on gpt-4.1. That lets me replay any routing strategy without running a model: route the question, look up what the chosen design did and what it cost, add the router’s own cost.
• 18 new questions written to fool the vocabulary. “How many minutes did the double-charge outage last?” sounds like a count; it is one number on one page. “Give me the full list of steps to fail over to the backup acquirer” sounds like a set; it is a procedure. “Compare the recovery point objective of payments-db and ledger-db” sounds like analysis; it is two sentences on two pages. These have no computed answers and were never run through the designs; they test only whether the router reads shape or keywords. Each has an expected route decided by where the answer lives in the data.
• The same 20 matrix questions again, with the router given one extra thing: a compact description of what the graph actually holds.

Three router models: gpt-4o-mini, gpt-4.1 and gpt-5-mini. Strategies compared against them: always one design, all three at once, and an oracle that picks the cheapest design that answered correctly, which is the ceiling any router can reach.

What routing is worth

Start with the ends of the scale. Running all three designs on every question and keeping the best answer scores 92% and costs 1.99 cents a question. The oracle scores the same 92% for 0.58 cents, because it never pays for a design that was not needed. That gap, 1.41 cents on every question, is what a routing decision is worth at its best. Always running the hybrid, the design that can do everything, lands at 91% and 0.97 cents: most of the saving, not all of it, and one point short on accuracy.

Now the routers. All three models score 83% and send roughly two thirds of the questions to the graph. gpt-4o-mini does it for 0.64 cents a question, gpt-5-mini for 0.63, gpt-4.1 for 0.81, where the difference is almost entirely the router’s own price. So a router costing a hundredth of a cent captured most of the oracle’s saving. It also lost eight points of accuracy against always-hybrid, and I want to be precise about where.

Split the 32 by origin and the picture sharpens. On the first twelve questions, every router scores 100%, the same as always-hybrid, and the gpt-4o-mini router does it for 0.94 cents against the hybrid’s 1.26, a quarter less. On the twenty matrix questions, the routers score 72.5% against the hybrid’s 85%. Every point of that difference comes from one cell: text, worked out. The trap from the second article. The router reads “which run books contain a rollback procedure” as a set, which it is, and sends it to the graph, which is right in shape and wrong in fact, because procedures were never loaded into the graph. The graph agent answers that no run books mention rollback. The router did its job. The data had not been given the shape the router assumed.

A router can only choose among the cells that have an owner. Where no design answers, the router cannot help, and it will not know that it did not help.

Over-routing and under-routing

gpt-4o-mini differed from the oracle on ten of the 32 questions. Seven of those were over-routes: the chosen design answered correctly but a cheaper one would have too. Four lookups about ownership, on-call and dependencies went to the graph at 0.42 cents instead of search at 0.31, because the router’s rules say ownership is structure, and they are not wrong. Three questions went to the hybrid when a cheaper design would have done. Over-routing is the cheap mistake: you pay a little more and you get the answer.

Three misroutes cost correctness. Two sit in the trap cell: text-only questions sent to a graph that had nothing to query. The third sent a lookup to the graph and got a partial, because the question said “services” and the graph returned a data store as well, which the judge did not forgive.

That asymmetry is the first design rule I would carry out of this: when in doubt, over-route. A wrong cheap answer is the expensive one, because somebody acts on it. The hybrid is the right default for doubt. The problem, as the next section shows, is that the router is never in doubt.

The words lie, and the router mostly sees through them

The 18 routing questions were written to separate reading shape from matching keywords. By the strict count, where only the cheapest correct design is a hit, gpt-4.1 and gpt-5-mini routed 16 of 18 as expected and gpt-4o-mini 13 of 18. Two of the labels are judgement calls, and the table marks them: the on-call lookup, which the graph answers as well as search does, and the two-part question about personal data, which the hybrid answers as well as the graph does. Count those as hits and the two larger models are 18 of 18. I report the strict number because it is the one I would have to defend.

The successes are the interesting part. “How many minutes did the double-charge outage last?” went to search on all three models, despite “how many”. “Give me the full list of steps to fail over to the backup acquirer” went to search, despite “full list”. “What changed after the payments database migration outage, and is anything still pending?” went to search on two of three, despite “pending” sounding like a status query. A router that reads the question as a whole handles the vocabulary that trips keyword rules.

The failures are instructive too. “Compare the recovery point objective of payments-db and ledger-db” went to the graph on all three models, with confidence of 0.90 to 0.95. The RPO is a sentence on each store’s page; the graph, before the second article’s extension, did not hold it. The word “compare” won. And “which data stores hold personal data, and which services write to them” went to the hybrid on two models, an over-route: both halves are in the graph, but a question with two parts reads as “mixed”. Over-routing again, and again the safe direction.

The confidence that means nothing

In the first article I wrote that over-routing costs money, under-routing costs correctness, and that the confidence threshold “is the dial between them”. I measured the dial. It is not connected to anything.

Every miss came with high confidence. gpt-4.1 reported 0.95 or more on all 32 decisions, including the six where the routed design did not answer correctly. gpt-4o-mini never went below 0.70 on any of its fifty decisions, and its three misses that cost correctness all carried 0.90. gpt-5-mini’s lowest was 0.86. The fallback to the hybrid, which triggers below 0.6, fired zero times in 150 decisions across the three models. The router is not calibrated; it is confident.

The sweep makes it plain. Raise the threshold from 0 to 0.85 and the score does not move. Raise it past the model’s habitual confidence and every question falls through to the hybrid, and you are paying always-hybrid prices with a router in front. gpt-5-mini, whose confidence spread is slightly wider, catches one miss at a threshold of 0.9. That is the whole benefit, on this data, of asking the model how sure it is.

I do not think this is a property of these three models. A model asked to grade its own decision in the same breath as making it will mostly agree with itself. If you want a signal, you need one the router cannot generate.

The signal the router cannot see, and the design can

There is a better signal, and it was in the recorded runs all along. On four of the five trap questions, the graph agent never ran a query. It read the ontology, saw that action items, procedures, retention and recovery objectives were not there, and said so. On the fifth it ran one query and got zero rows. The design knew it had nothing. The router did not.

So I replayed one more strategy over the recorded runs: route as before, and if the chosen design is the graph and the agent either gives up without a query or gets only empty results, escalate to the hybrid and pay for both. On gpt-4o-mini this fires on exactly the five trap questions and nowhere else. The score goes from 83% to 91%, the same as always-hybrid, at 0.75 cents against the hybrid’s 0.97. A quarter cheaper, for the same accuracy, with a rule that costs nothing to implement: a count of rows.

That is the second design rule. Do not fall back on the router’s confidence. Fall back on the design’s result. An empty graph result, a search whose top passages do not mention the entity asked about, an agent that answers without calling a tool: these are observable, and they are honest in a way a self-reported number is not.

The router that knows what the graph holds

The trap cell exposed a deeper limit. The router reads the question. It does not know the data. “Which run books contain a rollback procedure” is a set, and the router is right to say so; the mistake is assuming the set exists somewhere queryable. So I gave the router one more thing: a compact list of what the graph actually holds, one line per class with its properties and relationships, 242 tokens, and one rule: if a question needs something complete or counted that the graph does not hold, send it to the hybrid.

Figure 6 is one question, end to end, from the records. Without the hint, gpt-4.1 routed the rollback question to the graph at 0.95, and the graph agent answered that no run books mention it. With the hint, the same model routed it to the hybrid at 0.80, with the reason “this information is only in document prose, not the graph”, and the hybrid searched the run books and named both. After the structure from the second article was loaded and the hint updated to say so, the router sent it back to the graph at 0.90, “a graph query over Run book and Procedure”, and the graph agent answered with one query.

Across the set, the hint took all three routers from 72.5% to 82.5% before the structure existed, by moving three to five of the trap questions away from the graph. It did not reach always-hybrid’s 85%, because the hybrid itself only answered half the trap questions, and because gpt-4o-mini used the new rule to over-route a counting question that the graph could have answered. A router that knows the data is a better router. It is not a substitute for the data having the right shape.

After the structure was built, the same routers, told nothing new, scored 97.5% on the matrix set, against 95% for always-hybrid, at 0.60 cents against 0.77. With the owner of every cell in place, a hundredth-of-a-cent router beat the design that can do everything, on both score and price. That is the sentence I wanted from this whole series, and I did not get to write it until the trap was closed.

The other router: letting the platform pick the model

The model router is a different animal, and it is worth being precise about what it is. On Azure AI Foundry it is a deployment that fronts a pool of models. Every call to it is classified and handed to one of them, in one of three modes: cost, balanced or quality. You never name the model. The response tells you which one answered, and nothing else. I ran all three designs on the twenty matrix questions through each mode, the same questions and judge as before, so the fixed-model numbers from the earlier articles are directly comparable.

Four things came out of it.

The pool is not yours, and it moved under me. When I first wired the model router into this setup, a week before the previous article’s runs, the pool served three models. By the time of these runs it served nine, including three from a newer generation and one that is not an OpenAI model at all. In cost mode, the small models took almost every call. In quality mode, the graph and hybrid agents were served by the same frontier model on every single call. In balanced mode, the router escalated between a third and a half of the agents’ calls to a frontier model, and it was the mode where the hybrid scored lowest, 87.5%, partly because one question, about what breaks if the event stream fails, was refused outright by a content filter on the reasoning model that happened to be chosen. A question about an outage, flagged as a policy violation, by a model I did not pick. That is the price of not picking.

Cheap and good are not opposites here. Cost mode’s hybrid scored 92.5%, higher than gpt-4.1’s 85% on the same questions, for 0.19 cents a question against 0.80. The small models in the pool read documents far more willingly than gpt-4.1 did, nineteen full document reads across the five trap questions against none, which is why they did better in that cell, 70% against 50%. Quality mode’s hybrid scored the same 92.5% for 1.94 cents: ten times the price for the same answers. Classic RAG did not care which mode it ran in, 60 to 62.5% across all three, against 65% on gpt-4.1. When the design is the ceiling, the model router is choosing how much to pay to hit it.

The model router cannot fix the shape. Look at the cells. Through every mode, GraphRAG scored 100% on the connected, worked-out cell and 0% on the trap, and Classic RAG scored 100% on the written-down cells and 0 to 10% on the connected, worked-out one. The same cells, with the same owners and the same orphan, as on every fixed model in two articles. The model router picks a brain. It cannot pick a store, and it cannot know that action items were never loaded into the graph. That decision is upstream of it, and it is the question router’s.

Latency needs an asterisk, and the asterisk is instructive. The median time for the agent designs through the model router came out at 27 to 35 seconds against 3 to 5 seconds on fixed models. That number is real and it is not the models’ fault. Nearly every agent step took 13 to 15 seconds, regardless of mode or served model, while plain search calls took two to four. When I disabled the client library’s silent retries and ran the same calls again, they failed instantly with a token rate limit for the served model on the router resource. The agents’ prompts, with the ontology and the tool definitions, exhaust the quota of a pool model in a few calls; the SDK waits and retries, and the wait shows up as model latency. In a clean micro-benchmark on an idle quota the router answered the full ontology prompt with tools in 1.5 to 2.5 seconds, the same as gpt-4.1. So the measurement is correct, the explanation is quota, and the lesson is operational: a model router shares each pool model’s rate limit with whatever else uses it, and you will not see that in the response, only in the clock.

Where that leaves the model router in this stack: underneath, not in front. It is a good way to stop paying frontier prices for routine calls, and in cost mode it beat a fixed mid-range model on both price and score. It is not a routing decision in the sense this article is about, because it does not know what the question needs, only how hard the prompt looks. Put the question router first. Let the model router decide how much brain the chosen design gets.

The stack, then

Put the pieces in order and the architecture is short.

1. A question router in front, small, cheap, structured output, with the shapes and the designs in its prompt, and a line for each thing your graph holds. Give it the rule to over-route when a question needs something the graph lacks.
2. Three designs behind it. Search for pages, a graph with an ontology for structure, an agent over both for the questions that need both.
3. A fallback on results, not on confidence. An empty query, a search that misses the entity, an agent that answers without a tool: escalate to the hybrid and pay for both. On this data that rule alone recovered every point the router had lost.
4. A model router underneath, if the price is right for your call mix. It decides how much brain each call gets. It does not know, and cannot be told, what the question needs.
5. Collect the questions, keep the routing log, and read the misses. Every misroute in this article was found by comparing the router’s choice with what the designs actually did. That comparison is free once both are recorded, and it is the only calibration the router will ever get.

The honest caveats

Thirty-two answered questions and eighteen routing-only ones, on one invented company. The strategies were replayed over recorded gpt-4.1 outcomes, not re-run, which is exactly why they are comparable and also why they inherit every quirk of those runs, including the judge’s strictness on one ambiguous lookup. The oracle is defined from the same outcomes and is the ceiling for these questions, not a benchmark. The 18 routing questions have expected routes that I decided from the data; the two that could be argued either way are marked in Table 2 and counted strictly. The data hint was written knowing which facts the trap questions needed, which is what the second article’s first step tells you to do, and also a reason to believe it would be less sharp written cold. And the escalation rule was replayed, not deployed: it is a count of rows, so I expect it to behave, but I have not watched it in production.

Closing remarks

The router is where the money is decided, and it is almost free. On questions the designs can answer, a hundredth-of-a-cent decision matched the best design’s accuracy at a quarter less cost, and the oracle shows there is more to take.

A router cannot route to a design that does not exist. Every point it lost was in the cell nobody owned. Give the router a list of what the graph holds and it steps around the trap; build the structure and the trap is gone, and the router beats the hybrid outright.

Do not trust the confidence. Trust the result. Every miss came with a confidence of 0.86 or higher. The design that was wrongly chosen knew it had nothing, and said so, and that is the signal to fall back on.

So, one sentence: route first, measure the routes, and when a design comes back empty-handed, let that be the dial.

Naissus Pay, its people, systems and incidents are invented for this work. Prices are Azure list prices as of 30 September 2026. Four models in the model-router pool (gpt-6-luna, gpt-6-sol, gpt-6-astra and grok-4–1-fast-reasoning) have no Microsoft price page I could find; their rates are the Global Standard figures reported by third-party trackers. The first two articles are here and here.

Route First, Retrieve Second was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.

A rule I had never measured

In the first article I argued that questions have a shape, and that the shape, not the model, decides whether a retrieval system can answer them. Near the end of the talk that article came from, there was one slide I had drawn from experience rather than from data. Two questions, four cells. I called it “my rule”, because that is what it was: the way I decide, after years of shipping these systems, what to build for a given company.

Here is the rule. Before you choose a vector database, a graph database, a model or a framework, ask two things about the questions your users will bring:

1. Is the answer written down somewhere, or does it have to be worked out? A procedure in a run book is written down. “Which of our systems are in audit scope” is worked out: a rule applied to everything you own.
2. Where does the knowledge live: in text, or in connected data? A wiki page is text. A service catalogue, an org chart, a ticket tracker are connected data, whether or not anyone has ever called them a graph.

Two questions, two answers each, four cells. Each cell has a design that owns it, and one of them has a trap in it. I have drawn this on whiteboards for clients more times than I can count. What I had never done was test it. This article is the test.

The four cells

Laid out as a matrix, the rule looks like this.

Text, written down. Someone wrote the answer, and your job is to find it. This is Classic RAG, and it is the cheapest thing that works. If every question your users ask lands here, stop reading and ship it.

Connected, written down. The answer is a single fact: who is on call, which team owns this, what this service depends on. It lives in structured data and it is also, usually, repeated on some page. Any design answers it. The point is that none of them should be paid for it. This is a lookup, a form field, an API call. The language model is at most there to write the query.

Connected, worked out. What breaks if this database fails. Which teams own nothing critical. Which services touch restricted data. The answer is not in any record; it is between them, or it is a rule applied to all of them. A graph with an ontology owns this cell, because a database can traverse, count and compare exhaustively, and a language model cannot.

Text, worked out. The trap. The answer must be assembled from many pages, and nothing structured holds it. “How many promises from our postmortems are still open?” “Which run books cover a rollback?” “List every project we did in retail.” This is the cell my first chatbot died in, years ago, and the rule says something uncomfortable about it: no retrieval trick fixes it. Build the structure first.

That last claim is the one I most wanted to check, because it is the expensive one. If I am wrong, a bigger context window or a better reranker solves the problem for a fraction of the cost. If I am right, a lot of teams are tuning the wrong thing.

The experiment

I used the same invented company as before. Naissus Pay is a fictional payments company with 29 services, 19 data stores, 8 teams, 20 incidents with postmortems, 10 run books, 6 architecture decisions and 4 policies. Its knowledge exists twice, generated from one source of truth: as 105 wiki-style pages, chunked into 108 passages of about 17,000 tokens in total, in Azure AI Search; and as a knowledge graph of 165 nodes and 386 relationships with an ontology on top, in Neo4j. The graph was loaded from the structured source, the way a company would export it from its service catalogue. Nothing was extracted from text by a model.

Then I wrote twenty new questions, five per cell, each with a ground truth computed from the data. I placed them deliberately: not by how they sound, but by where the answer actually is. “Which postmortems have an action item about adding an alert” sounds like a search. It is not. Action items live only inside the twenty postmortem pages, and the graph, loaded from the catalogue, has no idea they exist. That is the trap cell by construction.

The three designs are the ones from the first article, unchanged. Classic RAG: hybrid search with a semantic reranker, five passages, one model call. GraphRAG: an agent that reads the ontology, resolves names and writes read-only Cypher, with no access to prose at all. The Agentic Hybrid: the same agent with four tools over both stores. All three ran on gpt-4.1, and an LLM judge with the ground truth and a list of key facts graded every answer correct, partial or incorrect. The judge is strict about sets: “exactly these five” with four listed is partial, not correct.

Two things are new. To test the claim that no retrieval trick fixes the trap, I ran Classic RAG again with 10, 20 and 50 passages instead of 5, and then once more with no retrieval at all: all 108 passages in the prompt, the brute-force ceiling. And after the first round, I did what the matrix tells you to do with the trap cell, and measured again. That comes last.

What happened, cell by cell

Text, written down: RAG 100%, hybrid 100%, graph 10%. Exactly as drawn. Freeze periods, the reasoning behind the event stream, what to do when the bank rail rejects a batch: Classic RAG found the page every time, in about two seconds, for less than half a cent. The graph agent, which holds no prose, said so, honestly and unhelpfully: “The content of the change management policy, including details about freeze periods, is not present in the knowledge graph.” It picked up a single partial on the incident question by returning the node’s metadata. The one thing I would add to the slide is a warning: in this cell, the graph is not merely useless, it is expensive and useless, at more than twice RAG’s cost per question, because the agent kept trying.

Connected, written down: everyone passes. RAG 100%, graph 90%, hybrid 90%. The one miss is instructive. Asked “which services does checkout-api call directly”, both agents returned the four services and also session-cache, which is a data store checkout-api depends on. The query was right; the question was ambiguous; the judge was strict. In the real world that is a conversation, not a defect. What matters in this cell is the bill: 0.31 cents for RAG, 0.42 for the graph agent, 0.60 for the hybrid, for a fact that a form field could return for nothing. The slide says “just a query”, and the numbers say the same: this is the cell where the chatbot should not be in the loop at all.

Connected, worked out: graph 100%, hybrid 100%, RAG 20%. The mirror image of the first cell. “If events-stream goes down, which services are affected, directly or indirectly?” The true answer is seventeen services, because the stream feeds notifications, fraud scoring, loyalty, audit logging and analytics, and those feed checkout, onboarding, chargebacks and the wallet. Classic RAG, reading the five best pages, named two and hedged a third. The graph agent wrote one query with a variable-length path and listed all seventeen. “Which teams own no Tier 1 service?” RAG answered that “the passages do not provide information about any teams that own no Tier 1 service”, which is true of the passages and false of the company. The graph agent wrote a NOT EXISTS pattern and named the two teams. Nothing in this cell can be found; it can only be computed.

Text, worked out: RAG 40%, graph 0%, hybrid 50%. The trap, measured. The graph agent scored zero, exactly as the matrix predicts, because it had nothing to query. Classic RAG, with five passages, counted four open action items where there are eleven, missed one of the five postmortems that promised an alert, and declared that no data store has a recovery time objective above two hours, because none of the three that do were in its passages.

The hybrid’s failure is the one I keep thinking about. Asked how many action items are still open, it did the right thing: it recognised a counting question and went to the graph. It wrote a query against an ActionItem label with an OWNED_BY relationship to teams. Neither existed. The query returned nothing, and the agent answered: "There are no open postmortem action items currently." The model had the right design in its head. The data had not been given the shape the model assumed. I cannot think of a cleaner demonstration of what the trap cell is: not a failure of retrieval, not a failure of reasoning, but a shape that nobody built.

Can you brute-force the trap?

The obvious objection to “build the structure first” is that context windows are now enormous. Why not just retrieve more, or retrieve everything? I tried both.

More passages help the trap cell, up to a point. Going from 5 to 50 passages took the text-worked-out score from 40% to 80%, and the cost from 0.37 to 1.40 cents per question, roughly four times. At 50 passages the model has almost half the company in front of it, and it can do a fair amount of counting. It still got the open action items wrong, in the other direction this time: thirteen instead of eleven, with two teams over-counted. Reading forty pages and tallying across them is not what language models are reliable at, and the judge noticed.

The connected cell barely moved. Twenty, thirty, forty percent. The chain question stayed wrong at every setting, because no number of pages contains the seventeen-service answer; each page describes one service and what it calls, and the model would have to hold the whole dependency graph in its head and walk it backwards. That is a traversal, and traversal is what databases are for.

And something slipped. At 50 passages the lookups, which had been perfect, dropped to 80%. Asked which architecture decisions concern the card gateway, the model now had more candidates to be wrong about, and it added a decision that merely mentions the gateway in passing. More context, more confident noise.

Then I removed retrieval altogether and put all 108 passages, about 17,000 tokens, into one prompt. Two surprises. First, it was cheaper than fifty passages, 0.99 cents against 1.40, because the corpus is identical on every call and Azure’s prompt caching billed almost all of it at the cached rate. Second, it was worse. The trap cell fell back to 60%, the lookups stayed at 80%, and on the gateway question the model produced a decision that does not exist, attaching the title of one record to the identifier of another. It also took 5.6 seconds at the median, more than twice Classic RAG. The whole company in the prompt is a worse reader than five good passages on simple questions, and an unreliable accountant on the hard ones.

So the honest answer to the objection is this. Brute force bought the trap cell forty points, at four times the price, with a ceiling of 80%, on a corpus small enough to fit in a single prompt. Your corpus does not fit in a prompt. And even here, it never reached the number a database returns for free.

Then I did what the matrix says

The rule for the trap cell is “build the structure first”. That is easy to write and expensive to ignore, so I did it, and measured.

Four things that had existed only in the pages got a place in the graph. Action items became nodes, each linked to its incident and assigned to its team, with a status. Run book procedures became nodes under their run book. Data stores got their recovery objectives and their retention, in years, as properties. Services got their availability target and latency budget. In the demo this is a loader reading the source of truth; in a real company it is four exports: the incident tracker, the run book index, the disaster-recovery register and the SLO catalogue. Every one of those systems already exists in every company that has postmortems and run books. I did not extract anything from text with a model, because I did not have to, and neither would you.

The cost of the structure: 62 new nodes, 101 new relationships, properties on 48 existing nodes, and two classes, three relationship types and one business definition added to the ontology. The ontology grew from 1,854 to 2,205 tokens. Then I ran the same twenty questions, on the same model, with the two agent designs.

GraphRAG on the trap cell: 0% before, 100% after. The same question that produced “the knowledge graph does not contain information about postmortem action items” now produced the right eleven, with the right count per team, from one query:

MATCH (i:Incident)-[:HAS_ACTION_ITEM]->(a:ActionItem {status: 'open'})-[:ASSIGNED_TO]->(t:Team)
RETURN t.name AS team, count(a) AS open_items

The retention question became WHERE d.retention_years > 5. The recovery-time question became WHERE d.rto_minutes > 120. The rollback question became a match on a procedure name. The alert question, which is a classification over free text, became WHERE toLower(a.name) CONTAINS 'alert', which is a cheap trick, and it worked, because the text was now short, labelled and attached to the thing it describes. Each answer was exact, complete, and cost about half a cent.

The hybrid: 50% before, 90% after. Its one remaining miss is worth a sentence. On the alert question it was offered both stores and chose the documents, because the question sounds like a search, and it missed one postmortem. The pure graph agent had no such choice and got it right. An agent with more options is an agent with more ways to take the familiar road.

Nothing else moved. The three cells that already had an owner scored the same before and after, which is the result you want from adding structure: it fixes the thing it was built for and breaks nothing.

The same picture on a smaller model

To check that the cells are a property of the designs and not of gpt-4.1, I ran the three designs again on gpt-5-mini, on the original graph, without the structure.

The shape is identical. Classic RAG is perfect on both written-down cells and collapses on both worked-out ones. The graph agent owns the connected, worked-out cell and scores zero in the trap, for the same reason: there is nothing to query. The hybrid did a little better in the trap on this model, 70% against 50%, because it leaned harder on reading whole documents, and a little worse on the connected cell, where it twice added an item the judge did not accept. The prices are different, about a fifth of gpt-4.1’s, and the latency is higher. The cells do not move. Which is the point: the matrix describes the data and the questions, not the model, and swapping the model changes the bill, not the shape.

The matrix, measured

Here is the slide again, now with numbers on it.

Three things to take from the cost table. Classic RAG costs the same everywhere, about four tenths of a cent, whether it is right or wrong, which is exactly the problem: it has no way of knowing it is in the wrong cell. The graph agent is cheapest in the trap cell before the structure exists, because it gives up fast; honesty is cheap. And the hybrid charges a premium in every cell, which is the price of not having decided. That premium is worth paying for questions that straddle cells, and worth routing around for everything else.

What this means if you are deciding

I write these for two readers: the engineer who will build it, and the person who will pay for it. For both, the matrix turns into a procedure.

Start by sorting the questions, not the data. Take a hundred real questions from your tickets, your chat logs, your on-call channel. Put each one in a cell. In my twenty, five were pages. In a real log the proportion is not kinder, and the ones that hurt are the ones that sound simple: “how many”, “which ones”, “list all”, “is anything missing”. Those words mean “worked out”, and worked out means a database, not a search.

The written-down cells are nearly free. Treat them that way. Classic RAG owns one of them, and a plain lookup owns the other. If a vendor is charging you agent prices for “who owns this service”, you are paying for a decision that was never needed.

The connected, worked-out cell is where a graph earns its keep. Not because graphs are fashionable, but because the question is a traversal, a count or a rule, and those are database operations. The ontology is what lets a model turn the question into the query; without it, as the first article showed, the same graph is blind.

The trap cell is not a retrieval problem, and it will not respond to retrieval budgets. More passages took it to 80% at four times the cost and then stalled. The whole corpus in the prompt did worse. Structure took it to 100% for half a cent a question. The structure was not expensive, because it was not extracted; it was exported from systems you already run. The real cost is organizational: somebody has to own the fact that action items live in the tracker and the tracker feeds the graph. That person is rarely in the AI team.

When the text really is only text, when nobody has ever classified your projects by industry and there is no system to export from, then yes, you classify once, with a model if you like, as a batch job with a human checking the output. But you store the result as structure and query it forever. That is still building the structure first. It is not retrieval.

And route. Every real company is a mix of all four cells. The mature architecture is not a bet on one design; it is three designs behind something cheap that reads the shape of each question before anything runs. I described the router in the first article, and nothing here changes it, except that the trap cell now has a destination.

The honest caveats

Twenty questions, chosen to exhibit the cells, on one invented company. The per-cell results are the point; the overall averages are not a benchmark. The judge is a language model with a reference answer, and I read every verdict; the three partials that came from ambiguous questions are noted above. The whole-corpus baseline benefited from prompt caching in a way a real workload with varied prefixes would not. And the structure I added was chosen knowing the questions, which is exactly what step one tells you to do, and also exactly why you should collect the questions before you build.

One more. The corpus here is 17,000 tokens. The brute-force experiment is only possible because of that, and I ran it so that nobody has to wonder. On a corpus of 17 million tokens, there is no brute force; there are only the four cells.

Closing remarks

Two questions before any technology. Is the answer written down, or must it be worked out? Does the knowledge live in text, or in connected data? Everything downstream, the store, the design, the model, the bill, follows from those two answers. The rule I had been drawing from experience held in three cells out of four, and the fourth held once I did what the rule said.

Written down is cheap. Worked out is a database. Search finds what somebody wrote. Counting, chaining, comparing and applying a rule are not search, and no amount of context turns them into search. The measured ceiling for brute force was 80%, at four times the price. The database returned 100% for half a cent.

The trap is a shape nobody built. The agent that answered “there are no open action items” had already written the right query. It was querying a structure that did not exist yet. The fix was four exports and a loader. The hard part was knowing which four.

So, if there is room for only one sentence: before you tune retrieval, ask whether the answer was ever written down. If it was not, retrieval was never the job.

Naissus Pay, its people, systems and incidents are invented for this work. Prices are Azure OpenAI list prices as of 30 September 2026. The first article, with the three designs, the ontology and the router, is here.

Written Down or Worked Out? The Two Questions That Decide Your AI Architecture was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.

The day my RAG was perfectly wrong

Two years into shipping retrieval-augmented systems, the business came to the R&D team with a request that sounded trivial. Make the company chatbot answer questions like these:

“Which of our use cases involve RPA in banking?” “List every retail project we have delivered.”

Every case study was already in the knowledge base. I expected an afternoon of work.

The chatbot found the five best-matching case studies, offered three examples, cited every one, and stopped. Retrieval was flawless. The answer was fluent. And it was wrong, because the true answer was every retail project across the whole archive, and “related to RPA in banking” needed every case study classified by industry and by technology. Top-k retrieval sees five pages. “All” needs each one.

Nothing was broken. That was the painful part.

I did what engineers do: I worked around it. Custom metadata on every chunk, industry, technology, year, so I could filter. It worked until someone asked a question that needed a tag nobody had added. Then files that listed every article on a topic, chained to each other, an index kept by hand. Properties on things. Links between things. I was building a graph by hand, and I had not admitted it yet.

The lesson took me longer to accept than to write: the answer was not in any passage. It was in all of them at once. No amount of better retrieval fixes that, because retrieval finds passages. The question was asking for a shape.

This article is for the engineer who has shipped a RAG system and hit this wall, and for the product owner who has to decide what to build next and what it will cost. Everything below was measured on a working system, and the numbers are real.

Questions have a shape

Here is the reframing that changed how I design these systems. Before asking which model, which vector database, or which chunking strategy, ask what shape the answer has.

• A page. “How do I roll back the ledger service?” Somebody wrote the answer down. The job is to find it by meaning, and search does that very well.
• A chain. “If this database fails, what else breaks?” The answer lives between records. The page about the database does not mention the services four hops away that depend on it.
• A set. “How many incidents did we have, by severity?” The answer needs every record. Sampling the top five and counting them is not an approximation; it is a different question.
• An absence. “Which critical services have no runbook?” No document says “there is no runbook here”. The answer can only be found by comparing two complete sets.
• A definition. “Which services are in audit scope?” A rule applied to data: a service is in scope if it touches a store holding card data, or calls something that does. Nobody wrote the list, and the list changes whenever a dependency changes.
• All of the above. “That outage where customers were charged twice: what failed, who depends on it, what did we promise to fix?” Structure and prose in one answer.

The first shape is the one everyone demos. The other five are the ones that arrive in week two of production, usually from the business, usually phrased as if they were simple.

Here is the claim the rest of the article defends: what a question-answering system can answer is bounded by the shape of the data it keeps and the shape of the questions it is asked. The model only decides how fluently it hits or misses that bound.

Data has a shape too

Consider one fact, kept two ways.

On the left, a wiki page. The checkout service calls the payments service, the fraud scoring service, the merchant service. After embedding, this passage is a vector of 1,536 numbers that captures what it resembles. Ask it anything that sounds like a payment question and it will surface.

On the right, the same sentence as edges. Checkout depends on payments. Payments depends on the card vault. The vault depends on its database. Each edge is a fact about how two things relate, and a database can follow them in any direction, at any depth, exhaustively.

A vector knows what a sentence looks like. A graph knows what it is attached to. Similar is not the same.

This is also why the advanced RAG toolbox plateaus on the hard shapes. I have used all of it, in production, and it pays. A context prefix on every chunk, the technique Anthropic calls contextual retrieval, measured at a 35% reduction in retrieval failures, and 67% with contextual keyword search and reranking on top. Hybrid search plus a semantic reranker, which took Microsoft’s own benchmark from 43.8 to 60.1 NDCG@3. Query rewriting, caching, humans in the loop. Every technique fixes one specific failure of finding. None of them changes what kind of thing you can find. On multi-hop questions, the MultiHop-RAG benchmark put GPT-4 at 0.56 accuracy with retrieved chunks and 0.89 when handed the right evidence. The gap is not retrieval quality. It is shape.

Facts and meaning are different things

Two words get mixed up constantly in this space: knowledge graph and ontology. The difference is the heart of the matter, so let me use a family.

The facts: Ana is Marko’s mother. Marko is Petar’s father. That is a knowledge graph.

The meaning: mother and father are kinds of parent, and a parent’s parent is a grandparent. That is an ontology, in Gruber’s 1993 words “an explicit specification of a conceptualization”.

Now ask: who are Petar’s grandparents? Nobody ever wrote “Ana is Petar’s grandmother”. No fact says it. The facts plus the meaning answer it, every time, for every family.

The same split exists in a company. The facts: the vault database stores card numbers; the payments service depends on the vault. The meaning: a card number is a kind of cardholder data, which is a kind of restricted data; “depends on” chains; and a service is in PCI scope when it touches a store holding cardholder data, or directly calls a service that does.

Walk it left to right. Card numbers are cardholder data. Two stores hold them: the vault database, obviously, and the settlement files, because the acquiring banks deliver full card numbers in their nightly files and somebody put those files in a bucket. Three services touch those stores: the vault, the reconciliation job, and the analytics pipeline. Two more call the vault directly. Five services, three teams.

Ask any engineer in that company and you will get two names. Nobody would have listed the analytics pipeline, including the analytics team. No document says it. No single edge says it. The shape says it.

Facts change every day: a deploy, a hire, an incident. Meaning changes when the business or the regulator does. That difference matters for cost and ownership, and I will come back to it.

Three designs, one corpus

To test this properly I needed a company whose data I could reason about completely, so I invented one. Naissus Pay is a fictional payments company: a hosted checkout, card payments through two acquiring banks, a ledger and merchant payouts, fraud scoring, a consumer wallet, eight engineering teams. Its knowledge exists twice, generated from a single source of truth:

• As pages, the way a wiki holds it: 105 documents, 108 chunks, about 17,000 tokens. Service pages, data store pages, team pages, 20 incident postmortems with action items, 10 runbooks, 6 architecture decisions, 4 policies. Written the way wikis are written: each page describes its subject and what it depends on. Nobody maintains the page of what depends on it.
• As a graph, the way a service catalogue holds it: 165 nodes, 386 relationships, in Neo4j. Teams own services, services depend on services, stores and external providers, stores hold categories of data in a taxonomy, incidents affect services and have root causes, people are on call. On top, an ontology with 16 classes, 13 relationship types and 6 business definitions.

Because both come from one source of truth, every question has a computed ground truth, and an LLM judge can grade each answer against it. One honesty note that matters for the cost discussion later: the graph was loaded from the structured source, the way a real company would export it from a service catalogue or a CMDB. It was not extracted from the text by a model.

Design 1: Classic RAG

This is a strong baseline, not a straw man. Hybrid search in Azure AI Search, vectors and BM25 fused by reciprocal rank, then the semantic reranker, then one generation over the top five passages. Trimmed of its tracing, the whole pipeline is this:

async def run_rag(ctx, question: str) -> str:
    vector = (await ctx.llm.embed([question]))[0]
    hits = await ctx.search.hybrid_search(question, vector, top_k=5)

    passages = "\n\n".join(f"[{h.rank}] {h.content}" for h in hits)
    messages = [
        {"role": "system", "content": RAG_SYSTEM},   # answer only from the passages, cite them, say what is unconfirmed
        {"role": "user", "content": f"Passages:\n\n{passages}\n\nQuestion: {question}"},
    ]
    result = await ctx.llm.chat(ctx.model, messages)
    return result.content

async def hybrid_search(self, query: str, vector: list[float], top_k: int = 5) -> list[Hit]:
    return await self._run(
        search_text=query,                                    # BM25 over title, section, content, entity ids
        vector_queries=[VectorizedQuery(vector=vector, k_nearest_neighbors=50, fields="content_vector")],
        query_type="semantic",                                # the semantic reranker re-scores the fused list
        semantic_configuration_name="default",
        top=top_k,
    )

The real search function also reports whether the reranker ran and falls back to plain hybrid search when it is unavailable; that is left out here. The design is fast, cheap and honest about its limits: the prompt tells the model to say what the passages do not support. It never sees more than five chunks, and it cannot follow a reference from one page to another.

Design 2: GraphRAG, an ontology and a traversal

A note on the name. “GraphRAG” in this article means an agent that queries a curated knowledge graph through an ontology. It is not Microsoft’s GraphRAG, which extracts entities from text and summarizes communities for whole-corpus questions. Same family, different job.

The design has no prose at all, so the three columns isolate what structure alone contributes. The agent gets two tools, a name resolver backed by a full-text index and a read-only Cypher runner, and an ontology in its system prompt. The loop is about forty lines:

async def run_agent(ctx, system: str, question: str, tools: list[dict]) -> str:
    messages = [{"role": "system", "content": system}, {"role": "user", "content": question}]

    for _ in range(ctx.settings.agent_max_steps):          # 8
        result = await ctx.llm.chat(ctx.model, messages, tools=tools)
        messages.append(result.message)
        if not result.tool_calls:                         # nothing more to fetch: this is the answer
            return result.content

        outcomes = await asyncio.gather(*(                # independent tool calls run in parallel
            execute(ctx, call.function.name, parse_arguments(call.function.arguments))
            for call in result.tool_calls
        ))
        for call, payload in zip(result.tool_calls, outcomes):
            messages.append({"role": "tool", "tool_call_id": call.id,
                             "content": json.dumps(payload, default=str)})

The guard around the queries the model writes is the part I would not ship without. The regex is a courtesy; the database is the real guard:

async def run_read(self, cypher: str, params: dict | None = None) -> dict:
    if _WRITE_CLAUSE.search(cypher):
        raise CypherRejected("Only read-only Cypher is allowed.")
    async with self._driver.session(database=self._database,
                                    default_access_mode=neo4j.READ_ACCESS) as session:
        result = await session.run(neo4j.Query(cypher, timeout=20), params or {})
        columns = list(result.keys())
        records = await result.fetch(self._row_limit + 1)   # 200, plus one to detect truncation
    rows = [[_plain(v) for v in record.values()] for record in records[: self._row_limit]]
    return {"columns": columns, "rows": rows, "truncated": len(records) > self._row_limit}

The ontology is defined once in code and used three ways: to label nodes at load time, to render the semantic layer the agent reads, and to draw the ontology view in the demo UI. A business definition carries both its meaning and its query pattern:

OntologyRule(
    "PciScopedService",
    "A Service is in PCI scope if (a) it DEPENDS_ON a CardholderDataStore directly, "
    "or (b) it DEPENDS_ON, directly, a service that meets (a). Services further upstream are out of scope.",
    "MATCH (d:DataStore)-[:STORES]->(:DataCategory)-[:SUBCATEGORY_OF*0..]->(:DataCategory {name: 'CardholderData'}) "
    "MATCH (a:Service)-[:DEPENDS_ON]->(d) "
    "OPTIONAL MATCH (b:Service)-[:DEPENDS_ON]->(a) "
    "WITH collect(DISTINCT a) + collect(DISTINCT b) AS scoped UNWIND scoped AS s RETURN DISTINCT s.name",
    aliases=("PCI scope", "in scope for PCI", "card data compliance scope"),
)

Rendered into the prompt, the agent sees the classes with their properties, each relationship with what it means and whether it is transitive, the data taxonomy, and definitions like the one above with the words people actually use for them. About 1,900 tokens, roughly 2,300 for the whole system prompt once the tool definitions are added, which prompt caching turns into a rounding error after the first call.

Design 3: the agentic hybrid

The same loop, with four tools: search the documents, read a whole document by id, resolve names in the graph, run Cypher. The prompt tells the agent which store fits which need:

1. The knowledge graph (Neo4j) holds structure: what exists, how it is connected, who owns it,
   how it is classified, what happened and when. It is exact, complete and countable.
2. The document library (Azure AI Search) holds prose: procedures, reasoning, incident narratives,
   recommendations, action items, numbers inside text. It is found by meaning and returns the
   best few passages, never everything.

Choose per question:
- Relationships, impact, ownership, counts, complete lists, "which ones have no ...",
  classification: use the graph.
- "How do I", "why did we", "what happened", "what does the runbook say": use the documents.
- A vague reference ("that outage where ..."): search the documents to identify it,
  then use its id in the graph.
- Both: get the exact set from the graph first, then read the documents for exactly those items.

The two sources share identifiers. Every search hit lists `entity_ids`, which are node names
in the graph. Graph nodes carry `doc_id`, which read_document accepts.

That last paragraph is the piece that makes the design work. Every chunk carries the entity ids it mentions; every node carries its document id. The agent can find an incident by its description in the text, step onto the graph with its id, follow the dependencies, and step back to the page for the recommendations.

One detail worth copying: every tool takes a reasoning argument, one sentence on why the model is making the call. It costs a few tokens and buys a readable trace on every model, reasoning or not.

The experiment

Twelve questions, each labelled with its shape and with a ground truth computed from the data. Three pipelines. Same model, same corpus, same chunking, same embeddings, same judge. Only the retrieval design differs.

For every run I recorded the answer, the judge’s verdict (correct, partial, incorrect), wall-clock time, model and tool calls, tokens, and cost at Azure list prices as of 30 September 2026. The judge, gpt-4.1 with the ground truth and a list of key facts, is strict about completeness: an answer that says “exactly these five” and lists four is partial, not correct.

Two caveats before the numbers. Twelve questions is a small set, chosen to exhibit each shape, so the per-shape results are the point and the overall averages are not a benchmark. And an LLM judge is a judge, not an oracle; I read every verdict. I also did not test a plain relational baseline; for the counting and absence questions a well-modeled table would likely match the graph, and the decision table at the end says so. The graph earns its place on chains and definitions.

What happened

Classic RAG and GraphRAG are mirror images. RAG is perfect on pages and useless on definitions and counts. GraphRAG is perfect on chains, sets, absences and definitions, and useless on pages, because it holds no prose: asked for the rollback procedure, it answered that the runbook exists and the steps are not in the graph, which is honest and unhelpful. The hybrid was correct on all twelve.

Three worked examples, with the actual outputs.

An absence. “Which Tier 1 services have no runbook?” Classic RAG retrieved five runbook chunks and concluded:

All listed Tier 1 services have a runbook specified: ledger-service, payouts-service, payments-api, fraud-scoring, card-gateway. There are zero Tier 1 services without a runbook mentioned in the provided passages.

Which is true of the passages and false of the company. The graph agent wrote one query and got the three services that have none:

MATCH (s:Service {tier: 1})
WHERE NOT (s)<-[:COVERS]-(:Runbook)
RETURN s.name

A definition. “Which services are in PCI scope?” Classic RAG found the policy page, cited the rule, and then applied it only to the five passages it had, producing five names: two of them wrong, two of the real ones missing, and a claim that the payments database is a cardholder data store, which it is not. The graph agent used the definition from the ontology and returned the five correct services in 2.9 seconds.

Then I switched the ontology off. Same graph, same model, same question, but the agent received only what a database can report about itself: labels, properties, relationship types, no meaning. It searched for a category called “PCI”:

MATCH (s:Service)-[:DEPENDS_ON]->(:DataStore)-[:STORES]->(d:DataCategory {label: 'PCI'})
RETURN s.name AS service_name

found nothing, and answered: “No services are currently marked as in PCI scope in the knowledge graph.” Every fact needed for the answer was in that database. Nobody had told the model that a card number is cardholder data, or what scope means.

All of the above. “The outage where customers got charged twice: which component was the root cause, which services depend on it, and what did the postmortem recommend?” Classic RAG found the right postmortem by meaning, even though the page says “billed two times”, and got the root cause and the three action items. For the dependants it listed what happened to be in its passages: two of the eight, plus the failed component itself. Partial. The graph agent, which can match words but not meaning, picked the wrong incident, a reconciliation incident from the following year whose root cause was the acquiring bank, and then traversed from the acquirer instead of the gateway. Incorrect, and confident. The hybrid searched the documents first, identified INC-2025–007, took the component name into the graph:

MATCH (s:Service)-[:DEPENDS_ON*1..]->(c:Component {name: 'card-gateway'})
RETURN DISTINCT s.name

got all eight dependents, and read the action items from the page it had already found. Four model calls, three tool calls, 7.2 seconds, 2.7 cents. Correct on all three parts.

A bigger model does not fix the wrong shape

This is the result I did not expect and now consider the most important one. Classic RAG scored exactly 46% on gpt-4o-mini, gpt-4o, gpt-4.1, gpt-5-mini, gpt-5 and o4-mini. On five of the six the verdicts were identical, the same four correct and the same three partial; o4-mini turned one correct and one incorrect into two partials and landed on the same score. When retrieval is the ceiling, a bigger model is a more eloquent way of being wrong.

The two agent designs do depend on the model, because the model has to use tools well. GraphRAG ranged from 46% on the gpt-4o family to 62% on gpt-5 and gpt-5-mini. The hybrid went from 71% on gpt-4o-mini to 100% on gpt-4.1, gpt-5 and gpt-5-mini. Interestingly, gpt-5-mini reached 100% on the hybrid at $0.0021 per question, half the price of gpt-4.1 running Classic RAG, at the cost of latency: a median of 6.0 seconds against 3.9.

What the ontology is worth, and what everything costs

Three things to read from this.

The ontology is the difference between a graph that answers and a graph that does not. With the meaning removed, GraphRAG fell from 58% to 21%, and the hybrid from 100% to 62%. The hybrid survived better only because the page questions still worked. Everything that needed a definition, a classification or a transitive relationship failed. The facts were unchanged.

Per-question cost is not where the money goes. The hybrid costs about three times Classic RAG, and three times half a cent is still almost nothing. The agents use four to six times the tokens, mostly the ontology sent on every call, but prompt caching bills the repeated part at a fraction of the price. The literature agrees on the shape of the trade: agentic retrieval needs about 3.3 times the input tokens and 1.5 times the time of single-shot RAG (Ferrazzi et al., ACL 2026), and buys a different class of answer for it.

Indexing is where the money goes, if you extract. Embedding this corpus cost $0.0017 and took 1.4 seconds. Loading the graph from the source of truth took about the same. Building a graph by having a model read your documents is the expensive path: Microsoft’s own worked example needs about two million LLM tokens to index a 55,000-token book, between $0.48 and $28 depending on the model, against $0.0056 for embeddings, and independent measurements put graph construction at 40 to 60 times the time of vector indexing. So when someone asks whether GraphRAG is too expensive, the honest answer is: it is, if you extract; it is not, if you already have the structure. And most companies do. It is in the service catalogue, the CMDB, the HR system, the classification policy. The cost of a graph is mostly the cost of admitting you already own one.

The line item nobody budgets is a person. An ontology needs an owner, because meaning changes when the business does.

Route first, retrieve second

If each design wins its own shape, the production answer is not to pick one. It is to read the shape of each question before retrieving anything.

The router is a small model with strict structured output. It cannot answer outside the schema, so it cannot invent a fourth route or leave out the confidence:

SCHEMA = {
    "type": "object",
    "properties": {
        "shape": {"type": "string", "enum": ["page", "chain", "set", "absence", "definition", "mixed"]},
        "route": {"type": "string", "enum": ["rag", "graph", "hybrid"]},
        "confidence": {"type": "number", "description": "0 to 1, how sure you are of the route"},
        "reasoning": {"type": "string", "description": "one sentence, at most 30 words"},
    },
    "required": ["shape", "route", "confidence", "reasoning"],
    "additionalProperties": False,
}

result = await llm.chat(model, [{"role": "system", "content": ROUTER_PROMPT},
                                {"role": "user", "content": f"Question: {question}"}],
                        response_schema=SCHEMA)
decision = json.loads(result.content)
fallback = decision["confidence"] < 0.6 and decision["route"] != "hybrid"
route = "hybrid" if fallback else decision["route"]       # a wrong cheap route costs more than an expensive right one

The prompt describes the six shapes, the three designs and a few rules of thumb, with generic examples only; none of the twelve test questions is in it. Measured against the design that scored best on each question:

The two misses err in opposite directions, which is the useful thing to know. gpt-4o-mini sent the restricted-data question to the hybrid, which still answered correctly, at twice the cost of the graph. gpt-5-mini sent the SEV1-incidents question to the graph, which answered the structured half and not the action items. Over-routing costs money; under-routing costs correctness. The confidence threshold is the dial between them.

On the PCI question, routed with gpt-4o-mini: the router cost $0.0002 and took 0.7 seconds, GraphRAG then cost $0.0071, and the two columns that did not run cost nothing. Running all three would have cost $0.019. The saving on that question was 61%.

What users can expect, by design

This is the part I wish someone had written for me before my first production system, because the user’s experience is set by the design long before the model is chosen.

How to decide

Two questions, before any technology.

Most companies are a mix of all four, which is why the mature architecture is three designs behind a router, not a bet on one.

In practice:

1. Collect the questions people actually ask. From tickets, chat logs, on-call channels. Not the ones you imagine.
2. Sort them by shape. Page, chain, set, absence, definition, mixed. In this experiment only three of the twelve were pages, and real question logs are not kinder.
3. See what structure you already own. Dependencies in the service catalogue, ownership in HR, categories in the classification policy. Load it; do not extract it.
4. Write the ontology for the definitions your business uses, and give it an owner.
5. Pick the cheapest design per shape, put a router in front, and measure with ground truth. Twelve questions with computed answers told me more than any benchmark.

Closing remarks

Three things I would carry out of this work.

Similar is not the same. Retrieval by resemblance finds what a question sounds like. It cannot find what the question is attached to, count what it covers, or notice what is missing. Better retrieval finds better passages; it does not change what a passage is.

Structure answers what text cannot. Route to it; do not default to it. The graph won every structural shape and lost every page. The hybrid won everything at three times the price. The economics of the whole thing are decided by one cheap decision made before retrieval starts.

Models will be swapped every year. Your ontology will still describe your business in five. Six models produced the same 46% on Classic RAG. One ontology turned a 21% graph into a 58% graph and a 62% hybrid into a 100% one. The durable asset is not the model and not the vector index. It is the explicit statement of what your data means.

So the rule of thumb, if there is only room for one: always start with the questions. Their shape tells you everything else.

Naissus Pay, its people, systems and incidents are invented for this work. Prices are Azure OpenAI list prices as of 30 September 2026.

References

• Lewis et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS. https://arxiv.org/abs/2005.11401
• Gao et al. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. https://arxiv.org/abs/2312.10997
• Anthropic (2024). Introducing Contextual Retrieval. https://www.anthropic.com/news/contextual-retrieval
• Microsoft (2023). Azure AI Search: Outperforming vector search with hybrid retrieval and reranking. https://techcommunity.microsoft.com/blog/azure-ai-foundry-blog/azure-ai-search-outperforming-vector-search-with-hybrid-retrieval-and-reranking/3929167
• Tang and Yang (2024). MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries. https://arxiv.org/abs/2401.15391
• Edge et al. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. https://arxiv.org/abs/2404.16130
• Microsoft (2024). GraphRAG Costs Explained: What You Need to Know. https://techcommunity.microsoft.com/blog/azure-ai-foundry-blog/graphrag-costs-explained-what-you-need-to-know/4207978
• Han et al. (2025). RAG vs. GraphRAG: A Systematic Evaluation and Key Insights. https://arxiv.org/abs/2502.11371
• Xiang et al. (2025). When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation. https://arxiv.org/abs/2506.05690
• Sequeda, Allemang and Jacob (2023). A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model’s Accuracy. https://arxiv.org/abs/2311.07509
• Ferrazzi et al. (2026). Is Agentic RAG worth it? An experimental comparison of RAG approaches. ACL Industry Track. https://arxiv.org/abs/2601.07711
• Hogan et al. (2021). Knowledge Graphs. ACM Computing Surveys. https://arxiv.org/abs/2003.02320
• Gruber (1993). A Translation Approach to Portable Ontology Specifications. Knowledge Acquisition 5(2).

Your Data Has a Shape. Does Your AI Know It? was originally published in AI Advances on Medium, where people are continuing the conversation by highlighting and responding to this story.