OKF + RAG: The Ultimate AI Agent Architecture
Every AI agent has a memory problem. Ask it a well-known fact, and it answers brilliantly. Ask it about your company’s specific churn rate formula, your internal API schema, or your production runbook for a database failover, and it confidently fabricates an answer that sounds perfect but is completely wrong.
This is the hallucination problem, and it is the single biggest blocker preventing enterprises from trusting AI agents with real work.
For the past two years, the industry’s answer has been RAG (Retrieval-Augmented Generation): connect the AI to a vector database full of your company’s documents, let it search for relevant context, and paste it into the prompt.
RAG works well for large, unstructured document corpuses, but it has a critical flaw: it is probabilistic. It finds documents that are semantically similar to your question, not documents that are factually authoritative. For mission-critical data — like the exact formula for calculating revenue, or the precise API contract for a payment endpoint — “similar” is not good enough.
In June 2026, Google Cloud introduced the Open Knowledge Format (OKF), a vendor-neutral, open specification for storing curated, authoritative knowledge as simple Markdown files. By combining OKF’s deterministic precision with RAG’s probabilistic reach, we arrive at what many are calling the Ultimate AI Agent Architecture.
The Two Memory Systems
Think of a human expert joining your company on Day 1.
They are handed two things:
- A Company Wiki (OKF): A curated, well-organized internal wiki containing canonical definitions, API docs, metric formulas, and runbooks. Everything in it has been reviewed, approved, and version-controlled. When you need the official answer, you go here.
- A Search Engine for Everything Else (RAG): Access to thousands of slack messages, email threads, support tickets, and meeting transcripts. When you need to find what people said about a topic, or discover information buried in unstructured archives, you search here.
Neither system alone is sufficient. Together, they cover the full spectrum of organizational knowledge.
What is OKF? (Open Knowledge Format)
OKF is radically simple by design. An OKF “knowledge bundle” is just a directory of Markdown files stored in Git. Each file represents a single concept — a database table, a metric, an API endpoint, a runbook — and includes a block of YAML frontmatter at the top for machine-readable metadata.
Get Ravish Khullar’s stories in your inbox
Join Medium for free to get updates from this writer.
Here is an example of an OKF file for a business metric:
---
type: metric
title: "Monthly Churn Rate"
description: "Official formula for calculating monthly customer churn."
owner: "data-engineering"
tags: [revenue, kpi, board-report]
timestamp: 2026-06-20T10:00:00Z
---
# Monthly Churn Rate
The official churn rate formula used in all board reports and investor decks:
Churn Rate = (Customers Lost During Month / Customers at Start of Month) × 100
### Rules
- **Do NOT** use trial accounts in the denominator.
- **Do NOT** count plan downgrades as churn.
- Source of truth: `analytics.monthly_churn_summary` table.
### Related
- [Monthly Active Users](mau.md)
- [Revenue Dashboard](revenue_dashboard.md)Why This Matters
- No Vector Math. No Hallucination. When the agent needs the churn formula, it reads this file directly. There is no embedding, no cosine similarity, no “top-k nearest neighbors.” It gets the exact, approved answer every time.
- Git-Ops Native. Because OKF bundles live in Git, every change is tracked through pull requests, code reviews, and commit history. You have a full audit trail of who changed the churn formula, when, and why.
- Zero Infrastructure. No vector database. No embedding pipeline. No chunking strategy. No re-indexing jobs. Just files in a folder.
What is RAG? (Retrieval-Augmented Generation)
RAG is the industry-standard pattern for connecting an LLM to large, unstructured data. It works by:
- Ingesting thousands of documents (PDFs, emails, Confluence pages, Slack messages).
- Chunking them into smaller segments.
- Embedding each chunk into a high-dimensional vector using an embedding model.
- Storing the vectors in a vector database (like Pinecone, Weaviate, or pgvector).
- Searching at query time: the user’s question is embedded into the same vector space, and the database returns the most semantically similar chunks.
- Augmenting the LLM’s prompt with those chunks so it can generate an informed answer.
RAG excels at reach — it can surface relevant information from thousands of unstructured documents that no human could manually organize.
But RAG struggles with precision. Because it relies on semantic similarity (vector math), it can return documents that are related to your question but not authoritative. If three different Slack threads define churn rate slightly differently, RAG might retrieve the wrong one.
The Hybrid Architecture: OKF + RAG
The ultimate agent architecture uses an Intelligent Router that classifies incoming queries and directs them to the appropriate memory system.
How the Router Decides
Implementation Example
Here is a simplified Python implementation of the hybrid router:
from openai import OpenAI
import os
import glob
import yaml
client = OpenAI()
# 1. Load the OKF knowledge bundle from the local directory
def load_okf_bundle(bundle_path: str) -> dict:
"""Reads all Markdown files in the OKF directory into a searchable dict."""
knowledge = {}
for filepath in glob.glob(f"{bundle_path}/**/*.md", recursive=True):
with open(filepath) as f:
content = f.read()
# Extract the title from YAML frontmatter
if content.startswith("---"):
_, frontmatter, body = content.split("---", 2)
meta = yaml.safe_load(frontmatter)
title = meta.get("title", os.path.basename(filepath))
knowledge[title.lower()] = body.strip()
return knowledge
# 2. Search OKF (deterministic, keyword-based)
def search_okf(query: str, okf_knowledge: dict) -> str | None:
"""Simple keyword match against OKF titles."""
for title, content in okf_knowledge.items():
if title in query.lower():
return content
return None
# 3. Search RAG (probabilistic, vector-based)
def search_rag(query: str) -> str:
"""Placeholder for your vector DB search (Pinecone, Weaviate, etc.)."""
# results = vector_db.similarity_search(query, top_k=5)
return "RAG context: [retrieved chunks would appear here]"
# 4. The Intelligent Router
def answer_query(query: str, okf_bundle_path: str) -> str:
okf_knowledge = load_okf_bundle(okf_bundle_path)
# Try OKF first (deterministic path)
okf_result = search_okf(query, okf_knowledge)
if okf_result:
context = f"[SOURCE: Official Knowledge Base (OKF)]\n{okf_result}"
else:
# Fall back to RAG (probabilistic path)
context = f"[SOURCE: Document Search (RAG)]\n{search_rag(query)}"
# Send to LLM with the retrieved context
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "Answer using ONLY the provided context."},
{"role": "user", "content": f"Context:\n{context}\n\nQuestion: {query}"}
]
)
return response.choices[0].message.content
Conclusion
The future of enterprise AI agents is not a choice between OKF and RAG — it is a hybrid architecture that leverages both.
OKF gives your agent a curated, hallucination-proof memory for the facts that matter most: your KPIs, your API contracts, your operational runbooks. RAG gives your agent the ability to explore vast archives of unstructured knowledge that no human could manually organize.
By combining a deterministic, Git-controlled knowledge base with a probabilistic vector search engine, you build an AI agent that is both precise when it needs to be and exploratory when it needs to discover.
