Building your AI agent without an LLM
I am a technology academic at the University of Surrey and I’ve been researching a lot about AI agents over the last couple of years. And I found out that, as at the time I’m writing this article (November 2026), only tech companies are really into AI agents. Everybody else thinks AI is ChatGPT.
This is disappointing to a tech enthusiast, but then I digged deeper to find out why this is going on. I interviewed about 30 business leaders from small, medium and large businesses, from tech and non tech backgrounds, and I found out some clear common themes:
- AI agents are not reliable to behave. They aren’t like code: code you test, you know if you do the same request 1000 times, you’ll get the same behaviour 1000 times. AI agents, if you send a complex flow for them to do 1000 times, you’ll get 1000 different answers;
- A lot of people simply aren’t into sending their business information to OpenAI, Google or Anthropic. Or their personal data. They want their things to stay inside their fence, and sometimes they are legally required to never share that data, so that really limits what they can do via agentic AI;
- Every time an LLM thinks, you pay a few pennies (or even a few dollars if the request is complex). Over time, it adds up. You simply can’t automate a takeaway using Claude Sonnet and MCP — it simply doesn’t pay.
- Setting all of that up is way too complicated. You need to understand guardrails, execution graphs, prompt engineering, context compression, etc etc! It’s just crazy! You need to be a master geek to get things right!
So I got intrigued about this problem and decided to go on a journey to solve it. I am not as big as big tech and I have practically no funding, but I have ideas. And I am working on an idea that will help all these people who can’t get the benefit of automation to enhance their productivity in a safe manner.
Get Fabio Dias’s stories in your inbox
Join Medium for free to get updates from this writer.
I researched NLP (Natural Language Processing) for decades, and started building a Small Language Model based purely on rules-based deterministic reasoning. Think about it as a Siri 2.0 — something intelligent enough to plan, reason, understand context, take complex prompts and break them into a series of hierarchical tasks, but not as clever as an LLM because is non-generative.
Now make your Siri 2.0 understand MCP and also understand what it cannot do and connect to an LLM, only if approved, to do the things it cannot do.
In my tests, I was able to get something fully deterministic that runs on a CPU and can do some quite complicated things like a systematic literature review, and some simple things like order a coffee or call an Uber. And it can schedule tasks, monitor events, search on the web and summarise information.
But with several big wins:
- It’s deterministic. I can debug it. If I run the same sequence of prompts 1000 times, I get 1000 times the same workflow.
- It keeps my data, my prompts, everything I have inside my own machine.
- It cost me zero, runs locally on my CPU, and gives me replies in 22 milliseconds on average, way faster than an LLM
I still haven’t won the part of making it simple to use, but that’s what I’m targeting next. And also I still have a lot of research on the language side to make it understand context better, reason better, etc.
I am starting this thread to document my journey, and also to let everyone know that I plan to make this public, and free. I already have a Python SDK, search stalwart-agents and you’ll find it. I’m building a web UI and I’m hoping to help researchers and takeaways alike.
Anyone who wants to give feedback, test, or just ask questions, please do reach out to me or visit my website!
