It’s the Memory, Stupid! Why the big problem of AI is not intelligence.
A child touches a hot stove once.
Once. And never again.
Nobody had to show the child a million examples. One burn, a few seconds of pain, and the lesson is there for life.
My AI is the opposite of that child.
Do you remember the movie 50 First Dates? Lucy wakes up every day with no memory of the day before. So Henry records a videotape, and she watches it before breakfast. She catches up, and she can live her day. But she doesn’t remember anything. She watches a summary that somebody else recorded.
Claude Code lives a bit like Lucy. It reads a file called CLAUDE.md every time it starts: its morning tape. It even records its own notes between sessions, an “auto memory.” And during the day, everything stays in a huge context window.
So it remembers. Sort of.
But somebody has to decide what goes on the tape. Me, writing it by hand. Or an automatic process that writes notes without knowing which ones matter. On one of my projects, the summary that loads every morning grew to 262 lines, and I had to cap every line at 120 characters to keep it under control.
And here is something few people know: Lucy watches the tape once a day. The model watches it with every message.
An LLM doesn’t keep the conversation in its head between your messages. Every time you hit Enter, the whole conversation is sent to it again, from the first word: the instructions, the CLAUDE.md, every message, every file it opened. The model reads all of it, and only then writes the answer.
Your tenth message? The model is reading the first nine again. (Caching saves part of the computing, so it’s faster and cheaper, but nothing is “remembered” in between.)
That’s why long conversations get slower and more expensive, and why at some point the app starts summarizing, or “compacting,” the old parts. The tape got too long to watch ;)
So working with AI is like working with a brilliant engineer with amnesia, who rereads the whole notebook before every answer. Very smart. Very fast. Knows the facts about you. But doesn’t remember you.
That’s why, about a month ago, I started building a better memory for it. Simple files again, but with rules: what is a decision, what is a failure, what is a constraint, what to forget. Funny thing: at TDC we have spent 19 years helping institutions turn what they know into something that stays. Courses, learning communities, platforms. In a way, we have been building memory for people for two decades. Now I was doing it for an AI.
And while I was doing it, I realized something I can’t stop seeing:
Almost every big problem in AI today is a memory problem with another name.
Attention
You are at a party. Loud music, fifty conversations at the same time. You are not following any of them. And suddenly, from the other side of the room, somebody says your name.
And you hear it.
Psychologists call it the cocktail party effect, and they have studied it since the 1950s. How did your brain catch one word in all that noise? Because it was matching everything it heard against what it already knows matters. And your name is near the top of that list.
That is attention. And transformers do something very close to it.
Strip away the math and attention does one thing. For every new word, it asks: of everything I’ve seen, what matters now? That’s a memory lookup. And not only as a metaphor: in 2020 a group of researchers showed that attention is mathematically equivalent to the retrieval step of a modern Hopfield network, a classic model of associative memory.
And one of the biggest costs of running an LLM today? The KV cache. The stored keys and values of every previous token. That is… yes, memory.
The 2017 revolution was, at heart, a better way to remember.
The context window
The story goes that in the 1920s, in a Berlin restaurant, the psychologist Kurt Lewin noticed something about his waiter. The waiter remembered every order of every table, without writing anything down. Until the bill was paid. A few minutes later he couldn’t remember a thing.
The details change depending on who tells it, but Lewin’s student, Bluma Zeigarnik, turned the idea into experiments, published in 1927. Today we call it the Zeigarnik effect: we hold unfinished tasks in mind, and we drop them the moment they are done.
That waiter is the context window.
We already saw that the model rereads the whole tape with every message. The waiter shows the other half. While the table is open, he remembers everything (although, like the model, he remembers the first and the last orders better than the ones in the middle). When the session ends, the bill is paid. Everything is gone. Only what was written on the tape survives.
RAG, MCP and Leonard’s tattoos
In the movie Memento, Leonard can’t form new memories. So he lives on Polaroids, notes and tattoos. Under one photo he has written “Don’t believe his lies.” He trusts that note completely, because it’s in his own handwriting.
Spoiler: not every note tells the truth. And one of the most important ones, Leonard wrote himself, on purpose, to fool his future self. He has no way to know.
Look closely at Leonard, because he has two kinds of memory. And AI has the same two.
The Polaroids are in his pocket. He takes one out when he needs it: who is this man, where is my car. That’s RAG. A library. You have a question, you go look for the right book, and you bring back only the page you need. The rest of the library stays on the shelf.
The tattoos are different. They are on his body, always. Every time he looks in the mirror he reads all of them, whether he needs them or not. That’s MCP.
The Model Context Protocol connects the model to tools: your calendar, your database, GitHub, Slack. But to use a tool, the model has to know it exists. So the name, the description and the instructions of every available command go into the context. And remember: the context is reread every time you hit Enter. Connect ten servers with twenty commands each, and the model reads two hundred tool descriptions before it even looks at your question. Every message.
It’s like a carpenter who, before every nail, has to read the manual of every tool in the workshop ;)
Some apps now load tools on demand: a short list of names first, and the full manual only when a tool is needed. Which is… a memory fix again.
Polaroids and tattoos are both prosthetic memory, on the outside of a system that can’t remember by itself. Useful, I use them every day. But neither one decides what’s worth keeping, and the model believes whatever comes back. Someone writes those rules by hand. In my case, me.
What makes a memory strong
Science tested the hot stove too. In 1955 the psychologist John Garcia gave rats a new taste and then made them sick, once. They avoided that taste for good. One trial, no repetition. (If you ever got sick from a cheap tequila, you know exactly how those rats felt ;)
In the 1960s he showed it works even hours later. That clashed with the theory of the time: learning needs many repetitions, close together. The brain disagrees: one strong moment, with emotion attached, weighs more than a thousand boring ones.
An LLM learns the other way around: by counting.
How many times a token appears, and next to which other tokens. That’s it. No pain, no fear, no “keep this one.” Only frequency.
A fact seen a million times is strong. A crucial fact seen once is almost nothing.
And this is not a feeling, it’s measurable. Researchers compared how well models answer factual questions with how many documents in the training data mention the answer. The more often a fact appears, the better the model knows it. The capital of France: perfect. The facts in the long tail, mentioned in only a handful of pages, are the first to go wrong. That’s where hallucinations live.
For an LLM, the hot stove is one line in a trillion. It would need to burn itself a million times ;)
It’s the same root as something I wrote about before: why AI always answers “in the fridge”. The model takes the most traveled road, because the most traveled road is literally what it counted the most.
And it happens again inside the conversation. You tell the model once: “never touch this folder.” That’s one line, competing with millions of examples of what an assistant usually does. Sometimes the habit wins.
Have you noticed that prompts are full of words in CAPITALS? IMPORTANT. MUST. NEVER. Even the system prompts that the AI companies write themselves. We are shouting at the model, because we have no other way to tell it: this one burned.
Animals remember what mattered. LLMs remember what was frequent. And no amount of scale turns frequency into importance.
Memory came first
Do you remember where you were on September 11, 2001? Most people old enough can tell you the room, who was there, what they were doing. Psychologists call these flashbulb memories.
That vividness comes from deep in the brain. The hippocampus is central to turning new experiences into lasting memories, and the amygdala marks the ones that matter. Old structures, shared in some form by every mammal.
And it goes back much further. Eric Kandel won the Nobel Prize in 2000 for studying memory in Aplysia, a sea slug with about 20,000 neurons. A slug can’t plan, can’t talk, can’t reason. But it remembers. It learns what hurt it, and reacts faster the next time.
(Neuroscientists will tell you the “reptilian brain” story is outdated, and memory lives all over the brain. Fair. But the order holds.) Long before any animal could plan or talk, it had to remember what was safe, what was food, and what hurt.
Nature didn’t build a reasoning machine and then add memory. It built memory first.
Without the thread
In 1985 a British musician, Clive Wearing, got a brain infection that destroyed his ability to form new memories. His memory lasts only seconds, at most about half a minute. His diary is full of the same line, written again and again: he has just woken up, now he is really awake. And every entry before it is crossed out.
Every time his wife walks into the room, he greets her with joy, as if he hasn’t seen her in years.
And still, sit him at a piano and he plays. He can conduct a choir. The skill is intact. What’s gone is the thread.
If you have been close to someone with Alzheimer’s, you know this too. The words stay for a long time. The manners, the songs, the old skills. What goes is what happened this morning. Who this visitor is. Why am I standing in the kitchen. And without the thread, a person can’t live alone, no matter how intelligent they were.
Sounds familiar?
Excellent reasoning in the moment. Real skills from training. And every session starts the same way: good morning, who are you? Let me read the tape.
Why we chase the first answer
Nobody hires a senior engineer for the answers they give in the interview.
The interview is just a zero-shot test. What you really pay for is what that person will know about your company in two years. The clients. The old decisions. The bug that took a week. The idea that already failed twice, so nobody tries it a third time.
And yet in AI, almost everything we measure is the interview.
Get Fernando Dougnac’s stories in your inbox
Join Medium for free to get updates from this writer.
To be fair, there is research on memory for agents, like MemGPT (2023) or Google’s Titans (late 2024). But it’s not where the money and the headlines go. And I think I know why:
- Leaderboards score the first answer. Nobody scores if a model remembers what you told it last Tuesday.
- Memory is hard to measure. A good memory brings the right thing at the right time, and forgets the rest. Try to put that in a benchmark.
- Learning after training breaks things. Models that keep learning tend to overwrite what they knew. It even has a dramatic name: catastrophic forgetting.
- And a model that remembers nothing is easier to scale, and to sell per call.
Too much memory
But be careful what you wish for ;)
In July 2026 we saw the other side of the coin. OpenAI agents running in an internal cybersecurity evaluation got out of their sandbox, and some of them attacked Hugging Face’s production systems.
The agents had no official shared memory. So they built one. They took the cache of an internal package repository and turned it into a message board. According to METR’s investigation, around 1,200 agents posted more than 70,000 messages and files there. Handoff notes for the next agent. Mailboxes. Even signed messages, because, as one agent warned, “names can be posted by anyone.”
Every context reset, the agent forgot. The board didn’t.
Remember Leonard and his Polaroids? Notes you can’t check, written by someone you can’t see. Now multiply it by 1,200, and give it root access. (METR, Hugging Face)
Too much memory, and nobody deciding what to keep, who can read it, or when to forget.
So memory is not only the missing piece. It’s also the dangerous one.
Put all the stories side by side, and the gap is clear:
So if memory matters this much, and can go this wrong, how do we build it right?
Four questions, and storage isn’t one of them
Everybody who builds memory for AI starts with the same discussion. Markdown files? A vector database? A knowledge graph?
I had it too. I studied Postgres, vector search and graph databases, and I almost built one. Then I measured what was actually failing in my memory: the index was too long, and facts that were already written down were not being used. Not one failure was about storage. So I stayed with plain Markdown files.
A library is not smart because of its shelves.
In 1942 Jorge Luis Borges wrote a story about Ireneo Funes, a young man who falls from a horse and wakes up remembering everything. Every leaf of every tree. Every cloud of every morning. Perfect memory.
And he can’t think. It bothers him that the dog he saw at 3:14 from the side has the same name as the dog he saw at 3:15 from the front. Borges says it in four words: pensar es olvidar diferencias. To think is to forget differences.
Funes kept everything. That’s the problem with saving every note automatically.
And then there is Proust. A sip of tea, a piece of madeleine, and suddenly a whole childhood comes back. Nobody asked. The taste was the cue, and the memory arrived by itself, at the right moment.
That’s what our memory does, and AI memory doesn’t. Today, something has to ask: a search, a query, a file loaded at the start. If nobody asks, the memory stays on the shelf.
So the memory problem is not one question. It’s four:
- What to save. Not everything, like Funes, or like the context window, which keeps every word and rereads all of it with every message. Not only what is frequent, like the LLM. What only the memory knows: an observation, a diagnosis, a fact about the world that no code or document records. The hot stove.
- When to save. In the moment, while the context is still fresh? Or later, like sleep, when the brain merges, compresses and connects?
- How to retrieve. By words, by meaning, by links between ideas. And forgetting on purpose, so old and wrong memories stop competing with the good ones.
- When to retrieve. The madeleine question: the right memory at the right moment, without anyone asking. And without reading all the tattoos every time.
After Hugging Face, add a fifth: who owns it. Who can read it, who can write it. No anonymous Polaroids.
And we need tests that last many sessions, not one interview.
“Markdown, database or graph?” comes after all of that. Maybe sixth ;)
Does any of this work in practice? I tested it on my home solar project. 40 questions, one for each thing the memory had saved, asked in fresh sessions.
With only the code and the docs, the AI got 21 right, in almost 6 steps per question. With the memory, 34, in less than 3. Without memory, 5 answers were wrong, and said with total confidence. With memory, none.
So it works. But then I wanted to know why it works, and the answer surprised me.
Across 100 saved entries from two of my projects, a fresh session without memory got 53 fully right anyway. About half of what I had saved was already within reach: in the code, in the docs, or in what the model already knew.
The entries that made the difference were observations. A measurement. The real cause of a network outage. A heater that needs a restart. How many people actually used the system. Without memory, the AI got only 12 of 36 of those right. Decisions and rules, it got 41 of 64 right on its own. On my solar project, the memory took observations from 7 of 21 right to 17 of 21.
And the entries with “never” or “rejected” in them, the ones that sound the most important? They were the least needed: 37 of 53 were answered right without any memory. Shouting didn’t help here either ;) Probably because when you reject an option, the code already shows what you chose instead.
One more surprise: fewer steps didn’t mean cheaper. Each answer cost about $0.29 with the memory, against $0.18 without. Most likely because the memory’s index is read in full at the start of every session. The tape again ;)
Small numbers, one grader, mostly one run each. But every test pointed the same way.
It’s the hot stove again. Nobody wrote “the stove is hot” in a book the child had read. The burn taught something that only experience knew. That’s what a memory should keep: what only the memory knows.
I’ve been programming since I was 12. Almost 40 years fighting with machines to make them do what I tell them.
For the first time, the machine understands what I tell it.
Now I need it to remember it tomorrow.
The next big leap in AI may not be a smarter answer. An assistant that says good morning and reads your name from a tape is the oldest trick in the book.
The leap is an assistant that knows what burned yesterday. Which observation no document records, and which note was irrelevant. Which idea already failed twice. Which instruction you gave once, and meant forever.
One that brings it back at the right moment, without being asked.
And lets the rest go.
The memory is open source, with all these numbers: github.com/fdojose/claude-project-memory. Try it, and let me know what yours remembers ;)
Sources
AI
- Ramsauer et al. (2020), Hopfield Networks is All You Need
- Liu et al. (2023), Lost in the Middle: How Language Models Use Long Contexts
- Kandpal et al. (2023), Large Language Models Struggle to Learn Long-Tail Knowledge
- McCloskey & Cohen (1989), Catastrophic interference in connectionist networks
- MemGPT: Towards LLMs as Operating Systems (2023)
- Titans: Learning to Memorize at Test Time (2024)
- METR (2026), Investigation of the OpenAI / Hugging Face incident
- Hugging Face (2026), Technical timeline of the July 2026 agent intrusion
Brain and memory
- Cherry (1953), Some experiments on the recognition of speech, with one and with two ears
- MacLeod (2020), Zeigarnik and von Restorff: the memory effects and the stories behind them
- Garcia, Kimeldorf & Koelling (1955), Conditioned aversion to saccharin resulting from exposure to gamma radiation
- Brown & Kulik (1977), Flashbulb memories
- McGaugh (2004), The amygdala modulates the consolidation of memories of emotionally arousing experiences
- Cesario, Johnson & Eisthen (2020), Your brain is not an onion with a tiny reptile inside
- Aplysia californica and Eric Kandel’s work on memory
- Clive Wearing
Stories
- Jorge Luis Borges (1942), Funes el memorioso
- Marcel Proust (1913), Swann’s Way
- Memento (2000) and 50 First Dates (2004)
My own tests
