Large Language Models (LLMs)
How they work · How they learn · How they’re used
1. What an LLM is — and how it learned
“An LLM is a computer program that, given any piece of text, predicts the next word.”
That single sentence is the entire trick. Everything else an LLM appears to do — answering questions, summarising documents, telling stories, writing code, holding a conversation — is the same trick repeated many times. The model predicts one next word, attaches it to the end, and predicts another. After hundreds of repetitions you have a paragraph, an essay, or a story.
Get VoiceAIEngineer’s stories in your inbox
Join Medium for free to get updates from this writer.
The name unpacks neatly. “Large” — billions of internal numbers, called parameters, set by training. “Language” — it works with text. “Model” — a neural network: layers of mathematics that take numbers in and produce numbers out.
How it learned: a guessing game played at scale
Nobody hand-codes the billions of parameters. They are learned by playing one game, trillions of times hide a word, ask the model to guess it, compare the guess to the real word, and nudge the parameters in the right direction. As a side effect — the surprising part — the model also picks up grammar, world knowledge, reasoning patterns, even the ability to follow instructions.
2. Inside the LLM — the algorithm
When you give an LLM your text and ask it to continue, the same six steps run for every word it produces. The numbered chain below is the entire procedure — every chatbot, every voice assistant, every code generator runs on it.
Why it can answer almost anything
Almost any useful task can be re-framed as next-word prediction. Answering a question is predicting the words that would naturally follow it. Translating is predicting what would follow “In French, …”. Writing code is predicting what would follow a function signature. The model never “understands” in a human sense — it just got very good at the predict-what-comes-next game, on enormous amounts of human- written text.
3. Where the data comes from
An LLM in a real product gets its information from three different places. A well-designed assistant picks the right source for each request.
4. Using an LLM in a voice assistant
A voice assistant is more than “just an LLM.” It is a pipeline of small specialised components, with the LLM acting as the brain in the middle. Audio is captured, cleaned, converted into text, enriched with retrieval
or tools, sent to the LLM, and the LLM’s reply is streamed back through TTS so the user hears the answer while the model is still generating.
Where LLMs differ from the older NLU approach
Pre-LLM assistants used Natural Language Understanding (NLU): small models trained on a fixed list of intents and slots. Fast and deterministic — but rigid. LLMs are a generalist alternative — they can do the same intent classification and gracefully handle anything else, at the cost of higher latency and probabilistic output. Most production assistants use both: a small NLU acts as a router for known fast intents (“set AC to 22”, “call emergency”); the LLM handles open questions, multi-step requests, and
conversations.
5. Strengths, limits, and the road ahead
What LLMs are very good at
- Generating fluent, coherent, natural-sounding text in dozens of languages.
- Generalising — handling phrasings, formats, and tasks they were never explicitly taught.
- Following multi-step instructions and producing structured outputs (JSON, tool calls).
- Summarising, translating, classifying, and rewriting — all without task-specific training.
What they still struggle with
- Hallucination — confidently producing wrong information. Mitigate with RAG and tools.
- Freshness — they don’t know anything past their training cutoff. Mitigate with web search or live APIs.
- Private and personal data — they can’t access your files or calendar. Mitigate with tool calls.
- Cost and latency — bigger and slower than classical models. Mitigate with quantization and routing.
- Determinism — same input can give different output. Mitigate with temperature 0 and structured outputs.
Where the field is heading
- Smaller, faster models that run on-device — Gemma, Phi, Llama 3.2 already do this on phones.
- Native multimodal — voice in, voice out, vision in. Sub-second full-duplex conversations.
- Agents — LLMs that don’t just answer, but plan, act, and use tools across many steps.
- Tighter coupling with retrieval and tools — fewer hallucinations, more grounded answers.
The whole guide in five lines
1 - An LLM is a giant autocomplete with billions of internal numbers tuned by training.
2 - To answer you, it runs the same six steps — tokenize, embed, transformer, score, pick, repeat once per word.
3 - All its built-in knowledge lives in its trained numbers, not in a live lookup.
4 - For fresh facts, private data, or real actions, pair the LLM with RAG and tool calls.
5 - In voice assistants the LLM is the brain in a longer pipeline of small specialised components.
