How AI Actually Works: The Big Picture Before the Maths
AI is not magic. Underneath ChatGPT, image generators and fraud detectors is one simple idea: learn a pattern from examples, then use it on new inputs.
I’m trying to understand AI at a foundational level: not just how to use the tools, but what is actually happening inside them. Why does AI need linear algebra? Where does probability come in? What does a model “learn”, exactly?
This blog is where I write it down as I go, in two parallel tracks. Foundations covers the maths and core ideas inside every model. What’s new covers recent AI tools and innovations, explained through those same foundations. This first post maps both.
The one idea: learning a function from examples
In normal programming, you write the rules. To flag a spam email, you might code: if the subject contains “FREE MONEY”, mark it spam. The computer follows your rules exactly.
Machine learning flips this around. You don’t write the rules. You give the computer thousands of emails already labelled “spam” or “not spam”, and it works out the rules itself.
Mathematically, every AI system is trying to find a function:
y = f(x)Here x is the input (an email, a photo, a sentence), y is the output (spam or not, cat or dog, the next word), and f is the unknown rule connecting them. We never write f by hand. We pick a flexible family of functions and let the data choose the best one from that family.
That’s it. Every model you’ve heard of, from linear regression to GPT, is a different way of choosing that family and searching inside it.
The four ingredients of every AI system
Strip away the hype and every model, small or huge, is built from the same four parts.
1. Data. Examples of inputs, usually with the correct answer attached. Photos labelled “cat”, house details with their sale prices, sentences with the word that came next. The model can only learn what’s in the data, so its quality and bias become the model’s quality and bias.
2. Model. The family of functions we search in. A straight line is a model. A decision tree is a model. A neural network is a very flexible model with millions or billions of adjustable numbers called parameters (or weights).
3. Loss function. A score for how wrong the model currently is. Predict a house at ₹80 lakh when it sold for ₹1 crore, and the loss measures that ₹20 lakh gap. Lower loss means better predictions.
4. Optimisation. The method for adjusting the parameters to reduce the loss. The most common one is gradient descent: work out which direction makes the loss go down, take a small step that way, and repeat.
Training is these four parts running in a loop. The model makes predictions on the data, the loss scores them, the optimiser nudges the parameters, and the cycle repeats thousands or millions of times until the loss stops improving.
A tiny example: predicting house prices
Say we want to predict a flat’s price from its size. We collect a few past sales:
Get Abhijna’s stories in your inbox
Join Medium for free to get updates from this writer.
Size (sq ft) | Price (₹ lakh)
— — — — — — — — — — — — —
600 | 48
900 | 70
1,200 | 95
1,500 | 118
Model. We guess the relationship is roughly a straight line:
\text{price} = w \times \text{size} + bThe two numbers w (the slope) and b (the starting value) are the parameters. Learning means finding good values for them.
Loss. For each flat, we compare the predicted price with the real one and square the difference, so big misses count more. Averaging that over all flats gives the mean squared error:
L(w, b) = \frac{1}{n} \sum_{i=1}^{n} \left( y_i - (w x_i + b) \right)^2Optimisation. We start with random w and b, which give terrible predictions and a high loss. Calculus tells us how the loss changes when we nudge w or b slightly. We nudge both in the direction that lowers the loss, then repeat. After enough steps we settle near w ≈ 0.078 and b ≈ 0.5, which predicts a 1,000 sq ft flat at about ₹79 lakh.
That is genuinely all training is. A neural network does exactly this, just with millions of parameters instead of two.
Where neural networks and LLMs fit in
A straight line can’t capture most real patterns. Whether a photo shows a cat depends on thousands of pixels interacting in complicated ways. We need a far more flexible family of functions.
A neural network is that family. It stacks many simple steps: multiply the inputs by weights, add them up, pass the result through a small non-linear function, and feed it to the next layer. Each step is almost as simple as our house-price line. Stack enough of them and the network can approximate almost any pattern.
The training recipe doesn’t change. Data, model, loss, optimisation. The only new piece is backpropagation, an efficient way to use the chain rule from calculus to find how every one of the millions of weights affects the loss.
Large language models like ChatGPT and Claude are neural networks trained on one deceptively simple task: given some text, predict the next word. The input x is the text so far, the output y is the next word, and the loss measures how surprised the model was by the real next word. Do that over a huge slice of the internet with billions of parameters, and grammar, facts and reasoning patterns emerge as side effects of getting better at that one prediction.
Where the maths shows up
Each of the four ingredients leans on a branch of maths. That’s why AI courses start with linear algebra and statistics, and it’s what the Foundations track will unpack.
Data leans on statistics and probability: how a sample represents the real world, and how bias sneaks in. The model is linear algebra, since a neural network is mostly vectors and matrices multiplied together. The loss comes from probability and information theory, which is where softmax and cross-entropy come from. Optimisation is calculus: gradient descent and backpropagation are derivatives and the chain rule at work.
The takeaway
Every AI system, from a two-parameter line to a chatbot with billions of weights, does the same thing: learn a function from examples by repeatedly lowering a loss. Data, model, loss, optimisation. Keep those four words in mind and any AI headline becomes easier to decode.
Stay curious, and see you next epoch.
