The Number e and the Logarithm — Explained Intuitively

Why every loss function in machine learning seems to involve a logarithm — and where this strange function actually comes from.

Mike Schaeffer

7 min read

·

May 11, 2026

--

6

Listen

Share

Press enter or click to view image in full size

This is Part 1 of a 3-part series that builds the intuition behind machine learning’s favorite math from the ground up — no formula memorization required.

• Part 1 — The Number e and the Logarithm (you’re here): why the logarithm is the one function that turns multiplication into addition.
• Part 2 — Shannon Entropy Demystified: how that one property became the foundation of all information theory.
• Part 3 — Cross-Entropy Demystified: the price of a wrong model, and the loss function behind half of modern AI.

This is where the series starts — no prior reading needed.

Introduction

In machine learning, we constantly minimize Cross-Entropy Loss and maximize Log-Likelihood. Why does the logarithm show up everywhere? To understand what this function is actually doing, without memorizing formulas, let’s forget about gradients for five minutes and look at bank accounts instead.

If you already grasp the intuition behind logarithms in machine learning, feel free to skip ahead to the next article about entropy a key metric in Machine Learning. There, we’ll build information entropy from the ground up, following in the footsteps of Claude Shannon.

Where the number e comes from

Here’s the most interesting thought experiment. We deposit $1 at 100% annual interest. The question: how much do we have at the end of the year if we reinvest more and more often?

• Once a year: (1 + 1)¹ = 2
• Twice (every six months, 50% each): (1 + 1/2)² = 2.25
• Four times (quarterly, 25% each): (1 + 1/4)⁴ ≈ 2.441
• 12 times (monthly): (1 + 1/12)¹² ≈ 2.613
• 365 times (daily): (1 + 1/365)³⁶⁵ ≈ 2.7146
• 8,760 times (hourly): ≈ 2.71813
• → ∞ (continuously): e ≈ 2.71828…

There it is — the function that describes continuous growth: the exponential. No matter how finely we slice the compounding period, the account never grows past 2.71828… That limit is the number e.

What eⁿ means

e is what we ended up with after one year. We took $1, kept it growing at 100% with infinitely frequent reinvestment, and by year’s end we had ≈ $2.72.

Now imagine we didn’t stop after a year, but kept going under the same conditions:

• After 1 year: e¹ ≈ $2.72
• After 2 years: e² ≈ $7.39
• After 3 years: e³ ≈ $20.09
• After 5 years: e⁵ ≈ $148
• After 10 years: e¹⁰ ≈ $22,026

So eⁿ tells us how much $1 becomes after n years, if we keep growing it at 100% with continuous reinvestment.

The intuition: in year one, the account grows by a factor of e. In year two, it grows by a factor of e again — but starting from $2.72, not $1. In year three, again by a factor of e, but from $7.39. And so on. Every year we multiply by e.

That’s why eⁿ = e · e · e · … (n times). This is the essence of the exponential — a quantity that multiplies itself at every step.

The logarithm — the reverse question

Before, we were asking: “n years have passed — how much money do I have?” Answer: eⁿ.

Now we flip it: “I have $X — how many years have passed?” Answer: ln(X).

Same numbers, going backwards:

• $2.72 → 1 year passed → ln(2.72) = 1
• $7.39 → 2 years → ln(7.39) = 2
• $20.09 → 3 years → ln(20.09) = 3
• $148 → 5 years → ln(148) = 5

So ln(X) tells us how many years we need to keep $1 growing at 100% continuously to reach $X.

Another way to feel this: ln(X) answers the question “by how many factors of e has my account grown?” Grown by e? That’s 1. Grown by e·e? That’s 2. By e·e·e? That’s 3. The logarithm simply counts how many times that “yearly e-multiplication magic” has fired.

That’s why ln(1) = 0: if the account has exactly what we put in, no time has passed — growth hasn’t started yet.

And ln(X) for X less than 1 is negative. Intuitively: the account shrank, as if growth ran “backwards in time.” This fact will come in handy — probabilities are also less than 1, and their logarithms will always be negative.

The logarithm in probability — when we need it

In ML, models often estimate independent events, and the joint probability ends up being a product. Multiplying thousands of numbers smaller than 1 is painful. The logarithm solves that pain. But before we see how, let’s spell out three reasons why:

Reason 1. Multiplying thousands of probabilities on a computer leads to underflow — the number gets so tiny that floating-point precision rounds it to zero. We need to turn multiplication into addition.

Reason 2. In ML, we constantly need to take derivatives. The derivative of a sum is just a sum of derivatives. The derivative of a product of thousands of terms is a combinatorial nightmare (the Leibniz rule), demanding enormous amounts of computation.

Reason 3 — a sneak preview. Maximizing log-likelihood, which we’re about to discover here, is the mathematical twin of minimizing cross-entropy and KL divergence. One follows from the other. But to see this connection without memorizing formulas, we first need to understand what entropy really is. That’s for the next article.

Now to business. Let’s start with how probabilities actually work. They combine in two different ways: by multiplying, and by adding.

When probabilities multiply

Independent events — when one has no effect on the other:

• Two coins. Heads on the first and heads on the second are unrelated.
• A coin and a die. The outcomes know nothing about each other.

The key property: their probabilities multiply.

• P(two heads in a row) = 0.5 · 0.5 = 0.25
• P(heads and a six) = 0.5 · (1/6) ≈ 0.083
• P(10 heads in a row) = 0.⁵¹⁰ ≈ 0.001

The keyword is “and”. When someone asks “what’s the probability that this and that both happen” — the probabilities multiply.

When probabilities add

Mutually exclusive events — events that cannot happen at the same time:

• One die roll. Either 1, or 2, or 3… but not “1 and 2 simultaneously.” P(1 or 2) = 1/6 + 1/6 = 2/6.
• One card from a deck. Either an ace, or a king. P(ace or king) = 4/52 + 4/52 = 8/52.
• One coin. P(heads or tails) = 0.5 + 0.5 = 1.

The keyword is “or”.

A handy mnemonic

• “AND” → multiply (both conditions must hold simultaneously — so there are fewer chances than for either one alone)
• “OR” → add (any one of the events will do — the constraint is looser, the probability is larger)

The logarithm shows up specifically in the multiplication scenario (the “and” — independent events stacking up). With addition it isn’t needed; that case is already convenient.

The key property of the logarithm: why this function specifically?

Here’s the crux. We have a probability: 0.5 · (1/6) = 1/12. The probabilities multiplied.

What’s the problem? Multiplication is inconvenient for the practical reasons above.

Wouldn’t it be great if they added instead, when events combine?

Stating the requirement. We know probabilities multiply: p(joint) = p₁ · p₂. We want some equivalent “weights” that add: weight(joint) = weight(p₁) + weight(p₂).

So we need a function f such that:

f(p₁ · p₂) = f(p₁) + f(p₂)

That is, a function that turns a product into a sum. This is a very special requirement. Most functions can’t do this: squaring? (p₁·p₂)² ≠ p₁² + p₂². Square root? No. Division? No.

Does such a function even exist? Yes. And there’s essentially only one (up to a choice of base) — the logarithm. Its signature property:

log(a · b) = log(a) + log(b)

Where does this come from? Remember: e² · e³ = e⁵ — when you multiply exponentials, the exponents add. The logarithm is exactly “pull the exponent out.” So when you multiplied two numbers, their “exponents” added — and the logarithm just makes that visible.

Let’s check on our example (from here on we’ll use the natural logarithm ln; more on the choice of base in a moment).

• ln(0.5) ≈ −0.69
• ln(1/6) ≈ −1.79
• ln(1/12) ≈ −2.48

And indeed: −0.69 + (−1.79) = −2.48. ✓

So we found our “quantity that adds.” For heads it’s −0.69, for a six it’s −1.79, for both together it’s −2.48.

It’s awkward that the numbers are negative. That’s because probabilities are less than 1, and the logarithm of a number less than 1 is negative. The fix is simple: stick a minus sign in front. −log(p), and everything’s positive.

Choice of base is choice of units

One last important thing. The logarithm has a “base” — log₂, log₁₀, ln. The choice of base is the choice of measurement units, not of meaning.

• log₂ → “bits” → handy for computers (binary system)
• ln (natural log, base e) → “nats” → handy for math and physics (derivatives are nicer)
• log₁₀ → “dits” (a.k.a. hartleys) → historical, from the era of slide-rule calculations

They all measure the same thing, just with different rulers. The same 100 cm = 1 meter = 0.001 kilometers. Same content, different format.

What we can do now

We’ve assembled a tool: the logarithm is the unique function that turns multiplication into addition. It shows up wherever something piles up through multiplication — compound interest, independent events, probabilities.

This tool looks abstract, but it solves very concrete problems. In the next article, we’ll look at how Claude Shannon, in 1948, used exactly this property of the logarithm to lay the foundations of all modern information theory — from ZIP compression to neural network training.