Member-only story
No LLM Is Actually Free — I Learned That the Hard Way (Twice)
A $20 surprise, a summer spent doing token math by hand, and what finally made LLM costs make sense
I still remember opening the billing page and seeing $20 next to an API I thought I was barely using. Twenty dollars isn’t a lot of money. It also isn’t a lot of money I could explain. I hadn’t changed how the app worked. I hadn’t sent it more traffic. I just… hadn’t been watching, and somewhere in that gap, cost had quietly added up into a number I couldn’t account for.
That moment stuck with me longer than it probably should have, because it wasn’t really about the $20. It was about realizing I had no idea why it was $20 and not $2 or $200. And once I started paying attention, I noticed the same blind spot showing up everywhere I looked at LLM-powered products — including, eventually, my own.
This is the story of how that blind spot led me to build cost observability into an AI product from day one, using Langfuse, and what I learned about why LLM cost is so much harder to reason about than it looks.
“Free” is doing a lot of work in that sentence
Before the $20 bill, I’d already run into a smaller version of the same problem during an internship at a company in Houston. Part of managing cost there meant estimating it by hand — counting how many tokens a query was likely to generate, multiplying by a rate I’d…
