Member-only story
Local LLMs For Data Work: A Hardware-Honest Guide To What Actually Runs
Benchmarks tell you what scores highest. They don’t tell you what runs on your machine without freezing.
Here’s how to choose by memory bandwidth and workflow, not by leaderboard position.
Not a medium member, Read it for free here.
You’ve probably seen the leaderboards. Qwen 3.5–27B hits 49.2% on SWE-bench. DeepSeek V3.2 is great at tackling algorithmic tasks. Llama 3.3 8B scores 73.0 on MMLU. The numbers look incredible, and you start thinking, I should be running one of these locally for my SQL generation, my code debugging, my schema documentation.
Then you pull the model, watch your RAM thrash for four minutes, and get 2.3 tokens per second, which is slower than you can read.
The leaderboard didn’t lie to you. It just answered a different question than the one you were asking.
For any developer or data and ai professional, the right question is never “what’s the best model?” It’s “what’s the best model my hardware can actually feed fast enough to be useful in the middle of a debugging session?” Those are completely different questions, and answering the wrong one costs you anywhere from an afternoon of frustration to a thousand dollars of hardware you didn’t need.
Here’s how to think through the hardware-first selection process: what the real constraint is, what the…
