Member-only story
Real-Time Embedding Updates: Keeping Vector Search Fresh Without Full Reindex
A few months back, one of our product catalogs started drifting. Someone would update a listing — new price, new description — and our semantic search would keep surfacing the old version for another six hours. Not because anything was broken, but because “fresh” wasn’t part of our vector pipeline’s vocabulary. We treated embeddings like a nightly batch job: rebuild the index, swap it in, done. That worked fine until the business started asking for search results that reflected what was true right now, not what was true at 2 AM.
This is the problem most teams run into once vector search moves from “nice demo” to “production feature”: full reindexing doesn’t scale with update frequency, but naive real-time writes introduce consistency headaches that full reindexing conveniently sidesteps. Here’s how we think about the trade-offs, and the patterns that actually held up.
Why full reindex breaks down
Rebuilding an entire vector index — recomputing embeddings for every document and rebuilding the ANN structure (HNSW, IVFFlat, whatever you’re on) — is simple and correct. It’s also O(n) in the worst way: every small change costs you the same as a million changes. Once your catalog crosses a few hundred thousand items, or your update rate crosses “a…
