I'm working on a news app where articles are updated frequently. How does Qdrant handle frequent point updates? Does it cause significant re-indexing lag, or is the process seamless enough to maintain real-time search accuracy as news stories evolve?
3 answers
The engine is designed for high-frequency updates. When you update a point in Qdrant, it uses a "write-ahead log" strategy. The change is immediately reflected in the "gray" part of the index, meaning it is searchable right away even before the HNSW graph is fully optimized. There is a background process that periodically merges these changes into the main index structures. You can tune the 'indexing_threshold' in the configuration; a higher threshold means less frequent background work but slightly slower search on the very latest data. For a news app, the default settings are usually sufficient to keep things feeling "live."
Are you updating the vector embeddings themselves or just the metadata payload associated with the news articles?
I recommend using the "upsert" command. It handles both new points and updates to existing ones perfectly without needing separate logic.
Exactly, Scott. Upsert is the standard way to go. It simplifies the application code significantly when dealing with streaming data sources.
Mostly the metadata, like "last updated" timestamps and view counts, but occasionally we re-embed the text if a major correction is made to the story. I was worried that changing the vector would trigger a massive recalculation of the whole neighborhood in the graph, but it sounds like the background merging process handles that quite elegantly without locking the whole database.