We are building a RAG (Retrieval-Augmented Generation) system for our customer support bot. Everyone keeps mentioning Vector Databases. Why can’t I just store my embeddings in a standard Postgres table using pgvector? Is there a specific scale where a dedicated vector DB becomes mandatory?
3 answers
The "pgvector" vs "Dedicated Vector DB" debate comes down to scale and features. For a few hundred thousand vectors, pgvector is amazing because you keep your metadata and vectors in one place. However, dedicated systems like Pinecone or Milvus are built from the ground up for "Approximate Nearest Neighbor" (ANN) searches at massive scale. They offer better latencies when you hit millions of embeddings and provide specialized indexing like HNSW that is highly optimized for high-dimensional data. If your bot needs to search across millions of documents in milliseconds, the managed scaling of a dedicated vector DB will save you a lot of infrastructure headaches.
How many dimensions are your embeddings, and what is the expected growth rate of your document library over the next year as you add more support articles and data?
Start with pgvector. It’s much easier to manage one database than two. You can always migrate to Milvus later if you actually hit a performance wall at 1 million records.
I second Kevin’s advice. Premature optimization is the root of all evil. pgvector is surprisingly robust for most early-to-mid stage AI startups.
Thomas, we are using OpenAI's text-embedding-3-small which has 1536 dimensions. We expect to have about 50,000 documents initially, but that could grow to 500,000 quickly. Since we're already at 1536 dimensions, I'm worried about the "curse of dimensionality" affecting the search speed. Do you think Postgres can handle that many dimensions efficiently, or will the search accuracy start to degrade significantly as the library grows?