We are building a SaaS platform where each user needs their own isolated search space. Should we create a separate collection in Qdrant for every tenant, or is it better to use a single collection with a 'tenant_id' payload filter to ensure data isolation and performance?
3 answers
For most SaaS use cases, using a single collection with payload filtering is the way to go. Qdrant is highly optimized for filtering, and creating thousands of individual collections can lead to massive overhead in terms of memory and management. When you use a 'tenant_id' as a keyword filter, the engine uses bitmasking to quickly exclude irrelevant vectors before the actual HNSW search happens. This keeps the search speed nearly identical to a smaller, dedicated collection. However, if you have a "VIP" tenant with millions of vectors, moving them to a dedicated shard or collection might be worth the extra operational complexity.
How many total tenants are you expecting to support? If the number is in the thousands, have you looked into the memory overhead per collection?
Payload filtering is definitely easier to maintain. Just ensure you have indexed the 'tenant_id' field specifically to keep the filtering step fast.
I agree with Rebecca. Without that keyword index on the payload, the performance will degrade significantly as your total vector count grows across all tenants.
We expect around 5,000 tenants. Each collection in this database has a baseline RAM requirement for the index metadata, so 5,000 collections would likely crash our current cluster. We need to be very careful with how we allocate resources to avoid spiraling cloud costs while maintaining sub-second query latency for our users.