We are building a SaaS platform where each client has private data. Can we use a single Chroma DB instance with filtered collections, or should we deploy separate instances per client to ensure there is no leakage of sensitive vector embeddings between accounts?
3 answers
Architecting a secure multi-tenant system with Chroma DB is entirely possible through collection-level isolation. You should create a unique collection for each tenant ID. This ensures that a query from 'Tenant A' can never accidentally retrieve vectors belonging to 'Tenant B'. In my experience developing a multi-user legal assistant, we used an API gateway to validate the user's token and then programmatically selected the correct collection name. This is much more resource-efficient than spinning up hundreds of Docker containers for separate database instances, which would drastically increase your cloud infrastructure costs and maintenance overhead.
Susan, does the performance of the Chroma DB server degrade when you have thousands of small collections instead of one massive collection with metadata filters?
For extreme security requirements, I suggest using separate namespaces or disk-persisted folders for each client to provide physical data isolation.
Laura is right; physical isolation is the safest bet. I’ve used different path locations for Chroma DB persistence in high-security medical apps to meet strict compliance standards.
Michael, the overhead of many collections is relatively low compared to the risk of metadata filtering errors. In our Chroma DB setup, we found that having 500 collections was actually faster for targeted searches than a single 5-million-vector collection where every query had to filter through a 'tenant_id' metadata field.