I'm moving from a traditional SQL DB to Milvus for our recommendation engine. How should I structure my schema to filter by category, price, and region while keeping the vector search efficient? Should I use dynamic fields or explicitly define every metadata column in the collection schema?
3 answers
Explicitly defining your fields is always better for performance if your schema is stable. While dynamic fields are flexible, they don't allow for the same level of indexing optimization as predefined scalar fields. For fields like 'region' or 'category', you should definitely create an index on those scalar columns. This allows the search engine to perform "pre-filtering," where it narrows down the vector space based on your metadata before doing the heavy math of the similarity search. This drastically reduces the number of distance calculations the CPU has to perform, leading to much faster response times for the end-user.
Have you looked at the maximum length constraints for string fields in the latest version? Sometimes long text descriptions can bloat the segment size
Yes, Milvus supports boolean expressions for range filtering. Just make sure you use the correct data type like FLOAT or INT64 for the price column.
Brandon is right. As long as you index that price field, the range query will be nearly instantaneous and won't slow down your vector similarity results.
I haven't checked the latest limits, but we are mostly dealing with short tags. My main concern is the 'price' field—can we do range queries efficiently alongside the vector search? We want users to find "items similar to this one but under $50." Is the scalar index capable of handling that kind of numerical range filtering effectively during the vector search phase?