I'm building a retail bot that needs to recognize thousands of different products. We often get new items with only 2-3 sample images. Is "Prototypical Networks" still the best way to handle this, or should I be looking at Siamese Networks with triplet loss?
3 answers
For a retail environment with thousands of classes, Siamese Networks (or more modern versions using Cosine Similarity Loss) are generally more scalable than Prototypical Networks. Prototypical Networks work by calculating a "mean" vector for each class, which is great for 5-10 classes but gets messy with 2,000. With Siamese Networks, you are teaching the model to learn a "Metric Space" where images of the same product are close together and different products are far apart. This way, when a new product arrives, you just save its embedding in a database (like FAISS). You don't need to retrain the whole model; you just do a nearest-neighbor search.
If we use Siamese Networks, how do we choose the "Hard Negatives" for the triplet loss? I’ve heard that picking the wrong pairs can make the model converge very slowly.
Contrastive Learning is the modern evolution here. Look into "SimCLR" or "MoCo." They use self-supervised learning to create incredibly robust embeddings that work perfectly for few-shot tasks.
Martha is right. Self-supervised pre-training on a large unlabeled dataset of your store's products will give you a much stronger baseline for any few-shot task later on.
You've hit on the most difficult part of metric learning, Jeffrey. If you pick "Easy Negatives" (e.g., comparing a soda bottle to a shoe), the loss becomes zero almost instantly and the model stops learning. We use "Semi-Hard Negative Mining." We look for images that are currently labeled as different but are mathematically very close in the embedding space. This forces the model to learn the tiny details—like the difference between a 12oz Coke and a 16oz Coke. Using a library like "PyTorch Metric Learning" simplifies this because it has these mining strategies built-in as standard functions.