My marketing agency wants to move away from stock photos and start using Generative AI to create consistent brand imagery. We’ve experimented with Midjourney, but it’s hard to keep the characters and style consistent across different campaigns. I’m looking into training a custom LoRA on Stable Diffusion XL using our own brand guidelines and product photos. Does anyone have tips on the ideal dataset size? Also, how do I prevent the model from "drifting" too far from the original product's proportions?
3 answers
For a high-quality LoRA, you usually need 20-50 high-resolution images. Consistency is more important than quantity. Make sure your captions (using BLIP or manually) are very descriptive. To prevent "drifting," use a technique called "ControlNet." This allows you to lock in the geometry or edges of your product while the AI generates the style around it. Also, set your "rank" (network dimension) carefully; a rank of 16 or 32 is usually the sweet spot for style. If you go too high, you’ll overfit and the model will only generate exact replicas of your training data.
Are you planning to use a specific base model like SDXL or the older 1.5? The prompting strategies and training requirements differ significantly between them, especially regarding aspect ratios.
Focus on your negative prompts as much as your training. Using terms like "distorted," "extra limbs," or "blurry" in your inference settings will save you a lot of post-processing time.
So true, Linda! A solid negative prompt template is worth its weight in gold when you're trying to push out professional-grade assets on a tight deadline for a client.
William, we are definitely going with SDXL because of its native support for different aspect ratios and better text rendering. Emily, regarding your question about proportions, make sure to include "regularization images" in your training set. These are images of the same category (e.g., if you're training a shoe, use images of generic shoes) to tell the model, "this is a shoe, but my specific product is a unique version of it." This prevents the model from forgetting what a normal object looks like.