I’ve been testing various and noticed they often struggle with multi-step reasoning that GPT-5 handles easily. Is this a fundamental architectural limit of open weights, or just a lack of diverse fine-tuning data? I’d love to hear from anyone who has managed to get open models to follow strict JSON formatting as reliably as the top-tier closed models.
3 answers
The gap you're seeing is largely due to the RLHF (Reinforcement Learning from Human Feedback) phase. OpenAI has a massive lead in human-preference data, which teaches the model exactly how to follow complex, multi-layered instructions. However, tools like "instructor" and libraries that enforce grammar at the sampler level (like llama.cpp's GBNF grammars) can actually make open models 100% reliable for JSON output. If you constrain the model's output via logic rather than just hoping it follows the prompt, you can achieve "frontier" levels of reliability on a much smaller budget.
Do you think the trend toward "small language models" will eventually make the massive scale of something like GPT-5 obsolete for most developers?
I've found that Mistral Large actually holds its own quite well against GPT-5 for complex instruction following if you use a few-shot prompting strategy.
Few-shotting is definitely a superpower for open models. It bridges that "intelligence gap" significantly by giving the model a clear pattern to follow.
Scott, I think we'll see a hybrid approach. A small, fast open model will handle the routing and simple tasks, while the "heavy lifter" like a GPT-5 or a massive open-weight 400B model will only be called for the truly difficult reasoning steps. This "agentic" workflow is much more efficient than using a giant model for every single "hello world" request, and it's where the industry is clearly heading.