I’ve heard great things about the DeepSeek reasoning models. Can I run the 32B version on a MacBook M2 with 16GB of unified memory using Ollama? Or should I stick to the smaller 7B distilled versions? I'm looking for the best balance between reasoning "thinking" time and actual hardware performance.
3 answers
On a 16GB Mac, the 32B model is going to be a struggle because it will consume almost all your unified memory, leaving nothing for the OS. You'll likely see "out of memory" errors or extremely slow response times. I recommend pulling the 7B or 14B versions via Ollama. The 14B version is surprisingly capable of complex logic and fits comfortably within 10-12GB of VRAM. I tested the 7B version last month, and while it's fast, the "thinking" process is much more robust on the mid-sized models if you can afford the memory overhead.
Are you using the default 4-bit quantization, or are you trying to run a higher precision version of the model?
The 8B distilled version based on Llama 3 is incredibly snappy on M-series chips and handles most coding tasks quite well.
Agreed, Pamela. For a 16GB machine, that 8B version is the most "usable" daily driver when running through Ollama alongside other open apps like VS Code.
I usually just use the standard ollama pull deepseek-r1 command, which I believe defaults to Q4_K_M. It seems to be the sweet spot for consumer hardware, though I wonder if Q2 would let me squeeze the bigger model in.