Model weights, explained in plain English
Parameters, “active” parameters, 4-bit, GGUF, MLX. Here’s what the labels on a model download actually mean, and how to tell if it fits your Mac.
Decoding a model name
Take a made-up example: Qwen3.6-35B-A3B-Instruct-MLX-4bit
| Piece | Meaning |
|---|---|
Qwen3.6 | Model family and version |
35B | 35 billion parameters in total. This sets how much memory you need. |
A3B | About 3 billion active per word (a mixture-of-experts model). This sets how fast it runs. |
Instruct / it / Chat | Tuned to follow instructions and chat. “Base” models are raw and not meant for chatting. |
MLX or GGUF | File format: MLX for Apple’s framework, GGUF for llama.cpp, LM Studio and Ollama |
4bit / Q4_K_M / Q8_0 | Quantization level: how compressed the weights are |
Memory math in one table
| Model size | 16-bit (original) | 8-bit | 4-bit |
|---|---|---|---|
| 8B | ~16GB | ~8.5GB | ~4.5–5GB |
| 27B | ~54GB | ~29GB | ~15–16GB |
| 35B | ~70GB | ~37GB | ~19–21GB |
| 120B | ~240GB | ~127GB | ~66–72GB |
| 320B | ~640GB | ~340GB | ~175–190GB |
Approximate weight sizes only. Add a few GB (more for long chats) and leave room for macOS.
Common questions
What are model weights?
Weights are the billions of numbers a model learned during training. Downloading an “open-weight” model means downloading that file of numbers so you can run the model on your own computer.
What does 27B mean?
It means the model has about 27 billion parameters (weights). More parameters usually means more knowledge and ability, but also more memory and slower answers.
What are active parameters in a mixture-of-experts (MoE) model?
An MoE model is split into many “experts”, and only a few run for each word. A model labelled 35B-A3B has 35B parameters in total but only about 3B active per word. You still need memory for all 35B, but it runs about as fast as a 3B model.
What is quantization (4-bit, 8-bit)?
Quantization stores each weight with fewer bits so the model is smaller. Original weights are usually 16-bit. 8-bit halves the size with almost no quality loss; 4-bit cuts it to roughly a quarter with a small loss, and is the most popular choice on Macs.
How much memory does a model need?
Rough rule: at 4-bit, about 0.55–0.6GB per billion parameters; at 8-bit, about 1.05–1.1GB; at 16-bit, about 2GB. Then add a few GB for the conversation and leave room for macOS.
What is GGUF?
GGUF is a single-file model format from the llama.cpp project. It is used by llama.cpp, LM Studio and Ollama, and works on Macs, Windows and Linux. File names like Q4_K_M or Q8_0 describe the quantization level.
What is MLX?
MLX is Apple’s open-source machine-learning framework for Apple silicon. MLX model versions (often from the mlx-community group on Hugging Face) are tuned for Macs and are frequently a bit faster there. LM Studio can run MLX models, and mlx-lm runs them from the command line.
Should I pick GGUF or MLX?
On a Mac, try MLX first if your app supports it, since it is often faster. GGUF has the widest selection and works everywhere. Both are fine.
Quick glossary
- Unified memory
- On Apple silicon, the CPU and GPU share one pool of RAM. That’s why Macs with lots of memory can run models that would need several expensive graphics cards on a PC.
- Memory bandwidth
- How fast the chip can read memory. Generating each word means reading the active weights, so higher bandwidth means faster answers.
- Context window
- How much text the model can consider at once. Longer conversations use more memory (the “KV cache”).
- Tokens per second
- Generation speed. Roughly 10+ feels usable for chat; 30+ feels quick.
- Open-weight vs open-source
- Open-weight means you can download the weights. It doesn’t always mean the training data or code is open, and licenses vary.
- Hugging Face
- The main website where open models are published and downloaded.
Mac deal alerts for local AI
Get a short email when a refurbished or open-box Mac with lots of memory drops in price, plus new open models that fit on a Mac.