What are ternary LLMs (BitNet b1.58)?
A ternary LLM constrains every weight to one of three values — −1, 0 or +1 — which needs about 1.58 bits of storage each (log₂3 ≈ 1.58), against 16 bits for a standard model. The idea comes from Microsoft Research's paper "The Era of 1-bit LLMs" (arXiv 2402.17764, February 2024) and its BitNet b1.58 architecture.
How it differs from quantization
Quantization shrinks a model after training and loses some accuracy. BitNet trains in low precision from scratch: a BitLinear layer replaces the standard linear layer, and because the weights are −1/0/+1, the matrix multiplies collapse into additions and subtractions with no floating-point multiply.
What the paper reports
| Metric — 3B params, vs an FP16 LLaMA of equal size | Result |
|---|---|
| Perplexity and zero-shot accuracy | Matched |
| GPU memory | 3.55× lower |
| Inference speed | 2.71× faster |
Energy use for the core arithmetic drops substantially as well, because additions cost far less than floating-point multiplies.
The catch
The gains are largest with kernels and hardware built for ternary math. Microsoft has since released an actual model (BitNet b1.58 2B4T, April 2025) and a runtime (bitnet.cpp), but the ecosystem is early — and you cannot convert an existing FP16 checkpoint into one.