What is unified memory, and why does it matter for AI?

Unified memory is a single pool of RAM shared by the CPU and GPU with no copying between them — the architecture Apple Silicon uses, and the reason a Mac can run models that need more memory than any consumer graphics card offers. On an M-series Ultra the GPU can address up to 512 GB at ~800 GB/s.

Why it matters for AI

A discrete GPU is limited to its soldered VRAM — 24 GB on a flagship consumer card, 48-80 GB on workstation and data-center parts. Unified memory removes that ceiling: a 128 GB Mac loads a 70B model at full precision, or several quantised models at once, with no multi-GPU orchestration and no PCIe transfer.

Discrete GPU:   CPU RAM  ──PCIe (≈32 GB/s)──  GPU VRAM (fast, small)
Unified:        CPU + GPU  ───  one pool (≈500-800 GB/s, large)

The trade-off

Unified memory bandwidth (500-800 GB/s) is roughly half that of a dedicated GPU's VRAM (900-1000+ GB/s), so on a model that fits both, the discrete card generates tokens per second faster. Unified memory wins when capacity is the constraint; the GPU wins when speed is, and for training — where the CUDA ecosystem still dominates — the GPU wins outright.