Compute
Reference parts lists for local AI machines — the components to hit a target model and a throughput band, priced from what you can buy now. Estimates, not test rigs; confirm the model fit with the hardware sizer.
Llama-3-70B (Q4_K_M), roughly 12-16 tok/s
48GB of VRAM fits a 4-bit 70B with room for context. The bottleneck is the PCIe link between the two cards — expect lower throughput than a single card with the same VRAM.
Mixtral 8x22B (FP16), roughly 20-30 tok/s
192GB of VRAM runs a full-precision Mixtral or several quantised 70Bs concurrently. Blower-style cards and real airflow are not optional at this density.
No reader setups published yet. Send yours — parts list, the model you run and the throughput you measured — through the contact page.
