Hardware Architecture Center
Silicon
Unleashed.
Build, benchmark, and deploy. Discover the exact hardware specifications required to run the next generation of open-weight models directly on your desk.
Indexed
44 systems
Calculator
Live model lookup
Live VRAM Calculator
Connected to the HuggingFace API. Paste any Model ID to calculate the exact VRAM requirements based on neural weight parameters and 20% Context Window overhead.
Highest rated right now
The top-scoring systems in the compute index — by our design score, not by payout.
Mac Studio (M3 Ultra)
Apple's compact workstation, now with 819 GB/s of unified memory bandwidth.
Apple MacBook Pro 16" (M3 Max)
The definitive mobile workstation for local AI development. Powered by the M3 Max chip with a 40-core GPU and up to 128GB of unified memory running at 400GB/s, it runs massive LLMs entirely on-device without thermal throttling.
Some links on item pages may earn a commission. It never affects the score.
The Bottleneck Analyzer
Select your architecture to reveal memory bandwidth limitations.
Maximum Memory Bandwidth
Theoretical 7B Inference (Q4)
252 t/s
The Hardware Explorer
Filter and search across our entire database of Local AI hardware.
Apple M4 Max SoC
Mac Studio (M3 Ultra)
Apple M5 iPad Pro (OLED)
Asus ZenBook A16 OLED
Nvidia GeForce RTX 5090 (32GB)
Wooting 80HE
Apple MacBook Air (M5)
ROG Xbox Ally X20
Apple MacBook Pro 16" (M3 Max)
ThinkPad T14s Gen 6 Snapdragon
Nvidia GeForce RTX 5070 Ti (16GB)
Nintendo Switch 2
+ 32 more items. Refine your search.
The Hardware Tiers
View all rigs →Top Pick
Mac Studio (M3 Ultra)
Apple's compact workstation, now with 819 GB/s of unified memory bandwidth.
Top Pick
Apple MacBook Pro 16" (M3 Max)
The definitive mobile workstation for local AI development. Powered by the M3 Max chip with a 40-core GPU and up to 128GB of unified memory running at 400GB/s, it runs massive LLMs entirely on-device without thermal throttling.
Top Pick
Mac Studio M2 Ultra
The ultimate local AI workstation for running massive 70B+ parameters models effortlessly thanks to 192GB of unified memory.
Deep dives & guides
Build blueprintsFrequently Asked Questions
Local AI Architecture
To run Llama 3 70B at a usable 4-bit quantization (Q4_K_M), you need approximately 40GB to 48GB of VRAM. This typically requires either a Mac Studio M2/M3 Ultra with unified memory, or a multi-GPU setup like 2x RTX 4090 or 2x RTX 3090.
Apple's Unified Memory offers massive capacity at a lower price point than enterprise GPUs, making it excellent for running huge models (like 70B or 104B parameters) that wouldn't fit on a single consumer GPU. However, for smaller models that fit entirely within a single RTX 4090 (24GB), the dedicated GPU will offer significantly faster token generation speeds.
Yes, you can offload layers to system RAM using frameworks like llama.cpp. However, system RAM bandwidth is typically around 50-90 GB/s, whereas a dedicated GPU like the RTX 4090 provides over 1000 GB/s. Offloading to system RAM will cause a massive bottleneck, resulting in very slow token generation speeds.
Currently, the best budget GPU for Local AI is a used RTX 3090 (24GB VRAM). It offers the same VRAM capacity as the RTX 4090 at a fraction of the cost, making it the king of budget inference. For brand new options, the RTX 4060 Ti 16GB offers a good entry point.