Skip to content
Fathom Layer
Buying guidesGuide2 MIN READ

Building a Tesla P40 rig for local AI: what you're really signing up for

The Tesla P40 is the cheapest route to a large VRAM pool: 24 GB per card on the used market for a fraction of a consumer 24 GB GPU. Two of them give you 48 GB for the price of one modern card. The catch is that it is a 2016 Pascal data-center card, and several things you take for granted do not work.

What you give up

Missing Consequence
Flash-attention (needs Ampere+) Slower prompt processing, more VRAM per token of context
FP16 tensor cores FP16 compute is slow; use it as an INT8/Q4 card
Active cooling It is a passive server card — you must add a blower shroud + fan
Display output Headless only; you need an iGPU or a cheap second card for the OS
Modern driver niceties Pascal is on the legacy driver branch; check CUDA version support

A realistic build

2× Tesla P40 (48 GB)      used
1× used server/HEDT board with 2 full x16 slots + above-4G decoding / ReBAR
1× 850–1000W PSU (P40 is 250W each)
2× 3D-printed blower shrouds + 2× 40mm high-static-pressure fans
1× cheap GT 710 or an iGPU for display

Enable Above 4G Decoding and Resizable BAR in BIOS or the cards may not initialise together.

Performance expectations

With llama.cpp (no flash-attention), a 4-bit 70B across two P40s lands around 8–12 tok/s — usable, not fast. Prompt processing on long contexts is where you feel the missing flash-attention most. A single modern 24 GB card is 2–3× faster on the models that fit it; the P40 rig exists purely to run models that don't.

The rule

Buy a P40 rig if your goal is "run a 70B at home for the lowest possible cost" and you accept ~10 tok/s, fan noise and a weekend of assembly. If you want speed, or you value your time, a single RTX 3090 or a Mac Studio is the better spend.

END OF ANALYSIS

Related Intelligence