Power and HBM4: The Two Hard Limits on AI Buildout

Power and HBM4: The Two Hard Limits on AI Buildout

Model progress is not what gates the next wave of AI capacity. Two physical inputs do: firm power delivered to the datacenter, and HBM4 / advanced-packaging throughput at the fab. Both have lead times measured in years, and both are concentrated in a handful of suppliers.

The numbers

• A single large training cluster now draws 150-300 MW. Grid interconnection queues in the major US markets run 24-48 months.
• HBM is >90% supplied by three vendors. HBM4 ramps through 2026-2027; every high-end accelerator competes for the same stacks.
• Advanced packaging (CoWoS-class) capacity is the real ceiling on accelerator output — more binding than wafer starts.

Accelerator cost of ownership, 2026
  silicon + packaging   ~45%
  HBM stack             ~30%   <- the shortage tax
  board + system        ~15%
  power + cooling (3yr) ~10%   <- rising fast

Decision matrix

If you are...	Your binding constraint	Hedge
Building a cluster	Firm power contract + packaging allocation	Sign both 18-24 months ahead; co-locate with generation
Buying cloud capacity	Reserved-instance availability in your region	Multi-region reservations; accept older-gen silicon for non-training
Selling inference	Your COGS floor is set by HBM price	Optimize for tokens/sec/GB-HBM, not raw FLOPs

Playbook

1. Treat power procurement as a core competency, not a facilities line item.
2. Design workloads to run on N-1 generation silicon — it is available and 30-40% cheaper.
3. Watch the HBM4 ramp; the shortage tax compresses when it clears, and inference economics move with it.