Running WebNN inference in a Next.js app

WebNN lets a web page run neural-network inference on the device's NPU, GPU or CPU through the OS's own ML runtime — no model download of a runtime, no server round-trip, data never leaves the browser. In a Next.js app it is a client-only concern.

The setup

1. Client component only. WebNN is a browser API — the code must run after hydration, never on the server. Wrap it in "use client" and load the model in a useEffect.
2. Feature-detect and degrade. Not every browser/engine ships WebNN yet. Check for navigator.ml; fall back to a WebGPU or WASM path when it's absent.
3. Load the model as a static asset. Put the .onnx (or framework equivalent) in public/; fetch it once and cache it (the model is often several MB).
4. Pick the device preference. Request "npu" for sustained low-power work (transcription, background effects), "gpu" for bursty heavy ops, "cpu" as the floor.

if (navigator.ml) {
  const context = await navigator.ml.createContext({ deviceType: "npu" });
  // build graph, compute
} else {
  // WebGPU / WASM fallback
}

What it's good for

Use	Device	Notes
Live transcription / translation	NPU	Runs continuously without draining battery
Background blur, image cleanup	NPU/GPU	Real-time on modern hardware
Small on-device generation	GPU	Feasible, tighter context window
Training / fine-tuning	—	Not what WebNN is for; use WebGPU

Failure modes

• SSR crash — any WebNN reference outside a client component and effect breaks the build.
• No fallback — the feature silently disappears for a chunk of users.
• Huge model, no cache — every page load re-downloads it. Use the Cache API or IndexedDB.
• Assuming NPU exists — request a device preference; the runtime picks what's available.

The rule

Treat WebNN as progressive enhancement: the app works without it, and users with a supported browser get local, private, instant inference. Ship the fallback first, add WebNN behind the feature check.