vLLM vs SGLang vs Ollama: which serving engine
The question stopped being "which model?" and became "which engine runs it best on my hardware?". Three names cover most of the decision. Here is how they differ in the way that matters — without benchmark numbers that will not transfer to your setup.
The split
| Ollama | vLLM | SGLang | |
|---|---|---|---|
| Made for | One machine, you | A service, many users | A service, shared-prefix workloads |
| Setup | One command | Docker + Python + config | Docker + Python + config |
| Key trick | Wraps llama.cpp cleanly | PagedAttention + continuous batching | RadixAttention (reuses shared prefixes) |
| Quantised weights | GGUF, off the shelf | Supported, more setup | Supported, more setup |
When each wins
- Ollama — development, a personal assistant, a single-user tool. The convenience is the point, and for one request at a time the throughput difference does not exist.
- vLLM — you are serving a feature to real traffic. Continuous batching and the paged KV cache keep the GPU full across many concurrent requests; this is the safe production default.
- SGLang — production and your requests share a large common prefix: a heavy system prompt, a RAG context block, multi-turn chat. RadixAttention caches that shared work instead of recomputing it per request.
What decides it in practice
Time to first token and inter-token latency under your concurrency, on your GPU, with your prompt lengths. The engines trade places depending on all three. Run a load test that mimics production before committing — and see the inference engine entry for the underlying mechanics.
END OF ANALYSIS