Read this before buying
Staff count does not size a server. Model size, quantisation, context length, concurrent requests and latency targets do. Use the sizing calculator for a provisional range, and measure your own workload before committing. We can run that measurement for you.
Evidence status for every figure on this page: Estimated Prices are approximate 2026 street prices from public listings.
| GPU | VRAM | Bandwidth | Est. 8B Q4 tok/s | ~Price |
|---|---|---|---|---|
| NVIDIA B200 | 192 GB | 8000 GB/s | ~480 | $40,000 |
| Apple M3 Ultra | 192 GB | 800 GB/s | ~48 | $4,000 |
| Apple M4 Ultra | 128 GB | 1000 GB/s | ~60 | $4,500 |
| NVIDIA H100 80GB | 80 GB | 3350 GB/s | ~201 | $30,000 |
| NVIDIA A100 80GB | 80 GB | 2039 GB/s | ~122 | $15,000 |
| NVIDIA RTX 6000 Ada | 48 GB | 960 GB/s | ~58 | $6,800 |
| Apple M4 Max | 36 GB | 410 GB/s | ~25 | $2,400 |
| NVIDIA RTX 5090 | 32 GB | 1792 GB/s | ~108 | $2,500 |
| NVIDIA RTX 4090 | 24 GB | 1008 GB/s | ~60 | $1,600 |
| AMD Radeon RX 7900 XTX | 24 GB | 960 GB/s | ~58 | $900 |
| NVIDIA RTX 3090 | 24 GB | 936 GB/s | ~56 | $700 |
| NVIDIA RTX 5070 Ti | 16 GB | 896 GB/s | ~54 | $750 |
| NVIDIA RTX 4070 Ti SUPER 16GB | 16 GB | 672 GB/s | ~40 | $800 |
| AMD Radeon RX 9070 XT | 16 GB | 640 GB/s | ~38 | $600 |
| NVIDIA RTX 4060 Ti 16GB | 16 GB | 288 GB/s | ~17 | $450 |
| Steam Deck OLED (AMD APU) | 16 GB | 89 GB/s | ~5 | $549 |
| NVIDIA RTX 5070 | 12 GB | 672 GB/s | ~40 | $550 |
| NVIDIA RTX 3060 12GB | 12 GB | 360 GB/s | ~22 | $250 |
| NVIDIA Jetson Orin Nano 8GB | 8 GB | 68 GB/s | ~4 | $199 |
| Raspberry Pi 5 (8GB) | 0 GB | 25 GB/s | — | $80 |
Estimate: tokens/sec ≈ bandwidth × 0.06 for an 8B model at Q4_K_M, single request, 2,048-token context. Batched serving raises total throughput but lowers per-user speed. Long contexts need much more memory for the KV cache.
What we mean by a benchmark
A useful benchmark records the model and runtime versions, hardware, quantisation, input and output lengths, concurrency, time to first token, end-to-end latency, resource use and task quality. Our labelling rules are in the editorial policy.