Can I run it?
Pick a model and your hardware to see how much memory it needs and roughly how fast it will run.
✓ Runs: best at FP16, needs 17.6 GiB, about 43 tokens/s
| Quantization | Weights | KV | Total | Usable | Result | tok/s (est.) |
|---|
| Q4_K_M | 4.6 | 1.1 | 6.5 GiB | 24.0 GiB | Fits | 142 |
| Q5_K_M | 5.4 | 1.1 | 7.3 GiB | 24.0 GiB | Fits | 121 |
| Q6_K | 6.3 | 1.1 | 8.2 GiB | 24.0 GiB | Fits | 105 |
| Q8_0 | 8.1 | 1.1 | 10.1 GiB | 24.0 GiB | Fits | 81 |
| FP16 | 15.3 | 1.1 | 17.6 GiB | 24.0 GiB | Fits | 43 |
Estimates, not benchmarks. How we estimate · DeepSeek-R1-0528-Qwen3-8B · GeForce RTX 4090
For AI agents and developers: the same calculation is available from the API, e.g. /api/v1/fit?model=…&hardware=…&context=8192, or through the MCP server at /mcp.