# Local fleet benchmark — 2026-07-03 17:43

## Performance (sorted by generation tok/s)

| Model | Gen tok/s | Prompt tok/s | Cold load | TTFT | Placement | OCR | Status |
|---|--:|--:|--:|--:|:--|:--:|:--|
| `gemma4:e2b` | 100.0 | 610 | 0.2s | 0.3s | 100% GPU | PASS | ok |
| `qwen3-coder:30b` | 85.7 | 297 | 0.1s | 0.2s | 100% GPU | - | ok |
| `qwen3-vl:30b-a3b-instruct` | 85.4 | 129 | 0.1s | 0.4s | 100% GPU | PASS | ok |
| `qwen3.6:35b-a3b` | 74.5 | 250 | 0.2s | 0.3s | 100% GPU | - | ok |
| `gpt-oss:20b` | 72.9 | 516 | 0.2s | 0.4s | 100% GPU | - | ok |
| `gemma4:26b` | 70.2 | 212 | 0.2s | 0.4s | 100% GPU | PASS | ok |
| `gemma4:e4b` | 66.0 | 369 | 0.2s | 0.3s | 100% GPU | PASS | ok |
| `glm-4.7-flash:latest` | 65.5 | 150 | 0.1s | 0.3s | 100% GPU | - | ok |
| `qwen3-vl:8b` | 60.4 | 363 | 0.1s | 0.2s | 100% GPU | PASS | ok |
| `qwen3:8b` | 59.0 | 244 | 0.1s | 0.2s | 100% GPU | - | ok |
| `gemma4:12b` | 35.9 | 167 | 0.2s | 0.5s | 100% GPU | PASS | ok |
| `qwen3:14b` | 35.7 | 200 | 0.1s | 0.3s | 100% GPU | - | ok |
| `qwen3.6:27b` | 18.3 | 71 | 0.2s | 0.7s | 100% GPU | - | ok |
| `gemma4:31b` | 16.6 | 66 | 0.2s | 0.8s | 100% GPU | PASS | ok |

## Routing correctness

| Case | Expected | Routed to | HTTP | Verdict |
|---|---|---|--:|:--:|
| short chat | `qwen3:8b` | `qwen3:8b` | 200 | PASS |
| code | `glm-4.7-flash` | `glm-4.7-flash` | 200 | PASS |
| repo code | `qwen3-coder:30b` | `qwen3-coder:30b` | 200 | PASS |
| reasoning | `qwen3.6:35b-a3b` | `qwen3.6:35b-a3b` | 200 | PASS |
| vision | `qwen3-vl:8b` | `qwen3-vl:8b` | 200 | PASS |
| ocr/doc | `qwen3-vl:30b-a3b-instruct` | `qwen3-vl:30b-a3b-instruct` | 200 | PASS |