# Routing

## How `model: "auto"` decides

`choose_model()` in [`router.py`](../router.py) runs cheap→expensive, first match wins:

1. **Image in the request** → vision model (`vision_heavy` if the text mentions
   OCR/document/table, else `vision`).
2. **Request carries `tools`** → agentic model (`code_heavy` if the text looks
   code-ish, else `agentic`).
3. **Text heuristics** → code regex → coding model; reasoning words → reasoning
   model; very short (< `SHORT_CHARS`) → the small fast model.
4. **Tiny classifier** — anything left over is labeled by a small always-warm
   model (`CLASSIFIER`) into code / reason / vision / general.

The chosen model, plus any fallback, is reported in the response headers
`x-router-initial-model` (the decision) and `x-router-model` (what actually served).

## The model map — customize this for your machine

Two dicts at the top of `router.py`:

```python
M = {
    "fast":   "qwen3:8b",       "general": "qwen3:14b",
    "code":   "glm-4.7-flash",  "code_heavy": "qwen3-coder:30b",
    "reason": "qwen3.6:35b-a3b", "agentic": "glm-4.7-flash",
    "vision": "qwen3-vl:8b",    "vision_heavy": "qwen3-vl:30b-a3b-instruct",
}
CLASSIFIER = "qwen3:8b"
```

Point these at models you've actually pulled (`ollama list`). Names must match
exactly, including any `:tag`. Restart the router after editing.

`FALLBACK` maps each model to a backup used when it errors or isn't pulled yet —
so the router degrades gracefully instead of failing.

## Explicit model override

Any request that names a real model (not `auto`) is passed straight through:
`{"model": "glm-4.7-flash", ...}`. Unknown names are handled by LiteLLM's catch-all
(`model_name: "*"` in `litellm.config.yaml`), which forwards *any* name to Ollama —
so `:tag` variants and models you pull later work without config changes.

## Policy: forcing a client onto a fleet (the "uncensored" example)

Put a token in `.uncensored_key`. Any request whose `Authorization: Bearer <token>`
matches is forced through `choose_uncensored()` — the same content heuristics, but
only ever selecting from the `UNCENSORED` map — **regardless of the model asked for**.
This pins a specific client (identified by its key) to a specific fleet.

Clients can also opt in per-request with `model: "auto-uncensored"` (no key needed).

Use this pattern for any policy split (a "coding-only" client, a "cheap models"
client, etc.): add a map + a `choose_*` function + a trigger in `do_POST`.

## Adding a model from Hugging Face

Ollama pulls GGUF repos directly:
```bash
ollama pull hf.co/USER/REPO:Q4_K_M          # tag must match the quant in the filename
ollama cp   hf.co/USER/REPO:Q4_K_M  friendly-name   # optional: rename
```
Then add `friendly-name` to the relevant map in `router.py`. Multi-file repos:
Ollama auto-pulls a vision `mmproj` if present; `mtp`/draft files are ignored.
Big prompts truncate at Ollama's default context — make a bigger-context copy with
`./scripts/make-context-variant.sh <model> <num_ctx>`.