# Deploying llm-router

## Run manually
```bash
./scripts/setup.sh      # once: venv + LiteLLM
./run.sh                # foreground: LiteLLM (:4000) + router (:8080)
```
`run.sh` starts LiteLLM in the background, waits for it, then runs the router in
the foreground. Ctrl-C stops both (it tracks PIDs in `.litellm.pid` / `.router.pid`).

## Run as a boot service
```bash
./deploy/install-service.sh
```
- **macOS** → a launchd user agent (`~/Library/LaunchAgents/com.llm-router.plist`),
  `KeepAlive` (auto-restart), starts at login.
- **Linux** → a systemd user service (`~/.config/systemd/user/llm-router.service`),
  `Restart=always`. Use `loginctl enable-linger $USER` to run without an active login.

The installer prints status / logs / removal commands for your platform.

Restart after editing `router.py` or config:
```bash
# macOS
launchctl kickstart -k gui/$(id -u)/com.llm-router
# Linux
systemctl --user restart llm-router
```

## Windows

```powershell
powershell -ExecutionPolicy Bypass -File .\scripts\setup.ps1        # venv + LiteLLM
powershell -ExecutionPolicy Bypass -File .\scripts\pull-models.ps1  # pull models
powershell -ExecutionPolicy Bypass -File .\run.ps1                  # LiteLLM + router
```
`run.ps1` starts LiteLLM in the background, waits for it, then runs the router in
the foreground; closing it stops both.

**Run at login (Task Scheduler):**
```powershell
$action  = New-ScheduledTaskAction -Execute "powershell.exe" `
  -Argument "-ExecutionPolicy Bypass -WindowStyle Hidden -File `"$PWD\run.ps1`""
$trigger = New-ScheduledTaskTrigger -AtLogOn
Register-ScheduledTask -TaskName "llm-router" -Action $action -Trigger $trigger `
  -Settings (New-ScheduledTaskSettingsSet -RestartCount 3 -RestartInterval (New-TimeSpan -Minutes 1))
```
Requires **Python 3.12** (`winget install Python.Python.3.12`) and Ollama for
Windows. The router (stdlib Python) and LiteLLM run the same as on macOS/Linux.

## Networking

- The router binds `ROUTER_HOST` (default `0.0.0.0` = reachable on the LAN).
- Reach it from other machines at `http://<this-host-ip>:8080/v1` or, on the same
  LAN, `http://<hostname>.local:8080/v1` (mDNS; more stable than a DHCP IP).
- Set `ROUTER_HOST=127.0.0.1` to make it local-only.

### Exposing beyond the LAN

Turn on auth first — put a token in `.apikey` (see `apikey.example`) and restart.
Then front it with one of:

- **Reverse proxy (recommended for a fixed setup)** — e.g. Caddy:
  ```
  router.example.com {
      reverse_proxy localhost:8080
  }
  ```
  Caddy handles TLS; the router handles auth. Pair with `ROUTER_HOST=127.0.0.1`
  so the only way in is through the proxy.
- **Quick tunnel (ephemeral)** — `cloudflared tunnel --url http://localhost:8080`
  gives a temporary `https://…trycloudflare.com` URL. Good for a quick test; the
  URL changes on restart and it's internet-facing, so keep `.apikey` set.

### A note on Ollama itself
The router connects to Ollama at `ROUTER_OLLAMA` (default `http://127.0.0.1:11434`).
If you want *other machines* to reach Ollama directly (not just via the router),
set Ollama's own `OLLAMA_HOST=0.0.0.0:11434` and restart Ollama — but that is the
**server bind** var and is unrelated to `ROUTER_OLLAMA` (the router's client URL).
Don't set `ROUTER_OLLAMA` to `0.0.0.0` — it's a connect address.

## Health & logs
```bash
curl -s http://localhost:8080/healthz          # {"ok":true}
curl -s http://localhost:8080/v1/models        # route targets
tail -f launchd.out.log                        # macOS service logs (router access + decisions)
tail -f litellm.log                            # LiteLLM backend
```
Each routed request logs a line like `[router] router -> glm-4.7-flash`. Responses
carry `x-router-model`, `x-router-initial-model`, and `x-router-decided-by` headers.