Initial commit: llm-router — smart OpenAI/Anthropic/Ollama router
A stdlib pre-router in front of Ollama with LiteLLM backend: - auto model selection by content/tools/modality, with fallbacks - OpenAI /v1, Anthropic /v1/messages, and Ollama-native /api/* endpoints - Whisper-shaped /v1/audio/transcriptions + in-chat audio - key-based fleet policies (e.g. force a client onto uncensored models) - optional Bearer auth; launchd/systemd service install - benchmark harnesses (speed, quality, agentic tool use) with sample results Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,11 @@
|
||||
# Hard agentic benchmark (multi-turn tool loop) — 2026-07-03 19:09
|
||||
|
||||
Real tool execution with results fed back; thinking enabled.
|
||||
|
||||
| Model | Overall | Seq | Parallel | Chain | Select | Recover | Abstain | Honesty |
|
||||
|---|--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
|
||||
| `glm-4.7-flash` | **100%** | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
|
||||
| `gemma4:26b` | **86%** | 1 | 1 | 1 | 1 | 0 | 1 | 1 |
|
||||
| `qwen3.6:35b-a3b` | **86%** | 1 | 1 | 1 | 1 | 0 | 1 | 1 |
|
||||
| `qwen3-coder:30b` | **86%** | 1 | 1 | 0 | 1 | 1 | 1 | 1 |
|
||||
| `gpt-oss:20b` | **71%** | 1 | 1 | 1 | 1 | 0 | 1 | 0 |
|
||||
Reference in New Issue
Block a user