Joseph CostaandClaude Opus 4.8
9938d46a67
A stdlib pre-router in front of Ollama with LiteLLM backend: - auto model selection by content/tools/modality, with fallbacks - OpenAI /v1, Anthropic /v1/messages, and Ollama-native /api/* endpoints - Whisper-shaped /v1/audio/transcriptions + in-chat audio - key-based fleet policies (e.g. force a client onto uncensored models) - optional Bearer auth; launchd/systemd service install - benchmark harnesses (speed, quality, agentic tool use) with sample results Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
12 lines
551 B
Markdown
12 lines
551 B
Markdown
# Hard agentic benchmark (multi-turn tool loop) — 2026-07-03 19:09
|
|
|
|
Real tool execution with results fed back; thinking enabled.
|
|
|
|
| Model | Overall | Seq | Parallel | Chain | Select | Recover | Abstain | Honesty |
|
|
|---|--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
|
|
| `glm-4.7-flash` | **100%** | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
|
|
| `gemma4:26b` | **86%** | 1 | 1 | 1 | 1 | 0 | 1 | 1 |
|
|
| `qwen3.6:35b-a3b` | **86%** | 1 | 1 | 1 | 1 | 0 | 1 | 1 |
|
|
| `qwen3-coder:30b` | **86%** | 1 | 1 | 0 | 1 | 1 | 1 | 1 |
|
|
| `gpt-oss:20b` | **71%** | 1 | 1 | 1 | 1 | 0 | 1 | 0 |
|