Joseph CostaandClaude Opus 4.8
9938d46a67
A stdlib pre-router in front of Ollama with LiteLLM backend: - auto model selection by content/tools/modality, with fallbacks - OpenAI /v1, Anthropic /v1/messages, and Ollama-native /api/* endpoints - Whisper-shaped /v1/audio/transcriptions + in-chat audio - key-based fleet policies (e.g. force a client onto uncensored models) - optional Bearer auth; launchd/systemd service install - benchmark harnesses (speed, quality, agentic tool use) with sample results Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
551 B
551 B
Hard agentic benchmark (multi-turn tool loop) — 2026-07-03 19:09
Real tool execution with results fed back; thinking enabled.
| Model | Overall | Seq | Parallel | Chain | Select | Recover | Abstain | Honesty |
|---|---|---|---|---|---|---|---|---|
glm-4.7-flash |
100% | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
gemma4:26b |
86% | 1 | 1 | 1 | 1 | 0 | 1 | 1 |
qwen3.6:35b-a3b |
86% | 1 | 1 | 1 | 1 | 0 | 1 | 1 |
qwen3-coder:30b |
86% | 1 | 1 | 0 | 1 | 1 | 1 | 1 |
gpt-oss:20b |
71% | 1 | 1 | 1 | 1 | 0 | 1 | 0 |