Initial commit: llm-router — smart OpenAI/Anthropic/Ollama router
A stdlib pre-router in front of Ollama with LiteLLM backend: - auto model selection by content/tools/modality, with fallbacks - OpenAI /v1, Anthropic /v1/messages, and Ollama-native /api/* endpoints - Whisper-shaped /v1/audio/transcriptions + in-chat audio - key-based fleet policies (e.g. force a client onto uncensored models) - optional Bearer auth; launchd/systemd service install - benchmark harnesses (speed, quality, agentic tool use) with sample results Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Executable
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# One-time setup: create the Python 3.12 venv and install LiteLLM + Pillow.
|
||||
# The pre-router itself needs no venv (stdlib only); this is for the backend proxy.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
PY="${PYTHON:-python3.12}"
|
||||
if ! command -v "$PY" >/dev/null 2>&1; then
|
||||
# common macOS Homebrew fallback
|
||||
[ -x /opt/homebrew/opt/python@3.12/bin/python3.12 ] && PY=/opt/homebrew/opt/python@3.12/bin/python3.12
|
||||
fi
|
||||
command -v "$PY" >/dev/null 2>&1 || { echo "Need Python 3.12. Set \$PYTHON, or:"; \
|
||||
echo " macOS: brew install python@3.12"; echo " Linux: install python3.12"; exit 1; }
|
||||
|
||||
echo ">> using $($PY --version) at $(command -v "$PY" 2>/dev/null || echo "$PY")"
|
||||
[ -d .venv ] || "$PY" -m venv .venv
|
||||
.venv/bin/pip install -q --upgrade pip
|
||||
.venv/bin/pip install -q -r requirements.txt
|
||||
.venv/bin/python -c "import litellm, PIL" && echo ">> deps OK (litellm, pillow)"
|
||||
echo ""
|
||||
echo "Next:"
|
||||
echo " ./scripts/pull-models.sh # pull a model fleet (edit the list first)"
|
||||
echo " ./run.sh # start LiteLLM + router"
|
||||
Reference in New Issue
Block a user