Joseph CostaandClaude Opus 4.8
9938d46a67
A stdlib pre-router in front of Ollama with LiteLLM backend: - auto model selection by content/tools/modality, with fallbacks - OpenAI /v1, Anthropic /v1/messages, and Ollama-native /api/* endpoints - Whisper-shaped /v1/audio/transcriptions + in-chat audio - key-based fleet policies (e.g. force a client onto uncensored models) - optional Bearer auth; launchd/systemd service install - benchmark harnesses (speed, quality, agentic tool use) with sample results Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
24 lines
1.0 KiB
Bash
Executable File
24 lines
1.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# One-time setup: create the Python 3.12 venv and install LiteLLM + Pillow.
|
|
# The pre-router itself needs no venv (stdlib only); this is for the backend proxy.
|
|
set -euo pipefail
|
|
cd "$(dirname "$0")/.."
|
|
|
|
PY="${PYTHON:-python3.12}"
|
|
if ! command -v "$PY" >/dev/null 2>&1; then
|
|
# common macOS Homebrew fallback
|
|
[ -x /opt/homebrew/opt/python@3.12/bin/python3.12 ] && PY=/opt/homebrew/opt/python@3.12/bin/python3.12
|
|
fi
|
|
command -v "$PY" >/dev/null 2>&1 || { echo "Need Python 3.12. Set \$PYTHON, or:"; \
|
|
echo " macOS: brew install python@3.12"; echo " Linux: install python3.12"; exit 1; }
|
|
|
|
echo ">> using $($PY --version) at $(command -v "$PY" 2>/dev/null || echo "$PY")"
|
|
[ -d .venv ] || "$PY" -m venv .venv
|
|
.venv/bin/pip install -q --upgrade pip
|
|
.venv/bin/pip install -q -r requirements.txt
|
|
.venv/bin/python -c "import litellm, PIL" && echo ">> deps OK (litellm, pillow)"
|
|
echo ""
|
|
echo "Next:"
|
|
echo " ./scripts/pull-models.sh # pull a model fleet (edit the list first)"
|
|
echo " ./run.sh # start LiteLLM + router"
|