ginnoirandClaude Opus 4.8
8e7682985d
New stacks/llm/ serves Qwen2.5-14B-Instruct (Q4_K_M GGUF) via llama.cpp's OpenAI-compatible server on the Tesla P100 (CDI nvidia.com/gpu=0), published on 172.20.0.1:8090 for the host-side Hermes agent. vLLM was rejected: the P100 (cc 6.0) lacks the DP4A INT8 instructions its AWQ/GPTQ kernels need. Includes design spec and implementation plan under docs/superpowers/. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
5 lines
304 B
Bash
5 lines
304 B
Bash
# llm stack secrets — read directly by the container via env_file.
|
|
# llama.cpp's server reads LLAMA_API_KEY from the environment (no --api-key flag,
|
|
# no ${VAR} interpolation), matching the repo's pure-env_file convention.
|
|
LLAMA_API_KEY=fc0c0baea5a9ca4eba94a38c8b7c65153594225afbc0b788eba0652fcfdf0069
|