ginnoirandClaude Opus 4.8 8e7682985d feat(llm): add llama.cpp inference stack for Hermes (Qwen2.5-14B on P100) ...

New stacks/llm/ serves Qwen2.5-14B-Instruct (Q4_K_M GGUF) via llama.cpp's
OpenAI-compatible server on the Tesla P100 (CDI nvidia.com/gpu=0), published on
172.20.0.1:8090 for the host-side Hermes agent. vLLM was rejected: the P100
(cc 6.0) lacks the DP4A INT8 instructions its AWQ/GPTQ kernels need.

Includes design spec and implementation plan under docs/superpowers/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-26 16:22:57 -05:00

..

plans

feat(llm): add llama.cpp inference stack for Hermes (Qwen2.5-14B on P100)

2026-06-26 16:22:57 -05:00

specs

feat(llm): add llama.cpp inference stack for Hermes (Qwen2.5-14B on P100)

2026-06-26 16:22:57 -05:00