fix(llm): --parallel 1 so a single request gets the full 64k context

With the default 4 slots, llama-server splits ctx into 32k per sequence, which
fails Hermes' 64K minimum. One slot serves the full 65536 per request (serial
agent use; concurrent calls queue).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
ginnoir
2026-06-26 16:42:41 -05:00
co-authored by Claude Opus 4.8
parent 265b407e8d
commit 74791105b3
+2
View File
@@ -32,6 +32,8 @@ services:
- "/models/Qwen2.5-14B-Instruct-Q4_K_M.gguf"
- "--alias"
- "qwen2.5-14b-instruct"
- "--parallel"
- "1"
- "-ngl"
- "99"
- "--ctx-size"