fix(llm): override-kv context_length=65536 so slot isn't capped to 32k

llama-server caps the slot to the GGUF training context (32768) and ignores the
YaRN-extended size, leaving per-seq context at 32k. Raise qwen2.context_length
metadata to 65536 so the full window is served per request.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
ginnoir
2026-06-26 16:45:33 -05:00
co-authored by Claude Opus 4.8
parent 74791105b3
commit 13424fcf75
+2
View File
@@ -44,6 +44,8 @@ services:
- "2"
- "--yarn-orig-ctx"
- "32768"
- "--override-kv"
- "qwen2.context_length=int:65536"
- "--flash-attn"
- "on"
- "--cache-type-k"