fix(llm): override-kv context_length=65536 so slot isn't capped to 32k
llama-server caps the slot to the GGUF training context (32768) and ignores the YaRN-extended size, leaving per-seq context at 32k. Raise qwen2.context_length metadata to 65536 so the full window is served per request. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
74791105b3
commit
13424fcf75
@@ -44,6 +44,8 @@ services:
|
||||
- "2"
|
||||
- "--yarn-orig-ctx"
|
||||
- "32768"
|
||||
- "--override-kv"
|
||||
- "qwen2.context_length=int:65536"
|
||||
- "--flash-attn"
|
||||
- "on"
|
||||
- "--cache-type-k"
|
||||
|
||||
Reference in New Issue
Block a user