fix(llm): use --flash-attn on (this llama.cpp build requires explicit value)
The server-cuda image parses -fa as --flash-attn [on|off|auto], so a bare -fa swallowed the following --cache-type-k as its value and crash-looped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
8e7682985d
commit
605c6d3709
@@ -36,7 +36,8 @@ services:
|
|||||||
- "99"
|
- "99"
|
||||||
- "--ctx-size"
|
- "--ctx-size"
|
||||||
- "32768"
|
- "32768"
|
||||||
- "-fa"
|
- "--flash-attn"
|
||||||
|
- "on"
|
||||||
- "--cache-type-k"
|
- "--cache-type-k"
|
||||||
- "q8_0"
|
- "q8_0"
|
||||||
- "--cache-type-v"
|
- "--cache-type-v"
|
||||||
|
|||||||
Reference in New Issue
Block a user