feat(llm): add ornith-1.0-9b coding model to llama-swap menu
DeepReinforce Ornith-1.0 (dense 9B on Qwen 3.5, Q5_K_M, MIT), an agentic-coding model. Tool-calls + <think> work under --jinja; native 256k so no YaRN. Loads at ~7.7GB VRAM @ 64k. Benchmark (docs/2026-06-27-ornith-9b-benchmark.md): quality ties gpt-oss-20b but gen is ~2.5-3x slower (dense 9B active vs gpt-oss MoE 3.6B active on the compute-bound P100). Default stays gpt-oss-20b; ornith kept as a coding specialist in the menu. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
dc2225d384
commit
009a474e90
@@ -3,10 +3,10 @@
|
||||
# only one fits in VRAM at a time, so selecting a different model triggers a
|
||||
# brief reload. Default is chosen by Hermes (model.default = gpt-oss-20b).
|
||||
#
|
||||
# All four serve 64k context (Hermes' minimum). Args are the validated Pascal
|
||||
# All serve 64k context (Hermes' minimum). Args are the validated Pascal
|
||||
# config: q8_0 KV (q4_0 V-cache is pathological on GP100), flash-attn on,
|
||||
# --parallel 1 so one sequence gets the full 64k. gpt-oss/gemma4 are natively
|
||||
# >=128k so no YaRN/override-kv needed.
|
||||
# --parallel 1 so one sequence gets the full 64k. gpt-oss/gemma4/ornith are
|
||||
# natively >=128k so no YaRN/override-kv needed.
|
||||
#
|
||||
# Excluded: qwen3-30b-a3b-2507 (Q3) — OOMs at 64k in 16GB, so it can't meet
|
||||
# Hermes' 64k minimum on this GPU.
|
||||
@@ -27,6 +27,15 @@ models:
|
||||
-m /models/gpt-oss-20b-mxfp4.gguf --alias gpt-oss-20b
|
||||
${common}
|
||||
|
||||
"ornith-1.0-9b":
|
||||
# DeepReinforce Ornith-1.0, dense ~9B on Qwen 3.5 (Q5_K_M). MIT. Agentic-coding
|
||||
# tuned: <think> block (-> reasoning_content under --jinja) + Qwen3 XML tool calls.
|
||||
# Native 256k so no YaRN. Recommended sampling: temp 0.6 / top_p 0.95 / top_k 20.
|
||||
cmd: >
|
||||
/app/llama-server --port ${PORT} --host 0.0.0.0
|
||||
-m /models/ornith-1.0-9b-Q5_K_M.gguf --alias ornith-1.0-9b
|
||||
${common}
|
||||
|
||||
"gemma-4-26b-a4b":
|
||||
# gemma4 MoE, 4B active / 26B total (UD-Q3_K_M). Quality-leaning; ~147 tok/s prefill.
|
||||
cmd: >
|
||||
|
||||
Reference in New Issue
Block a user