feat(llm): add ornith-1.0-9b coding model to llama-swap menu

DeepReinforce Ornith-1.0 (dense 9B on Qwen 3.5, Q5_K_M, MIT), an
agentic-coding model. Tool-calls + <think> work under --jinja; native
256k so no YaRN. Loads at ~7.7GB VRAM @ 64k.

Benchmark (docs/2026-06-27-ornith-9b-benchmark.md): quality ties
gpt-oss-20b but gen is ~2.5-3x slower (dense 9B active vs gpt-oss MoE
3.6B active on the compute-bound P100). Default stays gpt-oss-20b;
ornith kept as a coding specialist in the menu.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
ginnoir
2026-06-27 13:51:19 -05:00
co-authored by Claude Opus 4.8
parent dc2225d384
commit 009a474e90
2 changed files with 73 additions and 3 deletions
+12 -3
View File
@@ -3,10 +3,10 @@
# only one fits in VRAM at a time, so selecting a different model triggers a
# brief reload. Default is chosen by Hermes (model.default = gpt-oss-20b).
#
# All four serve 64k context (Hermes' minimum). Args are the validated Pascal
# All serve 64k context (Hermes' minimum). Args are the validated Pascal
# config: q8_0 KV (q4_0 V-cache is pathological on GP100), flash-attn on,
# --parallel 1 so one sequence gets the full 64k. gpt-oss/gemma4 are natively
# >=128k so no YaRN/override-kv needed.
# --parallel 1 so one sequence gets the full 64k. gpt-oss/gemma4/ornith are
# natively >=128k so no YaRN/override-kv needed.
#
# Excluded: qwen3-30b-a3b-2507 (Q3) — OOMs at 64k in 16GB, so it can't meet
# Hermes' 64k minimum on this GPU.
@@ -27,6 +27,15 @@ models:
-m /models/gpt-oss-20b-mxfp4.gguf --alias gpt-oss-20b
${common}
"ornith-1.0-9b":
# DeepReinforce Ornith-1.0, dense ~9B on Qwen 3.5 (Q5_K_M). MIT. Agentic-coding
# tuned: <think> block (-> reasoning_content under --jinja) + Qwen3 XML tool calls.
# Native 256k so no YaRN. Recommended sampling: temp 0.6 / top_p 0.95 / top_k 20.
cmd: >
/app/llama-server --port ${PORT} --host 0.0.0.0
-m /models/ornith-1.0-9b-Q5_K_M.gguf --alias ornith-1.0-9b
${common}
"gemma-4-26b-a4b":
# gemma4 MoE, 4B active / 26B total (UD-Q3_K_M). Quality-leaning; ~147 tok/s prefill.
cmd: >