Carnice-9b: A 9B Local Agent Execution Specialist

Carnice-9b is a 9-billion-parameter model by kai-os on Hugging Face, purpose-built for local agent execution. It is based on Qwen3.5-9B, a dense 9B model using a Gated DeltaNet hybrid architecture (mixing attention with state-space models), and follows the Hermes Agent direction: terminal operations, browser automation, file handling, and structured tool use.

• License: Apache-2.0
• Base model: Qwen3.5-9B (dense, hybrid attention/SSM)
• Focus: Hermes-style agent trajectories, trained in two stages
• Quantization: Separate GGUF repo available

Why It Takes a Different Path

While most 9B models compete on MMLU and chat leaderboards, Carnice-9b puts its training budget entirely into real workflows:

• TerminalBench 2.0-style scenarios — command-line operations, filesystem management
• Browser automation — web navigation, search, data extraction
• Structured tool use — JSON function calling, XML-style tool tags
• Multi-step task execution — including error recovery and self-correction

Training Details (Inferred from the 27B Version)

The 9B inherits the training recipe of Carnice-V2-27B, which is publicly documented:

• Full SFT merge (not LoRA adapters), saved as BF16 safetensors
• Training data from reasoning conversation sampling with the Qwen3.5-thinking chat template
• Assistant-only SFT masking — the model learns only from response tokens, improving tool-call precision
• Unsloth training stack, converted to GGUF via llama.cpp, BFCL smoke test included
• IFEval improvement: strict accuracy jumped from 85% to 90%; loss dropped from 0.607 to 0.414

Runtime Pitfalls & Configuration

• Requires a recent llama.cpp build — the Qwen3.5/3.6 Gated DeltaNet hybrid layers error out on older versions
• GGUF files avoid the tensor prefix bug, but IQ quantizations still need the latest build
• Recommended configs:
• 16GB VRAM: Q4_K_M or IQ2_M
• 8GB VRAM: Q2_K or IQ2_M
• Trained context length up to 32,768; tune KV cache for long contexts to avoid OOM

Ecosystem

Carnice-9b is embedded in the Hermes Agent ecosystem, which includes a CLI/TUI interface, server gateway, RL training environment, and benchmark suite. Carnice is one of its recommended models. LocalAI has gallery-level support for carnice-v2-27b, and the hipfire registry lists carnice:9b and carnice:27b tags.

Honest Limitations

1. Not a general-purpose crusher — the base Qwen3.5-9B trails gpt-oss-120b on LiveCodeBench (65.6) and OJBench (29.2); Carnice is specialized, not universal. 2. Text-only training — image/video capability is inherited from the base model with no extra SFT. 3. A rare niche — few 9B models focus on terminal/browser/tool-use, so Carnice fills a gap. 4. Ecosystem-dependent strength — its value is maximized within the Hermes Agent stack.

Getting Started

Load the GGUF in LM Studio or a fresh llama.cpp build, pair it with the Hermes Agent framework, and test terminal and browser tool calling. Run TerminalBench 2.0 or BFCL to compare it against the Qwen3.5-9B base. For pure chat, it may not beat the base model — but for agent scenarios, it is the real star.

References

1. Hugging Face main repo: https://huggingface.co/kai-os/Carnice-9b 2. Hugging Face GGUF repo: https://huggingface.co/kai-os/Carnice-9b-GGUF 3. 27B version: https://huggingface.co/kai-os/Carnice-V2-27b-GGUF 4. Hackernoon article: https://hackernoon.com/this-9b-model-was-trained-for-tool-calling-terminal-work-and-browser-automation 5. Hermes Agent deep dive: https://dev.to/truongpx396/hermes-agent-deep-dive-build-your-own-guide-1pcc