antirez's DwarfStar: Redis Creator's Deliberately Narrow Inference Engine
What does the creator of Redis build next? Salvatore Sanfilippo (antirez) answered with DwarfStar (ds4), a deliberately narrow local LLM inference engine that has been trending on GitHub.
Continuing the Redis Philosophy
Redis succeeded by doing one thing—memory caching—extremely fast rather than building a general-purpose database. Twenty years later, antirez applies the same philosophy to inference:
> DwarfStar is self-contained and deliberately narrow, not a general GGUF runner.
ds4 supports only three models: DeepSeek V4 Flash, GLM 5.2, and DeepSeek V4 PRO—but runs them to their fullest potential. Models can be swapped when better open weights appear.
Why Not Just Use vLLM or llama.cpp?
General GGUF runners make many compromises to support every model. Notably, vLLM has dropped support for older GPUs (Ada Lovelace architecture), leaving enterprise cards like the L40S stranded. ds4's opportunistic strategy: follow the best open weights, run only those models, but run them fastest.
Technical Highlights
Backends
- Metal: primary target for Macs with 96GB+ RAM; SSD streaming for smaller machines
- NVIDIA CUDA: including multi-GPU systems and DGX Spark
- ROCm: Strix Halo systems (e.g., Framework Desktop)
- 120 t/s aggregate generation speed
- 2000 t/s prefill
- SSD streaming: run on machines with insufficient RAM, even MacBooks
- Tensor parallelism: two MacBook M5 Max / M3 Ultra machines running 4-bit DeepSeek Flash or GLM 5.2 over RDMA
- Pipeline parallelism: chain multiple systems to pool RAM for larger models
- Micro batching: separate batching for decode and prefill
- colibrì: 1,300 lines of C running a 744B-parameter model
- Euclid-MCP: outsourcing reasoning to specialized tools
- vLLM dropping old GPUs: generic systems' compromises become specialized systems' opportunities
- Project: https://github.com/antirez/ds4
- Author: Salvatore Sanfilippo (antirez, creator of Redis)
- License: MIT
- Backends: Metal / CUDA / ROCm
Measured Performance
On 8x NVIDIA L40S cards with multi-user sessions:
This turns "old cards abandoned by vLLM" into a multi-user LLM server—likely ds4's most practical selling point for enterprises.
Other Features
Self-Contained Engineering
Model loading, prompt rendering, tool calling, KV state, HTTP server, and coding agent are all built and tested together—one integrated system, not assembled components.
Candid Disclosure of AI-Assisted Development
The README states openly:
> This software is developed with strong assistance from GPT 5.5, 5.6, Claude Fable and with humans leading the ideas, testing, and debugging. We say this openly because it shaped how the project was built. If you are not happy with AI-developed code, this software is not for you.
This represents a new engineering paradigm: human architects + AI programmers.
Credit to llama.cpp
Though ds4 does not link GGML, the README acknowledges:
> This project would not exist without llama.cpp and GGML.
Some source-level snippets (GGUF quantization layout, CPU quant/dot logic, specific kernels) are retained or adapted under MIT, with GGML's copyright notice preserved in the LICENSE file—an ideal example of open-source community interaction.
Engineering Insight: The Victory of Deliberate Narrowness
ds4 resonates with recent work:
Key takeaway: in the "bigger and stronger" AI narrative, *deliberate narrowness* is an underrated strategy. Redis proved it wins in databases; ds4 is testing whether it holds for inference engines.
---