Key points
- The hidden cost of AI coding: The default
grep → read_file → sub-agent recursionpipeline dominates Claude Code token spend. Stephan Tulkens estimates grep+read consumes ~50x the characters of the actually needed snippet, so most of the daily token budget is structural waste, not prompt-engineerable. - What Semble is: A zero-dependency, CPU-only code search layer from MinishLab (Stephan Tulkens, Thomas van Dongen), released May 2026. Marketed as 98% token reduction while keeping 99% retrieval quality. Integrates via MCP server (
claude mcp add semble) and asemble searchCLI. It hit 135 HN points, 37 comments, and 2,500+ GitHub stars in its first week. - Architecture, four layers: 1. *Chunking*: Chonkie over Tree-sitter ASTs, respecting function, class, and module boundaries. Supports Python, JS/TS, Rust, Go, Java, C/C++, Kotlin, Ruby, PHP, Swift. 2. *Dual retrieval*: potion-code-16M (16M params, ~60MB, MinishLab-trained static embeddings via vocabulary lookup → mean pooling → SIF weighting → L2 norm) plus BM25 for symbol precision. 3. *Fusion*: Reciprocal Rank Fusion with adaptive weights—symbol queries like
- Why it runs on CPU: No matrix multiplies, no attention, no GPU memory copies. Embedding lookup → BM25 → RRF → heuristic rerank gives the 1.5ms query latency.
- Benchmarks (19 languages, 63 repos, ~1,250 pairs): | Metric | Semble | CodeRankEmbed Hybrid (137M) | grep+read | |---|---|---|---| | NDCG@10 | 0.854 | 0.862 | ~0.3–0.4 | | Index time | ~250ms | ~50s (GPU) | 0s | | Query latency | ~1.5ms | ~15ms (GPU) | ~100ms | | Token cost | ~2% of grep+read | N/A | 100% | | Hardware | CPU | GPU/API | CPU | | External deps | None | Model/API key | None |
- Competitive landscape: grep/ripgrep (fast, cheap, no semantics), Semble (sweet spot for agent cost/latency/quality), CodeRankEmbed Hybrid (highest quality, needs GPU/API), claude-context / Zilliz (vector DB integration with Claude Code, but extra service and cost).
- Real limitations: 1. Long-function truncation: 500-line
- Adoption guidance:
- *Strong fit*: heavy Claude Code / Cursor / Codex use on >10k-line repos, monthly token spend >$100, frequent 200k-context overflow, multi-repo jumping.
- *Worth trying*: small/medium projects, existing in-house RAG to benchmark against, sub-10ms latency needs.
- *Skip*: <1k-line repos where ripgrep is fine; deep cross-file reasoning; teams already on Sourcegraph / Copilot Workspace; non-agent workflows.
- Project: https://github.com/MinishLab/semble
- Star history: https://star-history.com/#MinishLab/semble&Date
- Dev.to token-cost write-up: https://dev.to/sahil_kat/cut-claude-code-token-usage-98-with-purpose-built-mcps-4h0c
- Hacker News thread: https://news.ycombinator.com/item?id=48169874
- MinishLab org: https://github.com/MinishLab
- Docs: https://minish.ai/packages/semble/introduction/
- potion-code-16M model: https://huggingface.co/minishlab/potion-code-16M
- CodeRankEmbed Hybrid baseline: Semble README benchmarks section
- Chonkie: https://github.com/bhavnicksm/chonkie
- Model2Vec paper: https://arxiv.org/abs/2411.07163
Foo::bar boost BM25; natural-language queries like "how does authentication work" stay balanced.
4. *Reranking*: Five code-aware signals—definition boost, identifier stemming, file coherence, noise penalty (tests, compat/ shims, .d.ts stubs), and adaptive weighting.
Real-world: 50k-line repos drop from ~4,200 tokens per grep to ~84 tokens via Semble; 100k-line repos drop from 15,000–20,000 to 300–500 tokens. semble savings estimates savings as (file_chars − snippet_chars) / 4. At 40 searches/day that is ~20M tokens/year, worth thousands of dollars in API cost and—more importantly—frees the context window for longer conversations and reasoning chains.
init() blocks or templates stay as a single chunk; cross-function event flows get split.
2. Static embedding ceiling: process() in a data pipeline vs. a payment gateway gets the same vector; transformers can contextually attend but static models cannot.
3. Agent trust deficit—the biggest practical pit. Agents may ignore MCP tools and fall back to grep unless AGENTS.md / CLAUDE.md mandates semble search. Catch-22: if the agent obeys complex instructions, you need Semble less; if it doesn't, the hint is just another prompt.
4. Index rebuild cost on very large repos (Linux kernel, Chromium) and possible staleness on fast-moving branches despite local-path watchers.
5. Semble only solves retrieval, not reasoning—LLM inference tokens still dominate after retrieval.
Bottom line
Semble is not "small model beats big model"—it is small model + classical IR + domain heuristics winning under tight agent constraints. Three trade-offs define the win: drop dynamic semantics for ~1000x faster lookup; drop end-to-end learning for interpretable, tunable rules; drop generality for single-task focus. Whether it becomes Agent infrastructure depends on whether Claude Code, Cursor, and Codex make semantic retrieval the default instead of leaving it to user-configured MCP servers.