Codebase-Memory-MCP: Indexing the Linux Kernel in 3 Minutes with 120x Fewer Tokens

> Paper: Vogel M, et al. (2026). *Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP*. arXiv:2603.27277 > Project: https://github.com/DeusData/codebase-memory-mcp > Authors: Martin Vogel, Falk Meyer-Eschenbach, Severin Kohler, Elias Grünewald, Felix Balzer (Berlin team) > License: MIT > Languages: 158 (Tree-Sitter grammars) > Hybrid LSP: Python, TS/JS/JSX/TSX, PHP, C#, Go, C, C++, Java, Kotlin, Rust

One-line positioning

This is not another code search tool. It turns a codebase's structure into a queryable knowledge graph, exposed to any AI coding assistant via 14 MCP tools. One graph, one indexing pass — and every subsequent query costs under 1ms and roughly 1,000 tokens, instead of the tens of thousands consumed by file-by-file grep.

Key points

• RAM-first, three-phase pipeline: Parse (Tree-Sitter ASTs across 158 languages) → Build (6-stage parallel pipeline with cascade call resolution, Louvain community detection, XXH3 hashing) → Serve (14 MCP tools over SQLite). Indexing happens entirely in memory with LZ4 HC compression, then flushes to disk once — eliminating disk I/O bottlenecks.
• Linux kernel benchmark: 28,000,000 LOC / 75,000 files indexed in 3 minutes on Apple M3 Pro → 2,100,000 nodes and 4,900,000 edges. BFS call-chain tracing runs at ~0.3ms; dead-code detection ~150ms.
• Token efficiency: 5 structured queries cost ~3,400 tokens vs. ~412,000 tokens for file-by-file exploration — a ~120x reduction. The key insight: traditional agents re-discover structure every session; the graph pre-computes it once.

Comparison with CodeGraph

Both projects target the same pain point — AI assistants wastefully re-exploring codebases each session — but take different routes:

| Dimension | CodeGraph | Codebase-Memory-MCP | |---|---|---| | Deployment | Node.js/npm required | Single binary, zero dependencies | | Index speed | Moderate | Linux kernel 28M LOC / 3 min | | Storage | Memory + files | RAM-first + SQLite persistence | | Graph sharing | Basic | Compressed snapshots committed to git | | Security | Standard | 8-layer CI + 70+ AV engines + SLSA L3 | | Languages | 16+ | 158 (Tree-Sitter) + 10 (Hybrid LSP) | | Token efficiency | High | 10-120x reduction | | Incremental sync | Limited | Background watcher + XXH3 content hashing |

The 14 MCP tools

• Indexing (4): index_repository, list_projects, delete_project, index_status
• Query (4): search_graph (structured search), trace_call_path (BFS, depth 1-5), query_graph (Cypher-like subset), ingest_traces
• Analysis (3): detect_changes (git diff impact analysis), get_graph_schema, get_architecture
• Code (3): get_code_snippet, search_code (BM25 + FTS5), manage_adr

The most practical is trace_call_path: change a function, see who is affected in one call at ~0.3ms.

Hybrid LSP: type resolution without language servers

Tree-Sitter ASTs don't resolve which module user.profile.display_name() actually refers to. Codebase-Memory compiles a lightweight C type-resolution engine (structurally compatible with tsserver, pyright, gopls, Roslyn, JDT, rust-analyzer) into the binary. No language-server processes, no per-project config, no API keys. Highlights include Python generics/Pydantic/SQLAlchemy 2.0, TS JSX component dispatch, C++ templates and macro handling, Rust trait bounds and UFCS, and Go interface satisfaction.

8-layer security audit

MCP servers are the most vulnerable link in LLM toolchains — users grant filesystem, process, and network access with little verification. Codebase-Memory's CI suite includes:

1. Static whitelist audit of dangerous libc calls (system, popen, fork, execvp) 2. Post-build binary string audit (only GitHub API + localhost URLs; no embedded credentials) 3. Network egress monitoring via strace on Linux 4. Installer sandbox path validation (blocks writes to ~/.ssh, ~/.gnupg, ~/.aws) 5. Hardened smoke tests 6. Graph UI audit (no external domains/trackers; HTTP binds 127.0.0.1 only) 7. MCP robustness tests: 23 adversarial JSON-RPC payloads (SQL injection, path traversal, ReDoS, etc.) 8. SHA-256 integrity checks on 72 vendored files (incl. 66 Tree-Sitter grammars)

Release gating adds VirusTotal (70+ engines, zero tolerance), SLSA L3 provenance via Sigstore cosign, CodeQL SAST, OpenSSF Scorecard, CycloneDX SBOM, and sanitizer soak tests.

Shareable graphs and benchmarks

Graph snapshots (.codebase-memory/graph.db.zst) compress 8-13:1 via zstd and can be committed to git — one person indexes, the whole team imports, skipping full re-indexing.

On 31 real repositories across 12 question types:

| Metric | MCP agent (graph) | Exploration agent (files) | Gap | |---|---|---|---| | Quality score | 0.83 | 0.92 | 90% | | Tool calls/question | 2.3 | 4.8 | 2.1x fewer | | Tokens/question | ~1,000 | ~10,000 | 10x fewer | | Query latency | <1ms | 10-30s | >100x faster |

Graph agents win on structural queries (hub detection, caller ranking, cross-file dependency chains) in 19/31; file agents win where full source context is needed (16/31). Weakest case: macro-heavy C (0.58 vs 1.00), since macros aren't in Tree-Sitter ASTs. Optimal architecture is hybrid.

Limitations

1. C macros, dynamic reflection, and runtime-generated code are absent from the graph 2. 83% vs 92% quality gap still requires fallback to file exploration 3. Single-repository only; multi-repo dependency tracking is future work 4. Runtime behavior and dynamic dispatch are not captured 5. Benchmarks all on Apple M3 Pro 6. Built-in semantic search (Nomic nomic-embed-code, 768d int8) depends on that model's quality

Conclusions

1. This is not a "code search tool" but code-structure infrastructure — index once, benefit forever. 2. The 3-minute kernel index is an engineering result of the RAM-first pipeline (LZ4, Aho-Corasick, parallel workers), not marketing. 3. The 120x token reduction comes from precomputation: pay the indexing cost once; queries become constant-time. 4. The single zero-dependency binary is the biggest engineering bet — language parsing, type inference, graph storage, and MCP protocol all in one C binary. 5. The 8-layer security audit responds to the MCP ecosystem's trust crisis and should become an industry standard. 6. Versus CodeGraph: same forest, different trees — pragmatic sufficiency vs. performance obsession. Choose based on team size and performance sensitivity.

> "In the era of AI coding, what developers need is not faster grep, but precomputed structure."

*Reference: Vogel M, et al. (2026). Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP. arXiv:2603.27277.*