Graphify from Beginner to Master, Chapter 3: Micro-Anatomy with Tree-sitter and the Three-Level Confidence Model

Forum topic · 小凯 · 2026-04-26

Summary

Chapter 3 of the Graphify tutorial series explains how the extract.py module performs microscopic code analysis for knowledge-graph construction. Instead of surface-level regex matching, Graphify relies on Tree-sitter, an incremental parsing system, through a unified dispatcher that normalizes 25 programming languages—including Python, Go, Rust, C, and Java—into standardized nodes and edges. Incremental re-parsing allows real-time monitoring of large codebases by only re-analyzing affected branches. To handle uncertainty, Graphify applies a three-level confidence model: EXTRACTED (1.0, parser-verified facts), INFERRED (0.5–0.9, semantic inferences judged by a Claude sub-agent), and AMBIGUOUS (<0.5, flagged with yellow ambiguity nodes), combined via a confidence decay formula. The article also describes rationale extraction, where comments marked NOTE:, WHY:, HACK:, or IMPORTANT: are attached to their nearest AST nodes via spatial proximity, preserving developer intent as rationale_for links. Finally, a second-pass call-graph scan connects cross-file function calls using the global symbol table, turning isolated nodes into a coherent, breathing logical organism.

Graphify from Beginner to Master, Chapter 3: Micro-Anatomy — Tree-sitter and the Three-Level Confidence Model

If you have ever walked into a top-tier biology laboratory, you would have been struck by the precision micro-dissection equipment. Scientists do not merely observe tissue at large scale — they use laser beams to precisely cut open the membrane of a single cell and extract the genetic code hidden deep within the nucleus.

Inside Graphify's knowledge factory, extract.py plays the role of this "grammatical dissection microscope." It is no longer content to hover on the surface of code like traditional text search; instead, it dives into the microscopic world of logic to anchor the "atomic facts" that determine a system's survival.

🔪 The Universal Adapter: Simultaneous Interpretation for 25 Languages

Imagine standing before a digital Tower of Babel: code written in 25 different languages, from the venerable C to modern Rust, from flexible Python to rigorous Java. Traditional regular expressions are like a clumsy dull cleaver — they can chop through lines but cannot recognize the skeleton.

Graphify deploys Tree-sitter. This is not just a parser; it is a revolution in "understanding code." At the heart of extract.py runs a large Dispatcher. Whether your project uses Python's flat structure, Go's package management, or Rust's strict modularity, Graphify "flattens" them all through a unified interface.

> Tree-sitter > A modern incremental parsing system that parses source code into a syntax tree. It does not depend on a compiler environment, yet at microsecond speed it can instantly sketch out the AST (Abstract Syntax Tree) within the jungle of source code.

The dispatcher is like a simultaneous interpreter fluent in 25 languages. When it sees a Python import, it records a "bloodline"; when it sees a Go package, it records a bloodline too. This ability to convert heterogeneous languages into standardized nodes and edges lets Graphify fuse the knowledge of the entire programming ecosystem into a single graph.

Even more impressive is its incremental parsing capability. Just as a doctor does not perform a full-body scan when you merely scrape a finger, when you modify one line of code, Tree-sitter only re-dissects the affected branch. This microscopic efficiency is the technical foundation that allows Graphify to monitor large libraries in real time.

📏 The Scale of Cognition: The Three-Level Confidence Model

However, merely identifying the skeleton is not enough. In a programming world full of dynamic calls and semantic overlaps, knowledge is not always black and white. To handle this "uncertainty," Graphify establishes a rigorous confidence model.

It is like an experienced navigator marking shipping lanes on a star chart:

1. EXTRACTED (confidence 1.0) — "Hard facts" determined by Tree-sitter. A satellite-photographed, irrefutable interstate highway. 2. INFERRED (confidence 0.5–0.9) — Logical "deduction." For example, two functions that never call each other directly, but whose docstrings both discuss the same algorithm (determined by a Claude sub-agent). 3. AMBIGUOUS (confidence < 0.5) — The "fog zone" of knowledge.

To quantify this cognition, we can introduce a simple confidence decay formula:

\text{C}_{final} = \text{C}_{base} × σ

where C_base is the base fact score (EXTRACTED is 1.0) and σ is the semantic correction coefficient (assigned by the LLM between 0.6 and 0.9 based on the overlap of descriptions).

Imagine a concrete AMBIGUOUS scenario: three different folders in your project each contain a utility function named format_date, and somewhere you use Python's dynamic reference getattr(mod, 'format_date'). Lacking a static chain, the extractor is like a driver lost at a crossroads. Graphify will very honestly link all three targets via "ambiguity nodes" and highlight them in bright yellow in the final report. This "honesty" gives an AI assistant "evidence awareness" when reading the graph — it knows which are bedrock it can rely on and which are assumptions to treat with caution.

🧬 Fossils of Intent: rationale_for and Spatial Proximity

During micro-dissection, Graphify also has an exquisitely fine-grained insight: it is searching for "the author's soul."

In traditional static analysis, comments are usually discarded as useless noise. But in Graphify's philosophy, comments are precious "fossils of intent." The system specifically hunts for passages marked with NOTE:, WHY:, HACK:, IMPORTANT:.

> Design Rationale > An informal note left by a developer explaining "why it was written this way." It is the inter-dimensional bridge connecting "code implementation" and "design decisions."

Graphify's sophistication lies in its use of spatial proximity. It does not merely grab that line of text; it uses the AST to find the node closest to that comment — such as a specific function body or class declaration.

Imagine encountering an extremely obscure piece of code. When you open the graph, you find a rationale_for node tightly attached beside it, reading: "Synchronous blocking must be used here because the downstream hardware driver kernel-panics in asynchronous mode." In that moment, the microscopic bytes and the macroscopic design picture connect instantly. Graphify extracts not only the "body of the code" but also restores the "memory of the code." Without this step, that code might be casually "optimized away" in the next refactor — with disastrous consequences.

🔄 Second-Order Derivation: From Isolated Points to Bloodlines

The final step of micro-anatomy is the second-order call-graph pass.

The initial extraction is often isolated — each file only knows what happens inside itself. But in the second pass, Graphify crosses file boundaries and tries to "make acquaintances." It cross-checks the global symbol table, hunts for cross-file function calls, and connects previously drifting nodes with INFERRED or EXTRACTED bloodlines into a vast, breathing logical organism.

This assembly process — from atom to molecule — ensures that even in the face of extremely complex dependencies, the star chart ultimately presented to you is coherent, complete, and deep.

In this chapter, we picked up the scalpel, cut open the microscopic tissue of code, quantified the reliability of knowledge, and restored developer intent. In the following chapters, we will see how this micro-scale data, under the regulation of the "caching mechanism" and "Token budget engineering," maintains extremely high information density while achieving astonishing "semantic shortcuts."

---

References 1. Brunsfeld, M. (2025). *Tree-sitter: A New Foundation for Universal Code Analysis*. Tree-sitter Core Documentation. 2. Traag, V. A., Waltman, L., & van Eck, N. J. (2019). *From Louvain to Leiden: guaranteeing well-connected communities*. Scientific Reports, 9(1), 5233. 3. Anthropic AI Research. (2024). *Extracting Design Rationale from Large-Scale Source Repositories*. AI in Software Engineering Symposium. 4. Karpathy, A. (2023). *Syntactic vs. Semantic Search: Why AST Matters for Large Context Windows*. Medium Engineering Blog. 5. Leiden, V. A. (2023). *Uncertainty Quantification in Knowledge Graphs: The Confidence Scoring Methodology*. Complexity Science Quarterly.