Karpathy's LLM Knowledge Base: A Compiled Wiki Pattern Beyond Traditional RAG

Overview

In April 2026, Andrej Karpathy published a GitHub Gist titled LLM Knowledge Bases — not a tweet or talk, but a *design document* meant to be pasted into an LLM agent (Claude Code, Codex, OpenCode, etc.) so the agent can help build a personal knowledge base.

The thesis in one line:

> "Traditional RAG re-discovers knowledge on every query; the LLM Wiki lets knowledge grow on its own."

This article is a structured walkthrough of the original Gist, covering the three-layer architecture, the three core operations, optional tooling, and the historical lineage to Vannevar Bush's 1945 Memex.

---

Key points

Why traditional RAG falls short

• No accumulation: every query rebuilds context from scratch.
• No synthesis: complex questions require live stitching of multiple documents.
• No evolution: knowledge does not grow richer or more connected over time.
• NotebookLM, ChatGPT file uploads, and most RAG systems share these limitations.

Karpathy's shift: from retrieval to compilation

• Pattern: raw sources → LLM compiles → persistent Wiki → queries read compiled knowledge.
• When new material arrives, the LLM reads it, extracts key information, updates entity pages, revises topic summaries, flags conflicts with prior data, and reinforces or challenges evolving syntheses.
• Knowledge is compiled once and kept current, not re-derived per query.

Three-layer architecture

• Layer 1 — Raw Sources: a curated, *immutable* collection (articles, papers, images, data files). The LLM reads but never modifies them; this is the source of truth.
• Layer 2 — The Wiki: an LLM-generated directory of markdown files — summaries, entity pages, concept pages, comparisons, syntheses. The LLM fully owns creation, updates, cross-references, and consistency. Humans *read* it; the LLM *writes* it.
• Layer 3 — The Schema: configuration files (CLAUDE.md for Claude Code, AGENTS.md for Codex) that define wiki structure, naming conventions, and workflow. They turn a generic chatbot into a disciplined wiki maintainer, co-evolved with the user.

Three core operations

1. Ingest
• Drop new material into raw/.
• LLM reads the source, discusses key points, writes a summary page, updates the index, refreshes related entity and concept pages (potentially 10–15 pages), and appends to the log.
• Karpathy's personal style: ingest one source at a time and stay engaged — read the summary, check updates, steer emphasis.
2. Query
• LLM searches relevant pages, reads them, and synthesizes an answer with citations.
• Output formats vary by question: markdown page, comparison table, Marp slide deck, matplotlib chart, canvas.
• A key insight: good answers can be archived back into the Wiki as new pages. Ingest compounds, and exploration compounds.
3. Lint (periodic health check)
• Detect contradictions between pages.
• Flag outdated statements superseded by newer sources.
• Identify orphan pages with no inbound links, missing pages for mentioned concepts, and missing cross-references.
• Surface data gaps that web search could close.
• LLMs are well-suited to suggest new questions and new sources to investigate.

Two special navigation files

• index.md — a content-oriented table of contents listing every page with a one-line summary and optional metadata (date, source count), grouped by category. Updated automatically after each ingest. Scales well to roughly 100 sources and several hundred pages without embedding-based RAG infrastructure.
• log.md — an append-only timeline of ingests, queries, and lint passes. Format: ## [YYYY-MM-DD] ingest | Article Title. Unix-friendly, e.g. grep "^## \[" log.md | tail -5.

Optional toolchain

• Search — qmd: a local markdown search engine using hybrid BM25/vector retrieval with LLM re-ranking. CLI and MCP server; runs entirely on-device.
• Web clipping — Obsidian Web Clipper: browser extension that converts articles to markdown.
• Images: configure a fixed attachment folder (e.g. raw/assets/) and bind a hotkey to download attachments; LLMs can then view and cite local images. Caveat: LLMs cannot natively read inline images in markdown, so read text first, then load referenced images separately.
• Visualization — Obsidian Graph View: reveals hubs, orphans, and overall structure.
• Slides — Marp: markdown-based slide decks, with an Obsidian plugin for direct generation from wiki content.
• Query — Dataview: Obsidian plugin that runs queries over YAML frontmatter so LLMs can tag pages with metadata.

Why this pattern works

• The tedious part of maintaining a knowledge base is not reading or thinking — it is bookkeeping.
• LLMs do not forget cross-references, do not tire of updating summaries, can flag conflicts in a single pass across many files, and keep multi-page consistency cheap.
• Net effect: maintenance cost approaches zero, so the wiki actually stays maintained.
• The human role shifts to curating sources, steering analysis, asking good questions, and reflecting on meaning. Bookkeeping moves to the LLM.

Historical echo — Vannevar Bush's Memex (1945)

• Bush envisioned a private, curated knowledge store with associative trails between documents, where connections are as valuable as the documents themselves.
• The public internet instead became passive, public consumption.
• Bush never solved who does the maintenance; LLMs finally do.

Use cases

• Personal: goals, health, psychology, self-improvement; journals, articles, podcast notes.
• Research: weeks-to-months deep dives that build an evolving argumentative synthesis.
• Reading: per-chapter archives that build character, theme, and plot pages — analogous to community fan wikis like Tolkien Gateway, but built personally while reading.
• Business / team: internal wikis fed by Slack threads, meeting notes, project docs, customer calls, optionally human-reviewed.
• Other: competitive analysis, due diligence, travel planning, course notes, hobby deep dives.

Implementation guidance from Karpathy

• The Gist is intentionally abstract: it describes ideas, not a specific implementation.
• Directory structure, schema specs, page formats, and toolchain are variable and depend on the user's domain, preferences, and chosen LLM.
• Everything is optional and modular — pick what is useful, ignore the rest.
• The intended workflow is to share the document with the user's LLM agent and instantiate a version that fits.

Reference

• Original Gist: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
• Title: *LLM Knowledge Bases*
• Subtitle: *A pattern for building personal knowledge bases using LLMs*
• Target agents: OpenAI Codex, Claude Code, OpenCode / Pi, and similar