When Code Is No Longer Code: Karpathy's Software 3.0 Revelations

Forum topic · 小凯 · 2026-05-03

Summary

A detailed Chinese-language forum post analyzing Andrej Karpathy's Software 3.0 framework as presented at Sequoia's AI Ascent 2026. The post traces the paradigm shifts from Software 1.0 (human-written rules) through Software 2.0 (trained neural networks) to Software 3.0, where humans program via prompts, context, tool use, and memory — "the context window is the new program." It explains why Karpathy says vibe coding only "raises the floor" while agentic engineering "raises the ceiling," highlights the December 2025 inflection point when AI-generated code became reliably trustworthy, and introduces key ideas: LLMs as jagged "ghosts" rather than animals, verifiability as the predictor of AI adoption speed, app categories collapsing into a single model call, the reversal of computing architecture where GPUs wrap CPUs, and why understanding, taste, and judgment remain un-outsourceable human responsibilities. The post includes Karpathy's hiring recommendations, the MenuGen payment bug example, and full reference links to source materials.

Key points

A widely-shared Chinese forum post on zhichai.net provides an in-depth breakdown of Andrej Karpathy's Software 3.0 talk at Sequoia AI Ascent 2026, including his admission: *"I have never felt this far behind."*

The three paradigm shifts

Software 1.0: Humans write explicit rules — if/else, loops, functions. Seventy years of classical programming.
Software 2.0 (Karpathy, 2017): Humans curate data and define objective functions; the program is "learned" into neural network weights. Example: Tesla's vision-based autonomous driving.
Software 3.0 (now, not a prediction): Humans program via prompts, context, tool use, examples, memory, and instructions. The LLM acts as an interpreter executing the context. Per Karpathy: *"The context window is the new program."* A natural-language Markdown install guide functionally replaces a brittle .sh script — the LLM is literally "a sophisticated English interpreter."

The December 2025 turning point

Karpathy described using Cursor, Claude Code, and Codex as "useful but needing frequent corrections" — until December 2025, during a break, when generated code suddenly became larger, more coherent, and reliable. Hence: *"I have never felt this far behind."*

Floor vs. ceiling

Vibe coding raises the floor: near-zero barrier to prototyping, but it does not ensure quality, security, maintainability, or taste. Fine for prototypes; dangerous for production.
Agentic engineering raises the ceiling: humans design specs, supervise plans, review diffs, write tests, build eval loops, manage permissions, isolate workspaces, and keep taste — while agents generate, code, test, and check each other. Core question: *how to coordinate fallible, stochastic agents for speed without sacrificing quality.*

The MenuGen payment bug illustrates this: an agent matched Stripe buyer emails to Google login emails — plausible code, wrong system design (persistent user IDs were the correct approach).

LLMs are ghosts, not animals

> "These are spiky entities. They are fallible and stochastic, but extremely powerful."

LLMs exhibit jagged intelligence — superhuman on some tasks, baffling failures on others. This demands human taste, judgment, and oversight.

Verifiability framework

AI adoption speed depends on how verifiable a domain's outputs are, not on importance:

Code (compiles, tests run) → fastest progress
Math proofs (checkable steps) → rapid progress
Creative writing, medical diagnosis → slow, cautious adoption

Advice for founders: seek valuable but overlooked high-verifiability domains, not the crowded code-generation赛道.

Other notable claims

Programming units changed: from lines of code to delegated "macro actions" — implement this feature, refactor this subsystem, research this library. Karpathy admitted to a self-described "AI psychosis" after 16-hour daily agent sessions.
App collapse: MenuGen shows some app categories collapsing into a single model call — no backend, database, or deployment needed.
Hiring: LeetCode-style puzzles are mismatched; instead, *"Build a substantial project with agents, deploy it, make it secure, and then have adversarial agents try to break it."* The "10x engineer" gap may become far more extreme.
Architecture inversion: GPUs will wrap CPUs; AI-native operating systems are already emerging (Copilot in Windows, Apple Intelligence).
Understanding is un-outsourceable: *"You can outsource your thinking, but never your understanding."* Humans must still grasp storage, views, memory copies, invariants, identity, security boundaries, and system shape.

Closing reflection

When intelligence becomes cheap, understanding becomes scarce. The post updates Feynman's maxim for the agent era: *"What I cannot verify, I do not understand."* The floor has been raised; the ceiling is being redefined.

References

Karpathy, Software 2.0 (2017) — https://karpathy.medium.com/software-2-0-a64152b37c35
AI Ascent 2026 full video — https://www.youtube.com/watch?v=96jN2OCOfLs
Karpathy tweet summary — https://x.com/karpathy/status/2049903821095354523
Stephanie Zhan tweet summary — https://x.com/stephzhan/status/2049518659513852109
Sequoia Inference analysis — https://inferencebysequoia.substack.com/p/andrej-karpathys-software-30-and
Karpathy Bear Blog full notes — https://karpathy.bearblog.dev/sequoia-ascent-2026/
Anthropic, Effective Harnesses for Long-Running Agents — https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents