Which Layer of the AI Funnel Are You On?

On October 11, Andrej Karpathy reposted an old piece from April, adding a new framing: the gap in how people perceive large language model capabilities is still growing — no longer two groups talking past each other, but an increasingly steep funnel.

The funnel has four layers. His numbers are off-the-cuff estimates — he admits he didn't check sources: roughly 6 billion people (75% of the world's population) have barely touched a large language model; 1 to 2 billion occasionally use free chat products; about 20 million use frontier models on code and math; and only around 5,000 people have access to lab-internal systems.

The numbers are rough, but the shape holds up. OpenAI said at this year's DevDay that ChatGPT has 1.2 billion weekly active users; Google says the Gemini app exceeds 1 billion monthly active users — the two overlap, so the second layer's 1–2 billion range is reasonable. Microsoft's earnings reports put GitHub Copilot at 50 million users, more than double the 20 million figure, but a large share of Copilot is just code completion — Karpathy himself says he no longer writes code with completion; OpenAI says Codex has 5 million weekly users, including over a million non-programmers. The 20 million magnitude stands.

The top 5,000 have no public numbers to verify. But that layer has recently leaked glimpses of light twice. Once when OpenAI released 722 math manuscripts — all generated by one unreleased model — posted to GitHub on October 6, with 3 retracted the next day over a symbol error and 14 revised. The other was OpenAI's Hugging Face incident, which months later outsiders are still piecing together. Karpathy's comment on both was a single sentence: human review and understanding are starting to fall behind.

1. The AI You Use Is Not What Engineers Use

What does AI look like to someone on layer 2 of the funnel? A webpage, a dialog box, one question, one answer. It's smart, but you ask, it replies, and the interaction ends there.

Engineers on layer 3 see a different species. In tools like Claude Code, Codex, and Cursor, the model doesn't sit in a chat box waiting for questions — it gets a task, then reads the codebase, edits files, runs tests, reads errors, and iterates on its own. Weeks of work might become one sentence of description plus a wait. The typical experience at this layer: you hand off a task before leaving work and come back the next day to review it.

At layer 4, labs run fleets of agents working on large projects for weeks at a time. Outsiders can only guess what that looks like from "leaked light" like the 722 manuscripts.

Same company, same brand name — behind each layer is a completely different system. The price lines are clear too: free, tens to hundreds of dollars per month, not for sale. The layering isn't just a perception problem; it's physical.

2. Why Code and Math Fell First

Karpathy's explanation for the funnel has three parts: big achievements need big problems; capability peaks appear in digital, verifiable, valuable work; and tiered access. The middle one is the key.

Reinforcement learning needs an automatic judge that can score results. Code has one: tests either pass or they don't, errors are errors, and nobody needs to be consulted about whether a line of code is right. Math is more extreme: a correct proof is correct, and formal systems can check it line by line. In these two domains, every step a model takes, a machine immediately tells it whether the step was right — exactly where reinforcement learning thrives.

Most chat-product tasks have no judge. Did it write a good apology letter? There's no test suite to run. Without a scorer, reinforcement learning can't spin up, and progress accrues only slowly through pretraining. So the same industry develops different time-flow rates: capability in code and math climbs a tier every few months, while chat experiences change far more slowly. People on layer 2 think AI is roughly what it is; people on layer 3 are already working alongside tools that do the work themselves — they aren't lying; they're seeing two different species.

3. The Experiment Where AI Made People 19% Slower

In July 2025, evaluation firm METR ran a randomized controlled experiment, and the results surprised many.

The study recruited 16 experienced open-source developers working on real issues in repositories they had maintained for years — 246 tasks total. Half the tasks allowed AI tools (the then-latest models plus products like Cursor), half forbade them, with screens recorded and time tracked to the second.

The developers predicted AI would make them 20% faster. Measured result: the group allowed AI tools finished 19% slower.

Where did the time go? Three places: writing prompts, waiting for generation, and the most expensive — reviewing the AI's code. How well did these developers know their own repos? Many changes they could make eyes-closed; verifying AI-generated solutions cost extra time. Their gut said faster; the stopwatch said slower.

This experiment is often read as "AI coding is useless" — the opposite of what it shows. Its real lessons are two. First, scope: these were senior developers in the repos they know best; on unfamiliar codebases, boilerplate, or brand-new projects, AI's benefits are a different matter. Second, and more important: your gut is unreliable. Even this group of top professionals misjudged their own efficiency by 39 percentage points.

4. Calculate How Much AI Actually Saves You

You can run METR's method at a smaller scale yourself — an afternoon is enough.

Step 1, gather tasks. Pick 10–20 similar tasks from your real work, similar difficulty — e.g., all small features, or all fixes of one bug class.

Step 2, alternate groups. Don't do all-AI in the morning and all-manual in the afternoon — you get more skilled as you go, so the afternoon is naturally faster. Randomly split the tasks and shuffle the order.

Step 3, stopwatch from task start to "I'm willing to own this." That phrase is key: AI finishing isn't finishing — it counts only after you've read it, revised it, and the tests pass. METR's 19% largely hides in the "read and revise" part.

Step 4, log rework. How many places did you edit the AI's code? How many times did it fail?

Step 5, compute the ratio: total time without AI divided by total time with AI. Above 1, AI is helping you; below 1, you're helping AI. Compute per task type — boilerplate, debugging, refactoring, unfamiliar domains — the ratios usually differ a lot.

This method won't produce paper-grade conclusions, but it produces your own numbers — far more reliable than "feels much faster." That's exactly how those senior developers felt too.

More and more light is leaking from the top of the funnel, and not enough human reviewers can keep up. The steepest part of Karpathy's funnel isn't between the 6 billion and the 5,000 — it's the distance between the mental picture each layer has of the layer above it and the real thing.

Which layer are you on? Tell us in the comments.