Two Cents Tutoring: StudentBench and the 918x Price Gap
It's 1 a.m., the desk lamp still on. Someone in your GRE prep group just posted an ad: a top-agency math tutor, $75 an hour, ten-session minimum, next opening Wednesday. Meanwhile, a free chat window sits in the corner of your screen — a "teacher" who's always available, charges nothing, and answers a combinatorics question in three seconds with three different solutions.
Which one would you choose? A paper posted to arXiv in September 2026 — StudentBench, first-authored by Curtis Northcutt (MIT-trained, founder of cleanlab, known for data-quality research) — moved this question from intuition into the lab: it put expert human tutors and AI tutors into the same randomized controlled experiment, with the same tests, the same students, and the same measuring stick.
The result in one sentence: AI tutoring achieved statistically equivalent GRE learning gains compared to expert human tutors. And comparing cost per point of improvement: human tutoring cost $4.81, AI cost $0.0052 — a 918x price gap.
The Tutoring Market: An Old Craft Workshop
GRE tutoring is priced on scarcity. A good tutor's hours cannot be replicated; the $75 hourly rate sells not just a lesson but possession of a scarce slice of human time. Its other face is filtering: whoever can afford $750 for ten sessions stays in the classroom; everyone else goes back to free forum threads.
In the past few years, many test-takers have quietly already been using ChatGPT as a free tutor. Khan Academy embedded AI tutoring into its curriculum with Khanmigo. The real question shifted from "can AI teach" to "does AI teach as well" — a question intuition and word-of-mouth cannot answer.
The 2,383-Person Controlled Experiment
Randomized controlled trials are nearly a luxury in education research: ethical review blocks randomizing students into "worse" conditions, costs rival small clinical trials, and attrition undermines conclusions. Education research has relied on observational data, where student quality and teaching quality are hopelessly tangled.
StudentBench built a platform that sidesteps all three obstacles: free and open, students randomized into three groups upon registration, online delivery removing geographic limits, and every message automatically logged.
- AI tutor group: students learned with an LLM-driven tutor — over 175,000 student–AI dialogue messages recorded verbatim.
- Human tutor group: expert GRE tutors providing one-on-one online tutoring on the same material.
- Control group: self-study, no tutoring.
- Northcutt, C., Hasmani, I., Feng, K., Khangi, T., Plesner, A., & Mueller, J. (2026). *StudentBench: AI and human tutoring yield equivalent GRE learning gains*. arXiv:2609.28470. https://arxiv.org/abs/2609.28470
- Platform: https://studentbench.org
Total sample: 2,383 real students, measured via pretest → learning → posttest learning gains.
Two layers of results:
Overall: AI tutoring and human tutoring produced statistically equivalent learning gains — p = .015. This is not a classic significance test asking "is there enough evidence of a difference?" — an absence of evidence of difference never proves sameness. This paper used an equivalence test (TOST): it first defines how close counts as "close enough," then tests whether the difference falls within that margin. The p = .015 means the equivalence claim itself holds.
By domain: across seven GRE areas — quantitative arithmetic, algebra, geometry, data analysis; verbal text completion, reading comprehension, sentence equivalence — the best-performing AI tutor exceeded the human tutors' average in 5 of 7 domains. Not a crush everywhere; AI won through having no weak spots, while human tutors showed distinctive strengths and relative weak areas.
Decomposing Teaching into Five Processes
A second study had expert human tutors act as blind judges in 2,028 pairwise rubric evaluations, comparing AI-generated and human-written lessons and practice problems. Teaching was decomposed into five dimensions:
1. Lesson planning — structuring a lesson around Vygotsky's zone of proximal development and Bruner's scaffolding: a staircase whose steps are low enough not to trip you, steep enough to ascend. 2. Practice-problem creation — difficulty gradients and distractors that diagnose specific misconceptions. 3. Conversational pedagogy — guiding, probing, offering ladders rather than answers (Socratic questioning). 4. Cost — computed, not judged. 5. Engagement — whether students keep going.
No single AI led in all five; different AI tutors won different dimensions, and the overall composite matched humans. AI didn't win with one spectacular skill — it won by having no obviously lagging skill. Bad news for craft workshops whose identity rests on one specialty.
The 918x Gap
Cost per one-percentage-point improvement: human tutor $4.81, AI tutor $0.0052. Ratio ≈ 918.
Education has promised "democratization" at least three times. Printing made books cheap — but early buyers were those who could already afford them. MOOCs around 2012 promised world-class courses for all; completion rates exposed the flaw — one-way broadcast content has no one to catch you when you slip. Both revolutions made content cheap, never interaction. Content was never education's scarcest ingredient; "someone watching you, responding to you, waiting where you get stuck" was. AI tutoring is the first thing to make interaction cheap too. One AI tutor achieved all this while keeping learning gains statistically equivalent to human tutors (p = .044). It didn't cut corners on effectiveness — it's just cheap.
As Northcutt posted on social media: "One hour with an expert GRE tutor: $75." No commentary followed. Sometimes numbers speak for themselves.
Equivalence Is Not Identity: Two Marksmen
Statistical equivalence is the source of all misreadings. Imagine two marksmen averaging a 9-ring hit: one's shots cluster in a coin-sized area around the 9-ring, the other's scatter between 7 and 10. Same average, wildly different distributions. Equivalence testing says the averages coincide within a confidence interval — not that every student's experience is the same.
Four limitations must be stated:
1. Only GRE, only short-term. Gains measured immediately after learning. Long-term retention? Not tested. 2. Human tutors were "experimental tutors" in an online setting — not three-year teacher–student relationships with memory of your past mistakes and pre-exam anxiety. 3. 5/7 domains ≠ comprehensive superiority. AI still trailed in two domains. "AI education comprehensively surpasses human teachers" is rumor; "equivalent GRE learning gains" is the actual claim. 4. Most effective ≠ best experience. Encouragement, trust, someone genuinely caring about your future — none of that is in the rubric.
Accurate conclusion: in the specific scenario of GRE prep and the specific metric of short-term gains, the best AI tutor already stands in the same confidence interval as human experts — while on price, it stands in another galaxy.
Faster Teachers, Better Teachers
A finding buried in the data analysis may be the most durable: in quantitative sessions, faster AI replies → more student messages → more correct practice problems → larger learning gains, with each link significant at p < .002. It sounds less like AI research than table-tennis common sense: the faster the feeder, the more rallies, the quicker the touch.
A $75 human session spends much time waiting — for you to understand, compute, formulate questions. AI has no waiting cost; response-latency compression converts directly into dialogue rounds, and dialogue rounds into practice volume. The old wisdom "practice makes perfect" gets re-engineered: students aren't more diligent — the cost of diligence approaches zero. (This is correlational, not causal.)
What the Paper Doesn't Say
The paper's honesty is in its title: GRE learning gains. No "revolutionizes education," no "end of teachers."
Fair skepticism first: publication bias favors "AI crushes" or "AI fails" headlines; "equivalent, but don't get excited" is the least newsworthy and most weighty result type.
Sharper layer: Goodhart's law — once a measure becomes a target, it stops being a good measure. If AI tutors optimize for GRE scores and every student has one, the "ability" content of scores dilutes; the exam becomes a human-machine test-taking exercise. How much discrimination does that hard threshold retain?
The paper's real contribution may be the StudentBench platform itself: open infrastructure that continuously collects large-scale tutoring dialogue data and runs randomized comparisons, turning "AI vs human tutoring" from a one-off experiment into a repeatable public question.
Questions it cannot answer: when a student can't learn even with AI, do they need a better algorithm or a real person? When everyone can afford a master teacher, does competition shift from "who has a good teacher" to "who wants to learn more"? And which parts of a $75/hour master's craft cannot be replicated at $0.0052?
Aftertaste: When Scarcity Changes
The cruelty of GRE prep used to be that good teachers were rare and expensive, so "meeting a good teacher" was a function of luck and class. This paper doesn't prove AI is a good teacher — it proves that on a rigorously measured task, the best AI now stands in the same confidence interval as the best humans.
The 918x gap means the midnight hesitation is disappearing. Industrialization didn't eliminate craft; it eliminated "only a few can afford it." Master teachers likely won't disappear — like handmade leather shoes after the industrial age, they'll become luxury goods. The truly scarce resource will shift from "good teachers" to something else:
People who want to learn.
That's the story's quietest turn: centuries of promised educational democratization may be completed not by a grand institution, but by a stream of three-second replies.
---
References