Sherpa: Teaching LLMs to Teach Adaptively via Multi-Turn RL

Forum topic · 小凯 · 2026-10-08

Summary

Sherpa is a multi-turn reinforcement learning framework that trains large language models to become adaptive teachers rather than mere problem solvers. Developed by researchers including Weixian Xu and Diyi Yang (arXiv:2610.08778), Sherpa instantiates multiple LLM-based student archetypes conditioned on distinct learning preferences and trains a teacher model to adjust its instruction by directly maximizing these simulated students' learning outcomes. Unlike prior approaches that rely on demonstrations, preference data, or predefined pedagogical criteria, Sherpa grounds teacher training in individual student results, which vary substantially across learners. In evaluations, Sherpa-trained teacher LLMs improved performance across all prototype students by an average of 20.5 percentage points and raised the overall MathTutorBench teaching score from 52.5% to 79.2%. Human studies showed the trained teacher was preferred over the base model in 79.6% of pairwise comparisons, paving the way for AI tutors that adapt to real students.

Sherpa: Teaching LLMs to Teach Adaptively

Field: NLP Authors: Weixian Xu, Yanzhe Zhang, Zora Zhiruo Wang, Changyu Chen, Diyi Yang Published: 2026-10-06 arXiv: 2610.08778

Abstract

Large language models (LLMs) have become increasingly capable problem solvers, but being able to solve a problem is not the same as being able to teach it. Existing approaches to training LLMs as teachers rely on demonstrations, preference data, or predefined pedagogical criteria that specify what good teaching looks like. However, these signals are often not grounded in individual student learning outcomes, where effective teaching strategies can vary substantially across learners.

To address this, the authors introduce Sherpa, a multi-turn reinforcement learning framework that instantiates multiple student archetypes with LLMs conditioned on distinct learning preferences and trains a teacher model to adapt its instruction by directly maximizing their learning outcomes.

Results

Teacher LLMs trained with Sherpa improved the performance of all prototype students by an average of 20.5 percentage points.
On MathTutorBench evaluation, Sherpa raised the overall teaching score from 52.5% to 79.2%.
Human studies found the trained teacher was preferred over the base model in 79.6% of pairwise comparisons.

Sherpa trains LLM teachers to adapt to diverse simulated students, paving the way for AI tutors that can teach real students effectively.

--- *Source: zhichai.net forum post, auto-collected 2026-10-08.*