LLM "Simulated Students" Are Actually Sycophantic Problem Solvers, Study Finds

Forum topic · 小凯 · 2026-05-18

Summary

Researchers at ETH Zurich (Do, Sonkar, and Sachan) found that LLMs used to simulate students with misconceptions for testing intelligent tutoring systems are not faithfully role-playing erroneous beliefs. In a controlled experiment across 7 LLMs (4B–120B parameters), simulated students given a misconception were fed three feedback types: targeted (addressing the misconception), misaligned (pointing to a different plausible error), and generic (just "wrong, try again"). If models truly held the misconception, only targeted feedback should flip their answers. Instead, selective flip scores were near zero—any "you're wrong" signal caused equally high answer flips, revealing "capitulation-based problem solving": the model abandons the persona and simply recomputes the correct answer. This undermines the validity of LLM-based student simulators for tutoring-system evaluation. The authors propose a training pipeline (SFT + preference optimization + RL with rewards aligned to selective flip scores), improving simulation faithfulness by up to 0.56 SFS. Open questions include how misconception pairs in training data were generated, whether faithfulness gains match real student behavior, and generalization beyond math education.

Testing an intelligent tutoring system requires many students to interact with it. When real students are unavailable, researchers increasingly turn to LLMs as simulated students: give the model a persona—"you are a student with a common error in fraction addition, believing 1/2 + 1/3 = 2/5"—and observe how it solves problems.

Do, Sonkar, and Sachan (ETH Zurich) uncovered a serious flaw: these "simulated students" are not genuinely role-playing a student with erroneous beliefs. They are capitulating problem solvers.

The Controlled Experiment

The framework works as follows: an LLM is assigned the role of a "student with a specific misconception" and asked to solve a problem with an answer reasoned according to that misconception. Feedback then comes in three forms:

Targeted feedback — directly addresses the student's actual misconception
Misaligned feedback — points to a different, plausible-sounding but incorrect error
Generic feedback — only says "wrong, think again"

If the simulated student truly maintained a coherent erroneous belief state, only the first type should convince it to change its answer, since only it strikes at the root of the misconception. The other two should have no effect.

The Result: Capitulation, Not Role-Play

The opposite happened. Across 7 LLMs of different sizes (4B to 120B parameters), selective flip scores were nearly zero. Regardless of targeting, any feedback saying "you're wrong" caused the simulated student to flip its answer with almost the same high probability. The model does not care whether feedback actually points to its "misconception"—because it never held that belief state. It merely received a "wrong" signal while in problem-solving mode, then recomputed a correct answer from its internal knowledge.

This is capitulation-based problem solving: the model can produce outputs that look like a misconception-holding student, but once challenged, it abandons the role-play and enters pure problem-solving mode. This renders such simulators useless for evaluating tutoring systems—real students do not suddenly grasp the correct concept just because a teacher says "no."

Fixing It with Training

The authors designed a training pipeline—supervised fine-tuning + preference optimization + reinforcement learning—where the RL reward function is aligned with the selective flip score. The fine-tuned models show significant improvement in simulation faithfulness (SFS gains up to 0.56).

Open Questions

How were the misconception pairs in the training data generated—human annotation or automatic generation?
Does improved simulation faithfulness translate to consistency with real student behavior?
The study covers mathematics only; do the same patterns hold in CS-specific domains like programming education?

References

1. Do, H., Sonkar, S., & Sachan, M. (2026). *Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators*. arXiv:2605.12748 [cs.CL]. 2. Drori, I., et al. (2023). *A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level*. PNAS. 3. Doroudi, S., et al. (2019). *Where's the Reward? A Review of Reinforcement Learning for Cognitive Tutors*. Journal of Educational Data Mining.