Adaptive learning systems face a fundamental measurement problem: how do you quantify how much effort a student actually puts in? Total time is unreliable—five minutes on a hard problem might mean deep thinking, or it might mean the student zoned out. Accuracy doesn't work either—a correct answer could be a lucky guess, and a wrong answer doesn't necessarily mean the student didn't try.
Borchers, Zhang, Yang, Nagashima, and Domingue (from Stanford's Education Data Science lab and learning engineering teams) used a more refined signal: the response time between each step of problem solving. They analyzed real logs from eight classroom deployments—2020 to 2023, six US schools, 794 students solving problems in an algebra tutoring system—and used hierarchical models to estimate each student's "response-time propensity," then examined its relationship with learning efficiency.
The key finding is not "faster is better" or "slower is better"—it depends on the student.
- For high-proficiency students, slower response times predict higher learning efficiency. They are engaged in constructive processing, genuinely thinking at each step rather than skipping by intuition. For them, being slow is a good sign.
- For low-proficiency students, slow response times show a weak or even negative relationship with learning efficiency. They aren't thinking—they are struggling unproductively or drifting off. "Slow" signals wasted effort.
- The study uses algebra tutoring data—do response-time patterns behave the same in programming education or CS algorithm practice?
- Is "slow" for high-proficiency students genuine deep processing, or perfectionist over-checking?
- The paper uses a continuous propensity score, but real deployments need an explicit decision boundary for the slow/fast threshold.
There is also a timing-window effect: the correlation between response time and learning efficiency is strongest early in a practice sequence and decays as practice continues. This means the best moment to detect low engagement is the early phase of each new practice set—if a student is slow right at the start, that's a red flag; if they slow down midway, they may simply be entering deeper processing.
Open questions:
References
1. Borchers, C., Zhang, L., Yang, K., Nagashima, T., & Domingue, B. W. (2026). *Understanding Student Effort Using Response-Time Propensities During Problem Solving*. arXiv:2605.08943 [cs.CY]. 2. Aleven, V., et al. (2017). *Instruction Based on Adaptive Learning Technologies*. Handbook of Research on Learning and Instruction. 3. Baker, R. S., et al. (2008). *Developing More Generalizable Detectors of Student Gaming the System*. User Modeling 2008.