The biggest challenge for intelligent tutoring systems (ITS) is not the algorithms — it's that students don't use them. Many students start their first week with enthusiasm, fade in the second week, and disappear entirely by the third. If the system could know in advance "roughly how many minutes will this student practice next week, and how many skill points will they master," tutors and the system could intervene before a student gives up.
Qiu, Thomas, Guo, Aleven, and Borchers (again from CMU LearnLab, plus Conrad Borchers — the same author as the earlier response-time paper) framed this "engagement prediction" task as supervised learning on ITS logs: predicting two core metrics — weekly practice minutes and weekly newly mastered skills. The data come from interaction logs of 425 middle school students over an entire school year.
Fifteen predictive models (from linear regression to neural networks) were compared against heuristic baselines. The baseline approach is common and crude — "take the percentile from previous weeks and use it directly as the prediction" — a default in many educational applications. Result: feature-based models achieved 22–33% lower mean absolute error than the heuristics. The heuristic systematically overestimates — it assumes student effort is stable, but in reality effort in online learning behaves like a roller coaster.
Feature analysis revealed interesting differences: effort prediction relies mainly on recent activity features — if a student studied 60 minutes last week, this week will likely be similar. But progress prediction depends on learner-state and content-difficulty signals — a student stuck on a difficult knowledge point may master zero skills even after spending a lot of time.
They also ran a small qualitative validation: after seeing the system-generated predictive features, 8 university tutors reasoned about effort targets and progress targets differently — consistent with the patterns in the quantitative analysis. Tutors used "last week's activity" to infer effort, and "where the student is stuck" to infer progress.
What remains unclear: the prediction horizon is week-by-week — finer granularity (daily or even per-session) might be more valuable for interventions. The 425-person sample may be diluted in weekly prediction — some students may have only a few weeks of valid data. The tutor interviews involved only 8 people — whether the patterns generalize requires confirmation with a larger sample.
---
References
1. Qiu, E. S., Thomas, D. R., Guo, B., Aleven, V., & Borchers, C. (2026). *From Heuristics to Analytics: Forecasting Effort and Progress in Online Learning*. arXiv:2605.12788 [cs.LG].
2. Baker, R. S., & Inventado, P. S. (2014). *Educational Data Mining and Learning Analytics*. Learning Analytics.
3. Aleven, V., et al. (2022). *Example-Tracing Tutors: Intelligent Tutor Development for Non-Programmers*. International Journal of AI in Education.