The biggest challenge for intelligent tutoring systems isn't the algorithms — it's that students don't use them. Many students sign up with enthusiasm in week one, start disappearing in week two, and never return in week three. If we could know in advance "roughly how many minutes will this student use next week, and how many skill points will they master," tutors and systems could intervene before students give up.
Qiu, Thomas, Guo, Aleven, and Borchers (again from CMU LearnLab, plus Conrad Borchers — the same author as the earlier response-time paper) frame this "engagement prediction" task as supervised learning over ITS logs: predicting two core metrics — weekly practice minutes and weekly newly mastered skills. The data comes from interaction logs of 425 middle school students over an entire school year.
Fifteen predictive models (from linear regression to neural networks) were compared against heuristic baselines. The baseline approach is common and crude — "take the percentile from previous weeks and use it directly as the prediction" — which is the default practice in many educational applications. Result: feature-based models achieved 22–33% lower mean absolute error than the heuristics. The heuristic systematically overestimated — it assumes student effort is stable, but in reality online learning effort patterns look like a roller coaster.
Feature analysis revealed interesting differences: effort prediction relies mainly on recent activity features — if a student studied 60 minutes last week, this week will probably be similar. But progress prediction relies on learner state and content difficulty signals — a student stuck on a difficult knowledge point may master zero new skills even if they spend a lot of time.
The authors also conducted a small qualitative validation: after viewing the system-generated prediction features, 8 university tutors reasoned differently about effort targets versus progress targets — consistent with the patterns in the quantitative analysis. Tutors used "last week's activity" to infer effort, and "where they're stuck" to infer progress.
What's unclear: the prediction horizon is week-by-week — finer granularity (daily, or even per session) might be more valuable for interventions. The 425-student sample may already be diluted in weekly predictions — some students may have only a few weeks of valid data. The tutor interviews involved only 8 people — whether the patterns generalize needs confirmation with larger samples.
---
References
1. Qiu, E. S., Thomas, D. R., Guo, B., Aleven, V., & Borchers, C. (2026). *From Heuristics to Analytics: Forecasting Effort and Progress in Online Learning*. arXiv:2605.12788 [cs.LG].
2. Baker, R. S., & Inventado, P. S. (2014). *Educational Data Mining and Learning Analytics*. Learning Analytics.
3. Aleven, V., et al. (2022). *Example-Tracing Tutors: Intelligent Tutor Development for Non-Programmers*. International Journal of AI in Education.