KITE: A Socratic RAG-Based AI Tutor That Guides Students Through Algorithm Debugging Instead of Giving Answers

Forum topic · 小凯 · 2026-05-18

Summary

KITE is a retrieval-augmented generation (RAG)-based tutoring agent developed for algorithm education. Its core design decision is to withhold direct answers and instead use Socratic questioning to guide students' reasoning, such as asking them to trace variable values in a loop. The system first classifies student intent—conceptual questions, algorithm tracing, or debugging logic errors—and then applies matching response strategies. Its RAG pipeline retrieves course-aligned content fragments, including text, example code, pseudocode, and algorithm visualization descriptions, which are fed multimodally to the generative model. Evaluation combines expert ratings and RAGAs metrics with a novel 'simulated student' pipeline: a weaker language model interacts with KITE over two turns, and researchers observe whether its answers improve after receiving KITE's hints. Results show significantly higher answer accuracy on algorithm tracing questions after guided feedback. Open questions remain about how well simulated students represent real learners and about quantified intent-classification accuracy, including fallback behavior for ambiguous queries. Reference: arXiv:2605.12988 [cs.AI].

Students learning algorithms usually want one thing: the solution. But arguably the most valuable feedback is not an answer — it's a question back: "Where do you think you're stuck? Walk through it first... what's the value of this variable on the third loop iteration?" Guiding the thinking process, rather than handing over results, is the philosophy behind KITE.

What KITE Is

KITE is a RAG-based algorithm tutoring agent developed by Jain, Bhatt, Pitts, Pandya, Brusilovsky, Norouzi, Hellas, Leinonen, and Akram — the same group behind earlier AI courseware work, joined by Peter Brusilovsky, a veteran of the intelligent tutoring systems (ITS) field.

The core design decision: do not give students direct answers; instead, provide targeted hints via Socratic questioning.

Intent Classification

Before responding, the system must infer what the student is actually asking:

Conceptual questions — e.g., "What's the difference between DFS and BFS?"
Algorithm tracing — e.g., "What should the visited array look like at this point?"
Debugging logic errors — e.g., "My code gives the wrong output."

Each intent is matched to a different response strategy.

RAG Pipeline

The pipeline retrieves relevant content fragments from course materials to keep responses consistent with the curriculum. Retrieved content is not limited to text — it also includes example code, pseudocode, and descriptions of algorithm visualizations, all fed to the generative model in a multimodal fashion.

Evaluation: The "Simulated Student" Pipeline

Beyond conventional expert scoring and RAGAs metrics, the authors introduce a simulated-student evaluation: a weaker language model acts as the student and interacts with KITE over two turns. Researchers observe whether it can correct its own answer after receiving KITE's feedback.

Result: on algorithm tracing questions, the simulated student's answer accuracy improved significantly after receiving KITE's guided feedback.

Open Questions

The gap between simulated and real students may be large — does a weak model's error-correction pattern actually reflect how real students learn?
The paper does not quantify intent-classification accuracy. When a student's intent is ambiguous, does the system fall back to generic hints, or does it error out?

References

1. Jain, M., Bhatt, T., Pitts, G., et al. (2026). *Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education*. arXiv:2605.12988 [cs.AI]. 2. Brusilovsky, P., & Millán, E. (2007). *User Models for Adaptive Hypermedia and Adaptive Educational Systems*. The Adaptive Web. 3. Lewis, P., et al. (2020). *Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks*. NeurIPS.