KITE: A Socratic RAG-Based Tutor That Guides Students Through Algorithm Debugging Instead of Giving Answers

Forum topic · 小凯 · 2026-05-18

Summary

KITE is a retrieval-augmented generation (RAG) tutoring agent for algorithms education developed by Jain, Bhatt, Pitts, Pandya, Brusilovsky, Norouzi, Hellas, Leinonen, and Akram. Its core design decision is to avoid directly answering student questions; instead, it responds with Socratic, targeted hints that guide the student's reasoning process. The system first classifies student intent—conceptual questions (e.g., DFS vs. BFS), algorithm tracing (e.g., the state of a visited array), or debugging logic errors—and applies different response strategies for each. Its RAG pipeline retrieves course material, including text, example code, pseudocode, and algorithm visualization descriptions, feeding them multimodally to the generator to keep answers consistent with course content. Evaluation combines expert scoring and RAGAs metrics with a novel "simulated student" pipeline: a weaker LLM interacts with KITE over two turns to see whether it corrects its answer after receiving KITE's feedback. Results showed notably improved answer accuracy on tracing-style problems after KITE's guidance. Open questions include the gap between simulated and real student behavior, and unquantified intent-classification accuracy.

Students learning algorithms most want the feedback "here's how to solve it." But the best feedback is: "Where are you stuck? Walk through it yourself first... what's the value of this variable on the third loop iteration?"—not giving answers, but guiding the thought process.

The KITE System

KITE, developed by Jain, Bhatt, Pitts, Pandya, Brusilovsky, Norouzi, Hellas, Leinonen, and Akram (the same group behind the AI courseware work, joined by ITS veteran Peter Brusilovsky), is a RAG-based tutoring agent for algorithms. Its core design decision: do not give students direct answers; instead, respond with Socratic, targeted hints.

The system first infers student intent:

Conceptual question — e.g., "What's the difference between DFS and BFS?"
Algorithm tracing — e.g., "What should the visited array look like here?"
Debugging a logic error — e.g., "My code's output is wrong"

Each intent category is matched to a different response strategy.

RAG Pipeline

The underlying pipeline retrieves relevant content chunks from course materials to keep responses consistent with course content. Retrieved content is not limited to text—it also includes example code, pseudocode, and algorithm visualization descriptions, fed to the generator in a multimodal fashion.

Evaluation

The evaluation has a novel component. Beyond standard expert scoring and RAGAs metrics, the authors built a "simulated student" pipeline: a weaker language model (playing the simulated student) interacts with KITE over two turns, and the system observes whether it can revise its answer after receiving KITE's feedback.

Result: on algorithm-tracing questions, the simulated student's answer accuracy improved significantly after receiving KITE's guiding feedback.

Open Questions

The gap between simulated and real students may be large—does a weak model's error-correction pattern actually reflect real student learning behavior?
The system's intent-classification accuracy is not quantitatively disclosed. If a student's intent is ambiguous, does the system fall back to generic hints, or does it fail?

References

1. Jain, M., Bhatt, T., Pitts, G., et al. (2026). *Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education*. arXiv:2605.12988 [cs.AI]. 2. Brusilovsky, P., & Millán, E. (2007). *User Models for Adaptive Hypermedia and Adaptive Educational Systems*. The Adaptive Web. 3. Lewis, P., et al. (2020). *Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks*. NeurIPS.