Prompt-Engineering Framework for Personalized AI Teaching Assistants (Jill Watson)

Forum topic · 小凯 · 2026-09-06

Summary

A study on arXiv (2509.00005) by Saptarshi Basu, Sandeep Kakar, and Ashok Goel presents a prompt-engineering framework for personalizing general-purpose LLM/RAG-based AI teaching assistants such as Jill Watson across academic disciplines. The framework adapts responses using six learner-specific dimensions—self-assessment, abstraction preference, verbosity preference, perceptual orientation, information processing style, and level of understanding—yielding 96 distinct learner profiles. Student queries are analyzed with Bloom's Taxonomy to estimate cognitive complexity per interaction. Learner attributes and cognitive assessments are encoded into structured prompts, conditioning the LLM without retraining. Evaluation combined NLP metrics with a five-participant human study, showing perceived differences in response style and structure under personalized conditions; statistical analysis identified learner attributes correlated with measurable response changes. The findings offer preliminary evidence that prompt-based personalization can support adaptive behavior in LLM-driven educational agents, enabling scalable, flexible, real-time micro-level personalization.

Overview

Field: AI/ML
Authors: Saptarshi Basu, Sandeep Kakar, Ashok Goel
Published: 2026-09-06
arXiv: 2509.00005

AI teaching assistants powered by large language models (LLMs) offer scalable educational support but often provide limited personalization. This study presents a prompt-engineering-based framework for personalizing general-purpose LLM/RAG-based AI teaching assistants such as Jill Watson across academic disciplines and courses.

Key Points

The framework adapts responses using six learner-specific dimensions: self-assessment, abstraction preference, verbosity preference, perceptual orientation, information processing style, and level of understanding, yielding 96 distinct learner profiles.
Student queries are additionally analyzed using Bloom's Taxonomy to estimate cognitive complexity at the interaction level.
Learner attributes and cognitive assessments are encoded in structured prompts, conditioning the LLM without retraining the model.
The framework was evaluated experimentally using NLP metrics and a human study with five participants.
Results showed perceived differences in response style and structure under personalized conditions; statistical analysis identified learner attributes associated with measurable changes in responses.
The findings provide preliminary evidence that prompt-based personalization can support adaptive behavior in LLM-driven educational agents.

Significance

The approach demonstrates a lightweight, scalable path to hybrid micro-level personalization in general-purpose AI teaching assistants, avoiding costly model fine-tuning while supporting real-time adaptation across disciplines and courses.