Keep Yelling Assistant: A Vision-Language Model for Emotionally Expressive Responses to Risky Driving

Forum topic · 小凯 · 2026-07-21

Summary

Researchers introduce Keep Yelling Assistant (KYA), a vision-language pipeline that detects risky driving behaviors in real time and generates emotionally expressive responses to improve driver awareness and comfort. The vision module uses YOLOv8 variants to detect nearby vehicles and identify dangerous maneuvers such as sudden cut-ins, extracting normalized driving metrics including relative distance, speed, and projected reach time into a structured behavior log. The language module processes this log with user-defined emotional tone settings (neutral, humorous, analytical) and generates verbal reactions via large language models including ChatGPT-4o, Claude 3, Gemini 2.5, and Copilot. Evaluation used dashcam videos of risky driving and a user study with 108 participants who rated response styles and emotional alignment. All models scored favorably, with preferences varying by persona; the YOLOv8s plus ChatGPT-4o combination achieved the highest score of 4.29/5.00. KYA represents a new paradigm for emotionally intelligent in-vehicle AI (arXiv:2507.15486).

Paper Overview

Field: cs.CV Authors: Harine Choi, Eun Hak Lee, Zhengzhong Tu Published: 2026-07-21 arXiv: 2507.15486

Abstract

This study introduces a vision-language pipeline that detects risky driving behaviors and generates emotionally expressive responses to support driver awareness and comfort. Although vision-language models have advanced perception and reasoning in autonomous driving, existing systems rarely consider the emotional dimension or real-world user experience. Keep Yelling Assistant (KYA) detects high-risk driving maneuvers in real time, such as sudden cut-ins. It then produces emotional responses through a large language model tailored to driver preferences.

Framework

The framework comprises two core modules:

Vision module: Uses YOLOv8 variants to detect nearby vehicles and identify risky behaviors such as sudden cut-ins. Key driving metrics, including relative distance, speed, and projected reach time, are extracted and normalized to produce a structured behavior log.
Language module: Processes this log with user-defined emotional tone settings (e.g., neutral, humorous, analytical) and generates verbal reactions using state-of-the-art large language models, including ChatGPT-4o, Claude 3, Gemini 2.5, and Copilot.

Evaluation

The system was evaluated using dashcam videos containing risky driving behaviors and a user study involving 108 participants. Participants selected preferred response styles, and the LLMs were evaluated based on emotional alignment. All models received favorable ratings, although preferences varied across personas. Notably, the combination of YOLOv8s and ChatGPT-4o achieved the highest score of 4.29 out of 5.00.

Conclusion

By integrating real-world perception with emotionally adaptive dialogue, KYA introduces a new paradigm for emotionally intelligent in-vehicle artificial intelligence.

--- *Auto-collected on 2026-07-21*