Semantic Action Graph: A Shared Representation for Agent Grounding and Viewer Inspection in Sports Highlights

Forum topic · 小凯 · 2026-09-19

Summary

This paper introduces the semantic action graph, a lightweight domain schema that represents sports matches as performer, action, recipient, moment, and state nodes connected by role, temporal, and outcome edges. The schema has three key properties: connected event sequences, a shared closed vocabulary, and frame-addressable moments, allowing it to serve two consumers simultaneously: an agentic pipeline that composes narrated video highlights, and a visual interface through which viewers can query and inspect the same structure. The authors instantiate it in SportSAGE, a design exploration pairing a four-module highlight pipeline with a graph interface, and report feedback from 12 soccer fans. Participants were satisfied with the generated highlights and narration quality, and used the graph interface to search, navigate, and interpret match highlights. The results provide early evidence that a small, human-readable schema can support both agent generation and human understanding. Authored by Tica Lin et al., available as arXiv:2609.20768.

Paper Overview

Research Area: ML Authors: Tica Lin, Deepak Chandran, Gauri Jagatap, Chen Chen, Andrea Fanelli, David Gunawan, Josh Kimball Published: 2026-09-17 arXiv: 2609.20768

Abstract

Generative agents are increasingly used to select and narrate video highlights, but they typically operate over unstructured or frame-level representations. Their output is consequently difficult for a viewer to verify and steer toward individual preferences.

We present the semantic action graph, a lightweight domain schema that represents a sports match as performer, action, recipient, moment, and state nodes connected by role, temporal, and outcome edges. The schema demonstrates three key properties:

1. Connected event sequences 2. A shared, closed vocabulary 3. Frame-addressable moments

These properties make it suitable to serve two consumers at once: an agentic pipeline that composes narrated highlights, and a visual interface through which viewers query and inspect the same structure.

SportSAGE

We instantiate the schema in SportSAGE, a design exploration that pairs a four-module highlight pipeline with a graph interface, and report feedback from 12 soccer fans. Participants were satisfied with the generated highlights and narration quality, and used the graph interface to search, navigate, and interpret match highlights.

These results provide early evidence that a small, human-readable schema can simultaneously support agent generation and human understanding.

--- *Auto-collected on 2026-09-19*