SymAnt: Neuro-Symbolic Long-Horizon Action Anticipation from Short Video Observations

Forum topic · 小凯 · 2026-10-31

Summary

SymAnt is a novel neuro-symbolic framework for long-horizon action anticipation, introduced in arXiv paper 2510.25911 by Fnu Aryan, Yaqi Xie, and Joseph Campbell (published October 28, 2025). As robots and intelligent systems increasingly interact with humans, they must predict long sequences of future actions from brief observations. Purely neural models typically need extensive observation data and are hard to interpret or adjust. SymAnt unifies neural network expressiveness with structured reasoning over scene graphs and knowledge graphs. Given a short video, it performs a joint search over both graphs to build a symbolic context, enabling unified reasoning about spatial relationships, object attributes, affordances, and associated actions. This context serves as a prior for an iterative refinement process that progressively generates and improves future action sequences. Predictions are grounded in the symbolic context, making the approach parameter-efficient, editable by users without retraining, and effective with short observation windows. The authors validate SymAnt on five datasets: Breakfast, 50 Salads, EPIC-KITCHENS, EGTEA Gaze+, and Assembly101.

Paper Overview

Field: Computer Vision Authors: Fnu Aryan, Yaqi Xie, Joseph Campbell Published: 2025-10-28 arXiv: 2510.25911

Abstract

As robots and intelligent systems increasingly interact with humans, they must anticipate long sequences of future actions from only brief observations. While purely neural models typically require extensive observations to predict future actions and remain difficult to interpret and adjust, symbolic priors offer a structured alternative.

To this end, the authors introduce SymAnt, a novel neuro-symbolic approach that unifies the expressiveness of neural networks with the structured reasoning of scene graphs and knowledge graphs.

How It Works

Given a short video, SymAnt performs a joint search over both the scene graph and the knowledge graph to construct a symbolic context, enabling unified reasoning over spatial relationships, object attributes, affordances, and associated actions.
This symbolic context serves as a prior for an iterative refinement process that progressively generates and improves future action sequences.
By grounding predictions in the symbolic context, SymAnt predicts long-horizon actions from short observation windows in a parameter-efficient manner, while allowing users to directly edit the underlying knowledge to adjust predictions without further training.

Evaluation

The authors demonstrate SymAnt's effectiveness through experiments on five diverse datasets, particularly when only short observation contexts are available:

Breakfast
50 Salads
EPIC-KITCHENS
EGTEA Gaze+
Assembly101

Full paper: arXiv:2510.25911