From Seeing to Acting: Smart Glasses as First-Person Intelligence Platforms

Forum topic · 小凯 · 2026-08-27

Summary

A survey by Jiangning Zhang, Haojun Chen, and Yong Liu (arXiv:2608.24877, CV) presents the first unified framework for studying smart glasses as first-person intelligence platforms. The paper argues that smart glasses are evolving from capture-and-display accessories into systems that connect human perception, persistent context, and digital or physical action. Because their on-body viewpoint aligns with the wearer's vision, audition, motion, and hand-object interaction, they must operate under tight energy, thermal, privacy, and feedback constraints. The authors identify the core challenge as whether a complete system—not an isolated model—can sustain a reliable, temporally valid, correctable, and governable perception-state-action loop. The survey formalizes first-person data streams and constraint-aware task utility, characterizes devices along eight verifiable hardware capability axes, organizes the literature around seven interdependent foundational capabilities, and introduces an L0-L5 maturity framework spanning capture, reactive perception, contextual assistance, persistent state, governed action, and embodied coupling.

Overview

Field: Computer Vision Authors: Jiangning Zhang, Haojun Chen, Yong Liu Published: 2026-08-25 arXiv: 2608.24877

Abstract

Smart glasses are evolving from capture and display accessories into first-person intelligence platforms that connect human perception, persistent context, and digital or physical action. Their on-body viewpoint aligns with the wearer's vision, audition, motion, and hand-object interaction, but must operate under tight energy, thermal, privacy, and feedback constraints.

Despite rapid progress in augmented reality, egocentric vision, multimodal models, human-computer interaction, and embodied intelligence, the literature remains fragmented across devices, tasks, and benchmarks. The key challenge is not whether a model can recognize, answer, remember, or act in isolation, but whether a complete system can sustain a reliable, temporally valid, correctable, and governable perception-state-interaction-action loop.

Contributions

First systematic survey of smart glasses through a unified framework
Formalizes first-person data streams and constraint-aware task utility
Characterizes devices along eight verifiable hardware capability axes
Organizes the literature around seven interdependent foundational capabilities
Introduces an L0-L5 maturity framework spanning capture, reactive perception, contextual assistance, persistent state, governed action, and embodied coupling

--- *Auto-collected on 2026-08-27.*