Modeling Subjective Urban Perception with Human Gaze: Place Pulse-Gaze Dataset and Gaze-Guided Framework

Forum topic · 小凯 · 2026-05-04

Summary

This paper, 'Modeling Subjective Urban Perception with Human Gaze' (arXiv: 2605.00764) by Lin Che, Xi Wang, Marc Pollefeys, Konrad Schindler, Martin Raubal, and Peter Kiefer, argues that computer vision models of urban perception should consider not just what people see, but how they see. The authors introduce Place Pulse-Gaze, a dataset pairing street-view images with synchronized eye-tracking recordings and individual subjective perception labels (e.g., safety, vibrancy, aesthetics). Building on it, a Gaze-Guided Urban Perception Framework uses gaze data to direct model attention to regions humans actually fixate on, models subjective ratings from gaze patterns, and captures inter-individual differences. Key insights: attention is unevenly distributed toward salient cues (faces, text, danger signals); gaze sequences reflect the cognitive process of forming judgments; and demographic differences in fixation explain divergent evaluations of identical scenes. The takeaway for perception AI: subjective experience is an active, constructive process shaped by observation behavior, so gaze data offers an interpretable behavioral signal for modeling it.

> Paper: Modeling Subjective Urban Perception with Human Gaze > Authors: Lin Che, Xi Wang, Marc Pollefeys, Konrad Schindler, Martin Raubal, Peter Kiefer > arXiv: 2605.00764 | 2026-04-30

The AI That Only Takes Photos, Never Watches People

Walking down a street, you don't just register buildings, roads, and vehicles — you feel safety, vibrancy, beauty, or oppression. But traditional computer vision systems only "take photos": they analyze pixels and detect objects without ever asking:

Does this place feel safe or dangerous?
Is this street lively or dead?
Does this neighborhood look affluent or impoverished?

They ignore the most critical layer: human subjective perception.

Urban Perception: From Images to Feelings

Urban perception research studies how people subjectively evaluate urban environments — not objective measurements like building height or road width, but subjective experiences like "safe," "beautiful," or "boring."

Limitations of existing computational approaches:

They model perception directly from street-view images
They ignore the human perception process
They don't know where people look, how they look, or why they rate scenes the way they do

It's like judging whether a dish tastes good from a photo — without knowing what the eater looked at, smelled, or tasted.

Place Pulse-Gaze: Eye-Tracking + Street View + Subjective Ratings

The paper introduces the Place Pulse-Gaze dataset and a Gaze-Guided Urban Perception Framework built on it.

Dataset innovations:

Street-view images + synchronized eye-tracking recordings + individual perception labels
It captures not only "how people rated a scene" but "where they were looking while forming that rating"

Framework design:

1. Gaze guidance: eye-tracking data directs the model to regions humans actually fixate on 2. Perception modeling: subjective ratings are predicted based on gaze patterns 3. Individual differences: the framework captures how perception varies across individuals

It's like giving AI "human eyes" — seeing not just the image, but *how* humans look at the image.

Why Eye-Tracking Data Matters

Eye-tracking reveals several key insights:

1. Attention is not uniform scanning. People don't evenly survey a scene; certain regions (faces, text, danger signals) attract more fixations. These high-attention regions are crucial for perception.

2. Gaze trajectories reflect cognitive processes. What you look at first, next, and for how long mirrors how a subjective judgment forms. A person feeling unsafe may quickly scan escape routes; a person feeling pleased may linger on aesthetic details.

3. Explainable individual differences. People of different cultural backgrounds, genders, and ages may fixate on different regions — explaining why the same street scene receives different ratings.

How You Observe Shapes What You Observe

The core philosophical point: an environment's "objective" properties are not the same as a person's "subjective" experience. Perception is not passive reception but active construction — and the manner of construction (gaze patterns) determines the outcome (subjective evaluation).

Traditional computer vision assumes image information is objective and perception directly processes that information. Place Pulse-Gaze shows instead that perception is an active process: how people look determines what they see.

Takeaways for Perception AI Builders

If you're building perception AI systems, ask yourself:

1. Does my model account for "how humans look," not just "what humans see"? 2. Could eye-tracking data enhance my scene understanding? 3. Can subjective perception be modeled from behavioral data such as gaze? 4. Am I focusing only on perception *outcomes* while ignoring the perception *process*?

The central lesson: to understand human experience of the world, you cannot study the world alone — you must also study how humans experience it.

A city is not the sum of its buildings. It is the space of how people feel, walk, gaze, and remember. For AI to truly understand cities, it must understand how humans interact with them. The eye is not just a visual organ — it is a window into the mind. Through gaze data, we glimpse not only "what is seen" but "how it is felt."