SIMON: Saliency-Aware Multi-View Neural Decoding for EEG-to-Image Retrieval

> Paper: SIMON: Saliency-aware Integrative Multi-view Object-centric Neural Decoding > Authors: YuSheng Lin, Ji-Hwa Tsai, Chun-Shu Wei > arXiv: 2605.00401 | 2026-04-29

1. The Brain-Computer Interface That "Only Looks at the Center"

Imagine controlling image retrieval with your brainwaves (EEG):

Problems with existing methods:

• They assume the viewer always looks at the center of an image
• Visual feature extraction is focused on the center
• But human attention is content-driven
• We look at the most "salient" regions of an image

The contradiction:

• You see a cat in the corner
• But the system assumes you're looking at the center
• The extracted features don't match your attention
• EEG decoding fails

This is the "geometry-semantics separation" problem.

2. SIMON: A Saliency-Aware Multi-View Framework

The paper proposes SIMON:

Core idea: > When humans view images, attention is drawn to salient regions. EEG decoding should be based on salient regions, not a fixed center.

Technical approach:

1. Saliency detection

• Analyzes the salient regions of an image
• Identifies the parts most likely to attract attention
• E.g., a cat's face, vivid colors

2. Multi-view feature extraction

• Not just the center
• But features from multiple viewpoints
• Center view + salient-region views

3. Integrative decoding

• Combines multi-view information
• Matches it against EEG signals
• Retrieves the most relevant images

4. Zero-shot retrieval

• No paired training data required
• Leverages pretrained models
• Generalizes to novel images

An analogy:

• Traditional methods: assume you always look at the center of a road sign
• SIMON: knows you'll look at the most prominent text on the sign
• More accurate decoding

3. Why Does Saliency Matter So Much?

The problem with center bias:

• It doesn't match human behavior: attention is content-driven, not position-driven; the fixed-center assumption is wrong
• Feature mismatch: EEG encodes the content of attention, but features are extracted from the center — the two don't align

Advantages of saliency:

• Cognitive plausibility: humans really are drawn to salient regions, so features align with attention and match EEG better
• Higher accuracy: decoding is based on the true attentional focus rather than an assumed one, improving retrieval accuracy

4. A Feynman-Style Takeaway: Understanding Attention Is Key to Understanding Perception

Feynman noted that "knowing the name of something" and "understanding something" are entirely different.

For brain-computer interfaces:

> Assuming people always look at the center is an oversimplification of human perception. SIMON's insight is that attention is content-driven — we look at the most salient parts of an image. Understanding this is what makes correct EEG decoding possible.

This echoes a core point from cognitive science:

• Attention is selective
• It is not uniform
• Understanding attention patterns = understanding perception

5. Questions Worth Asking

If you work on brain-computer interfaces or visual understanding, ask yourself:

1. "Does my model assume a fixed attention pattern?" 2. "Could saliency better match human attention?" 3. "Could multi-view features improve decoding accuracy?" 4. "Have I considered the reality of human cognition?"

SIMON reminds us: a brain-computer interface is not just an engineering problem — it is a cognitive one.

When a decoding system understands the real rules of human attention, it can read true intent from brainwaves. In the future of BCIs, the best systems won't be the most powerful ones, but the ones that best understand human perception.

In the code of brainwaves, attention is the key decoder.