See & Sniff: Learning Visuo-Olfactory Representations with SmellNet-V

Forum topic · 小凯 · 2026-06-28

Summary

See & Sniff is a self-supervised framework for learning joint visual-olfactory representations, introduced alongside SmellNet-V, a scalable visuo-olfactory dataset. The key insight is that odor identity is largely invariant to visual transformations within a semantic category, allowing smell-only samples to be synthetically paired with semantically aligned in-the-wild web images. This converts unimodal olfactory datasets into cross-modal benchmarks without costly co-collection. The framework uses dense local alignment to learn joint representations and naturally produces smell saliency maps for spatial grounding of odor sources. The authors also introduce a pixel-level smell grounding task and evaluation benchmark. The method outperforms olfaction-only baselines by 7% on olfactory classification and generalizes to cross-modal retrieval and smell grounding, establishing visuo-olfactory learning as a new direction in multimodal perception.

Paper Overview

Research Area: Computer Vision Authors: Seongyu Kim, Seungwoo Lee, Hyeonggon Ryu, Joon Son Chung, Arda Senocak Published: 2026-06-25 arXiv: 2606.27307

Abstract

While modern multimodal models integrate vision with language, audio, or touch, olfaction remains largely unexplored due to the lack of paired visuo-olfactory data. The authors introduce SmellNet-V, a scalable visuo-olfactory dataset built on the insight that odor identity is largely invariant to visual transformations within a semantic category. This allows smell-only samples to be synthetically paired with semantically aligned in-the-wild web images, converting a unimodal olfactory dataset into a cross-modal benchmark without costly co-collection.

Building on this dataset, they propose See & Sniff, a self-supervised framework that learns joint visuo-olfactory representations via dense local alignment and naturally produces smell saliency maps for spatial grounding of odor sources. They further introduce a pixel-level smell grounding task and evaluation benchmark.

Key Results

Outperforms olfaction-only baselines by 7% on olfactory classification
Generalizes to cross-modal retrieval and smell grounding
Produces smell saliency maps enabling spatial localization of odor sources

Why It Matters

The work establishes visuo-olfactory learning as a new direction in multimodal perception, sidestepping the data scarcity problem that has kept olfaction largely unexplored in modern multimodal models.

---

*Auto-collected on 2026-06-28*