English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

RS-JEPA: Compact Joint-Embedding Predictive Architecture for Cross-Sensor Remote Sensing Image Retrieval

Forum topic · 小凯 · 2026-11-12

Summary

RS-JEPA (arXiv:2611.14496) is a compact self-supervised framework for cross-sensor remote sensing image retrieval, matching scenes across SAR-optical and panchromatic-multispectral sensors without manual labels. Unlike prior methods that reconstruct sensor-specific pixel details, align only co-registered twins, or rely on transformers with 118-398M parameters, RS-JEPA builds on a shared Vision Mamba encoder processing images as ordered patch sequences. It introduces scan-path views, treating two reading orders of the same image as two views with identical pixels, predicting hidden patches in one order from the visible patches in the other. Adversarial View Coupling (AVC) learns the hardest visible-patch partitions to maintain prediction difficulty, while a neighborhood consensus loss aligns scene similarity across sensors. Experiments on BEN-14K, CBRSIR-VS, and DSRSID benchmarks show state-of-the-art F1@5 scores of 72.92, 80.35, and 94.43 across twelve reimplemented baselines, with a retrieval network of only 3.88M parameters (14.57M during training), 8-16x smaller than prior JEPA retrievers. Code is available at github.com/Tanishq251/RSJEPA.

RS-JEPA: Remote Sensing Joint-Embedding Predictive Architecture

Field: Computer Vision Authors: Tanishq Rachamalla, Srishti Kaushik, Muhammad Attique Khan, Gustau Camps-Valls, Swalpa Kumar Roy Published: 2026-11-10 arXiv: 2611.14496

Abstract

Cross-sensor remote sensing image retrieval aims to find scenes with the same content across heterogeneous sensors, such as SAR and optical or panchromatic (PAN) and multispectral (MS) imagery, without manual labels. Existing self-supervised methods either reconstruct sensor-specific pixel detail, align each image only with its co-registered twin, or rely on large transformers with 118-173M parameters (and up to 398M).

RS-JEPA is a compact joint-embedding predictive framework built on a Vision Mamba encoder shared by both sensors. Because a state-space encoder reads an image as an ordered sequence of patches, the authors propose scan-path views: two reading orders of the same image are treated as two views with identical pixels, and hidden patches in one order are predicted from visible patches in the other. To maintain prediction difficulty, Adversarial View Coupling (AVC) learns the hardest visible-patch partitions, and a neighborhood consensus loss aligns scene similarity across the two sensors.

Key Results

  • Extensive experiments on SAR-optical (BEN-14K, CBRSIR-VS) and PAN-MS (DSRSID) benchmarks
  • Highest F1@5 among 12 reimplemented methods: 72.92, 80.35, and 94.43 respectively
  • Retrieval network of only 3.88M parameters (14.57M during training), 8-16x fewer than previous JEPA-based retrievers
  • Trained on co-registered pairs, consistent with prior methods
  • Highlights the potential of "reading order" as a source of self-supervised views

Code

github.com/Tanishq251/RSJEPA

--- *Auto-collected on 2026-11-12*

Tags

#remote-sensing#self-supervised-learning#joint-embedding-predictive-architecture#vision-mamba#cross-sensor-retrieval#sar#computer-vision#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637070