Towards Scaling Marine Perception with Synthetic Data

Forum topic · 小凯 · 2026-09-20

Summary

This paper addresses the scarcity of labeled real-world training data for underwater machine learning by extending OceanSim, an IsaacSim-based underwater perception simulator, with a Synthetic Data Generation (SDG) pipeline. The pipeline generates large-scale, automatically labeled, photorealistic datasets with configurable scene appearance, structure, and sensor settings, enabling scalable training data for underwater perception tasks. The authors evaluate the pipeline on a real-world sea urchin detection task, studying how different forms of synthetic scene variation affect sim-to-real transfer performance. Based on the experiments, the paper discusses key insights, main limitations of the current pipeline, and future directions for improving underwater rendering fidelity, scene diversity, and sim-to-real generalization evaluation. The work was authored by Haoyu Ma, Onur Bagoren, Anja Sheppard, and colleagues at the University of Michigan field robotics group, and is available on arXiv (2609.20680) with open-source code on GitHub.

Paper Overview

Research area: Computer Vision Authors: Haoyu Ma, Onur Bagoren, Anja Sheppard, Elias Fandi, Ashrith Edukulla, Tanner Aslan, Natasha Sieh, Jingyu Song, Katherine A. Skinner Published: 2026-09-17 arXiv: 2609.20680

Abstract

Scalable machine learning in challenging underwater environments is strongly limited by the lack of labeled real-world training data. This data is often expensive and laborious to gather, making large-scale real-world data challenging to gather and curate. However, simulated data can help close the gap, enabling many learning-based tasks for underwater perception.

In this work, the authors extend OceanSim, an IsaacSim-based underwater perception simulator, with a Synthetic Data Generation (SDG) pipeline for training models to be used in underwater scenarios. The proposed pipeline enables users to generate large, automatically labeled, photorealistic datasets with configurable scene appearance, structure, and sensor settings.

Key Contributions

An SDG pipeline built on OceanSim for generating large-scale, auto-annotated, photorealistic underwater datasets
Configurable scene appearance, structure, and sensor settings for controlled data variation
Evaluation on a real-world sea urchin detection task, analyzing how different forms of synthetic scene variation affect sim-to-real performance
Discussion of limitations and future directions for underwater rendering fidelity, scene diversity, and sim-to-real generalization evaluation

Resources

Paper: https://arxiv.org/abs/2609.20680
Code: https://github.com/umfieldrobotics/OceanSim

--- *Auto-collected on 2026-09-20*