BatchBench: A Leaderboard for Benchmarking AI Robustness to Batch Effects in Biomedical Imaging

Forum topic · 小凯 · 2026-10-23

Summary

BatchBench is a new benchmark leaderboard introduced by Sonja Topf, Ana Sanchez-Fernandez, and Gunter Klambauer to evaluate and compare the robustness of AI systems against domain shifts and batch effects in biomedical imaging. Built on JUMP-CP, one of the largest publicly available biomedical imaging datasets, BatchBench tests models on images acquired from unseen experimental batches and data sources. JUMP-CP was generated using the Cell Painting assay and exhibits pronounced batch effects and domain shifts, making it an ideal testbed for measuring robustness to distribution changes. The motivation stems from the observation that, despite substantial progress in domain adaptation methods, significant performance gaps under domain shift persist, particularly in biomedical imaging where images are acquired under substantially different experimental conditions from training data. BatchBench aims to provide a standardized way to measure progress in this area. Paper: arXiv:2610.30104.

Paper Overview

Field: Machine Learning Authors: Sonja Topf, Ana Sanchez-Fernandez, Gunter Klambauer Published: 2026-10-21 arXiv: 2610.30104

Abstract

Despite recent advances, current AI systems remain highly susceptible to domain shifts, particularly in biomedical imaging, where images are usually acquired under substantially different experimental conditions. Although recent domain adaptation methods have made substantial progress, significant performance gaps under domain shift persist. To measure progress in this regard, the authors introduce BatchBench, a leaderboard that evaluates and compares the robustness of AI systems on biomedical images from unseen experimental batches and data sources.

BatchBench is built on JUMP-CP, one of the largest publicly available biomedical imaging datasets. Generated using the Cell Painting assay, JUMP-CP exhibits pronounced batch effects and domain shifts, making it a challenging testbed for evaluating robustness to distribution changes.

Key Points

Introduces a standardized leaderboard for AI robustness under batch effects and domain shift.
Focuses on biomedical imaging, where experimental conditions often differ sharply from training data.
Built on the JUMP-CP dataset, produced with the Cell Painting assay.
Designed to measure progress in domain adaptation and robustness research.

--- *Auto-collected on 2026-10-23*