Cross-Model Agreement as a Deployment-Time Reliability Signal for Automatic Polyp Segmentation (RBQE, arXiv 2609.10495)

Forum topic · 小凯 · 2026-09-11

Summary

In real-time colonoscopy, ground-truth annotations are unavailable at inference, so polyp segmentation models can fail silently. This paper proposes Referee-Based Quality Estimation (RBQE), a reference-free framework that measures agreement between a primary segmentation model and an independently trained referee model on the same image. The method is evaluated on a standardized 1,223-image external benchmark drawn from four public datasets. With a common Agreement Dice descriptor, a same-architecture referee differing only in random initialization already yields a useful reliability signal (ROC-AUC = 0.923), showing independent training alone is sufficient. Cross-architecture referees improve performance further: SegFormer-B0 achieves the strongest results (ROC-AUC = 0.960), outperforming the same-architecture control, UNet++, and a test-time augmentation baseline, while a prompt-coupled MedSAM referee underperforms. On a restricted evaluation excluding trivially separable empty-mask cases, ROC-AUC falls to 0.876 (SegFormer-B0), yet RBQE's margin over baselines widens. RBQE also supports selective prediction, increasing the mean Dice of retained predictions as low-agreement cases are rejected, with only one additional deterministic forward pass at inference. Paper by Siddharth Gupta and Jitin Singla, arXiv:2609.10495.

Paper Overview

Field: Computer Vision (medical imaging)
Authors: Siddharth Gupta, Jitin Singla
Published: 2026-09-09
arXiv: 2609.10495

The Problem

Method: Referee-Based Quality Estimation (RBQE)

Evaluation

Benchmark: standardized 1,223-image external set drawn from four public polyp datasets.
Design axes: four referee configurations separate *referee independence* from *architectural diversity*.
Descriptor: a common Agreement Dice descriptor across configurations.

Key Findings

A same-architecture referee differing from the primary model only in random initialization already yields a useful reliability signal: ROC-AUC = 0.923, showing that independent training alone is sufficient.
Cross-architecture referees improve further: SegFormer-B0 achieves ROC-AUC = 0.960, significantly outperforming the same-architecture control and UNet++, and exceeding a representative Test-Time Augmentation baseline by 0.055 ROC-AUC under an identical protocol.
A prompt-coupled MedSAM referee underperforms despite maximal architectural diversity.
Restricted evaluation (excluding trivially separable empty-mask cases): ROC-AUC falls to 0.876 (SegFormer-B0, 1,046 images) and 0.783 (same-architecture control, 975 images), yet RBQE's margin over both baselines widens on this identical subset.
Selective prediction: the mean Dice of retained predictions increases as low-agreement cases are progressively rejected.
Cost: only one additional deterministic referee forward pass at inference.

Conclusion

Cross-model agreement is a practical, interpretable, and inexpensive reliability signal for automated polyp segmentation at deployment time, supporting safer clinical use through failure detection and selective prediction.

--- *Source: arXiv paper page — https://arxiv.org/abs/2609.10495*