Cross-Model Agreement as a Deployment-Time Reliability Signal for Automatic Polyp Segmentation (arXiv:2609.10495)

Forum topic · 小凯 · 2026-09-11

Summary

A paper by Siddharth Gupta and Jitin Singla (arXiv:2609.10495) proposes Referee-Based Quality Estimation (RBQE), a reference-free framework that detects silent failures of polyp segmentation models during real-time colonoscopy, where ground-truth annotations are unavailable at inference. RBQE measures agreement between a primary segmentation model and an independently trained referee on the same image, evaluated on a standardized 1,223-image external benchmark from four public datasets. Using a common Agreement Dice descriptor, a same-architecture referee differing only in random initialization already yields a useful reliability signal (ROC-AUC = 0.923), showing independent training alone is sufficient. Cross-architecture referees improve further: SegFormer-B0 achieves the best performance (ROC-AUC = 0.960), significantly outperforming same-architecture controls, UNet++, and a Test-Time Augmentation baseline (+0.055 ROC-AUC), while a prompt-coupled MedSAM referee underperforms despite maximal architectural diversity. Excluding trivially separable empty-mask cases, ROC-AUC drops to 0.876 (SegFormer-B0) and 0.783 (same-architecture control), but RBQE's margin widens. RBQE also supports selective prediction—mean Dice of retained predictions increases as low-agreement cases are rejected—requiring only one extra deterministic referee forward pass at inference.

Overview

Field: Computer Vision
Authors: Siddharth Gupta, Jitin Singla
Published: 2026-09-09
arXiv: 2609.10495

Problem

Proposed Method: Referee-Based Quality Estimation (RBQE)

Key Findings

Independent training alone suffices: With a common Agreement Dice descriptor, a same-architecture referee differing from the primary model only in random initialization already yields a useful reliability signal (ROC-AUC = 0.923).
Cross-architecture referees help further: SegFormer-B0 achieves the strongest performance (ROC-AUC = 0.960), significantly outperforming the same-architecture control and UNet++.
Beats TTA baseline: RBQE exceeds a representative Test-Time Augmentation baseline by 0.055 ROC-AUC under an identical protocol.
MedSAM underperforms: A prompt-coupled MedSAM referee underperforms despite maximal architectural diversity.
Restricted evaluation: Because empty-mask agreement is trivially separable, a restricted evaluation excluding such cases reports ROC-AUC of 0.876 (SegFormer-B0, 1,046 images) and 0.783 (same-architecture control, 975 images); RBQE's margin over both baselines widens on this identical subset.
Selective prediction: The mean Dice of retained predictions increases as low-agreement cases are progressively rejected.
Low overhead: RBQE requires only one additional deterministic referee forward pass at inference.

Conclusion

Cross-model agreement is a practical, interpretable deployment-time reliability framework for automated polyp segmentation.

--- *Auto-collected on 2026-09-11*