Paper Overview
Field: Computer Vision Authors: Johannes Theodoridis, Johannes Maucher, Andreas Schilling Published: 2025-06-11 arXiv: 2506.08640
Abstract
We propose Differences in Detection (DnD), an intuitive method to compare two object detection models. Based on the same matching algorithm, it complements the standard metrics of mean Average Precision (mAP) and TIDE error analysis with the ability to compare two models directly. More specifically, we calculate the intersection of ground truth labels that are recognized by both models, followed by the corresponding difference sets and the complement set of ground truth labels that are missed by both models. The resulting comparison is more direct and intuitive than a comparison of independent summary statistics. It reveals individual and shared mistakes and becomes particularly interesting when combined with error types. In this case, the differences in detection errors can be analyzed naturally in a standard confusion matrix.
Key Ideas
- DnD (Differences in Detection): a set-based method for directly comparing two object detection models using the same matching algorithm as standard evaluation.
- Complements mAP and TIDE error analysis rather than replacing them.
- Computes:
- The intersection of ground truth labels detected by both models
- The difference sets (labels detected by only one model)
- The complement set (labels missed by both models)
- More direct and intuitive than comparing independent summary statistics.
- Reveals individual and shared mistakes; combined with error type analysis, differences in detection errors map naturally onto a standard confusion matrix.
*Auto-collected on 2026-06-09*