Differences in Detection (DnD): An Intuitive Method to Compare Object Detection Models

Forum topic · 小凯 · 2026-06-09

Summary

Differences in Detection (DnD) is a method proposed by Johannes Theodoridis, Johannes Maucher, and Andreas Schilling for directly comparing two object detection models. Built on the same matching algorithm used in standard evaluation, DnD complements the conventional mean Average Precision (mAP) metric and TIDE error analysis by enabling head-to-head model comparison rather than comparing independent summary statistics. The method computes the intersection of ground truth labels recognized by both models, along with the corresponding difference sets and the complement set of ground truth labels missed by both models. This reveals each model's individual mistakes as well as shared errors in a direct and intuitive way. When combined with error type analysis, differences in detection errors can be naturally examined within a standard confusion matrix, making the approach particularly useful for explainability and error diagnosis in computer vision research.

Paper Overview

Research Area: Computer Vision (CV) Authors: Johannes Theodoridis, Johannes Maucher, Andreas Schilling Published: 2025-06-11 arXiv: 2506.08640

Abstract

We propose Differences in Detection (DnD), an intuitive method to compare two object detection models. Based on the same matching algorithm, it complements the standard metrics of mean Average Precision (mAP) and TIDE error analysis with the ability to compare two models directly. More specifically, we calculate the intersection of ground truth labels that are recognized by both models, followed by the corresponding difference sets and the complement set of ground truth labels that are missed by both models. The resulting comparison is more direct and intuitive than a comparison of independent summary statistics. It reveals individual and shared mistakes and becomes particularly interesting when combined with error types. In this case, the differences in detection errors can be analyzed naturally within a standard confusion matrix.

Key Ideas

Direct model comparison: Instead of only comparing aggregate metrics like mAP, DnD compares which ground truth objects each of two models actually detects.
Set-based analysis: The method computes the intersection (objects detected by both models), difference sets (objects detected by only one model), and the complement (objects missed by both models).
Complementary tool: DnD uses the same matching algorithm as standard evaluation and augments mAP and TIDE error analysis rather than replacing them.
Error type integration: Combined with error type analysis, detection differences can be examined through a standard confusion matrix, improving explainability of model behavior.