Integrity Review Report: The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection

Academic fraud report · Geng Detector

Summary

This report evaluates the IJCV 2021 paper (DOI: 10.1007/s11263-020-01400-4) by Bergmann et al., a journal extension of the CVPR 2019 MVTec AD dataset paper. The verdict is CLEAN — no academic misconduct indicators were detected. Key checks confirmed: (1) Table 1 image counts sum exactly to 3629 training and 1258 test defect images, a natural-looking tally rather than a rounded fabrication; (2) AU-PR values in Table 2 show organic decimal distributions (e.g., 0.711, 0.042, 0.035, 0.568, 0.017); (3) Table 5 algorithm scores exhibit realistic large variance (f-AnoGAN: 0.251 Carpet vs 0.825 Hazelnut), inconsistent with fabricated uniformly high results; (4) the CVPR 2019 → April 2020 submission → November 2020 acceptance → January 2021 publication timeline is standard; (5) full hyperparameter disclosure and explicit declaration of the authors' affiliation with MVTec Software GmbH and HALCON library strengthen authenticity. Pixel-level image analysis was not performed since only text was available.

Verdict

Clean (no academic misconduct indicators detected). The paper passes all six checks for image reuse, data fabrication, image splicing, statistical anomalies, output irregularities, and citation/methodology issues.

Key findings

Arithmetic in Table 1 is internally consistent: 280+264+245+230+247 (Texture) + sum of Object categories = 3629 training images; 382 + 876 = 1258 test defect images, both matching the stated totals.
AU-PR values in Table 2 (e.g., 0.711, 0.042, 0.035, 0.568, 0.017) show natural decimal tails, not the uniform-0-or-5 pattern typical of fabricated numbers.
Algorithm performance in Table 5 (AU-ROC) shows realistic high variance across categories (f-AnoGAN ranges from 0.251 to 0.825), inconsistent with the uniformly inflated results that fabrication often produces.
Timeline is internally consistent: CVPR 2019 conference version → submitted 15 April 2020 → accepted 2 November 2020 → published online 6 January 2021.
The Student-Teacher method ranks consistently at Rank 1 across AU-ROC, AU-PRO, AU-IoU, and AU-PR in Table 3, consistent with its use of pretrained features.
Hyperparameters are reported in full (latent dim 128, image size 256×256, Adam lr=2e-4), indicating genuine experimental work.
Authors disclose affiliation with MVTec Software GmbH and explicitly note that the optimized implementation is available in HALCON, an unusually transparent conflict-of-interest disclosure.

Evidence highlights

DOI: 10.1007/s11263-020-01400-4
Training set total verified: 3629; test defect set total verified: 1258 (Table 1).
f-AnoGAN AU-ROC range across categories: 0.251 (Carpet) to 0.825 (Hazelnut), Table 5.
Submission-to-acceptance interval: ~6.5 months (15 April 2020 → 2 November 2020), within normal IJCV turnaround.

Notes

Limits of this review: only the manuscript text was provided; pixel-level image-duplication and splicing analysis of Figures 1–10 was therefore not possible. The textual claim that "each dataset image shows a unique physical sample" reduces the likelihood of physical-sample recycling, but cannot substitute for forensic image analysis.
The paper is a dataset contribution rather than a hypothesis-driven study, so p-value fabrication is not applicable; AUC-based evaluation is the field standard.
The report was AI-assisted; final determination of academic misconduct requires institutional investigation.