Fabricated SOTA Figures and Self-Contradictory Ablation in 'Pathology-Aware Reconstruction with Discriminative Knowledge Boosting Alignment for CheX-ray Vision-Language Pre-training' (ACM MM 2025)

Academic fraud report · Geng Detector

Summary

This report flags serious internal numerical inconsistencies in the ACM Multimedia 2025 paper 'Pathology-Aware Reconstruction with Discriminative Knowledge Boosting Alignment for CheX-ray Vision-Language Pre-training' (DOI: 10.1145/3746027.3755336). The central charge is that headline Table 1 reports an AUC of 81.4 for the full model on ChestX-ray14 (1% data), whereas the ablation tables (Tables 4 and 5) place the same full model at 80.8, and Section 5.1 text states a 2.0% gain over MRM (78.8), which arithmetically yields 80.8, not 81.4. Additionally, Table 4 shows that adding the proposed PAR module on top of DKBA degrades AUC from 81.0 to 80.8, contradicting the paper's claimed complementarity. A citation error was also identified. Verdict is high-confidence: arithmetic contradictions across three independent locations in the manuscript constitute strong prima-facie evidence of data fabrication or uncritical AI-assisted rewriting. Uncertainty: numeric contradiction is solid; intent cannot be proven from the manuscript alone.

Verdict

🔴 Confirmed fabrication / strong prima facie evidence. Independent numerical statements inside a single manuscript contradict each other on the headline metric (ChestX-ray14 AUC at 1% data). Taken together with a counter-intuitive negative-ablation on a proposed core module, the manuscript fails basic internal-consistency checks expected of an ACM MM paper.

Key findings

Spatial–temporal collision between headline table and ablation tables. The full model ('Ours') on ChestX-ray14 (1%) is reported as 81.4 AUC in Table 1, but the identical configuration is reported as 80.8 AUC in Table 4 (1% subset footnote) and Table 5 (same setting, MGCA prior).
Section 5.1 text independently corroborates 80.8, not 81.4. The paper states the framework achieves a 2.0% AUC improvement over MRM on ChestX-ray14 (1%). Table 1 lists MRM at 78.8, so 78.8 + 2.0 = 80.8, matching the ablation tables but not the headline 81.4.
Negative ablation on a core contribution. In Table 4, DKBA alone scores 81.0, while DKBA + PAR drops to 80.8. Section 5.3.1 claims PAR and DKBA are complementary; the table indicates PAR slightly hurts the final model on this benchmark.
Citation mis-attribution (Finding 4). Section 2.1 assigns the MedUnic description to reference [24], but [24] corresponds to Liu et al., 'Improving medical vision-language contrastive pretraining with semantics-aware triage,' IEEE TMI 2023 (SAT). The actual MedUnic reference is [34] (Wan et al., NeurIPS 2024).
Unexplained missing Med-Unic metrics on CXR14 (Finding 5). The note 'Due to the lack of Med-Unic official checkpoints' leaves CXR14 blank but reports Med-Unic on CXP, RSNA, and COVIDx; the stated reason does not fully match the reporting pattern. Motive cannot be proven from the manuscript.

Evidence highlights

Table 1 (main): 'Ours' on ChestX-ray14 (1%) = 81.4 AUC.
Table 4 (ablation, 1% sub-setting per footnote): ✓PAR + ✓DKBA = 80.8 AUC.
Table 5 (ablation over priors, 1% data): best prior MGCA + full model = 80.8 AUC.
Section 5.1, paragraph 1: '…our framework achieves a 2.0% AUC score improvement over MRM.' MRM in Table 1 = 78.8; 78.8 + 2.0 = 80.8, confirming the ablation value rather than the headline value.
Table 4 (ChestX-ray14 column), DKBA alone = 81.0, versus DKBA + PAR = 80.8 (Δ = −0.2).
Section 2.1: '[24]' mislabeled as MedUnic; reference list [24] is Liu et al., IEEE TMI 2023 (semantics-aware triage / SAT).

Notes

The contradictions are arithmetic and reproducible from the manuscript text; intent (deliberate fabrication vs. careless copy-editing/AI-assisted rewriting) cannot be determined from the manuscript alone.
Confidence is high on Findings 1–4 (internally provable). Finding 5 is plausible but lacks direct evidence and should be treated as speculative.
DOI: 10.1145/3746027.3755336
Recommended follow-up: request raw training logs for the ChestX-ray14 1% configuration from the authors; file a PubPeer comment; notify the ACM MM 2025 program committee / Area Chair. Final adjudication rests with the conference investigation.