Geng Investigation Report: Enhancing representation in radiography-reports foundation model (Nature Communications, 2024)

Academic fraud report · Geng Detector

Summary

This report evaluates a 2024 Nature Communications paper (DOI: 10.1038/s41467-024-51749-0) proposing MaCo, a masked contrastive learning framework for radiograph-report foundation models. Verdict: highly suspicious. Three issues were flagged. First, Table 4 reports that an ImageNet-pretrained ResNet baseline on the RSNA detection fine-tuning task drops from mAP 12.4 (10% labels) to 8.0 (100% labels), a counter-intuitive degradation unaddressed in the manuscript. Second, Table 3 shows MaCo exceeding MedKLIP on the small (~200-image) COVID Rural segmentation set by 32.9% (10%) and 31.1% (100%) Dice, a margin considered implausible for same-architecture (ViT) pretraining comparisons. Third, References 8, 10, and 13 cluster 2024 IEEE ISBI self-citations on closely related topics. Severity ratings: two issues marked red/orange, one yellow. Limitations: numerical claims are taken from the report; raw logs, training code, and seed configurations were not independently verified, and final misconduct determinations require institutional investigation.

Verdict

Highly suspicious (orange). The manuscript contains multiple anomalies that, taken together, warrant formal inquiry by the editorial office, but no finding is independently confirmed beyond the reported numbers.

Key findings

Counter-intuitive baseline regression on RSNA detection (Table 4): ImageNet-pretrained ResNet baseline mAP reportedly falls from 12.4 at 10% labels to 8.0 at 100% labels, i.e., roughly a one-third relative drop when supervision is increased tenfold. The authors do not discuss this in the text.
Implausibly large segmentation gains on COVID Rural (Table 3): MaCo Dice reported as 68.3 (10%) and 75.1 (100%), versus MedKLIP 35.4 and 44.0, i.e., absolute gains of +32.9 and +31.1 Dice points. The COVID Rural set is small (~200 images), making such margins inconsistent with typical ViT-pretraining improvements.
Dense ISBI 2024 self-citations (References 8, 10, 13): Multiple overlapping co-authors (Huang W., Liu J., Yang H.) cite their own freshly accepted ISBI 2024 papers on closely related topics, raising concerns about citation padding rather than methodological dependence.

Evidence highlights

DOI: 10.1038/s41467-024-51749-0
Table 4, Fine-tuning detection (RSNA): ImageNet baseline mAP 12.4 (10% labels) → 8.0 (100% labels).
Table 3, Fine-tuning segmentation (COVID Rural): MedKLIP Dice 35.4 (10%) / 44.0 (100%); MaCo Dice 68.3 (10%) / 75.1 (100%); absolute gains +32.9 and +31.1.
References list includes Liu et al. ISBI 2024 (Ref 8), Yang et al. ISBI 2024 (Ref 10), Huang et al. ISBI 2024 (Ref 13), with overlapping authorship.
Received date: 15 August 2024.

Notes

The mAP regression and the ~30-point Dice jump are reported as observed in the published tables; this report has not independently re-run the experiments or inspected training logs, checkpoints, or seeds.
Possible benign explanations (e.g., mislabeled baselines, broken evaluation pipeline, learning-rate/schedule not tuned for the ResNet baseline, cherry-picked MedKLIP checkpoint) cannot be ruled out without code, configuration files, and per-seed outputs.
The ISBI self-citations are a soft signal; dense self-citation alone is not evidence of misconduct but is relevant context given the other anomalies.
Per the disclaimer, final determinations of fabrication or selective reporting require investigation by the journal and/or the authors' institution. Suggested follow-ups include requesting raw training logs, baseline hyperparameter and seed disclosures, and posting structured queries on PubPeer.