Paper Slam 4/25: When AI Starts to 'See' — Diagnosing Lesions in Video and Restoring Hallucinated Photos

Forum topic · 小凯 · 2026-04-28

Summary

This forum post reviews two arXiv papers (2604.21814 and 2604.21879) that tackle opposite sides of the same question: what happens when AI mediates how machines and people 'see'? The first, DiCE (Divide-then-Diagnose) from HKUST and Renji Hospital, addresses ultra-long capsule endoscopy videos (8-12 hours, ~100,000 frames, fewer than 10 diagnostically relevant). It reframes the task as diagnosis-driven video summarization, introduces the VideoCAP dataset (240 full-length videos, 7.24 million frames, report-based annotations, 12 lesion classes), and uses a three-stage pipeline: a lightweight Selector, a Context Weaver with two-level hierarchical clustering, and an Evidence Converger for multi-frame voting. With a 0.2B DINOv3 backbone, it beats 8B-parameter MLLM baselines (lesion detection rate 44.12%, diagnostic yield 20%). The second paper, from the University of Toronto and Samsung AI Center, tackles hallucinations introduced by on-camera generative AI ISPs (AI super-resolution, low-light enhancement) that can alter text and semantics. It embeds ~180KB of per-image metadata — a lightweight encoder plus a tiny image-specific MLP — enabling restoration of the pre-AI 'unhallucinated' image in ~3 seconds without ISP access, outperforming blind restoration baselines across super-resolution and low-light tasks. The post compares both via a Feynman-style lens, arguing that understanding problem structure matters more than model scale.

This post is a structured review and comparison of two papers that both confront the question: when AI intervenes in how we 'see', is it revealing truth or fabricating illusions?

Paper 1: DiCE — Divide-then-Diagnose (arXiv:2604.21814)

Problem. Capsule endoscopy videos run 8-12 hours (~100,000 frames) with fewer than 10 diagnostically relevant frames. Even with AI-assisted frame screening, clinician review still exceeds 1 hour, and only ~8% of AI-flagged frames contain meaningful lesions. Existing AI works frame-by-frame, ignoring context.

Contribution. The authors redefine the task as *diagnosis-driven CE video summarization* and build VideoCAP: 240 full-length videos (7.24M frames) from two clinical centers at Renji Hospital, annotated from clinical diagnostic reports (not per-frame anomaly labels) with 12 lesion classes.

Method (three stages, mimicking clinician workflow):

Selector: lightweight frozen DINOv3 (0.2B) + MLP classifier with high recall to filter ~100K frames into candidates.
Context Weaver: two-level hierarchical clustering — coarse *anatomical context anchoring* (cover different gut regions) and fine *lesion context refinement* (group frames of the same lesion).
Evidence Converger: sums softmax distributions within each context, prunes inconsistent frames and low-confidence contexts, and outputs a representative frame (medoid) with an aggregated label. Key motivation: baseline methods show 46-65% label disagreement between adjacent keyframes within 30 seconds — single-frame predictions are nearly unusable.

Results. Lesion Detection Rate 44.12% (best baseline 35.29%), Sensitivity 85.29%, lowest temporal error (54.86s) and redundancy (77.81%), Diagnostic Yield 20% (baselines 12.5-15%). Ablations: replacing hierarchical clustering with 300s time windows drops LDR by 22.76%; replacing multi-frame voting with single-frame confidence drops Diagnostic Yield from 20% to 2.5%. Adjacent-keyframe label inconsistency falls from 46-65% to 5.1-10.7% (8-9x improvement).

Caveats. VideoCAP labels are retrospective matches to clinical reports, so the model may learn to reproduce reports rather than make independent diagnoses. Diagnostic Yield of 20% means doctors still need full review in practice.

Paper 2: Addressing Image Authenticity When Cameras Use Generative AI (arXiv:2604.21879)

Problem. On-device AI ISPs (digital zoom super-resolution, low-light enhancement) can hallucinate content at capture time — changing a blurred license-plate "RM" into "AN", altering character colors. Existing authentication tools detect *post-capture* tampering; prior work (Punnappurath et al., 2024) only flags suspicious pixels via metadata masks without restoring them.

Method.

Key insight: hallucinations arise mainly from the *enhancement* end of the ISP pipeline; the intermediate image before AI enhancement is still faithful.
Blind restoration fails because mappings like low-light enhancement are one-to-many (the true darkness is ambiguous).
Solution: per-image restoration. A frozen modality-specific encoder (~127KB, NAFNet-based) maps the enhanced image y to latent features w; a tiny two-layer MLP (~53KB) is finetuned per image (~1000 iterations, ~3s on V100) to predict a residual: x̂ = y − Θ([x, y, w]). Reconstruction L2 loss only — no GAN/perceptual losses, which would themselves hallucinate.
Total ~180KB of parameters embedded in JPEG/HEIC metadata; restoration requires no ISP access.

Results. PSNR 32.96 dB on DIV2K 4x SR (RealESRGAN), 31.26 dB on MARCONet text SR, 36.34 dB on LOL low-light — beating SIREN, NeRF, hashgrid, and a 64MB blind NAFNet. Also beats JPEG residual compression at equal metadata size (and its size is resolution-independent). Qualitatively recovers "GQB" from "608" and an "i" from "l".

Caveats. Requires paired access to the pre-enhancement intermediate image (realistic only for camera vendors); metadata tamper-resistance (encryption/watermarking) is left open; future end-to-end AI ISPs may eliminate the intermediate point the method relies on.

Comparison and Takeaways

Shared structure: extremely low signal-to-noise ratio, high cost of errors, and failure of conventional approaches. Both solutions aggregate evidence in *context* rather than trusting single observations — DiCE via spatio-temporal clustering and voting, the second via a global encoder plus per-image refinement.
Key divergence: DiCE is *data/task-directed* (redefining the task and benchmark beats 8B models with 0.2B parameters); the authenticity paper is *model-directed* (a compact, deployable inverse-mapping scheme).
Opposite stances on AI trust: DiCE extends constructive trust in AI-assisted diagnosis; the second paper builds defensive tooling against AI's own fabrications.
Feynman-style conclusion: naming ≠ understanding. Both papers succeed by abandoning generality and confronting problem-specific structure — evidence that in some problems, *understanding the problem beats scaling the model*.

---

Paper index

"Divide-then-Diagnose: Weaving Clinician-Inspired Contexts for Ultra-Long Capsule Endoscopy Videos." arXiv:2604.21814 [cs.CV], April 2026.
Umar Masud, Abhijith Punnappurath, Luxi Zhao, David B. Lindell, Michael S. Brown. "Addressing Image Authenticity When Cameras Use Generative AI." arXiv:2604.21879 [cs.CV], April 2026.