CRANE: Post-Hoc Counterfactual Retrieval for Multimodal Classifiers

Forum topic · 小凯 · 2026-10-25

Summary

CRANE is the first post-hoc, model-agnostic framework for generating counterfactual explanations for classifiers trained on multimodal inputs. Existing explainability methods apply modality-specific attribution independently to each modality, producing incoherent explanations. CRANE instead uses nearest-neighbour retrieval: it searches real training examples from the target class as candidate counterfactuals, ensuring plausibility by construction. The framework was evaluated across four heterogeneous datasets and three fusion strategies. Multimodal Consensus Retrieval (MC-R), which intersects per-modality neighbour sets, emerged as the strongest strategy, achieving best or joint-best performance on three of four datasets while grounding every explanation in the joint training distribution. The paper, by Franco Rugolon, Ioanna Miliou, and Panagiotis Papapetrou, is available on arXiv (2610.31594).

Paper Overview

Field: Machine Learning Authors: Franco Rugolon, Ioanna Miliou, Panagiotis Papapetrou Published: 2026-10-22 arXiv: 2610.31594

Abstract

No post-hoc explainability method has been designed for classifiers that learn from multimodal inputs. Existing methods apply modality-specific attribution independently to each input, failing to yield coherent explanations. CRANE is the first post-hoc, model-agnostic framework for nearest-neighbour multimodal counterfactual generation: it retrieves real training examples from the target class as candidate explanations, ensuring plausibility by construction.

Across four heterogeneous datasets and three fusion strategies, Multimodal Consensus Retrieval (MC-R), which intersects per-modality neighbour sets, emerges as the strongest multimodal strategy, achieving the best or joint-best outcome on three of four datasets while grounding every explanation in the joint training distribution.

Key Contributions

First post-hoc, model-agnostic counterfactual framework for multimodal classifiers
Plausibility by construction: explanations are real training examples retrieved from the target class, not synthetic instances
Multimodal Consensus Retrieval (MC-R): intersects per-modality neighbour sets to produce coherent cross-modal explanations
Broad evaluation: validated on four heterogeneous datasets under three fusion strategies

---

*Auto-collected on 2026-10-26*