From Interpretability Methods to Interpretable Models: A Position Paper on the Future of XAI in Computer Vision

Forum topic · 小凯 · 2026-09-08

Summary

A position paper by Julien Colin, Nuria Oliver, and Thomas Serre (arXiv:2609.05399) argues that explainable AI (XAI) research in computer vision has over-invested in building and comparing interpretability methods—attribution, feature visualization, concept-based, and circuit-based approaches—while neglecting the core questions those methods were meant to answer: how interpretable are our models, and is interpretability improving as models evolve? The authors propose shifting the field's focus from methods to models along two complementary lines. The first is immediately achievable: using existing tools to characterize and compare what different models represent and compute. The second is harder and largely neglected: assessing whether a model can genuinely be understood by the humans who rely on it, especially the independent evaluators on whom trust and certification depend. The paper reframes XAI evaluation around model-level interpretability and human understanding rather than method benchmarks.

Paper Overview

Research Area: Computer Vision (CV) Authors: Julien Colin, Nuria Oliver, Thomas Serre Published: 2026-09-04 arXiv: 2609.05399

Abstract (translated/summarized from the original)

More than a decade in, explainable AI (XAI) for computer vision has assembled a mature toolbox: attribution, feature visualization, concept-based, and circuit-based methods. Yet almost all of the field's effort has gone into building and comparing these methods, and little into the question they were meant to answer—how interpretable are our models, and are we making progress as they evolve?

The authors argue for shifting the field's focus from methods to models, along two complementary lines:

1. Within reach: existing tools already let us characterize and compare what different models represent and compute.

2. Harder and largely neglected: whether a model can actually be understood by the humans who rely on it—the independent evaluators on whom trust and certification depend.

Key Takeaways

XAI method development has outpaced actual model interpretability assessment.
The community should evaluate and compare models' interpretability, not just interpretability methods.
Human understanding of models (especially for independent certification and trust) remains an open, under-addressed challenge.

---

*Auto-collected on 2026-09-08. Original Chinese forum post via zhichai.net.*