From Interpretability Methods to Interpretable Models: A Position Paper on Explainable AI in Computer Vision

Forum topic · 小凯 · 2026-09-08

Summary

A new arXiv position paper (2609.05399) by Julien Colin, Nuria Oliver, and Thomas Serre argues that explainable AI (XAI) research in computer vision should shift its focus from building interpretability methods to making models themselves interpretable. The authors observe that after more than a decade, XAI has assembled a mature toolbox—covering attribution, feature visualization, concept-based, and circuit-based methods—but nearly all effort has gone into constructing and comparing these tools rather than answering the questions they were meant to address: how interpretable are our models, and is interpretability actually improving as models evolve? The paper proposes progress along two complementary lines. First, an achievable near-term goal: using existing tools to characterize and compare what different models represent and compute. Second, a harder and largely neglected goal: assessing whether a model can genuinely be understood by the humans who rely on it—the independent evaluators on whom trust and certification depend. The work is relevant to researchers in computer vision and XAI.

Paper Overview

Field: Computer Vision (CV)
Authors: Julien Colin, Nuria Oliver, Thomas Serre
Published: 2026-09-04
arXiv: 2609.05399

Key Arguments

More than a decade into explainable AI (XAI) for computer vision, the field has assembled a mature toolbox of methods:

Attribution methods
Feature visualization
Concept-based methods
Circuit-based methods

However, the authors argue that almost all of the field's effort has gone into building and comparing these methods, and little into the question these methods were meant to answer: how interpretable are our models, and are we making progress as they evolve?

Proposed Shift: From Methods to Models

The paper argues for shifting the field's focus from interpretability *methods* to interpretable *models*, along two complementary lines:

1. Within reach: Existing tools already allow us to characterize and compare what different models represent and compute.

2. Harder, largely neglected: Whether a model can actually be understood by the humans who rely on it—the independent evaluators on whom trust and certification depend.

Original Abstract (excerpt)

> More than a decade in, explainable AI (XAI) for computer vision has assembled a mature toolbox: attribution, feature visualization, concept-based, and circuit-based methods. Yet almost all of the field's effort has gone into building and comparing these methods, and little into the question they were meant to answer---how interpretable are our models, and are we making progress as they evolve? We argue for shifting the field's focus from methods to models, along two complementary lines. One is already within reach: existing tools let us characterize and compare what different models represent and compute. The other is harder, and largely neglected: whether a model can actually be understood by the humans who rely on it---the independent evaluators on whom trust and certification depend...

Read the full paper on arXiv: 2609.05399