Summary
A paper by Leyla Roksan Caglar, Pedro A. M. Mediano, and Baihan Lin (arXiv:2604.21909, posted 2026-04-23) examines how humans and deep vision models differ in the types of classification errors they make, beyond matching accuracy rates. Using response data from matched human and deep visual models on natural image classification across 12 perturbation types, the authors quantify asymmetries in confusion matrices and link them to generalization geometry via a rate-distortion (RD) framework, summarized by three geometric features: slope (beta), curvature (kappa), and efficiency (AUC). Key findings: humans show broad but weak asymmetries, while deep vision models exhibit sparser, stronger directional collapses; robustness training reduces global asymmetry but fails to restore the human-like breadth-strength distribution of graded similarity; mechanistic simulations show different asymmetry organizations shift the RD frontier in opposite directions even when performance is matched. The work positions directional confusions and RD geometry as compact, interpretable signatures of inductive bias under distribution shift.
Paper Overview
Field: Computer Vision (CV)
Authors: Leyla Roksan Caglar, Pedro A. M. Mediano, Baihan Lin
Posted: 2026-04-23
arXiv: 2604.21909
Abstract (Translation)
Humans and modern vision models can achieve similar classification accuracy, yet they make systematically different types of errors — the difference lies not in error frequency, but in who is mistaken for whom, and in which direction. The authors show that these directional confusions reveal divergent inductive biases that accuracy alone cannot see.
Using matched human and deep visual model responses on natural image classification tasks across 12 perturbation types, the study quantifies asymmetries in confusion matrices and connects them to the geometry of generalization through a rate-distortion (RD) framework, summarized by three geometric features: slope (β), curvature (κ), and efficiency (AUC).
Key findings:
- Humans exhibit broad but weak asymmetries, while deep vision models show sparser, stronger directional collapses.
- Robustness training reduces global asymmetry, but cannot recover the human-like breadth-versus-strength distribution of graded similarity.
- Mechanistic simulations show that different organizations of asymmetry move the RD frontier in opposite directions, even when performance is matched.
These results position directional confusions and RD geometry as compact, interpretable signatures of inductive bias under distribution shift.
---
*Auto-collected on 2026-04-27*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177618801