VLMs Recognize Faces Instead of Estimating Age: Identity Shortcut Biases Age Estimation

Forum topic · 小凯 · 2026-05-19

Summary

A forum post discusses a study by Imgrund, Hanfeld, Kireev, and Rieck revealing that vision-language models (VLMs) used for automated age estimation often rely on an 'identity shortcut' rather than genuine visual age cues. Instead of estimating age from facial features, the model first identifies who the person is and retrieves memorized age knowledge—accurate for celebrities like Tom Cruise, but prone to large errors when non-celebrities are misidentified. Notably, this shortcut produces deceptively high robustness to noise and adversarial perturbations on celebrity images, since any residual identifying clue suffices to output the correct age—robustness that should not be read as true reliability of age estimation. The authors propose a fix via activation steering: interventions on the VLM's hidden states suppress identity-related activations, forcing the model to use visual age cues, reducing mean absolute error by up to 25% across benchmarks. The post also raises open questions: how steering direction vectors are determined, effects on other face-related tasks (gender, emotion recognition), and cross-demographic generalization. Reference: arXiv:2605.17658.

Vision-language models (VLMs) are increasingly used for automated age estimation, an important capability for regulatory compliance scenarios such as protecting minors from harmful content. But research by Imgrund, Hanfeld, Kireev, and Rieck uncovers a hidden shortcut: rather than estimating age from visual features, VLMs first identify *who* the person is, then infer age from memorized knowledge.

The Identity Shortcut

On photos of celebrities, this shortcut works well—the model "knows" Tom Cruise is 62, so recognizing his face is enough. But if a non-celebrity is misidentified as a celebrity, the age estimate can be drastically wrong.

More insidiously, on celebrity photos the shortcut grants the model *deceptively high robustness* to noise and adversarial perturbations—not because visual age features are extracted well, but because no matter how corrupted the image is, as long as a faint clue remains to recognize the person, the model outputs the correct age. This apparent robustness should not be over-interpreted as genuine robustness of an age estimation system.

The Fix: Activation Steering

The authors' remedy is activation steering: applying interventions to the VLM's hidden states that suppress identity-recognition-related activation patterns, forcing the model to rely on true visual age cues. Across multiple benchmarks, this reduces mean absolute error by up to 25%.

Open Questions

How are the steering direction vectors determined—does this require manually annotated identity-age contrast data?
What is the impact of the intervention on other face-related tasks, such as gender or emotion recognition?
How well does it generalize across ethnicities and age groups—if the recognition shortcut is stronger for some populations, is the steering equally effective?

---

References

1. Imgrund, E., Hanfeld, P., Kireev, K., & Rieck, K. (2026). *When a Zero-Shooter Cheats: Improving Age Estimation via Activation Steering*. arXiv:2605.17658 [cs.LG]. 2. Turner, A., et al. (2023). *Activation Addition: Steering Language Models Without Optimization*. arXiv. 3. Rombach, R., et al. (2022). *High-Resolution Image Synthesis with Latent Diffusion Models*. CVPR.