[论文] Inside the Unfair Judge: A Mechanistic Interpretability Account o...

论文概要

研究领域: ML
作者: Zixiang Xu, Sixian Li, Huaxing Liu, Xiang Wang, Shuai Li
发布时间: 2026-07-13
arXiv: 2607.11871

中文摘要

现有LLM-as-judge评分偏差研究主要在输入-输出层面工作：扰动输入、测量分数变化并提出提示层缓解方案。本文认为这些偏差同样可在裁判的隐藏状态中获得表征层面的解释，与输入-输出视角互补且在操作上更有用。跨七个裁判模型、七种偏差类型和九个基准报告了三项发现。几何结构：基线裁判输入占据紧凑的激活流形，而偏差输入沿低维类型特定子空间位移，该子空间随深度锐化且被三类估计器一致恢复。因果控制：沿此子空间操控隐藏状态可双向驱动评分，正向偏移在干净输入上复现有偏评分，反向偏移在偏差输入上恢复基线评分，而匹配的随机方向产生小一个数量级的偏移。操作性：向同一偏差方向特征的简单线性投影可预判裁判在三个完全未见基准上的失败，显著优于基于文本的替代方案。将偏差读取为激活几何而非输入-输出噪声，在单一框架内统一了几何结构、因果控制和操作预测。

原文摘要

Existing studies of LLM-as-judge scoring bias work predominantly at the input-output level: they perturb inputs, measure score deltas, and propose prompt-level mitigations. We argue that the same biases admit a representation-level account in the judge's hidden state, complementary to the input-output view and operationally useful in ways it does not afford. We report three findings, across seven judges, seven bias types, and nine benchmarks. Geometry: baseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth and is recovered consistently by three families of estimators. Causal control: steering hidden states along this subspace drives scoring in both directions, forward shifts repro...

自动采集于 2026-07-15

#论文 #arXiv #ML #小凯