[论文] Label-Wording Augmentation in a Typed-Decision Encoder: Gains on Unfam...

论文概要

研究领域: NLP
作者: Vladimir Beskorovainyi
发布时间: 2026-11-05
arXiv: 2611.08439

中文摘要

类型决策模型对调用方在请求时提供的选项进行打分，因此其精度取决于模型开发者无法控制的标签措辞。我们比较了一个 307M 参数多语言交叉编码器 Ru-Jev 的两个版本——它在十三个俄语任务上训练——较新的版本增加了一种数据增强，以多种表面形式呈现标签。固定问题不变，仅改变十八个路由标签的措辞方式，我们评估了十套标签词汇表，由三个独立流程生成并检查了与训练数据的重叠。在五个候选上使用全部十八个标签，其中八个问题还额外测试了十八个候选。在由普通词或外来词（英语衍生术语、办公俚语、部门名称）组成的留出词汇表上，增强版本在所有设置下都更精确，最高提升 0.24。十四项增益中十一项通过 Holm 校正（俚语十八候选和描述句五候选未通过），术语在所有标签上两种设置的标签级区间均不含零。在缩写和不透明代码上没有可用增益：缩写上不更好，在干净切片上更差；不透明代码在两个版本移除与训练选项相关的标签后都不高于随机水平。在大多数精度提升的单元格中，风险-覆盖面积下降，但十八候选时术语和俚语上的自信错误增加。在两个两个版本都未见过的数据集上，精度分别变动 -8.4 和 +5.6 个百分点。我们还报告了比较两个版本时的四个评估陷阱。

原文摘要

Typed-decision models score options that the caller supplies at request time, so their accuracy depends on label wording that the model's developer does not control. We compare two releases of a 307M-parameter multilingual cross-encoder, Ru-Jev, trained on thirteen Russian-language tasks; the later release adds an augmentation that presents labels in several surface forms. Holding questions fixed and varying only how eighteen routing labels are worded, we evaluate ten label vocabularies, produced by three separate processes and checked for overlap with the training data, with five candidates per question and, for eight of them, also with eighteen. On held-out vocabularies made of ordinary or borrowed words (English-derived jargon, office slang, department names), the augmented release is more accurate in every setting, by up to 0.24. Eleven of these fourteen gains survive a Holm correction (slang with eighteen candidates and descriptive sentences with five do not), and for jargon on all labels the label-level interval excludes zero in both settings. On abbreviations and opaque codes it gives no usable gain: on abbreviations it is not better, and worse on the clean slice, and on opaque codes both releases stay at or below chance once labels related to training options are removed. In most cells where accuracy rises, the risk-coverage area falls, but with eighteen candidates confident errors rise on jargon and slang. On two datasets neither release has seen, accuracy moves by -8.4 and +5.6 points. We also report four evaluation hazards from comparing the two releases.

自动采集于 2026-11-08

#论文 #arXiv #NLP #小凯