[论文] CritICL: Inference-Time Weak-to-Strong Generalization from Small Langu...

论文概要

研究领域: NLP
作者: Yufan Wu, Yinghui He, Zhengyi Hu, Lang Wei, Ruichen Li, Qifan Yang, Ting Zhu
发布时间: 2026-08-27
arXiv: 2608.27455

中文摘要

推理时间扩展的最新进展显著提升了大语言模型（LLMs）的推理性能，但这些方法通常依赖重复生成或外部验证。为解决这一局限，我们提出了CritICL，一种新颖的推理时间框架，在保持高效率的同时提升推理能力。核心洞察是：LLM的失败模式在同一模型族内呈现跨规模的结构化模式。CritICL不将失败视为不受欢迎的输出，而是将其作为指导来源。具体而言，我们利用较弱模型的失败模式，并通过基于批判的上下文示例将其融入推理过程。我们提出两个变体：CritICL-dynamic自适应预测输入特定的失败模式并检索批判；CritICL-static使用全局失败模式配置文件提供稳定指导。实验结果表明，CritICL始终优于标准上下文学习，性能与测试时间扩展方法相当或更优，同时需要的生成次数和token成本显著更少。

原文摘要

Recent advances in inference-time scaling have significantly improved the reasoning performance of large language models (LLMs). However, these methods typically rely on repeated generation or external verification. To address this limitation, we introduce CritICL, a novel inference-time framework that improves reasoning while maintaining high efficiency. Our key insight is that LLM failure modes exhibit structured patterns across model scales within the same family. Instead of treating failures as undesirable outputs, CritICL leverages them as a source of guidance. Specifically, we utilize failure modes derived from weaker models and incorporate them into inference through critique-based in-context examples. We propose two variants: CritICL-dynamic, which adaptively predicts input-specifi...

自动采集于 2026-08-30

#论文 #arXiv #NLP #小凯