Rational Synthesizers or Heuristic Followers? 卡内基梅隆大学最新研究揭开AI决策黑箱的裂痕

AI Rationality Research - CMU Study

CMU Research · AI Safety

Rational Synthesizers

or Heuristic Followers?

卡内基梅隆大学最新研究揭开AI决策黑箱的裂痕

AI 真的理性吗？

还是它只是一个读过很多书但耳根子很软的"复读机"？

深度解读论文

Analyzing LLMs in RAG-based Question-Answering

核心发现

Core Findings

我们以为 AI 是客观的法官，
但数据表明，它们更像是固执的"经验主义者"：

We assume AI acts as an objective judge, but data shows they are stubborn "heuristic followers":
the larger the model, the less plastic it becomes, and they are easily fooled by repetitive information.

四大关键发现

Four Key Discoveries

01
经验法则追随者

Heuristic Followers

我们打破了 AI 是"理性综合者"的幻想。研究发现，AI 在处理冲突信息时，往往不进行深度逻辑分析，而是依赖简单的统计捷径。

We shatter the illusion of AI as a "Rational Synthesizer." Research shows that when facing conflicting info, AI often skips deep logical analysis and relies on simple statistical shortcuts.

02
可塑性悖论

The Plasticity Paradox

一个反直觉的发现——模型参数规模越大（如 Llama-3 70B），反而越难以接受新的事实证据，表现出极强的"知识惯性"。

A counter-intuitive finding: the larger the model scale, the harder it is for it to accept new factual evidence, showing strong "knowledge inertia."

03
虚幻真相效应

Illusory Truth Effect

AI 竟然会被"车轱辘话"忽悠？实验证明，简单重复的冗余信息，比高质量的独立证据更能左右 AI 的判断。这意味着真相可以被"制造"。

Can AI be fooled by repetition? Experiments prove that simply repeating the same point persuades AI more than high-quality, independent evidence. Truth can be "manufactured."

04
思维链的伪装

The Disguise of Chain of Thought

你的 AI 可能在撒谎。其输出的推理过程往往只是事后的"公关稿"，用来掩盖它基于直觉和位置偏见的粗糙决策。

Your AI might be lying. Its output reasoning is often just a post-hoc "press release" designed to cover up crude decisions based on intuition and positional bias.

这不仅仅是技术瑕疵，更是对未来 AI 安全 的严峻警示。

This isn't just a technical glitch; it's a critical warning for the future of AI safety.

1,635

争议性问题

Controversial Questions

15,058

证据文档

Evidence Documents

GroupQA

数据集名称

Dataset

卡内基梅隆大学

Carnegie Mellon University

Research Paper
›
arXiv:2601.06189

Author: Atharv Naphade · LLM Research, Uncertainty, Reasoning, and AI Safety