LLMs Get Lost in Multi-Turn Conversation: Why AI Performance Drops 39% in Multi-Turn Chats

Forum topic · 小凯 · 2026-05-22

Summary

A Chinese tech forum post analyzes the ICLR 2026 Outstanding Paper 'LLMs Get Lost In Multi-Turn Conversation' by Microsoft Research and Salesforce Research. Using a 'Sharded Simulation' method that splits single-turn requests into multiple incremental turns, the researchers found LLM performance drops up to 39%: overall accuracy fell from 92.4% to 53.5%, code generation from 88.1% to 45.2%, and math reasoning from 76.5% to 39.8%. Notably, model aptitude declined only ~16%, while unreliability (inconsistency) surged — an effect the authors call 'Intelligence Collapse.' The post identifies three root causes: premature commitment to early assumptions (path dependency), verbosity inflation that pollutes context with the model's own filler text, and an amplified lost-in-the-middle effect. The proposed fix is to have the model first collect requirements into a complete spec, then start a fresh session to execute it. Paper reference: arXiv:2505.06120.

Introduction

LLMs impress in single-turn tasks — writing code, answering questions, discussing philosophy. But in multi-turn conversations, they often seem to 'lose their soul.' A recent paper from Microsoft Research and Salesforce Research, 'LLMs Get Lost In Multi-Turn Conversation' (ICLR 2026 Outstanding Paper Award), pinpoints this problem as Intelligence Collapse: a non-linear degradation of output quality as interaction turns increase in sequential logic tasks.

Key Findings

The researchers introduced Sharded Simulation, splitting a requirement expressible in one sentence into 7–8 fragments delivered over multiple turns:

| Metric | Single-Turn (Full) | Multi-Turn (Sharded) | Drop | | :--- | :---: | :---: | :---: | | Overall Accuracy | 92.4% | 53.5% | -38.9% | | Code Generation | 88.1% | 45.2% | -42.9% | | Math Reasoning | 76.5% | 39.8% | -36.7% |

Crucially, the drop is driven less by lost capability than by lost reliability:

$Reliability \downarrow = \Delta Aptitude (16\%) + \Delta Consistency (112\%)$

Models don't become less knowledgeable — they become wildly inconsistent, sometimes right, sometimes wrong.

Three Root Causes

1. Premature Commitment — Models rush to make assumptions and produce solutions after only fragmentary early input, locking into wrong paths early. Even when later turns correct the course, the underlying logic remains trapped (path dependency).

2. Verbosity Inflation — Politeness-driven filler text generated by the model itself pollutes the context window, lowering the signal-to-noise ratio until the model can't distinguish user instructions from its own rambling.

3. Enhanced Lost-in-the-Middle — Despite long context windows, models attend heavily to the first and last turns while key constraints in the middle turns get ignored, with very low citation rates.

The Fix: Aggregate Before Acting

The paper's remedy: don't let the model think and act simultaneously across turns. Instead, have it act as a requirements collector — gather all fragments into a complete specification, then start a fresh session where the model executes against that spec. This recovers most of the lost performance.

Paper Reference

Title: *LLMs Get Lost In Multi-Turn Conversation*
Authors: Philippe Laban (Microsoft Research), Hiroaki Hayashi (Salesforce Research), Yingbo Zhou, Jennifer Neville
Venue: ICLR 2026 Outstanding Paper Award
Link: arXiv:2505.06120
Institutions: Microsoft Research & Salesforce Research

Bottom line: Even top-tier LLMs lose ~39% of their performance under incremental, multi-turn interaction. When working with AI, prefer consolidating requirements into a complete spec before executing.