LLMs Get Lost in Multi-Turn Conversation: Inside the 39% Performance Drop (ICLR 2026)

Forum topic · 小凯 · 2026-05-22

Summary

A Chinese tech forum post discusses the ICLR 2026 paper 'LLMs Get Lost In Multi-Turn Conversation' by researchers from Microsoft and Salesforce. Using a 'Sharded Simulation' method that splits single-sentence requirements across multiple turns, the study finds LLM performance drops by an average of 39% (accuracy 92.4% to 53.5%, code generation down 42.9%, math reasoning down 36.7%). Notably, model aptitude falls only about 16%, while reliability collapse stems from a 112% surge in inconsistency—the models become unstable rather than less capable. The post identifies three root causes: premature commitment to early assumptions (path dependency), verbosity inflation that pollutes the context window with low-signal filler, and an enhanced 'lost in the middle' effect where mid-conversation details are ignored. The recommended fix is to collect requirements first, compile them into a full specification, then execute in a fresh session. The paper earned an ICLR 2026 Outstanding Paper Award; authors include Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville.

Digital Maze: When LLMs Lose Their Soul in Multi-Turn Chat

Opening: A Sudden 'IQ Waterloo'

Large language models are famously capable—writing code, discussing poetry and philosophy. But while they excel in single-turn conversations, they often become confused and unreliable in multi-turn dialogue or group chats.

A recent paper from Microsoft and Salesforce researchers, 'LLMs Get Lost In Multi-Turn Conversation' (ICLR 2026, Outstanding Paper Award), pinpoints this phenomenon: virtually all models fall victim to the 'multi-turn curse.'

> Intelligence Collapse: the non-linear degradation of output quality as a system processes logical tasks across an increasing number of interactions.

The Data: A 39% Cliff

The researchers introduced Sharded Simulation: taking a requirement that fits in one sentence and splitting it into seven or eight pieces, delivered turn by turn.

The result? When models receive this 'toothpaste-squeeze' style input, performance drops by 39%—like telling a chef to fetch meat, then to cut it, then to add sugar, one step at a time.

| Dimension | Single-turn (Full) | Multi-turn (Sharded) | Drop | | :--- | :---: | :---: | :---: | | Overall accuracy | 92.4% | 53.5% | -38.9% | | Code generation | 88.1% | 45.2% | -42.9% | | Math reasoning | 76.5% | 39.8% | -36.7% |

More strikingly, the study found that a model's baseline aptitude drops by less than 20%—what really collapses is reliability. The model isn't incapable; it becomes wildly inconsistent.

\text{Reliability} \downarrow = \Delta \text{Aptitude (16\%)} + \Delta \text{Consistency (112\%)}

Three Root Causes

1. Premature Commitment

Models rush to act on incomplete information. By turns two or three, they've already made assumptions—like a fortune teller pronouncing your fate before you give your birth date. Once the initial direction is wrong, later corrections barely help: the model agrees verbally, but its underlying logic is stuck. This is path dependency.

2. Verbosity Inflation

In multi-turn exchanges, models pad their replies with polite filler. This self-generated noise pollutes the context window, and the model eventually can't distinguish user instructions from its own fabricated text—the signal-to-noise ratio plummets.

3. Enhanced Lost-in-the-Middle

Despite claims of million-token context windows, models in long conversations remember only the beginning and end. Key constraints mentioned mid-conversation are cited at alarmingly low rates.

The Fix: Don't 'Think and Do' at Once

The authors propose: summarize before acting. Instead of grinding through multi-turn back-and-forth, have the model first act as a 'requirements collector.' Once all fragments are gathered, compile them into a complete specification, then start a fresh session and execute against that spec. This recovers most of the lost performance.

Paper References

Title: *LLMs Get Lost In Multi-Turn Conversation*
Authors: Philippe Laban (Microsoft Research), Hiroaki Hayashi (Salesforce Research), Yingbo Zhou, Jennifer Neville
Venue: ICLR 2026 Outstanding Paper Award
Link: arXiv:2505.06120
Institutions: Microsoft Research & Salesforce Research
TL;DR: Reveals the 'reliability collapse' common in incremental interactions, showing even top LLMs lose 39% of performance due to the structural interference of multi-turn conversation.

--- *All cited data are based on the paper's reported measurements. The goal is to expose the fragility of digital minds—not to disparage them. Stay clear-eyed, not blindly trusting, when working with models.*