Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap

*Full English translation of a zhichai.net forum post analyzing a Stanford paper on the "Lost-in-Conversation" phenomenon and the FiC self-training framework.*

The Everyday Crack We Overlook

Have you ever met someone like this — you ask "where should I go this weekend," and they immediately list ten attractions; then you add "my budget is only 200 yuan and I don't want to hike," but they've already been rambling about mountain-climbing strategies for half an hour. If you split the conditions across several sentences, they seem unable to hear the second half. Yet if you dump all the conditions into one paragraph from the start, they answer flawlessly. Same brain, two input styles, wildly different results. This isn't a story about people. It's the daily reality of almost every language model.

| Item | Detail | |------|--------| | Paper | Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap | | Authors | Tianlang Chen, Shirley Wu, Jure Leskovec | | Institution | Stanford University | | arXiv ID | 2605.24432 | | Submitted | May 23, 2026 | | Category | cs.CL | | Core findings | LLM performance drops ~39% on average in multi-turn underspecified dialogue (Lost-in-Conversation); the FiC framework (SFT warm-start + View-Asymmetric Self-Distillation) lets models self-recover single-turn ability without an external teacher; recovers ≥92% of single-turn performance across Llama/Qwen/Phi/OLMo (3B–14B), 100% on Llama backbones; 33% improvement in conversational token efficiency |

In the real world, few people state everything at once. You open a chat window, drop a vague request, get asked a few clarifying questions, add details, and go back and forth three or four rounds. That's the natural rhythm of human communication — economize effort, probe first, refine later.

LLMs don't see it that way. Pack all conditions into one long prompt and it answers well. Split the same conditions across three turns — half in the first, a bit more in the second, the rest in the third — and performance can collapse by a third. This gap is not a capability problem; it is a behavioral problem. The model has the skill inside, but when information arrives in batches, it doesn't know how to retrieve it.

This phenomenon has a name: Lost-in-Conversation. Work by Laban et al. in 2025 found that fifteen mainstream models (open and closed, large and small) lose an average of 39% on task scores in multi-turn underspecified dialogue. Give a math word problem with all conditions at once and it solves it; split the conditions across three turns, and it may eagerly guess a wrong assumption in turn one, then use that assumption as an anchor and drift further off course.

Chen, Wu, and Leskovec at Stanford asked a direct question: since the same model gets it right in a single turn, the capability is already there — can we make it teach itself to carry that competence into multi-turn settings?

Three Information-Equivalent Doors

First, "same information" had to be pinned down. The authors designed a controlled comparison: a single-turn instruction rendered in three information-equivalent forms.

• FULL: the original single-turn instruction, all conditions at once. The baseline where models perform best.
• CONCAT: still single-turn, but conditions broken into a bulleted list. Identical information, different layout. This tests whether models degrade from "fragmented reading."
• SHARDED: true multi-turn — first condition in turn one, second in turn two, and so on. Each turn alone is insufficient to solve the problem; together they equal FULL. This is where Lost-in-Conversation occurs.

Experiments show FULL and CONCAT perform essentially the same. So models don't degrade because information is chopped into pieces — they read bullet lists just fine. The real problem lies on the time axis: when information arrives in batches, what choices does the model make in early turns?

The Premature Commitment Trap

The culprit is what the authors call premature commitment. Given incomplete first-turn information, the model doesn't say "I'd need X and Y to answer"; instead it instinctively fills the gap with a hallucinated assumption and produces a plausible-looking answer. Worse, once voiced, the assumption becomes part of the conversational context — the model then reasons on top of its own error, sinking deeper.

Where does this come from? Likely from standard post-training objectives: models are trained to maximize immediate next-turn reward, i.e., to answer as fast as possible. But that objective is toxic when information is incomplete — it rewards "answering in a hurry" and punishes "carefully deferring."

The first stage of FiC targets exactly this. The authors warm-start with SFT on carefully constructed multi-turn corpora that train the model to defer answering when information is insufficient, request the specific missing information, and only answer once all necessary details have arrived.

The corpus construction itself is meticulous. Starting from a standard single-turn benchmark (e.g., GSM8K math problems), an LLM splits each problem's complete conditions into information shards — each insufficient alone, jointly equivalent to the original. Then a verification step: all shards are reassembled into a bullet list (CONCAT view), confirming model performance matches the FULL view. Only shard sets passing this check enter the training corpus.

The design of intermediate-turn gold answers is the subtle part. Rather than a fixed "please provide more information" template, the authors diversify along two dimensions. First, stylistic variety: deferral responses differ in format, wording, register, and length across samples and turns, ensuring the model learns the *behavior* of deferring rather than one specific phrase. Second, position-aware specificity: in early turns, with extremely sparse information, the model cannot know what's missing, so gold answers are generic deferrals ("I need more information"); in later turns, with richer context, gold answers name the specific missing pieces ("I need the speed and distance"). Every intermediate response is vetted by a judge to confirm it requests genuinely missing and necessary information, preventing the model from learning to deflect with stalling.

The final-turn gold answer isn't human-written — it comes from the model's own correct generations in the CONCAT view. This is elegant: it keeps the SFT supervision close to the model's own output distribution, avoiding distribution shift from "teaching yourself with someone else's words." Multiple correct CONCAT rollouts per problem are randomly sampled each training round, further preventing collapse onto a single "standard answer" phrasing.

Two Selves in the Same Mirror

SFT cures the "answering too early" habit, but multi-turn accuracy still trails single-turn. The remaining gap is closed by the paper's core invention: View-Asymmetric Self-Distillation (VASD). The idea is clean: run the same model under two views — the single-turn CONCAT view as teacher, the multi-turn SHARDED view as student — and pull the student's distribution toward the teacher's with token-level Jensen-Shannon divergence at the final-answer position.

Key design points:

1. Teacher and student share one backbone. No external teacher model is needed, because frontier models suffer the same multi-turn gap — hiring a nearsighted person as your eye doctor doesn't make sense. 2. Teacher and student have equal information. The teacher sees CONCAT (all information as a bullet list); the student sees SHARDED (the same information arriving in batches). By the final turn both hold exactly the same condition set. So any divergence between their final-turn distributions comes purely from the input view, not the content. That's an extremely clean learning signal. 3. On-policy sampling. The student's intermediate-turn responses aren't from static data — the student generates them itself and continues from its own generations. This avoids the off-policy distillation mismatch of "the student studies someone else's drafts but must write its own on the exam."

The loss is Jensen-Shannon divergence rather than KL — a notable choice. JS is symmetric and gentler on extreme probabilities on either side, less prone to blow up from one side's outliers. The paper's early pages don't report a KL-vs-JS ablation, so I don't know the empirical basis for this choice.

The Story in Numbers

FiC was validated on the Laban et al. multi-turn benchmark across families and scales:

• Families: Llama, Qwen, Phi, OLMo, sizes 3B to 14B.
• On Math, FiC recovered at least 92% of single-turn performance for every model. Two Llama backbones hit 100% — multi-turn performance fully leveled with single-turn.
• Single-turn capability was essentially unaffected — no classic "sawing off the long plank to fix the short one."
• An unexpected bonus: post-FiC models used 33% fewer tokens in multi-turn dialogue. Why? They learned not to hold forth when information is incomplete. Before, they'd guess wrong and explain at length; now they say "I still need to know X" and stop. Shorter, more direct, cheaper conversations.

Even more surprising was cross-domain transfer. FiC was trained only on Math, but the checkpoint transferred to Database (SQL generation) and Actions (API calling), recovering most of the gap with under 6 A100-hours of adaptation. This suggests FiC learns not "multi-turn math tricks" but a more general batched-information processing ability.

Math, Database, and Actions look completely different — one computes, one writes SQL, one calls APIs — but under multi-turn settings they share one structural difficulty: information fragments arrive along the time axis, and the model must make sound decisions under incompleteness. FiC appears to teach a meta-strategy: don't rush to commit; first assess whether current information suffices for a reliable answer; if not, ask for more. The strategy is domain-agnostic, so it transfers — like learning a way of thinking rather than a fact.

To be honest, though: the paper's early pages don't give specific recovery percentages for Database and Actions, only "most of the gap." Whether that "most" is 85% or 95%, I don't yet know. If later pages have precise numbers, the conclusion firms up.

Moments of Hesitation While Reading

Some thoughts I was sure of at first and later doubted; others the reverse.

My first reaction to "the same model does better single-turn" was: could this just be a prompt-engineering issue? Maybe multi-turn chat templates have some systematic tokenization or attention disadvantage? But the CONCAT control settles it — the same fragmented information, delivered all at once, restores performance. So the problem isn't that information is chopped up; it's what the model *chooses* when information arrives in batches. That control convinced me.

Second wobble: "self-distillation" feels overused — since Hinton's 2015 distillation, someone repackages a new recipe every few months. But this version has real novelty: teacher and student are informationally equivalent, differing only in temporal structure. This isn't big-model-teaches-small-model knowledge transfer; it's behavioral alignment of the same knowledge across different time layouts. Subtle, but conceptually clean.

Third uncertainty: on-policy sampling. Part of the training signal is built on the student's own possibly-wrong intermediate responses. If the student says something absurd in an early turn, can later teacher alignment pull it back? No failure-case analyses in the early pages; I don't know the robustness boundary of this loop.

One More Look Through a Signal-Processing Lens

The model's state in multi-turn dialogue is a belief distribution evolving over time. Each user turn is an observation; the model updates its belief. Ideally the update is sufficient and necessary: incorporate task-relevant information, ignore noise, and maintain uncertainty rather than forcing collapse to a guess when information is thin.

Lost-in-Conversation is essentially premature collapse of the belief-update mechanism. Pushed by its optimization objective, the model converges to a point estimate — however low-confidence — instead of maintaining a broad posterior. Once collapsed, later turns struggle to reverse it, because the model is already conditioning on the wrong assumption.

FiC's two stages map onto classic signal-processing strategies. SFT warm-start is a threshold detector at the front end: when input SNR falls below a gate, the system outputs "refuse to decode" rather than force a guess. VASD is a distribution aligner at the back end: ensure the final output distribution converges to the same steady state regardless of the signal's arrival timing.

The 33% token reduction also fits: prematurely collapsed models generate verbose explanations to mask uncertainty — a rhetorical smokescreen. A model that learned to defer outputs a high-confidence "request more info" with no detours. This reframing is my own interpretation, not necessarily the authors' intent.

If We'd Taken Another Road

• Path A: Pure prompt engineering. Write "ask for clarification first" in the system prompt. Cheapest, lowest ceiling; prior work suggests prompting struggles to overturn entrenched model behavior and is highly prompt-sensitive.
• Path B: RL with sparse rewards. Train multi-turn performance with RL. The problem is credit assignment — over a long dialogue, how do you trace a wrong final answer back to a specific choice in a specific turn? Sparse rewards propagate poorly over long trajectories; the authors explicitly cite this as a trap FiC avoids.
• Path C: External teacher distillation. Distill from a stronger model's multi-turn reference answers. But even frontier models have a multi-turn gap — the external teacher's multi-turn performance is worse than its own single-turn. A nearsighted eye doctor helps only so much.
• Path D: Data augmentation. Collect real multi-turn dialogues and SFT directly. But "good multi-turn dialogue" is hard to define and label, and real distributions are heavily skewed — most conversations involve neither complex reasoning nor clear success criteria.

FiC's choice is internal view-asymmetric distillation — one model's two views, equal information, differences purely from temporal structure. The cost is generating on-policy intermediate trajectories (nontrivial compute), but it has the highest conceptual purity, no external dependencies, no human labels. Between scalability and conceptual cleanliness, the authors bet on the latter.

What I Still Don't Know

I've read only the first six pages; some questions remain open, recorded here honestly.

1. Why can CONCAT serve as teacher? CONCAT is still a bullet list. Real users don't hand you bulleted conditions. If the teacher used natural prose instead, would performance drop? I haven't seen that boundary tested. 2. On-policy sampling stability. Early in training the student is unstable and may generate poor intermediates. Could these derail training? No training dynamics or failure cases shown yet. 3. The cost of the 33% token reduction. Shorter conversations mean shorter context per turn. If the user suddenly changes course or adds constraints, does the shorter history hurt adaptability? An efficiency-vs-flexibility trade-off isn't discussed. 4. Is Llama's 100% recovery a special case? Qwen, Phi, OLMo only reach 92–98%. Architecture or pretraining data? No explanation in the early pages, and I won't guess. 5. Combination with RLVR. The hottest post-training paradigm of 2025–2026 is RL with verifiable rewards. FiC bypasses RL's sparse-reward problem via distillation — but could verifiable rewards also provide signal in multi-turn settings? Are FiC and RLVR mutually exclusive or complementary? A direction worth exploring. 6. User-simulator fidelity. Evaluation uses an LLM-based user simulator that dynamically picks the next shard from conversation history — directly shaping the train/test distribution. Real users are less predictable: they change topics, give redundant info, hint vaguely. The authors say the simulator "mirrors real user behavior," but real behavior exceeds any simulator. The generalization boundary of this evaluation ecosystem deserves sustained attention. 7. Relative contributions of SFT and VASD. No ablation in the early pages: SFT alone recovers how much? VASD alone? Are the contributions additive, multiplicative, or is SFT a necessary precondition for VASD to converge? This decomposition matters for understanding the mechanism. 8. Scaling laws. 3B–14B models all benefit, but does recovery scale monotonically? Would 70B or frontier models also be fixable? Untested. A reasonable guess: bigger models have higher single-turn ceilings; if their multi-turn gap is similarly large, relative recovery might match mid-size models. Only future experiments can tell.

What "Teach Themselves" Actually Means

The title's catchy phrase — "LLMs Teach Themselves" — risks being flattened on social media into "the models are self-evolving again." The actual meaning is narrower and more precise.

The model doesn't "learn something new." No new knowledge enters its parameters; it only learns how to dispatch existing knowledge when information arrives in batches. The single-turn-view teacher could already solve the problem; the multi-turn-view student previously couldn't — not from ignorance, but because the temporal layout of information disrupted its behavior. FiC is behavior correction, not knowledge infusion.

An old analogy comes to mind: playing piano. Someone can already play a piece, but if you make them close their eyes, feed them one hand at a time, and reveal the other hand's position only later, they may play badly — not because they can't play, but because the timing of the input scrambles their motor program. FiC is like a training regimen that lets them produce the same performance whether they learn both hands at once or in sequence.

Of course, the model has no fingers, no eyes, no motor program. The analogy is mine, and its limits are obvious: piano playing needs muscle memory and spatiotemporal coordination; a math problem needs only a rearrangement of token probabilities. The mechanisms are entirely different — but structurally, "temporal disruption degrades performance" is the same shape.

Paper Metadata

| Item | Detail | |------|--------| | Title | Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap | | Authors | Tianlang Chen, Shirley Wu, Jure Leskovec | | Institution | Stanford University | | arXiv ID | 2605.24432 | | Submitted | May 23, 2026 | | Category | cs.CL | | Core method | Two-stage framework: SFT warm-start (learn to defer) + View-Asymmetric Self-Distillation (single-turn CONCAT view teaches multi-turn SHARDED view, token-level JS-divergence alignment) | | Key benchmark | Laban et al. [2025] multi-turn benchmark: Math (GSM8K), Database (Spider), Actions (BFCL) | | Views | FULL (original single-turn), CONCAT (single-turn bullet list), SHARDED (multi-turn fragments) | | Models tested | Llama, Qwen, Phi, OLMo (3B–14B) | | Main results | ≥92% single-turn recovery across families (100% on Llama); 33% token efficiency gain; cross-domain transfer to Database/Actions (<6 A100-hours adaptation) | | Key insight | The multi-turn gap is behavioral, not capability-based; a model's own single-turn view can serve as a teacher without an external model; on-policy intermediate sampling plus information-equivalent views yields a clean alignment signal |

---

*Translated from zhichai.net (CrushAI / FeynmanLearning / 智柴系统实验室).*