English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Prompt Repetition: How Simply Repeating a Prompt Makes Non-Reasoning LLMs Smarter at Zero Cost

Forum topic · ✨步子哥 · 2026-01-19

Summary

A Google Research study (arXiv:2512.14982) found that simply repeating the user prompt twice significantly improves large language model performance when reasoning is disabled. Because causal LLMs process tokens with masked attention, each token only attends to preceding content, causing problems in options-first multiple-choice formats where questions appear after choices. Duplicating the full prompt lets every token attend to all other prompt tokens in the second copy, restoring near-bidirectional attention. Tested across seven models (Gemini 2.0 Flash/Lite, GPT-4o/GPT-4o-mini, Claude 3 Haiku/Sonnet, DeepSeek V3) on benchmarks including ARC, OpenBookQA, GSM8K, MMLU-Pro, MATH, plus custom NameIndex and MiddleMatch tasks, prompt repetition won 47 of 70 model-benchmark combinations with zero losses. On NameIndex, Gemini 2.0 Flash-Lite accuracy jumped from 21.33% to 97.33%. Crucially, the extra tokens are processed during the parallelized prefill phase, leaving output length and latency essentially unchanged, unlike chain-of-thought prompting. With reasoning enabled, effects are neutral to mildly positive, and both techniques can be combined.

The Discovery

A Google Research paper (arXiv:2512.14982) demonstrates a remarkably simple technique: repeating the user prompt twice (input changes from <QUERY> to <QUERY><QUERY>) substantially improves LLM performance when the model answers directly without reasoning. The trick adds no output length and no generation latency.

Why It Works: Causal Attention Limits

Large language models are causal (decoder-only) models using masked attention: each token only sees tokens before it. This creates order sensitivity:

  • In question-first multiple-choice formats, the model can still "hold" the question in context while reading options.
  • In options-first formats, the model processes the options before ever seeing the question, hurting performance.
  • By appending a full copy of the prompt, every token in the second copy can attend to every other prompt token (the first copy provides the context). This effectively approximates bidirectional attention without changing the model.

    Results: 47 Wins, 0 Losses

    Testing on 7 models from different families — Gemini 2.0 Flash and Flash-Lite, GPT-4o and GPT-4o-mini, Claude 3 Haiku and Sonnet, DeepSeek V3 — across ARC, OpenBookQA, GSM8K, MMLU-Pro, MATH, plus two custom long-context tasks:

  • Prompt repetition won 47 of 70 model-benchmark combinations with 0 losses (McNemar test, p<0.1 for many gains).
  • Gains are largest in options-first settings, where causal limitations bite hardest.
  • On NameIndex (locating a name in a long list), Gemini 2.0 Flash-Lite accuracy jumped from 21.33% to 97.33%. MiddleMatch (matching the middle element of long sequences) showed similar dramatic gains, since both tasks depend on long-range attention.
  • Zero Cost Efficiency

    Unlike chain-of-thought or "think step by step" prompting, which multiply output tokens and latency, prompt repetition costs almost nothing:

  • The duplicated tokens are processed in the parallelized prefill phase; generation speed is untouched.
  • Measured output length and end-to-end latency match baselines (minor exceptions for long inputs on Claude models).
  • A control using period padding to equal length showed no improvement, confirming the benefit comes from content repetition and attention symmetry, not added length.
  • Coexistence with Reasoning

    With reasoning enabled, prompt repetition is neutral to slightly positive (5 wins, 1 loss, 22 ties) — reasoning already involves internal re-processing. The two techniques can be combined: use repetition alone for speed, both together for maximum accuracy.

    Variants and Related Work

  • Verbose repetition (adding explanatory phrases) and triple repetition perform comparably or better; triple repetition gives extra gains on NameIndex and MiddleMatch.
  • Related work includes Chain-of-Thought (arXiv:2201.11903), zero-shot reasoning via re-reading (arXiv:2309.06275), and repetition improving embeddings (arXiv:2402.15449).

Outlook

The authors outline 13 future directions: fine-tuning models for repeated prompts, periodic repetition during generation, keeping KV-cache only for the second copy, extending to non-text modalities, and analyzing attention pattern changes. Prompt repetition's zero cost, universality, and drop-in deployment make it a candidate default setting for non-reasoning LLM applications.

References

1. Yaniv Leviathan, Matan Kalman, Yossi Matias. Prompt Repetition Improves Non-Reasoning LLMs. arXiv:2512.14982, 2025. 2. Jason Wei et al. Chain-of-thought prompting elicits reasoning in large language models. arXiv:2201.11903, 2023. 3. Takeshi Kojima et al. Large language models are zero-shot reasoners. arXiv:2205.11916, 2023. 4. Xiaohan Xu et al. Re-reading improves reasoning in large language models. arXiv:2309.06275, 2024. 5. Jacob Mitchell Springer et al. Repetition improves language model embeddings. arXiv:2402.15449, 2024.

Tags

#llm#prompt-engineering#prompt-repetition#google-research#causal-attention#transformer#inference-efficiency#benchmarks

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/176415302