Jumping the Line: A 20-Token Suffix Can Trick LLM Schedulers Into Predicting Short Outputs
This post is a Chinese-language walkthrough of the paper *Jumping the Line: Exploiting Length Predictions in LLM Scheduling* (Dai, Shahout & Sharif, arXiv:2610.03430), the second part of a three-part series on trusting LLM schedulers' length predictions.
Background
Modern LLM serving resembles a hospital: GPUs are exam rooms, KV caches are beds, and requests are patients. FCFS scheduling suffers from head-of-line blocking, so systems use shortest-job-first (SJF/SRPT) policies that minimize average completion time — but only if job sizes are known in advance. LLM output lengths are unknown at arrival, so systems like TRAIL (NeurIPS 2025), LTR, and ELIS use lightweight probes on the model's intermediate-layer representations to predict response length and prioritize short requests.
This creates a four-link trust chain: (1) users control request text → (2) text determines internal representations → (3) probes map representations to length estimates → (4) schedulers rank by those estimates. The first two links are fully attacker-controlled.
The JIL Attack
JIL manufactures a mismatch between predicted and actual length. A 20-token suffix is optimized with GCG (the gradient-based optimizer from Zou et al. 2023's jailbreak attacks), minimizing the probe's predicted length n̂ via gradients flowing back through probe g_θ into the transformer's layer l. Deployment requires only appending the suffix to one's own request — no system modification, no touching other requests, one forward pass on the victim model.
Unlike sponge-example attacks that drain compute by forcing longer outputs, JIL steals *priority*, which is exclusive and zero-sum.
Experimental Results
Setup: four models (Llama-3-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, Llama-3.1-70B-Instruct), each with an independently trained probe; 200 Alpaca + 200 UltraChat + 100 ARC-Challenge + 100 HellaSwag prompts per model; TRAIL/vLLM on a single NVIDIA B200; paired clean/attacked workloads at 64 and 16 sequence slots.
- Predicted length drops up to 83.4% across all models; UltraChat (multi-turn chat) shows partial robustness for some models, especially Qwen2.5-7B.
- Up to 1.53x faster completion for targeted requests; effects are strongest under 16-slot (high-pressure) capacity. Benign requests shift backward in the queue, with those originally near the front slowed most often.
- Actual output length barely changes — the mismatch is preserved. But quality suffers: reference win rates drop on Alpaca (largest for Llama-3-8B, with nearly unchanged length), and UltraChat judge medians are -1 across all models.
- Early stopping backfires: milder suffixes (20/40/60-token targets) retain speed gains but show *lower* win rates than the full attack — partial suffixes may land on representation positions most toxic to content.
- Cross-model transfer works both ways between Llama-3-8B and Mistral-7B (74.8% mean prediction drop when directly optimizing on Mistral), suggesting probes share common surface-cue shortcuts.
- Defenses: bucketing predictions into coarse bins and imposing prediction floors both reduce attack gains and benign tail latency. Bucketing helps most when concurrency is constrained (16 slots), trading SJF precision for manipulation resistance.
- Dai, Y., Shahout, R., & Sharif, M. (2026). Jumping the Line: Exploiting Length Predictions in LLM Scheduling. arXiv:2610.03430.
- Shahout, R., et al. (2025). Don't Stop Me Now: Embedding Based Scheduling for LLMs. NeurIPS 2025.
- Zou, A., et al. (2023). Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043.
- Choi, S., et al. (2025). ELIS. arXiv:2505.09142.
- Qiu, H., et al. (2024). Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction. arXiv:2404.08509.
- Wu, H., et al. (2022). Scalable Verification of GNN-based Job Schedulers. OOPSLA.
- Mitzenmacher, M. (2019). Scheduling with Predictions and the Price of Misprediction. arXiv:1902.00732.
- Shumailov, I., et al. (2021). Sponge Examples: Energy-Latency Attacks on Neural Networks. EuroS&P.
Broader Context
The attack surface generalizes anywhere model-derived predictions allocate scarce resources: Certaindex compute allocation, speculative decoding draft selection (OUTLETS), and even GNN cluster schedulers (Wu et al. 2022's Decima verification). Mitzenmacher's 2019 "price of misprediction" frames the cost. The series' third installment will address rebuilding trustworthy schedulers when readings are both unreliable (part one) and manipulable (part two).