Paper Overview
Research Area: NLP Authors: Cheng Qian, Wenting Zhao, Liangwei Yang Published: 2026-08-13 arXiv: 2508.03418
Summary
Recent work on distillation typically transfers the capabilities of large models to smaller ones by updating the smaller models' parameters through teacher forcing, online distillation, and related training-time methods. This paper asks whether such transfer can happen at test time instead.
The authors study strong-to-weak scaffolding: whether a stronger builder model can construct inference-time harnesses that help a weaker target model solve tasks more reliably, without any parameter updates.
Methodology
- Four representative theory-of-mind benchmarks are used.
- Each builder model uses 5% of the data as a validation set.
- The harness is refined over multiple iterative rounds.
- The final harness is evaluated on the full test set.
- Test-time capability transfer is highly effective: average target model performance nearly doubles from 0.49 to 0.91.
- Gains mainly come from:
- Offloading unstable model reasoning to deterministic code
- Benchmark-specific routing
- Strict answer-format enforcement
- Gains do not come from encouraging broader reasoning or wider sampling by the target model.
- The builder model's reasoning effort monotonically improves harness quality.
- Plateau effects are modest relative to the builder model's own capability.
- Weaker target models obtain the largest gains.
Key Findings
Conclusion
Inference-time harness design is an important complement to traditional training-time distillation, enabling strong models to transfer cognitive structure to weak models without retraining.
Original Abstract (excerpt)
> Recent work on distillation transfers the capabilities of large models to smaller ones often by updating the latter's parameters...
--- *Auto-collected on 2026-08-14*