Learning to Think from Multiple Thinkers: Active Learning with Multi-Thinker Chain-of-Thought Supervision

Forum topic · 小凯 · 2026-04-29

Summary

This paper by Nirmit Joshi, Roey Magen, and Nathan Srebro (arXiv:2504.20632, April 2025) studies learning with Chain-of-Thought (CoT) supervision from multiple thinkers, all providing correct but systematically different solutions, such as step-by-step math solutions written by different people or execution traces of different programs solving the same problem. The authors consider function classes that are computationally easy to learn from CoT supervision of a single thinker, yet hard to learn from end-result supervision alone. They prove that under cryptographic assumptions, learning can be computationally hard when relying passively on CoT data from two or a few different thinkers. In contrast, they present a generic, computationally efficient active learning algorithm in which each thinker contributes only a small amount of CoT data independent of the target accuracy, and the required number of thinkers grows as log(1/ε)·loglog(1/ε). The work highlights a fundamental separation between passive and active learning settings for multi-source CoT supervision.

Paper Overview

Field: Machine Learning
Authors: Nirmit Joshi, Roey Magen, Nathan Srebro
Published: 2025-04-29
arXiv: 2504.20632

Original Abstract

We study learning with Chain-of-Thought (CoT) supervision from multiple thinkers, all of whom provide correct but possibly systematically different solutions, e.g., step-by-step solutions to math problems written by different thinkers, or step-by-step execution traces of different programs solving the same problem. We consider classes that are computationally easy to learn using CoT supervision from a single thinker, but hard to learn with only end-result supervision, i.e., without CoT (Joshi et al. 2025). We establish that, under cryptographic assumptions, learning can be hard from CoT supervision provided by two or a few different thinkers, in passive data-collection settings. On the other hand, we provide a generic computationally efficient active learning algorithm that learns with a s... (abstract truncated at source)

Key Takeaways (from the Chinese summary)

The setting: CoT supervision comes from multiple thinkers, each correct but potentially systematically different in style or approach.
Under cryptographic assumptions, passive learning from CoT data of only two or a few thinkers can be computationally hard.
A generic, computationally efficient active learning algorithm is provided, where each thinker supplies only a small amount of CoT data, independent of the target accuracy ε.
The number of thinkers needed scales as log(1/ε)·loglog(1/ε).

---

*Auto-collected on 2026-04-29.*