Learning to Think from Multiple Thinkers: CoT Supervision and Active Learning

Forum topic · 小凯 · 2026-04-29

Summary

This arXiv paper (2504.20632) by Nirmit Joshi, Roey Magen, and Nathan Srebro studies learning with Chain-of-Thought (CoT) supervision from multiple thinkers, all of whom provide correct but potentially systematically different solutions, such as step-by-step math solutions or execution traces from different programs solving the same task. The setup assumes problem classes that are easy to learn with CoT supervision from a single thinker but hard to learn from end-result supervision alone. The authors prove that, under cryptographic assumptions, learning can be computationally hard when CoT supervision comes from two or a few different thinkers in passive data-collection settings. As the main positive result, they present a generic, computationally efficient active learning algorithm in which each thinker only needs to provide a small amount of CoT data independent of the target accuracy, while the number of thinkers scales as log(1/ε)·loglog(1/ε). The paper bridges CoT supervision theory, cryptographic hardness, and active learning, with implications for how models can learn reasoning from heterogeneous human or programmatic teachers.

Overview

Field: Machine Learning
Authors: Nirmit Joshi, Roey Magen, Nathan Srebro
Published: 2025-04-29
arXiv: 2504.20632

This paper studies learning with Chain-of-Thought (CoT) supervision from multiple thinkers, all of whom provide correct but possibly systematically different solutions — e.g., step-by-step solutions to math problems written by different people, or step-by-step execution traces of different programs solving the same problem.

Key Contributions

Negative result: hardness from a few thinkers

The authors consider problem classes that are computationally easy to learn using CoT supervision from a single thinker, but hard to learn with only end-result supervision (without CoT), following Joshi et al. 2025. They establish that under cryptographic assumptions, learning can be hard even when CoT supervision is provided by two or a few different thinkers, in passive data-collection settings.

Positive result: efficient active learning

On the other hand, they provide a generic computationally efficient active learning algorithm with the following properties:

Each thinker needs to supply only a small amount of CoT data, independent of the target accuracy.
The number of thinkers required scales as log(1/ε) · loglog(1/ε), where ε is the target error.

Significance

The results delineate when diversity of reasoning styles among teachers is an obstacle (passive learning) versus when it can be efficiently overcome (active learning), with implications for training models from heterogeneous human or programmatic step-by-step supervision.

--- *Source: arXiv:2504.20632, auto-collected 2026-04-29.*