English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original
智柴网 中文原文

AIR: Adaptive Interleaved Reasoning with Code in Multimodal LLMs — Paper Explainer

Forum topic · Anonymous · 2026-06-23

Summary

This zhichai.net forum post explains AIR (Adaptive Interleaved Reasoning with Code in MLLMs), a training framework that teaches multimodal large language models to interleave visual perception, note-taking, external queries, and code execution—much like Sherlock Holmes switching between observation, measurement, and deduction. The post identifies two weaknesses in prior tool-use approaches: reliance on hand-crafted heuristic rules for deciding when to call tools, and a focus on visual operations at the expense of numerical computation. AIR's solution has three components: (1) a two-stage cold-start data pipeline using a teacher model with execution-based verification, step decomposition, and diversity augmentation; (2) reinforcement-learning dataset curation with correctness, difficulty, and diversity filtering; and (3) a group-constrained reward function that scores process quality across multiple reasoning trajectories, yielding adaptive tool-calling behavior. Reported results include a 6.1 percentage-point average gain after RL training, a 9.9 point gain on samples requiring interleaved reasoning, and over 95% tool-call success on benchmarks like MathVista, MathVision, and MMMU. The post also covers limitations (teacher-model dependency, limited tool ecosystem, code-execution safety) and future directions such as multi-modal interleaving, metacognition, and multi-agent collaboration.

AIR: Adaptive Interleaved Reasoning with Code in Multimodal LLMs — Paper Explainer

> *"When you have eliminated the impossible, whatever remains, however improbable, must be the truth."* — Sherlock Holmes

This post from zhichai.net explains the paper AIR (Adaptive Interleaved Reasoning with Code in MLLMs) (arXiv:2606.23678), which trains multimodal LLMs to reason the way a detective works: alternating between perception, recording, querying, computing, and verification instead of answering in a single pass.

Background: why multimodal models struggle

Modern MLLMs (GPT-4V, Claude 3, Gemini) excel at describing images, recognizing objects and text, and interpreting charts — but fail at mathematical computation that requires combining visual extraction with multi-step calculation. The reason: training data rarely pairs "reading a chart" with "doing the math." OpenAI's o3 showed the value of interleaved reasoning — observe, write code, execute, and decide the next step based on results.

Existing tool-use approaches have two limitations:

  • Hand-crafted heuristic rules decide when to invoke tools (e.g., "if math problem, call calculator"), which generalize poorly.
  • Prior work focuses on visual operations (image editing, detection) and neglects numerical computation.

    • AIR's three components

    1. Cold-start data construction (two stages)

  • A teacher model (e.g., GPT-4V) generates chain-of-thought solutions on vision-math problems.
  • AIR refines them via execution verification (code must actually run and match claimed results), step decomposition (each step = one subtask: observe, compute, query, verify), and diversity augmentation (multiple solution paths per problem).

    • 2. RL dataset curation

    A multi-layer filtering system:
  • Correctness filter: keep only data where code execution matches the final answer.
  • Difficulty filter: keep the "sweet spot" of problems that require tools but are solvable; down-weight trivial ones and shelve the hardest.
  • Diversity filter: ensure coverage of different reasoning patterns, avoiding over-representation of any single strategy.

    • 3. Adaptive tool calling via group-constrained rewards

    Instead of a pure outcome reward, AIR combines:
  • Process rewards: credit for correctly extracting key visual information, calling appropriate tools, successful code execution, and coherent intermediate steps.
  • Group constraint: generate multiple reasoning trajectories per problem, group them by tool-calling strategy, compare group-level performance, and boost rewards for whole groups that perform better — so the model learns *strategy selection*, not just individual skills.
  • The result is emergent, adaptive tool use: the model learns when to write code, when to query, and when to request more visual detail, without hand-coded rules.

    • Experimental results

    Evaluated on MathVista, MathVision, and MMMU:

  • +6.1 percentage points average performance after RL training.
  • +9.9 percentage points on samples genuinely requiring interleaved multi-step reasoning with tools.
  • >95% of tool calls execute successfully and produce valid results.

    • Ablations show:

  • Without cold-start data, RL training is unstable and prone to collapse.
  • Group-constrained rewards outperform single outcome-based rewards, especially on complex reasoning samples.
  • Data filtering substantially improves training efficiency and final performance.

    • Significance and applications

    AIR demonstrates that models can learn general-purpose tool-use strategies through RL rather than predefined rules, moving multimodal AI from pattern matching toward genuine reasoning. Potential applications include:

  • Scientific discovery: extract data from experiment plots, run statistics, generate visualizations.
  • Financial analysis: pull metrics from statement screenshots, compute ratios and growth rates.
  • Education: solve homework photos step by step, invoking calculation tools at key steps.

    • Limitations and future directions

  • Teacher-model dependency: cold-start data inherits the teacher's biases.
  • Limited tool ecosystem: experiments mainly use Python execution; real-world use needs databases, APIs, and specialized software.
  • Safety: auto-generated code requires sandboxing and strict permission control.
  • Future work: interleaving more modalities (audio, video, sensor data), metacognition (reasoning about its own reasoning), and multi-agent collaborative reasoning.

    • References

  • Han, C., Lan, X., & Qiu, H. (2026). AIR: Adaptive Interleaved Reasoning with Code in MLLMs. arXiv:2606.23678.
  • OpenAI. (2024). Introducing OpenAI o3 and o3-mini. https://openai.com/index/openai-o3-system-card/
  • Lu, P., et al. (2023). MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts. arXiv:2310.02255.
  • Yue, X., et al. (2023). MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark. arXiv:2311.16502.
  • Wei, J., et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022.
  • Ouyang, L., et al. (2022). Training language models to follow instructions with human feedback. NeurIPS 2022.
*Explainer posted June 24, 2026.*

Tags

#multimodal-llm#reinforcement-learning#tool-use#interleaved-reasoning#math-reasoning#paper-explainer#code-execution#air

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178208052