Pistis Technical Report: Multimodal LLM Family with Interleaved Distillation and Reinforcement Learning

Forum topic · 小凯 · 2026-09-26

Summary

Pistis is a family of 27B- and 9B-parameter multimodal large language models built on Qwen3.6 and Qwen3.5, developed through a general and scalable post-training framework. The framework begins with large-scale multimodal supervised fine-tuning (SFT), then applies Interleaved Distillation and Reinforcement Learning (IDRL), a novel post-training paradigm that tightly integrates on-policy distillation and reinforcement learning within a single training loop. By alternating between the two objectives, IDRL enables more effective knowledge transfer, greater optimization stability, and more precise credit assignment for long-horizon agentic trajectories. Each model size yields two specialized variants: Pistis-Thinking for enhanced deep multimodal reasoning, and Pistis-Agentic, trained with agentic trajectory data to support long-horizon planning, iterative reasoning, and tool use, with particular strength in multimodal search. The report also introduces Pistis-Auto-Harnessing (PAH), a system-level method that iteratively refines an agent's reasoning scaffolding automatically, improving performance without updating model parameters or increasing interaction budgets. Paper: arXiv 2609.21938.

Pistis Technical Report

Field: Computer Vision Authors: Heyun Chen, Xiaohan Lan, Jiaxi Li Published: 2026-09-25 arXiv: 2609.21938

Overview

We introduce the Pistis model family, comprising 27B- and 9B-parameter multimodal large language models built on Qwen3.6 and Qwen3.5, developed through a general and scalable post-training framework.

The framework first establishes a solid foundation through large-scale multimodal supervised fine-tuning (SFT). On top of this, the authors propose Interleaved Distillation and Reinforcement Learning (IDRL), a novel post-training paradigm that tightly integrates on-policy distillation and reinforcement learning within a single training loop. By alternating optimization between the two objectives, IDRL achieves:

More effective knowledge transfer
Greater optimization stability
More precise credit assignment for long-horizon agentic trajectories

Model Variants

Each model size produces two specialized variants:

Pistis-Thinking: enhanced for deep multimodal reasoning
Pistis-Agentic: incorporates agentic trajectory data to support long-horizon planning, iterative reasoning, and tool use, with particular excellence in multimodal search

Pistis-Auto-Harnessing (PAH)

The report also introduces Pistis-Auto-Harnessing (PAH), a system-level method that automatically improves an agent's reasoning scaffolding through iterative refinement, boosting performance without updating model parameters or increasing the interaction budget.

Original Abstract

> We introduce the Pistis model family, comprising 27B- and 9B-parameter multimodal large language models built on Qwen3.6 and Qwen3.5, developed through a general and scalable post-training framework. We propose Interleaved Distillation and Reinforcement Learning (IDRL), a novel post-training paradigm that tightly integrates on-policy distillation and reinforcement learning within a single training loop.

--- *Auto-collected on 2026-09-26*