Trust Guided Decision Transformer (TGDT): Context Selection Before Value Guidance

Forum topic · 小凯 · 2026-09-29

Summary

Trust Guided Decision Transformer (TGDT) addresses performance degradation in Decision Transformers during long rollouts. The authors show that as the conditioning context drifts out of the training distribution, this drift is directly observable through the model's own next state prediction error, which rises and stays elevated during rollout. TGDT selects context before applying value guidance: at each step, it evaluates several recent context suffixes using rolling next state prediction error, calibrated against held-out offline data via split conformal prediction. Only suffixes whose error remains within the calibrated threshold are trusted, and a frozen critic then selects the highest-value action among these trusted suffixes. This reverses the ordering of value-only resilient selection, which may pick actions generated from contexts already flagged as unreliable. Experiments on D4RL navigation and locomotion tasks show that state prediction, critic guidance, and hard context resets each address only part of the problem, while TGDT reduces persistent high-error runs and achieves returns exceeding the original Decision Transformer, reset-based context control, and value-only context selection. Paper: arXiv:2609.31586.

Overview

Field: Machine Learning Authors: Chainesh Gautam, Raghuram Bharadwaj Diddigi, Chandramouli Kamanchi, Pankaj Dayama, Sumanta Mukherjee, Kameshwaran Sampath Published: 2026-09-25 arXiv: 2609.31586

Abstract (English, from the paper)

Decision Transformer performance degrades on long rollouts because the conditioning context drifts out of the training distribution. We show that this drift is visible through the model's own next state prediction error, which rises during rollout and stays elevated, giving a direct signal of when context has become unreliable. We introduce Trust Guided Decision Transformer (TGDT), which selects context before applying value guidance. At each step, TGDT evaluates several recent context suffixes using rolling next state prediction error, calibrated against held out offline data via split conformal prediction. It keeps only suffixes whose error stays within the calibrated threshold, then uses a frozen critic to choose the highest value action among the trusted suffixes. This reverses the ordering of value-only resilient selection, which may choose an action generated from context the model itself has flagged as unreliable. On D4RL navigation and locomotion tasks, experiments show that state prediction, critic guidance, and hard context resets each solve only part of the problem. TGDT reduces persistent high-error runs and achieves returns exceeding the original Decision Transformer, reset-based context control, and value-only context selection.

Key Contributions

Observability of context drift: Next state prediction error from the Decision Transformer itself serves as a direct, online signal of when the conditioning context has become unreliable.
Trust-first context selection: TGDT evaluates recent context suffixes via rolling prediction error and calibrates a threshold with split conformal prediction on held-out offline data.
Reversed ordering: Trust filtering happens *before* value guidance, avoiding actions derived from contexts the model already considers unreliable.
Empirical results: On D4RL navigation and locomotion benchmarks, TGDT outperforms the vanilla Decision Transformer, reset-based context control, and value-only context selection, reducing persistent high-error rollouts.

Reference

arXiv: <https://arxiv.org/abs/2609.31586>

--- *Auto-collected on 2026-09-29.*