English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original
智柴网 中文原文

Trust Guided Decision Transformer (TGDT): Fixing Context Drift in Decision Transformers

Forum topic · 小凯 · 2026-09-29

Summary

This post introduces Trust Guided Decision Transformer (TGDT), a paper (arXiv:2609.31586) by Chainesh Gautam et al. addressing performance degradation of Decision Transformers on long rollouts. The authors show that as the conditioning context drifts out of the training distribution, the model's own next state prediction error rises and remains elevated, providing a direct signal of when context has become unreliable. TGDT selects context before applying value guidance: at each step, it evaluates several recent context suffixes using rolling next state prediction error, calibrates thresholds on held-out offline data via split conformal prediction, and keeps only trusted suffixes. A frozen critic then picks the highest-value action among trusted suffixes, reversing the order of value-only selection, which may pick actions generated from contexts the model itself has flagged as unreliable. Experiments on D4RL navigation and locomotion tasks show that state prediction, critic guidance, and hard context resets each solve only part of the problem, while TGDT reduces persistent high-error runs and surpasses the original Decision Transformer, reset-based context control, and value-only context selection.

Paper Overview

Field: Machine Learning Authors: Chainesh Gautam, Raghuram Bharadwaj Diddigi, Chandramouli Kamanchi, Pankaj Dayama, Sumanta Mukherjee, Kameshwaran Sampath arXiv: 2609.31586

Summary

Decision Transformer performance degrades on long rollouts because the conditioning context drifts out of the training distribution. The authors show that this drift is visible through the model's own next state prediction error, which rises during rollout and stays elevated, giving a direct signal of when context has become unreliable.

They introduce Trust Guided Decision Transformer (TGDT), which selects context before applying value guidance:

  • At each step, TGDT evaluates several recent context suffixes using rolling next state prediction error.
  • Thresholds are calibrated against held-out offline data via split conformal prediction.
  • Only suffixes whose error stays within the calibrated threshold are kept.
  • A frozen critic then chooses the highest-value action among the trusted suffixes.

    • This reverses the order of value-only elastic selection, which may pick an action generated from a context the model itself has already flagged as unreliable.

    Results

    Experiments on D4RL navigation and locomotion tasks show that state prediction, critic guidance, and hard context resets each address only part of the problem. TGDT reduces persistent high-error runs and achieves returns exceeding:

  • The original Decision Transformer
  • Reset-based context control
  • Value-only context selection

Original Abstract (excerpt)

> Decision Transformer performance degrades on long rollouts because the conditioning context drifts out of the training distribution. We show that this drift is visible through the model's own next state prediction error, which rises during rollout and stays elevated, giving a direct signal of when context has become unreliable. We introduce Trust Guided Decision Transformer (TGDT), which selects context before applying value guidance...

---

*Auto-collected on 2026-09-29. Source: arXiv:2609.31586*

Tags

#decision-transformer#reinforcement-learning#offline-rl#conformal-prediction#d4rl#machine-learning#arxiv#sequence-modeling

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178635334