Paper Overview
- Field: Machine Learning
- Authors: Junru Zhang, Jinyong Wen, Bin Dou, Wuliang Huang, Baokun Wang, Yu Cheng, Weiqiang Wang, Cheng Peng
- Published: 2026-10-19
- arXiv: 2610.22400
- Replaces autoregressive user encoders with a diffusion language model (dLLM) framework, enabling bidirectional information flow.
- Four-stage pipeline: denoise → factorize → route → compress.
- Learnable meta-slots separate denoising states into cross-modal semantics and modality-specific behavioral factors.
- BehaviorQA: new large-scale corpus with explicit behavioral rationales for supervision.
- Lightweight scenario-guided router with input-level residual pathway, applied before fusion and compression.
- Validated on ten industrial scenarios plus online A/B tests; outperforms SOTA in unified and scenario-adaptive settings.
Abstract
Existing large language model (LLM)-based user encoders serialize heterogeneous behavioral logs and compress them into static representations through autoregressive (AR) prediction. However, their unidirectional constraint limits bidirectional information flow, hindering cross-modal interactions and the joint preservation of multi-source signals. Consequently, the learned static representations struggle to accommodate the distinct evidence requirements of diverse downstream scenarios.We address these limitations by unlocking the iterative denoising process of diffusion language models (dLLMs) as a multi-granularity reconstruction space. Based on this insight, we propose FINDERS (FactorIzatioN of Diffusion statEs for Routing across Scenarios), a router-equipped dLLM framework comprising four operations: denoise, factorize, route, and compress.
Learnable meta-slots organize complementary denoising states into transferable cross-modal semantics and modality-specific behavioral factors. To provide grounded supervision, we construct BehaviorQA, a large-scale corpus containing explicit behavioral rationales. Guided by scenario queries, a lightweight router dynamically emphasizes relevant slot directions via an input-level residual pathway before cross-source fusion and final compression.
Extensive experiments and online A/B tests across ten industrial scenarios demonstrate that FINDERS consistently outperforms state-of-the-art methods in both unified and scenario-adaptive settings. This work effectively pushes the boundaries of dLLMs beyond language generation, demonstrating their immense potential as powerful user representation learners.