[论文] Rethinking User Encoders: Routing Diffusion States for Scenario-Ada...

论文概要

研究领域: ML 作者: Junru Zhang, Jinyong Wen, Bin Dou, Wuliang Huang, Baokun Wang, Yu Cheng, Weiqiang Wang, Cheng Peng 发布时间: 2026-10-19 arXiv: 2610.22400

中文摘要

现有基于 LLM 的用户编码器通过自回归（AR）预测把异构行为日志压缩成静态表示，单向约束限制了双向信息流，阻碍了跨模态交互与多源信号的联合保持，难以兼顾多样下游场景的不同证据需求。FINDERS 把扩散语言模型（dLLM）的迭代去噪过程解锁为多粒度重构空间，提出配备路由器的 dLLM 框架，包含去噪、分解、路由、压缩四步操作。可学习的 meta-slot 把互补的去噪状态组织成可迁移的跨模态语义与模态特有行为因子；为提供落地监督，团队构建了含显式行为理由的大规模语料 BehaviorQA；轻量路由器在跨源融合与最终压缩前，按场景查询经输入级残差通路动态强调相关 slot 方向。十个工业场景的实验与在线 A/B 测试表明，FINDERS 在统一设定与场景自适应设定下均稳定超越 SOTA。本工作把 dLLM 的潜力从语言生成推向用户表示学习。

原文摘要

Existing large language model (LLM)-based user encoders serialize heterogeneous behavioral logs and compress them into static representations through autoregressive (AR) prediction. However, their unidirectional constraint limits bidirectional information flow, hindering cross-modal interactions and the joint preservation of multi-source signals. Consequently, the learned static representations struggle to accommodate the distinct evidence requirements of diverse downstream scenarios. We address these limitations by unlocking the iterative denoising process of diffusion language models (dLLMs) as a multi-granularity reconstruction space. Based on this insight, we propose FINDERS (FactorIzatioN of Diffusion statEs for Routing across Scenarios), a router-equipped dLLM framework comprising four operations: denoise, factorize, route, and compress. Learnable meta-slots organize complementary denoising states into transferable cross-modal semantics and modality-specific behavioral factors. To provide grounded supervision, we construct BehaviorQA, a large-scale corpus containing explicit behavioral rationales. Guided by scenario queries, a lightweight router dynamically emphasizes relevant slot directions via an input-level residual pathway before cross-source fusion and final compression. Extensive experiments and online A/B tests across ten industrial scenarios demonstrate that FINDERS consistently outperforms state-of-the-art methods in both unified and scenario-adaptive settings. This work effectively pushes the boundaries of dLLMs beyond language generation, demonstrating their immense potential as powerful user representation learners.

---

*自动采集于 2026-10-19*

#论文 #arXiv #ML #小凯