[论文] On the Token Collapse Phenomenon in Textual Graph Soft Prompting

论文概要

研究领域: ML
作者: Andrea Giuseppe Di Francesco, Rishabh Jain, Franco Maria Nardini, Nicola Tonellotto, Pietro Liò, Fabrizio Silvestri
发布时间: 2026-10-22
arXiv: 2610.31572

中文摘要

图软提示通过GNN学习的token将图结构注入LLM提示。本文发现G-Retriever中的token坍缩现象：跨三个基准、三个图编码器、两个LLM家族及冻结/LoRA适配模型，图token在输入间几乎无法区分。将token替换为数据集均值或其他示例的token在准确性上无影响。从不访问图的学习常量提示匹配或超过G-Retriever。坍缩token主要充当通用软提示。token变异性本身不能证明模型使用了图信息：通过干预和解码探针，token可能反映问题、答案先验或图派生内容，LLM可能使用也可能不使用。结果提供了一种评估图token对齐的实用方法。

原文摘要

Graph soft prompting injects graph structure into LLM prompts via GNN-learned tokens. We uncover token collapse in G-Retriever: across three benchmarks, three graph encoders, two LLM families, and both frozen and LoRA-adapted models, graph tokens become almost indistinguishable across inputs. Swapping a token with the dataset mean or one from another example produces no answer-level effect. A learned constant prompt that never accesses the graph matches or surpasses G-Retriever on all benchmarks. The collapsed token functions primarily as a generic soft prompt. Token variability alone does not prove the model uses graph information: via interventions and decoding probes, a token can reflect the question, an answer prior, or graph-derived content that the LLM may or may not use.

自动采集于 2026-10-26

#论文 #arXiv #ML #小凯