Paper Overview
Research area: Machine Learning Authors: Andrea Giuseppe Di Francesco, Rishabh Jain, Franco Maria Nardini, Nicola Tonellotto, Pietro Liò, Fabrizio Silvestri Published: 2026-10-22 arXiv: 2610.31572
Summary
Graph soft prompting injects graph structure into LLM prompts via GNN-learned tokens. This paper uncovers the phenomenon of token collapse in G-Retriever: across three benchmarks, three graph encoders, two LLM families, and both frozen and LoRA-adapted models, graph tokens become almost indistinguishable across inputs.
Swapping a token with the dataset mean or a token from another example produces no answer-level effect. A learned constant prompt that never accesses the graph matches or surpasses G-Retriever on all benchmarks. The collapsed token functions primarily as a generic soft prompt.
Key Findings
- Token variability alone does not prove the model uses graph information.
- Via interventions and decoding probes, a token can reflect the question, an answer prior, or graph-derived content that the LLM may or may not use.
- The results provide a practical methodology for evaluating graph token alignment.
Original Abstract
Graph soft prompting injects graph structure into LLM prompts via GNN-learned tokens. We uncover token collapse in G-Retriever: across three benchmarks, three graph encoders, two LLM families, and both frozen and LoRA-adapted models, graph tokens become almost indistinguishable across inputs. Swapping a token with the dataset mean or one from another example produces no answer-level effect. A learned constant prompt that never accesses the graph matches or surpasses G-Retriever on all benchmarks. The collapsed token functions primarily as a generic soft prompt. Token variability alone does not prove the model uses graph information: via interventions and decoding probes, a token can reflect the question, an answer prior, or graph-derived content that the LLM may or may not use.
--- *Automatically collected on 2026-10-26*