[论文] Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs

论文概要

研究领域: ML
作者: Zhewei Chen, Hao Zhu, Jiaojiao Jiang, Ahad N. Zehmakan
发布时间: 2026-10-07
arXiv: 2610.10520

中文摘要

GNN到MLP的蒸馏旨在保留消息传递教师的预测准确率，同时在推理时部署无图的MLP。现有方法主要迁移节点级预测或使用基于置信度的重加权，但并未指明学生应在何处保留教师由图诱导的几何结构。我们发现这一遗漏导致学生表示空间中两种谱失效模式：在稀疏图上，学生遭受谱欠拟合——遗漏集中在边界区域附近的高能教师方向；在稠密图上，学生遭受谱过拟合——保留了教师已通过聚合坍缩的虚假方向。受能量加权的教师-学生对齐目标启发，我们提出Graph Geometry-aware MLP（G²MLP），一个由Ollivier-Ricci曲率指导的训练时蒸馏框架。曲率识别两种谱误差的集中位置，用于在预测级和表示级对齐之间分配监督。部署的模型仍是标准MLP，推理时无需访问图。在节点分类基准上，G²MLP一致优于无图蒸馏基线，在两种机制下均减小教师-学生的秩差距，且无需架构改动即可迁移到Graph Transformer教师和链接预测任务。

原文摘要

GNN-to-MLP distillation aims to retain the predictive accuracy of a message-passing teacher while deploying a graph-free MLP at inference. Existing methods mainly transfer node-wise predictions or use confidence-based reweighting, but they do not specify where the student should preserve the teacher's graph-induced geometry. We show that this omission leads to two spectral failure modes in the student's representation space. On sparse graphs, the student suffers from spectral underfit, missing high-energy teacher directions concentrated near boundary regions. On dense graphs, it suffers from spectral overfit, retaining spurious directions that the teacher has collapsed through aggregation. Motivated by an energy-weighted teacher-student alignment objective, we propose Graph Geometry-aware ...

自动采集于 2026-10-09

#论文 #arXiv #ML #小凯