Research
Global Communication or Graph-Specific Memory?
arXiv:2610.05874v1 Announce Type: new Abstract: Scalable Graph Transformers are commonly trained and evaluated on static large graphs in a transductive setup. Many scalable Graph Transformer component
arXiv:2610.05874v1 Announce Type: new Abstract: Scalable Graph Transformers are commonly trained and evaluated on static large graphs in a transductive setup. Many scalable Graph Transformer components can be formulated as a constant-size shared memory, similar to virtual nodes, providing compressed information about the whole graph. The counterpart of these models in language models and other domains is justified as the input changes, and this mechanism learns to compress some useful information about the input. In transductive learning on a single fixed graph, however, any shared memory can be seen as a constant at test time. This raises the question of what exactly this shared memory does in this static setup. We give preliminary evidence that optimizing a shared memory directly performs similarly to global communication methods, and so normal local message-passing models can embed similar information in their weights. Thus, these settings may be a poor fit for evaluating global communication in graph neural networks.
Related
- CoRe-GNN: Multilevel Message passing on Coarsened graphs
- Learning How Much to Think: Difficulty-Aware Dynamic MoEs for Graph Node Classification
- GIST: Gauge-Invariant Spectral Transformers for Scalable Graph Neural Operators
- Scalable Peptide Design via Memory-Efficient Equivariant Transformer
Source: arXiv cs.LG | 2026-10-06