DeKoR: Decomposed Knowledge Routing for Efficient GNN--LLM Fusion

arXiv:2610.14710v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown strong potential for text-attributed graph learning by leveraging rich pre-trained knowledge and language understanding capabilities. Combining LLMs with Graph Neural Networks (GNNs) offers a promising paradigm for expanding the modeling capabilities of graph learning. However, LLMs and GNNs follow distinct learning paradigms, leading to a pronounced mismatch between their knowledge spaces. Such discrepancies make effective knowledge fusion nontrivial. Moreover, existing methods often incur substantial additional computation for only limited performance gains. To address these challenges, we propose DeKoR, an efficient framework for GNN and LLM knowledge fusion. To enable efficient knowledge transfer, DeKoR decomposes LLM knowledge into complementary local and global views and uses the LLM as an offline teacher to distill them into lightweight experts. To enable selective knowledge fusion, an adaptive router learns how much each node should rely on each expert. Extensive experiments on multiple benchmarks demonstrate that DeKoR outperforms existing baselines on both node and subgraph classification, achieving superior effectiveness and efficiency.

Related

• Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations
• Colorful Talks with Graphs: Human-Interpretable Graph Encodings for Large Language Models
• LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretraining

Source: arXiv cs.LG | 2026-10-14