HyperGraphPro: Progress-Aware Reinforcement Learning for Structure-Guided Hypergraph RAG

arXiv:2601.17755v2 Announce Type: replace Abstract: Graph Retrieval-Augmented Generation (GraphRAG) has emerged as a promising paradigm that organizes external knowledge into structured graphs of entities and relations, enabling large language models (LLMs) to perform complex reasoning beyond text-chunk retrieval. Recent advances have integrated reinforcement learning (RL) into agentic GraphRAG approaches, enabling iterative interactions with knowledge graphs during training. However, existing RL-based methods suffer from two key limitations: (1) they primarily depend on semantic similarity for retrieval, often overlooking the underlying graph topology, and (2) they rely on sparse, outcome-level rewards that fail to capture the quality of intermediate retrieval steps and their dependencies. To address these limitations, we propose HyperGraphPro, a progress-aware agentic framework for graph-based retrieval and multi-step reasoning. HyperGraphPro introduces a structure-aware hypergraph retrieval mechanism that jointly considers semantic relevance and graph connectivity, promoting coherent traversal along multi-hop reasoning paths. Furthermore, we design a progress-based stepwise policy optimization that provides dense learning signals by modulating advantages according to intermediate reasoning progress within a graph, rather than relying solely on final outcomes. Experiments on multi-hop question answering benchmarks demonstrate that HyperGraphPro consistently improves reasoning accuracy and generation quality over existing GraphRAG methods.

Related

• GenProve: Learning to Generate Text with Fine-Grained Provenance
• Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
• Reinforcement-aware Knowledge Distillation for LLM Reasoning
• ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework

Source: arXiv cs.CL | 2026-04-14