HIERA: Workload-Aware Planning Across Implementation Spaces for GPU Kernel Optimization

arXiv:2608.21157v1 Announce Type: cross Abstract: High-performance GPU kernels underpin modern deep learning and scientific computing. As workloads become increasingly diverse and GPU hardware evolves rapidly, developing efficient methods for automated GPU kernel generation and optimization has become increasingly important. Existing LLM-based methods typically optimize within a fixed implementation space, limiting either optimization flexibility or search efficiency. We propose extsc{HIERA}, a hierarchical search-space planning framework for GPU kernel optimization. extsc{HIERA} constructs contract-augmented task specifications, selects an appropriate implementation space across PyTorch operators, CUDA libraries, and custom CUDA kernels, and uses profiling feedback and expert knowledge to guide structured iterative refinement. Experiments on KernelBench across multiple various workload levels and base LLMs show that extsc{HIERA} delivers stronger overall implementation validity, sample efficiency, and optimization performance than existing training-free methods, while remaining competitive with the training-based CUDA-L1 without additional model training. A case study on a specialized stencil operator from scientific computing further achieves a (1.53imes) speedup over cuDNN, demonstrating the potentiality of the general framework beyond standard machine-learning workloads.

Related

• Record-Remix-Replay: Hierarchical GPU Kernel Optimization using Evolutionary Search
• Xe-Forge: Multi-Stage LLM-Powered Kernel Optimization for Intel GPU
• Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts
• Learning When to Optimize: Verified Optimization Skills from Expert GPU-Kernel Lineages
• AgentKernelArena: Generalization-Aware Benchmarking of GPU Kernel Optimization Agents

Source: arXiv cs.AI | 2026-08-24