SpikeOcc: Towards Resilient Multimodal 3D Occupancy Perception via Spiking Experts

arXiv:2610.15745v1 Announce Type: new Abstract: Multimodal 3D occupancy prediction underpins safe autonomous driving through comprehensive scene understanding. However, existing methods have largely pursued higher accuracy through sophisticated fusion, leaving resilience to missing sensors underexplored. In this paper, we investigate spiking neural networks (SNNs) as a means of building more resilient multimodal perception. Still, the discrete nature of spiking neurons makes optimization challenging. To overcome this difficulty, SpikeOcc is introduced with two complementary components, a Spiking Mixture-of-Experts (SMoE) and Sensor Availability Learning (SAL). Specifically, SMoE organizes RGB, LiDAR, and joint fusion into specialized pathways, enabling full-width unimodal representations while retaining the original multimodal features via a bounded residual. Beyond sensor dropout, SAL employs a staged curriculum with explicit routing supervision, progressively improving robustness under severe sensing conditions. Remarkably, SAL also yields notable gains for conventional models, highlighting its broader potential for resilient multimodal learning. Extensive experiments demonstrate that SpikeOcc maintains competitive full-sensor accuracy while markedly raising the performance floor under sensor outages. SpikeOcc illustrates the promise of neuromorphic computing, pointing toward more reliable and energy-efficient autonomous driving.

Related

• SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction
• Semantic Occupancy Prediction with Dual Range-Voxel Representation
• GaussianSeed: Hierarchical Gaussian Seeding for High-Resolution 3D Occupancy Prediction
• VEOcc: Voxel-Centric Online Semantic Occupancy Prediction For Embodied Scene Understanding
• SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Prediction

Source: arXiv cs.CV | 2026-10-14