Computational Onboard Data Management for Heterogeneous Autonomous Vehicle Streams

arXiv:2511.19453v2 Announce Type: replace-cross Abstract: Autonomous vehicles continuously generate large volumes of heterogeneous sensor and state data streams. While this data supports real-time autonomy, it also enables emerging vehicle computing applications that require retained historical context. Existing in-vehicle logging stacks are largely replay- or compliance-oriented: rich sensor streams are often discarded after online processing or recorded only in limited windows for offline replay. This creates a new onboard vehicle-data operating point: transforming transient multimodal streams into compact, long-horizon, queryable history under embedded resource constraints. We present Autonomous Vehicle Storage (AVS), a computational and hierarchical in-vehicle data management system for this operating point. AVS introduces a vehicle-specialized architecture including (i) utility-aware modality preprocessing before persistence, (ii) a chunk-indexed append-only logger for sustained ingest, time-range lookup, and prefix recovery, and (iii) an SSD-HDD hot/cold hierarchy that supports direct cold queries. Evaluated on a real L4 platform with three days of traces and concurrent LiDAR, camera, and GPS logging, AVS reduces storage by 8.0-8.7x relative to raw rosbag2 from its modality and task aware preprocessing, delivers hot-tier (SSD) query Time To First Byte (TTFB) of 0.9-9.1 ms and cold-tier (HDD) query TTFB of 25.6-44.0 ms, recovers from power loss in 9.98 s with a bounded 0.24 s loss window, and sustains background archival to HDD at about 107-108 MB/s. It shows the feasibility of AVS on real in-vehicle platforms, and it also exposes key tradeoffs and open challenges for vehicle data engineering.

Source: arXiv cs.RO | 2026-10-06