HF is quietly becoming the best place to store data, public AND private, especially for brutal domains like robotics and video AI where the …

HF is quietly becoming the best place to store data, public AND private, especially for brutal domains like robotics and video AI where the files are massive, append-only, and never stop growing. Example? Public robotics datasets exploded from 1,000 in early 2025 to 60,000 today, and there's twice as many private ones. Why? A single robot records at 140 MB/s, all day, forever. That data has to be stored, streamed to GPUs, and shipped back to hardware on repeat. Get it wrong and your GPUs sit idle at 0 MB/s waiting for a dataset to land. Get it right (stream straight from the Hub, pre-warmed cache) and those same GPUs scream along at ~1,326 MB/s, fully fed. 🚀 Here's how LeRobot + Hugging Face Storage Buckets pull it off: https://huggingface.co/spaces/imstevenpmwork/LeRobot_and_HF_Buckets#phase-1-collect-and-ingest

Source: Clem Delangue (X) | 2026-06-23