Real-time inference for robots at Physical Intelligence

Physical Intelligence (Pi) is building a general-purpose robotic intelligence system whose core Visual-Language-Action (VLA) model takes visual observations, natural-language instructions, and the robot's proprioceptive state as input and outputs motor commands in a continuous closed-loop control cycle. To validate every model revision at scale, Pi runs evaluations on real robots performing real tasks — amounting to thousands of inference cycles running 24/7 across a growing robot fleet — offloading that GPU compute to Modal's cloud platform, which simplified operations, enabled rapid experimentation with larger models, and added only 10–15 ms of network overhead. To handle the strict latency demands of the robotic control loop, Pi worked with Modal to build a QUIC-based, UDP transport with automatic NAT traversal, establishing a persistent bidirectional channel between the robot runtime and the Modal GPU container so the robot streams observations and receives action outputs continuously rather than issuing independent TCP requests.

Source: tools

Related