Model Releases
Decoupled DiLoCo: A new frontier for resilient, distributed AI training
Decoupled DiLoCo is a distributed architecture that enables training of large language models across distant data centers using lower bandwidth and improved hardware resilience by dividing training in
Decoupled DiLoCo is a distributed architecture that enables training of large language models across distant data centers using lower bandwidth and improved hardware resilience by dividing training into decoupled compute "islands" with asynchronous data flow. Built on Google's Pathways framework, it allows asynchronous training where chip failures in one area don't interrupt others, and successfully trained a 12 billion parameter model across four U.S. regions more than 20 times faster than conventional synchronization methods. The approach enables mixing different hardware generations in a single training run while maintaining comparable ML performance.
Source: Google DeepMind | 2026-04-22