- Meta says Muse Wonder training on Prometheus held more than 85% goodput in pre-training and more than 90% in reinforcement learning runs.
- Job interruptions fell more than 99% from a mid-2025 peak of nearly 1,000 per 8,000 GPUs per day, according to Meta.
- Prometheus runs Nvidia GB200 and GB300 racks today, with Vera Rubin NVL72 systems arriving next and the Louisiana Hyperion site built on the same blueprint.
Meta has published the first detailed engineering account of Prometheus, its gigawatt-scale AI supercluster in New Albany, Ohio, confirming that the completed first phase trained its Muse Wonder frontier models and that the company is now scaling the site toward roughly 1 million GPUs. The post from Meta's infrastructure team says the cluster already links hundreds of thousands of Nvidia GPUs across multiple buildings, some up to 100 km apart, so that they run as a single machine.
Prometheus trained Muse Wonder on hundreds of thousands of Nvidia GPUs
Two years ago the New Albany campus served ordinary web traffic. Meta rebuilt it in phases, mixing conventional data halls, AI-optimized buildings and rapid-deployment structures, and stitched them together with what it describes as millions of miles of fiber strand and tens of thousands of switches. Each building-level Ethernet fabric connects up to 20,736 GPUs, and an aggregation network joins those fabrics across distances from 2 km to more than 100 km.
The hardware spans three Nvidia generations. Catalina racks use the GB200 NVL72, Clemente racks use the GB300 NVL72 with 288 GB of memory per GPU, and a third platform, San Miguel, will bring Vera Rubin NVL72 racks drawing more than 170 kW each.
| Platform | Nvidia system | Memory per GPU | Rack power |
|---|---|---|---|
| Catalina | GB200 NVL72 | 192 GB HBM3e | ~120 kW (two-rack pair) |
| Clemente | GB300 NVL72 | 288 GB HBM3e | ~135 kW |
| San Miguel | Vera Rubin NVL72 | 288 GB HBM4 | 170 kW+ |
Source: Engineering at Meta, October 19, 2026.
Reliability, measured in GPU hours, is the real engineering result
The most useful numbers in Meta's account concern failure. A 72-GPU rack shares one high-bandwidth domain, which Meta says enlarges the blast radius of a single fault ninefold compared with the eight-GPU servers of the H100 era. Meta answers that by running each rack as a 64-GPU slice with eight in-rack spares and holding about 2% extra racks in reserve. For a training run that needs 32,768 active GPUs, it provisions 37,656, so roughly 4,900 GPUs, about 13% of the footprint, sit ready as spares.
Meta reports a mean time between interruptions of about 47,000 GPU hours during the Muse Wonder runs and more than 54,000 GPU hours after later firmware fixes, with a stated goal of about 100,000. The company credits a co-engineering arrangement with Nvidia, whose teams work on site at New Albany and share telemetry for joint triage of every halt.
Meta keeps about one GPU in eight on standby for its largest training runs, a reminder that the cost of frontier AI infrastructure now includes a large reserve of hardware bought specifically to sit idle until something breaks.
The disclosure follows Meta's release of Muse Spark Open and its push into in-house silicon, and it gives the clearest view yet of the AI infrastructure behind those models. Prometheus shows that Meta's frontier ambitions now rest as much on keeping hundreds of thousands of GPUs running in lockstep as on the models themselves, and the 99% cut in interruptions is the figure that explains how the Muse Wonder runs finished at all.
Santage is committed to independent, transparent journalism. This article is produced in accordance with Santage's Editorial Standards and aims to provide accurate and timely information. Readers are encouraged to verify information independently.