The A3 uses the Nvidia H100 GPU, the successor to the popular A100 that powered the previous A2. It is also used to power ChatGPT, the AI writer that kickstarted the generative AI race when it launched in November last year. Additionally, the A3 is the first VM where GPUs will use Google’s custom-designed 200 Gbps VPUs, which allows for ten times the network bandwidth of the previous A2 VMs.
The A3 will also make use of Google’s Jupiter data centre, which can scale to tens of thousands of interconnected GPUs, and “allows for full-bandwidth reconfigurable optical links that can adjust the topology on demand.” Google claims that the workload bandwidth of the A3 is indistinguishable from more expensive off-the-shelf non-blocking network fabrics, resulting in a lower total cost of ownership. The A3 also provides up to 26 exaFlops of AI performance, considerably improving the time and costs for training large ML models.
