What does it take to scale up a data center to accommodate large AI clusters powered by 32,000 GPUs? Loi Nguyen, EVP and GM, Optical from Marvell explains:
- The importance of connectivity in scaling up data centers: Not only do you need the GPUs themselves, but you also need high-speed optics and low latency switches to connect these GPUs together. A 32k GPU cluster would require around 70,000 optics connected via approximately 2,000 switches.
- The power consumption of a large-scale data center: A 32k GPU cluster would consume about 45 megawatts of power, which is significantly more than the typical data center’s 32 megawatts. This presents a major scaling challenge.
- The role of Marvell in scaling bandwidth: The optic industry has done a good job in scaling optics, with optic bandwidth increasing by about 400% in the past 10 years. 800G optics are here. Next up, 1.6 Tbps. And beyond?
Have a tech update that you want to brief us on? Contact info@nextgeninfra.io!