Google reveals self-developed AI supercomputer with outstanding advantages

Google, a subsidiary of Alphabet Inc., on April 4 revealed new details about the supercomputers it uses to train its own artificial intelligence (AI) models, emphasizing that these systems are faster and more energy efficient than equivalent systems from rival technology company Nvidia Corp.

Google has developed an application-specific integrated circuit called the Tensor Processing Unit (TPU). This is the company’s fourth-generation TPU. Google uses these chips for more than 90% of its AI training, the process of feeding data through models to make them useful for tasks like answering text queries like humans or generating images.

In a scientific journal published the same day, Google detailed how it connected more than 4,000 chips together to form a supercomputer, using optical switches it developed itself to help connect the computers together.

Improving these connections is a key part of the competition among AI supercomputer companies, as the multilingual convolutional models that power cutting-edge technologies like Google’s Bard or OpenAI’s ChatGPT chatbot are growing too large to fit on a single chip. Instead, those large language models are stored on thousands of chips, then aggregated over weeks or longer to train intelligent information processing models.

Google's PaLM, the most comprehensive multi-language model to date, was trained over 50 days on two 4,000-chip supercomputers.

Google says its supercomputers make it faster and easier to reconfigure connections between chips, avoiding problems and tweaking to improve performance. They are 1.7 times faster and 1.9 times more energy efficient than a comparable system from Nvidia Corp., which was developed on the A100 chip and was available at the same time as the fourth-generation TPU.

Although details of the supercomputer have only just been revealed, the cutting-edge system has been deployed to a Google data center since 2020 in Mayes County, Oklahoma, in the US.

Google added that the startup Midjourney used Google's supercomputer to train its model, which generates new images based on requests with a few words in text form.

VNA

Source