Nvidia Tops Benchmark Tests Speed of Systems Training ChatGPT-Like Chatbots

The MLPerf benchmark is based on GPT-3, an AI model used to train ChatGPT, the viral chatbot developed by OpenAI.

By Reuters | Updated: 28 June 2023 19:22 IST

Nvidia Tops Benchmark Tests Speed of Systems Training ChatGPT-Like Chatbots

Only two chip firms -- Nvidia and Intel's Habana Labs -- submitted results for the benchmark

Highlights

Nvidia's largest system submitted in partnership with AI cloud startup
Generally, more chips and a bigger system mean faster training
Habana Labs is an AI chip company acquired by Intel

MLCommons, a group that develops benchmark tests for artificial intelligence (AI) technology, on Tuesday unveiled results for a new test that determines system speeds when training algorithms used for chatbots like ChatGPT -- and Nvidia came out on top.

The MLPerf benchmark is based on GPT-3, an AI model used to train ChatGPT, the viral chatbot developed by OpenAI and backed by Microsoft. However, because the model is huge, the benchmark only uses a representative portion.

"This was our most expensive benchmark so far," MLCommons Executive Director David Kanter told Reuters. "We spent over 600K hours of accelerator compute time to develop it, plus some fantastically talented engineers."

Kanter declined to disclose the cost of development, only saying it was in the millions of dollars.

Only two chip firms -- Nvidia and Intel's Habana Labs -- submitted results for the benchmark, with the fastest time coming from systems using the latest H100 chip from Nvidia, the uncontested leader in hardware for training AI.

Nvidia's largest system submitted in partnership with AI cloud startup CoreWeave used 3,584 H100 chips, resulting in a training time of 10.94 minutes. Habana Labs, an AI chip company acquired by Intel, ran the benchmark in 311.945 minutes with a much smaller system equipped with 384 Gaudi2 chips.

Generally, more chips and a bigger system mean faster training.

Intel's Jordan Plawner, senior director of AI Products, said the results demonstrated the potential of Gaudi2, which will have a software update in September to boost speed.

"You will get a 1.5X to 2X speed up on the Habana results. So that's when we see Habana Gaudi2 being really competitive and lower priced than H100," Plawner told Reuters.

Plawner declined to say how much a Gaudi2 chip costs, but said the industry needs a second supplier of chips for AI training, and the MLPerf results show Intel can fill that need.

© Thomson Reuters 2023

Apple unveiled its first mixed reality headset, the Apple Vision Pro, at its annual developer conference, along with new Mac models and upcoming software updates. We discuss all the most important announcements made by the company at WWDC 2023 on Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.