Meta Llama 3.1 405B Released as Company's Largest Open Source AI Model to Date, Beats OpenAI's GPT-4o

Meta has also released upgraded versions of its Meta Llama 3 8B and 70B models.

Written by Akash Dutta, Edited by David Delima | Updated: 24 July 2024 12:28 IST

Meta Llama 3.1 405B Released as Company's Largest Open Source AI Model to Date, Beats OpenAI's GPT-4o

Photo Credit: Meta

The AI model supports English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai languages

Highlights

Meta says the Llama 3.1 405 outperforms GPT-4o in some benchmarks
To download the AI model, one would need 750GB of storage space
Meta Llama 3.1 405B AI model was trained on more than 15 trillion tokens

Meta on Tuesday released its latest and largest artificial intelligence (AI) model to the public. Called Meta Llama 3.1 405B, the company says the open-source model outperforms major closed AI models such as GPT-4, GPT-4o, and Claude 3.5 Sonnet across several benchmarks. The previously released Llama 3 8B and 70B AI models have also been upgraded. The newer versions were distilling from the 405B model and now offer a 1,28,000 tokens context window. Meta claims both of these models are now the leading open-source large language models (LLMs) for their sizes.

Meta Llama 3.1 405B AI Model Released

Announcing the new AI model in a blog post, the technology conglomerate said, “Llama 3.1 405B is the first openly available model that rivals the top AI models when it comes to state-of-the-art capabilities in general knowledge, steerability, math, tool use, and multilingual translation.”

Notably, 405B here refers to 405 billion parameters, which can be understood as the LLM's number of knowledge nodes. The higher the parameter size, the more adept an AI model is in handling complex queries. The context window of the model is 128,000 tokens. It supports English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai languages.

Meta Seeks Patent for This Feature Inspired by the Apple Vision Pro

The company claims the Llama 3.1 405B was evaluated on more than 150 benchmark tests across multiple expertise. Based on the data shared in the post, Meta's AI model scored 96.8 in the Grade School Math 8K (GSM8K) GPT-4's 94.2, GPT-4o's 96.1, and Claude 3.5 Sonnet's 96.4. It also outperformed these models in the AI2's Reasoning Challenge (ARC) benchmark for science proficiency, Nexus for tool use, and the Multilingual Grade School Math (MGSM) benchmark.

Meta's largest AI model was trained on more than 15 trillion tokens with more than 16 thousand Nvidia H100 GPUs. One of the major introductions in the Llama 3.1 405B is the official support for tool-calling which will allow developers to use Brave Search for web searches, Wolfram Alpha to perform complex mathematical calculations, and Code Interpreter to generate Python code.

Since the Meta Llama 3.1 405B is available in open source, individuals can access it from either the company's website or from its Hugging Face listing. However, being a large model, it requires roughly 750GB of disk storage space to run. For inferencing, two nodes on Model Parallel 16 (MP16) will also be necessary. Model Parallelism 16 is a specific implementation of model parallelism where a large neural network is separated into 16 devices or processors.

Apart from being available publicly, the model is also available on major AI platforms by AWS, Nvidia, Databricks, Groq, Dell, Azure, Google Cloud, Snowflake, and more. The company says a total of 25 such platforms will be powered by Llama 3.1 405B. For safety and security, the company has used Llama Guard 3 and Prompt Guards, two new tools that safeguard the LLM from potential harm and abuse.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Meta, Llama, AI, Artificial Intelligence, AI Model, LLM

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More