Mistral Launches 8X22B Mixture of Experts AI Model in Open Source, Benchmarks Reveal Big Upgrades

Mistral released the open-source 8X22B AI model via a torrent magnet link.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 11 April 2024 15:05 IST

Mistral Launches 8X22B Mixture of Experts AI Model in Open Source, Benchmarks Reveal Big Upgrades

Photo Credit: Pexels/Tara Winstead

Mistral’s model outperforms Gemini 1.0 Ultra in Hellaswag benchmark, as per scores posted on Hugging Face

Highlights

Mistral 8x22B is an autocomplete data model
Mistral could release an instruct or chat version of the model later
The model follows after Mistral 8x7B model released last year

Mistral released its latest artificial intelligence (AI) model, 8X22B, on Wednesday. The mixture-of-expert AI model follows the 8X7B model released in December 2023 and offers a larger parameter size. The company known for its fully open-source AI models released the latest model unconventionally, without any announcement posts or blog posts accompanying it. While there are no benchmarks provided by the AI firm itself, the Hugging Face community users tested it and posted benchmark scores of the 8X22B model. The results appear to close the gap with the closed models from OpenAI and Google.

The official X (formerly known as Twitter) account of Mistral released the 8X22B AI model via a torrent magnet link, continuing its unconventional method of dropping AI models without an announcement and directly for people to download. Mistral is also one of the only truly open-source platforms that not only makes the weights open but also the entire architecture. However, it should be noted that most devices are not equipped to run powerful AI models on-device, and it might cause the device to malfunction. The total file size is 262GB.

Mistral's 8X22B is an autocomplete AI model. These are typically different from instruct or chat variants of AI models. OpenAI's ChatGPT and Google's Gemini AI are chat models, whereas Meta's Code Llama 7B and 13B are instruct models. Chat models are those which can understand natural language and contextual queries to provide the correct response. Instruct models are mainly used by developers who ask the AI model to perform a specific task. In contrast, an autocomplete model completes the sentence that has been provided in the prompt.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

Early benchmark scores based on a Hugging Face community post show that the latest Mistral model offers significant upgrades over its predecessors and closes the gap with other major models. In the Hellaswag benchmark, Mistral 8X22B scored 88.9, closely trailing GPT-4 at 95.3, Claude 3 Opus at 95.4, and Gemini 1.5 Pro at 92.5. However, it outscores GPT-3.5 at 85.5 and Gemini 1.0 Ultra at 87.8.

Comparing the benchmarks with Mistral's own models, the 8X22B falls between Mistral-medium and Mistral-large models. With a later release of an instruct or chat variant, it might even fare similarly to Mistral-large.

Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Mistral, LLM, Artificial Intelligence, AI, AI models

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More