Elon Musk’s xAI Takes Its AI Chatbot Grok Open Source for Researchers and Developers

xAI has open-sourced the base model weights and network architecture of its Grok-1 LLM.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 18 March 2024 11:24 IST

Elon Musk’s xAI Takes Its AI Chatbot Grok Open Source for Researchers and Developers

Photo Credit: Midjourney/xAI

xAI’s Grok AI open-source LLM comes with a commercial usage licence

Highlights

Grok-1 is a Mixture-of-Experts model with 314 billion parameters
The open-sourced model is from the chatbot’s pre-training phase
xAI said the LLM is not fine-tuned for any specific application

Elon Musk's artificial intelligence (AI) venture, xAI, made its large language model (LLM) Grok-1 available in open source on March 17. Last week, the billionaire revealed on his social media platform X (formerly known as Twitter) that the AI chatbot will be open-sourced. Now, it is available for researchers and developers. Notably, the xAI developers stated that only the pre-trained LLM has been released in the public domain, which means while the weights and network architecture can be accessed to build on top of it, Grok does not come with any training data.

Announcing the open release, xAI stated in a blog post, “We are releasing the base model weights and network architecture of Grok-1, our large language model. Grok-1 is a 314 billion parameter Mixture-of-Experts model trained from scratch by xAI.” The AI firm also noted that the LLM is being provided in open source under the Apache 2.0 licence. Interested individuals can access the AI model from GitHub.

As per a report by VentureBeat, the Apache 2.0 licence includes commercial usage, as well as modification and redistribution. This means developers can build on top of the LLM, fine-tune it for specific applications, and sell it. However, the model cannot be trademarked and developers will have to mention any changes made to the original code.

Samsung Wins Patent for a Tri-Fold Smartphone With Specialised Barrier Layer for Improved Durability

But while a commercial licence is being provided, the version of Grok-1 that has been made available is from October 2023, before the model was trained using data from X and it gives a unique personality to the Grok chatbot. This means any researchers or developers intending to use the model will have to procure the data themselves.

xAI stated that the open-sourced Grok-1 is a 314 billion parameter Mixture-of-Experts (MoE) model. The parameter size is much larger than other LLMs in the public domain, such as Meta LLaMa 2 or the Mistral 8x7B. The larger the parameter size, the bigger the context window of the AI. This enables it to provide more contextually accurate and thorough responses.

Grok was unveiled on November 3, 2023, as a chatbot designed to compete with the major players in the segment. It was made available to a limited set of users who purchased the X Premium+ subscription. Musk also said at the time that Grok will get a standalone app.

Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Elon Musk, Grok, chatbots, xAI, Artificial Intelligence, AI

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More