Meta Releases AI Coding Model Code Llama 70B; Calls It ‘Largest’ and ‘Best-Performing’ in Llama Family

Code Llama 70B scored 53 percent in accuracy on the HumanEval benchmark and 62.4 percent on the MBPP benchmark, beating GPT 3.5 in both metrics.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 31 January 2024 12:46 IST

Meta Releases AI Coding Model Code Llama 70B; Calls It ‘Largest’ and ‘Best-Performing’ in Llama Family

Photo Credit: Meta

Code Llama 70B is available in the foundational model version, as well as Python and Instruct versions

Highlights

Code Llama 70B is free for research and commercial use
Meta’s Code Llama 70B is built on Llama 2 foundational model
It was trained on 1 trillion tokens of code and code-related data

Meta has recently released Code Llama 70B, the latest update to the company's open-source artificial intelligence (AI) coding model. Announcing the release, the California-based tech conglomerate called it “the largest and best-performing model in the Code Llama family.” As per the company's report, Code Llama 70B scored 53 percent in accuracy on the HumanEval benchmark, highlighting capabilities nearing OpenAI's GPT 4 that scored 67 percent. The latest AI assistant joins the company's existing coding models Code Llama 7B, Code Llama 13B, and Code Llama 34B.

Meta CEO Mark Zuckerberg announced Code Llama 70B via a Facebook post and said, “We're open sourcing a new and improved Code Llama, including a larger 70B parameter model. Writing and editing code has emerged as one of the most important uses of AI models today. [..] I'm proud of the progress here, and looking forward to including these advances in Llama 3 and future models as well.”

Code Llama 70B is available in three versions — the foundational model, the Code LLama - Python, and Code Llama - Instruct, as per Meta's blog post. Python is for the specific programming language and Instruct has natural language processing (LNP) capabilities, which means you can use this even if you do not know how to code.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

The Meta AI coding assistant is capable of generating both codes and natural language responses, the latter being important for explaining codes and answering queries related to them. The 70B model has been trained on 1 trillion tokens (roughly 750 million words) of coding and code-related data. Like all LLama AI models, Code Llama 70B is also free for research and commercial purposes. It is hosted on Hugging Face, a coding repository.

Coming to benchmarks, the company has posted its scores in accuracy and compared them against all rival coding-focussed AI models. On the HumanEval benchmark, 70B scored 53 percent and on the Mostly Basic Python Programming (MBPP) benchmark, it received 62.4 percent. In both benchmarks, it has outscored OpenAI's GPT 3.5 which received 48.1 percent and 52.2 percent, respectively. GPT 4 posted only its HumanEval accuracy scores online and netted 67 percent, edging the Llama Code 70B parameter by a fairly healthy margin.

In August 2023, Meta released Code Llama, which was based on the Llama 2 foundational model and trained specifically on its coding-based datasets. It accepts both codes and natural language for prompts and can generate responses in both, as well. Code Llama can generate, edit, analyse, and debug codes. Its Instruct version can also help users understand the codes in natural language.

Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Meta, Artificial intelligence, AI

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More