Meta Releases 'Segment Anything Model 2' With AI-Powered Object Identification in Images and Videos

Meta’s Segment Anything Model (SAM 2) arrives a year after its predecessor was announce by the Facebook parent firm.

Written by Akash Dutta, Edited by David Delima | Updated: 30 July 2024 14:12 IST

Meta Releases 'Segment Anything Model 2' With AI-Powered Object Identification in Images and Videos

Photo Credit: Meta

Segment Anything Model 2 is based on a simple transformer architecture with streaming memory

Highlights

SAM 2 can segment and track any object in a video or image
Meta said the AI model can enable advanced video editing and generation
The open-source AI model is available on GitHub

Meta released a new artificial intelligence (AI) model on Monday that can perform complex computer vision tasks. Dubbed Segment Anything Model 2 (SAM 2), it follows after its predecessor that was launched last year and was incorporated in Instagram's Backdrop and Cutouts tools. The successor to the model now comes with advanced capabilities and the company said it can perform segment identification and tracking even on videos. Like most of Meta's large language models (LLMs), SAM 2 is also an open-source AI model.

Meta's Segment Anything Model 2 Unveiled

In a newsroom post, Meta announced the new AI model which focuses on segment analysis on videos primarily, while improving its image segmentation capabilities. Highlighting the accomplishments of its predecessor, Meta said the AI model was used in Instagram's Backdrop and Cutouts features, while marine scientists used it to “segment sonar images and analyse coral reefs, satellite imagery analysis for disaster relief, and in the medical field, segmenting cellular images and aiding in detecting skin cancer".

View this post on Instagram

A post shared by Mark Zuckerberg (@zuck)

SAM 2 is capable of object segmentation in an image and video as well as track it across different frames of a video in real-time. The AI can also track and segment objects in scenarios where the objects move fast, change in appearance, or are concealed by other objects or an entirely different scene.

The foundation model for prompt-based visual segmentation is built on a simple transformer architecture. It has a streaming memory that allows it to process videos in real-time. The company also claimed that the model was trained on its largest video segmentation dataset dubbed SA-V dataset.

Instagram Now Lets You Create AI Characters Using Meta AI Studio

Meta said the AI model can help ease the process of video editing or AI-based video generation, as well as to power new experiences in the company's mixed-reality ecosystem. The object tracking capability in videos can also assist in faster annotation of visual data to train other computer vision systems, the company added.

Since it is an open-source AI model, the company has hosted its weights on its GitHub page. Interested individuals can download and test out the AI model. Notably, it is licenced under the Apache 2.0 licence which allows for research, academic, and non-commercial usage.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Meta, AI, Artificial Intelligence, AI image generator, AI Models, LLM

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More