Malicious Machine Learning Models Discovered on Hugging Face: Report

Cybersecurity researchers found that malware was being distributed on Hugging Face by abusing Pickle file serialisation.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 10 February 2025 18:49 IST

Highlights

The malicious code in these ML models is called “nullifAl”
Hugging Face has removed the ML models after it was reported
The malicious ML models were stored in PyTorch format

Malicious Machine Learning Models Discovered on Hugging Face: Report

Pickle is a Python module used for serialising and deserialising ML model data

Photo Credit: Hugging Face

Hugging Face, the artificial intelligence (AI) and machine learning (ML) hub, is said to contain malicious ML models. A cybersecurity research firm discovered two such models that contain code that can be used to package and distribute malware to those who download these files. As per the researchers, threat actors are using a hard-to-detect method, dubbed Pickle file serialisation, to insert malicious software. The researchers claimed to have reported the malicious ML models, and Hugging Face has removed them from the platform.

Researchers Discover Malicious ML Models in Hugging Face

ReversingLabs, a cybersecurity research firm, discovered the malicious ML models and detailed the new exploit being used by threat actors on Hugging Face. Notably, a large number of developers and companies host open-source AI models on the platform that can be downloaded and used by others.

The firm discovered that the modus operandi of the exploit involves using Pickle file serialisation. For the unaware, ML models are stored in a variety of data serialisation formats, which can be shared and reused. Pickle is a Python module that is used for serialising and deserialising ML model data. It is generally considered an unsafe data format as Python code can be executed during the deserialisation process.

DeepSeek-R1 Is Not Fully Open-Source, Hugging Face Wants to Change That

In closed platforms, Pickle files have access to limited data that comes from trusted sources. However, since Hugging Face is an open-source platform, these files are used broadly allowing attackers to abuse the system to hide malware payloads.

During the investigation, the firm found two models on Hugging Face that contained malicious code. However, these ML models were said to escape the platform's security measures and were not flagged as unsafe. The researchers named the technique of inserting malware “nullifAI” as “it involves evading existing protections in the AI community for an ML model.”

These models were stored in PyTorch format, which is essentially a compressed Pickle file. The researchers found that the models were compressed using the 7z format which prevented them from being loaded using PyTorch's “torch.load()” function. This compression also prevented Hugging Face's Picklescan tool from detecting the malware.

The researchers claimed that this exploit can be dangerous as unsuspecting developers who download these models will unknowingly end up installing the malware on their devices. The cybersecurity firm reported the issue to the Hugging Face security team on January 20 and claimed that the models were removed in less than 24 hours. Additionally, the platform is said to have made changes to the Picklescan tool to better identify such threats in “broken' Pickle files.

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.