MediaTek Announces Optimisation of Microsoft’s Phi-3.5 AI Models on Dimensity Chipsets

The models are now optimised for the MediaTek Dimensity 9400, 9300, and 8300 chipsets.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 24 December 2024 13:40 IST

MediaTek Announces Optimisation of Microsoft’s Phi-3.5 AI Models on Dimensity Chipsets

Photo Credit: MediaTek

The Phi-3.5 Mixture of Experts (MoE) supports 1,28,000 tokens context length

Highlights

Microsoft released the Phi-3.5 small language models in August
Phi-3.5 is also optimised for MediaTek's NeuroPilot SDK toolkit
MediaTek said its mobile platforms will support on-device AI inference

MediaTek announced on Monday that it has now optimised several of its mobile platforms for Microsoft's Phi-3.5 artificial intelligence (AI) models. The Phi-3.5 series of small language models (SLMs), comprising Phi-3.5 Mixture of Experts (MoE), Phi-3.5 Mini, and Phi-3.5 Vision, was released in August. The open-source AI models were made available on Hugging Face. Instead of being typical conversational models, these were instruct models that require users to input specific instructions to get the desired output.

MediaTek Optimises Dimensity Chipsets for Phi-3.5 SLMs

In a blog post, MediaTek announced that its Dimenisty 9400, Dimensity 9300, and Dimensity 8300 chipsets are now optimised for the Phi-3.5 AI models. With this, these mobile platforms can efficiently process and run inference for on-device generative AI tasks using MediaTek's neural processing units (NPUs).

Optimising a chipset for a specific AI model involves tailoring the hardware design, architecture, and operation of the chipset to efficiently support the processing power, memory access patterns, and data flow of that particular model. After optimising, the AI model will show reduced latency and power consumption, and increased throughput.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

MediaTek highlighted that its processors are not only optimised for Microsoft's Phi-3.5 MoE but also for Phi-3.5 Mini which offers multi-lingual support and Phi-3.5 Vision which comes with multi-frame image understanding and reasoning.

Notably, the Phi-3.5 MoE has 16x3.8 billion parameters. However, only 6.6 billion of them are active parameters when using two experts (typical use case). On the other hand, Phi-3.5 features 4.2 billion parameters and an image encoder, and the Phi-3.5 Mini has 3.8 billion parameters.

Coming to performance, Microsoft claimed that the Phi-3.5 MoE outperformed both Gemini 1.5 Flash and GPT-4o mini AI models on the SQuALITY benchmark which tests readability and accuracy when summarising a block of text.

While developers can leverage Microsoft Phi-3.5 directly via Hugging Face or the Azure AI Model Catalogue, MediaTek's NeuroPilot SDK toolkit also offers access to these SLMs. The chip maker stated that the latter will enable developers to build optimised on-device applications capable of generative AI inference using the AI models across the above mentioned mobile platforms.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Microsoft, MediaTek, MediaTek Dimensity 9400, MediaTek Dimensity 9300, MediaTek Dimensity 8300, AI, Artificial Intelligence, Phi

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More