Tencent InstantMesh, an AI Model Capable of 3D Rendering Static Images Unveiled

Tencent’s InstantMesh is an instant 3D mesh generation tool that uses a Large Reconstruction Model (LRM).

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 18 April 2024 18:16 IST

Tencent InstantMesh, an AI Model Capable of 3D Rendering Static Images Unveiled

Photo Credit: Hugging Face/Tencent

Tencent’s InstantMesh model is an upgrade over its older Instant3D framework

Highlights

Tencent says its AI model can generate 3D renders within 10 seconds
The AI model is available to use on Hugging Face
Earlier, Stability AI also released its Stable Video 3D rendering model

Tencent has released a new artificial intelligence (AI) model, dubbed InstantMesh, that can render 3D objects using a static photo. The new AI model is an upgrade over the company's older Instant3D framework and now uses a combination of a multiview diffusion model and a sparse-view reconstruction model based on the large reconstruction model (LRM) architecture. Tencent has also made the InstantMesh model open source and has offered a preview app for enthusiasts to test out its capabilities or generate and export 3D renders.

The company published a pre-print version of its research paper on arXiv. Notably, arXiv does not conduct peer reviews, so it is difficult to say whether the model has been assessed. However, the company has already made the AI model available in open source on Hugging Face, so developers can test its efficiency. For enthusiasts, there is an app view available as well where they can add a photo and watch it turn into a 3D render. We, at Gadgets 360, tested out the platform and found that the renders were created in under 10 seconds, as the company claimed. However, the quality of the renders felt quite low quality. An X (formerly known as X) user posted a video of using the AI model, and you can see the results below.

🤯InstantMesh from Tencent is insane - Super fast Image-to-3D with high quality output

⬇️ Link below - Generate a 3D model from a single image in 30 seconds for free 🔥🔥 pic.twitter.com/Dft4xF3vQm
— Victor M (@victormustar) April 15, 2024

Coming to the technology behind the AI model, the company uses two different architectures — a multiview diffusion model and an LRM architecture. The former helps in processing the image as input and generates different dimensions which are not visible in the image, and the LRM constructs an orbital view object that can be experienced in a 3D environment.

According to Tencent, InstantMesh solves the Janus problem in the world of 3D rendering. The Janus problem is a phenomenon in 3D rendering space where, since the model has to “imagine” different sides of the reference object and create them, it creates multiple canonical views of the object instead of a cohesive 3D object. The company solves the issue by using a novel view generator fine-tuned from Stable Diffusion.

The research paper also shared benchmark scores compared to different existing models, including Stability AI's Stable Video 3D, which was recently launched. Based on the scores, InstantMesh performed better than SV3D on Google Scanned Objects (GSO) and OmniObject3D (Omni3D) orbit views. SV3D fared better in a couple of parameters in the Omni3D benchmark, which corresponded to the resolution of the output, but Tencent said that it was intentional. “We argue that the perceptual quality is more important than faithfulness, as the “true novel views” should be unknown and have multiple possibilities given a single image as reference,” the company explained.

Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Tencent, Artificial intelligence, AI

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More