Nvidia Executive Claims AI-Powered Robots in the Future Will Be Trained on Simulation, Share a Hive Mind

Nvidia’s Jim Fan said that the majority of AI robots in the future will not be trained in isolation.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 26 December 2024 18:59 IST

Nvidia Executive Claims AI-Powered Robots in the Future Will Be Trained on Simulation, Share a Hive Mind

Photo Credit: Reuters

Jim Fan highlighted the example of the City of Tokyo releasing a 3D digital twin of the city on cloud

Highlights

AI-powered robots are also known as embodied AI
Jim Fan said most embodied AI agents will be born in simulation in future
These robots will also connect with other robots to complete tasks

Nvidia's Senior Research Manager and Lead of the Embodied AI division Generalist Embodied Agent Research (GEAR) Lab, Jim Fan, highlighted how artificial intelligence (AI) robots in the future could be trained and function. The executive claimed that soon embodied AI agents will be born in simulation where they will learn and gain expertise in specific tasks. He also highlighted that soon entire cities, houses, and factories will be transported to simulations, making training AI on real-world scenarios possible. He also postulated that future AI robots could share a hive mind.

Nvidia's Jim Fan Shares How AI Robots Could Be Trained in the Future

Fan shared his ideas in a post on X (formerly known as Twitter), as he highlighted that the City of Tokyo has released a 3D digital twin of the entire city in a high-resolution point cloud, and made it free to download. Sharing the development, the Nvidia executive stated that this trend of taking real-world institutions to digital simulations is only going to grow in the future.

“It's an inevitable trend that more and more cities, houses, and factories will be transported into simulations,” Fan said.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

Sharing the example, Fan also added that in the future, robots will not be trained in isolation. Currently, robots are trained in controlled environments where they can learn how to move around, identify objects, pick and drop things, and complete tasks. However, many have claimed that this method of training does not prepare the robots for various challenges they might face in real-world scenarios.

Offering a solution to the problem, the Nvidia research manager claimed that soon embodied AI agents will be simulated as an iron fleet. Notably, embodied AI agents are AI systems integrated with a physical body or a simulated embodiment, enabling them to interact with and perceive the world like humans or animals.

Fan said that future AI robots will be deployed in real-time graphics engines and will be scaled across a huge cluster. Such training methods will also produce trillions of high-quality training tokens, he added. Taking the hypothetical vision further, he said, “The majority of embodied agents will be born in sim, and transferred zero-shot to our real world when they are ready.”

This Iron Man-Inspired Robot Helps People with Paraplegia Walk Again

Once these AI robots have been deployed in the real world, such as factories or as assistants in offices and homes, they will also share a hive mind. Such hive minds can enable the embodied agents to learn from thousands of use cases as well as coordinate difficult tasks with multi-agent efforts.

Robots May Soon Detect Emotions by Measuring Your Sweat

While this might appear to be science fiction, Fan, who is leading the embodied AI division at Nvidia, is confident that this will be the direction to train highly efficient robots in the near future. And it appears that Nvidia is already moving in that direction. He shared that the company's Santa Clara headquarters buildings are “designed and rendered in Omniverse, a GPU-accelerated graphics platform, before materialising in atoms.”

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Nvidia, Embodied AI, AI Robots, Robots, Simulation

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More