Nvidia Releases Chat With RTX, an AI Chatbot That Runs Locally on Windows PC

To run Nvidia’s Chat with RTX, a Windows PC with RTX 30 or 40 series GPU with a minimum of 8GB VRAM is required.

Written by Akash Dutta, Edited by Manas Mitul | Updated: 14 February 2024 17:30 IST

Nvidia Releases Chat With RTX, an AI Chatbot That Runs Locally on Windows PC

Photo Credit: Nvidia

Since Chat with RTX runs locally, the data fed to it is not shared with a third party

Highlights

Nvidia’s Chat with RTX is a demo app and is available for free
Chat with RTX feeds on local data provided by the user
The Nvidia chatbot can summarise YouTube videos and documents

Nvidia has released an artificial intelligence (AI)-powered chatbot called Chat with RTX that runs locally on a PC and does not need to connect to the Internet. The GPU maker has been at the forefront of the AI industry since the generative AI boom, with its advanced AI chips powering AI products and services. Nvidia also has an AI platform that provides end-to-end solutions for enterprises. The company is now building its own chatbots, and Chat with RTX is its first offering. The Nvidia chatbot is currently a demo app available for free.

Calling it a personalised AI chatbot, Nvidia released the tool on Tuesday (February 13). Users intending to download the software will need a Windows PC or workstation that runs on an RTX 30 or 40-series GPU with a minimum of 8GB VRAM. Once downloaded, the app can be installed with a few clicks and be used right away.

Since it is a local chatbot, Chat with RTX does not have any knowledge of the outside world. However, users can feed it with their own personal data, such as documents, files, and more, and customise it to run queries on them. One such use case can be feeding it large volumes of work-related documents and then asking it to summarise, analyse, or answer a specific question that could take hours to find manually. Similarly, it can be an effective research tool to skim through multiple studies and papers. It supports text, pdf, doc/docx, and xml file formats. Additionally, the AI bot also accepts YouTube video and playlist URLs and using the transcriptions of the videos, it can answer queries or summarise the video. For this functionality, it will require internet access.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

As per the demo video, Chat with RTX essentially is a Web server along with a Python instance that does not contain the information of a large language model (LLM) when it is freshly downloaded. Users can pick between Mistral or Llama 2 models to train it, and then use their own data to run queries. The company states that the chatbot leverages open-source projects such as retrieval-augmented generation (RAG), TensorRT-LLM, and RTX acceleration for its functionality.

According to a report by The Verge, the app is approximately 40GB in size and the Python instance can occupy up to 3GB of RAM. One particular issue pointed out by the publication is that the chatbot creates JSON files inside the folders you ask it to index. So, feeding it your entire document folder or a large parent folder might be troublesome.

Is the Samsung Galaxy Z Flip 5 the best foldable phone you can buy in India right now? We discuss the company's new clamshell-style foldable handset on the latest episode of Orbital, the Gadgets 360 podcast. Orbital is available on Spotify, Gaana, JioSaavn, Google Podcasts, Apple Podcasts, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Nvidia, Artificial intelligence, AI, chatbots

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More