Gemini ‘Ask About This Screen’ and YouTube Video Summaries Features Rolling Out to Users

Google first previewed these Gemini AI-powered features at Google I/O in May.

Written by Akash Dutta, Edited by David Delima | Updated: 28 August 2024 16:56 IST

Gemini ‘Ask About This Screen’ and YouTube Video Summaries Features Rolling Out to Users

Photo Credit: Google

The ‘Ask about this video’ feature only works with YouTube

Highlights

Gemini captures a temporary screenshot to process the screen
The feature was spotted in Google app version 15.33
Gemini can also capture a YouTube video's URL to generate a summary

Gemini for Android is getting two new features that will allow users to ask questions about their smartphone's current screen. Dubbed ‘Ask about this screen' and ‘Ask about this video', these tools capture a screenshot and process the content of the screen. Then, when prompted by the user, it can answer with relevant information that the user is seeking. The video feature, which only appears to be compatible with YouTube, can also provide video summaries provided there are captions added to the video.

Gemini Now Lets Users Ask Queries About Their Screen

Gadgets 360 staff members spotted the features in the Google app version 15.33.38.28.arm64 and later in several different smartphones. These appear on the top of the Gemini floating window when a user summons the AI assistant. Those who have access to the feature will see a separate rectangular strip with the text “Ask about this screen” and a screenshot icon. If Gemini was activated while playing a video on YouTube, the text is replaced with “Ask about this video” and the YouTube icon.

Gemini Ask about this screen feature

Tapping the icon for ‘Ask about this screen' captures a temporary screenshot within Gemini, and then the user can ask any query about the screen. The responses will only be based on what the AI can see in the screenshot, however, users can also prompt Gemini to access the Internet to find more information. One of its use cases could be asking for a short summary of a news article.

To ask a follow-up question, users can just tap the microphone icon or the keyboard icon at the bottom of the floating window. Notably, the captured screenshot is not saved on the device. Additionally, long screenshots are not supported with this feature.

The ‘Ask about this video' feature only works for a YouTube video where captions have been added. In our testing, we found that no other video platform, despite the status of captions, was compatible with the feature. However, on a YouTube video, using the feature allows users to ask questions about the video. The answers are generated based on the captions, and the video itself is not analysed.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Gemini, Google, AI, Artificial Intelligence, YouTube

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More