Prime Highlights: 

Google is rolling out new real-time AI capabilities, allowing Gemini to “see” screens and camera feeds, enabling users to interact with visual content and ask questions. 

These advanced features are being introduced to users with the Google One AI Premium plan, offering enhanced functionality. 

Key Background: 

Google has begun rolling out new AI-powered capabilities for its Gemini platform, offering users the ability to interact with real-time video and screen feeds. These features are now available to select Google One AI Premium subscribers, providing an advanced level of functionality. The rollout follows nearly a year after Google first demonstrated the underlying technology, known as “Project Astra,” which powers these new capabilities. 

A key aspect of the new features is Gemini’s ability to “see” screens and camera feeds in real-time, allowing users to ask questions about the visual content. According to a Google spokesperson, the new tools are being introduced gradually, starting with the Google One AI Premium plan. The rollout has already been noted by some users, with one Reddit user showcasing the feature on their Xiaomi phone. 

One of the standout features is Gemini’s screen-reading capability, which enables the AI to interpret and respond to the content displayed on a user’s screen. The second major capability involves live video interpretation, where Gemini can analyze the feed from a smartphone camera in real-time. In a demonstration video shared by Google, a user asks Gemini to suggest a paint color for freshly-glazed pottery, showcasing the practical applications of this technology. 

These new features position Gemini at the forefront of AI assistants, reinforcing Google’s leadership in the field. This development comes as Amazon prepares to debut its Alexa Plus upgrade in limited early access, and Apple faces delays with its upgraded Siri. Despite competition, Samsung’s Bixby continues to be the default assistant on its devices, while Gemini remains the standard for Google’s smartphones. This advancement underscores Google’s commitment to enhancing its AI offerings and providing more interactive and practical tools for its users.