Galaxy S25 and Pixel phones will be first to get Google's Project Astra features

At Google I/O 2024, the company told us about Project Astra — a prototype multimodal AI agent that can understand the world around its users. So far, Astra has shown impressive real-time interaction capabilities. It has been in the hands of Google’s trusted testers for several months, who have used it for a range of tasks, including generating real-time recipe recommendations, creating stories based on artwork, taking guided city tours, solving math problems, and more. But today, Google has announced that the advanced AI agent is ready for its public debut.

Project Astra allows users to open their phone’s camera and start asking questions about what the viewfinder is seeing in real time. Google calls this Astra’s live video streaming feature. Think of it like Google Lens supercharged by a conversational AI. Users can also share their phone screens with Astra to unlock next-level interactive assistance, allowing it to guide them through tasks, provide explanations, or even directly interact with elements on their screen.

On the sidelines of the Galaxy S25 launch, Google has announced that Project Astra’s screen sharing and live video streaming features will be coming to the standalone Gemini mobile app for Android in the “coming months.”

That means you’ll soon be able to use the Gemini app to ask the AI about your surroundings in real-time. You’ll also be able to share your screen and interact with the AI using real-time voice.

There’s a catch, though. Project Astra’s features will initially be available exclusively on the Galaxy S25 series and Pixel phones. They should eventually see a wider rollout through the Gemini Android app.