What is Project Astra? Gemini Live, Android XR, and everything else you need to know
Project Astra began as an ambitious research project designed to bolster Google’s AI efforts when it was first announced last year. Now, it’s a product you can actually use as Gemini Live on millions of smartphones — both iOS and Android devices. However, there’s a lot more to Project Astra than the live video and screen sharing capabilities that recently became available for Gemini users.
Focused initially on bringing versatile, multimodal AI capabilities to Android, Project Astra has evolved into a far-reaching research project aimed at achieving Google’s vision for a universal AI assistant. That means Project Astra is working on key AI features like agentic intelligence, native audio, memory, accessibility, tool use, and much more.
In other words, the upgraded Gemini Live experience isn’t the finish line for Project Astra — it’s just the beginning. If you’re curious about all Project Astra encompasses and how it’ll affect platforms like Android and Android XR, you’re in the right place. Let’s break down everything you need to know about Project Astra and how you can try it now.
Project Astra: What is it?
Project Astra is known for a few high-profile applications already, but Google’s definition of the project is simple. It’s a “research prototype exploring breakthrough capabilities for Google products — on the way to building a universal AI assistant,” according to the Project Astra website.
That’s a vague explanation of Project Astra, and it’s probably on purpose. While the project began as a vector for testing multimodal Gemini support for live audio and video, it’s expanded to become much more. Now, parts of Project Astra are operating under the Gemini app arm at Google, like the ones working on Gemini Live. Others are still working on experimental prototypes and applications for Google DeepMind.
Here are a few Project Astra features and applications we currently know about:
- Multimodal live video and screen sharing support (Gemini Live)
- Native audio dialogue
- Proactive responses without lag or interruption
- Context-aware dialogue
- Agent highlighting that understands the context of your device screen
- Tool use and interface control to complete actions on your behalf
- Personalized reasoning specific to the user
- Content retrieval from the web or past interactions
- Multimodal memory
- Visual Interpreter to support the blind and low-vision community
- Integration with prototype Android XR glasses
Currently, many of these features are still in the prototype stage and are only available to select testers. The big exception is Gemini Live, which we’ll talk about in detail later.
Project Astra: How it works
Project Astra is a tweaked version of Gemini with enhanced multimodal and agentic capabilities. For trusted testers, Project Astra is available as a standalone app. It works by utilizing multiple knowledge sources and advanced reasoning to provide more comprehensive assistance than a typical search engine or AI chatbot. This could include Gemini models, information found on the web, visual or auditory cues, data that’s on your screen, or personalized knowledge from your Google activity or past interactions.
It works by quickly processing all these data points at once, and the result is Project Astra being able to communicate with a user in natural language with accurate context. This is achieved by using native audio dialogue that can understand different accents and emotions in 24 languages.
Newly announced at Google I/O 2025 are Project Astra’s agentic capabilities. In other words, the prototype AI assistant will soon be able to take actions on your behalf. It can understand what’s on your screen in the context of your conversation, using visual cues to show you where to look. It can also utilize Google features such as Search, Gmail, Calendar, Maps, and more to complete tasks on your behalf.
You can see this in action in the I/O demo video for Project Astra, which demonstrates how a user can control their phone and solve daily problems hands-free.
Watch On
We see the person in the video asking Project Astra to find manuals, highlight key points in PDF files, and call businesses on their behalf. This is a perfect, real-world example of Google’s vision for Project Astra. In the end, this universal AI assistant should be able to control your phone — and perhaps your computer or smart glasses — without you needing to lift a finger.
Project Astra: Features and platforms
The long-term goal is for Project Astra to expand to multiple platforms, and for now, Google has its sights on smartphones and smart glasses. The company says that this prototype “integrates seamlessly with prototype glasses to see the world as you see it — for an even more immersive and helpful experience.”
After trying the experience on prototype Android XR glasses at Google I/O, it’s clear Astra aims to bring everything we’ve already covered — including multimodality, memory, visual analysis, and agentic control — to a wearable form factor. This is a long way out from reaching consumers, as Google’s smart glasses aren’t expected to hit shelves until 2026 at the earliest.
Available now is the Project Astra experience for Android and iOS smartphones, also known as Gemini Live. With the upgraded Gemini Live functionality, you can use your phone’s camera to have a conversation about what you see. You can also share your screen to talk about what’s currently on your smartphone. In both situations, Gemini can leverage knowledge stored in its models and the web for context.
Project Astra: Availability
Project Astra’s Gemini Live features are currently available for all Android and iOS devices that can install the Gemini app. They’re free for everyone to try now.
You can sign up to try the unreleased Project Astra app and its features for yourself by joining the Trusted Tester Program Waitlist.
Post Comment