Press Esc to close

Gemini Live's Guided Vision tells you where to point your camera

Official Google artwork for Guided Vision, with the words Guided Vision and Visual assistance tailored to your needs in white above a glowing blue pill shape on a dark background

Image: Google

Google is rolling out Guided Vision in Gemini Live on compatible Android devices. Share your camera, and Gemini describes what's around you, answers follow-up questions and, crucially, tells you how to move your phone when it can't get a good look. Google says it built the feature alongside the blind and low-vision community, a point it repeated when announcing the launch on X.

The camera coaching is the clever bit. A camera assistant is only useful if it can see what you're asking about, which is hard to guarantee when you can't see the screen yourself. With Guided Vision switched on, if your camera is too high, too close or slightly off to the side, Gemini will ask you to pan slowly to the right, tilt down or step back until it has a clear enough view to answer your question.

Google has a long list of everyday jobs in mind. You can have it read a tiny nutrition label, an appliance dial, your washing machine's cycles or a printed menu in a dimly lit restaurant. It can help you find a dropped earbud on the floor or the black pepper in a crowded spice cabinet, tell you whether a striped shirt goes with a pair of trousers, or describe the layout of an unfamiliar room.

Frame from Google's Guided Vision demo video. A phone running Gemini Live points its camera at shelves of house-shaped spice jars, and a caption bubble from Gemini reads: The oregano jar is on the second shelf from the top

Image: Google

To teach Gemini Live this kind of real-world back-and-forth, Google partnered with Aira, which visually interpreted data for tens of thousands of hours in total. More than 1,000 members of Aira's Trusted Tester network stress-tested the model across their daily routines, and Aira specialists worked with Google's teams to set and evaluate its safety guardrails.

It also works in multiple languages. Google says blind and low-vision communities in India, Brazil, Singapore, Indonesia, Japan and other countries tested it to make the descriptions and cues feel natural. And while accessibility is the focus, Google points out it's handy for older adults, people with low literacy, or anyone squinting at fine print in bad lighting.

There are three ways to start it. In the Gemini app, open your profile settings and turn on Use Guided Vision in Live, then open Gemini Live and tap to share your camera. You can also go to Android Settings > Accessibility > Vision assistance > Guided Vision and set up a shortcut, so the floating accessibility button, a two-finger swipe or pressing both volume keys launches it. TalkBack users can tap with three fingers to bring up the TalkBack menu and pick Guided Vision from there.

Google is clear about the limits, too. Like any generative AI tool, Guided Vision can make mistakes. It isn't a medical device, a mobility aid or a replacement for a white cane, and it isn't meant for navigation or obstacle detection. Google says people should keep using their usual mobility aids and safe-travel practices when getting around.

Google first previewed Guided Vision in its September Android Drop. To use it, you'll need Android 9 or newer and a region and language where Gemini Live is supported. Make sure your Gemini app and Android system software are fully updated.

Comments