Google Launches Guided Vision for Real-Time AI Camera Descriptions
Google introduces Guided Vision within Gemini Live and TalkBack on Android, offering real-time audio descriptions of surroundings and fine print for users.
Google is rolling out a new accessibility tool called Guided Vision for compatible Android devices. Announced as part of a recent set of Pixel updates, the feature uses artificial intelligence to deliver real-time audio descriptions of objects, small text, and environments captured by a smartphone camera.
How Guided Vision Works
Users can activate Guided Vision by sharing their camera within Gemini Live or through Google TalkBack. Additionally, individuals using devices running Android 9 and above can configure an accessibility shortcut directly from the Settings app. The functionality is built to assist users who are blind or have low vision, as well as anyone needing contextual help in specific scenarios. Similar to Apple's VoiceOver Live Recognition feature on the iPhone and Vision Pro, the tool aims to make physical surroundings more accessible.
When pointing a phone camera at an object, the system can read fine print, describe details, identify surrounding items, and answer follow-up questions. For instance, a user can ask Gemini to read an expiration date on a product. If an item is outside the current camera frame, Gemini will provide audio cues to help users correctly align their device.
Safety and Limitations
While the technology offers robust descriptive capabilities, Google explicitly cautions users against relying on Guided Vision for navigation, safe-travel guidance, or obstacle detection. The company notes that the feature is not intended to replace traditional mobility tools such as canes.
What it means for developers
As multimodal AI features like Guided Vision become standard across consumer platforms, developers are increasingly looking to integrate advanced vision and audio capabilities into their own applications. Building these experiences often requires testing multiple foundational models to find the right balance of speed and accuracy. Developers can try top AI models cheaply through one API at https://apixoai.online to experiment with similar multimodal workflows.
Source: Google’s new Guided Vision feature can help you read the fine print — The Verge AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
