Google, alongside the launch of the new Pixel 10 series phones, also announced Visual Guidance in Gemini Live, along with other improvements that not only make it sound more natural but also allows it to connect to more Google apps, expanding its capabilities further.
Visual Guidance in Gemini Live
Now, when you share your camera, Gemini not only sees what you see, but it also provides visual guidance by highlighting things directly on your screen. “These real-time visual cues create a powerful new way to learn and solve problems together,” said Google.
As for how it works, Google explained that with some examples such as: “Can’t decide between two pairs of sneakers? Gemini can see them both and highlight the one that best matches the outfit you’re envisioning. Or maybe you’re trying to figure out the correct tool to use. Point your camera at your tool box, and Gemini can point out the right one.”
Visual guidance in Gemini Live will be available on the Pixel 10 series as soon as devices hit shelves on August 28, and will start rolling out to other Android devices that week. We’ll expand to iOS devices in the coming weeks.
Support for New Google Apps in Gemini Live
“Juggling your schedule for the week? Talk through your appointments on your Google Calendar, and ask Gemini to set a reminder in Google Tasks for you to pick up your prescription before the pharmacy closes. Or, if you’re brainstorming a new recipe for dinner, you can ask Gemini to add all the ingredients to a new shopping list in Google Keep,” said Google.
Also Read: Google Tensor G5: How Has it Improved and What’s New?
These capabilities are powered by integrations with Google Calendar, Keep, and Tasks apps, available starting today. And soon, the company will be adding the Messages, Phone, and Clock apps and bringing additional capabilities into Google Maps integration, so Gemini Live will be able to help you in even more ways.
New Model Updates for Gemini Live
Google is also launching new model updates soon that will dramatically improve how Gemini Live uses the key elements of human speech, like intonation, rhythm and pitch, allowing for a more responsive and expressive conversation. In the coming weeks, you’ll be able to:
- Experience more natural and intuitive interactions. Gemini will respond more appropriately to what you say. If you’re talking about a stressful topic, it might respond with a calmer, more measured voice.
- Control how Gemini speaks. You’ll be able to ask Gemini to speak more slowly if you’re taking notes, faster if you’re in a hurry or even in a fun accent to liven things up.
- Enjoy dramatic storytelling. Ask Gemini to tell you about the Roman empire from the perspective of Julius Caesar himself, and get a rich, engaging narrative complete with character accents.