Google now lets you search videos with voice

Google’s new AI tools are aimed at improving search and expanding it with multimodality.

One of the major focus areas of AI-developing companies this year has been expanding their services with multimodality. This enables the processing of not only written content but also audio, pictures, and videos.

Google discussed its multimodality capabilities extensively at its developer conference, and now it offers some of these tools to users, as well as several other AI-based improvements.

Starting this week, users can record videos and ask questions about objects in motion using voice commands. Google provided an example where, at an aquarium, you could film interesting fish and then ask about them using a voice query.

AI would analyze both the video content and the spoken query to deliver relevant information.

To use the feature, you need to open Lens, a visual-based search tool, on the Google app and hold down the shutter button to record while asking your question out loud. This will produce an AI Overview and resources from across the web.

The feature is currently available for Search Labs users enrolled in the “AI Overviews and More” experiment, with support for English queries.

Voice queries for pictures, though, are available for all users via their Google app. They can access this by holding the shutter button while pointing a camera at an object and asking a question.

Other improvements

In addition to video search, Google is introducing several other AI-based improvements. One of these will result in showing more information about products that users are looking for, such as previews, price information across retailers, and where to buy them.

AI will also be used to organize some of the search queries, starting with recipes on mobile devices for US users. After typing a search for a recipe, you will get top recipes as well as the ability to search separate ingredients, find new recipes with mentioned ingredients, and more.

Meanwhile, AI Overviews will now officially include ads, though some users may have already seen them. Google has been testing the ads since May.

Finally, Google is updating its Circle to Search tool with a Shazam-like feature. Circle to Search, now available for 150 million Android users, will be able to identify songs without switching apps.