In the coming months, Google will introduce a new way to search visually, with the ability to ask questions about what the user see.
This new tool is an aplication of Google’s Multitask Unified Model (MUM), which was presented as a milestone for understanding information at Google I/O earlier in 2021.
With the new cappability users can can tap on the Lens icon when looking at a picture of a shirt, for example, and ask Google to find the same pattern but on another article of clothing, like socks or handbags, etc..
This helps when the user is looking for something that might be difficult to describe accurately with words alone.
Instead of writing “white floral Victorian socks” and perhaps not finding the exact pattern the searcher is looking for, combining images and text into a single query makes it easier efor you finding what you are inetested in as well as letting the user asking google in a more natural way.
The point-and-ask mode of searching will make it easier to find whatever the user looks for not only using a photo, but also find the exact moment in a video that can help the search.
