May 02, 2016

This blog post is a work in progress and will detail the process that I took to create the Visaurant project.

Visaurant (as you can see in the video below) is a reimagination of the way users search through images that they are interested in. One prime use case for Visaurant is in sorting and filtering through food images (hence VIS -ual rest- AURANT).

Background

So why do we want a visual search application?

Images represent a wealth of information. As cliche as it sounds, 'a picture is worth a thousand words' especially when it comes to looking for food items that seem appetizing. Speaking from personal experience, I often use apps like Yelp by:

1. Going to the images of a particular restaurant
2. Looking for the pictures that look good
3. Reading the caption
4. Going back to the reviews to find the reviews regarding the item that I saw

While this process works, it is by no means streamlined. Having to go back and forth between the images, captions, and reviews is a hassle and quite often overwhelming because there are so many images to sift through. So although images are powerful and contain a lot of information, if the images are not ordered or organized then it becomes difficult to find what I care about (like the example below...).

Visaurant aims to change that by:

1. Clustering visually similar images to allow users to quickly narrow down to images of interest
2. Parsing through captions to extract reviews that are relevant to the image that was selected

What is happening in the video?

1. User gets to select a group of images from a restaurant that looks interesting (4 selections)
2. Recommendations will be made based on the user's selections
3. User can once again select the restaurant that looks most appealing
4. Restaurant specific page will show up allowing the user to hover over images that look interesting and images within the same visual cluster will be highlighted
5. User can select an image and all images will be filtered for images in the same cluster
6. After filtering down to a single cluster, user can select a specific image to bring up reviews that have matching words with the image's caption