Charles Explorer logo
🇬🇧

Interactive Product Search Based on Global and Local Visual-Semantic Features

Publication at Faculty of Mathematics and Physics, Faculty of Arts |
2018

Abstract

In this paper, we present a prototype web application of a product search engine of a fashion e-shop. Today, e-shop product metadata consist of text description, simple attributes (price, size, color, fabric, etc.) and visual information (product photo).

Search engines used in e-shops mostly provide text and attribute/category interface for product filtering. In our model, we focus on the visual information applied in an interactive query-by-example scenario.

The global visual descriptors may be often ambiguous and may not correspond well with the intended mental query of the user. Therefore, we proposed and evaluated model and GUI allowing user to guide the query process by selecting image regions (patches) of interest within the query.

In the demo evaluation, we show that allowing user to specify relevant image patches led to a significant improvement of the results' relevance in the vast majority of tested queries.