Products on e-commerce websites are usually organized based on seller-provided product attributes. Customers looking for a product typically have certain needs or product use-cases in mind, for e.g., a headphone for gym classes, or a printer for a small business. However, they often struggle to map these use-cases to product attributes and subsequently fail to find the product they need. In this talk, we present a use-case based shopping (UBS) ML system that facilitates use-case based customer experiences (CXs). The UBS system recommends dominant product use-cases to customers along with most relevant products for those use-cases. Use-cases and their definitions vary across product categories and marketplaces (MPs). This makes training supervised models for thousands of e-commerce categories and multiple MPs infeasible by collecting large amount training data needed to train these models. In this talk, we present our work on scaling the UBS model by instruction tuning an LLM for our task.

Navigating vast e-commerce websites with extensive product cata- logs can be a daunting challenge for shoppers. To assist customers in finding the products they desire, e-commerce platforms provide product attribute filters, commonly referred to as "refinements." These refinements serve as a vital navigational aid, enabling cus- tomers to refine their search results based on specific product at- tributes such as material, color, size, brand, etc. However, on mobile devices refinements are not easily discoverable due to lack of screen space. To improve discoverability, contextually relevant refinements are suggested in-line on search page by refinement recommenda- tion systems. In the work, we discuss the evolution of refinement recommendations strategies i.e, a) search query-based classifica- tion approach, for a given search query we train a classification model, with the refinements as labels b) session-based classification approach, for the given sequence of session interactions we train a sequence classification model, with the refinements as labels and c) session-based generation approach, with the sequence of session interactions as input and output as the refinement name.

Finding the right product on e-commerce websites with millions of products is a daunting task for a large set of customers. On the search page, product attribute filters a.k.a. "refinements" emerge as a convenient navigational option for customers to narrow down the search results along product attributes of their choice (e.g., Material:Cotton, Color:Black for 'shirt'). However, on mobile devices, refinements are not easily discoverable due to lack of screen space. To improve discoverability, contextually relevant refinements are suggested in-line on search page by refinement recommendation systems. Existing works on refinement recommendations primarily rely on the search context as input, and are trained using aggregated refinement preferences 'explicitly' expressed by customers. These solutions fail to capture 'implicit' preferences expressed during the customer shopping mission through in-session browsing activity. In this paper, we propose a session-based recommendation system (SBRS) which recommends refinements by inferring product attribute preferences of customers based on the sequence of products viewed earlier in the session. For the task of refinement recommendation, we propose a) AttriBERT, a model which extends BERT architecture to learn from the attribute values of products and b) a novel product representation strategy, which represents each product as a dictionary of attribute:value pairs (e.g., RAM Size:64GB). We evaluate our approach on RecSys 2022 Challenge and Amazon e-commerce datasets. Our approach consistently outperforms various state-of-the-art sequence models on the task of session-based refinement recommendation.

The boom of mobile devices and cloud services has led to an explosion of personal photo and video data. However, due to the missing user-generated metadata such as titles or descriptions, it usually takes a user a lot of swipes to find some video on the cell phone. To solve the problem, we present an innovative idea called Visual Memory QA which allow a user not only to search but also to ask questions about her daily life captured in the personal videos. The proposed system automatically analyzes the content of personal videos without user-generated metadata, and offers a conversational interface to accept and answer questions. To the best of our knowledge, it is the first to answer personal questions discovered in personal photos or videos. The example questions are "what was the lat time we went hiking in the forest near San Francisco?"; "did we have pizza last week?"; "with whom did I have dinner in AAAI 2015?".

The ubiquity of mobile devices and cloud services has led to an unprecedented growth of online personal photo and video collections. Due to the scarcity of personal media search log data, research to date has mainly focused on searching images and videos on the web. However, in order to manage the exploding amount of personal photos and videos, we raise a fundamental question: what are the differences and similarities when users search their own photos versus the photos on the web? To the best of our knowledge, this paper is the first to study personal media search using large-scale real-world search logs. We analyze different types of search sessions mined from Flickr search logs and discover a number of interesting characteristics of personal media search in terms of information needs and click behaviors. The insightful observations will not only be instrumental in guiding future personal media search methods, but also benefit related tasks such as personal photo browsing and recommendation. Our findings suggest there is a significant gap between personal queries and automatically detected concepts, which is responsible for the low accuracy of many personal media search queries. To bridge the gap, we propose the deep query understanding model to learn a mapping from the personal queries to the concepts in the clicked photos. Experimental results verify the efficacy of the proposed method in improving personal media search, where the proposed method consistently outperforms baseline methods.

As mobile displays proliferate across the world, and fixed video displays expand in size, consumers demand enhanced visual experiences while watching such displays. Competitive low-end to high-end markets are migrating to high-definition (HD) output displays driven by consumer demand. In order to generate high quality visual experiences on HD displays, video processing algorithms are becoming increasingly complex. Such complex HD algorithms require up to tera-operations per second to generate acceptable HD video output. Silicon Hive's HiveFlex Video VSP1 processor services the needs all low to high end video displays by using a unique scaleable tiled architecture. The architecture of each tile is flexible, low-cost and low-power, resulting in an attractive IP solution targeting consumer video signal processing (VSP) systems on chips (SoCs). The Video VSPI tile presented here performs a variety of video coding algorithms commonly used in post processing of HDTV signals such as H.264 decoding, de-interlacing, picture-rate up-conversion and others. H.264 or MPEG4-AVC is an advanced video coding standard targeting multiple markets such as broadcast and hand-held entertainment. In partnership with Silicon Hive, AllGo Embedded Systems has developed an H.264 decoder that is optimized for the HiveFlex VSP architecture. Multiple VSP tiles with configurable number of issue slots and SIMD architecture are used efficiently to implement H.264 video decoding at HD resolution