Due to the emergence of several nutrition-related mobile applications and websites in recent years, as well as the massive amount of crowd-sourced nutrition data, searching and finding relevant results has become increasingly difficult for users. This problem becomes even more challenging when dealing with crowd-sourced food names that are noisy and not well-structured. Because food names are short in length, it is difficult to incorporate existing methods to achieve an optimal matching quality. Despite several recent studies on nutrition data, these challenges remain. In this paper, we propose a novel learning-to-rank framework for crowd-sourced food names that has significant real-world applications, including food search and food recommendations. In particular, we propose a deep learning based, multi-modal learning-to-rank model that leverages the text describing a food name and the numerical values that represent its nutritional information. To this end, we also introduce a novel type of loss-function, which extends standard triplets hinge loss function into a multi-modal scenario. The proposed model is flexible and supports various data types as well as an arbitrary number of modalities. The effectiveness of our proposed model is demonstrated through several experiments on real-data, consisting of more than six million instances.

We study the problem of how to match a formally structured restaurant menu item to a large database of less structured food items that has been collected via crowd-sourcing. At first glance, this problem scenario looks like a typical text matching problem that might possibly be solved with existing text similarity learning approaches. However, due to the unique nature of our scenario and the need for scalability, our problem imposes certain restrictions on possible machine learning approaches that we can employ. We propose a novel, practical, and scalable machine learning solution architecture, consisting of two major steps. First we use a query generation approach, based on a Markov Decision Process algorithm, to reduce the time complexity of searching for matching candidates. That is then followed by a re-ranking step, using deep learning techniques, to meet our required matching quality goals. It is important to note that our proposed solution architecture has already been deployed in a real application system serving tens of millions of users, and shows great potential for practical cases of user-entered text to structured text matching, especially when scalability is crucial.