Abstract In traditional machine teaching, a teacher needs to teach a concept to a learner by means of a finite set of examples, the witness set. But concepts can have many equivalent representations. This redundancy strongly affects the search space, to the extent that teacher and learner may not be able to easily determine the equivalence class of each representation. In this common situation, instead of teaching concepts, we explore the idea of teaching representations. We work with several teaching schemas that exploit representation and witness size (Eager, Greedy and Optimal) and analyze the gains in teaching effectiveness, both theoretically, and also experimentally for languages where redundancy can vary (DNF expressions and Turing-complete P3 programs). Our theoretical and experimental results indicate that there are various types of redundancy, related, e.g,. to the spread of the redundant representations, handled better by the new Greedy schema introduced here than by the Eager schema. For P3 programs witness sets found by Greedy are usually smaller than the programs they identify, corroborating previous results that conveying information efficiently is a leitmotif of machine teaching.

Large language models have become multimodal, and many of them are said to integrate their modalities using common representations. If this were true, a drawing of a car as an image, for instance, should map to a similar area in the latent space as a textual description of the strokes that form the drawing. To explore this in a black-box access regime to these models, we propose the use of machine teaching, a theory that studies the minimal set of examples a teacher needs to choose so that the learner captures the concept. In this paper, we evaluate the complexity of teaching vision-language models a subset of objects in the Quick, Draw! dataset using two presentations: raw images as bitmaps and trace coordinates in TikZ format. The results indicate that image-based representations generally require fewer segments and achieve higher accuracy than coordinate-based representations. But, surprisingly, the teaching size usually ranks concepts similarly across both modalities, even when controlling for (a human proxy of) concept priors, suggesting that the simplicity of concepts may be an inherent property that transcends modality representations.

Large Language Models (LLMs) have become the state-of-the-art technology in a variety of language understanding tasks. Accordingly, many commercial organizations have been increasingly trying to integrate LLMs in multiple areas of their production and analytics. A typical scenario is the need for answering questions over a domain-specific, private collection of documents, such that the answer is supported by evidence clearly referenced from those documents. The Retrieval-Augmented Generation (RAG) framework has been recently used by many applications for this kind of scenarios, as it intuitively bridges dedicated data collections and state-of-the-art generative models. Yet, LLMs are known to present data contamination, a phenomenon in which their performance on evaluation data relevant to a task is influenced by said data being already incorporated to the LLM during training phase. In this paper, we assess the performance of LLMs within the domain of Equinor, the largest energy company in Norway. Specifically, we address question answering with a RAG-based approach over a novel data collection not available for well-established LLMs during training, in order to study the effect of data contamination for this task. Beyond shedding light on LLM performance for a highly-demanded, realistic industrial scenario, we also analyze its potential impact for an ensemble of personas in Equinor with particular information needs and contexts.

Abstract Web search is an experience that naturally lends itself to recommendations, including query suggestions and related entities. In this article, we propose to recommend specific tasks to users, based on their search queries, such as planning a holiday trip or organizing a party. Specifically, we introduce the problem of query-based task recommendation and develop methods that combine well-established term-based ranking techniques with continuous semantic representations, including sentence representations from several transformer-based models. Using a purpose-built test collection, we find that our method is able to significantly outperform a strong text-based baseline. Further, we extend our approach to using a set of queries that all share the same underlying task, referred to as search mission, as input. The study is rounded off with a detailed feature and query analysis.

We address the problem of constructing a knowledge base of entity-oriented search intents. Search intents are defined on the level of entity types, each comprising of a high-level intent category (property, website, service, or other), along with a cluster of query terms used to express that intent. These machine-readable statements can be leveraged in various applications, e.g., for generating entity cards or query recommendations. By structuring service-oriented search intents, we take one step towards making entities actionable. The main contribution of this paper is a pipeline of components we develop to construct a knowledge base of entity intents. We evaluate performance both component-wise and end-to-end, and demonstrate that our approach is able to generate high-quality data.

Web search is a human experience of a very rapid and impactful evolution. It has become a key technology on which people rely daily for getting information about almost everything. This evolution of the search experience has also shaped the expectations of people about it. The user sees the search engine as a wise interpreter capable of understanding the intent and meaning behind a search query, realizing her current context, and responding to it directly and appropriately[1]. Search by meaning, or semantic search, rather than just literal matches, became possible by a large portion of IR research devoted to study semantically more meaningful representations of the information need expressed by the user query. Major commercial search engines have indeed responded to user expectations, capitalizing on query semantics, or query understanding. They introduced features which not only provide information directly but also engage the user to stay interacting in the search engine result page (SERP). Direct displays (weather, flight offers, exchange rates, etc.), rich vertical content (images, videos, news, etc.), and knowledge panels, are examples of this recent evolution trend into answer engines [10].

We address the problem of generating query suggestions to support users in completing their underlying tasks (which motivated them to search in the first place). Given an initial query, these query suggestions should provide a coverage of possible subtasks the user might be looking for. We propose a probabilistic modeling framework that obtains keyphrases from multiple sources and generates query suggestions from these keyphrases. Using the test suites of the TREC Tasks track, we evaluate and analyze each component of our model.

Identifying the target types of entity-bearing queries can help improve retrieval performance as well as the overall search experience. In this work, we address the problem of automatically detecting the target types of a query with respect to a type taxonomy. We propose a supervised learning approach with a rich variety of features. Using a purpose-built test collection, we show that our approach outperforms existing methods by a remarkable margin.

We introduce Nordlys, a toolkit for entity-oriented and semantic search. It provides functionality for entity cataloging, entity retrieval, entity linking, and target type identification. Nordlys may be used as a Python library or as a RESTful API, and also comes with a web-based user interface. The toolkit is open source and is available at http://nordlys.cc.