Lexical Alignment is a phenomenon often found in human–human conversations, where the interlocutors converge during a conversation to use the same terms and phrases for the same underlying concepts. Alignment (linguistic) is a mechanism used by humans for better communication between interlocutors at various levels of linguistic knowledge and features, and one of them is lexical. The existing literature suggests that alignment has a significant role in communication between humans, and is also beneficial in human–agent communication. Various methods have been proposed in the past to measure lexical alignment in human–human conversations, and also to implement them in conversational agents. In this research, we carry out an analysis of the existing methods to measure lexical alignment and also dissect methods to implement it in a conversational agent for personalizing human–agent interactions. We propose a new set of criteria that such methods should meet and discuss the possible improvements that can be made to existing methods.

An essential characteristic for unbiased and diligent information-seeking that can enable informed opinion formation and decision-making is intellectual humility (IH), the awareness of the limitations of one’s knowledge and opinions. While researchers have recognized the potential to boost IH in individuals, the effect of such interventions on their search behavior, along with the broader significance of IH in the context of web search on debated topics remains unexplored. In this paper, we present the results of a preregistered user study (N = 299) that we conducted to (1) test the effect of three interventions that boost self-reported IH on opinionated individuals’ search behavior and (2) explore the role of IH in the search process of opinionated individuals more broadly. IH-boosting interventions did not affect search behavior; we attribute this to the high familiarity of the search environment, prompting searchers to default to their usual search behavior. Still, explorations of the role of IH in the search process indicate that IH and IH-related search intentions should be considered as relevant factors in the pursuit of supporting unbiased and diligent search on debated topics. Based on our exploratory findings, we argue that future research should investigate interventions that are more directly integrated into the search process, as well as such that combine boosting IH with encouraging searchers to approach the search task in an IH-driven way and promoting transparency for appropriate reliance on the search system and ranking.

When people use web search engines to find information on debated topics, the search results they encounter can influence opinion formation and practical decision-making with potentially far-reaching consequences for the individual and society. However, current web search engines lack support for information-seeking strategies that enable responsible opinion formation, e.g., by mitigating confirmation bias and motivating engagement with diverse viewpoints. We conducted two preregistered user studies to test the benefits and risks of an intervention aimed at confirmation bias mitigation. In the first study, we tested the effect of warning labels, warning of the risk of confirmation bias, combined with obfuscations, hiding selected search results per default. We observed that obfuscations with warning labels effectively reduce engagement with search results. These initial findings did not allow conclusions about the extent to which the reduced engagement was caused by the warning label (reflective nudging element) versus the obfuscation (automatic nudging element). If obfuscation was the primary cause, this would raise concerns about harming user autonomy. We thus conducted a follow-up study to test the effect of warning labels and obfuscations separately. According to our findings, obfuscations run the risk of manipulating behavior instead of guiding it, while warning labels without obfuscations (purely reflective) do not exhaust processing capacities but encourage users to actively choose to decrease engagement with attitude-confirming search results. Therefore, given the risks and unclear benefits of obfuscations and potentially other automatic nudging elements to guide engagement with information, we call for prioritizing interventions that aim to enhance human cognitive skills and agency instead.

Explainable Artificial Intelligence (XAI) focuses on research and technology that can explain an AI system’s functioning and its underlying methods, and also on making these explanations better through personalization. Our research study investigates a natural language personalization method called lexical alignment in understanding an explanation provided by a conversational agent. The study setup was online and navigated the participants through an interaction with a conversational agent. Participants faced either an agent designed to align its responses to those of the participants, a misaligned agent, or a control condition that did not involve any dialogue. The dialogue delivered an explanation based on a pre-defined set of causes and effects. The recall and understanding of the explanations was evaluated using a combination of Yes-No questions, a Cloze test (fill-in-the-blanks), and What-style questions. The analysis of the test scores revealed a significant advantage in information recall for those who interacted with an aligning agent against the participants who either interacted with a non-aligning agent or did not go through any dialogue. The Yes-No type questions that included probes on higher-order inferences (understanding) also reflected an advantage for the participants who had an aligned dialogue against both non-aligned and no dialogue conditions. The results overall suggest a positive effect of lexical alignment on understanding of explanations.

As interactive narratives, by definition, change according to the user’s choices (dynamic story), so do the dialogue utterances by the characters. Writing all possible utterances manually faces scaling problems. This motivates the use of natural language generation techniques. We present ExpReal, a surface realizer and templating language that allows authors of interactive narratives to write flexible and enriched templates while maintaining control over their use. Templates are automatically selected based on author-specified conditions relating to the world state (e.g. characters’ emotions) or the current task at hand. ExpReal has been developed to support at least three languages (English, French and Dutch).

An important objective in health-technology is the ability to gather information about people’s well-being. Structured interviews can be used to obtain this information, but are time-consuming and not scalable. Questionnaires provide an alternative way to extract such information, though typically lack depth. In this paper, we present our first prototype of the BLISS agent, an artificial intelligent agent which intends to automatically discover what makes people happy and healthy. The goal of Behaviour-based Language-Interactive Speaking Systems (BLISS) is to understand the motivations behind people’s happiness by conducting a personalized spoken dialogue based on a happiness model. We built our first prototype of the model to collect 55 spoken dialogues, in which the BLISS agent asked questions to users about their happiness and well-being. Apart from a description of the BLISS architecture, we also provide details about our dataset, which contains over 120 activities and 100 motivations and is made available for usage.

We describe a prototype of Churnalist, a headline generator for creating contextually-appropriate fictional headlines that can be used as flavor text in games. Churnalist creates new headlines by remixing existing headlines. It extracts seed words from free text input, searches for related words in a dataset of word embeddings and uses these words in the new headlines. The system requires no linguistic expertise or hand-coded language models from the user.

The design of Tangible User Interfaces (TUIs) has often involved children. Nowadays, the term co-creation is gaining momentum and extending the already established role of children as co-designers. In this sense children become creators of new personalized experiences not only during the design process but also when using the final interactive products. Such dynamic opens new possibilities reshaping the role and the development of tangibles by providing users "building blocks" for creative experimentations. This workshop aims at bringing together researchers and practitioners from relevant disciplines and expertise to reflect on the co-creation approach, and on the possibilities that it opens for the development of innovative tools that place the users in the center of the creative process. Contributions to this workshop may address diverse aspects related to co-creation from different perspectives. The ultimate aim of the workshop is to discuss the research challenges and the pedagogical issues that need to be addressed when designing TUIs for co-creation.

We propose using real-world datasets to generate textual game assets for serious games. As an example, we used a dataset of P2000 crisis event messages to generate descriptive texts that can be transformed into new game assets by game writers, thereby reducing the writing effort required during the development phase of an adaptive serious game. In this paper we describe this first attempt and we discuss the challenges and possibilities of using open data for textual asset generation.

This paper presents a new version of a football reports generation system called PASS. The original version generatedDutch text and relied on a limited handcrafted knowledge base.We describe how, in a short amount of time, we extended PASS to produce English texts, exploiting machine translation and Wikidata as a large-scale source of multilingual knowledge.

This paper presents SimpleNLG-NL, an adaptation of the SimpleNLG surface realisation engine for the Dutch language.It describes a novel method for determining and testing the grammatical constructions to be implemented, using target sentences sampled from a treebank.

We present anew dialogue engine called Flipper 2.0 (Flipper) which aims to help developers of embodied conversational agents (ECAs) to quickly and flexibly create dialogues. Flipper provides a technically stable and robust dialogue management system to integrate with other components of ECAs such as behaviour realisers. We compare Flipper with state-of-the-art dialogue design systems. We describe the details of our dialogue engine, how it handles dialogue management and how it supports the authoring of dialogues. We demonstrate the use of the dialogue engine with examples of design patterns and discuss practical applications. Finally we give recommendations on the cases in which it is beneficial to use Flipper.

Remote mixed reality (MR) collaboration systems allow for multimodal, real-time support from remote experts. We present our open toolkit that provides a flexible end-to-end solution for building such systems using off-the-shelf hardware. From related work, three core design aspects have been identified: 1) the independence of the viewpoint that the visitor (the remote expert) can take in relation to position and viewpoint of the visitee, 2) the immersiveness of the presentation technology for visitor and visitee, and 3) the extent to which the visitor's body is represented in the visitee's environment. This paper describes the implementation of our system, which includes these aspects. In a study aimed at validating whether we implemented these core aspects to good effect, conducted with a collaborative puzzle application built with our toolkit, we examine how variations of these aspects contribute to usability, performance and social presence related metrics.

In our ongoing research, we argue that storytelling activities can be used as creative tasks to stimulate creativity in children, one of the so-called 21st century skills. In this paper, we lay the foundations for our project on digitally supported storytelling, by gathering the viewpoints on storytelling and creativity expressed by local primary school teachers and reviewing methods reported in the literature. We report the main characteristics and features to consider in creative storytelling activities, and present our current work on a distributed storytelling interface for supporting children's creative skills development that meets the practical requirements found in the literature and expressed by the teachers.

Touch displays moving autonomously, i.e., being self-actuated, are starting to be part of system prototypes reported in the literature. In the future, user interactions with such prototypes may take place while the display is moving. However, since current prototypes do not yet require this type of interaction, there is a lack of empirical evidence reporting issues related to touch input in such conditions. This leads us to propose two basic questions: Can we request users to deliver touch gestures during the actuation? Which aspects should we take into account when having touch input in these moving devices? In order to start to answer these questions, we report in this paper a first study to get an insight into how people perform and feel it when they have to carry out touch input on a self-actuated tablet. The preliminary results show that the self-actuated tablet does not necessarily need to be still during touch interaction, and that single-touch gestures based on drag or tap are preferable over others. Furthermore, the results also revealed an issue with tap gestures because of the movement of the tablet.

We present a demonstration of the ARIA framework, a modular approach for rapid development of virtual humans for information retrieval that have linguistic, emotional, and social skills and a strong personality. We demonstrate the framework's capabilities in a scenario where `Alice in Wonderland', a popular English literature book, is embodied by a virtual human representing Alice. The user can engage in an information exchange dialogue, where Alice acts as the expert on the book, and the user as an interested novice. Besides speech recognition, sophisticated audio-visual behaviour analysis is used to inform the core agent dialogue module about the user's state and intentions, so that it can go beyond simple chat-bot dialogue. The behaviour generation module features a unique new capability of being able to deal gracefully with interruptions of the agent.

For many applications measuring the similarity between documents is essential. However, little is known about how users perceive similarity between documents. This paper presents the first large-scale empirical study that investigates perception of narrative similarity using crowdsourcing. As a dataset we use a large collection of Dutch folk narratives. We study the perception of narrative similarity by both experts and non-experts by analyzing their similarity ratings and motivations for these ratings. While experts focus mostly on the plot, characters and themes of narratives, non-experts also pay attention to dimensions such as genre and style. Our results show that a more nuanced view is needed of narrative similarity than captured by story types, a concept used by scholars to group similar folk narratives. We also evaluate to what extent unsupervised and supervised models correspond with how humans perceive narrative similarity.

As in-vehicle information systems are increasingly able to obtain and deliver information, driver distraction becomes a larger concern. In this paper we propose that informative interruption cues (IIC) can be an effective means to support drivers' attention management. As a first step, we investigated the design and presentation modality of IIC that conveyed not only the arrival but also the priority level of a message. Both sound and vibration cues were created for four different priority levels and tested in 5 task conditions that simulated possible perceptional and cognitive load in real driving situations. Results showed that the cues were quickly learned, reliably detected, and quickly and accurately identified. Vibration was found to be a promising alternative for sound to deliver IIC, as vibration cues were identified more accurately and interfered less with driving. Sound cues also had advantages in terms of shorter response time and more (reported) physical comfort.

Many researchers argue, in assessing the benefits of Advanced Driver Assistance Systems (ADAS) it has to be taken into account that any gains in terms of security may be again reduced by the fact they affect the drivers' behavior. In this paper, we present results of a driving simulation study in which we compare driving performance as well as driver stress in three conditions: 1) when no assistance is available (NOASSIST) ; 2) when the system is working as it should (WORK); 3) when the system suddenly fails (FAIL). Results show that the driving performance is severely affected by system failures. The drivers' ability to effectively react to suddenly appearing obstacles FAIL is significantly lower than in NOASSIST. At the same time, the stress level is significantly higher in FAIL compared to WORK.

Local danger warning is an important function of Advanced Driver Assistance Systems (ADAS) to improve the safety of driving. The user interface (the warning presentation) is particularly crucial to a successful danger avoidance. We present a user study investigating various warning presentations using a scenario of emergent road obstacles. Two presentation factors were selected: modality and level of assistance. The modality factor had 4 variants: speech warning, visual and speech warning, visual warning with blinking cue, and visual warning with sound cue. The level of assistance varied between with or without action suggestions (AS). In accordance with the ISO usability model, a total of 6 measurements were derived to assess the effectiveness and efficiency of the warnings and the drivers' satisfaction. Results indicate that the combination of speech and visual modality leads to the best performance as well as the highest satisfaction. In contrast, purely auditory and purely visual modalities were both insufficient for presenting high-priority warnings. AS generally improved the usability of the warnings especially when they were accompanied by supporting information so that drivers could validate the suggestions.

In this study, we argue that modality planning in multimodal presentation systems needs to consider the modality characteristics at not only the presentational level but also the cognitive level, especially in a situation where the information load is high and the user task is time-critical. As a first step towards automatic cognitive-aware modality planning, we integrated the effect of different modalities on cognitive load and performance, using a high-load information presentation scenario. Mainly based on modality-related psychology theories, we selected five modality conditions (text, image, text+image, text+speech, and text+sound) and made hypotheses about their effects on cognitive load. Modality effects were evaluated by two cognitive load measurements and two performance measurements. Results confirmed most of the predicted modality effects, and showed that these effects become significant when the information load and the task demand are high. The findings of this study suggest that it is highly necessary to encode modality-related principles of human cognition into the modality planning procedure for systems that support high-load human-computer interaction.

We present the Virtual Guide, an embodied conversational agent that can give directions in a 3D virtual environment. We discuss how dialogue management, language generation and the generation of appropriate gestures are carried out in our system. Index Terms: dialogue management, language generation, embodied conversational agents

Work on expressive speech synthesis has long focused on the expression of basic emotions. In recent years, however, interest in other expressive styles has been increasing. The research presented in this paper aims at the generation of a storytelling speaking style, which is suitable for storytelling applications and more in general, for applications aimed at children. Based on an analysis of human storytellers' speech, we designed and implemented a set of prosodic rules for converting "neutral" speech, as produced by a text-to-speech system, into storytelling speech. An evaluation of our storytelling speech generation system showed encouraging results.

This article challenges the received wisdom that template-based approaches to the generation of language are necessarily inferior to other approaches as regards their maintainability, linguistic well-foundedness, and quality of output. Some recent NLG systems that call themselves “template-based” will illustrate our claims.

In human-human communication, dialogue participants are con-tinuously sending and receiving signals on the status of the inform-ation being exchanged. We claim that if spoken dialogue systems were able to detect such cues and change their strategy accordingly, the interaction between user and systemwould improve. Therefore, the goals of the present study are as follows: (i) to find out which positive and negative cues people actually use in human-machine interaction in response to explicit and implicit verification questions and (ii) to see which (combinations of) cues have the best predictive potential for spotting the presence or absence of problems. It was found that subjects systematically use negative/marked cues (more words, marked word order, more repetitions and corrections, less new information etc.) when there are communication problems. Using precision and recall matrices it was found that various combinations of cues are accurate problem spotters. This kind of information may turn out to be highly relevant for spoken dia-logue systems, e.g., by providing quantitative criteria for changing the dialogue strategy or speech recognition engine.

This paper discusses an experiment concerning the assignment of contrastive accents, i.e., accents used to indicate the presence of contrastive information. The experiment tested which of two existing approaches to the determination of contrastive information gives the most natural results. According to one approach the presence of `alternative items' is the only condition for the assignment of contrastive accent; according to the other the presence of parallelism between sentences also is a condition. The experimental results indicate that the presence of alternative items combined with parallelism always triggers a preference for contrastive accent, whereas in the absence of parallelism, accent assignment seems to depend on the degree of coherence of the utterance.

The defining property of a Concept-to-Speech system is that it combines language and speech generation. Language generation converts the input concepts into natural language, which speech generation subsequently transforms into speech. Potentially, this leads to a more `natural sounding' output than can be achieved in a plain Text-to-Speech system, since the correct placement of pitch accents and intonational boundaries ---an important factor contributing to the `naturalness' of the generated speech--- is co-determined by syntactic and discourse information, which is typically available in the language generation module. In this paper, a generic algorithm for the generation of coherent spoken monologues is discussed, called D2S. Language generation is done by a module called LGM which is based on TAG-like syntactic structures with open slots, combined with conditions which determine when the syntactic structure can be used properly. A speech generation module (SGM) converts the output of the LGM into speech using either phrase-concatenation or diphone-synthesis.

Probably the best current algorithm for generating definite descriptions is the Incremental Algorithm due to Dale and Reiter. If we want to use this algorithm in a Concept-to-Speech system, however, we encounter two limitations: (i) the algorithm is insensitive to the linguistic context and thus always produces the same description for an object, (ii) the output is a list of properties which uniquely determine one object from a set of objects: how this list is to be expressed in spoken natural language is not addressed. We propose a modification of the Incremental Algorithm based on the idea that a definite description refers to the most salient element in the current context satisfying the descriptive content. We show that the modified algorithm allows for the context-sensitive generation of both distinguishing and anaphoric descriptions, while retaining the attractive properties of Dale and Reiter's original algorithm. 1. INTRODUCTION In their interesting 1995 paper, Dale & Reiter ...

Being able to predict the placement of contrastive accent is essential for the assignment of correct accentuation patterns in spoken language generation. I discuss two approaches to the generation of contrastive accent and propose and alternative method that is feasible and computationally attractive in data-to-speech systems.