Figure 1: Unity Environment: The game view showing the four virtual managers (A); The Wizard of Oz interface (B); The game view showing the virtual environment (C)Abstract "Double Jeopardy" describes the phenomenon where individuals belonging to multiple marginalised groups experience greater discrimination than those not affected by intersectional biases.This topic has been studied within sociology and economics, where double jeopardy tendencies have predominantly been observed in relation to income disparities.However, to our knowledge, no study has explicitly examined whether a double jeopardy effect can be measured in interpersonal interactions, specifically, whether individuals with intersecting marginalisations such as gender and race are perceived significantly more negatively.Intelligent Virtual Agents (IVAs) offer an ideal means of investigating such phenomena due to their resource-efficient, standardisable adaptability and consistent interactivity.We implemented a demonstrator to examine whether a cumulative bias effect of racial and gender bias could be detected in interactions with IVAs and to what extent this effect is influenced by participants' prior implicit biases.However, our analyses did not provide evidence of a double jeopardy effect when interacting with IVAs.Although the findings did not align with our hypotheses, this study offers an initial starting point for future IVA-based investigations into the double jeopardy phenomenon and provides methodological foundations for further research.

Storytelling has accompanied humans since the beginning of mankind, and the reception of stories has become one of the most popular leisure activities. With their multimodal abilities, social robots bear great potential as storytellers. In this contribution, we present the Automatic Multimodal Robotic Storyteller – a system implemented for the Pepper robot which allows to automatically annotate a given story with emotions and perform it by the robot using emotional body language, emotion-inducing music, and colored lights. The system is empirically based on the results of several studies that are briefly summarized in this work. In an encompassing study, we compared the final Automatic Multimodal Robotic Storyteller to traditional storytelling media formats, namely text and audio book. While reading a story is preferred for the perceived control it offers, robotic storytellers are as well received as today’s traditional storytelling media in most aspects of storytelling experience, and even surpass the audio book in terms of emotion induction. Given this potential, our Automatic Multimodal Robotic Storyteller system could help robotic storytellers become a common storytelling medium in the future by making them accessible to a wide range of stories, by simply applying our pipeline to any given story in text format. To do so, the system will be made publicly available upon acceptance of this manuscript. The approach can also serve as guidance for other robotic storytellers.

The phenomena of in-group favoritism and out-group discrimination arise as early as preschool age. Living in increasingly diverse societies, it is therefore important to research these behaviors with children of that age. However, there exists no widely applicable tool that allows to measure in-group out-group bias in young children.

Storytelling is a long-established human tradition for entertainment and knowledge transfer. Social robots are emerging as a new storytelling medium, being able to imitate human storytelling using gestures but also extend it by adding, e.g., sound effects to the experience. Due to COVID-19 restrictions, we conducted an online video-based study to investigate the effects of congruent respectively incongruent or no gesture usage in combination with additional non-speech sounds, i.e. sound effects and background music, on recipients' transportation into the story told, emotion induction, and perception of the robot. Results indicate no effect of additional non-speech sound integration on the variables listed above. Contradicting with related findings from in-person studies, we found a no significant differences between congruent, incongruent and no gesture usage. Last, no interplay of additional sounds and gesture congruence was identified. Future studies should provide deeper insights into the importance of multimodal congruence in video-taped robots and the possible advantages of adding non-speech sounds to online but also in-person robotic storytelling as well as their interplay in in-person HRI.

Enabling artificial agents to be truly socially interactive and unobtrusive requires them to understand the social context of a situation. As a variety of different social context cues exist, both user studies and methods for context classification systems of agents are required to reliably categorize and recognise social contexts and respond appropriately and autonomously towards users. Compiling datasets from the real world can often be expensive, time-consuming and lack scalability. Thus, virtually simulated social contexts can be used to generate training data for context detection systems, but also stimuli material for user studies in a fast and cost-effective way. As a proof of concept, we introduce virtual social contexts and test their applicability for social context research with objective and subjective measurements. Therefore, we first implement virtual social contexts to compile our own dataset1. With subjective participant ratings, we evaluate the applicability of virtual social contexts as stimuli material for user studies. For this, we replicate a previous study with recordings of real social contexts using recordings of virtual social contexts instead. For our objective measurements, we use the computer vision models of YOLOv8 on object detection and pose estimation to train a classification model on our virtual image dataset and evaluate it on an image dataset of real social contexts. The results of the subjective and objective analyses indicate, that virtually generated social contexts can be used both for training classification systems and as stimuli material in user studies.

Positive contact with authorities can be an important pillar for our society, as it fosters trust and confidence in the legal and governmental systems, helping maintain social order and cooperation. Hence, for instance, the police engages in public relations efforts. However, due to staff shortages, this is only possible to a limited extent. Additionally, the representation of individuals with immigrant backgrounds within the police force is low. Nevertheless, for individuals with immigrant backgrounds, the ingroup effect can be particularly beneficial in facilitating pleasant personal interactions. To investigate the potential positive effect of a pleasant interaction with the police, and the impact of cultural similarity, on the perceived trust in the police, we implemented an interactive scenario containing either a prototypical German police officer, or one with a mixed-cultural background. In a user study with individuals of the same mixed-cultural background, we measured the trust in the police before and after the interaction. In addition, we assessed the perception of the agent in terms of similarity, warmth, competence, empathy, trust towards the agent, as well as the interaction with the agent in general. Our results reveal, that through interaction with a virtual police officer, indeed the trust in the police could be increased. However, we did not find the intended effect of ingroup similarity, showing no increased trust in the mixed-cultural condition compared to the German condition. We consider these results promising in regards to effectively being able to use virtual agents as a tool to provide positive personal interactions with authorities.

Technology-supported learning is an integral part of everyday learning. Adaptive tutoring systems are already widespread, but they often lack the social component of learning. Social robots can provide personalized tutoring as well as social interaction and can thus address this challenge. In this contribution, we present a model to personalize the behavior of a social robot tutor for higher education which has both a theoretical as well as an empirical foundation: Theoretically, it is based on the well-known Self-Determination Theory (SDT) that assumes that intrinsic motivation is strongly related to the fulfillment of three basic human needs, namely the need for autonomy, competence, and relatedness. Empirically it is based on the results of four studies with university students that demonstrate that particularly adaptation on a social level is beneficial. The final model considers the personalization of the robotic tutor’s behavior based on the current learning content and its social behavior and also considers a long-term perspective for multi-session learning. Thereby each of the three layers of SDT is addressed in multiple ways. This is one of the first models that contain a long-term perspective and is particularly designed for higher education.

Social robots are finding their way into our society and must adapt to the social norms of everyday life. It is often unavoidable for robotic assistants to interrupt humans in their activities. These interruptions must therefore be designed in such a way that they have as little negative impact as possible on the ongoing activity and the perception of the robot. In this study, we investigate whether interruptions regarding health-promoting behavior delivered by a social robot in an office situation should be phrased politely rather than directly. A system was implemented to trigger either politely or directly phrased interruptions in a laboratory user study with participants executing an office task. We examined the perception of the robot, the task, and the interruption, as well as whether the health-promoting behavior was performed by the users. The results show that there were no significant differences in the perception of the robot and the interruptions, while the temporal demand of the task was perceived to be higher in the polite condition. In addition, the intention to follow the polite prompts was significantly higher, but there was no difference in the actual compliance with the prompts in the study itself. Index Terms—social robot, politeness, interruptions

Virtual environments (VEs) can be designed to evoke specific emotions for example by using colored light, not only applicable for games but also for virtual storytelling with a single storyteller. Social robots are perfectly suited as storytellers due to their multimodality. However, there is no research yet on the transferability of robotic storytelling to virtual reality (VR). In addition, the transfer of concepts from VE design such as adaptive room illumination to robotic storytelling has yet not been tested. Thus, we conducted a study comparing the same robotic storytelling with a physically embodied robotic storyteller and in VR to investigate the transferability of robotic storytelling to VR. As a second factor, we manipulated the room light following design guidelines for VEs or kept it constant. Results show that a virtual robotic storyteller is not perceived worse than a physically embodied storyteller, suggesting the applicability of virtual static robotic storytellers. Regarding emotion-driven lighting, no significant effect of colored lights on self-reported emotions was found, but adding colored light increased the social presence of the robot and its’ perceived competence in both VR and reality. As our study was limited by a static robotic storyteller not using bodily expressiveness future work is needed to investigate the interaction between well-researched robot modalities and the rather new modality of colored light based on our results.

Storytelling is an important method not only in entertainment but also for education or therapy. Humor can improve storytelling due to its social functions. We investigate whether the humorous function of laughter can also be beneficial for robotic storytelling. We conducted a study focusing on the effects of voice type (human vs. synthetic) and laughter (presence vs. absence) on the robot’s perceived humorous appeal as well as recipients’ transportation, i.e. their absorption into the story, and emotions. No significant differences were found regarding the manipulations. The type of voice and use of laughter are therefore less relevant than assumed. However, it is shown that negative emotions are significantly reduced by the humorous story presented by the robot. Furthermore, the story was perceived as more funny and elicited increased transportation when being received by the robot compared to reading the story. In conclusion, this study indicates that robots are generally suitable for humorous storytelling and interesting results for future research on humorous robotic storytelling.

This paper presents an interaction study with a mixed-cultural Intelligent Virtual Agent (IVA) that investigates the impact of perceived ingroup similarity on the empathy of human interlocutors, and the perceived competence and warmth of the IVA. We herefore implemented an interactive scenario with either an IVA that should be perceived as an ingroup member or a neutral IVA. Our results show that the IVA in the ingroup condition triggered significantly more empathy and was rated significantly more competent than the IVA in the neutral condition. Regarding perceived warmth, both IVAs were rated rather high, revealing no significant differences. With these results we contribute to the goal of raising empathy towards IVAs with mixed-cultural backgrounds, to be potentially used in scenarios that aim at studying and reducing implicit racial bias.

The social context of a situation is crucial for people to decide how to behave and interact. For artificial agents to be truly socially interactive and unobtrusive it is equally important to understand the social context of a situation. Training these agents to reliably recognize social context is a difficult task, as big data sets of training data are required and a wide variety of different social context cues exist. In this contribution, we describe the creation of virtually simulated situations using virtual agents that demonstrate different social contexts. With this approach, we provide a fast and cost-effective way to create diverse and adaptable training data, by being able to generate video material of the virtual scenes. A suitable context detection system can then be used to support the classification of social context and decide upon, for example, whether a user is available for an interaction, or not. Further, these virtual social context situations can be also used for virtual reality studies to investigate how users react to interruptions in social contexts.

This paper presents the development and preliminary evaluation of a personality enhancer behaviour model for virtual audiences to increase their realism and individualism. We conducted a systematic literature review and identified sixteen posture parameters to modify seated animations dynamically, calling them personality enhancers. We grouped them into four main categories: body, gaze, face and gesture behaviour modifiers. We implemented a unique animation modifier system on top of a game engine to apply these personality enhancers on existing pre-recorded generic seated animations. The first results with sixty participants in an online video survey show that the model can successfully simulate individuals with low and high levels of extroversion as well as with low and high levels of emotional stability.

This paper presents an interaction study that investigates whether participants adapt their verbal and non-verbal behaviour towards an IVA that shows non-native speech patterns in a way that is congruent to observations previously made in human-human and human-agent interaction. In a between-subjects design, participants either interacted with an IVA speaking native-accented German, an IVA speaking German with a non-native accent, or an IVA speaking grammatically incorrect German with a non-native accent in VR. The results reveal that participants partly adapt their verbal and non-verbal behaviour when interacting with the IVAs with non-native speech patterns, which constitutes important implications for the cost-efficient design of enculturated IVAs.

This paper presents a system consisting of two tools to facilitate the cost-effective implementation of mixed-cultural intelligent virtual agents (IVAs) for both basic and applied research. The first tool automatically introduces grammatical mistakes and non-native accents into a given text to produce synthetic non-native speech. The second tool is a Wizard-of-Oz scenario builder, created with the game engine Unity, that can construct natural interaction scenarios with a dynamic number of (mixed-cultural) IVAs. It enables the control of the appearance and non-verbal behaviour of the IVAs as well as their verbal behaviour by integrating the first tool. Both tools can be either used separately or combined for a wide range of applications in both basic research, for example to investigate the perception of mixed-cultural cues in IVAs, and applied research, for example to create virtual interventions with IVAs.

Learning to read can be difficult, but is a crucial skill that is needed for the whole lifespan. Deficiencies in reading capabilities can often be traced back to the initial attempts to learn reading during primary school. While there exist a number of analogue and digital tools to support reading acquisition for primary school children, they often don't account for their heterogeneity. This paper presents work in progress on integrating an adaptive pedagogical agent in a mobile reading app designed for second graders with reading difficulties. We apply Reinforcement Learning to adapt the feedback behavior of the agent to each child's learning needs and elaborate on our methodological approach. With it, we contribute a novel concept for adapting a pedagogical agent's feedback behavior within the critical context of reading development.

Advancing digitization in working environments brings up the necessity of lifelong learning as well as technology-supported employee training. Research on social robots has already demonstrated their potential to support adults in their learning process. In this study, we focus on potential benefits of applying a social robot for employee training. We conducted a field study in cooperation with a company and set-up two conditions: a robot-supported learning environment and the onscreen learning environment the company usually uses for employee training. Our results show a positive perception of the robot and participants of the robot condition reported significantly more enjoyment while learning. Half of the participants are willing to use it again in the future: some prefer the robot-supported training over the onscreen training, while others were interested to use both options. However, the other half stick with onscreen learning in the future and there were no significant differences between motivation and learning success between the two groups.

This paper introduces the Pepper Python Unity Toolkit (PePUT), a toolkit for controlling and using the social robot Pepper via Unity and Python. As toolkit components, we present implementations for the speech- and tablet-control, as well as animation and navigation, which can be directly used within Unity. With it, we provide the opportunity of a virtual testbed for the social robot Pepper. In addition, we highlight potential use cases for the components, in particular in a smart environment, as well as the use of PePUT as a research tool. The toolkit with the presented components and the source files are publicly available as open-source project1under the MIT license via GitLab for usage, replication, and extension.

This paper presents an exploratory study investigating the impact of non-native accented speech on the perception of Intelligent Virtual Agents (IVAs). In an online study, native English speakers watched a video of an IVA holding a monologue whilst speaking English with either a Spanish, Hindi or Mandarin accent that was either recorded by native speakers of that respective language (natural speech) or synthetically generated (synthetic speech). The results showed a significant impact of naturalness of speech on the IVAs perceived warmth and a significant interaction of accent and naturalness of speech on its perceived competence. The naturalness of speech impacted the participants’ perception of the IVA as a non-native speaker of English, and the correctness of the attributed mother tongue in the Spanish and the Mandarin accent condition. These results are a valuable contribution to research on mixed-cultural IVAs in general and non-native speech as a cultural cue more specifically.

This study investigates how different levels of a non-native Turkish accent in German speech are perceived by Turkish-German listeners, using either natural or synthetic speech. The participants listened to six audio recordings and rated the respective speaker regarding her mother tongue, warmth, competence, and intelligibility. The results show that the naturalness of speech had no impact on the non-native speakers’ ability to assign the correct mother tongue to the respective speaker. It did, however have an impact on the speakers’ perceived warmth, competence and intelligibility.

Social robots can use biomimetic modalities such as gestures to convey emotions. The use of colored light for emotion expression is also possible, but rarely explored. In this paper, colored LEDs are used in addition to contextual gestures to communicate emotions in robotic storytelling. Results show that adding colored light to the storytelling did not improve the recipients' transportation into the story. Their cognitive absorption was significantly decreased. The users' perception of the NAO robot was not influenced by colored lights except for animacy which was higher using only white LEDs. Problems might have been mismatches between colors and emotions, the lack of emotional gestures, and the too small light emission from NAO's eye LEDs. Since smart rooms lighting can affect the users' whole visual field, they could enhance emotional effects. Thus, future studies should investigate the integration of a robotic storyteller in smart rooms allowing for smart light control.

This paper presents an exploratory study which investigates the impact of different non-native accents and the naturalness of speech on the correct assignment of an Intelligent Virtual Agent’s (IVA) mother tongue, as well as its perceived warmth, competence and intelligibility. An online-experiment with a between subjects design was conducted, in which the participants, who were native speakers of German, watched a video of an IVA that spoke German with a non-native accent. The IVA’s speech was either synthetically generated or pre-recorded using non-native speakers. The participants experienced an IVA with either a Turkish, Italian or Polish accent, based on the most frequent accents in the German-speaking area. The results revealed that the IVA’s accent impacted its perceived warmth, but not its perceived competence and intelligibility. The IVA’s naturalness of speech played no role in its classification as a non-native speaker of German but on the correctness of the assigned mother tongue within the Polish accent condition. These results give valuable insight in the perception of non-native speaking IVAs and constitute helpful implications for future research with mixed-cultural IVAs.

Learning in universities challenges students to engage in self-directed learning, which requires a high degree of self-motivation while individual support by teachers is limited. Research on social robots has already demonstrated their potential to support students in their learning process. In this paper, we focus on the benefits of adaptivity of a robotic tutor in a higher education scenario. To this end, we conducted a field study over three sessions over the course of a semester and implemented two conditions (adaptive and non-adaptive) of a robotic tutor to support students with exam preparation. After participant learned with both version in random order in the first two sessions, their preferred condition was used in the third session. Our results show that significantly more students preferred to learn with the adaptive robotic tutor. Additionally, participation resulted in significantly better exam performance compared to the average of the course. However, there was no significant difference in the learning experience such as motivation or need satisfaction between conditions.

Social robots found their way into several economic fields of our daily life, e.g., as info points in shopping malls or hotels. However, research on robots in the tourism sector is still in its infancy. Conducting three online studies, we examined the design of robotic speech for a robotic concierge. Providing self-disclosure via speech or using local dialect did not affect robot acceptance and attitudes. In contrast, informal speech led to higher likeability and perceived warmth of the robot concierge. Overall, our results reveal positive evaluations of our robotic concierge regardless of its manipulation.

Social robots are applied in several fields such as education and entertainment. Using the method of storytelling, these two fields can be combined to allow for informal learning. As social robots can benefit students' motivation and learning process and storytelling is a consolidated method for vocabulary acquisition, integrating learning content into a robotic storyteller's narration might facilitate implicit learning. Focusing on language learning, we conducted an online study to compare a robotic storyteller for vocabulary learning to traditional media, e.g. text. Results show that a robotic storyteller is as effective as traditional media concerning long-term memory. While it is also perceived equally useful, participants were more satisfied when reading text. No differences were indicated for transportation into the story. This preliminary online study indicates the suitability of robotic storytelling for implicit language learning, however, future studies should examine live and co-located scenarios to investigate their full potential.

Separating and recycling waste is an important topic to protect our environment and achieve a more sustainable future. However, recycling also is a complex process, as each type of waste needs a specific recycling method. This comes along with multiple recycling containers, each relevant for one specific type of waste. Ensuring a correct recycling process therefore not only requires specific infrastructure, but also a respective attitude and knowledge of the population. Stressing the need for an accessible educational opportunity addressing waste separation, we present a mobile Augmented Reality (AR) application that guides a user through a recycling process and hence scaffolds the learning of proper recycling of each type of waste. The app further provides a prototypical implementation of a product scanner, that identifies the waste type based on a marker and assists the recycling on a case-by-case decision. Using self-determination theory as a framework, we integrated gamification elements, aiming for enhanced need satisfaction, motivation and user experience. In a user study, we compared the gamified version to a control version, with both app versions yielding a high acceptance, user experience, and waste separation behavior. This indicates the importance of providing easy-to-use mobile apps allowing for a learning and assistance of proper recycling.

This paper presents the implementation of a prototype including a virtual environment in which humans are exposed to a seemingly autonomous Intelligent Virtual Agent (IVA) that ostracises them by using its smartphone during a conversation (a phenomenon called phubbing with adverse outcomes in interhuman communication). Based on the temporal need threat model of ostracism, the effects of phubbing were examined in an online experiment with three conditions (no vs. proactive vs. reactive phubbing) using an IVA. The results showed a significant effect on the warmth attributed to the IVA. Consequently, our study provides partial support that the negative effects of phubbing can be transfered to human-agent interaction. It is thereby not only in line with the underlying theory, but also provides a prototype suitable for future research with phubbing in the context of IVAs.

Abstract Reading is an essential ability and a cornerstone of education. However, learning to read can be challenging for children. To scaffold young learners, a number of reading interventions were developed, including a syllable-based approach in German, which has proven to be successful, but resource and time consuming through individual interaction by educators. To improve the reach of the reading intervention, we present the first step towards a digital intervention, following an iterative design approach. In this contribution, we present the implementation of a digital prototype, developed with the feedback of expert evaluations, as well as an interview study with second graders. The results of interviews with children showed that the app is suitable to be applied in the target age group, that children had fun using it and were motivated to further do so. In a next step towards a meaningful digitalization of the analogue intervention, we extended the application based on the evaluation results and conducted a usability evaluation of the extended app. The study as well as the usability evaluation provides design implications for iteratively transferring the analogue concept into a digital application.

The benefits of social robots in educational contexts were mainly investigated with children, but also bear great potential to support learners and teachers in higher education. To further explore the potential of social robots in the context of university teaching, we implemented a robot-supported learning environment as a complementary training to a university course. To learn more about the students' perspective and attitudes towards the integration of robots in their education, we conducted a field study with qualitative interviews as data collection method. Our results show a clear positive perception of the robot-supported learning environment, and indicate a positive impact on the learning outcomes. Most students suppose an additional value in the presence of the robot compared to traditional on-screen scenario or self-study, and perceived the robot to increase their motivation, attention and concentration. We found a clear interest of the students to use the learning environment again in the future. However, more individualized feedback was desired.

The present contribution investigates the effects of spoken language varieties, in particular non-standard / regional language compared to standard language (in our study: High German), in social robotics. Based on (media) psychological and sociolinguistic research, we assumed that a robot speaking in regional language (i.e., dialect and regional accent) would be considered less competent compared to the same robot speaking in standard language (H1). Contrarily, we assumed that regional language might enhance perceived social skills and likability of a robot, at least so when taking into account whether and how much the human observers making the evaluations talk in regional language themselves. More precisely, it was assumed that the more the study participants spoke in regional language, the better their ratings of the dialect-speaking robot on social skills and likeability would be (H2). We also investigated whether the robot's gender (male vs. female voice) would have an effect on the ratings (RQ). H1 received full, H2 limited empirical support by the data, while the robot's gender (RQ) turned out to be a mostly negligible factor. Based on our results, practical implications for robots speaking in regional language varieties are suggested.

This paper presents an experiment evaluating the effects of virtual agents' language proficiency on whether they are perceived as a native speakers or not. Our first results indicate that beyond 10% word order mistakes and 25% infinitive mistakes, virtual agents are perceived as non-native speakers, even though their appearance and non-verbal behaviour were not altered. We believe these thresholds constitute interesting guidelines possibly simplifying the design of non-native speaker simulations.

Culture, amongst other individual and social factors, plays a crucial role in human-human interactions. If robots should become a part of our society, they should be able to act in culture-specific manners as well. In this paper, we showcase the implementation of a cultural dichotomy, namely individualism vs. collectivism, in a social robots' conversation. Presenting these conversations to human observers from Germany and Japan, we investigate whether the implemented differences are recognized as such, and whether stereotypical culture-specific behaviours that correspond to the observers' cultural background is preferred. Results suggest that the manipulations in behaviour had the intended effect, but are not reflected in personal preferences.

The integration of migrants and refugees is currently a severe challenge for European states. Especially the imparting of culture- and gender-specific behaviours is an important issue. Social robots might be a valuable tool to introduce refugees to culture-specific behaviours of their host country. In this paper, we investigate the general acceptance of a social robot as well as users' perception of a robot presenting stereo-typical Arabic vs. German female non-verbal behaviour to Syrian newcomers to Germany. Our preliminary study revealed a generally positive attitude towards robots and the idea of an educational robot. Culture-specific manipulations were reflected in participants' partial preference for the Arabic version, but not in participants' perceptual ratings.

People tend to adapt to their conversation partners. In mixed-cultural settings, with a native and a non-native speaker, adaptation can manifest itself in the usage of simplified language or increased usage of nonverbal scaffolding to foster understanding. In this contribution, we address the question whether the phenomenon of AFC (adapted, foreigner-directed communication) is also shown towards a virtual agent. We therefore implemented a demonstrator with a local and a foreign agent that interact with human users in a direction giving scenario. A user study revealed that participants behaved differently towards the two agents and adapted both, their verbal and their non-verbal behaviour. We are thus confident, that our demonstrator is well suited to systematically study AFC.

In order to enable a social agent to behave in a believable and realistic way, it needs a wide range of information in the form of both low-level value-based data as well as high-level semantic knowledge. In this work we propose a system that puts a virtual reality layer between the real world and an agent's knowledge representation. This mirror world allows the agent to use its abstract representation of the environment and inferred events as an additional source of knowledge when reasoning about the real world. Additionally, users and developers can use the mirror world, with its visualized data and highlighting of the agent's reasoning, for further understanding of the agent's behavior, debugging and testing, or the simulation of additional sensor input.

This paper introduces an interactive surface concept for Mixed Reality (MR) tabletop games that combines a variable (LCD and/or projection) screen configuration with the detection of finger touches, in-air gestures, and tangibles. It is low-cost and minimally requires an ordinary table, a TV screen, and a Kinect v2 sensor. Existing applications can easily be connected by being compliant to standards. The concept is intended to foster further research on collaborative tabletop situations, not limited to games, but also including learning, meetings, and social interaction.

Providing effective feedback can empower users to change their behaviour and take the necessary actions to reduce their energy consumption. The types of feedback that allow comparison of energy usage seem to be particularly valuable. This paper introduces the time-stack visualization, which has been designed to support comparisons of individual and collective energy usage data. It also describes a user study conducted to compare the effectiveness of time-stack against a similar visualization called time-pie. The results show that although the two visualizations are generally comparable in their effectiveness, users rate time-stack more favourably.

The outcome of interpersonal interactions depends not only on the contents that we communicate verbally, but also on nonverbal social signals. Because a lack of social skills is a common problem for a significant number of people, serious games and other training environments have recently become the focus of research. In this work, we present NovA ( No n v erbal behavior A nalyzer), a system that analyzes and facilitates the interpretation of social signals automatically in a bidirectional interaction with a conversational agent. It records data of interactions, detects relevant social cues, and creates descriptive statistics for the recorded data with respect to the agent's behavior and the context of the situation. This enhances the possibilities for researchers to automatically label corpora of human--agent interactions and to give users feedback on strengths and weaknesses of their social behavior.

Through interaction with the virtual environment and virtual characters, users are able to influence the storyline of many games. The design choice for the style of interactivity can thereby have a crucial influence on the user's experience. However, only a few approaches evaluate different interaction modalities for one system to investigate the impact of design choice on the users' experience. In this paper, we present an experimental approach in which we first reflect on design alternatives concerning a specific element of interactive narratives-user-character dialog-and then investigate user responses to different design options (round-based dialog versus continuous dialog). Results of an experimental evaluation study show that users tend to prefer continuous interaction in a soap-opera-like game environment using typed text input to communicate with virtual characters that act and react using speech output, although the recognition rate of user utterances of the continuous version was slightly worse compared to the round-based version.

In recent years a growing number of information visualization systems have been developed to assist users with monitoring their energy consumption, with the hope of reducing energy use through more effective user-awareness. Most of these visualizations can be categorized into either some form of a time-series or pie chart, each with their own limitations. These visualization systems also often ignore incorporating contextual (e.g. weather, environmental) information which could assist users with better interpretation of their energy use information. In this paper we introduce the time-pie visualization technique, which combines the concepts of timeseries and pie charts, and allows the addition of contextual information to energy consumption data.

In social robotics, the behavior of humanoid robots is intended to be designed in a way that they behave in a human-like manner and serve as natural interaction partners for human users. Several aspects of human behavior such as speech, gestures, eye-gaze as well as the personal and social background of the user need therefore to be considered. In this paper, we investigate interpersonal distance as a behavioral aspect that varies with the cultural background of the user. We present two studies that explore whether users of different cultures (Arabs and Germans) expect robots to behave similar to their own cultural background. The results of the first study reveal that Arabs and Germans have different expectations on the interpersonal distance between themselves and robots in a static setting. In the second study, we use the results of the first study to investigate the users' reactions on robots using the observed interpersonal distances themselves. Although the data of this dynamic setting is not conclusive, it suggests that users prefer robots that show behavior that has been observed for their own cultural background before.

In this paper, we present a modeling approach for the management of highly interactive, multithreaded and multimodal dialogues. Our approach enforces the separation of dialogue content and dialogue structure and is based on a statechart language enfolding concepts for hierarchy, concurrency, variable scoping and a detailed runtime history. These concepts facilitate the modeling of interactive dialogues with multiple virtual characters, autonomous and parallel behaviors, flexible interruption policies, context-sensitive interpretation of the user's discourse acts and coherent resumptions of dialogues. An interpreter allows the realtime visualization and modification of the model to allow a rapid prototyping and easy debugging. Our approach has successfully been used in applications and research projects as well as evaluated in field tests with non-expert authors. We present a demonstrator illustrating our concepts in a social game scenario.