Creating 3D assets is essential for game development. Modeling requires time and technical skills, and fast prototyping is challenging, especially for novice designers. To make complex 3D modeling more accessible while strengthening creativity, we present CreatureCreator, a tool that enables users to generate 3D models from an image of a created artifact. We investigated two input modalities: drawing with pen and paper and sculpting with clay. Adding an analog input modality to the generation process could lower the entry barrier for diverse user groups, enabling intuitive and creative modeling. In a within-subject study with game design-experienced participants (n=20), both modalities scored highly for user experience, engagement, and creativity support, with the clay input especially valued for its generated models and enhanced sense of control. The interaction workflow supports playful exploration by allowing users to experiment with parameters and using AI as a production and/or ideation partner.

Embodiment of users within robotic systems has been explored in human-robot interaction, most often in telepresence and teleoperation. In these applications, synchronized visuomotor feedback can evoke a sense of body ownership and agency, contributing to the experience of embodiment. We extend this work by employing embreathment, the representation of the user's own breath in real time, as a means for enhancing user embodiment experience in robots. In a within-subjects experiment, participants controlled a robotic arm, while its movements were either synchronized or non-synchronized with their own breath. Synchrony was shown to significantly increase body ownership, and was preferred by most participants. We propose the representation of physiological signals as a novel interoceptive pathway for human–robot interaction, and discuss implications for telepresence, prosthetics, collaboration with robots, and shared autonomy.

Dark patterns are deceptive strategies that recent work in humancomputer interaction (HCI) has captured throughout digital domains, including social networking sites (SNSs).While research has identified difficulties among people to recognise dark patterns effectively, few studies consider vulnerable populations and their experience in this regard, including people with attention deficit hyperactivity disorder (ADHD), who may be especially susceptible to attention-grabbing tricks.Based on an interactive web study with 135 participants, we investigate SNS users' ability to recognise and avoid dark patterns by comparing results from participants with and without ADHD.In line with prior work, we noticed overall low recognition of dark patterns with no significant differences between the two groups.Yet, ADHD individuals were able to avoid specific dark patterns more often.Our results advance previous work by understanding dark patterns in a realistic environment and offer insights into their effect on vulnerable populations.

We present a sensory substitution-based method for representing locations of remote objects in 3D space via haptics. By imitating auditory localization processes, we enable vibrotactile localization abilities similar to those of some spiders, elephants, and other species. We evaluated this concept in virtual reality by modulating the vibration amplitude of two controllers depending on relative locations to a target. We developed two implementations applying this method using either ear or hand locations. A proof-of-concept study assessed localization performance and user experience, achieving under 30° differentiation between horizontal targets with no prior training. This unique approach enables localization by using only two actuators, requires low computational power, and could potentially assist users in gaining spatial awareness in challenging environments. We compare the implementations and discuss the use of hands as ears in motion, a novel technique not previously explored in the sensory substitution literature.

Robot appearance crucially shapes Human-Robot Interaction (HRI) but is typically described via broad categories like anthropomorphic, zoomorphic, or technical. More precise approaches focus almost exclusively on anthropomorphic features, which fail to classify robots across all types, limiting the ability to draw meaningful connections between robot design and its effect on interaction. In response, we present METAMORPH, a comprehensive framework for classifying robot morphology. Using a metamodeling approach, METAMORPH was synthesized from 222 robots in the IEEE Robots Guide, offering a structured method for comparing visual features. This model allows researchers to assess the visual distances between robot models and explore optimal design traits tailored to different tasks and contexts.

Affordances - i.e. possibilities for action that an environment or objects in it provide - are important for robots operating in human environments to perceive. Existing approaches train such capabilities on annotated static images or shapes. This work presents a novel dataset for affordance learning of common household tasks. Unlike previous approaches, our dataset consists of video sequences demonstrating the tasks from first- and third-person perspectives, along with metadata about the affordances that are manifested in the task, and is aimed towards training perception systems to recognize affordance manifestations. The demonstrations were collected from several participants and in total record about seven hours of human activity. The variety of task performances also allows studying preparatory maneuvers that people may perform for a task, such as how they arrange their task space, which is also relevant for collaborative service robots.

In this explorative study, we investigated how a zoomorphic dog-like robot could encourage users to engage in active behavior through nudges with different levels of intrusiveness. We examined three hypotheses on the effectiveness of different intensity levels, the impact of zoomorphic design on emotional responses, and the influence of prior dog experience on interactions. Using a within-subject approach with 34 participants, we found that low-intensity nudges were most sufficient at encouraging active behavior, contradicting our first hypothesis. Participants generally responded positively to the zoomorphic design, supporting our second hypothesis. Our third hypothesis was approved as participants with prior dog experience perceived the robot as less intrusive and reported higher user experience. This finding was complemented by these specific users putting a powerful mental model of a real dog directly on our robot dog, sometimes even resulting in projecting attributes like "thirst" or other desires to the robot. We call this new finding "Fluffy Tightrope" - the balancing act between realism and abstraction in designing a zoomorphic robot, so that users with animal experience respond to nudges, but do not project too strong wishes and desires onto the robot resulting in misinterpretations.

This study explores which factors of the visual design of a robot may influence how humans would place it in a collaborative cooking scenario and how these features may influence task delegation. Human participants were placed in a Virtual Reality (VR) environment and asked to set up a kitchen for cooking alongside a robot companion while considering the robot's morphology. We collected multimodal data for the arrangements created by the participants, transcripts of their think-aloud as they were performing the task, and transcripts of their answers to structured post-task questionnaires. Based on analyzing this data, we formulate several hypotheses: humans prefer to collaborate with biomorphic robots; human beliefs about the sensory capabilities of robots are less influenced by the morphology of the robot than beliefs about action capabilities; and humans will implement fewer avoidance strategies when sharing space with gracile robots. We intend to verify these hypotheses in follow-up studies.

Fidelity describes how closely a replication resembles the original. It can be helpful to analyze how faithful interactions in virtual reality (VR) are to a reference interaction. In prior research, fidelity has been restricted to the simulation of reality—also called realism. Our definition includes other reference interactions, such as superpowers or fiction. Interaction fidelity is a multilayered concept. Unfortunately, different aspects of fidelity have either not been distinguished in scientific discourse or referred to with inconsistent terminology. Therefore, we present the Interaction Fidelity Model (IntFi Model). Based on the human-computer interaction loop, it systematically covers all stages of VR interactions. The conceptual model establishes a clear structure and precise definitions of eight distinct components. As a communication tool, it helps teams to understand and discuss fidelity in VR. It was reviewed through workshops with fourteen VR experts. We provide guidelines, diverse examples, and educational material to apply the IntFi Model universally to any VR experience and propose foundational research opportunities.

As the gaming industry advances, it seeks increased immersion. Concurrently, recent advancements in speech recognition substantially enhanced its accuracy, creating opportunities for speech-based interactions in gameplay. Integrating speech into single-player games with non-player characters (NPCs) introduces a new social dimension. Yet, like toxicity in social media and multiplayer games, anti-social behavior can infiltrate NPC interactions, generating a toxic gaming atmosphere. We designed a speech-based game called “A Day at the Office” and conducted a within-subject study with 26 participants to explore player interactions with NPCs possessing different human qualities. Furthermore, we introduced courtesy-based game mechanics, where NPCs react to players’ choice of words and loudness, investigating player experiences. Results revealed that participants spoke differently with NPCs based on their human qualities, such as gender and hierarchical roles. Moreover, the courtesy-based interactions fostered greater cognitive engagement, suggesting that tactful interactions contribute to enhanced immersion and presence in the game.

Studies show that evoking empathy can positively affect the host society's acceptance of refugees, whose numbers are increasing due to political instability and environmental disasters. To examine how commercially available serious games designed to foster empathy with refugees are perceived by refugees and residents, we invited both groups to play two games: Path Out and Bury Me, My Love. Following the gaming session, we used semi-structured interviews, creating a safe space, with the refugees (n=6) to ask about their personal experiences. Residents (n=5) joined a focus group to facilitate a broader discussion. Our results show that the employed game mechanics, such as decisions with unclear consequences and narratives, generally evoked empathy. Yet, the residents missed a more extensive background on the circumstances to create a deeper connection with the games’ protagonists, and both groups wished for more agency, leaving them with a feeling of passivity. These aspects are crucial to take into account when designing new games to foster empathy for marginalized groups.

Advances in natural language processing and understanding have led to a rapid growth in the popularity of conversational user interfaces (CUIs). While CUIs introduce novel benefits, they also yield risks that may exploit people’s trust. Although research looking at unethical design deployed through graphical user interfaces (GUIs) established a thorough understanding of so-called dark patterns, there is a need to continue this discourse within the CUI community to understand potentially problematic interactions. Addressing this gap, we interviewed 27 participants from three cohorts: researchers, practitioners, and frequent users of CUIs. Applying thematic analysis, we construct five themes reflecting each cohort’s insights about ethical design challenges and introduce the CUI Expectation Cycle, bridging system capabilities and user expectations while considering each theme’s ethical caveats. This research aims to inform future development of CUIs to consider ethical constraints while adopting a human-centred approach.

Recent advances in language processing and speech recognition open up a large opportunity for video game companies to embrace voice interaction as an intuitive feature and appealing game mechanics. However, speech-based systems still remain liable to recognition errors. These add a layer of challenge on top of the game’s existing obstacles, preventing players from reaching their goals and thus often resulting in player frustration. This work investigates a novel method called context-aware speech recognition, where the game environment and actions are used as supplementary information to enhance recognition in a speech-based game. In a between-subject user study (N = 40), we compared our proposed method with a standard method in which recognition is based only on the voice input without taking context into account. Our results indicate that our proposed method could improve the player experience and the usability of the speech system.

With video games and Esports becoming ever-present, playing as long as possible to improve proficiency has become a common strategy among video game enthusiasts. This behavior does not only bear increased health and social risks, but is also connected to performance decline, which itself inhibits skill acquisition. To promote health and advance self-regulated learning, we investigate the effects of distributed practice on skill acquisition in an ecologically valid Esports task. As research about distributed practice in video games is restricted to simple games or short practice sessions, we explore the impact of distributed versus massed practice with an extended training period of 45 minutes. Using (n = 16) participants and a complex contemporary fighting game, we show that massed practice schedules do not provide any advantage over distributed ones - suggesting that practice time can be evenly distributed without improvement losses, which would alleviate health risks connected to longer sessions.

This study explores the prevalence of dark patterns in mobile games that exploit players through temporal, monetary, social, and psychological means. Recognizing the ethical concerns and potential harm surrounding these manipulative strategies, we analyze user-generated data of 1496 games to identify relationships between the deployment of dark patterns within "dark" and "healthy" games. Our findings reveal that dark patterns are not only widespread in games typically seen as problematic but are also present in games that may be perceived as benign. This research contributes needed quantitative support to the broader understanding of dark patterns in games. With an emphasis on ethical design, our study highlights current problems of revenue models that can be particularly harmful to vulnerable populations. To this end, we discuss the relevance of community-based approaches to surface harmful design and the necessity for collaboration among players/users and practitioners to promote healthier gaming experiences.

Rhythm is essential in early music education. To acquire rhythm abilities, the synchronisation between body and rhythm patterns is fundamental. Extensive research has shown the link between the auditory-motor system and rhythm acquisition; however, other sensorimotor systems also improve rhythm skills. Rhythm acquisition can be trained using new technologies, such as video games; however, most of these use mainly auditory stimuli to improve it. Therefore, in this work, we explore tactile stimuli to improve rhythm skills using a video game and how visual elements affect rhythm perception using different stimuli. We evaluate and compare the synchronisation of 27 participants following either vibrotactile or auditory stimuli emitted through a rhythm video game in two difficulty levels and we evaluated the user experience. Consistent with the literature, our results indicate that participants synchronised slightly better using auditory than vibrotactile stimuli and players’ performance was better when they had visual guides. A fatigue effect persisted and increased over playtime in both conditions. In the current study, usability measures indicate no significant differences. These findings suggest that vibrotactile stimuli are also suitable to train rhythm skills in a video game environment and the game-elements play an important role in players' performance.

With the rise of popularity and societal importance of video games, trends of grinding play for excessive amounts of time to improve proficiency becomes more and more common. This bears not only physical and mental health risks, but has already shown to be detrimental for skill acquisition, defeating its purpose in the first place. For the sake of health promotion, well-being, and the advancement of self-regulated learning, we specifically investigate the impact of detailed feedback on player performance. As research about comparable activities in related domains suggest inconclusive implications for when feedback in video game learning should be presented, we particularly explore the impact of delivering immediate versus delayed feedback, during or after play. Using (n = 30) participants and the well-investigated Super Mario Bros., we communicate insights in favor for deploying feedback immediately when polishing one's video game related skills such as in speedrunning.

Interest in unethical user interfaces has grown in HCI over recent years, with researchers identifying malicious design strategies referred to as “dark patterns”. While such strategies have been described in numerous domains, we lack a thorough understanding of how they operate in social networking services (SNSs). Pivoting towards regulations against such practices, we address this gap by offering novel insights into the types of dark patterns deployed in SNSs and people’s ability to recognise them across four widely used mobile SNS applications. Following a cognitive walkthrough, experts (N = 6) could identify instances of dark patterns in all four SNSs, including co-occurrences. Based on the results, we designed a novel rating procedure for evaluating the malice of interfaces. Our evaluation shows that regular users (N = 193) could differentiate between interfaces featuring dark patterns and those without. Such rating procedures could support policymakers’ current moves to regulate deceptive and manipulative designs in online interfaces.

Research in HCI has shown a growing interest in unethical design practices across numerous domains, often referred to as “dark patterns”. There is, however, a gap in related literature regarding social networking services (SNSs). In this context, studies emphasise a lack of users’ self-determination regarding control over personal data and time spent on SNSs. We collected over 16 hours of screen recordings from Facebook’s, Instagram’s, TikTok’s, and Twitter’s mobile applications to understand how dark patterns manifest in these SNSs. For this task, we turned towards HCI experts to mitigate possible difficulties of non-expert participants in recognising dark patterns, as prior studies have noticed. Supported by the recordings, two authors of this paper conducted a thematic analysis based on previously described taxonomies, manually classifying the recorded material while delivering two key findings: We observed which instances occur in SNSs and identified two strategies — engaging and governing — with five dark patterns undiscovered before.

We present the Meta-Ontology for Introspection (MOI): Inspired by fundamental processes of the human mind, cognitive architectures (CAs) explore ever more methods to leverage metacognition. Still, an ontological model to trace metacognitive experiences for learning or as input for metacognitive control routines has yet to be developed. Based on a review of existing standards, we formally identify the relevant scope in the form of Competency Questions (CQs) and extend SOMA, a well-established formal ontology initially designed to interpret episodic memories of a robotic CA. The resulting MOI can model a CA’s software and capabilities of single components, trace information processing and inter-component communication, label self-lived mental events, and capture causal relationships. We evaluate MOI via the CQs and exemplarily demonstrate its reasoning capabilities.

While a growing number of households contain home assistants, they mainly remain voice-only devices where the virtual agent is not represented visually. The visual representation of the agent is limited to the device's housing and abstract light animations that signify the assistant's state to its users. However, the audio channel is limited in conveying information beyond semantic content. Embodied virtual assistants can enhance interaction with conversational interfaces by adding a visual layer to further convey personality and human characteristics. In this work, we conducted an online survey (N=78) to explore people's preferences for visualizing humanoid assistants. Our findings suggest that participants prefer an agent who appears mature, healthy, competent, and attractive. Furthermore, demographic similarities between the users and agents are wished for the agent to look more relatable. We discuss these findings and their implications for the design of virtual humanoid home assistants.

With rapid advances in artificial intelligence and natural language processing, voice assistants are evolving into advanced digital personal assistants capable of complex tasks. As they become more proficient at understanding people’s behaviors, preferences, intentions, and surroundings, opportunities for proactive interactions emerge. However, despite their potential benefits, people still find certain proactive agent interactions inappropriate and invasive, such as correcting or nudging the user. This study investigates humor’s potential to enhance the desirability of proactive agent comments, given its stress-relieving and acceptance-promoting characteristics. We investigate how infusing humor into VA statements affects perceptions of appropriateness and desirability in proactive interventions. We designed storyboards showcasing voice assistants’ proactive actions in everyday situations and social contexts. Participants () assessed these scenarios in an online questionnaire across multiple criteria. Our results reveal that humor’s impact on proactive statement desirability is contingent on participants’ perceptions of voice assistants and their subjective judgment of the humor.

Video game testing has become a major investment of time, labor, and expense in the game industry. Particularly the balancing of in-game units, characters, and classes can cause long-lasting issues that persist years after a game's launch. While approaches incorporating artificial intelligence have already shown successes in reducing manual effort and enhancing game development processes, most of these draw on heuristic, generalized, or optimal behavior routines, while actual low-level decisions from individual players and their resulting playing styles are rarely considered. In this article, we applydeep player behavior modelingto turn atomic actions of 213 players from six months of single-player instances within the MMORPG Aion into generative models that capture and reproduce particular playing strategies. In a subsequent simulation, the resulting generative agents (“replicants”) were tested against common NPC opponent types of MMORPGs that iteratively increased in difficulty, respective to the primary factor that constitutes this enemy type (Melee, Ranged, Rogue, Buffer, Debuffer, Healer, Tank, or Group). As a result, imbalances between classes as well as strengths and weaknesses regarding particular combat challenges could be identified and regulated automatically.

Providing haptic feedback in virtual reality to make the experience more realistic has become a strong focus of research in recent years. The resulting haptic feedback systems differ greatly in their technologies, feedback possibilities, and overall realism making it challenging to compare different systems. We propose the Haptic Fidelity Framework providing the means to describe, understand and compare haptic feedback systems. The framework locates a system in the spectrum of providing realistic or abstract haptic feedback using the Haptic Fidelity dimension. It comprises 14 criteria that either describe foundational or limiting factors. A second Versatility dimension captures the current trade-off between highly realistic but application-specific and more abstract but widely applicable feedback. To validate the framework, we compared the Haptic Fidelity score to the perceived feedback realism of evaluations from 38 papers and found a strong correlation suggesting the framework accurately describes the realism of haptic feedback.

Advances in speech recognition, language processing and natural interaction have led to an increased industrial and academic interest. While the robustness and usability of such systems are steadily increasing, speech-based systems are still susceptible to recognition errors. This makes intelligent error handling of utmost importance for the success of those systems. In this work, we integrated anticipatory error handling for a voice-controlled video game where the game would perform a locally optimized action in respect to goal completion and obstacle avoidance, when a command is not recognized. We evaluated the user experience of our approach versus traditional, repetition-based error handling (). Our results indicate that implementing anticipatory error handling can improve the usability of a system, if it follows the intention of the user. Otherwise, it impairs the user experience, even when deciding for technically optimal decisions.

When playing sports in virtual reality foot interaction is crucial for many disciplines. We investigated how the visibility of the foot influences penalty shooting in soccer. In a between-group experiment, we asked 28 players to hit eight targets with a virtual ball. We measured the performance, task load, presence, ball control, and body ownership of inexperienced to advanced soccer players. In one condition, the players saw a visual representation of their tracked foot which improved the accuracy of the shots significantly. Players with invisible foot needed 58% more attempts. Further, with foot visibility the self-reported body ownership was higher.

The use of procedural content generation (PCG) in the context of video games has increased over the years as it provides an economical way to generate game content whilst enhancing their variety and replayability. For city-building games, this approach is often utilized to predefine map layouts, terrains, or cityscapes for the player. One core aspect of facilitating enjoyment in these games comes from creative expressivity. PCG, in this context, may support creativity by lowering the technical complexity for content creation, or it may hinder creativity by taking away control and freedom from the user. To examine these potential effects, this paper investigates if PCG has an impact on players' creativity in the context of VR city-building games. We present a VR prototype that provides varying degrees of procedural content: No PCG, terrain generation, city generation, and full (city + terrain) generation. In a remote user study, these conditions were compared regarding their capability to support creativity. Statistical tests for equivalence revealed that the presence of PCG did not affect creativity in any way. Our work suggests that PCG can be a useful integration into city-building games without notably decreasing players' ability to express themselves creatively.

Many serious games are most effective when played regularly; however, little is known about how individual game elements support player adherence over time. This work draws on evidence from existing frameworks and game design theories as well as from the design of casual games to investigate how individual game mechanics affect player attrition in a serious game. We implemented a math-learning game in which we could individually layer various game mechanics, and over the course of 3 weeks, 99 participants played one of six versions: Baseline, Rewards, Novelty, Completion, Waiting, or Blocking. We compared the game versions by analyzing the players’ performance as well as behaviour. Using survival analysis, we identified that the addition of Completion and Blocking mechanics facilitated the strongest sustained engagement. These findings are congruent with existing theories of player experience and promote the development of guidelines on designing for sustained engagement in serious games.

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave many pieces of information necessary for performing the task unspecified. Humans can solve such problems fast as we reduce the search space by recourse to prior knowledge such as a connected collection of plans that describe how certain goals can be achieved at various levels of abstraction. Rather than enumerating fine-grained physical contexts SOMA sets out to include socially constructed knowledge about the functions of actions to achieve a variety of goals or the roles objects can play in a given situation. As the human cognition system is capable of generalizing experiences into abstract knowledge pieces applicable to novel situations, we argue that both physical and social context need be modeled to tackle these challenges in a general manner. The central contribution of this work, therefore, lies in a comprehensive model connecting physical and social entities, that enables flexibility of executions by the robotic agents via symbolic reasoning with the model. This is, by and large, facilitated by the link between the physical and social context in SOMA where relationships are established between occurrences and generalizations of them, which has been demonstrated in several use cases in the domain of everyday activites that validate SOMA.

The use of voice assistants (VAs) is spreading widely. Most common VAs consist of a single, usually female voice that responds to the user’s inquiry. We designed a VA system appearing as a group of agents, each with a different voice and a specialized task domain. We conducted a quantitative user study comparing our multi-agent approach with a conventional single-agent assistant in a smart home scenario as virtual reality (VR) simulation. The results show significantly higher user experience ratings for the multi-agent concept. Based on our findings, we discuss the potentials and challenges of designing multi-party VA systems.

Questionnaires are among the most common research tools in virtual reality (VR) user studies. Transitioning from virtuality to reality for giving self-reports on VR experiences can lead to systematic biases. VR allows to embed questionnaires into the virtual environment which may ease participation and avoid biases. To provide a cohesive picture of methods and design choices for questionnaires in VR (inVRQ), we discuss 15 inVRQ studies from the literature and present a survey with 67 VR experts from academia and industry. Based on the outcomes, we conducted two user studies in which we tested different presentation and interaction methods of inVRQs and evaluated the usability and practicality of our design. We observed comparable completion times between inVRQs and questionnaires outside VR (nonVRQs) with higher enjoyment but lower usability for \inVRQs. These findings advocate the application of inVRQs and provide an overview of methods and considerations that lay the groundwork for inVRQ design.

Many online games suffer when players drop off due to lost connections or quitting prematurely, which leads to match terminations or game-play imbalances. While rule-based outcome evaluations or substitutions with bots are frequently used to mitigate such disruptions, these techniques are often perceived as unsatisfactory. Deep learning methods have successfully been used in deep player behavior modelling (DPBM) to produce non-player characters or bots which show more complex behavior patterns than those modelled using traditional AI techniques. Motivated by these findings, we present an investigation of the player-perceived awareness, believability and representativeness, when substituting disconnected players with DPBM agents in an online-multiplayer action game. Both quantitative and qualitative outcomes indicate that DPBM agent substitutes perform similarly to human players and that players were unable to detect substitutions. Notably, players were in fact able to detect substitution with agents driven by more traditional heuristics.

Balancing games and producing content that remains interesting and challenging is a main cost factor in the design and maintenance of games. Dynamic difficulty adjustments (DDA) can successfully tune challenge levels to player abilities, but when implemented with classic heuristic parameter tuning (HPT) often turns out to be very noticeable, e.g. as "rubber-banding". Deep learning techniques can be employed for deep player behavior modeling (DPBM), enabling more complex adaptivity, but effects over frequent and longer-lasting game engagements, as well as how it compares to HPT has not been empirically investigated. We present a situated study of the effects of DDA via DPBM as compared to HPT on intrinsic motivation, perceived challenge and player motivation in a real-world MMORPG. The results indicate that DPBM can lead to significant improvements in intrinsic motivation and players prefer game experience episodes featuring DPBM over experience episodes with classic difficulty management.

Questionnaires are among the most common research tools in virtual reality (VR) evaluations and user studies. However, transitioning from virtual worlds to the physical world to respond to VR experience questionnaires can potentially lead to systematic biases. Administering questionnaires in VR (inVRQs) is becoming more common in contemporary research. This is based on the intuitive notion that inVRQs may ease participation, reduce the Break in Presence (BIP) and avoid biases. In this paper, we perform a systematic investigation into the effects of interrupting the VR experience through questionnaires using physiological data as a continuous and objective measure of presence. In a user study (n=50), we evaluated question-asking procedures using a VR shooter with two different levels of immersion. The users rated their player experience with a questionnaire either inside or outside of VR. Our results indicate a reduced BIP for the employed inVRQ without affecting the self-reported player experience.

Balancing the options available to players in a way that ensures rich variety and viability is a vital factor for the success of any video game, and particularly competitive multiplayer games. Traditionally, this balancing act requires extensive periods of expert analysis, play testing and debates. While automated gameplay is able to predict outcomes of parameter changes, current approaches mainly rely on heuristic or optimal strategies to generate agent behavior. In this paper, we demonstrate the use of deep player behavior models to represent a player population (n = 213) of the massively multiplayer online role-playing game Aion, which are used, in turn, to generate individual agent behaviors. Results demonstrate significant balance differences in opposing enemy encounters and show how these can be regulated. Moreover, the analytic methods proposed are applied to identify the balance relationships between classes when fighting against each other, reflecting the original developers' design.

Informed by modern imaging techniques, current medical navigation systems support physicians during a variety of interventions, such as needle-based operations. During these, an abundance of information is often displayed on monitors placed in positions that are uncomfortable for the operator to view. In this article, we address these issues with the concept and prototype of a customized flexible display that is placed on the patient’s body to provide the essential information at just the right location. We present an empirical evaluation comparing the flexible display against a control condition using a standard interventional monitor setup and an additional condition that combines both. Our results show that the flexible display significantly reduces task load while improving overall usability. Furthermore, we found indications that the flexible display reduces task completion time while also observing a negative effect on accuracy, which needs to be balanced carefully.

Professionals in domains like film, theater, or architecture often rely on physical models to visualize spaces. With virtual reality (VR) new tools are available providing immersive experiences with correct perceptions of depth and scale. However, these lack the tangibility of physical models. Using tangible objects in VR can close this gap but creates the challenges of producing suitable objects and interacting with them with only the virtual objects visible. This work addresses these challenges by evaluating tangibles with three haptic fidelities: equal disc-shaped tangibles for all virtual objects, Lego-built tangibles, and 3D-printed tangibles resembling the virtual shapes. We present results from a comparative study on immersion, performance, and intuitive interaction and interviews with domain experts. The results show that 3D-printed objects perform best, but Lego offers a good trade-off between fast creation of tangibles and sufficient fidelity. The experts rate our approach as useful and would use all three versions.

Dealing with fear of falling is a challenge in sport climbing. Virtual reality (VR) research suggests that using physical and reality-based interaction increases the presence in VR. In this paper, we present a study that investigates the influence of physical props on presence, stress and anxiety in a VR climbing environment involving whole body movement. To help climbers overcoming fear of falling, we compared three different conditions: Climbing in reality at 10 m height, physical climbing in VR (with props attached to the climbing wall) and virtual climbing in VR using game controllers. From subjective reports and biosignals, our results show that climbing with props in VR increases the anxiety and sense of realism in VR for sport climbing. This suggests that VR in combination with physical props are an effective simulation setup to induce the sense of height.

We propose an approach to facilitate adjustable grip for object interaction in virtual reality. It enables the user to handle objects with loose and firm grip using conventional controllers. Pivotal design properties were identified and evaluated in a qualitative pilot study. Two revised interaction designs with variable grip were compared to the status quo of invariable grip in a quantitative study. The users performed placing actions with all interaction modes. Performance, clutching, task load, and usability were measured. While the handling time increased slightly using variable grip, the usability score was significantly higher. No substantial differences were measured in positioning accuracy. The results lead to the conclusion that variable grip can be useful and improve realism depending on tasks, goals, and user preference.

Needle-guiding templates are used for a variety of minimally invasive medical interventions. While physically supporting needle placement with a grid of holes, they lack integrated information where needles need to be inserted. Physicians must manually determine the correct holes based on the output of planning software - a workflow that is error-prone and lengthy. We address these issues by embedding a display into the template using electroluminescence (EL) screen printing. The EL display is connected to planning software and illuminates the correct hole. In an empirical evaluation with physicians and researchers from the medical domain, we compare the illuminated against the conventional template as used in magnetic resonance imaging (MRI) guided prostate biopsies. Our results show that the EL display significantly improves task completion time by 51%, task load by 47% and usability by 30%.

Room-scale mapping facilitates natural locomotion in virtual reality (VR), but it creates a problem when encountering virtual walls. In traditional video games, player avatars can simply be prevented from moving through walls. This is not possible in VR with room-scale mapping due to the lack of physical boundaries. Game design is either limited by avoiding walls, or the players might ignore them, which endangers the immersion and the overall game experience. To prevent players from walking through walls, we propose a combination of auditory, visual, and vibrotactile feedback for wall collisions. This solution can be implemented with standard game engine features, does not require any additional hardware or sensors, and is independent of game concept and narrative. A between-group study with 46 participants showed that a large majority of players without the feedback did pass through virtual walls, while 87% of the participants with the feedback refrained from walking through walls. The study found no notable differences in game experience.

Enabling multiple-purpose robots to follow textual instructions is an important challenge on the path to automating skill acquisition. In order to contribute to this goal, we work with physical exercise instructions as an everyday activity domain where textual descriptions are usually focused on body movements. Body movements are a common element across a broad range of activities that are of interest for robotic automation. Developing a text-to-animation system, as a first step towards understanding language for machines, is an important task. The process requires natural language understanding (NLU) including non-declarative sentences and the extraction of semantic information from complex syntactic structures with a large number of potential interpretations. Despite a comparatively high density of semantic references to body movements, exercise instructions still contain a large amount of underspecified information. Detecting and bridging or filling such underspecified elements is extremely challenging when relying on methods from NLU alone. Humans, however, can often add such implicit information with ease, due to its embodied nature. We present a process that contains a combination of a semantic parser and a Bayesian network. It explicates the information that is contained in textual movement instructions so that an animation execution of the motion-sequences performed by a virtual humanoid character can be rendered. Human computation is then employed to determine best candidates and to further inform the models in order to increase performance adequacy.

We introduce WatchThru, an interactive method for extended wrist-worn display on commercially-available smartwatches. To address the limited visual and interaction space, WatchThru expands the device into 3D through a transparent display. This enables novel interactions that leverage and extend smartwatch glanceability. We describe three novel interaction techniques, Pop-up Visuals, Second Perspective and Peek-through, and discuss how they can complement interaction on current devices. We also describe two types of prototypes that helped us to explore standalone interactions, as well as, proof-of-concept AR interfaces using our platform.

Landmark-based navigation systems have proven benefits relative to traditional turn-by-turn systems that use street names and distances. However, one obstacle to the implementation of landmark-based navigation systems is the complex challenge of selecting salient local landmarks at each decision point for each user. In this paper, we present Pharos, a novel system that extends turn-by-turn navigation instructions using a single global landmark (e.g. the Eiffel Tower, the Burj Khalifa, municipal TV towers) rather than multiple, hard-to-select local landmarks. We first show that our approach is feasible in a large number of cities around the world through the use of computer vision to select global landmarks. We then present the results of a study demonstrating that by including global landmarks in navigation instructions, users navigate more confidently and build a more accurate mental map of the navigated area than using turn-by-turn instructions.

Matching game difficulty to player ability is a crucial step toward a rewarding player experience, yet making difficulty adjustments that are effective yet unobtrusive can be challenging. This paper examines the impact of automatic and player-initiated difficulty adjustment on player experience through two studies. In the first study, 40 participants played the casual game THYFTHYF either in motion-based or sedentary mode, using menu-based, embedded, or automatic difficulty adjustment. In the second study, we created an adapted version of the commercially available game fl0w to allow us to carry out a more focused study of sedentary casual play. Results from both studies demonstrate that the type of difficulty adjustment has an impact on perceived autonomy, but other player experience measures were not affected as expected. Our findings suggest that most players express a preference for manual difficulty choices, but that overall game experience was not notably impacted by automated difficulty adjustments.

Providing pedestrian navigation instructions on small screens is a challenging task due to limited screen space. As image-based approaches for navigation have been successfully proven to outperform map-based navigation on mobile devices, we propose to bring image-based navigation to smartwatches. We contribute a straightforward pipeline to easily create image-based indoor navigation instructions that allow users to freely navigate in indoor environments without any localization infrastructure and with minimal user input on the smartwatch. In a user study, we show that our approach outperforms the current state-of-the art application in terms of task completion time, perceived task load and perceived usability. In addition, we did not find an indication that there is a need to provide explicit directional instructions for image-based navigation on small screens.

Strong adaptability is a major requirement and challenge in the physiotherapeutic use of motion-based games for health. For adaptation tool development, tablets are a promising platform due to their similarity in affordance compared to traditional clipboards. In a comparative study, we examined three different input modalities on the tablet that allow for configuring joint angles: direct-touch, classic interface components (e.g. buttons and sliders), and a combination of both. While direct touch emerged as the least preferable modality, the results highlight the benefits of the combination of direct-touch and classic interface components as the most accessible modality for configuring joint angle ranges. Furthermore, the importance of configuring joint angles along three distinct axes and the interesting use-case of configuration tools as communication support emerged.

Exergames are increasingly considered as an exercise instruction modality in health applications. Studies are typically conducted in non-situated contexts and capture short-term effects. We present first results from a medium-scale study conducted over the course of 5 weeks and integrated into a normal rehabilitation program. The study features three groups, comparing manually adjustable exergames with the identical games in adaptive versions and manual physiotherapy interventions without games. The results indicate that the exergames and traditional therapy are comparable regarding measures of competence and enjoyment, while exergames led to significantly higher scores for autonomy, presence, and in a functional reach test. With traditional therapy, scores for tension-pressure and effort-importance were significantly higher. The initial results of the broader study presented in this paper deliver insights regarding motivational aspects of exergames and traditional therapy and point out which motivational aspects could be strengthened in future implementations.

Autonomous manipulation robots can be valuable aids as interactive agents in the home, yet it has proven extremely difficult to program their behavior. Imitation learning uses data on human demonstrations to build behavioral models for robots. In order to cover a wide range of action strategies, data from many individuals is needed. Acquiring such large amounts of data can be a challenge. Tools for data capturing in this domain must thus implement a good user experience. We propose to use human computation games in order to gather data on human manual behavior. We demonstrate the idea with a strategy game that is operated via a natural user interface. A comparison between using the game for action execution and demonstrating actions in a virtual environment shows that people interact longer and have a better experience when playing the game.

Map applications for smartwatches present new challenges in cartography, a domain in which large display sizes have significant advantages. In this paper, we introduce StripeMaps, a system that adapts the mobile web design technique of linearization for displaying maps on the small screens of smartwatches. Just as web designers simplify multiple column desktop websites into a single column for easier navigation on mobile devices, StripeMaps transforms any two-dimensional route map into a one-dimensional "stripe". Through a user study, we show that this simplification allows StripeMaps to outperform both traditional mobile map interfaces and turn-by-turn directions for pedestrian navigation using smartwatches. In addition to introducing StripeMaps, this paper also has a secondary contribution. It contains the first empirical comparison of different approaches for pedestrian smartwatch navigation and illuminates their pros and cons.

In this paper we present how to tag images automatically based on the image and sensor data from a mobile device. We developed a system that computes low-level tags using the image itself and meta data. Based on these tags and previous user tags we learn high-level tags. With a client-server-implementation we source out computational expensive algorithms to recommend the tags as fast as possible. We show what are the best feature extraction methods in combination with a machine learning technique to recommend good tags.

In this paper, we present results from an ongoing participatory design research project that focuses on the design and evaluation of a tagging-system and corresponding visualizations to support actors and visitors of an urban gardening project. The contributions of this work are threefold. First, it addresses the yet underexplored field of integrating context information beyond location and time into mobile tagging systems and folksonomies. Second, it suggests novel visualizations to explore tagged data and folksonomies beyond tag clouds and simple map representations on mobile phones. And third, it gives novel insights into supporting the embedded practices of a DIY (Do-It-Yourself) urban gardening community with interactive systems based on ethnographic fieldwork and participatory design.

The partnership of theater and digital media shows much potential for new means of storytelling. Digital scenery can be joined to the performer's action on stage; algorithmic influences can blur the linearity of a drama; interactive technology offers novel means of involving the audience in the creation of the piece. Interaction can thus enhance the dramaturgical possibilities of traditional theater. However, the narrative task also accompanies various new challenges for the designers of such a play. On the basis of our dance performance Parcival XX-XI, we define requirements for making an audience interact in a theatrical play and introduce four interaction-enabling criteria for theatrical performances that use gestural interfaces.

In multiagent-based simulation systems the agent programming paradigm is adopted for simulation. This simulation approach offers the promise to facilitate the design and development of complex simulations, both regarding the distinct simulation actors and the simulation environment itself. We introduce the simulation middleware PlaSMA which extends the JADE agent framework with a simulation control that ensures synchronization and provides a world model based on a formal ontological description of the respective application domain. We illustrate the benefits of an ontology grounding for simulation design and discuss further gains to be expected from recent advances in ontology engineering, namely the adaption of foundational ontologies and modelling-patterns.

Current planning tools for cycling trips do not sufficiently support the often explorative, spontaneous nature of cycling as route creation is not provided in a way suitable for usage on typical mobile devices in situ. In close cooperation with the target group, we developed and evaluated an approach where cycling routes based on selected images of points of interest can be generated on-the-fly.

While studies have shown the advantages of map-image-combinations for pedestrian navigation, none of them con-centrated on interaction. We suggest to combine an intuitive pitch gesture with the natural peephole metaphor not only for pedestrian navigation but also for virtual exploration with mobile devices and present a first prototype implement-ing our ideas.

We present the development and evaluation of an authoring system for image-based pedestrian navigation which lets authors take pictures and annotate instructions on the go in three interface variants. Results indicate that a freehand manner of photograph annotation is fastest, while authors strive toward visually pleasing annotation compositions.

Runtime testing is important for improving the quality of software systems. This fact holds true especially for systems which cannot be completely assembled at development time, such as mobile or ad-hoc systems. The concepts of built-in-test (BIT) can be used to cope with runtime testing, but to our knowledge there does not exist an implemented infrastructure for BIT. The MORABIT project realizes such an infrastructure and extends the BIT concepts to allow for a smooth integration of the testing process and the original business functionality execution. In this paper the requirements on the infrastructure and our solution are presented

Today component- and service-based technologies play a central role in many aspects of enterprise computing. However, although the technologies used to define, implement, and assemble components have improved significantly over recent years, techniques for verifying systems created from them have changed very little. The correctness and reliability of component-based systems are still usually checked using the traditional testing techniques that were in use before components and services became widespread, and the associated costs and overheads still remain high. This paper presents an approach which addresses this problem by making the system verification process more component-oriented. Based on the notion of built-in tests - tests that are packaged and distributed with prefabricated, off-the-shelf components - the approach and supporting infrastructure help to automate some of the testing process, thereby significantly reduces system testing effort. After providing an introduction to the principles behind component-based verification, and explaining the main features of the approach, we show by means of a small example how it can reduce system verification effort

What are the most suitable interaction paradigms for navigational and informative tasks for pedestrians? Is there an influence of social and situational context on multimodal interaction? Our study takes a closer look at a multimodal system on a handheld device that was recently developed as a prototype for mobile navigation assistance. The system allows visitors of a city to navigate, to get information on sights, and to use and manipulate map information. In an outdoor evaluation, we studied the usability of such a system on site. The study yields insight about how multimodality can enhance the usability of hand-held devices with their future services. We show, for example that for our more complicated tasks multimodal interaction is superior to classical unimodal interaction.

This paper presents SmartKom Mobile, the mobile version of the SmartKom system. SmartKom Mobile brings together highly advanced user interaction and mobile computing in a novel way and allows for ubiquitous access to multi-domain information. SmartKom Mobile is device-independent and realizes multi-modal interaction in cars and on mobile devices such as PDAs. With its siblings, SmartKom Home and SmartKom Public, it provides intelligent user interfaces for an extremely broad range of scenarios and environments.

This paper presents an architecture for image analysis that is based on feature hierarchies. The architecture allows for shift, scale and topological invariant detection of objects. Features are efficiently represented and combined dynamically during the detection process. The respective feature detectors are trained using a supervised learning scheme. The method discussed here can also solve the problem of segmenting an image into image regions that correspond to detected features. This segmentation can be done through backtracking of feature information in the feature hierarchy. We applied the method for a set of images where building facades are analyzed and show experimental results that demonstrate the capabilities of the system.

The presentation of information using multiple modalities influences the perception of users, their comfort, and their performance in using a computer-based information system. This paper presents a user study investigating the effects of different output modality-combinations on the effectiveness to transport information and on the user's acceptance of the system. We chose a tourist information system as a test environment and conducted the study on three different devices (PDA, TV set, and desktop computer) to investigate whether the best modality-combination depends on the used device. It turned out that the modality-combination of spoken text in connection with a picture was the most effective regarding recall-performance. This effect was strongest for users working with PDAs, which can be explained by the cognitive load theory. In contrast to this, participants ranked different modality combinations as most appealing, namely those with written text.

Most artificial neural networks used in practical applications are based on simple neuron types in a multi-layer architecture. Here, we propose to solve optimization problems using a fully recurrent network of spiking neurons mimicking the response behavior of biological neurons. Such networks can compute a series of different solutions for a given problem and converge into a periodical sequence of such solutions. The goal of this paper is to prove that neural networks like the SRM (Spike Response Model) are able to solve nonlinear optimization problems. We demonstrate this for the traveling salesman problem. Our network model is able to compute multiple solutions and can use its dynamics to leave local minima in which classical models would be stuck. For adapting the model, we introduce a suitable network architecture and show how to encode the problem directly into the network weights.

Biologically motivated neuronal models have become popular in auto-associative recurrent networks due to their ability to solve the binding problem and to segment complex scenes into previously stored components. Most approaches only use simple Hebbian learning which works best for orthogonal patterns. This paper presents a learning algorithm based on perceptron learning which enhances the storage capability in such neural networks and also allows correlated patterns. As these iterative learning algorithms allow weights to grow arbitrarily, the amount of network input may also grow arbitrarily and can cause desynchronization. We therefore incorporate a method to ensure a constant network input for trained patterns while facilitating the switching from one attractor to a different one when a sequence of patterns is generated.

A new model for chemosensory reception is presented. It models reacti(cid:173) ons between odor molecules and receptor proteins and the activation of second messenger by receptor proteins. The mathematical formulation of the reaction kinetics is transformed into an artificial neural network (ANN). The resulting feed-forward network provides a powerful means for parameter fitting by applying learning algorithms. The weights of the network corresponding to chemical parameters can be trained by presen(cid:173) ting experimental data. We demonstrate the simulation capabilities of the model with experimental data from honey bee chemosensory neurons. It can be shown that our model is sufficient to rebuild the observed data and that simpler models are not able to do this task.