During the design, it is important to evaluate the user experience of representative users in many human product interactions. But, in some cases, it is difficult or even impossible to recruit representative users because they have disabilities that do not allow them to take part in such investigations. Thus, alternative populations are widely studied. The most common way to replace real blind people is to use sighted but blindfolded users when studying design solutions. To test whether such alternative or proxy users can be used to represent blind people in social interactions, we examined the communication quality of 20 blind-sighted pairs and 20 blindfolded-sighted pairs in two different experiments. A prototype named E-Gaze glasses was evaluated as the testing tool. Results clearly show that the blindfolded participants achieved significantly higher communication quality than the blind participants. In qualitative data analysis, the blindfolded participants also reported their user experience of being blindfolded in conversations. Our qualitative results strengthen the conclusion that blindfolded users’ behaviour is different from real blind users’ behaviour. We recommend that blind users should not be substituted for blindfolded users in human product evaluations when communication quality is measured.

As the population ages rapidly, there is a strong focus on the healthy aging of older adults. A central part of healthy aging is keeping people connected in later social life. Exergames are recommended as one of the coping strategies to help improve health and quality of life in older adults. In our study, we developed an exergame called Social Balance Ball to engage older and younger people to play together, encouraging social interaction between generations. From May to July 2021, we evaluated this exergame in Shanghai, China, performing a user experiment with 18 unfamiliar young-old pairs under three test conditions (virtual player, mediated human player, and co-located human player). To evaluate the exergame, our main findings demonstrated that participants felt significantly perceived social interaction in mediated play and co-located play than in virtual play. Overall, older participants perceived significantly higher social interaction than younger participants. In this study, we contribute (1) empirical research findings on how the Social Balance Ball exergame enhances social interaction in generations; (2) design implications for informing future design and development of social exergames.

One of the most effective ways to improve quality of life in dementia is by exposing people to meaningful activities. The study of engagement is crucial to identify which activities are significant for persons with dementia and customize them. Previous work has mainly focused on developing assessment tools and the only available model of engagement for people with dementia focused on factors influencing engagement or influenced by engagement. This article focuses on the internal functioning of engagement and presents the development and testing of a model specifying the components of engagement, their measures, and the relationships they entertain. We collected behavioral and physiological data while participants with dementia (N = 14) were involved in six sessions of play, three of game-based cognitive stimulation and three of robot-based free play. We tested the concurrent validity of the measures employed to gauge engagement and ran factorial analysis and Structural Equation Modeling to determine whether the components of engagement and their relationships were those hypothesized. The model we constructed, which we call the ENGAGE-DEM, achieved excellent goodness of fit and can be considered a scaffold to the development of affective computing frameworks for measuring engagement online and offline, especially in HCI and HRI.

Almost all older people with dementia have progressive communication difficulties, which lead to increased social isolation and negative emotions. Thus, providing communication assistance for them is essential. This paper explores the feasibility of using social robots to assist older people with dementia in their face-to-face communication with others. We designed the behavior of a humanoid Pepper robot and made a Wizard of Oz prototype that the robot can serve as a personal memory assistant. The robot stores personal information for older people and assists in their communication through voice and screen display. In a video-based study with 88 participants, we investigated the effects of this assistive robot from a third-person observer perspective. Data were collected and analyzed using both three-way MANCOVAs for quantitative analysis and conventional content analysis for qualitative data. The results revealed that, by providing memory support, the robot significantly improved the observer's perceptions of an older person with dementia, including her perceived communication ability and performance, and personal image. Meanwhile, the communication is perceived to be significantly more effective when the robot assisted an older person. The willingness of others to communicate with more senior people also increased accordingly. Based on these findings, we present guidelines that may inform the design and development of communication assistant robots for older people with dementia.

This study investigates the effect of peripheral visual and haptic information in providing information about upcoming motions of a fully automated vehicle to an occupant engaging in a non-driving related task, i.e. reading. Two peripheral displays, one visual and the other haptic, were designed and tested. It was hypothesized that the peripheral information would enhance the users’ situation awareness and reduce their mental workload. The study was conducted with 18 participants driven around in a real-road environment in a multi-purpose vehicle that simulated a fully automated vehicle. The peripheral visual and haptic information significantly enhanced situation awareness but did not reduce the mental workload. Implications and future work are discussed.

This paper presents a crowd-sourcing video-based investigation on the users' perception of proactiveness towards an object with artificial eyes through winking. Based on both the session initiation protocol (SIP) and the perceptual crossing paradigm, we synthesized a set of minimally-designed video clips and tested them in Amazon Mechanical Turk with 240 participants. The results show that, in both single- or multi-user scenarios, winking can be a useful expression that makes the user view the object as being proactive and encourages reciprocal input. We also discuss how the object-initiated interaction extends the perceptual crossing paradigm in human-object communication.

As developments in artificial intelligence and robotics progress, more tasks arise in which humans and robots need to collaborate. With changing levels of complementarity in their capabilities, leadership roles will constantly shift. The research presented explores how people adapt their behavior to initiate or accommodate continuous leadership shifts in human-robot collaboration and how this influences trust and understanding. We conducted an experiment in which participants were confronted with seemingly conflicting interests between robot and human in a collaborative task. This was embedded in a physical navigation task with a robot on a leash, inspired by the interaction between guide dogs and blind people. Explicit and implicit feedback factors from the task and the robot partner proved to trigger humans to reconsider when to lead and when to follow, while the outcome of this differed across participants. Overall the participants evaluated the collaboration more positively over time, while participants who took the lead more often valued the collaboration more negatively than other participants.

Eye contact is crucial in social interactions, linking with sincerity and friendliness. However, blind people cannot see and make eye contact when they communicate with sighted people. It influences the involvement of blind people in blind-sighted conversations. Based on this context, we implemented Social glasses with an eye-tracking system, aiming to improve the communication quality between blind and sighted people in face-to-face conversations. Social glasses attempts to simulate the appropriate gaze for blind people, especially establishing the “eye contact” in blind-sighted conversations. To evaluate the impact of the interactive gaze displayed on the Social glasses, we performed dyadic-conversation tests under four experimental conditions (No Gaze, Constant Gaze, Random Gaze, and Interactive Gaze) for 40 participants. Quantitative results showed that the Interactive gaze has a positive impact on improving the communication quality between blind and sighted people, which were consistent with a qualitative analysis of the participants’ comments.

Driving is a social activity: Drivers need to coordinate and cooperate with each other to share the infrastructure. The relationship between drivers influences their driving behavior and experience. Lights, horn and speed are the most frequently used means to exchange information, limiting both the range and the bandwidth of the connectivity and leading to isolation, loneliness, and competition. We present “iSticker” and “MusicHound”, two concepts that aim to establish a connection by presenting similarity information between drivers. The two concepts were prototyped and evaluated with users in a driving simulator. The results showed that iSticker and MusicHound enhance drivers’ social closeness with each other and belongingness during the journey.

The wellbeing of people with dementia in long-term care facilities is hindered, as they spend most of their time alone with little engagement in meaningful activities and an absence of pleasant sensory stimulation. We designed an interactive system called LiveNature that adopts a novel combined approach involving an ambient display unit and an interactive robotic sheep, to offer long-term access and to engage people with dementia in long-term care facilities in rewarding experiences. LiveNature aims to provide holistic multi-sensory engagement to provoke positive emotions, increase social bonding, and restore attentiveness and communication. The design was implemented within a Dutch nursing home. An evaluation of the user experience and the effectiveness of the design was conducted in a real-life setting with nine participants, five family members, two caregivers and four volunteers, using observational rating scales and semi-structured interviews. The results of the rating scales and the findings from qualitative data showed evidence of enhanced positive engagement.

This study examines patterns of coordinated gaze between a child and a robot (NAO) during a card matching game, `Memory'. Dyadic gaze behavior like mutual gaze, gaze following and joint attention are indications both of child's engagement with the robot and of the quality of child-robot interaction. Eighteen children interacted with a robot tutor in two settings. In the first setting, the robot tutor gave clues to assist children in finding the matching cards, and in the other setting, the robot tutor only looked at the participants during the play. We investigated the coordination between child and robots' gaze behaviors. We found that more occurrences of mutual gaze and gaze following made the children aware of the gaze hints given by the robot and improved the efficacy of the robot tutor as a helping agent. This study, therefore, provides guidelines for gaze behaviors design to enrich child-robot interaction in a tutoring context.

To understand the behavior of moving entities in a given environment, one should be capable of predicting their motion, that is, to model their dynamics. In a setting where different behaviors can arise, one can assume that each of them corresponds to different motivational states of observed entities. Here, those motivations are understood as goal positions or spots where entities seek to arrive. To build prediction models based on that idea, we present an unsupervised method to estimate motivational spots actively. Additionally, we use the output of such process to refine an adaptive system modeling the dynamics of inferred hidden causes of observed data. The whole method uses deep variational methods, and particularly, the network estimating motivations is trained through dynamic programming. Results show that modeling the dynamics of entities can be better achieved by integrating information about motivational spots. Notably, a network modeling the dynamics converges faster through the incorporation of information about motivations.

This paper presents a method for observational learning in autonomous agents. A formalism based on deep learning implementations of variational methods and Bayesian filtering theory is presented. It is explained how the proposed method is capable of modeling the environment to mimic behaviors in an observed interaction by building internal representations and discovering temporal and causal relations. The method is evaluated in a typical surveillance scenario, i.e., perimeter monitoring. It is shown that the vehicle learns how to drive itself by simultaneously observing its surroundings and the actions taken by a human driver for a given task. That is achieved by embedding knowledge regarding perception-action couplings in dynamic representational states used to produce action flows. Thereby, representations link sensory data to control signals. In particular, the representational states associate visual features to stable action concepts such as turning or going straight.

It is not difficult to design an object that is sensitive to our gaze. The challenging part is to make the user realize it. And since people are not used to interacting with an object by simply moving their eyes (people use eyes to see, not to trigger something), the interaction itself become unfamiliar to them. Based on the gaze behavior of people socializing with others, we believe that the feeling of being looked back when interacting with a gaze sensitive object is rather crucial in order to overcome the problem of unfamiliarity and to make people naturally realize that the object was sensitive to their gaze behavior. To achieve this feeling, we conclude that eyes need to be presence in the user's view. In this paper, we present an anthropomorphize coffee machine called the CoffeePet, attached with two, small OLED screen that displays animated eyes. These eyes are responsive towards the user's gaze behavior. Furthermore, the CoffeePet will automatically start to brew the drink if the user manages to maintain a prolonged eye contact with it. In three experiments, we investigated the impact of the animated eyes in aiding the participants to become familiar and to realize that their gaze behavior influences the CoffeePet to react. Without being told on how to interact with the CoffeePet, participants were randomly assigned to participate in one of three conditions. 1) CoffeePet with watching eyes (eyes with direct gaze), 2) CoffeePet with interactive gaze model, and 3) CoffeePet with interactive gaze following. The results showed that the interactive sharing gaze did, in fact, lead the participants to become familiar and to realize that they can interact with the object by simply moving their eyes.

The feeling of being looked back when we look at someone and that someone is also aware that we are looking at him/her is a basic fundamental during social interaction. This situation can only occur if both realize the presence of each other. Based on these theories, this research is motivated in exploiting the possibility of designing for a gaze sensitive object - how people can relate to object by depending on their eyes only. In this paper, we present a gaze-activated coffee machine called the CoffeePet attached with two small, OLED screen that will displays animated eyes. These eyes are responsive towards the user's gaze behavior. Furthermore, we used a sensor module (HVC Omron) to detect and track the eyes of a user in real time. It gives the ability for the user to interact with the CoffeePet simply by moving their eyes. The CoffeePet is also able to automatically brew and pour the coffee out of its spout if it feels appropriate during the interaction. We further explain the description of the system, modification of the real product, and the experimental plan to compare the user's perception of the CoffeePet's eyes and to investigate whether the user realizes or not that their gaze behavior influences the CoffeePet to react.

This paper presents a study that analyzes the effects of robots' gaze hints on children's performance in a card-matching game. We conducted a within-subjects study, in which children played a card matching game "Memory" in the presence of a robot tutor in two sessions. In one session, the robot gave hints to help the child find matching cards by looking at the correct match and, in the other session, the robot only looked at the child and did not give them any help. Children performance was measured using the number of tries and overall time used to complete the game. Our findings show that the use of gaze hints (help condition) made the matching task significantly easier and that children used significantly fewer tries than without help.

With the growing availability of wearable technology, video recording devices have become so intimately tied to individuals, that they are able to record the movements of users' hands, making hand-based applications one the most explored area in First Person Vision (FPV). In particular, hand pose recognition plays a fundamental role in tasks such as gesture and activity recognition, which in turn represent the base for developing human-machine interfaces or augmented reality applications. In this work we propose a graph-based representation of hands seen from the point of view of the user, obtained through the shape-fitting capability of a modified Instantaneous Topological Map. Spectral analysis of the graph Laplacian allows to arrange eigenvalues in vectors of features, which prove to be discriminative in classifying the considered hand poses.

The study of engagement is central to improve the quality of care and provide people with dementia with meaningful activities. Current assessment techniques of engagement for people with dementia rely exclusively on behavior observation. However, novel unobtrusive sensing technologies, capable of tracking psychological states during activities, can provide us with a deeper layer of knowledge about engagement. We compared the engagement of persons with dementia involved in two playful activities, a game-based cognitive stimulation and a robot-based free play, using observational rating scales and electrodermal activity (EDA). Results highlight significant differences in observational rating scales and EDA between the two activities and several significant correlations between the items of observational rating scales of engagement and affect, and EDA features.

Driving is a social activity, and therefore driving is not only a matter of skills but also of emotion. Numerous studies show that aggressive driving makes a significant contribution to traffic accident involvement. In previous research, a concept based on Driver to Driver communication employing location-based services was proposed that enables road users to express their disapproval and appreciation of others' driving behavior. In the current study, a complete prototype based on this concept, which enables participants to send and receive feedback while driving and review their behavior afterwards, was developed. The acceptance and influence of driving behaviour of this concept were investigated in a driving simulator. It was found that the system positively influenced people's driving behavior and was accepted by most participants.

This paper describes a new approach in exploring preferred driving styles for autonomous vehicles through simulation of autonomous driving in real road conditions. A Wizard experiment with an equipped car was conducted to investigate the preferences of people with different driving styles, assertive and defensive, for three autonomous vehicle driving styles (defensive, assertive and light rail transit), inducing different acceleration forces, at three different road profiles. Subjective and objective measurements were collected. The results show that the defensive driving style was preferred and there were variations between participants related to their own driving style. The results indicate that the preferences of assertive drivers for the driving style of an autonomous vehicle may not match their own driving style. Yet, users of future autonomous vehicles should be able to indicate and adjust the driving behaviour of an autonomous vehicle to their own preferences in order to maximize comfort in travelling experience.

Eye gaze plays an essential role in social interaction which influences our perception of others. It is most likely that we can perceive the existence of another intentional subject through the act of cathing one another's eyes. Based on the notion of perceptual crossing, we aim to establish a meaningful social interaction that emerges out of the perceptual crossing between a person and an everyday object by exploiting the gazing behavior of the person as the input modality for the system. We investigated in literature the experiments that adopt the perceptual crossing as their foundation, lessons learned from literature were used as input for a concept to create meaningful social interaction. We used an eye-tracker to measure gaze behavior that allows the participant to interact with the object by using their eyes through active exploration. It creates a situation where both of them mutually becoming aware of each other's existence. Further, we discuss the motivation for this research, present a preliminary experiment that influences our decision and our directions for future work.

Our eye gaze is important during social interactions. It can generate significant social cues in nonverbal communication. The feeling of being look back when we are gazing at someone influences our social behaviour. In this paper, we propose an interactive coffee cup that is responsive whenever a person is fixating on it. Taking the user's gazing behaviour as our system input modality, we want to create an environment where a person may establish social interaction with an everyday object whenever he/she is looking at it. To make an object visible to the user's eyes and for the user to feel connected with the object, it is expected that the object to possess distinctive characteristics that can acknowledge the user that it is aware of being look at. By combining the recent technology of eye trackers, mechanical design and embedded electronics, we want to explore the possibility of nonverbal social interaction between a person and inanimate object that will respond when a person is looking at it to allow social interaction and to create a sense of emotional bond between the two.

Gaze is an important nonverbal cue in human - human communication, for example, in communicating direction of attention. Therefore, presumably being able to understand and provide gaze cues is an important aspect in robot's interactive behavior. While there is considerable progress, as regards the design of social gaze cues for robots, there is little that has been done to examine the ability of humans to read and accept help signals from a robot's gaze. In this study, we examine how people perceive gaze cues and head angles directed towards different target positions on a table when human and NAO robot are sitting against each other as in board game scenarios. From the results, we show that when the head pitch angle is higher (24±2) and the depth is less, approximately 20 cm from the robot, participants detected the positions with good accuracy. Unexpectedly, the locations on the left of the robot were detected with lower accuracy. In conclusion, we discuss the implications of this research for design of interaction settings between human and a robot that is intended for social and educational support.

In face-to-face communication, eye gaze is integral to a conversation to supplement verbal language. The sighted often uses eye gaze to convey nonverbal information in social interactions, which a blind conversation partner cannot access and react to them. In this paper, we present E-Gaze glasses (E-Gaze), an assistive device based on an eye tracking system. It simulates gaze for the blind person to react and engage the sighted in face-to-face conversations. It is designed based on a model that combines eye-contact mechanism and turn-taking strategy. We further propose an experimental design to test the E-Gaze and hypothesize that the model-driven gaze simulation can enhance the conversation quality between the sighted and the blind person in face-to-face communication.

Gaze and eye contact are frequently in social occasions used among sighted people. Gaze is considered as a predictor of attention and engagement between interlocutors in conversations. However, gaze signals from the sighted are not accessible for the blind person in face-to-face communication. In this paper, we present functional work-in-progress prototype, E-Gaze glasses, an assistive device based on an eye tracking system. E-Gaze simulates natural gaze for blind people, especially establishing the "eye contact" between blind and sighted people to enhance their engagement in face-to-face conversations. The gaze behavior is designed based on a turn-taking model, which interprets the corresponding relationship between the conclusive gaze behavior and the interlocutors' conversation flow.

Gaze signals, frequently used by the sighted in social interactions as visual cues, are hardly accessible for low-vision and blind people. A concept is proposed to help the blind people access and react to gaze signals in face-to-face communication. 20 blind and low-vision participants were interviewed to discuss the features of this concept. One feature of the concept is further developed into a prototype, namely Tactile Band, to aim at testing the hypothesis that tactile feedback can enable the blind person to feel attention (gaze signals) from the sighted, enhancing the level of engagement in face-to-face communication. We tested our hypothesis with 30 participants with a face-to-face conversation scenario, in which the blindfolded and the sighted participants talked about a given daily topic. Comments from the participants and the reflection on the experiment provided useful insights for improvements and further research.

This paper presents a Learning Analytics approach for understanding the learning behavior of students while interacting with Technology Enhanced Learning tools. In this work we show that it is possible to gain insight into the learning processes of students from their interaction data. We base our study on data collected through six laboratory sessions where first-year students of Computer Engineering at the University of Genoa were using a digital electronics simulator. We exploit Process Mining methods to investigate and compare the learning processes of students. For this purpose, we measure the understandability of their process models through a complexity metric. Then we compare the various clusters of students based on their academic achievements. The results show that the measured complexity has positive correlation with the final grades of students and negative correlation with the difficulty of the laboratory sessions. Consequently, complexity of process models can be used as an indicator of variations of student learning paths.

Gaze signals are frequently used by the sighted in social interactions as visual cues. However, these signals and cues are hardly accessible for people with visual disability. A conceptual design of E-Gaze glasses is proposed, assistive to create gaze communication between blind and sighted people in face-to-face conversations. We interviewed 20 totally blind and low vision participants to envision the use of the E-Gaze. We explained four features of E-Gaze to participants using persona and use scenarios. Participants discussed the features on their usefulness, efficiency and interest. The results helped us clarify the design direction and further research.

Producing educational serious games can be costly and time-consuming. The Service-Oriented Architecture (SOA) approach of software development can offer a solution to reduce costs and foment serious games development. In this work, we apply a model called Activity Theory-based Model of Serious Games (ATMSG) for identifying existing relevant components that can be reused for different educational serious games. We apply the derived structure to classify the elements of an existing game and to identify how it can be refactored and expanded following the SOA paradigm.

Classifying frames, or parts of them, is a common way of carrying out detection tasks in computer vision. However, frame by frame classification suffers from sudden significant variations in image texture, colour and luminosity, resulting in noise in the extracted features and consequently in the decisions taken. Support Vector Machines have been widely validated as powerful tools for frame by frame detection of non-separable datasets, but are extremely sensitive to these variations between adjacent frames, creating as consequence sudden flickering in the classification results. This work proposes a Dynamic Bayesian Network to smooth the classification results of Support Vector Machines (SVM) in detection tasks. The method is evaluated in First Person Vision (FPV) videos, where a SVM is used to decide whether or not the user's hands are in his field of view.

In the past decade, research on human–computer interaction has embraced psychophysiological user interfaces that enhance awareness of computers about conscious cognitive and affective states of users and increase their adaptive capabilities. Still, human experience is not limited to the levels of cognition and affect but extends further into the realm of universal instincts and innate behaviours that form the collective unconscious. Patterns of instinctual traits shape archetypes that represent images of the unconscious. This study investigated whether seven various archetypal experiences of users lead to recognisable patterns of physiological responses. More specifically, the potential of predicting the archetypal experiences by a computer from physiological data collected with wearable sensors was evaluated. The subjects were stimulated to feel the archetypal experiences and conscious emotions by means of film clips. The physiological data included measurements of cardiovascular and electrodermal activities. Statistical analysis indicated a significant relationship between the archetypes portrayed in the videos and the physiological responses. Data mining methods enabled us to create between-subject prediction models that were capable of classifying four archetypes with an accuracy of up to 57.1%. Further analysis suggested that classification performance could be improved up to 70.3% in the case of seven archetypes by using within-subject models.

The emergence of new wearable technologies, such as action cameras and smart glasses, has increased the interest of computer vision scientists in the first person perspective. Nowadays, this field is attracting attention and investments of companies aiming to develop commercial devices with first person vision (FPV) recording capabilities. Due to this interest, an increasing demand of methods to process these videos, possibly in real time, is expected. The current approaches present a particular combinations of different image features and quantitative methods to accomplish specific objectives like object detection, activity recognition, user-machine interaction, and so on. This paper summarizes the evolution of the state of the art in FPV video analysis between 1997 and 2014, highlighting, among others, the most commonly used features, methods, challenges, and opportunities within the field.

This paper proposes a method for interpretation of the emotions detected in facial expressions in the context of the events that cause them. The method was developed to analyze the video recordings of facial expressions depicted during a collaborative game played as a part of the Mars-500 experiment. In this experiment, six astronauts were isolated for 520 days in a space station to simulate a flight to Mars. Seven time-dependent components of facial expressions were extracted from the video recordings of the experiment. To interpret these dynamic components, we proposed a mathematical model of emotional events. Genetic programming was used to find the locations, types, and intensities of the emotional events as well as the way the recorded facial expressions represented reactions to them. By classification of different statistical properties of the data, we found that there are significant relations between the facial expressions of different crew members and a memory effect between the collective emotional states of the crew members. The model of emotional events was validated on previously unseen video recordings of the astronauts. We demonstrated that both genetic search and optimization of the parameters improve the accuracy of the proposed model. This method is a step toward automating the analysis of affective expressions in terms of the cognitive appraisal theory of emotion, which relies on the dependence of the expressed emotion on the causing event.

Abstract: Instruction in physics aims at achieving two goals: the acquisition of body of knowledge and problem solving skills in physics. This requires students to connect physical phenomena, physics principles, and physics sym-bols. Computer simulation provides students with graphical model that unites phenomenon and principles in physics. However, such minimally guided approach may harm learning since it overburdens the working mem-ory. Also, simulation is inadequate in promoting problem solving skills since students need to exercise with a variety of physics problems. Intelligent tutoring systems (ITS), in contrast, train students in solving physics problems. In this paper, we designed an online puzzle game in physics that combines simulation and pseudo-tutor (namely QTut). We addressed three challenges: extensibility, scalability, and reusability in designing our game. We conducted usability tests with 10 participants on the game prototype to study the user performances and perceptions for improvement. The results indicate the game as educative and moderately entertaining. The use of scaffolding in the game positively contributed to the game learning experience. Moreover, the game GUI expressed information well that made the game understandable, even with little instructions. 1

In this paper, we describe our ongoing work towards understanding how interactive comics could affect time perception. An interactive comic story, The Dreaming Wine, has been created based on our current understanding of interactive comics, time perception and the relation between both. As a starting point for our research, we conducted an experiment on whether the amount of panels would affect the time perception of the comic reader. A list of principles was applied in splitting comic panels. The findings of the experiment are expected to provide a better understanding of how panels in interactive comics have an effect on the reader’s time perception.

Detection of emotions of a crowd is a new research area, which has never, to our knowledge, been accounted for research in previous literature. A bio-inspired model for representation of emotional patterns in crowds has been demonstrated. Emotions have been defined as evolving patterns as part of a dynamic pattern of events. This model has been developed to detect emotions of a crowd based on the knowledge from a learned context, psychology and experience of people in crowd management. The emotions of multiple people making a crowd in any surveillance environment are estimated by sensors signals such as a camera and are being tracked and their behavior is modeled using bio-inspired dynamic model. The behavior changes correspond to changes in emotions. The proposed algorithm involves the probabilistic signal processing modelling techniques for analysis of different types of behavior, interaction detection and estimation of emotions. The emotions are recognized by the expectation of temporal occurrences of causal events modeled by Gaussian mixture model. The model has been evaluated using the simulated behavioral model of a crowd.

Lifelogging tools aim to precisely capture daily experiences of people from the first-person perspective. Although there have been numerous lifelogging tools developed for users to record the external environment around them, the internal part of experience characterized by emotions seems to be neglected in the lifelogging field. However, the internal experiences of people are important and, therefore, lifelogging tools should be able to capture not only the environmental data, but also emotional experiences, thereby providing a more complete archive of past events. Moreover, there are implicit emotions that cannot be consciously experienced, but still influence human behaviors and memories. It has been proven that conscious emotions can be recognized from physiological signals of the human body. This fact may be used to enhance life-logs with information about unconscious emotions, which otherwise would remain hidden. On the other hand, it is not clear if unconscious emotions can be recognized from physiological signals and differentiated from conscious emotions. Therefore, an experiment was designed to elicit emotions (both conscious and unconscious) with visual and auditory stimuli and to record cardiovascular responses of 34 participants. The experimental results showed that heart rate responses to the presentation of the stimuli are unique for every category of the emotional stimuli and allow differentiation between various emotional experiences of the participants.

Immersive mixed reality environments offer new possibilities to reproduce reality or embodied presence with constructing elaborate fantasy worlds and provide to the user an intense and seemingly real experience. Mixed reality gives possibilities to create deeply dimensional narratives and simulations that put the user in the center of the action. In this work we describe an interactive mixed reality installation named Alice, consisting of six separate stages based on the narrative “Alice's Adventures in Wonderland”. To be able to achieve the intended experience we have to build complex and heterogeneous distributed system, composed of sensors, actuators, virtual reality, application components and variety of processing components that manage the flow of context information between the sensors/actuators and applications.

This article describes research into online electronic payment systems, focusing on the aspects of payment systems that are critical for their acceptance by end users. Based on our earlier research and a diary study of payments with an online payment system and with online banking systems of a reputable bank, we proposed a set of 12 interaction design guidelines. The guidelines have been applied during the implementation and redesign of a new payment system. An extensive experimental comparison of the original version of the system with the one designed according to the design guidelines has provided confirmation of the relevance and adequacy of these guidelines for designing online payment systems.

Even in the current computer age, there are still many important application areas, such as early architectural design, where traditional tools like sketching on paper continue to be preferred by many professionals over computer-based tools. There is a growing awareness that there are often very good grounds for this preference. Hence, instead of trying to replace such traditional ways of working, it is now often considered more opportune to try and preserve the strengths of these traditional ways of working, while at the same time improving them by providing access to new media. This is one of the main objectives of the augmented reality approach that we adopt here. In this paper, we specifically discuss the realization of a tool for early architectural design on an existing augmented reality system, called the Visual Interaction Platform. We describe the development process, the resulting tool and its performance for elementary tasks such as positioning and overdrawing. We also identify directions for future research and applications.

An experiment was carried out to estimate the effect of auditory alarms on the work of a plant operator in the context of a computer simulation. The process simulator was implemented so that each of eight machines (computer numeric controlled [CNC] robots) produced sounds to indicate its status over time. Each sound was designed to reflect the ‘real-world’ semantic of the actual breakdown event. As many as 32 different auditory alarms plus six normal machine sounds could be played at once. We attempted to design the auditory alarms so that none would be masked (rendered inaudible) by other auditory alarms. Eight students of computer science operated our process simulation program of an assembly line with the eight CNC robots. Relevant information of disturbances and machine breakdowns was given in a visual (test condition 1), and in visual and auditory form (test condition 2). The results indicate that the additional feedback of auditory alarms significantly improves operator performance and increases some mood aspects positively.

This paper describes work in progress on automatic generation of “impact sounds” based on purely physical modelling. These sounds can be used as non-speech audio presentation of objects and as interaction mechanisms to non visual interfaces. Different approaches for synthesizing impact sounds, the process of recording impact sounds and the analysis of impact sounds are introduced. A physical model for describing impact sounds “spherical objects hitting flat plates or beams” is presented. Some examples of impact sounds generated by mentioned physical model and comparison of spectra of real recorded sounds and model generated impact sounds (generated via physical modelling) are discussed. The objective of this research project (joint project University of Zurich and Swiss Federal Institute of Technology) is to develop a concept, methods and a prototype for an audio framework. This audio framework shall describe sounds on a highly abstract semantic level. Every sound is to be described as the result of one or several interactions between one or several objects at a certain place and in a certain environment.

The current state of traditional software development is surveyed and essential problems are investigated on the basis of system-theoretical considerations. The concept of the optimisation cycle is presented. The relationship between several different kinds of local optimisation cycles with known techniques for user participation is discussed and integrated into a general concept of participatory software development. The pros and cons of essential problems known to obstruct optimal software development and possible ways of solving them are considered.>

Work psychologists have introduced a concept, in which dialog design is a part of task design. A set of criteria of user‐oriented dialog design is presented here. These criteria are consistently integrated into a control concept. Empirical investigations on some of these criteria are then described. In an experiment to prove the criterion transparency, a desktop interface (high transparency) and a conventional menu selection interface (low transparency) were compared. The main result is the clear superiority of the user interface with direct manipulation over the conventional user interface with menu selection. Support is another of the criteria of user‐oriented dialog design that was empirically investigated. The advantages of goal‐ and task‐oriented help messages are also described here. To make the criterion flexibility and the criterion user‐definability practicable, the implementation of a dialog handler for user‐tailorable systems is introduced. The criterion participation was investigated in field studies. It will be shown that in projects with active participation the costs were exceeded to a lesser degree than in projects with passive participation and to a much lesser degree than in projects without participation.