This study investigated the relationship between trust in automation, gaze behavior, and driving performance in beginner and experienced drivers during a simulated driving session. Twenty participants completed a 17-minute drive across three conditions: manual driving, non-critical automated driving, and critical automated driving, with a non-driving-related task (NDRT) introduced between conditions to assess visual attention. Driving performance was evaluated using the Standard Deviation of Lateral Position (SDLP), and eye-tracking data in terms of mean gaze duration (MGD). While both groups demonstrated increased trust in the automated system post-session, beginners showed greater lateral position variability in critical conditions, suggesting over-reliance on automation. Eye-tracking analysis revealed significant changes in glance behavior across driving conditions, particularly in response to critical events. These findings highlight how driver experience shapes interactions with automated systems, emphasizing the importance of trust calibration in automated driving scenarios.

Emotionally overheated social media discussions cause many harms both to the individuals participating in the discussion and to the wider society. To support emotion regulation in social media discussions, the strategy of affect labelling is explored. Two experiments were run to study the effects of emotion labels presented with social media messages on participants’ emotional responses to the messages. Altogether 84 participants read emotional social media messages presented without and with emotion labels (Experiment 1, N = 39), and without labels and with emotion / control labels (Experiment 2, N = 45) and rated their subjective emotional valence. Results show that the ratings of valence were significantly higher for the positive messages presented with label than for the messages presented as such without emotion label or presented with a control label. In addition, the results of the second experiment show that the negative social media messages were experienced as more negative when the messages were presented with an emotion label than without emotion label or with control label. The results suggest that affect labelling of social media messages is an ineffective strategy to down-regulate emotions but could be used to support and strengthen positive emotions.

Uncivil commenting on online news articles tends to harm journalists, news readers, and public discourse. Recent HCI research proposes that such behaviour could be mitigated by alerts (e.g. instructional pop-up notifications) to users while writing seemingly impolite messages. However, the design space of such alerts and the contextual factors influencing users’ perceptions of them remain largely unexplored. Informed by principles of behaviour design, we examined the impact of three factors on the perceived quality and expected effectiveness of alerts: the level of politeness of the comment that the alerted user was responding to and the form and amount of guidance in alert designs. We used a 2 × 2 × 3 factorial quasi-experiment, implemented as an international online survey (N = 248) targeting online news commenters. The results showed that the politeness of the recipient impacted how the comment was perceived but had no effect on the evaluation of the alert. Additionally, alerts with behavioural guidance were preferred over those without. As for effects of background variables, participants’ commenting frequency was not found to influence alert scoring. This study expands our understanding of the design space for user interface alerts and the contextual factors that influence their perceived quality.

This study represents the first exploration of active gaming based on mid-air gestural interaction with a fogscreen, focusing on its feasibility for children with attention deficits. The aim was to evaluate the practicality and effectiveness of gestural interaction in two interaction scenarios involving translation and selection of objects projected on the fog, through the assessment of interaction performance and subjective opinions. The proposed approach was investigated in a custom-designed puzzle-based game Tangram. Eighteen novice participants, namely, children with confirmed attention deficits or attention-deficit/hyperactivity disorder (ADHD) played the game collaboratively in pairs during six sessions. Our results demonstrated that gestural interaction with the fogscreen in the Tangram game was not only enjoyable and engaging but also facilitated effective task completion, as shown by an 81% game success rate. During object translation, the participants typically needed two to three attempts to accurately relocate a puzzle piece, with 42% of translation movements adjusting the pieces’ placements. Further, in object selection, it was underlined the importance of visual feedback corresponding to the user’s hand position for effective interaction. These insights underscore the potential implications of gestural interfaces for fogscreen interaction in active gaming for children.

This study examined the potential of mid-air gestural interfaces specifically designed for children, in the context of full-body motion control for confirmation actions while interacting with large conventional displays. The study aimed to investigate gestures for confirmation actions that were feasible for children, including children with challenges in behavior control and impulse inhibition. Two scenarios were explored, namely, active full-body gesturing (i.e., jumps and jumps with hands up) and freezing (i.e., motionless posture) to identify their usability for confirmation actions. Experiment I involved 11 typically developing children playing the game to check the system’s general functionality and robustness. Experiment II involved 18 children with attention deficit/hyperactivity disorder (ADHD) to evaluate a prolonged usage of the confirmation gestures in six game sessions. The results showed that the children were able to interact with the large conventional displays with consistent accuracy and responsiveness in both active gesturing and freezing scenarios. Moreover, jumps with rising hands were significantly preferred as an interaction input method. Further, there was no significant difference in ratings of the difficulty of freezing gesture, and children successfully repeated the predefined postures. The findings provided insights into the functionality and suitability of mid-air gestural interfaces for confirmation actions in interactive applications targeted to children.

While HCI literature offers general frameworks for understanding user-centred quality, specific application areas may call for more detailed contextualisation of it. This paper focuses on socio-technical context of online news commenting by investigating speculative UI interventions intended to influence users’ emotions and social behaviour. To understand the aspects of quality that matter to users in such UI interventions, we conducted an international online survey (N = 439) and qualitatively analysed respondents’ first impressions of eight different design proposals. The findings describe contextually relevant socio-technical viewpoints and offer actionable considerations for design. For example, the findings imply that designers should be mindful of possible unintentional misuse that may result from the UI reinforcing specific emotional states or affording stigmatisation of individual users. The study advances understanding of which aspects of quality should be considered when designing and deploying UI interventions for digital media services and evaluating them with potential end-users.

Abstract An online experiment investigated the perceived effects of a user interface (UI) intervention aiming to support online news commenters' emotion regulation. By describing the comment's tone to the user, the expected effect was activation of the implicit emotion regulation process of affect labeling (i.e. naming emotions). The perceived emotion- and behavior-related effects of the labeling intervention were investigated using the experimental vignette methodology. Participants read a vignette describing the behavior of an uncivil commenter and assessed the commenter's probable responses to the labeling intervention or a control intervention shown in the UI. The results showed that, when compared to a control condition, the labeling intervention was assessed to evoke positive emotions and to result in mitigation of uncivil behavior. This suggests that UI solutions that support emotion regulation are a promising approach to reducing uncivil comments that users might afterward regret, and hence potentially improving the quality of online discussions.

On digital media services, uncivil commenting is a persistent issue causing negative emotional reactions. One enabler for such problematic behavior is the user interface, conditioning, and structuring text-based communication online. However, the specific roles and influences of UIs are little understood, which calls for critical analysis of the current UI solutions as well as speculative exploration of alternative designs. This paper reports a research-through-design study on the problematic phenomenon regarding uncivil and inconsiderate commenting on online news, envisioning unconventional solutions with a critical voice. We unpack this problem area and outline critical perspectives to possible solutions by describing and analyzing four designs that propose to support emotion regulation by facilitating self-reflection. The design choices are further discussed in respect to interviews of ten news media experts. The findings are reflected against the question of how can critique meaningfully manifest in this challenging problem area.

The present aim was to investigate if angry, neutral, and happy facial expressions have different effects on haptic responses. Two experiments were conducted. In Experiment 1, participants (N = 24) were to respond as fast as they could to images of angry, neutral, and happy faces using a haptic device that measured the amount of applied force in newton units and the duration of touch in milliseconds. Experiment 2 was identical to Experiment 1, except that ratings of emotion-related experiences evoked by the stimuli were also collected in terms of valence and arousal. The results from both studies showed that the force of touch in response to angry expressions was significantly stronger than in response to happy expressions. The ratings of valence showed that angry, neutral, and happy faces were experienced statistically significantly different from each other along the dimension of valence. Ratings of arousal showed that angry faces were experienced as significantly more arousing than neutral and happy faces. The results suggest a relationship between the processing of emotion-related facial expressions and haptic response systems. The force of touch could be used as one of the measures conveying information about the user’s state of mind when interacting with haptic user interfaces.

The present aim was to investigate emotion-related physiological responses and subjective ratings of two groups of active gamers (N = 24) in response to both playing and watching a video of a first-person shooter game. Participants of one group had high preferences for game dynamics in first-person shooter games, whereas the other group disliked such dynamics. Electrodermal activity (EDA), heart rate (HR), and electromyographic (EMG) activity of the corrugator supercilii (i.e., brow furrowing) and the zygomaticus major (i.e., smiling) muscles were measured while playing and watching a gameplay video. After the playing and watching sessions, the participants rated their experienced level of valence and arousal. The results showed that those who liked the game dynamics showed comparable and stable levels of EDA and HR during both playing and watching. Those who disliked the game dynamics showed overall higher levels of EDA and HR during playing than watching a video, and a rising EDA tendency especially during watching a video. Playing evoked overall higher corrugator supercilii activity than watching in both groups. The group that liked the game dynamics showed a steep EMG increase in the activity of the corrugator supercilii, whereas the group that disliked the game dynamics showed less EMG increase. As for ratings of valence and arousal, both groups reported more positive valence and higher arousal after playing than after watching a video, and there were no differences between the groups. In sum, the results showed that player preferences were associated with players’ emotion-related physiological responses. The results also showed that playing as opposed to watching generated higher autonomic arousal, but only for players who disliked the dynamics of the game.

Semiautonomous driving still requires the driver's control and attention in certain situations. Especially control transitions, i.e. take-over and hand-over situations, are important for safety. Our aim was to study control transitions supported by unimodal (i.e. visual, auditory, or haptic) or multimodal (i.e. visual, auditory and haptic) signals indicating change from manual to autonomous driving and vice versa. The signals were abstract visual blinks, auditory beeps, or haptic vibrations. The task was to take over driving while either looking through the windshield or playing a game. In addition, in half of the control transitions a feedback signal indicated successful control transition. The results showed that a secondary task slowed down the reaction times, but there was a great variation between individuals. In general, the response to auditory signal was slower than to visual, haptic, or multimodal signals. Moreover, users preferred feedback during control transitions but this slowed down the reaction time.

With the proliferation of small-screen computing devices, there has been a continuous trend in reducing the size of interface elements. In virtual keyboards, this allows for more characters in a layout and additional function widgets. However, vision-based interfaces (VBIs) have only been investigated with large (e.g., full-screen) keyboards. To understand how key size reduction affects the accuracy and speed performance of text entry VBIs, we evaluated gaze-controlled VBI (g-VBI) and head-controlled VBI (h-VBI) with unconventionally small (0.4°, 0.6°, 0.8° and 1°) keys. Novices (N = 26) yielded significantly more accurate and fast text production with h-VBI than with g-VBI, while the performance of experts (N = 12) for both VBIs was nearly equal when a 0.8--1° key size was used. We discuss advantages and limitations of the VBIs for typing with ultra-small keyboards and emphasize relevant factors for designing such systems.

The aim was to study if odors evaporated by an olfactory display prototype can be used to affect participants' cognitive and emotionrelated responses to audio-visual stimuli, and whether the display can benefit from objective measurement of the odors. The results showed that odors and videos had significant effects on participants' responses. For instance, odors increased pleasantness ratings especially when the odor was authentic and the video was congurent with odors. The objective measurement of the odors was shown to be useful. The measurement data was classified with 100% accuracy removing the need to speculate whether the odor presentation apparatus is working properly.

Current aim was to introduce and initially evaluate the performance of a system called Extended Cognition, which was developed for enhancing the filtering of emotionally meaningful information from visual lifelogging data. The system is proposed to add facial expression markers to the stream of visual lifelogging data in order to later find special, possibly emotionally meaningful moments from the data. In an initial user study we collected subjective evaluations (N=10) about the ergonomics of the data measurement setup and the use of voluntary smiles and frowns for adding markers. Experience evaluations about the system were also collected. Results showed that data measurement setup was ergonomic to use and the system was evaluated positively. Smiling was rated as pleasant, natural, and functional expression for data marking purposes. Participants considered that, for example, the system would support memorization, and that it could be used to share important memories with others.

This study presents the results of a longitudinal study on multimodal text entry where objects were selected by gazing and smiling. Gaze was used to point at the desired characters and smiling movements were performed to select them. Participants (N=12) took part in the experiments where they entered text for a total of 2.5 hours in ten 15-minute-long sessions during one-month time period. The results showed that the text entry rate improved with practice from 4.1 to 6.7 words per minute. However, the learning curve had not reached its plateau phase at the end of the experiment. Subjective ratings showed that the participants appreciated this multimodal technique.

The aim of this paper was to evaluate the use of three facial actions (i.e. frowning, raising the eyebrows, and smiling) in selecting objects on a computer screen when gaze was used for pointing. Dwell time is the most commonly used selection technique in gaze-based interaction, and thus, a dwell time of 400 ms was used as a reference selection technique. A wireless, head-mounted prototype device that carried out eye tracking and contactless, capacitive measurement of facial actions was used for the interaction task. Participants (N=16) performed point-and-select tasks with three pointing distances (i.e. 60, 120 and 240 mm) and three target sizes (i.e. 25, 30 and 40 mm). Task completion times, pointing errors and throughput values based on Fitts' law were used to compare the selection techniques. The participants also rated the techniques with subjective ratings scales. The results showed that the different techniques performed equally well in many respects. However, throughput values varied from 8.38 bits/s (raising the eyebrows) to 15.33 bits/s (smiling) and were comparable to or, in the case of smiling, better than in earlier research with similar interaction techniques. The dwell time was found to be the least accurate selection technique in terms of the magnitudes of point-and-select errors. Smiling technique was rated as more accurate to use than the frowning or the raising techniques. The results give further support for methods that combine facial behavior to eye tracking when interacting with technology.

Emotional reactions to basic, artificial, yet carefully controllable point-light displays (PLDs) were investigated with ratings of valence, arousal, approachability and dominance. PLDs were varied by movement location (upper and lower) and intensity (⁠| $10^{\circ }$ |⁠, | $20^{\circ }$ | and | $30^{\circ }$ | angular change) for angular upward and downward movements. Half of participants (⁠| $N =28$ |⁠) were told that PLDs were related to face while to other half nothing was hinted. Results showed that | $20^{\circ }$ | and | $30^{\circ }$ | angle lower location upward movements were rated as significantly more pleasant, relaxing and approachable than corresponding upper location downward movements. Informed participants rated | $20^{\circ }$ | and | $30^{\circ }$ | angle lower movements as significantly more controllable than corresponding upper movements. Results are important from many perspectives, like for understanding human perceptual mechanisms. When using PLDs only a small amount of information needs to be transmitted. This enables low bandwidth requirements. As PLD visualizations are simple, there is no need for high-definition displays.

In curriculum visualization, information visualization techniques are used to communicate the structure and content of a curriculum to the stakeholders, such as students, lecturers, and administrators. Curriculum visualization has recently drawn a lot of interest as there are growing demands to form larger educational units and to find savings by eliminating overlap. We have developed a novel approach to analyse and visualize the contents of a curriculum. Our software tool for curriculum visualization uses a fast heuristic to automatically lay out a curriculum diagram, provides tools to process the content information, and offers several coordinated views to visualize the curriculum contents and overlap. The approach and software tool have already been used successfully in one survey and two others are underway.

This study aimed to investigate how the degree of naturalness and lexical emotional content of synthesised speech affects the subjective ratings of emotional experiences and how the naturalness of the voice affects the ratings of voice quality. Twenty-four participants listened to a set of affective words produced by three different speech synthesis techniques: formant synthesis, diphone synthesis and unit selection synthesis. The participants’ task was to rate their experiences evoked by the speech samples using three emotion-related bipolar scales for valence, arousal and approachability. The pleasantness, naturalness and clarity of the voices were also rated. The results showed that the affective words produced by the synthesisers evoked congruent emotion-related ratings in the participants. The ratings of the experienced valence and approachability were statistically significantly stronger when the affective words were produced by the more humanlike voices as compared to the more machinelike voice. The more humanlike voices were also rated as statistically significantly more natural, pleasant and clear than the less humanlike voice. Thus, our findings suggest that even machinelike voices can be used to communicate affective messages but that increasing the level of naturalness enhances positive feelings about synthetic voices and strengthens emotional communication between computers and humans.

Interpretations of vibrotactile stimulations were compared between two participant groups. In both groups, the task was to evaluate specifically designed tactile stimulations presented to the wrist or chest. Ascending, constant, and descending vibration frequency profiles of the stimuli represented information for three different speed regulation instructions: “accelerate your speed,” “keep your speed constant,” and “decelerate your speed,” respectively. The participants were treated differently so that one of the groups was first taught (i.e., primed) the meanings of the stimuli, whereas the other group was not taught (i.e., unprimed). The results showed that the stimuli were evaluated nearly equally in the primed and the unprimed groups. The best performing stimuli communicated the three intended meanings in the rate of 88% to 100% in the primed group and in the unprimed group in the rate of 71% to 83%. Both groups performed equally in evaluating “keep your speed constant” and “decelerate your speed” information. As the unprimed participants performed similarly to the primed participants, the results suggest that vibrotactile stimulation can be intuitively understood. The results suggest further that carefully designed vibrotactile stimulations could be functional in delivering easy-to-understand feedback on how to regulate the speed of movement, such as in physical exercise and rehabilitation applications.

Video-based human-computer interaction has received increasing interest over the years. However, earlier research has been mainly focusing on technical characteristics of different methods rather than on user performance and experiences in using computer vision technology. This study aims to investigate performance characteristics of novice users and their subjective experiences in typing text with several video-based pointing and selection techniques. In Experiment 1, eye tracking and head tracking were applied for the task of pointing at the keys of a virtual keyboard. The results showed that gaze pointing was significantly faster but also more erroneous technique as compared with head pointing. Self-reported subjective ratings revealed that it was generally better, faster, more pleasant and efficient to type using gaze pointing than head pointing. In Experiment 2, mouth open and brows up facial gestures were utilized for confirming the selection of a given character. The results showed that text entry speed was approximately the same for both selection techniques, while mouth interaction caused significantly fewer errors than brow interaction. Subjective ratings did not reveal any significant differences between the techniques. Possibilities for design improvements are discussed.

The effect of facial behaviour on gaze tracking accuracy was studied while using a prototype system that integrated head-mounted, video-based gaze tracking and a capacitive facial movement detection for respective pointing and selecting objects in a simple graphical user interface. Experiments were carried out to determine how voluntary smiling movements that were used to indicate clicks affect the accuracy of gaze tracking due to the combination of user eye movement behaviour and the operation of gaze tracking algorithms. The results showed no observable degradation of the gaze tracking accuracy when using voluntary smiling for object selections.

We present a novel vision-based perceptual user interface for hands-free text entry that utilizes face detection and visual gesture detection to manipulate a scrollable virtual keyboard. A thorough experimentation was undertaken to quantitatively define a performance of the interface in hands-free pointing, selection and scrolling tasks. The experiments were conducted with nine participants in laboratory conditions. Several face and head gestures were examined for detection robustness and user convenience. The system gave a reasonable performance in terms of high gesture detection rate and small false alarm rate. The participants reported that a new interface was easy to understand and operate. Encouraged by these results, we discuss advantages and constraints of the interface and suggest possibilities for design improvements.

The present aim was to investigate the functionality of a new wireless prototype called Face Interface. The prototype combines the use of voluntary gaze direction and facial muscle activations, for pointing and selecting objects on a computer screen, respectively. The subjective and objective functionality of the prototype was evaluated with a series of pointing tasks using either frowning (i.e., frowning technique) or raising the eyebrows (i.e., raising technique) as the selection technique. Pointing task times and accuracies were measured using three target diameters (i.e., 25, 30, 40 mm), seven pointing distances (i.e., 60, 120, 180, 240, 260, 450, and 520 mm), and eight pointing angles (0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°). The results showed that the raising technique was faster selection technique than the frowning technique for the objects that were presented in the pointing distances from 60 mm to 260 mm. For those pointing distances the overall pointing task times were 2.4 s for the frowning technique, and 1.6 s for the raising technique. Fitts’ law computations showed that the correlations for the Fitts’ law model were r = 0.77 for the frowning technique and r = 0.51 for the raising technique. Further, the index of performance (IP) value was 1.9 bits/s for the frowning technique and 5.4 bits/s for raising the eyebrows technique. Based on the results, the prototype functioned well and was adjustable so that two different facial activations can be used in combination with gaze direction for pointing and selecting objects on a computer screen.

Exposure to distressing computer-generated stimuli and feedback of physiological changes during exposure have been effective in the treatment of anxiety disorders (e.g., social phobia). Here we studied voluntary facial activations as a method for regulating more spontaneous physiological changes during virtual social stimulation. Twenty-four participants with a low or high level of social anxiety activated either the corrugator supercilii (used in frowning) or the zygomaticus major (used in smiling) facial muscle to keep a female or a male computer character walking towards them. The more socially anxious participants had a higher level of skin conductance throughout the trials as compared to less anxious participants. Within both groups, short-term skin conductance responses were enhanced both during and after facial activations; and corrugator supercilii activations facilitated longer term electrodermal relaxation. Zygomaticus major activations had opposite effects on subjective emotional ratings of the less and the more socially anxious. In sum, voluntary facial activations were effective in regulating emotional arousal during virtual social exposure. Corrugator supercilii activation was found an especially promising method for facilitating autonomic relaxation.

The present aim was to study the preference of tactile feedback stimulations given by non-physical (i.e., solid) piezo-actuated buttons. Participants (n=16) ranked 16 different tactile feedback stimuli varied by 4 output delays and 4 vibration durations. The results showed that the mean ranks of the stimuli differed significantly from each other. The timing parameters of delay and duration interacted with each other, for example, so that preference of certain vibration duration fluctuated in response to different output delays. Using a very short time window (i.e., 10-453 ms) combining both delay and duration parameters of the feedback could result either in favorable or significantly less favorable subjective experience. The results suggest that a preferred perception of tactile feedback from non-physical buttons requires careful design and controlling of the timing parameters.

This paper introduces a multi-actuator tactile device designed for remote touch communication. While closely-spaced high-frequency vibrotactile actuators can be difficult to distinguish, our system utilized four linear DC motors for presenting spatial tactile messages through low-frequency actuation. An experiment was conducted to determine accuracy for recognizing stimuli presented on the palm of the hand. Participants were asked to identify 10 predefined stimulus patterns created from the four linear actuators positioned in either a diamond or square configuration. Results showed that positional, linear, and circular stimuli were recognized with mean response accuracies of 98.8, 96.5, and 90.2%, respectively. No statistically significant differences were found between the actuator configurations. These findings can be utilized in developing a remote communication channel that supports the transfer of spatial aspects of touch such as mapping the location of finger touch of one user to tactile sensation on the palm of another user.

The present aim was to study if thermal stimuli presented to the palm can affect emotional responses when measured with emotion related subjective rating scales and changes in skin conductance response (SCR). Two target temperatures, cold and warm, were created by either decreasing or increasing the temperature of the stimulus 4 °C in respect to the participants current hand temperature. Both cold and warm stimuli were presented by using two presentation methods, i.e., dynamic and pre-adjusted. The results showed that both the dynamic and pre-adjusted warm stimuli elevated the ratings of arousal and dominance. In addition, the pre-adjusted warm and cold stimuli elevated the SCR. The results suggest that especially pre-adjusted warm stimuli can be seen as effective in activating the autonomic nervous system and arousal and dominance dimensions of the affective rating space.

The sense of touch has been shown to convey emotive information and nuances in face-to-face interpersonal communication, but its applications in mobile communication technologies are still limited. One of the challenges for such new communication medium is the interpretation of tactile messages. This paper presents a study with an early prototype of a mobile tactile device. Twenty novice participants interpreted four messages consisting of a four-channel vibrotactile stimulus, complemented with three cumulative textual cues regarding 1) the communication setting, 2) sender, and 3) situation. The subjective interpretations were assessed with four semantic differential scales, and the reasoning behind the interpretations was inquired by interviewing. The findings show that the intensity and, to some degree, the friendliness of the message could be identified from the tactile-only message. However, to correctly interpret the degree of formality or emotionality in the message, also contextual cues were needed.

A light-weight, wearable, wireless gaze tracker with integrated selection command source for human-computer interaction is introduced. The prototype system combines head-mounted, video-based gaze tracking with capacitive facial movement detection that enable multimodal interaction by gaze pointing and making selections with facial gestures. The system is targeted mainly to disabled people with limited mobility over their hands. The hardware was made wireless to remove the need to take off the device when moving away from the computer, and to allow future use in more mobile contexts. The algorithms responsible for determining the eye and head orientations to map gaze direction to on-screen coordinates are presented together with the one to detect movements from the measured capacitance signal. Point-and-click experiments were conducted to assess the performance of the multimodal system. The results show decent performance in laboratory and office conditions. The overall point-and-click accuracy in the multimodal experiments is comparable to the errors in previous research on head-mounted, single modality gaze tracking that does not compensate for changes in head orientation.

Abstract Emotion‐ and attention‐related subjective and physiological responses to virtual proximity and facial expressions of embodied computer agents (ECA) were studied. Thirty participants viewed female and male characters with a neutral, unpleasant, or pleasant facial expression. Agents' size was used to simulate three levels of proximity. Participants' electrical facial muscle and heart activity were registered, and subjective ratings of emotional and attentional experiences collected. Unpleasant and large (i.e., closer) agents were more alerting (i.e., unpleasant, arousing, and dominating) and attracted more stimulus‐driven attention than neutral, pleasant, and smaller (i.e., further away) agents. Pleasant agents attracted more voluntary attention than neutral and unpleasant agents. Heart rate (HR) responded to agent proximity, while the valence of the agent affected electrical facial muscle activity. Thus, the imitation of human social emotional cues in embodied computer agents (ECAs) could be used to regulate human–computer interaction. Copyright © 2010 John Wiley & Sons, Ltd.

Cutaneous saltation provides interesting possibilities for applications. An illusion of vibrotactile mediolateral movement was elicited to a left dorsal forearm to investigate emotional (i.e., pleasantness) and cognitive (i.e., continuity) experiences to vibrotactile stimulation. Twelve participants were presented with nine saltatory stimuli delivered to a linearly aligned row of three vibrotactile actuators separated by 70 mm in distance. The stimuli were composed of three temporal parameters of 12, 24 and 48 ms for both burst duration and inter-burst interval to form all nine possible uniform pairs. First, the stimuli were ranked by the participants using a special three-step procedure. Second, the participants rated the stimuli using two nine-point bipolar scales measuring the pleasantness and continuity of each stimulus, separately. The results showed especially the interval between two successive bursts was a significant factor for saltation. Moreover, the temporal parameters seemed to affect more the experienced continuity of the stimuli compared to pleasantness. These findings encourage us to continue to further study the saltation and the effect of different parameters for subjective experience.

A prototype of friction-based horizontally rotating fingertip stimulator was used to investigate emotional experiences and behavioral responses to haptic stimulation. The rotation style of 12 different stimuli was varied by burst length (i.e., 20, 50, 100 ms), continuity (i.e., continuous and discontinuous), and direction (e.g., forward and backward). Using these stimuli 528 stimulus pairs were presented to 12 subjects who were to distinguish if stimuli in each pair were the same or different. Then they rated the stimuli using four scales measuring the pleasantness, arousal, approachability, and dominance qualities of the 12 stimuli. The results showed that continuous forward-backward rotating stimuli were rated as significantly more unpleasant, arousing, avoidable, and dominating than other types of stimulations (e.g., discontinuous forward rotation). The reaction times to these stimuli were significantly faster than reaction times to discontinuous forward and backward rotating stimuli. The results clearly suggest that even simple haptic stimulation can carry emotional information. The results can be utilized when making use of haptics in human-technology interaction.

Haptic stimulation in motion has been studied only little earlier. To provide guidance for designing haptic interfaces for mobile use we carried out an initial experiment using C-2 actuators. 16 participants attended in the experiment to find out whether there is a difference in perceiving low-amplitude vibrotactile stimuli when exposed to minimal and moderate physical exertion. A stationary bike was used to control the exertion. Four body locations (wrist, leg, chest and back), two stimulus durations (1000 ms and 2000 ms) and two motion conditions with the stationary bicycle (still and moderate pedaling) were applied. It was found that cycling had significant effect on both the perception accuracy and the reaction times with selected stimuli. Stimulus amplitudes used in this experiment can be used to help haptic design for mobile users.

The aim in this work was to develop a method to visualize log files produced by a pilot study of an experimental e-learning platform. The e-learning system running on a web server generated detailed descriptions of its usage that were analyzed. The log files were produced to allow the project's researchers to study offline how the system was utilized. We used open-source software tools to create a flexible system that processed a large amount of log data into concise visual representations. The visualizations revealed a set of different interaction patterns during the pilot study and showed that each teacher had a large effect on students' interaction with the system. Thus, the results showed that the suggested visualization method can be used to support the evaluation and training of students' proficiency in a learning environment.

The present aim was to develop methods that estimate emotional experiences in real time from the electromyographic activity of two facial muscles: zygomaticus major (activated when smiling) and corrugator supercilii (activated when frowning). Ten subjects were stimulated with a series of emotionally arousing pictures and videos. After each stimulus the subjects rated the valence of their emotional experience on a nine-point bipolar dimensional scale. At the same time the computer estimated the subjects’ ratings on the basis of their electrical facial activity during each stimulation with 70 computational models. The models estimated the subjects’ ratings either categorically or dimensionally with regression models. The best categorical models were able to estimate negative and positive ratings with an average accuracy of over 70 and 80% for pictures and videos, respectively. The best correlations between the human ratings and machine estimations formed with the regression models were high (r>0.9). These findings indicate that models estimating psycho-emotional experiences on the basis of facial activity can be created successfully in several ways.

Feature-based method for detecting landmarks from facial images was designed. The method was based on extracting oriented edges and constructing edge maps at two resolution levels. Edge regions with characteristic edge pattern formed landmark candidates. The method ensured invariance to expressions while detecting eyes. Nose and mouth detection was deteriorated by happiness and disgust.

The goal of this research was to investigate neural network-based methods to be applied in the processing of biomedical signals. We developed a neural network-based method for the detection of voluntarily produced changes in facial muscle action potentials. Electromyographic signals were recorded from the corrugator supercilii and zygomaticus major facial muscles. The facial muscle action potentials of thirty subjects were measured while they performed a series of voluntary contractions of these muscles. Wavelet denoising or digital bandpass filtering was applied to the preprocessing of the signals. A neural network was exploited for an offline classification of various phases of these signals. The results show that the neural network-based technique developed functioned very well, producing a reliable recognition accuracy of 96 to 99%. Because of these promising results, we will proceed in the development of this method for real-time applications that benefit from the analysis of electromyographic signals.

New methods for unobtrusive monitoring of computer users' emotion psychophysiology are very much needed in human-computer interaction research. The present aim was to study heart rate changes during emotionally provocative stimulation. Six-second long auditory, visual, and audiovisual emotionally negative, neutral, and positive stimuli were presented to 24 participants. Heart rate responses were measured with a regular office chair embedded with electromechanical film (the EMFi chair) and with traditional earlobe photoplethysmography (PPG). Ratings of the stimuli were also collected. The results showed that the two heart rate measurements were significantly correlated, r = 0.99. In line with other studies the results showed that, in general, heart rate decelerated in response to emotional stimulation and it decelerated the most in response to negative stimuli as compared with responses to positive and neutral stimuli. Especially, emotional stimulation caused significant changes in heart rate at the 6th second from the stimulus onset. We suggest that the EMFi chair could be used in human-computer interaction for unobtrusive measurement of the user's emotional reactions.

The aim of this research was to develop methods for the automatic person-independent estimation of experienced emotions from facial expressions. Ten subjects watched series of emotionally arousing pictures and videos, while the electromyographic (EMG) activity of two facial muscles: zygomaticus major (activated in smiling) and corrugator supercilii (activated in frowning) was registered. Based on the changes in the activity of these two facial muscles, it was possible to distinguish between ratings of positive and negative emotional experiences at a rate of almost 70% for pictures and over 80% for videos. Using these methods, the computer could adapt its behavior according to the user's emotions during human-computer interaction.

The present study investigated the psychophysiological effects of positive and negative affective interventions in human–computer interaction during and after the interventions. Eighteen subjects were exposed to pre-programmed mouse delays in an interactive problem-solving task. Following the mouse delays three types of conditions were used: positive or negative interventions given via speech synthesizer, and no intervention. Facial electromyographic responses were recorded from the zygomaticus major and corrugator supercilii muscle sites. These muscles control smiling and frowning, respectively. Smiling activity was significantly higher during the positive than the other conditions. It was also significantly higher after the positive interventions than the no intervention condition. The frowning activity attenuated significantly more after the positive interventions than the no intervention condition. Following the positive interventions the users' problem solving performance was significantly better than after no intervention. In all, the results suggest that both types of affective intervention had beneficial effects over ignoring the user. The results suggest further that positive intervention may be especially useful.

The present aim was to study a new technique for human--computer interaction. It combined the use of two modalities, voluntary gaze direction and voluntary facial muscle activation for object pointing and selection. Fourteen subjects performed a series of pointing tasks with the new technique and with a mouse. At short distances the mouse was significantly faster than the new technique. However, there were no statistically significant differences at medium and long distances between the techniques. Fitts' law analyses were performed both by using only error-free trials and using also data including error trials (i.e., effective target width). In all cases both techniques seemed to follow Fitts' law, although for the new technique the effective target width correlation coefficient was smallerR= 0.776 than for the mouseR= 0.991. The regression slopes suggested that at very long distances (i.e., beyond 800 pixels) the new technique might be faster than the mouse. The new technique showed promising results already after a short practice and in the future it could be useful especially for physically challenged persons.

This paper investigated in two experiments pupillary responses to emotionally provocative sound stimuli. In experiment one, 30 subjects' pupillary responses were measured while listening to 10 negatively and 10 positively arousing sounds, and 10 emotionally neutral sounds. In addition, the subjects rated their subjective experiences to these stimuli. The results showed that the pupil size was significantly larger after highly arousing positive and negative stimuli than after neutral stimuli with medium arousal.