Thermal feedback has the potential to enrich immersive interaction, yet its role in material discrimination remains underexplored. We designed and evaluated a conductivity model that simulates transient heating and cooling profiles based on material properties of an object. Thirty-eight participants used a Meta Quest 3 headset and WEART TouchDIVER Pro gloves to classify virtual blocks (metal, glass, wood) under three conditions: visual-thermal congruence, thermal-only, and visual-thermal incongruence. Although the results show accuracy decreased when congruence decreased, the objects were rated consistently following their material conductivity properties (Metal > Glass > Wood), supporting the validity of the conductivity model. Haptic Experience (HX) ratings of realism, involvement, and harmony remained stable across tasks, while sorting difficulty was lowest under congruent and highest under incongruent visuals. Physiological baselines did affect performance. Our findings demonstrate that conductivity-based thermal rendering enables perceptually reliable material differences in VR, informing design and application of thermal haptics.

Abstract In this paper, we simulate a lifelike interaction with deformable bodies in virtual reality (VR) and provide force haptic feedback. Our method achieves physically compelling responses and stable feedback, which are essential for natural interaction with soft virtual components. Existing systems often suffer from instability or latency and rely on rigid body or primitive soft body models, which limit their application. We present a real-time VR framework that employs Extended Position Based Dynamics (XPBD) to enable deformable body simulation alongside stable haptic feedback. In our scenes, the proposed system achieves up to 96.3 FPS visual rendering on the VR system’s head-mounted display (HMD) while maintaining stability without frame loss or lag. Designed for lightweight deployment on consumer-grade hardware, our implementation integrates a haptic glove to capture finger motion and deliver multi-point tactile feedback. The simulation uses a unified solver that incorporates a set of position-based constraints and accounts for haptic force feedback. Our method supports scenes of up to 500 k vertices and requires no external tracking, relying solely on the HMD and controllers for positional input. We conducted a user study in which participants interacted with the system and rated deformations and haptic feedback. Results showed the visuals and the haptic responses were convincing and plausible. We demonstrate that our framework enables immersive, stable, deformable body interaction with synchronised visual and haptic feedback, supporting applications in medical simulation, physical skill training, and entertainment.

This study investigated the relationship between trust in automation, gaze behavior, and driving performance in beginner and experienced drivers during a simulated driving session. Twenty participants completed a 17-minute drive across three conditions: manual driving, non-critical automated driving, and critical automated driving, with a non-driving-related task (NDRT) introduced between conditions to assess visual attention. Driving performance was evaluated using the Standard Deviation of Lateral Position (SDLP), and eye-tracking data in terms of mean gaze duration (MGD). While both groups demonstrated increased trust in the automated system post-session, beginners showed greater lateral position variability in critical conditions, suggesting over-reliance on automation. Eye-tracking analysis revealed significant changes in glance behavior across driving conditions, particularly in response to critical events. These findings highlight how driver experience shapes interactions with automated systems, emphasizing the importance of trust calibration in automated driving scenarios.

Robots interacting with humans pose data privacy risks, potentially leading to uncontrollable leaks of personal information. Robots interact with older adults in public places, homes, care facilities, and hospitals. Older adults may have concerns about privacy issues related to robots and would benefit from developing their robot literacy skills regarding data privacy. Research on older adults’ robot literacy in relation to data privacy is scarce. We conducted a qualitative and explorative human-centered design study with care home residents (N=9) to explore their perceptions of and interest in robot data privacy literacy. In the study, they interacted with an early prototype of a robot-assisted learning application implemented on the social robot QTrobot. Participants were concerned about the "superpowers" and data storage of robots. Based on our findings and existing literature, we redesigned the prototype into "Myth Buster," a robot-assisted learning application aimed at enhancing older adults’ data privacy literacy regarding robots. Our work contributes to the understanding of older adults’ data privacy literacy, which is currently under-researched in Human-Robot Interaction. We also present design-relevant insights for developing robot-assisted data privacy learning applications to enhance robot literacy of older adults.

Abstract In Virtual Reality (VR), locomotion methods allow users to adjust their location in the virtual environment. These methods are sometimes used in 3D-VR medical image manipulation, which has gained interest in VR in the medical field due to the immersive environment and increased number of possible interaction techniques. However, the medical 3D-VR image manipulation context differs from typical VR locomotion, as the focus is not on the navigation of the VR space, but rather on the point of view of the user to the 3D image. For this study, we recruited 24 participants to find and observe easy targets from simplified medical images. We compared whether the three VR locomotion methods, Diving, Grabbing and Teleporting, were suitable for medical 3D image manipulation tasks. Diving was found to be significantly more successful than Teleporting while being an equally fast method. Locomotion method preferences varied. Therefore, a VR system is suggested to provide various manipulation methods for the user. However, the methods need development before they can be used with radiologists for actual medical image analysis tasks. We fill a research gap from the VR medical image manipulation context, where even though VR locomotion methods, such as Teleporting have been sometimes used, there has been a lack of interaction technique studies of these methods. Therefore, we provide necessary data on locomotion methods for the designers and developers of 3D-VR medical image applications. This is an initial study to validate different techniques for medical image analysis tasks in VR.

This study investigates the effects of dynamic and static visual manipulations, specifically darkening vignettes, on user experience and task performance in extended reality (XR) environments. By using heart rate (HR) as physiological feedback, the study explores how dynamic visual adaptations based on HR impact immersion, task focus, and user comfort. A user study (n = 16) found that while dynamic vignettes could increase immersion and encourage a more focused approach, they also induced discomfort, anxiety, and reduced task performance for some participants. The results reveal that although users may adapt to these visual effects over time, the variability in emotional and physical reactions suggests that such dynamic visual adaptations based on physiological signals should be carefully calibrated to balance enhanced immersion with user comfort. This research contributes to the understanding of adaptive XR environments and the role of physiological feedback in shaping user experience.

There are situations where we need to understand a person's health register data, get an overview of it and inspect the details of interest, preferably in a short period of time. Typical situations are a doctor seeing a patient for the very first time, or a researcher trying to understand what makes a person an outlier in a patient cohort. We have developed a visualization tool, called LifeTrack, that will show a person's whole health register history in a single view. The structure of the LifeTrack data view is based on concepts familiar to the target audience, and allows one to see both the overview and details of the data.

Whenever a diagnosis is given, a procedure is performed, or a drug is prescribed, it leads to an entry into an electronic health record (EHR) system. Previously, this data was difficult to utilize because of rules regarding confidentiality, but new security approaches and pseudonymization have enabled us to work with this data. Health-related data is voluminous and complex, and it can be difficult to abstract a meaningful overview. One of the complexities is its longitudinality. Often medical research is cross-sectional - we often take a point in time for analysis, when instead, it might be more beneficial to see the trajectory that led to the point in time. We are currently developing a trajectory visualization tool for longitudinal electronic health data. It is a web-based tool that interfaces with the OHDSI data infrastructure and visualizes the cohorts and concept sets (groups of medical codes) defined via the OHDSI Atlas GUI. Currently, our tool is in user testing and it will be deployed to a wider user group during the spring. The user feedback has been positive. Users find the tool especially useful in understanding and debugging their OHDSI Atlas cohort definitions.

Touch between people is an integral part of human life. Touch is used to convey information, emotions, and other social cues. Still, everyday remote communication remains mainly auditive or audio-visual. The theme of this article, interpersonal haptic communication, refers to any communication system that supports mediation of touch between two or more persons. We first present a scoping review of the state of the art in interpersonal haptic communication, including physiological and psychological basis of touch, affective and social touch, and mediated social touch. We then discuss emerging research themes that shape the future of interpersonal haptic communication, identify research gaps and propose key research directions for each theme. Finally, societal impact and ethical aspects are discussed.

To effectively utilize a gaze tracker in user interaction it is important to know the quality of the gaze data that it is measuring. We have developed a method to evaluate the accuracy and precision of gaze trackers in virtual reality headsets. The method consists of two software components. The first component is a simulation software that calibrates the gaze tracker and then performs data collection by providing a gaze target that moves around the headset's field-of-view. The second component makes an off-line analysis of the logged gaze data and provides a number of measurement results of the accuracy and precision. The analysis results consist of the accuracy and precision of the gaze tracker in different directions inside the virtual 3D space. Our method combines the measurements into overall accuracy and precision. Visualizations of the measurements are created to see possible trends over the display area. Results from selected areas in the display are analyzed to find out differences between the areas (for example, the middle/outer edge of the display or the upper/lower part of display).

In this study, we assessed the emotional dimensions (valence, arousal, and dominance) of the multimodal visual-cutaneous rabbit effect. Simultaneously to the tactile bursts on the forearm, visual silhouettes of saltatorial animals (rabbit, kangaroo, spider, grasshopper, frog, and flea) were projected on the left arm. Additionally, there were two locomotion conditions: taking-off and landing. The results showed that the valence dimension (happy-unhappy) was only affected by the visual stimuli with no effect of the tactile conditions nor the locomotion phases. Arousal (excited-calm) showed a significant difference for the three tactile conditions with an interaction effect with the locomotion condition. Arousal scores were higher when the taking-off condition was associated with the intermediate duration (24 ms) and when the landing condition was associated with either the shortest duration (12 ms) or the longest duration (48 ms). There was no effect for the dominance dimension. Similar to our previous results, the valence dimension seems to be highly affected by visual information reducing any effect of tactile information, while touch can modulate the arousal dimension. This can be beneficial for designing multimodal interfaces for virtual or augmented reality.

Vibrotactile feedback is widely used in mobile devices because it provides a discreet and private feedback channel. Gaze-based interaction, on the other hand, is useful in various applications due to its unique capability to convey the focus of interest. Gaze input is naturally available as people typically look at things they operate, but feedback from eye movements is primarily visual. Gaze interaction and the use of vibrotactile feedback have been two parallel fields of human–computer interaction research with a limited connection. Our aim was to build this connection by studying the temporal and spatial mechanisms of supporting gaze input with vibrotactile feedback. The results of a series of experiments showed that the temporal distance between a gaze event and vibrotactile feedback should be less than 250 ms to ensure that the input and output are perceived as connected. The effectiveness of vibrotactile feedback was largely independent of the spatial body location of vibrotactile actuators. In comparison to other modalities, vibrotactile feedback performed equally to auditory and visual feedback. Vibrotactile feedback can be especially beneficial when other modalities are unavailable or difficult to perceive. Based on the findings, we present design guidelines for supporting gaze interaction with vibrotactile feedback.

Abstract Kinaesthetic interaction using force-feedback devices is promising in virtual reality. However, the devices are currently not suitable for interactions within large virtual spaces because of their limited workspace. We developed a novel gaze-based kinaesthetic interface that employs the user’s gaze to relocate the device workspace. The workspace switches to a new location when the user pulls the mechanical arm of the device to its reset position and gazes at the new target. This design enables the robust relocating of device workspace, thus achieving an infinite interaction space, and simultaneously maintains a flexible hand-based kinaesthetic exploration. We compared the new interface with the scaling-based traditional interface in an experiment involving softness and smoothness discrimination. Our results showed that the gaze-based interface performs better than the traditional interface, in terms of efficiency and kinaesthetic perception. It improves the user experience for kinaesthetic interaction in virtual reality without increasing eye strain.

Semiautonomous driving still requires the driver's control and attention in certain situations. Especially control transitions, i.e. take-over and hand-over situations, are important for safety. Our aim was to study control transitions supported by unimodal (i.e. visual, auditory, or haptic) or multimodal (i.e. visual, auditory and haptic) signals indicating change from manual to autonomous driving and vice versa. The signals were abstract visual blinks, auditory beeps, or haptic vibrations. The task was to take over driving while either looking through the windshield or playing a game. In addition, in half of the control transitions a feedback signal indicated successful control transition. The results showed that a secondary task slowed down the reaction times, but there was a great variation between individuals. In general, the response to auditory signal was slower than to visual, haptic, or multimodal signals. Moreover, users preferred feedback during control transitions but this slowed down the reaction time.

This paper contains a succinct survey on the recent major advances in contactless ultrasonic tactile feedback. It is a haptic technology which enables easy mid-air interactions with rich multisensory feedback and creates effects which are otherwise impossible. It can become a disruptive technology for multimedia interfaces, applications, and mobile computing. We summarize and discuss the advantages, problems and applications of the technology. This survey provides an introduction to the topic for anyone interested in applying or researching it.

We present a live gigapixel virtual reality system employing a 360° camera, a superzoom camera with a pan-tilt robotic head, and a head-mounted display (HMD). The system is capable of showing on-demand gigapixel-level subregions of 360° videos. Similar systems could be used to have live feed for foveated rendering HMDs.

Selecting an object is a basic interaction task in virtual reality (VR) environments. Interaction techniques with gaze pointing have potential for this elementary task. There appears to be little empirical evidence concerning the benefits and drawbacks of these methods in VR. We ran an experiment studying three interaction techniques: ray casting, dwell time and gaze trigger, where gaze trigger was a combination of gaze pointing and controller selection. We studied user experience and interaction speed in a simple object selection task. The results indicated that ray casting outperforms both gaze-based methods while gaze trigger performs better than dwell time.

Interactions between humans and virtual environments rely on timely and consistent sensory feedback, including haptic feedback. However, many questions remain open concerning the spatial location of haptics on the user's body in VR. We studied how simple vibrotactile collision feedback on two less studied locations, the temples, and the wrist, affects an object picking task in a VR environment. We compared visual feedback to three visual-haptic conditions, providing haptic feedback on the participants' (N=16) wrists, temples or simultaneously on both locations. The results indicate that for continuous, hand-based object selection, the wrist is a more promising feedback location than the temples. Further, even a suboptimal feedback location may be better than no haptic collision feedback at all.

Additional illumination improves the capture of omnidirectional 360° video and images, especially for dark or high-contrast environments. There is no "behind" for 360° cameras, so the placement of lights is a problem. We explore ways to position lights on some 360° cameras, and propose two good locations.

Immersive 360° video needs new ways of interaction. We compared three different interaction methods to find out which one of them is the most applicable for controlling 360° video playback. The compared methods were: remote control, pointing with head orientation, and hand gestures. A WebVR-based 360° video player was built for the experiment.

We present a novel signaling method for head-mounted displays, which surpasses eye (pupil) and delivers guiding light signals directly to retina through tissue near the eyes. This method preserves full visual acuity on the display and does not block view to the scene, while also delivering additional visual signals.

The best way to construct user interfaces for smart glasses is not yet known. We investigated the use of eye tracking in this context in two experiments. The eye and head movements were combined so that one can select the object to interact by looking at it and then change a setting in that object by turning the head horizontally. We compared three different techniques for mapping the head turn to scrolling a list of numbers with and without haptic feedback. We found that the haptic feedback had no noticeable effect in objective metrics, but it sometimes improved user experience. Direct mapping of head orientation to list position is fast and easy to understand, but the signal-to-noise ratio of eye and head position measurement limits the possible range. The technique with constant rate of change after crossing the head angle threshold was simple and functional, but slow when the rate of change is adjusted to suit beginners. Finally the rate of change dependent on the head angle tends to lead to fairly long task completion times, although in theory it offers a good combination of speed and accuracy.

Navigation systems usually require visual or auditory attention. Providing the user with haptic cues could potentially decrease cognitive demand in navigation. This study is investigating the use of haptic eyeglasses in navigation. We conducted an experiment comparing directional haptic cues to visual cueing in a car navigation task. Participants (N=12) drove the Lane Change Test simulator with visual text cues, haptic cues given by the eyeglasses and haptic cues given by a car seat. The participants were asked to confirm the recognition of a directional cue (left or right) by pressing an arrow on a tablet screen and by navigating to the corresponding lane. Reaction times and errors were measured. The participants filled in the NASA-TLX questionnaire and were also interviewed about the different cues. The results showed that in comparison to the visual text cues the haptic cues were reacted to significantly faster. Haptic cueing was also evaluated as less frustrating than visual cueing. The haptic eyeglasses fared slightly, although not significantly, better than the haptic seat in subjective and objective evaluations. The paper suggests that haptic eyeglasses can decrease cognitive demand in navigation and have many possible applications.

There exists a large base of evidence for gender differences in human navigation. However, there is not much research on gender differences in collaborative aspects of navigation, including the interaction of individuals during collaborative wayfinding tasks in virtual environments. In light of this, we present a study of a collaborative virtual environment, Berlin Kompass. The goal of this study was to find out the main differences between genders in collaborative wayfinding. The application was evaluated in the context of foreign language learning in schools with over 200 students, where the users navigated through cityscapes while interacting verbally with each other. We collected and analyzed interaction logs, questionnaire data and audio and video recordings to gain insights into gender-related differences in wayfinding in virtual worlds. Our findings suggest that several differences that are evident in single user systems are not present when the collaborative aspect is added. Male users were more immersed during the task than females. One of the explaining factors for this might be video game experience. Genders also communicated differently -- males spoke in longer utterances whereas females had more, shorter utterances. Males referred more to relative directions and dynamic landmarks such as cars and pedestrians while navigating. Males with more video game experience also provided more positive subjective user experience feedback on the application.

Virtual and physical worlds are merging. Currently users of head-mounted displays cannot have unobtrusive tactile feedback while touching virtual objects. We present a mid-air tactile feedback system for head-mounted displays. Our prototype uses the focus of a modulated ultrasonic phased array for unobtrusive mid-air tactile feedback generation. The array and the hand position sensor are mounted on the front surface of a head-mounted virtual reality display. The presented system can enhance 3D user interfaces and virtual reality in a new way.

Anticipating the emergence of gaze tracking capable mobile devices, we are investigating the use of gaze as an input modality in handheld mobile devices. We conducted a study of combining gaze gestures with vibrotactile feedback. Gaze gestures were used as an input method in a mobile device and vibrotactile feedback as a new alternative way to give confirmation of interaction events. Our results show that vibrotactile feedback significantly improved the use of gaze gestures. The tasks were completed faster and rated easier and more comfortable when vibrotactile feedback was provided.

Consistent measuring and reporting of gaze data quality is important in research that involves eye trackers. We have developed TraQuMe: a generic system to evaluate the gaze data quality. The quality measurement is fast and the interpretation of the results is aided by graphical output. Numeric data is saved for reporting of aggregate metrics for the whole experiment. We tested TraQuMe in the context of a novel hidden calibration procedure that we developed to aid in experiments where participants should not know that their gaze is being tracked. The quality of tracking data after the hidden calibration procedure was very close to that obtained with the Tobii's T60 trackers built-in 2 point, 5 point and 9 point calibrations.

Eye tracking input often relies on visual and auditory feedback. Haptic feedback offers a previously unused alternative to these established methods. We describe a study to determine the natural time limits for haptic feedback to gazing events. The target is to determine how much time we can use to evaluate the user gazed object and decide if we are going to give the user a haptic notification on that object or not. The results indicate that it is best to get feedback faster than in 250 milliseconds from the start of fixation of an object. Longer delay leads to increase in incorrect associations between objects and the feedback. Delays longer than 500 milliseconds were confusing for the user.

Haptic feedback based on the sense of touch and movement is a promising area of human-computer interaction in the car context. Most user studies on haptic feedback in the car have been controlled experiments of specific types of haptic stimuli. For the study presented in this paper, twelve participants tried novel haptic feedback prototypes and evaluated communication scenarios in the physical car context. Our aim was to understand user experiences and usage potential of haptic feedback in the car. The qualitative results show that haptic feedback may offer support for safety and social communication, but can be hard to interpret. We propose design considerations for in-car haptics such as simplicity, subtleness and directionality.

Devices such as mobile phones have made it possible to take part in remote audio conferences regardless of one's physical location. Mobile phones also allow for new ways to interact with other conference participants. We present a study on evaluating the user experiences of a mobile audio conferencing system that was augmented with spatial audio, haptics, and gestures. In a user study groups of participants compared the augmented audio conference based on a mobile phone and headset to a traditional audio conference. The participants' task was to use the two alternative systems in given discussion tasks. The results of the subjective questionnaires showed that the augmented audio conference was perceived as more stimulating (e.g. creative), while the traditional audio conference was perceived as more practical (e.g. straightforward). The results of the group interviews indicated that spatial audio was the most desired feature, and that it had a positive effect on participants' perception of the conversation. Based on our findings, guidelines for the future development of similar systems are presented.

The spreading of mobile devices to all areas of everyday life impacts many contexts of use, including cars. Even though driving itself has remained relatively unchanged, there are now a wide variety of new in-car tasks, which people perform with both integrated infotainment systems and their mobile devices. To gain insights into this new task context and how it could be improved, we conducted a qualitative, contextual study in which we observed real-life car journeys with eight participants. The focus was on user interaction with touchscreen mobile devices, due to their wide range of functions and services. The findings show that the car is an extension of other contexts and it contains a rich set of infotainment tasks, including use of social media. Drivers emphasized gesture interaction and the use of non-visual modalities, for replacing visual information and notifying of changes in the driving context. Based on the findings, we present design implications for future in-car infotainment systems.

We present a location-based collaborative mobile learning platform called Seek'N'Share. It is comprised of a Web-based learning assignment editor and a mobile application for exploring and capturing multimedia content in the field. The editor enables drag-and-drop creation of learning tasks, areas and points of interest using an intuitive Web interface. Assignments are accessed with an Android application that uses location information to provide content and tasks to learners as they explore the environment. The mobile application enables the learners to record audio, video and take pictures of their environments. This supports the overall goal of putting together a presentation as the outcome of the learning activity by combining predefined, contextual information with user-generated content. The platform is currently piloted with local schools. Its novelty lies in its flexible support for creating location-based learning activities for unconstrained environments, and the possibility for the learners to collaboratively document their learning outcomes in situ.

In Virtual Learning Environments efficient navigation is a major issue, especially when it is used as a component in the learning process. This paper addresses the challenges in creating meaningful navigation routes from language learning perspective. The work is grounded on findings from a specific case on German language learning, wherein two remotely located users communicated in a wayfinding guidance scenario. The users navigated through 360-degree virtual panoramic images using body gestures and could receive communication help via spoken hints by pointing at objects in the scenery. An important design consideration is how to choose these objects, as they have both navigational importance and pedagogical significance in terms of learning the desired language. Wayfinding interactions from 21 participants were compared to the values provided by a landmark attraction model applied on the landmarks along the routes. The results show that there was a clear connection between prominence of landmarks and time spent on each panorama. This indicates that together with pedagogical planning, the model can aid in selecting the interactive content for language learning applications in virtual environments.

Interpretations of vibrotactile stimulations were compared between two participant groups. In both groups, the task was to evaluate specifically designed tactile stimulations presented to the wrist or chest. Ascending, constant, and descending vibration frequency profiles of the stimuli represented information for three different speed regulation instructions: “accelerate your speed,” “keep your speed constant,” and “decelerate your speed,” respectively. The participants were treated differently so that one of the groups was first taught (i.e., primed) the meanings of the stimuli, whereas the other group was not taught (i.e., unprimed). The results showed that the stimuli were evaluated nearly equally in the primed and the unprimed groups. The best performing stimuli communicated the three intended meanings in the rate of 88% to 100% in the primed group and in the unprimed group in the rate of 71% to 83%. Both groups performed equally in evaluating “keep your speed constant” and “decelerate your speed” information. As the unprimed participants performed similarly to the primed participants, the results suggest that vibrotactile stimulation can be intuitively understood. The results suggest further that carefully designed vibrotactile stimulations could be functional in delivering easy-to-understand feedback on how to regulate the speed of movement, such as in physical exercise and rehabilitation applications.

The present aim was to study the preference of tactile feedback stimulations given by non-physical (i.e., solid) piezo-actuated buttons. Participants (n=16) ranked 16 different tactile feedback stimuli varied by 4 output delays and 4 vibration durations. The results showed that the mean ranks of the stimuli differed significantly from each other. The timing parameters of delay and duration interacted with each other, for example, so that preference of certain vibration duration fluctuated in response to different output delays. Using a very short time window (i.e., 10-453 ms) combining both delay and duration parameters of the feedback could result either in favorable or significantly less favorable subjective experience. The results suggest that a preferred perception of tactile feedback from non-physical buttons requires careful design and controlling of the timing parameters.

This paper introduces a multi-actuator tactile device designed for remote touch communication. While closely-spaced high-frequency vibrotactile actuators can be difficult to distinguish, our system utilized four linear DC motors for presenting spatial tactile messages through low-frequency actuation. An experiment was conducted to determine accuracy for recognizing stimuli presented on the palm of the hand. Participants were asked to identify 10 predefined stimulus patterns created from the four linear actuators positioned in either a diamond or square configuration. Results showed that positional, linear, and circular stimuli were recognized with mean response accuracies of 98.8, 96.5, and 90.2%, respectively. No statistically significant differences were found between the actuator configurations. These findings can be utilized in developing a remote communication channel that supports the transfer of spatial aspects of touch such as mapping the location of finger touch of one user to tactile sensation on the palm of another user.

The present aim was to study if thermal stimuli presented to the palm can affect emotional responses when measured with emotion related subjective rating scales and changes in skin conductance response (SCR). Two target temperatures, cold and warm, were created by either decreasing or increasing the temperature of the stimulus 4 °C in respect to the participants current hand temperature. Both cold and warm stimuli were presented by using two presentation methods, i.e., dynamic and pre-adjusted. The results showed that both the dynamic and pre-adjusted warm stimuli elevated the ratings of arousal and dominance. In addition, the pre-adjusted warm and cold stimuli elevated the SCR. The results suggest that especially pre-adjusted warm stimuli can be seen as effective in activating the autonomic nervous system and arousal and dominance dimensions of the affective rating space.

The sense of touch has been shown to convey emotive information and nuances in face-to-face interpersonal communication, but its applications in mobile communication technologies are still limited. One of the challenges for such new communication medium is the interpretation of tactile messages. This paper presents a study with an early prototype of a mobile tactile device. Twenty novice participants interpreted four messages consisting of a four-channel vibrotactile stimulus, complemented with three cumulative textual cues regarding 1) the communication setting, 2) sender, and 3) situation. The subjective interpretations were assessed with four semantic differential scales, and the reasoning behind the interpretations was inquired by interviewing. The findings show that the intensity and, to some degree, the friendliness of the message could be identified from the tactile-only message. However, to correctly interpret the degree of formality or emotionality in the message, also contextual cues were needed.

We demonstrate a multimodal, multi-user, and multi-device conference program guide for conference participants. Its functionality includes access to the conference program with additional multimedia content, voting, feedback, and communication with the other participants. People can interact with the system in a multimodal way using spoken language, gestures and haptic feedback with mobile phones and shared public displays.

Human head gestures can potentially trigger different commands from the list of available options in graphical user interfaces or in virtual and smart environments. However, continuous tracking techniques are limited in generating discrete events which could be used to execute a predefined set of commands. In this article, we discuss a possibility to encode a set of discrete events by integrating continuous head movements and crossing-based interaction paradigm. A set of commands can be encoded through specific sequences of crossing points when a head-mouse cursor such as a scaled pointer interacts with a graphical object. The goal of the present experiment was testing the perceptual-motor performance of novices in target acquisition tasks using a subset of round head gestures and symbolic icons designating eight types of directional head movements. We have demonstrated that the novices can equally well execute round head gestures in clockwise and counter-clockwise directions by making two crossings for about 2 s or three crossings for about 3 s. None of the participants reported neck strain or other problems after 360 trials performed during a 40-min test in each of 5 days.

We report the results of a study focusing on the meanings that can be conveyed by audio and tactile icons. Our research considers the following question: how can audio and tactile icons be designed to optimise congruence between crossmodal feedback and the type of information this feedback is intended to convey? For example, if we have a set of system warnings, confirmations, progress up-dates and errors: what audio and tactile representations best match the information or type of message? Is one modality more appropriate at presenting certain types of information than the other modality? The results of this study indicate that certain parameters of the audio and tactile modalities such as rhythm, texture and tempo play an important role in the creation of congruent sets of feedback when given a specific type of information to transmit. We argue that a combination of audio or tactile parameters derived from our results allows the same type of information to be derived through touch and sound with an intuitive match to the content of the message.

Cutaneous saltation provides interesting possibilities for applications. An illusion of vibrotactile mediolateral movement was elicited to a left dorsal forearm to investigate emotional (i.e., pleasantness) and cognitive (i.e., continuity) experiences to vibrotactile stimulation. Twelve participants were presented with nine saltatory stimuli delivered to a linearly aligned row of three vibrotactile actuators separated by 70 mm in distance. The stimuli were composed of three temporal parameters of 12, 24 and 48 ms for both burst duration and inter-burst interval to form all nine possible uniform pairs. First, the stimuli were ranked by the participants using a special three-step procedure. Second, the participants rated the stimuli using two nine-point bipolar scales measuring the pleasantness and continuity of each stimulus, separately. The results showed especially the interval between two successive bursts was a significant factor for saltation. Moreover, the temporal parameters seemed to affect more the experienced continuity of the stimuli compared to pleasantness. These findings encourage us to continue to further study the saltation and the effect of different parameters for subjective experience.

We describe a simple and low-cost system that can help multiple sclerosis (MS) patients with asymmetric impairment to exert better grasp force control in manipulation tasks. The approach consists of measuring force vectors at the fingertips of the impaired hand, computing the force imbalance among the fingers, and providing corresponding haptic signals to the fingers of the opposite hand. Tests conducted on 24 MS patients indicated that for those with mild impairment, slightly better results were obtained with an ldquoevent-cuerdquo feedback (ECF) that alerted them when the grasp forces were straying outside of a desirable range. For patients with more severe impairment, better results were obtained by providing a proportional signal, in which the frequency and duty cycle of vibration pulses were correlated directly with the magnitudes of the fingertip forces. Post-test surveys of the patients also indicated that mildly impaired subjects preferred an event-cue feedback, and more severely impaired subjects preferred the proportional feedback.

A prototype of friction-based horizontally rotating fingertip stimulator was used to investigate emotional experiences and behavioral responses to haptic stimulation. The rotation style of 12 different stimuli was varied by burst length (i.e., 20, 50, 100 ms), continuity (i.e., continuous and discontinuous), and direction (e.g., forward and backward). Using these stimuli 528 stimulus pairs were presented to 12 subjects who were to distinguish if stimuli in each pair were the same or different. Then they rated the stimuli using four scales measuring the pleasantness, arousal, approachability, and dominance qualities of the 12 stimuli. The results showed that continuous forward-backward rotating stimuli were rated as significantly more unpleasant, arousing, avoidable, and dominating than other types of stimulations (e.g., discontinuous forward rotation). The reaction times to these stimuli were significantly faster than reaction times to discontinuous forward and backward rotating stimuli. The results clearly suggest that even simple haptic stimulation can carry emotional information. The results can be utilized when making use of haptics in human-technology interaction.

Haptic stimulation in motion has been studied only little earlier. To provide guidance for designing haptic interfaces for mobile use we carried out an initial experiment using C-2 actuators. 16 participants attended in the experiment to find out whether there is a difference in perceiving low-amplitude vibrotactile stimuli when exposed to minimal and moderate physical exertion. A stationary bike was used to control the exertion. Four body locations (wrist, leg, chest and back), two stimulus durations (1000 ms and 2000 ms) and two motion conditions with the stationary bicycle (still and moderate pedaling) were applied. It was found that cycling had significant effect on both the perception accuracy and the reaction times with selected stimuli. Stimulus amplitudes used in this experiment can be used to help haptic design for mobile users.

This paper reports the results of a study on the design and evaluation of the game and techniques which allow puzzles to be played in the absence of visual feedback. We have demonstrated that a camera-mouse can be used successfully for blind navigation and target location acquisition within a game field. To gradually teach the players the sequential learning method was applied. Blind exploration of the gamespace was augmented with sticky labels and overview sound cues, verbal and non-verbal, which can significantly reduce the cognitive load and facilitate mental matching and integration. The full-sticky labels technique does not require fine motor skills and allows a user to gain control over the game with a minimum level of skills. With the vertical sticky labels technique training was focused on the development of accurate head movements only on a horizontal plane. With practice, the players can use the non-sticky labels technique. After 240 trials (3–4 h), the cumulative experience of the blindfolded players was increased 22.5–27 times compared to the initial 10 trials.

We present a systematic study on gender classification with automatically detected and aligned faces. We experimented with 120 combinations of automatic face detection, face alignment and gender classification. One of the findings was that the automatic face alignment methods did not increase the gender classification rates. However, manual alignment increased classification rates a little, which suggests that automatic alignment would be useful when the alignment methods are further improved. We also found that the gender classification methods performed almost equally well with different input image sizes. In any case, the best classification rate was achieved with a support vector machine. A neural network and Adaboost achieved almost as good classification rates as the support vector machine and could be used in applications where classification speed is considered more important than the best possible classification accuracy.

Modern information technology allows us to seek out new ways to support the computer use and communication of disabled people. With the aid of new interaction technologies and techniques visually impaired and sighted users can collaborate, for example, in the classroom situations. The main goal of the MICOLE project was to create a software architecture that makes it easier for the developers to create multimodal multi-user applications. The framework is based on interconnected software agents. The hardware used in this study includes VTPlayer Mouse which has two built-in Braille displays, and several haptic devices such as PHANToM Omni, PHANToM Desktop and PHANToM Premium. We also used the SpaceMouse and various audio setups in the applications. In this paper we present a software architecture, a set of software agents, and an example of using the architecture. The example application shown is an electric circuit application that follows the single-user with many devices scenario. The application uses a PHANToM and a VTPlayer Mouse together with visual and audio feedback to make the electric circuits understandable through touch.

Trackball-mice are devices that include both a trackball and a mouse. In this paper we discuss our experiences in building and testing trackball-mouse prototypes. We report four experiments on user performance with the prototypes used as trackball-mice, conventional mice, and in two-handed configuration with a separate trackball for the non-dominant hand. The results show that user performance with the two-handed configuration was better than in one-handed operation of a trackball-mouse and in one-handed operation of a mouse. Trackball-mouse use and conventional mouse use were more evenly matched. However, Trackball-mouse operation involves a skill that most users do not have whereas mouse operation is familiar to most. Therefore, widespread introduction of trackball-mice does not appear to be justified on performance grounds alone. However, trackball-mice can be used as regular mice by ignoring the ball. This makes them compatible with traditional graphical user interfaces while offering two extra degrees of freedom in tasks where they are beneficial.

Visually impaired people have a lack of proper user interfaces to allow them to easily make use of modern technology. This problem may be solved with multimodal user interfaces that should be designed taking into account the type and degree of disability. The purpose of the study presented in this article was to create usable games for visually impaired children making use of low-cost vibro-tactile devices in multimodal applications. A tactile memory game using multimodal navigation support with high-contrast visual feedback and audio cues was implemented. The game was designed to be played with a tactile gamepad. Different vibrations were to be remembered instead of sounds or embossed pictures that are common in memory games for blind children. The usability and playability of the game was tested with a group of seven 12–13-year-old visually impaired children. The results showed that the game design was successful and a tactile gamepad was usable. The game got a positive response from the focus group.

Visually impaired children have a great disadvantage in the modern society since their ability to use modern computer technology is limited due to inappropriate user interfaces. The aim of the work presented in this paper was to develop a multimodal software architecture and applications to support visually impaired children and to enable them to interact equally with sighted children in learning situations. The architecture is based on software agents, and has specific support for visual, auditory and haptic interaction. It has been used successfully with different groups of 7-8-year-old and 12-year-old visually impaired children. In this paper we focus on the enabling software technology and interaction techniques aimed to realize our goal.

Visually impaired children do not have equal opportunities to learn and play compared to sighted children. Computers have a great potential to correct this problem. In this paper we present a series of studies where multimodal applications were designed for a group of eleven visually impaired children aged from 3.5 to 7.5 years. We also present our testing procedure specially adapted for visually impaired young children. During the two-year project it became clear that with careful designing of the tasks and proper use of haptic and auditory features usable computing environments can be created for visually impaired children.

As structured graphs are widely used in various application areas for conceptual modeling, the need for interactive layout adjusting methods is clearly recognized in the domain of graph drawing applications. We present a graph manipulation method where the process of adjusting and manipulating graphs is broken down into a group of direct and interactive tools. The tools are visualized as large tool-specific mouse cursors that are easily adjustable. Colors and highlighting are used to identify different tools and their operating ranges. According to the usability studies, the tools proved to be reasonably easy to use with only a short introductory time. The users could also select appropriate tools for each manipulation task without problems and were satisfied with the functionality as well as the visualization of the tools. The usefulness of the tools in manipulating graph drawings was also recognized.

Individual characters and text are the main inputs in many computing devices. Currently there is a growing trend in developing small portable devices like mobile phones, personal digital assistants, GPS-navigators, and two-way pagers. Unfortunately these portable computing devices have different user interfaces and therefore the task of text input takes many forms. The user, who in the future is likely to have several of these devices, has to learn several text input methods. We argue that there is a need for a universal text input method. A method like this would work on a wide range of interface technologies and allow the user to transfer his or her writing skill without device-specific training. To show that device independent text input is possible, we present a candidate for a device independent text entry method that supports skill transfer between different devices. A limited longitudinal study was conducted to achieve a proof of concept evaluation of our Minimal Device Independent Text Input Method (MDITIM). We found MDITIM writing skill acquired with a touchpad to work almost equally well on mouse, trackball, joystick and keyboard without any additional training. Our test group reached on average 41% of their handwriting speed by the end of the tenth 30-minute training session.

Object alignment is one of the basic operations in drawing programs. Current solutions provide mainly three ways for carrying out this operation: either by issuing an alignment command, or by using direct positioning with the help of gravity active points, or by making use of constraints. The first technique has limited functionality, and the other two may be difficult to learn for a novice. We describe here a new direct manipulation tool for alignment. We show that while direct manipulation helps to make the normal use of the tool intuitive, it also offers advanced functionality not found in current commercial products. We report on an empirical study in which we compared the ease of use, intuitiveness, learnability, and efficiency of alignment menus, palettes and the alignment stick. In the study novice users found the basic operation of the alignment stick natural and easy to learn. The increased functionality was best appreciated and utilized by the experienced users.

Current object-oriented drawing programs have an established way of drawing in which the shape of an object is controlled by manipulating control points. While the control points are intuitive in their basic use, it is not clear whether they make more complex drawing tasks manageable for the average user. In this paper we describe an alternative way of drawing and editing a drawing using new direct manipulation tools. Our approach resembles sculpting in two dimensions: the user begins with a large block and uses different tools to give it the desired shape. We also present a user evaluation in which the users could try our new tools and compare them to their previous experience of control points, The users claimed to understand the operations better with our tools than if they had needed to use curves and control points. However, our tools were better suited for sketching the artwork than for making very detailed drawings.

Current drawing programs provide mainly three ways for carrying out object alignment: either by issuing an alignment command, or by using direct positioning with the help of gravity active points, or by making use of constraints.The first technique has limited functionality, and the other two may be mysterious for a novice.We describe here a new direct manipulation tool for alignment.We show that while direct manipulation helps to make the tool intuitive, it has through iterative design evolved into a tool that also offers functionality not found in current commercial products.