We trained a model to distinguish an extreme high arousal, unpleasant drink from regular drinks based on a range of implicit behavioral and physiological responses to naturalistic tasting. The trained model predicted arousal ratings of regular drinks, highlighting the possibility to estimate affective experience without having to rely on subjective ratings.

Advances in robotics and multisensory displays allow extending telepresence ambitions beyond only the feeling of being present at a remote location In this paper, we discuss what may lie beyond telepresence and how we can transport both the functional and social self of a user. We introduce the embodiment illusion and its potential contribution to task performance and list important cues to evoke this illusion, including synchronicity in multisensory information, a first-person visual perspective, and a human-like visual appearance and anatomy of the telepresence robot. We also introduce the concept of social presence and the important bidirectional social cues it needs, including eye contact, facial expression, posture, gestures, and social touch. For all these multisensory and social cues, we explain how they can be implemented in a telepresence system and describe our solution consisting of a closed control pod and a humanoid telepresence robot.

This paper presents a proof-of-concept of a robotic teleoperation system, that provides the human operator a thermal sense in addition to the visual sense. With a sensor suite comprising a stereo camera, 360° camera and long-wave infra-red camera, our demonstrator pushes the boundaries of virtual-reality situational awareness by bringing not only 3D visual content but also a 360° thermal experience to the operator. The visual channel of our robotic teleoperation system is represented through a head-mounted-display and the thermal channel is displayed through directional heaters in the operator cockpit and a thermal glove. Initial tests showed that an operator successfully experienced a 360° remote environment, correctly distinguished between and interacted with hot and cold objects, and could notice the presence of nearby people outside her direct field-of-view, based on their emitted heat.

During the COVID-19 pandemic, many nursing homes had to restrict visitations. This had a major negative impact on the wellbeing of residents and their family members. In response, residents and family members increasingly resorted to mediated communication to maintain social contact. To facilitate high-quality mediated social contact between residents in nursing homes and remote family members, we developed an augmented reality (AR)-based communication tool. In this study, we compared the user experience (UX) of AR-communication with that of video calling, for 10 pairs of residents and family members. We measured enjoyment, spatial presence and social presence, attitudes, behavior and conversation duration. In the AR-communication condition, residents perceived a 3D projection of their remote family member onto a chair placed in front of them. In the video calling condition, the family member was shown using 2D video. In both conditions, the family member perceived the resident in the video calling mode on a 2D screen. While residents reported no differences in their UX between both conditions, family members reported higher spatial presence for the AR-communication condition compared to video-calling. Conversation durations were significantly longer during AR-communication than during video calling. We tentatively suggest that there may be (unconscious) differences in UX during AR-based communication compared to video calling.

Social VR shall allow natural communication between users with high social presence, as if users are in the same room. One way to increase social presence is to add haptic interaction to allow, for example, users to give each other a ”high-five” or to pass documents among them. In this paper, we present our web-based VR communication framework with an added haptic component to simulate touch. The goal of this framework is to enhance the VR communication experience and the social cues exchange between users in VR. We describe our method for rendering haptic feedback within the web-based framework and evaluate the perceived quality of our system with a user survey (with 119 participants). Our proof-of-concept system was rated positively, with the haptic component offering an enhanced quality of the VR experience for 78% of the participants.

High extraversion and conscientiousness and low neuroticism predict successful performance during and after stressful conditions. We investigated whether these personality factors are linked to stress sensitivity and to baseline physiology. Stress was induced through negative feedback on gaming performance. Stress sensitivity was determined as the difference in baseline physiological variables (skin conductance, heart rate and heart rate variability) before and after performing the game, as well as the difference in subjectively reported stress. While physiological results suggest that the game indeed induced stress, subjective reports do not. Maybe due to a low level of experienced stress, stress sensitivity (as indicated by the difference in heart rate) only correlates with conscientiousness and not with extraversion or neuroticism. The baseline measurements show the expected correlations between extraversion and both heart rate and heart rate variability—negative and positive respectively. The negative correlation between neuroticism and skin conductance is opposite to what we expected. While the exact mechanisms are not clear yet, the present results indicate that for healthy individuals, there are indeed measurable and consistent relations between physiology and personality. Hence, physiological indicators of personality may ultimately be of value as predictors of stress resiliency.

We tested how non-experts judge point probability for seven different visual representations of uncertainty, using a case from an unfamiliar domain. Participants (n = 140) rated the probability that the boundary between two earth layers passed through a given point, for seven different visualizations of the positional uncertainty of the boundary. For all types of visualizations, most observers appear to construct an internal model of the uncertainty distribution that closely resembles a normal distribution. However, the visual form of the uncertainty range (i.e., the visualization type) affects this internal model and the internal model relates to participants' numeracy. We conclude that perceived certainty is affected by its visual representation. In a follow-up experiment we found no indications that the absence (or presence) of a prominent center line in the visualization affects the internal model. We discuss if and how our results inform which visual representation is most suitable for representing uncertainty and make suggestions for future work.

Touch is an essential channel in interpersonal and affective communication, yet most social agents currently lack the capability to touch the user. In this paper we show the credibility of three premises that make the case that providing touch capability to social robots will increase their effectiveness in communicating emotions, building trust and achieving behavioral changes. The first premise is that humans can communicate distinct emotions through touch only, the second is that this is also possible through mediated (virtual) touch, and the third is that social agents can use the same mediated touch technology as effectively as humans. Based on a literature review, we also formulate ten design rules as guidance for the development of social agents that can touch. These rules concern parameters that regulate the meaning of touch cues like context and familiarity, the implicit and explicit meanings of touch, user characteristics, and parameters that can be communicated through affective touch.

We present the design of a haptic computer accessory that adds the experience of touch to online conversations. It enables dyads to exchange the physical experience of holding hands regardless of distance and location. Text, speech and video communication systems support the transmission of several nonverbal affective aspects of communication but not the impression of physical proximity which is essential to experience a more intimate conversation. In real life touching hands is an important empathic experience for dyads that elicits a strong sense of togetherness, while signaling trust, understanding and social support. We analyzed hand-holding behavior of parent-child dyads and classified these into five categories implemented by the device: (1) child holding parent's finger, (2) fingers on fingers (both on the inside), (3) palm on palm, (4) palm on outside of hand, and (5) child holding parent's fingers and palm. For each hand holding position there are distinct sending, receiving and neutral contact points. Adding the haptic sensation of holding hands to online conversations may foster a sense of proximity or "togetherness". Pilot user tests with prototypes support this concept and provide new insights for future developments.

The predictions of 13 computational bottom-up saliency models and a newly introduced Multiscale Contrast Conspicuity(MCC) metric are compared with human visual conspicuity measurements. The agreement between human visual conspicuity estimates and model saliency predictions is quantified through their rank order correlation. The maximum of the computational saliency value over the target support area correlates most strongly with visual conspicuity for 12 of the 13 models. A simple multiscale contrast model and the MCC metric both yield the largest correlation with human visual target conspicuity (>0:84). Local image saliency largely determines human visual inspection and interpretation of static and dynamic scenes. Computational saliency models therefore have a wide range of important applications, like adaptive content delivery, region-of-interest-based image compression, video summarization, progressive image transmission, image segmentation, image quality assessment, object recognition, and content-aware image scaling. However, current bottom-up saliency models do not incorporate important visual effects like crowding and lateral interaction. Additional knowledge about the exact nature of the interactions between the mechanisms mediating human visual saliency is required to develop these models further. The MCC metric and its associated psychophysical saliency measurement procedure are useful tools to systematically investigate the relative contribution of different feature dimensions to overall visual target saliency.

We developed a fast and efficient method to derive and apply a natural colour mapping for nighttime imagery from multi-band sensors. The colour mapping is derived from the combination of a multi- band image and a corresponding natural colour reference image. The mapping optimizes the match between the multi-band image and the reference image, and yields a night vision image with a colour distribution similar to that of the reference image. The actual mapping procedure is extremely simple and fast, and once it has been derived the colour mapping can be deployed in realtime to other multi-band images. Displaying night-time imagery in natural colours may help human observers to process this type of imagery faster and better, thereby improving situational awareness and reducing reaction times.

A number of segmentation techniques are compared with regard to their usefulness for region-based image and video fusion. In order to achieve this, a new multi-sensor data set is introduced containing a variety of infra-red, visible and pixel fused images together with manually produced "ground truth" segmentations. This enables the objective comparison of joint and unimodal segmentation techniques. A clear advantage to using joint segmentation over unimodal segmentation, when dealing with sets of multi-modal images, is shown. The relevance of these results to region-based image fusion is confirmed with task-based analysis and a quantitative comparison of the fused images produced using the various segmentation algorithms

Subjects used the dichoptic combination of a monocular image intensifier (NVG) and a monocular uncooled microbolometer (LWIR) to detect and localize both visual targets and camouflaged thermal targets while moving through a dimly lit complex environment. The NVG imagery enabled the subjects to move freely through the environment with high accuracy, but did not mediate the detection of camouflaged thermal targets. The LWIR mode mediated the detection of camouflaged thermal targets but did not allow the detection of visual targets, and provided insufficient detail to allow accurate movement through the environment Subjects were quite capable to dichoptically fuse the individual LWIR and NVG images, enabling them to detect all (visual and thermal) targets while moving accurately through the environment We conclude that dichoptic fusion of NVG and LWIR imagery is quite feasible and is a simple way to provide observers with enhanced situational awareness in nighttime operations

A technique that is functionally equivalent to the oriented smoothing concept and reduces numerical complexity and computational costs by eliminating the smoothness requirement from the iteration process is introduced. Local affine transformations are applied to propagate uniquely computed flow vectors into homogeneous regions and along edges in a single step. The window within which the local affine transformation is performed can adapt to the local structure of the intensity pattern in accordance with the oriented smoothness concept as formulated by H. Nagel (1987).>