This study examines eyeblink synchronization in interactions characterized by mutual gaze without task-related or conversational elements that can trigger similarities in visual, auditory, or cognitive processing. We developed a study design capable of isolating the role of gaze in human-human interaction and observed the blinking behavior of dyads with mobile eye tracking glasses under three conditions: face-to-face mutual gaze, mediated mutual gaze through a mirror, and self-directed gaze in a mirror. The results revealed that when the interaction was through direct mutual gaze, eyeblink synchronization increased concurrently with a more structured temporal pattern. Also, the sense of connection between partners mimicked the synchronization. These findings suggest that even minor deviations caused by mediated interaction lead to reduced synchronization and a weakened sense of connection among partners. The paper also discusses the need for methodologies to enhance the efficacy and authenticity of online environments and human-robot interaction.

Recognition of deceptive intentions from the eyes has been of appealing interest in the last decades but is still unresolved. Here, we report the development of a paradigm based on the Concealed Information Test enabling the study of various kinds of deception, that is, faking and concealing. Based on a card game, we compared fixation as well as pupil behaviour while participants were instructed to fake, conceal, or tell the truth. We realized two different layouts of stimulus presentation. Fixations differed between concealing versus faking and telling the truth; pupil size additionally unveiled the object of deception. We infer that various kinds of deceptive behaviour must be carefully distinguished, and contribute to how to use gaze measures as indicators of deceptive intentions.

The role of gaze in communication is diverse and of the essence, particularly thanks to its dual function which both encodes and signals mental and emotional states, and the direction and object of attention. To date, substantial research on the attentional and perceptual effects of gaze behavior has established four main forms of social eye gaze, and each form plays a role in establishing, grounding, and ensuring communication between interlocutors. However, while the perceptual effects of gaze on attentional and cognitive processes are well-elaborated, social and affective effects of gaze and, particularly, mutual gaze attracted little attention. The present review suggests that the mutual gaze has a unique role in social interactions since it has both perceptual and affective effects in social interaction, and the unique affective and social role of mutual gaze in real-time and naturalistic settings still needs further research.

Recent studies of eye movements in faking behaviours are reviewed. The best candidates to reveal deceptive intentions are the number and duration of fixations, while saccade amplitudes and pupil size show great potential. Special consideration about experimental design and individual abilities that could impact the physiological responses should be taken when studying faking intentions.

Interacting with a group of people requires to direct the attention of the whole group, thus requires feedback about the crowd’s attention. In face-to-face interactions, head and eye movements serve as indicator for crowd attention. However, when interacting online, such indicators are not available. To substitute this information, gaze visualizations were adapted for a crowd scenario. We developed, implemented, and evaluated four types of visualizations of crowd attention in an online study with 72 participants using lecture videos enriched with audience’s gazes. All participants reported increased connectedness to the audience, especially for visualizations depicting the whole distribution of gaze including spatial information. Visualizations avoiding spatial overlay by depicting only the variability were regarded as less helpful, for real-time as well as for retrospective analyses of lectures. Improving our visualizations of crowd attention has the potential for a broad variety of applications, in all kinds of social interaction and communication in groups.

In online lectures, showing an on-screen instructor gained popularity amidst the Covid-19 pandemic. However, evidence in favor of this is mixed: they draw attention and may distract from the content. In contrast, using signaling (e.g., with a digital pointer) provides known benefits for learners. But effects of signaling were only researched in absence of an on-screen instructor. In the present explorative study, we investigated effects of an on-screen instructor on the division of learners´ attention; specifically, on following a digital pointer signal with their gaze. The presence of an instructor led to an increased number of fixations in the presenter area. This did neither affect learning outcomes nor gaze patterns following the pointer. The average distance between the learner's gaze and the pointer position predicts the student's quiz performance, independent of the presence of an on-screen instructor. This can also help in creating automated immediate-feedback systems for educational videos.

Webcam-based eye-tracking promises easy and quick data collection without the need for specific or additional eye-tracking hardware. This makes it especially attractive for educational research, in particular for modern formats, such as MOOCs. However, in order to fulfill its promises, webcam-based eye tracking has to overcome several challenges, most importantly, varying spatial and temporal resolutions. Another challenge that the educational domain faces especially, is that typically individual students are of interest in contrast to average values. In this paper, we explore whether an attention measure that is based on eye movement synchronicity of a group of students can be applied with unreliably-sampled data. Doing so we aim to reproduce earlier work that showed that, on average, eye movement synchronicity can predict performance in a comprehension quiz. We were not able to reproduce the findings with unreliably-sampled data, which highlights the challenges that lie ahead of webcam-based eye tracking in practice.

Physiological responses are generally involuntary; however, real-time feedback enables, at least to a certain extent, to voluntary control automatic processes. Recently, it was demonstrated that even pupil dilation is subject to controlled interference. To address effects of training on the ability to exercise control on pupil dilation, the current study examines repeated exercise over seven successive days. Participants utilize self-induced changes in arousal to increase pupil diameter, real-time feedback was applied to evaluate and improve individual performance. We observe inter-individual differences with regard to responsiveness of the pupillary response: six of eight participants considerably increase pupil diameter already during the first session, two exhibit only slight changes, and all showed rather stable performance throughout training. There was a trend towards stronger peak amplitudes that tend to occur increasingly early across time. Hence, higher cognitive control on pupil dilations can be practiced by most users and may therefore provide an appropriate input mechanism in human-computer interaction.

Previous research suggests that self-relevant stimuli, such as one's own name, attract more attention than stimuli that are not self-relevant. In two experiments, we examined to which extent the own name is also less prone to inhibition than other names using a Go/NoGo approach. The pupil diameter was employed as psychophysiological indicator of attention. A total of 36 subjects performed various categorization tasks, with their own name and other names. Whereas in Go-trials, pupil dilation for own and other names did not differ, in NoGo-trials, significant larger pupil dilations were obtained for subjects' own names compared to other names. This difference was especially pronounced at larger intervals after stimulus onset, suggesting that inhibitory processing was less effective with one's own name.

Video-based gaze tracking is prone to brightness changes due to their effects on pupil size. Monocular observations indeed confirm variable fixation locations depending on brightness. In close viewing, pupil size is coupled with accommodation and vergence, the so-called near triad. Hence, systematic changes in fixation disparity might be expected to co-occur with varying pupil size. In the current experiment, fixation disparity was assessed. Calibration was conducted either on dark or on bright background, and text had to be read on both backgrounds, on a self-illuminating screen and on paper. When calibration background matches background during reading, mean fixation disparity did not differ from zero. In the non-calibrated conditions, however, a brighter stimulus went along with a dominance of crossed fixations and vice versa. The data demonstrate that systematic changes in fixation disparity occur as effect of brightness changes advising for careful setting calibration parameters.

Exploration in large multimedia collections is challenging because the user often navigates into misleading directions or information areas. The vision of our project is to develop an assistive technology that is able to support the individual user and enhance the efficiency of an ongoing exploratory search. Such a technical search aid should be able to find out about the user's current interests and goals. Respective parameters can be found in the central and in the peripheral nervous system as well as in overt behavior. Therefore, we aim at using eye movements, pupillometry and EEG to assess respective information. Here, we describe the set-up and the first results of a preliminary user study investigating the effects of searching an image collection on eye movements and pupil dilations. First data show that numbers of fixation, fixation durations as well as pupil dilations differ systematically when looking at a subsequently selected target as compared with not selected items. These results support our vision that further research additionally investigating EEG can in fact result in better predicting the searchers goals and next choices.

Modern interaction techniques like non-intrusive gestures provide means for interacting with distant displays and smart objects without touching them. We were interested in the effects of feedback modality (auditory, haptic or visual) and its combined effect with input modality on user performance and experience in such interactions. Therefore, we conducted two exploratory experiments where numbers were entered, either by gaze or hand, using gestures composed of four stroke elements (up, down, left and right). In Experiment 1, a simple feedback was given on each stroke during the motor action of gesturing: an audible click, a haptic tap or a visual flash. In Experiment 2, a semantic feedback was given on the final gesture: the executed number was spoken, coded by haptic taps or shown as text. With simultaneous simple feedback in Experiment 1, performance with hand input was slower but more accurate than with gaze input. With semantic feedback in Experiment 2, however, hand input was only slower. Effects of feedback modality were of minor importance; nevertheless, semantic haptic feedback in Experiment 2 showed to be useless at least without extensive training. Error patterns differed between both input modes, but again not dependent on feedback modality. Taken together, the results show that in designing gestural systems, choosing a feedback modality can be given a low priority; it can be chosen according to the task, context and user preferences.

Multimodal systems still tend to ignore the individual input behavior of users, and at the same time, suffer from erroneous sensor inputs. Although many researchers have described user behavior in specific settings and tasks, little to nothing is known about the applicability of such information, when it comes to increase the robustness of a system for multimodal inputs. We conducted a gamified experimental study to investigate individual user behavior and error types found in an actually running system. It is shown, that previous ways of describing input behavior by a simple classification scheme (like simultaneous and sequential) are not suited to build up an individual interaction history. Instead, we propose to use temporal distributions of different metrics derived from multimodal event timings. We identify the major errors that can occur in multimodal interactions and finally show how such an interaction history can practically be applied for error detection and recovery. Applying the proposed approach to the experimental data, the initial error rate is reduced from 4.9% to a minimum of 1.2%.

Controlling computers using eye movements can provide a fast and efficient alternative to the computer mouse. However, implementing object selection in gaze-controlled systems is still a challenge. Dwell times or fixations on a certain object typically used to elicit the selection of this object show several disadvantages. We studied deviations of critical thresholds by an individual and task-specific adaptation method. This demonstrated an enormous variability of optimal dwell times. We developed an alternative approach using antisaccades for selection. For selection by antisaccades, highlighted objects are copied to one side of the object. The object is selected when fixating to the side opposed to that copy requiring to inhibit an automatic gaze shift toward new objects. Both techniques were compared in a selection task. Two experiments revealed superior performance in terms of errors for the individually adapted dwell times. Antisaccades provide an alternative approach to dwell time selection, but they did not show an improvement over dwell time. We discuss potential improvements in the antisaccade implementation with which antisaccades might become a serious alternative to dwell times for object selection in gaze-controlled systems.

Eye typing could provide motor disabled people a reliable method of communication given that the text entry speed of current interfaces can be increased to allow for fluent communication. There are two reasons for the relatively slow text entry: dwell time selection requires waiting a certain time, and single character entry limits the maximum entry speed. We adopted a typing interface based on hierarchical pie menus, pEYEwrite [Urbina and Huckauf 2007] and included bigram text entry with one single pie iteration. Therefore, we introduced three different bigram building strategies. Moreover, we combined dwell time selection with selection by borders, providing an alternative selection method and extra functionality. In a longitudinal study we compared participants performance during character-by-character text entry with bigram entry and with text entry with bigrams derived by word prediction. Data showed large advantages of the new entry methods over single character text entry in speed and accuracy. Participants preferred selecting by borders, which allowed them faster selections than the dwell time method.

Pie menus offer several features which are advantageous especially for gaze control. Although the optimal number of slices per pie and of depth layers has already been established for manual control, these values may differ in gaze control due to differences in spatial accuracy and congitive processing. Therefore, we investigated the layout limits for hierarchical pie menu in gaze control. Our user study indicates that providing six slices in multiple depth layers guarantees fast and accurate selections. Moreover, we compared two different methods of selecting a slice. Novices performed well with both, but selecting via selection borders produced better performance for experts than the standard dwell time selection.

The potential of Augmented Reality (AR) to support industrial processes has been demonstrated in several studies. While there have been first investigations on user related issues in the long-duration use of mobile AR systems, to date the impact of theses systems on physiological and psychological aspects is not explored extensively. We conducted an extended study in which 19 participants worked 4 hours continuously in an order picking process with and without AR support. Results of the study comparing strain and work efficiency are presented and open issues are discussed.

Various interfaces for gaze control (which are recommended due to certain requirements of controlling a machine by gaze) have already been developed. One problem, especially for novice users, is that respective interfaces all look different and require different steps to use. As a means to unify interfaces for gaze control, pie menus are suggested. Such pEYEs allow for universal input in various applications usable by novices and by experts. We present two examples for pEYE interfaces; one eye-typing application and one desktop navigation. Observations in user studies indicate effective and efficient performance and a large acceptance.

Augmented Reality (AR) uses computer-generated virtual information to enhance the user’s information access. While numerous previous studies have demonstrated the large potential of AR to improve industrial processes by enhancing product quality and reducing production times it is still unclear if and how long term usage of such AR technology produces stress and strain. This paper presents an approach to use the analysis of Heart Rate Variability to objectively measure current user strain during different work tasks. Results of a user study comparing strain during an AR supported and a non-AR supported work task in a laboratory setting are presented and discussed.

We introduce two new six degree of freedom desktop input devices based on the key concept of combining forceless isotonic rotational input with force-requiring elastic translational input. The GlobeFish consists of a custom three degrees of freedom trackball which is elastically connected to a frame. The trackball is accessible from the top and bottom and can be moved slightly in all spatial directions by using force. The GlobeMouse device works in a similar way. Here the trackball is placed on top of a movable base, which requires to change the grip on the device to switch between rotating the trackball and moving the base.Our devices are manipulated with the fingertips allowing precise interaction with virtual objects. The elastic translation allows uniform input for all three axes and the isotonic trackball provides a natural mapping for rotations. Our user study revealed that the new devices perform significantly better in a docking task in comparison to the SpaceMouse, an integrated six degrees of freedom controller. Subjective data confirmed these results.