With the advent of consumer-grade electroencephalography (EEG) and virtual reality (VR) devices, the use of human cognitive processes directly as means for user-adapted interaction in immersive virtual environments has become increasingly relevant. In this study (N = 43), we investigate electroencephalography-based neurofeedback interaction in virtual reality (VR). Particularly, we investigate this phenomenon in the context of meditation which enables the studying of cognitive processes (attention, sense of presence, meditation depth) when using an immersive interface that is adaptive based on neural responses. A prototype virtual reality environment was built that employs head-mounted display (HMD), and the neurofeedback functionality guided the user to increase the neural indices of meditation-related concentration and relaxation. The observed findings provide evidence for the effectiveness of neurofeedback functionality and VR in evoking desired types of neural activation and subjective experiences.

We explore electroencephalography (EEG), electrodermal activity (EDA), and electrocardiography (ECG) as valid sources to infer humor appraisal in a realistic environment. We report on an experiment in which 25 participants browsed a popular user-generated humorous content website while their physiological responses were recorded. We build predictive models to infer the participants’ appraisal of the humorousness of the content and demonstrate that the fusion of several physiological signals can lead to classification performances up to 0.73 in terms of the area under the ROC curve (AUC). We identify that the most discriminative changes in physiological signals happen at the later stages of the information consumption process, reflected in changes on the upper EEG frequency bands, higher levels of EDA, and heart-rate acceleration. Additionally, we present a comprehensive analysis by benchmarking the predictive power of each of the physiological signals separately, and by comparing them to state-of-the-art facial recognition algorithms based on facial video recordings. The classification performance ranges from 0.88 (in terms of AUC) when combining physiological signals and video recordings, to 0.55 when using ECG signals alone.

Meditation in general and mindfulness in particular have been shown to be useful techniques in the treatment of a plethora of ailments, yet they can be challenging for novices. We present RelaWorld: a neuroadaptive virtual reality meditation system that combines virtual reality with neurofeedback to provide a tool that is easy for novices to use yet provides added value even for experienced meditators. Using a head-mounted display, users can levitate in a virtual world by doing meditation exercises. The system measures users' brain activity in real time via EEG and calculates estimates for the level of conCentration and relaxation. These values are then mapped into the virtual reality. In a user study of 43 subjects, we were able to show that the RelaWorld system elicits deeper relaxation, feeling of presence and a deeper level of meditation when compared to a similar setup without head-mounted display or neurofeedback.

Peripheral physiological signals, as obtained using electrodermal activity and facial electromyography over the corrugator supercilii muscle, are explored as indicators of perceived relevance in information retrieval tasks. An experiment with 40 participants is reported, in which these physiological signals are recorded while participants perform information retrieval tasks. Appropriate feature engineering is defined, and the feature space is explored. The results indicate that features in the window of 4 to 6 seconds after the relevance judgment for electrodermal activity, and from 1 second before to 2 seconds after the relevance judgment for corrugator supercilii activity, are associated with the users' perceived relevance of information items. A classifier verified the predictive power of the features and showed up to 14% improvement predicting relevance. Our research can help the design of intelligent user interfaces for information retrieval that can detect the user's perceived relevance from physiological signals and complement or replace conventional relevance feedback.

Term-Relevance Prediction from Brain Signals (TRPB) is proposed to automatically detect relevance of text information directly from brain signals. An experiment with forty participants was conducted to record neural activity of participants while providing relevance judgments to text stimuli for a given topic. High-precision scientific equipment was used to quantify neural activity across 32 electroencephalography (EEG) channels. A classifier based on a multi-view EEG feature representation showed improvement up to 17% in relevance prediction based on brain signals alone. Relevance was also associated with brain activity with significant changes in certain brain areas. Consequently, TRPB is based on changes identified in specific brain areas and does not require user-specific training or calibration. Hence, relevance predictions can be conducted for unseen content and unseen participants. As an application of TRPB we demonstrate a high-precision variant of the classifier that constructs sets of relevant terms for a given unknown topic of interest. Our research shows that detecting relevance from brain signals is possible and allows the acquisition of relevance judgments without a need to observe any other user interaction. This suggests that TRPB could be used in combination or as an alternative for conventional implicit feedback signals, such as dwell time or click-through activity.

We introduce interactive intent modeling, where the user directs exploratory search by providing feedback for estimates of search intents. The estimated intents are visualized for interaction on an Intent Radar, a novel visual interface that organizes intents onto a radial layout where relevant intents are close to the center of the visualization and similar intents have similar angles. The user can give feedback on the visualized intents, from which the system learns and visualizes improved intent estimates. We systematically evaluated the effect of the interactive intent modeling in a mixed-method task-based information seeking setting with 30 users, where we compared two interface variants for interactive intent modeling, namely intent radar and a simpler list-based interface, to a conventional search system. The results show that interactive intent modeling significantly improves users' task performance and the quality of retrieved information.

Advanced interactive visualization such as in virtual environments and ubiquitous interaction paradigms pose new challenges and opportunities in considering real-time responses to subliminal cues. In this paper, we propose a synthetic reality platform that, combined with psychophysiological recordings, enables us to study in realtime the effects of various subliminal cues. We endeavor to integrate various aspects known to be relevant to implicit perception. The context is of consumer experience and choice of an artifact where the generation of subliminal perception through an intelligent 3D interface controls the spatio-temporal aspects of the information displayed and of the emergent narrative. One novel contribution of this work is the programmable nature of the interface that exploits known perceptive phenomena (e.g. masking, crowding and change blindness) to generate subliminal perception.

To understand how implicit and explicit biofeedback work in games, we developed a first-person shooter (FPS) game to experiment with different biofeedback techniques. While this area has seen plenty of discussion, there is little rigorous experimentation addressing how biofeedback can enhance human-computer interaction. In our two-part study, (N=36) subjects first played eight different game stages with two implicit biofeedback conditions, with two simulation-based comparison and repetition rounds, then repeated the two biofeedback stages when given explicit information on the biofeedback. The biofeedback conditions were respiration and skin-conductance (EDA) adaptations. Adaptation targets were four balanced player avatar attributes. We collected data with psycho¬physiological measures (electromyography, respiration, and EDA), a game experience questionnaire, and game-play measures.