• First study to identify physiological correlates of social disconnection in the real-world. • Detected mood-related general social connectedness using wearable devices. • Developed a social density estimation model of people within a 5m vicinity of participants. • Detected social connectedness to people nearby. • Classification models detected correlates of social connection using physiological features from wearable devices. Loneliness, low mood, and social disconnection can have damaging effects on physical and mental health. Detecting these emotions in the context of everyday life is important as these psychological states can manifest differently outside the laboratory. Lifelogging and quantified self technologies, including wearable devices, offer an approach to continuously monitor physiological signals in the real-world. However, little work has been undertaken with these devices to detect social connection in everyday life. This paper presents a study that leveraged machine learning to infer mood and social connectedness in everyday life using multimodal lifelogging data collected via a wrist-worn wearable device. Fifty participants were supplied with a wearable device and smartphone that collected physiological and subjective data across two consecutive weekdays as they went about their daily lives. The analysis examined physiological correlates between a person’s psychological perceptions of general connectedness, as well as feelings of in-the-moment social connection to people within an estimated 5m vicinity and mood at specific timepoints using four machine learning classification models – k-Nearest Neighbour, Random Forest, Support Vector Machine and Naïve Bayes. Results demonstrated that Random Forest obtained the highest accuracy of 0.8 – 0.84 for the binary detection of mood and social connection in everyday life.

Stress is a negative emotion that is part of everyday life. However, frequent episodes or prolonged periods of stress can be detrimental to long-term health. Nevertheless, developing self-awareness is an important aspect of fostering effective ways to self-regulate these experiences. Mobile lifelogging systems provide an ideal platform to support self-regulation of stress by raising awareness of negative emotional states via continuous recording of psychophysiological and behavioral data. However, obtaining meaningful information from large volumes of raw data represents a significant challenge because these data must be accurately quantified and processed before stress can be detected. This work describes a set of algorithms designed to process multiple streams of lifelogging data for stress detection in the context of real world driving. Two data collection exercises have been performed where multimodal data, including raw cardiovascular activity and driving information, were collected from 21 people during daily commuter journeys. Our approach enabled us to 1) pre-process raw physiological data to calculate valid measures of heart rate variability, a significant marker of stress, 2) identify/correct artefacts in the raw physiological data, and 3) provide a comparison between several classifiers for detecting stress. Results were positive and ensemble classification models provided a maximum accuracy of 86.9 percent for binary detection of stress in the real-world.

The accurate classification of psychophysiological data is an important determinant of the quality when interacting with a physiological computing system. Previous research has focused on classification accuracy of psychophysiological data in purely mathematical terms but little is known about how accuracy metrics relate to users' perceptions of accuracy during real-time interaction. A group of 14 participants watched a series of movie trailers and were asked to subjectively indicate their level of interest in a binary high/low fashion. Psychophysiological data (EEG, ECG and SCL) were used to create a binary classification of interest via a Support Vector Machine (SVM) algorithm. After a period of training, participants received real-time feedback from the classification algorithm and perceptions of accuracy were assessed. The purpose of the study was to compare mathematical classification accuracy with the perceived accuracy of the system as experienced by the users. Results indicated that perceived accuracy was subject to a number of psychological biases resulting from expectations, entrainment and development of trust. The F1 score was generally a significant predictor of perceived accuracy.

The prospect of connecting the brain and body to a technological device can elicit a broad range of responses from potential users. Early adopters are thrilled by the possibility of a device that can interface directly to the human nervous system. For the vast majority, interest is tempered by caution, as nascent varieties of physiological computing systems raise as many questions as answers about how we will interact with computers in the future.

This article presents a psychophysiological construct of interest as a knowledge emotion and illustrates the importance of interest detection in a cultural heritage context. The objective of this work is to measure and classify psychophysiological reactivity in response to cultural heritage material presented as visual and audio. We present a data processing and classification framework for the classification of interest. Two studies are reported, adopting a subject-dependent approach to classify psychophysiological signals using mobile physiological sensors and the support vector machine learning algorithm. The results show that it is possible to reliably infer a state of interest from cultural heritage material using psychophysiological feature data and a machine learning approach, informing future work for the development of a real-time physiological computing system for use within an adaptive cultural heritage experience designed to adapt the provision of information to sustain the interest of the visitor.

Physiological data in its different dimensions—bioelectrical, biomechanical, biochemical, or biophysical—and collected through existing sensors or specialized biomedical devices, image capture, or other sources is pushing the boundaries of physiological computing for human-computer interaction (HCI). Although physiological computing shows the potential to enhance the way in which people interact with digital content, systems remain challenging to design and build. The aim of this special issue is to present outstanding work related to use of physiological data in HCI, setting additional bases for next-generation computer interfaces and interaction experiences. Topics covered in this issue include methods and methodologies, human factors, the use of devices, and applications for supporting the development of emerging interfaces.

The biocybernetic loop describes the data processing protocol at the heart of all physiological computing systems. The loop also encompasses the goals of the system design with respect to the anticipated impact of the adaptation on user behaviour. There are numerous challenges facing the designer of a biocybernetic loop in terms of measurement, data processing and adaptive design. These challenges are multidisciplinary in nature spanning psychology and computer science. This paper is concerned with the design process of the biocybernetic loop. A number of criteria for an effective loop are described followed by a six-stage design cycle. The challenges faced by the designer at each stage of the design process are exemplified with reference to a case study where EEG data were used to adapt a computer game.

BCIs are traditionally conceived as a way to control apparatus, an interface that allows you to "act on" external devices as a form of input control. We propose an alternative use of BCIs, that of monitoring users as an additional intelligent sensor to enrich traditional means of interaction. This vision is what we consider to be a grand challenge in the field of multimodal interaction. In this article, this challenge is introduced, related to existing work, and illustrated using some best practices and the contributions it has received.

Adaptive behavior is one of the main challenges in building computerized systems, especially in the case of systems which are delivering information to the end users. Indeed, since the information overload has become the main drawback for the future development of such systems (cf. Big Data challenge), there is a huge movement in the research community to develop concepts for better adaptation of the form and size of information that will be delivered to a user (usually taking different forms of the personalization). However, the main effort has been dedicated to the contextualization of the user's task in order to determine what is the best way to tailor/adapt the presentation of information to the user, neglecting the role of the user's internal context, expressed as the user's (short-term) interest. The same is valid for the AR systems. In this tutorial we present novel results in modeling users' interest in the context of AR systems and demonstrate some practical results in realizing such an approach in a multisensor AR system based on the usage of the see-through AR glasses. Due to the need for continuously adapt the AR content to the user's interest, such models are facing many challenges in sensing the user's behavior (using acoustic-, video-, gesture- and bio-sensors), interpreting it as an interest and deciding in real-time what kind of the adaptation to perform. We argue that this lead to a new class of AR system that we coined as adaptive AR (AR) systems. This work has been partially realized within the scope of the FP7 ICT research project ARtSENSE (www.artsense.eu), that is developing new AR concepts for improving personalized museum's experience. The tutorial will present practical results from applying the approach in three cultural heritage institutions in Europe (Paris, Madrid and Liverpool).

Pervasive adaptive systems are concerned with the construction of “smart” technologies capable of adapting to the needs of the individual in real time. In order to achieve this level of specificity, systems must be capable of monitoring the psychological status of the user and responding to these changes in real time and across multiple systems if necessary. This article describes a number of conceptual issues associated with this category of adaptive technology. The biocybernetic loop describes different approaches to monitoring the status of the user from physiological sensors to overt behavior. These data are used to drive real time system adaptation tailored to a specific user in a particular context. The rate at which the technology adapts to the individual user are described over three different phases of usage: awareness (short-term), adjustment (medium-term), and coevolution (long-term). An ontology is then proposed for the development of an adaptive software architecture that embodies this approach and may be extended to encompass several distinct loops working in parallel. The feasibility of the approach is assessed through implemented case studies of their performance and functionality.

Developing systems that support people in everyday life in a discrete and effective way is an ultimate goal of a new generation of technical systems. Physiological computing represents one means of creating a system to sense the user, analyse users' responses to system adaptation and respond dynamically. This process of adaptation is achieved by creating a biocybernetic loop that may operate on several, simultaneous timescales (minutes/hours/weeks/ months/years). In terms of architecture, it is argued that a ldquosense-analyse-reactrdquo system requires middleware with closed-loop control consisting of: (1) a tangible layer concerned with sensors and actuators, (2) a reflective layer containing a flexible representation of the user to guide system adaptation, and (3) an application layer representing application scenarios and the context for adaptation and evolution.

This review paper is concerned with the development of physiological computing systems that employ real-time measures of psychophysiology to communicate the psychological state of the user to an adaptive system. It is argued that physiological computing has enormous potential to innovate human–computer interaction by extending the communication bandwidth to enable the development of ‘smart’ technology. This paper focuses on six fundamental issues for physiological computing systems through a review and synthesis of existing literature, these are (1) the complexity of the psychophysiological inference, (2) validating the psychophysiological inference, (3) representing the psychological state of the user, (4) designing explicit and implicit system interventions, (5) defining the biocybernetic loop that controls system adaptation, and (6) ethical implications. The paper concludes that physiological computing provides opportunities to innovate HCI but complex methodological/conceptual issues must be fully tackled during the research and development phase if this nascent technology is to achieve its potential.

Physiological computing involves the direct interfacing of human physiology and computer technology, i.e. brain–computer interaction (BCI). The goal of physiological computing is to transform bioelectrical signals from the human nervous system into real-time computer input in order to enhance and enrich the interactive experience. Physiological computing has tremendous potential for interactive innovation but research activities are often disparate and uneven, and fail to reflect the multidisciplinary nature of the topic. This paper will provide a primer on detectable human physiology as an input source, a summary of relevant research and a research agenda to aid the future development of interactive systems that utilise physiological information.