Waveforms extracted via nasal thermistors are the most common signals used to study breathing function in sleep studies. In recent years, unobtrusive alternatives have been developed based on thermal imaging. Initially, the research aimed to produce a measurement on par with the clinical standard (the thermistor), but at a distance. Lately, there has been recognition that imaging is inherently multidimensional and can produce spatiotemporal and not just temporal signals - a development with significant diagnostic value. The extraction of 3D breathing information, however, has been based on inaccurate assumptions regarding the formation of the nasal thermal patterns sensed by the camera. The present paper corrects these assumptions, enabling the production of more accurate and complete multidimensional breathing signals.

Electrodermal activity (EDA) is an important affective indicator, measured conventionally on the fingers with desktop sensing instruments. Recently, a new generation of wearable, battery-powered EDA devices came into being, encouraging the migration of EDA sensing to other body locations. To investigate the implications of such sensor/location shifts in psychophysiological studies we performed a validation experiment. In this experiment we used startle stimuli to instantaneously arouse the sympathetic system of n=23 subjects while sitting. Startle stimuli are standard but minimal stressors, and thus ideal for determining the sensor and location resolution limit. The experiment revealed that precise measurement of small EDA responses on the fingers and palm is feasible either with conventional or mobile EDA sensors. By contrast, precise measurement of small EDA responses on the sole is challenging, while on the wrist even detection of such responses is problematic for both EDA modalities. Given that affective wristbands have emerged as the dominant form of EDA sensing, researchers should beware of these limitations.

This study explored various user interface designs to transition a two dimensional (2D) questionnaire from its paper-and-pencil testing format to the mobile platform. The current administration of the test limits its usage beyond the lab environment. Creating a mobile version would facilitate ubiquitous administration of the test. Yet, the mobile design must be at least as good as its paper-based counterpart in terms of input accuracy and user interaction efforts. We developed four user interface designs, each of which featured a specific interaction approach. These approaches included displaying the 2D space of the questionnaire in its original form (M1), inputting one variable at a time on the 2D space (M2), dissolving the 2D space into two one-dimensional ordinal scales (M3), and orienting the input selections to the diagonal axes (M4). The designs were tested by a total of 34 participants, aged 18 to 52 years. The study results find the first three interaction approaches (M1-M3) effective but the fourth approach inefficient. Furthermore, the results indicate that the two-tap designs (M2 and M3) are equally as good as the one-tap design (M1).

In this paper, we present a novel framework for quantifying physiological stress at a distance via thermal imaging. The method captures stress-induced neurophysiological responses on the perinasal area that manifest as transient perspiration. We have developed two algorithms to extract the perspiratory signals from the thermophysiological imagery. One is based on morphology and is computationally efficient, while the other is based on spatial isotropic wavelets and is flexible; both require the support of a reliable facial tracker. We validated the two algorithms against the clinical standard in a controlled lab experiment where orienting responses were invoked on n=18 subjects via auditory stimuli. Then, we used the validated algorithms to quantify stress of surgeons (n=24) as they were performing suturing drills during inanimate laparoscopic training. This is a field application where the new methodology shines. It allows nonobtrusive monitoring of individuals who are naturally challenged with a task that is localized in space and requires directional attention. Both algorithms associate high stress levels with novice surgeons, while low stress levels are associated with experienced surgeons, raising the possibility for an affective measure (stress) to assist in efficacy determination. It is a clear indication of the methodology's promise and potential.

Unlike machines, we humans are prone to boredom when we perform routine activities for long periods of time. Workers' mental engagement in boring tasks diminishes, which eventually, compromises their performance. The result is a double-whammy because the workers do not get job satisfaction and their employers do not receive optimal return on investment. This paper proposes a novel way for improving workers' mental engagement and hence, enjoyment, in routine activities. Specifically, we propose to blend in routine tasks mild mental/physical challenges. To test our hypothesis, we chose to experiment on a monitoring task typical of security guard operations. We combined this routine task with an iPhone-based game to make it more enjoyable. The results from 10 participants show that their mental engagement and enjoyment were significantly higher during the combined task.

User intervention in the periorbital thermal signal extraction process breaks down automation. This paper proposes a novel way to minimize user intervention. While previous work demonstrated the importance of accurate computation of the periorbital signal, the present method enables its automatic extraction at a reduced processing time. The proposed algorithm capitalizes on detection of involuntary eye blinking in the thermal imagery. The need for automation has emerged because of repetitive processing of the same subjects, aiming to validate improvements in the periorbital tissue tracking or segmentation algorithms. The proposed approach initiates the tracking and segmentation algorithms on the same spatio-temporal location in repetitive runs of the thermal clip. Thus, it does not only automate the process but also eliminates the variability introduced by manual intervention. We have tested the algorithm on thermal video clips of 39 subjects who faced stressful interrogation for a mock crime. The results show that the proposed method has reduced total processing time from a week down to a day.

One of the major challenges of video game design is to have appropriate difficulty levels for users in order to maximize the entertainment value of the game. Game players may lose interests if a game is either too easy or too difficult. This paper presents a novel methodology to improve user's experience in computer games by automatically adjusting the level of the game difficulty. The difficulty level is computed from measurements of the facial physiology of the players at a distance. The measurements are based on the assumption that the players' performance during the game-playing session alters blood flow in the supraorbital region, which is an indirect measurement of increased mental activities. This alters heat dissipation, which can be monitored in a contact-free manner through a thermal imaging-based stress monitoring and analysis system, known as StressCam.

An artificial vision system is presented for lie detection by analyzing face thermal image sequences.This system represents an alternative technique to the polygraph.Some of its features are: 1) it has no physical contact with the examinee, 2) it is non-intrusive, 3) it has a potential for private use, and 4) it can simultaneously analyze several persons.The proposed system is based on the detection of physiological changes in temperature in the lacrimal puncta area caused by the subtle increase in blood flow through the nearby vascular network.These changes take place when anxiety appears as a consequence of deception.Thus, the system segments the periorbital area, and tracks consecutive frames using the Kanade-Lucas-Tomasi algorithm.The results show a success rate of 79.2 % in detecting lies using a simple classification based on the comparison between the estimated temperatures in control questions, and the rest of the interrogation procedure.The performance of this system is comparable with previous works, where cameras with better specifications were used.