Facial micro-expressions are spontaneous, subtle, involuntary muscle movements occurring briefly on the face. The spotting and recognition of these expressions are difficult due to the subtle behavior, and the time duration of these expressions is about half a second, which makes it difficult for humans to identify them. These micro-expressions have many applications in our daily life, such as in the field of online learning, game playing, lie detection, and therapy sessions. Traditionally, researchers use RGB images/videos to spot and classify these micro-expressions, which pose challenging problems, such as illumination, privacy concerns and pose variation. The use of depth videos solves these issues to some extent, as the depth videos are not susceptible to the variation in illumination. This paper describes the collection of a first RGB-D dataset for the classification of facial micro-expressions into 6 universal expressions: Anger, Happy, Sad, Fear, Disgust, and Surprise. This paper shows the comparison between the RGB and Depth videos for the classification of facial micro-expressions. Further, a comparison of results shows that depth videos alone can be used to classify facial micro-expressions correctly in a decision tree structure by using the traditional and deep learning approaches with good classification accuracy. The dataset will be released to the public in the near future.

Neurochemical systems are well studied in animal learning; however, ethical issues limit methodologies to explore these systems in humans. Pupillometry provides a glimpse into the brain's neurochemical systems, where pupil dynamics in monkeys have been linked with locus coeruleus (LC) activity, which releases norepinephrine (NE) throughout the brain. The objective of my research is to understand the role of neurochemicals in human learning. Specifically, I aim to 1) Establish a non-invasive method to study the role of neurochemicals in human learning, 2) Develop methods to monitor learning in real time using pupillometry, and 3) Discover causal relationships between neurochemicals and learning in human subjects. In this article, to address Objective 1, we present evidence that pupil dynamics can be used as a surrogate measure of neurochemical activity during learning. Specifically, we hypothesize that norepinephrine modulates the encoding of memories, the influence of which can be measured with pupil dynamics. To examine this hypothesis a task-irrelevant learning paradigm was used, in which learning is boosted for stimuli temporally paired with task targets. We show that participants better recognize images that are paired with task targets than distractors and, in correspondence, that pupil size changes more for target-paired than distractor-paired images. To further investigate the hypothesis that NE nonspecifically guides learning for stimuli that are present with its release, a second procedure was used that employed an unexpected sound to activate the LC--NE system and induce pupil-size changes; results indicated a corresponding increase in memorization of images paired with the unexpected sounds. Together, these results suggest a relationship between the LC--NE system, pupil-size changes, and learning. My ongoing work aims to develop methods to monitor learning in real time by investigating the relationship between, pupil size changes, eye movement and learning in context of a free visual search task. Future work will investigate the causal relationship between neurochemicals, learning and pupil dynamics by using NE specific drugs to up- and down-regulate levels of NE during learning.

Drawing a box around an intended segmentation target has become both a popular user interface and a common output for learning-driven detection algorithms. Despite the ubiquity of using a box to define a segmentation target, it is unclear in the literature whether a box is sufficient to define a unique segmentation or whether segmentation from a box is ill-posed without higher-level (semantic) knowledge of the intended target. We examine this issue by conducting a study of 14 subjects who are asked to segment a boxed target in a set of 50 real images for which they have no semantic attachment. We find that the subjects do indeed perceive and trace almost the same segmentations as each other, despite the inhomogeneity of the image intensities, irregular shapes of the segmentation targets and weakness of the target boundaries. Since the subjects produce the same segmentation, we conclude that the problem is well-posed and then provide a new segmentation algorithm from a box which achieves results close to the perceived target.