Autonomous vehicles (AV) are a promising contemporary engineering innovation. Since they have no human drivers, their vehicle controllers must be fully informed of the activities of their occupants in vehicle seats so that occupant safety and comfort can be guaranteed. This paper introduces a system that employs capacitive sensing and machine learning to inform the controller of occupant activities, like actions and posture changes. The system facilitates capacitive sensing by deploying a capacitance-sensing mat on the vehicle seat. A sensing circuitry connected to the mat measures all its capacitances continuously. Since an occupant’s body induces variations in these capacitances, temporal sequences of capacitance measurements represent various occupant activities in the vehicle seat. Subsequently, the system converts capacitance measurements corresponding to every time step to two grayscale capacitance-sensing images (CSIs). The CSIs, in turn, yield a feature vector corresponding to every time step, thereby building a dataset of temporal sequences of features. The system’s machine-learning unit tracks and recognizes occupant actions and posture changes using an action-recognition block, which is essentially a long short-term memory (LSTM) network trained on a dataset of temporal sequences of either features or capacitance measurements corresponding to various occupant actions. If the occupant remains stationary, a switching block in the unit disables the action-recognition block and enables a posture-recognition block. The latter is a${k}$-nearest-neighbor (${k}$NN) classifier trained on a dataset of features to recognize stationary occupant postures. This paper validates the system’s performance and investigates its deployment for real-time tracking of occupant activities in AVs.

We used a gaze-contingent eye-tracking setup to investigate how peripheral vision before the saccade affects post-saccadic foveal processing. Studies have revealed robust changes in foveal processing when the target is available in peripheral vision (the extrafoveal preview effect). To further characterize the role of peripheral vision, we adopted a paradigm where an upright/inverted extrafoveal face stimulus was shown and changed orientation (invalid preview) on 50% of trials during the saccade. Invalid preview significantly reduced post-saccadic discrimination performance compared to valid preview (aka preview effect). In addition, the preview face varied in eccentricity and added noise which affected its visibility. Face visibility was operationalized by a lateralized face identification task, run in a separate session. A mixed model analysis suggests that visibility modulated the preview effect. Overall, these findings constrain theories of how preview effects might influence perception under natural viewing conditions.

We present PET- the Pascal animal classes Eye Tracking database. Our database comprises eye movement recordings compiled from forty users for the bird, cat, cow, dog, horse and sheep trainval sets from the VOC 2012 image set. Different from recent eye-tracking databases such as [1, 2], a salient aspect of PET is that it contains eye movements recorded for both the free-viewing and visual search task conditions. While some differences in terms of overall gaze behavior and scanning patterns are observed between the two conditions, a very similar number of fixations are observed on target objects for both conditions. As a utility application, we show how feature pooling around fixated locations enables enhanced (animal) object classification accuracy.

Most artworks are explicitly created to evoke a strong emotional response. During the centuries there were several art movements which employed different techniques to achieve emotional expressions conveyed by artworks. Yet people were always consistently able to read the emotional messages even from the most abstract paintings. Can a machine learn what makes an artwork emotional? In this work, we consider a set of 500 abstract paintings from Museum of Modern and Contemporary Art of Trento and Rovereto (MART), where each painting was scored as carrying a positive or negative response on a Likert scale of 1-7. We employ a state-of-the-art recognition system to learn which statistical patterns are associated with positive and negative emotions. Additionally, we dissect the classification machinery to determine which parts of an image evokes what emotions. This opens new opportunities to research why a specific painting is perceived as emotional. We also demonstrate how quantification of evidence for positive and negative emotions can be used to predict the way in which people observe paintings.