Mutual Information (MI) is often used for feature selection when developing classifier models. Estimating the MI for a subset of features is often intractable. We demonstrate, that under the assumptions of conditional independence, MI between a subset of features can be expressed as the Conditional Mutual Information (CMI) between pairs of features. But selecting features with the highest CMI turns out to be a hard combinatorial problem. In this work, we have applied two unique global methods, Truncated Power Method (TPower) and Low Rank Bilinear Approximation (LowRank), to solve the feature selection problem. These algorithms provide very good approximations to the NP-hard CMI based feature selection problem. We experimentally demonstrate the effectiveness of these procedures across multiple datasets and compare them with existing MI based global and iterative feature selection procedures.

Dimension reduction techniques form the core of predictive analytics systems and they help us create a new feature space that is more helpful in predicting response variables. But these techniques do not necessarily guarantee a better predictive capability as most of them are unsupervised, especially in regression learning. In regression analysis literature, supervised dimension reduction techniques have not been explored much and in this work we provide a solution to this through probabilistic topic models. In this work, we have shown that the double mixture structure of Latent Dirichlet Allocation topic model helps us 1) to visualize feature patterns, and 2) to project features onto a topic simplex that is more predictive of responses, when compared to popular techniques like PCA and KernelPCA. Until now, topic models have not been explored in a supervised context of video analysis and in this work we introduce a Regularized supervised topic model (RSLDA) that models video and audio features and has outperformed supervised dimension reduction techniques like SPCA and Correlation based feature selection algorithms. All the models discussed in this work have been evaluated to predict continuous human emotions from video data.

Popular domain adaptation (DA) techniques learn a classifier for the target domain by sampling relevant data points from the source and combining it with the target data. We present a Support Vector Machine (SVM) based supervised DA technique, where the similarity between source and target domains is modeled as the similarity between their SVM decision boundaries. We couple the source and target SVMs and reduce the model to a standard single SVM. We test the Coupled-SVM on multiple datasets and compare our results with other popular SVM based DA approaches.

Affective computing researchers have recently been focusing on continuous emotion dimensions like arousal and valence. This dual coordinate affect space can explain many of the discrete emotions like sadness, anger, joy, etc. In the area of continuous emotion recognition, Principal Component Analysis (PCA) models are generally used to enhance the performance of various image and audio features by projecting them to a new space where the new features are less correlated. We instead, propose that quantizing and projecting the features to a latent topic space performs better than PCA. Specifically we extract these topic features using Latent Dirichlet Allocation (LDA) models. We show that topic models project the original features to a latent feature space that is more coherent and useful for continuous emotion recognition than PCA. Unlike PCA where no semantics can be attributed to the new features, topic features can have a visual and semantic interpretation which can be used in personalized HCI applications and Assistive technologies. Our hypothesis in this work has been validated using the AVEC 2012 continuous emotion challenge dataset.

There is an increasing demand for applications that can detect changes in human affect or behavior especially in the fields of health care and crime detection. Detection of changes in continuous human affect dimensions from multimedia data precedes the exact prediction of an emotion as a continuum. With the growth in the dimensions of emotion space there is a need to discover latent descriptors (topics) that can explain these complex states. Considering that at every time step the audio/video frames constitute a set of such latent topics, the presence and absence of changes in emotion should effect the topics in those frames. Based on this assumption an Adaptive Temporal Topic model (ATTM) based change detection algorithm is presented that, at each time step, detects whether a significant change in human affect has occurred. ATTM is a probabilistic topic model that extends Latent Dirichlet Allocation model by incorporating the temporal dependencies between human audio/video `documents' and generates refined topics. The topics assigned to a document by ATTM are adapted to the presence or absence of a change in the affect dimension at that time step. ATTM along with different regression models has been tested on the multimodal Audio Visual Emotion Challenge (AVEC 2012) data and has shown promising results in comparison to existing temporal and non-temporal topic models.

Gesture spotting is the task of detecting and recognizing gestures defined in a vocabulary. The difficulty of gesture spotting stems from the fact that valid gestures appear sporadically in a continuous gesture stream, interspersed with invalid gestures (movements that do not correspond to any gesture contained in the vocabulary). In this paper, a novel method for designing threshold models from valid gesture models learnt through Adaptive Boosting is proposed. This threshold model is adaptive in nature and discriminates between valid and invalid gestures. Furthermore, a gesture spotting network consisting of the individual gesture models and the threshold model is proposed to perform the task of spotting and recognition simultaneously. This technique is evaluated in the context of spotting and recognizing activity gestures (hand gestures) from continuous accelerometer data streams. The proposed technique results in a precision of 0.78 and a recall of 0.93 out performing the HMM based threshold model which resulted in 0.4 and 0.81 precision and recall values.

Cooking activities are complex activities consisting of multiple steps or tasks. These tasks can be associated with one another based on two properties the temporal structure that defines the sequence of occurrence of tasks and the objects that are used in the activity. This paper develops cooking activity models for the purpose of task prediction based on these two properties. The temporal structure of the sequence of tasks is captured by the novel hierarchical state space markov chain (HMC) and the object usage is represented using the object based task group (OTG) models. A probabilistic task prediction algorithm that fuses the HMC and OTG models has been developed to predict the next most probable task, given that a sequence of tasks has been completed. The proposed models and algorithms have been evaluated on two complex cooking activities making brownies and making eggs, achieving a subject independent accuracy of 68.5% for predicting tasks, which is an improvement by an average of 6% in comparison to a Markov chain. The work done in the paper is first of its kind as it focuses on task prediction rather than task recognition. The task prediction framework described in the paper can be easily adapted to any complex activity supporting various annotation schemes and activity models.