This paper proposes a dataset augmentation method by fine-tuning pre-trained diffusion models. Generating images using a pre-trained diffusion model with textual conditioning often results in domain discrepancy between real data and generated images. We propose a fine-tuning approach where we adapt the diffusion model by conditioning it with real images and novel text embeddings. We introduce a unique procedure called Mixing Visual Concepts (MVC) where we create novel text embeddings from image captions. The MVC enables us to generate multiple images which are diverse and yet similar to the real data enabling us to perform effective dataset augmentation. We perform comprehensive qualitative and quan-titative evaluations with the proposed dataset augmentation approach showcasing both coarse-grained and fine-grained changes in generated images. Our approach outperforms state-of-the-art augmentation techniques on benchmark classification tasks. The code is available at https://github.com/rahatkutubi/MVC

Existing domain adaptation literature comprises multiple techniques that align the labeled source and unlabeled target domains at different stages, and predict the target labels. In a source-free domain adaptation setting, the source data is not available for alignment. We present a source-free generative paradigm that captures the relations between the source categories and enforces them onto the unlabeled target data, thereby circumventing the need for source data without introducing any new hyper-parameters. The adaptation is performed through the adversarial alignment of the posterior probabilities of the source and target categories. The proposed approach demonstrates competitive performance against other source-free domain adaptation techniques and can also be used for source-present settings.

Annotating a data sample in a multi-label learning problem requires a human oracle to consider the presence/absence of every possible label separately, which is extremely labor intensive. Active learning algorithms automatically identify the informative samples from large amounts of unlabeled data and significantly reduce human annotation efforts in inducing a classification model. Further, deep models have gained popularity to automatically learn representative features from a given dataset and have depicted promising empirical performance in a variety of applications. In this paper, we exploit the feature learning capabilities of deep neural networks and propose a novel framework to address the problem of multi-label active learning with label correlation. We integrate an active selection criterion to the objective function and train deep networks to optimize the function. Our extensive empirical studies on five benchmark multi-label datasets show that our methods outperform the state-of-the-art algorithms, corroborating their potential for real-world image classification applications.

In recent years, deep neural networks have emerged as a dominant machine learning tool for a wide variety of application domains. However, training a deep neural network requires a large amount of labeled data, which is an expensive process in terms of time, labor and human expertise. Domain adaptation or transfer learning algorithms address this challenge by leveraging labeled data in a different, but related source domain, to develop a model for the target domain. Further, the explosive growth of digital data has posed a fundamental challenge concerning its storage and retrieval. Due to its storage and retrieval efficiency, recent years have witnessed a wide application of hashing in a variety of computer vision applications. In this paper, we first introduce a new dataset, Office-Home, to evaluate domain adaptation algorithms. The dataset contains images of a variety of everyday objects from multiple domains. We then propose a novel deep learning framework that can exploit labeled source data and unlabeled target data to learn informative hash codes, to accurately classify unseen target data. To the best of our knowledge, this is the first research effort to exploit the feature learning capabilities of deep neural networks to learn representative hash codes to address the domain adaptation problem. Our extensive empirical studies on multiple transfer tasks corroborate the usefulness of the framework in learning efficient hash codes which outperform existing competitive baselines for unsupervised domain adaptation.

In the recent years, deep learning algorithms have achieved state-of-the-art performance in a variety of computer vision applications. In this paper, we propose a novel active learning framework to select the most informative unlabeled samples to train a deep belief network model. We introduce a loss function specific to the active learning task and train the model to minimize the loss function. To the best of our knowledge, this is the first research effort to integrate an active learning based criterion in the loss function used to train a deep belief network. Our extensive empirical studies on a wide variety of uni-modal and multi-modal vision datasets corroborate the potential of the method for real-world image recognition applications.

Mutual Information (MI) is often used for feature selection when developing classifier models. Estimating the MI for a subset of features is often intractable. We demonstrate, that under the assumptions of conditional independence, MI between a subset of features can be expressed as the Conditional Mutual Information (CMI) between pairs of features. But selecting features with the highest CMI turns out to be a hard combinatorial problem. In this work, we have applied two unique global methods, Truncated Power Method (TPower) and Low Rank Bilinear Approximation (LowRank), to solve the feature selection problem. These algorithms provide very good approximations to the NP-hard CMI based feature selection problem. We experimentally demonstrate the effectiveness of these procedures across multiple datasets and compare them with existing MI based global and iterative feature selection procedures.

Dimension reduction techniques form the core of predictive analytics systems and they help us create a new feature space that is more helpful in predicting response variables. But these techniques do not necessarily guarantee a better predictive capability as most of them are unsupervised, especially in regression learning. In regression analysis literature, supervised dimension reduction techniques have not been explored much and in this work we provide a solution to this through probabilistic topic models. In this work, we have shown that the double mixture structure of Latent Dirichlet Allocation topic model helps us 1) to visualize feature patterns, and 2) to project features onto a topic simplex that is more predictive of responses, when compared to popular techniques like PCA and KernelPCA. Until now, topic models have not been explored in a supervised context of video analysis and in this work we introduce a Regularized supervised topic model (RSLDA) that models video and audio features and has outperformed supervised dimension reduction techniques like SPCA and Correlation based feature selection algorithms. All the models discussed in this work have been evaluated to predict continuous human emotions from video data.

Popular domain adaptation (DA) techniques learn a classifier for the target domain by sampling relevant data points from the source and combining it with the target data. We present a Support Vector Machine (SVM) based supervised DA technique, where the similarity between source and target domains is modeled as the similarity between their SVM decision boundaries. We couple the source and target SVMs and reduce the model to a standard single SVM. We test the Coupled-SVM on multiple datasets and compare our results with other popular SVM based DA approaches.

There is an increasing demand for applications that can detect changes in human affect or behavior especially in the fields of health care and crime detection. Detection of changes in continuous human affect dimensions from multimedia data precedes the exact prediction of an emotion as a continuum. With the growth in the dimensions of emotion space there is a need to discover latent descriptors (topics) that can explain these complex states. Considering that at every time step the audio/video frames constitute a set of such latent topics, the presence and absence of changes in emotion should effect the topics in those frames. Based on this assumption an Adaptive Temporal Topic model (ATTM) based change detection algorithm is presented that, at each time step, detects whether a significant change in human affect has occurred. ATTM is a probabilistic topic model that extends Latent Dirichlet Allocation model by incorporating the temporal dependencies between human audio/video `documents' and generates refined topics. The topics assigned to a document by ATTM are adapted to the presence or absence of a change in the affect dimension at that time step. ATTM along with different regression models has been tested on the multimodal Audio Visual Emotion Challenge (AVEC 2012) data and has shown promising results in comparison to existing temporal and non-temporal topic models.