Many image classification tasks (e.g., medical image classification) have a severe class imbalance problem. Convolutional neural network (CNN) is currently a state-of-the-art method for image classification. CNN relies on a large training dataset to achieve high classification performance. However, manual labeling is costly and may not even be feasible for medical domain. In this paper, we propose a novel similarity-based active deep learning framework (SAL) that deals with class imbalance. SAL actively learns a similarity model to recommend unlabeled rare class samples for experts' manual labeling. Based on similarity ranking, SAL recommends high confidence unlabeled common class samples for automatic pseudo-labeling without experts' labeling effort. To the best of our knowledge, SAL is the first active deep learning framework that deals with a significant class imbalance. Our experiments show that SAL consistently outperforms two other recent active deep learning methods on two challenging datasets. What's more, SAL obtains nearly the upper bound classification performance (using all the images in the training dataset) while the domain experts labeled only 5.6% and 7.5% of all images in the Endoscopy dataset and the Caltech-256 dataset, respectively. SAL significantly reduces the experts' manual labeling efforts while achieving near optimal classification performance.

We describe a new and effective real-time solution for detecting video segments showing an instrument used during diagnostic or therapeutic operations in endoscopic procedures. In addition, we present a new method to collect a large training dataset: similarity-based data augmentation. This method automates most of the creation of a large training dataset and prevents extensive manual effort to collect and annotate training data by domain experts. Convolutional Neural Network (CNN) analysis using the training data collected with similarity-based data augmentation yields an average F1 score within 1% of that of the CNN analysis using a large manually collected training dataset.

With the advent of new media technology and the ability to identify more information about potential voters, political campaigns have aggressively changed their campaign strategies. Election campaigns increasingly rely on online video advertising to reach voters. Until now, the contents of these ads are manually coded for political science research to study campaign strategies. Manual coding is tremendously time consuming and not scalable to handle the expected increase in online ads. We make the first attempt to investigate automated coding of the content of political video ads for political science research. Specifically, we focus on the problem of classifying a political ad into one of these categories: attack ads, promoting ads, and contrast ads. Together with the domain expert, we introduce a concrete definition for each of these categories. We made available the ground truth labels of 773 political ads of the 2016 primary presidential election. We investigate the effectiveness of several classifiers using single modality and two modalities. The best average F1 score is 0.845 using text features from audio and embedded text in image frames.