When information retrieval systems return a ranked list of results in response to a query, they may be choosing from a large set of candidate results that are equally useful and relevant. This means we might be able to identify a difference between rankers A and B, where ranker A systematically prefers a certain type of relevant results. Ranker A may have this systematic difference (different “vibe”) without having systematically better or worse results according to standard information retrieval metrics. We first show that a vibe difference can exist, comparing two publicly available rankers, where the one that is trained on health-related queries will systematically prefer health-related results, even for non-health queries. We define a vibe metric that lets us see the words that a ranker prefers. We investigate the vibe of search engine clicks vs. human labels. We perform an initial study into correcting for vibe differences to make ranker A more like ranker B via changes in negative sampling during training.

An effective email search engine can facilitate users’ search tasks and improve their communication efficiency. Users could have varied preferences on various ranking signals of an email, such as relevance and recency based on their tasks at hand and even their jobs. Thus a uniform matching pattern is not optimal for all users. Instead, an effective email ranker should conduct personalized ranking by taking users’ characteristics into account. Existing studies have explored user characteristics from various angles to make email search results personalized. However, little attention has been given to users’ search history for characterizing users. Although users’ historical behaviors have been shown to be beneficial as context in Web search, their effect in email search has not been studied and remains unknown. Given these observations, we propose to leverage user search history as query context to characterize users and build a context-aware ranking model for email search. In contrast to previous context-dependent ranking techniques that are based on raw texts, we use ranking features in the search history. This frees us from potential privacy leakage while giving a better generalization power to unseen users. Accordingly, we propose a context-dependent neural ranking model (CNRM) that encodes the ranking features in users’ search history as query context and show that it can significantly outperform the baseline neural model without using the context. We also investigate the benefit of the query context vectors obtained from CNRM on the state-of-the-art learning-to-rank model LambdaMart by clustering the vectors and incorporating the cluster information. Experimental results show that significantly better results can be achieved on LambdaMart as well, indicating that the query clusters can characterize different users and effectively turn the ranking model personalized.

In this work, a novel acoustic characterization approach to multimedia event detection (MED) task for unconstrained and unstructured consumer-level videos through audio signal modeling is proposed. The key idea is to characterize the acoustic space of interest with a set of fundamental acoustic units around which a set of acoustic segment models (ASMs) is built. A vector space modeling technique to address MED is here adopted, where an incoming audio signal is first decoded into a sequence of acoustic segments. Then, a feature vector is generated by using co-occurrence statistics of acoustic units, and the MED final decision is implemented with a vector space language classifier. Experimental evidence on the TRECVID2011 MED demonstrates the viability of the proposed approach. Furthermore, it better accounts for temporal dependencies than previously proposed MFCC bag-of-word approaches.

Georgia Tech's Colleges of Engineering and Computing initiated the Vertically-Integrated Projects (VIP) program in January 2009. Undergraduate students that join VIP teams earn academic credit for participating in design efforts that assist faculty and graduate students with research and development issues in their technical areas. The teams are: multidisciplinary - drawing students from around the university; vertically-integrated - maintaining a mix of sophomores through PhD students each semester; and long-term - each undergraduate student may participate in a project for up to six semesters. We describe the Video and Image Annotation VIP (VIA-VIP) project, which provides undergraduates unique opportunities to learn and apply state-of-the-art video-mining algorithms by processing a large archive of football videos recorded from GT football games. Their results are documented. Based on their feedback we believe the VIA-VIP course is on track to meet the needs of undergraduates in areas they don't usually see in the traditional undergraduate classroom.

We propose a kernelized maximal-figure-of-merit (MFoM) learning approach to efficiently training a nonlinear model using subspace distance minimization. In particular, a fixed, small number of training samples are chosen in a way that the distance between function spaces constructed with a subset of training samples and with the entire training data set is minimized. This construction of the subset enables us to learn a nonlinear model efficiently while keeping the resulting model nearly optimal compared to the model from the whole training data set. We show that the subspace distance can be minimized through the Nystrom extension. Experimental results on various machine learning problems demonstrate clear advantages of the proposed technique over the case where the function space is built with randomly selected training samples. Additional comparisons with the model trained with the entire training samples show that the proposed technique achieves comparable results while reducing training time tremendously.

We propose a novel system to analyze gestural and nonverbal cues of participants in video conferencing. These cues have previously been referred to as “honest signals” and are usually associated with the underlying cognitive state of the participants. The presented system analyzes a set of audio-visual, non-linguistic features in real time from the audio and video streams of two participants in a video conference. We show how these features can be used to compute indicators of the overall quality and type of conversation being held. The system also provides visual feedback to the participants, who then have the choice of modifying their conversational style in order to achieve the desired outcome of the video conference. Experiments on real-life data show that the system can predict the type of conversation with high accuracy using the non-linguistic signals only. Qualitative user studies highlight the positive effects of increased awareness amongst the participants about their own gestural and non-verbal cues.

A detection-based paradigm decomposes a complex system into small pieces, solves each subproblem one by one, and combines the collected evidence to obtain a final solution. In this study of video story segmentation, a set of key events are first detected from heterogeneous multimedia signal sources, including a large scale concept ontology for images, text generated from automatic speech recognition systems, features extracted from audio track, and high-level video transcriptions. Then a discriminative evidence fusion scheme is investigated. We use the maximum figure-of-merit learning approach to directly optimize the performance metrics used in system evaluation, such as precision, recall, and F1 measure. Some experimental evaluations conducted on the TRECVID 2003 dataset demonstrate the effectiveness of the proposed detection-based paradigm. The proposed framework facilitates flexible combination and extensions of event detector design and evidence fusion to enable other related video applications.

We propose an incremental classifier learning framework that starts with a small amount of labeled training data to create an initial set of classifiers, and gradually incorporates unlabeled data into the incremental learning process to improve the models. A key to the effectiveness of the proposed framework is to judicially select a good incremental learning subset from all remaining unlabeled samples by computing a confidence measure and a margin-like discrimination score that measures potential contributions of the selection set to enhancing the existing models. To further refine the above selection set, class prior densities were also exploited. The proposed framework was tested on an automatic image annotation application using a subset of the Corel image set. When all data, including both initially labeled and incrementally learned samples, were used, the final model was shown to achieve a significant improvement over the initial set of classifiers in terms of micro-averaging F1 even when only a small number of images were initially labeled. Furthermore, when 30% of the images were initially labeled the incrementally learned models achieved comparable results to the case when models were created with all training data labeled.

In this paper, we present an experimental study on using high-dimensional image features to perform discriminative classifier combination for TRECVID concept detection. We combine a multi-class classifier with binary-class classifiers. After training a multi-class classifier, we train binary-class classifiers by decomposing a multi-class problem into several binary-class classification problems, and fuse them together using a discriminative classifier combination approach. This idea leverages on each classifier's properties; multi-class classifiers emphasize on segmenting a decision space optimally in terms of some overall performance criteria whereas binary classifiers focus on detecting corresponding positive samples locally. Testing on the TRECVID2005 development set with 39 LSCOM-Lite concepts by adding an additional set of 39 pairs of binary concept classifiers, the mean average precision was improved by 34.1% over our baseline system with only 39 multi-class concept classifiers. When compared with state-of-the-art systems our proposed method is quite competitive especially for concepts with a relatively small number of positive samples.