We present a novel action recognition method based on space-time locally adaptive regression kernels and the matrix cosine similarity measure. The proposed method uses a single example of an action as a query to find similar matches. It does not require prior knowledge about actions, foreground/background segmentation, or any motion estimation or tracking. Our method is based on the computation of novel space-time descriptors from the query video which measure the likeness of a voxel to its surroundings. Salient features are extracted from said descriptors and compared against analogous features from the target video. This comparison is done using a matrix generalization of the cosine similarity measure. The algorithm yields a scalar resemblance volume, with each voxel indicating the likelihood of similarity between the query video and all cubes in the target video. Using nonparametric significance tests by controlling the false discovery rate, we detect the presence and location of actions similar to the query video. High performance is demonstrated on challenging sets of action data containing fast motions, varied contexts, and complicated background. Further experiments on the Weizmann and KTH data sets demonstrate state-of-the-art performance in action categorization.

We present a novel face representation based on locally adaptive regression kernel (LARK) descriptors. Our LARK descriptor measures a self-similarity based on “signal-induced distance” between a center pixel and surrounding pixels in a local neighborhood. By applying principal component analysis (PCA) and a logistic function to LARK consecutively, we develop a new binary-like face representation which achieves state-of-the-art face verification performance on the challenging benchmark “Labeled Faces in the Wild” (LFW) dataset. In the case where training data are available, we employ one-shot similarity (OSS) based on linear discriminant analysis (LDA). The proposed approach achieves state-of-the-art performance on both the unsupervised setting and the image restrictive training setting (72.23% and 78.90% verification rates), respectively, as a single descriptor representation, with no preprocessing step. As opposed to combined 30 distances which achieve 85.13%, we achieve comparable performance (85.1%) with only 14 distances while significantly reducing computational complexity.

We present a visual saliency detection method and its applications. The proposed method does not require prior knowledge (learning) or any pre-processing step. Local visual descriptors which measure the likeness of a pixel to its surroundings are computed from an input image. Self-resemblance measured between local features results in a scalar map where each pixel indicates the statistical likelihood of saliency. Promising experimental results are illustrated for three applications: automatic target detection, boundary detection, and image quality assessment.

We present a generic detection/localization algorithm capable of searching for a visual object of interest without training. The proposed method operates using a single example of an object of interest to find similar matches, does not require prior knowledge (learning) about objects being sought, and does not require any preprocessing step or segmentation of a target image. Our method is based on the computation of local regression kernels as descriptors from a query, which measure the likeness of a pixel to its surroundings. Salient features are extracted from said descriptors and compared against analogous features from the target image. This comparison is done using a matrix generalization of the cosine similarity measure. We illustrate optimality properties of the algorithm using a naive-Bayes framework. The algorithm yields a scalar resemblance map, indicating the likelihood of similarity between the query and all patches in the target image. By employing nonparametric significance tests and nonmaxima suppression, we detect the presence and location of objects similar to the given query. The approach is extended to account for large variations in scale and rotation. High performance is demonstrated on several challenging data sets, indicating successful detection of objects in diverse contexts and under different imaging conditions.

We present an algorithm for detecting human actions based upon a single given video example of such actions. The proposed method is unsupervised, does not require learning, segmentation, or motion estimation. The novel features employed in our method are based on space-time locally adaptive regression kernels. Our method is based on the dense computation of so-called space-time local regression kernels (i.e. local descriptors) from a query video, which measure the likeness of a voxel to its spatio-temporal surroundings. Salient features are then extracted from these descriptors using principal components analysis (PCA). These are efficiently compared against analogous features from the target video using a matrix generalization of the cosine similarity measure. The algorithm yields a scalar resemblance volume; each voxel indicating the like-lihood of similarity between the query video and all cubes in the target video. By employing non-parametric significance tests and non-maxima suppression, we accurately detect the presence and location of actions similar to the given query video. High performance is demonstrated on a challenging set of action data indicating successful detection of multiple complex actions even in the presence of fast motions.

The optimal spatial adaptation (OSA) method [1] proposed by Boulanger and Kervrann has proven to be quite effective for spatially adaptive image denoising. This method, in addition to extending the Non-Local Means(NLM) method of [2], employs an iteratively growing window scheme, and a local estimate of the mean square error to very effectively remove noise from images. By adopting an iteratively growing space-time window, the method was recently extended to 3-D for video denoising in [3]. In the present paper, we demonstrate a simple, but effective improvement on the OSA method in both 2- and 3-D. We demonstrate that the OSA implicitly relies on a locally constant model of the underlying signal. Thereby, removing this constraint and introducing the possibility of higher order local regression models, we arrive at a relatively simple modification that results in an improvement in performance. While this improvement is observed in both 2-D and 3-D, we concentrate on demonstrating it in 3-D for the application of video denoising.

We present a novel approach to the problem of detection of visual similarity between a template image, and patches in a given image. The method is based on the computation of a local kernel from the template, which measures the likeness of a pixel to its surroundings. This kernel is then used as a descriptor from which features are extracted and compared against analogous features from the target image. Comparison of the features extracted is carried out using canonical correlations analysis. The overall algorithm yields a scalar resemblance map (RM) which indicates the statistical likelihood of similarity between a given template and all target patches in an image being examined. Performing a statistical test on the resulting RM identifies similar objects with high accuracy and is robust to various challenging conditions such as partial occlusion, and illumination change.