For more than a decade, deep learning algorithms have consistently achieved and improved upon the state-of-the-art performance on image classification tasks. However, there is a general lack of understanding and knowledge about the decision boundaries carved by these modern deep neural network architectures. Recently, an algorithm called DeepDIG was introduced to generate boundary instances between two classes based on the decision regions defined by any deep neural network classifier. Although it is very effective in generating boundary instances, the underlying algorithm was designed to work with two classes at a time in a non-commutative fashion which makes it ill-suited for multi-class problems with hundreds of classes. In this work, we extend the DeepDIG algorithm so that it scales linearly with the number of classes. We show that the proposed U-DeepDIG algorithm maintains the efficacy of the original DeepDIG algorithm while being scalable and more efficient when applied to larger classification problems. We demonstrate this by applying our algorithm on MNIST, Fashion-MNIST and CIFAR10 datasets. In addition to qualitative comparisons, we also perform extensive quantitative comparison by analyzing the margin between the class boundaries and the instances generated.

Image restoration methods aim to recover the underlying clean image from corrupted observations. The expected patch log-likelihood (EPLL) algorithm is a powerful image restoration method that uses a Gaussian mixture model (GMM) prior on the patches of natural images. Although it is very effective for restoring images, its high runtime complexity makes the EPLL ill-suited for most practical applications. In this paper, we propose three approximations to the original EPLL algorithm. The resulting algorithm, which we call the fast-EPLL (FEPLL), attains a dramatic speed-up of two orders of magnitude over EPLL while incurring a negligible drop in the restored image quality (less than 0.5 dB). We demonstrate the efficacy and versatility of our algorithm on a number of inverse problems, such as denoising, deblurring, super-resolution, inpainting, and devignetting. To the best of our knowledge, the FEPLL is the first algorithm that can competitively restore a pixel image in under 0.5 s for all the degradations mentioned earlier without specialized code optimizations, such as CPU parallelization or GPU implementation.

Patch priors have become an important component of image restoration. A powerful approach in this category of restoration algorithms is the popular expected patch log-likelihood (EPLL) algorithm. EPLL uses a Gaussian mixture model (GMM) prior learned on clean image patches as a way to regularize degraded patches. In this paper, we show that a generalized Gaussian mixture model (GGMM) captures the underlying distribution of patches better than a GMM. Even though GGMM is a powerful prior to combine with EPLL, the non-Gaussianity of its components presents major challenges to be applied to a computationally intensive process of image restoration. Specifically, each patch has to undergo a patch classification step and a shrinkage step. These two steps can be efficiently solved with a GMM prior but are computationally impractical when using a GGMM prior. In this paper, we provide approximations and computational recipes for fast evaluation of these two steps, so that EPLL can embed a GGMM prior on an image with more than tens of thousands of patches. Our main contribution is to analyze the accuracy of our approximations based on thorough theoretical analysis. Our evaluations indicate that the GGMM prior is consistently a better fit for modeling image patch distribution and performs better on average in image denoising task.

We consider patch matching as a recommendation system problem and introduce a new patch-matching approach using nearest neighbor-based collaborative filtering (NN-CF). Our approach involves recommending similar patches to a query patch with the help of other similar patches in a noisy image or an external database. Using user-oriented and item-oriented formulations of NN-CF, we present two variations of CF-based patch-matching criterion. To demonstrate the superior matches found with our method, we apply the new patch-matching scheme to patch-based image denoising and evaluate its effect on the denoising performance. We test the methods on two data sets with varying background and image complexities and under different levels of noise. The proposed method not only improves robustness to patch matching but also provides a new formulation to seamlessly combine internal and external denoising.

The paradigm of using clean patches from a targeted external database to design optimal denoising filters, called Targeted Image Denoising (TID), has been shown to outperform state-of-the-art denoising algorithms such as BM3D. In this paper, we introduce Targeted Video Denoising algorithm that extends the TID algorithm to denoise decompressed video without adding complexity. Our algorithm leverages the motion vectors generated during compression to establish temporal coherency between patches in consecutive frames. We test our algorithm on three decompressed video sequences with different foregrounds, backgrounds and movement patterns, and different noise level settings. Experimental results show that our approach is effective and performs better than original TID and the state-of-the-art video denoising algorithm.

Multi-task learning (MTL) improves the prediction performance on multiple, different but related, learning problems through shared parameters or representations. One of the most prominent multi-task learning algorithms is an extension to svms by Evgeniou et al. Although very elegant, multi-task svm is inherently restricted by the fact that support vector machines require each class to be addressed explicitly with its own weight vector which, in a multi-task setting, requires the different learning tasks to share the same set of classes. This paper proposes an alternative formulation for multi-task learning by extending the recently published large margin nearest neighbor (lmnn) algorithm to the MTL paradigm. Instead of relying on separating hyperplanes, its decision function is based on the nearest neighbor rule which inherently extends to many classes and becomes a natural fit for multitask learning. We evaluate the resulting multi-task lmnn on real-world insurance data and speech classification problems and show that it consistently outperforms single-task kNN under several metrics and state-of-the-art MTL classifiers.

Automatic classification of electroencephalography (EEG) signals, for different type of mental activities, is an active area of research and has many applications such as brain computer interface (BCI) and medical diagnoses. We introduce a simple yet effective way to use Kullback-Leibler (KL) divergence in the classification of raw EEG signals. We show that k-nearest neighbor (k-NN) algorithm with KL divergence as the distance measure, when used using our feature vectors, gives competitive classification accuracy and consistently outperforms the more commonly used Euclidean k-NN. We also develop and demonstrate the use of a KL-based kernel to classify EEG data using support vector machines (SVMs). Our KL-distance based kernel compares favorably to other well established kernels such as linear and radial basis function (RBF) kernel. The EEG data, used in our experiments for classification, was recorded while the subject performed 5 different mental activities such as math problem solving, letter composing, 3-D block rotation, counting and resting (baseline). We present classification results for this data set that are obtained by using raw EEG data with no explicit artifact removal in the pre-processing steps.