We present a novel framework for applying deep neural networks (DNN) to soft decoding of linear codes at arbitrary block lengths. Unlike other approaches, our framework allows unconstrained DNN design, enabling the free application of powerful designs that were developed in other contexts. Our method is robust to overfitting that inhibits many competing methods, which follows from the exponentially large number of codewords required for their training. We achieve this by transforming the channel output before feeding it to the network, extracting only the syndrome of the hard decisions and the channel output reliabilities. We prove analytically that this approach does not involve any intrinsic performance penalty, and guarantees the generalization of performance obtained during training. Our best results are obtained using a recurrent neural network (RNN) architecture combined with simple preprocessing by permutation. We provide simulation results that demonstrate performance that sometimes approaches that of the ordered statistics decoding (OSD) algorithm.

Multiple Kernel Learning (MKL) methods are known for their effectiveness in solving classification and regression problems involving multi-modal data. Many MKL approaches use linear combination of base kernels, resulting in somewhat limited feature representations. Several non-linear MKL formulations were proposed recently. They provide much higher dimensional feature spaces, and, therefore, richer representations. However, these methods often lead to non-convex optimization and to intractable number of optimization parameters. In this paper, we propose a new non-linear MKL method that utilizes nuclear norm regularization and leads to convex optimization problem. The proposed Nuclear-norm-Constrained MKL (NuC-MKL) algorithm converges faster, and requires smaller number of calls to an SVM solver, as compared to other competing methods. Moreover, the number of the model support vectors in our approach is usually much smaller, as compared to other methods. This suggests that our algorithm is more resilient to overfitting. We test our algorithm on several known benchmarks, and show that it equals or outperforms the state-of-the-art MKL methods on all these data sets.

Computer Aided Diagnosis (CADx) systems are designed to assist doctors in medical image interpretation. However, a CADx is often thought of as a black box whose diagnostic decision is not intelligible to a radiologist. Therefore, a system that uses semantic image interpretation, and mimics human image analysis, has clear benefits. We propose a new method for automatic textual description of medical image findings, such as lesions in medical images. The method performs joint estimation of semantic features of lesions from image measurements. We formalize this problem as learning to map a set of diverse medical image measurements to a set of semantic descriptor values. We use a structured learning framework to model individual semantic descriptors and their relationships. The parameters of the model are efficiently learned using the Structured Support Vector Machine (SSVM). The proposed approach generates radiological lexicon descriptors used to make a diagnosis of various diseases. This can help radiologists easily understand a diagnosis recommendation made by an automatic system, such as CADx. We apply the proposed method to publicly available and to proprietary breast and brain imaging datasets, and show that our method generates more accurate descriptions, as compared to other alternative approaches.

That most computer vision algorithms rely on parameters is a fact of life which cannot be avoided. For optimal algorithm performance, these parameters need to be tuned; generally speaking, this tuning is done manually or in some heuristic fashion. In this paper, we propose a new, general method for attacking the problem of parameter tuning, which is applicable to a wide variety of computer vision algorithms. Our method is semi-automatic: a user is given several pairs of outputs from a given vision algorithm, which have been generated by different parameter values; the user is then required to simply choose, for each pair, which output is preferred. Our method then finds the smoothest preference function which satisfies these user preferences. Using the theory of Reproducing Kernel Hilbert Spaces, we show how this problem can be reduced to a finite-dimensional convex optimization. We validate our parameter tuning scheme both on simulated data and on the problem of tuning the parameters of an image denoising algorithm.

Given a source object and a target object, we consider the problem of transferring the “color scheme” of the source to the target, while at the same time maintaining the target's original look and feel. This is a challenging problem due to the fact that the source and target may each consist of multiple colors, each of which comes in multiple shades. We propose a two stage solution to this problem. (1) A discrete color flow is computed from the target histogram to the source histogram; this flow is computed as the solution to a convex optimization problem, whose global optimum may be found. (2) The discrete flow is turned into a continuous color transformation, which can be written as a convex sum of Stretch-Minimizing Structure-Preserving (SMSP) transformations. These SMSP transformations, which are computed based on the color flow, are affine transformations with desirable theoretical properties. The effectiveness of this two stage algorithm is validated in a series of experiments.

In this paper we present a novel method for high quality real-time video enhancement; it improves the sharpness and the contrast of video streams, and simultaneously suppresses noise. The method is comprised of three main modules: (1) noise analysis, (2) spatial processing, based on a new multi-scale pseudo-bilateral filter, and (3) temporal processing that includes robust motion detection and recursive temporal noise filtering. To achieve video frame rates for HD signals used in high-end telepresence systems such as the HP Halo room, we employ the computational capacity of modern graphics cards (GPUs), and distribute the tasks of analysis and of processing between CPU and GPU. The proposed scheme allows achieving video quality which is comparable with high-end camera systems, while using much lower cost cameras and reducing channel bandwidth requirements.

The kernel density estimate (KDE) is a nonparametric density estimate which has broad application in computer vision and pattern recognition. In particular, the mean shift procedure uses the KDE structure to cluster or segment data, including images and video. The usefulness of these twin techniques—KDE and mean shift—on large data sets is hampered by the large space or description complexity of the KDE, which in turn leads to a large time complexity of the mean shift procedure that is superlinear in the number of points. In this paper, we propose a sampling technique for KDE paring, i.e., the construction of a compactly represented KDE with much smaller description complexity. We prove that this technique has good properties in that the pared-down KDE so constructed is close to the original KDE in a precise mathematical sense. We then show how to use this pared-down KDE to devise a considerably faster mean shift algorithm, whose time complexity we analyze formally. Experiments show that image and video segmentation results of the proposed fast mean shift method are similar to those based on the standard mean shift procedure, with the typical speed-up several orders of magnitude for large data sets. Finally, we present an application of the fast mean shift method to the efficient construction of multiscale graph structures for images, which can be used as a preprocessing step for more sophisticated segmentation algorithms.

The Mean Shift procedure is a well established clustering technique that is widely used in imaging applications such as image and video segmentation, denoising, object tracking, texture classification, and others. However, the Mean Shift procedure has relatively high time complexity which is superlinear in the number of data points. In this paper we present a novel fast Mean Shift procedure which is based on the random sampling of the Kernel Density Estimate (KDE). We show theoretically that the resulting reduced KDE is close to the complete data KDE, to within a given accuracy. Moreover, we prove that the time complexity of the proposed fast Mean Shift procedure based on the reduced KDE is considerably lower than that of the original Mean Shift; the typical gain is of several orders for big data sets. Experiments show that image and video segmentation results of the proposed fast Mean Shift method are similar to those based on the standard Mean shift procedure. We also present a new application of the Fast Mean Shift method to the efficient construction of graph hierarchies for images; the resulting structure is potentially useful for solving computer vision problems which can be posed as graph problems, including stereo, semi-automatic segmentation, and optical flow.

Many of today's real world applications need to handle and analyze continually growing amounts of data, while the cost of collecting data decreases. As a result, the main technological hurdle is that the data is acquired faster than it can be processed. Data reduction methods are thus increasingly important, as they allow one to extract the most relevant and important information from giant data sets. We present one such method, based on compressing the description length of an estimate of the probability distribution of a set points.

This paper targets denoising of digital photos taken by cameras with unknown sensor parameters and image processing pipeline. Common noise characteristics in such images originate from camera-internal processing, such as demosaicing, tone mapping, and JPEG compression. Three of the noise characteristics that are not adequately addressed by existing denoising algorithms are spatially correlated low-frequency noise, strong signal dependency of the noise level and high levels of the chrominance noise relative to the luminance noise. We propose a generic scheme that extends existing denoisers such as the bilateral filter to account for all the problems above. Our solution combines a novel progressive pyramidal filtering scheme to address the correlated noise, filter adaptation via local noise level estimation and luminance-guided chrominance filtering to address the low-SNR of the chrominance channels. We demonstrate the effectiveness of our solution for challenging realistic noisy photos.

In this paper, we propose a novel method for automatic dust and scratch detection and removal in scanned images. The proposed method combines morphological operations with local image analysis that guides adaptive, image dependent perceptual defect thresholding. This allows us to achieve reliable separation of defects and intrinsic image features, and yields high quality image restoration. The method is capable of detecting and removing defects of very low contrast and of various sizes; it is efficient computationally and can be utilized in real time applications.

In this work, we propose noise and signal activity estimation method that discriminates noise from signal based on local and global properties of the image data. The method yields pixel-wise maps of the noise variance and of the signal activity. Using these maps to guide imaging algorithms such as image enhancement and print defect detection improves their performance. The proposed method does not assume a white Gaussian noise model; it is very efficient computationally and, as such, is useful for a wide variety of applications.

It was previously shown that sparse representations can improve and simplify the estimation of an unknown mixing matrix of a set of images and thereby improve the quality of separation of source images. Here we propose a multiscale approach to the problem of blind separation of images from a set of their mixtures. We take advantage of the properties of multiscale transforms such as wavelet packets and decompose signals and images according to sets of local features. The resulting partial representations on a tree of data structure depict various degrees of sparsity. We show how the separation error is affected by the sparsity of the decomposition coefficients, and by the misfit between the prior, formulated in accordance with the probabilistic model of the coefficients distribution, and the actual distribution of the coefficients. Our error estimator, based on the Taylor expansion of the quasi log-likelihood function, is used in selection of the best subsets of coefficients, utilized in turn for further separation. The performance of the proposed method is assessed by separation of noise-free and noisy data. Experiments with simulated and real signals and images demonstrate significant improvement of separation quality over previously reported results.

A classical technique for reconstruction of emission tomography (ET) images from measured projections is based on the maximum likelihood (ML) estimation, achieved with the expectation maximization (EM) algorithm. We incorporate the wavelet transform (WT) and total variation (TV) based penalties into the ML framework, and compare performance of the EM algorithm and the previously proposed conjugate barrier (CB) algorithm. Using the WT- and TV-based penalties allows one to embed regularization procedures into the iterative process. In the case of the WT-based penalty, we impose a subset of wavelet coefficients with a desired resolution on the objective function. It appears that the CB algorithm outperforms substantially the EM algorithm in penalized reconstruction. The properties of the optimization algorithms along with WTand TV-based regularization are demonstrated on image reconstructions of a synthetic brain phantom, and the quality of reconstruction is compared with standard methods.

The blind source separation problem is concerned with extraction of the underlying source signals from a set of their linear mixtures, where the mixing matrix is unknown. It was discovered recently, that exploiting the sparsity of sources in their representation according to some signal dictionary, dramatically improves the quality of separation. It is especially useful in image processing problems, wherein signals possess strong spatial sparsity. We use multiscale transforms, such as wavelet or wavelet packets, to decompose signals into sets of local features with various degrees of sparsity. We use this intrinsic property for selecting the best (most sparse) subsets of features for further separation. Experiments with 1D signals and images demonstrate significant improvement of separation quality.

We consider a problem of blind source separation from a set of instan(cid:173) taneous linear mixtures, where the mixing matrix is unknown. It was discovered recently, that exploiting the sparsity of sources in an appro(cid:173) priate representation according to some signal dictionary, dramatically improves the quality of separation. In this work we use the property of multi scale transforms, such as wavelet or wavelet packets, to decompose signals into sets of local features with various degrees of sparsity. We use this intrinsic property for selecting the best (most sparse) subsets of features for further separation. The performance of the algorithm is ver(cid:173) ified on noise-free and noisy data. Experiments with simulated signals, musical sounds and images demonstrate significant improvement of sep(cid:173) aration quality over previously reported results.