Video frame interpolation is a long-studied problem in the video processing field. Recently, deep learning approaches have been applied to this problem, showing impressive results on low-resolution benchmarks. However, these methods do not scale-up favorably to high resolutions. Specifically, when the motion exceeds a typical number of pixels, their interpolation quality is degraded. Moreover, their run time renders them impractical for real-time applications. In this paper we propose IM-Net: an interpolated motion neural network. We use an economic structured architecture and end-to-end training with multi-scale tailored losses. In particular, we formulate interpolated motion estimation as classification rather than regression. IM-Net outperforms previous methods by more than 1.3dB (PSNR) on a high resolution version of the recently introduced Vimeo triplet dataset. Moreover, the network runs in less than 33msec on a single GPU for HD resolution.

We address single image super-resolution using a statistical prediction model based on sparse representations of low- and high-resolution image patches. The suggested model allows us to avoid any invariance assumption, which is a common practice in sparsity-based approaches treating this task. Prediction of high resolution patches is obtained via MMSE estimation and the resulting scheme has the useful interpretation of a feedforward neural network. To further enhance performance, we suggest data clustering and cascading several levels of the basic algorithm. We suggest a training scheme for the resulting network and demonstrate the capabilities of our algorithm, showing its advantages over existing methods based on a low- and high-resolution dictionary pair, in terms of computational complexity, numerical criteria, and visual appearance. The suggested approach offers a desirable compromise between low computational complexity and reconstruction quality, when comparing it with state-of-the-art methods for single image super-resolution.

The primary challenge in linear inverse problems is to design stable and robust decoders to reconstruct high-dimensional vectors from a low-dimensional observation through a linear operator. Sparsity, low-rank, and related assumptions are typically exploited to design decoders, whose performance is then bounded based on some measure of deviation from the idealized model, typically using a norm. This paper focuses on characterizing the fundamental performance limits that can be expected from an ideal decoder given a general model, i.e., a general subset of simple vectors of interest. First, we extend the so-called notion of instance optimality of a decoder to settings where one only wishes to reconstruct some part of the original high-dimensional vector from a low-dimensional observation. This covers practical settings, such as medical imaging of a region of interest, or audio source separation, when one is only interested in estimating the contribution of a specific instrument to a musical recording. We define instance optimality relatively to a model much beyond the traditional framework of sparse recovery, and characterize the existence of an instance optimal decoder in terms of joint properties of the model and the considered linear operator. Noiseless and noise-robust settings are both considered. We show somewhat surprisingly that the existence of noise-aware instance optimal decoders for all noise levels implies the existence of a noise-blind decoder. A consequence of our results is that for models that are rich enough to contain an orthonormal basis, the existence of an ℓ2/ℓ2instance optimal decoder is only possible when the linear operator is not substantially dimension-reducing. This covers well-known cases (sparse vectors, low-rank matrices) as well as a number of seemingly new situations (structured sparsity and sparse inverse covariance matrices for instance). We exhibit an operator-dependent norm which, under a model-specific generalization of the restricted isometry property, always yields a feasible instance optimality property. This norm can be upper bounded by an atomic norm relative to the considered model.

The cosparse analysis model for signals assumes that the signal of interest can be multiplied by an analysis dictionary , leading to a sparse outcome. This model stands as an interesting alternative to the more classical synthesis-based sparse representation model. In this paper, we propose a theoretical study of the performance guarantee of the thresholding algorithm for the pursuit problem in the presence of noise. Our analysis reveals two significant properties of Ω, which govern the pursuit performance: the first is the degree of linear dependencies between sets of rows in Ω, depicted by the cosparsity level. The second property, termed the restricted orthogonal projection property, is the level of independence between such dependent sets and other rows in Ω. We show how these dictionary properties are meaningful and useful, both in the theoretical bounds derived and in a series of experiments that are shown to align well with the theoretical prediction.

The synthesis-based sparse representation model for signals has drawn a considerable interest in the past decade. Such a model assumes that the signal of interest can be decomposed as a linear combination of a few atoms from a given dictionary. In this paper we concentrate on an alternative, analysis-based model, where an Analysis Dictionary multiplies the signal, leading to a sparse out-come. Our goal is to learn the analysis dictionary from a set of signal examples, and the approach taken is parallel and similar to the one adopted by the K-SVD algorithm that serves the corresponding problem in the synthesis model. We present the development of the algorithm steps, which include two greedy tailored pursuit algorithms and a penalty function for the dictionary update stage. We demonstrate its effectiveness in several experiments, showing a successful and meaningful recovery of the analysis dictionary.

We address the problem of denoising for image patches. The approach taken is based on Bayesian modeling of sparse representations, which takes into account dependencies between the dictionary atoms. Following recent work, we use a Boltzman machine to model the sparsity pattern. In this work we focus on the special case of a unitary dictionary and obtain the exact MAP estimate for the sparse representation using an efficient message passing algorithm. We present an adaptive model-based scheme for sparse signal recovery, which is based on sparse coding via message passing and on learning the model parameters from the data. This adaptive approach is applied on noisy image patches in order to recover their sparse representations over a fixed unitary dictionary. We compare the denoising performance to that of previous sparse recovery methods, which do not exploit the statistical dependencies, and show the effectiveness of our approach.

We revisit the problem of recovering a continuous-time signal lying within a known shift-invariant subspace from nonlinear and nonideal samples. Recently, an iterative algorithm for perfect recovery of such signals was proposed. This method requires operations which are not linear time-invariant (LTI), rendering it impractical due to its polynomial dependency on the data length. We describe an alternative iterative algorithm for recovering the signal, which involves only LTI operations. In the revised method, each iteration is much faster and implementation is simpler. Furthermore, the overall running time of our approach depends linearly on the number of samples.