Abstract This paper presents a hybrid scheme that integratedly uses self‐similarity prior and deep convolutional neural network (CNN) fusion for compression artifact reduction in low bit‐rate video applications. Based on the temporal correlation hypothesis, the self‐similarity prior is extended to the temporal domain by using as references not only the current decoded frame but also its neighbouring frames. Furthermore, being cognizant of that the bicubic downsampling process can typically improve the perceptual quality of a video coded at low bit‐rate, for each small patch in the current frame, we search for similar patches in down‐scaled versions of these references, and then form several self‐similarity prior based predictions by tiling these similar patches at corresponding positions. To further exploit information flow across scales, a deep CNN model is constructed that contains two sub‐networks to estimate the final output. One sub‐network takes the self‐similarity prior based predictions along with the decoded frame itself; and the other takes the down‐scaled versions of these frames as network input. Experimental results demonstrate that the proposed method can remarkably improve, both subjectively and objectively, quality of compressed video sequences of low bit‐rates.

Adversarial representation learning is a promising paradigm for obtaining data representations that are invariant to certain sensitive attributes while retaining the information necessary for predicting target attributes. Existing approaches solve this problem through iterative adversarial minimax optimization and lack theoretical guarantees. In this paper, we first study the “linear” form of this problem i.e., the setting where all the players are linear functions. We show that the resulting optimization problem is both non-convex and non-differentiable. We obtain an exact closed-form expression for its global optima through spectral learning and provide performance guarantees in terms of analytical bounds on the achievable utility and invariance. We then extend this solution and analysis to non-linear functions through kernel representation. Numerical experiments on UCI, Extended Yale B and CIFAR100 datasets indicate that, (a) practically, our solution is ideal for “imparting” provable invariance to any biased pre-trained data representation, and (b) the global optima of the “kernel” form can provide a comparable trade-off between utility and invariance in comparison to iterative minimax optimization of existing deep neural network based approaches, but with provable guarantees.

Patch-based denoising algorithms aim to reconstruct the clean image patch leaving behind the residual as contaminating noise. The residual should possess statistical properties of contaminating noise. However, it is very likely that the residual patch contains remnants from the clean image patch. In this letter, we propose a new residual correlation based regularization for image denoising. The regularization can effectively render residual patches as uncorrelated as possible. It allows us to derive an analytical solution for sparse coding (atom selection and coefficient calculation). It also leads to a new online dictionary learning update. The clean image is obtained through alternating between the two stages of sparse coding and dictionary updating. The performance of the proposed algorithm is compared with state-of-the-art denoising algorithms in terms of peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and feature similarity index (FSIM), as well as through visual comparison. Experimental results show that the proposed algorithm is highly competitive and often better than leading denoising algorithms. The proposed algorithm is also shown to offer an efficient complement to the benchmark algorithm of block-matching and 3D filtering (BM3D) especially.

The letter presents a covariance tensor based estimator that can effectively provide the model order for multiple sinusoids in noise. For moderate to high signal-to-noise ratios the estimator works effectively even for short length observations. Unlike the conventional methods that use a covariance matrix, the proposed estimator makes use of a three-way covariance tensor formed from the one-dimensional data. Estimate of the model order has been determined by processing the incremental Frobenius norms of the leading principal subtensors of the core tensor obtained via higher order singular value decomposition. Simulations under additive white Gaussian noise show that the new estimator offers significant improvements over conventional methods such as multiple signal classification (MUSIC), estimation of signal parameters via rotational invariance technique+maximum a posteriori, and entropy-based MUSIC. The improvement is particularly notable at larger orders or when frequency separation is small.

In this letter, a new subspace based estimator that can effectively provide the order and frequencies of multiple sinusoids in noise is proposed. The estimator, referred to as SAS-Est (Subspace Aligning and Separating Estimator), simultaneously seeks to separate the steering vectors from the noise subspace and align them to the signal subspace. The angles between subspaces and the generalized Kullback-Leibler divergence are used in characterizing the alignment and separation. Minimizing the divergence leads to maximal subspace separation and best alignment, thus allowing improved performance. Simulations in additive white Gaussian noise show that the new estimator offers an improvement for both model order and frequency estimation. When compared with other methods, the improvement is more pronounced for high model orders and low signal-to-noise ratio values.

For comments paper see D.B.H. Tay, ibid., vol.16, no.2, p.109-11, (2009). For original paper see R. Yu et al., ibid., vol.15, p.437-40, (2008). We would like to mention that a design method for orthogonal wavelets based on sum of squares decomposition is proposed. The method can guarantee the nonnegativity and leads to conjugate quadratic filters with certain properties.

This letter is concerned with design of halfband product filters for orthogonal wavelets. We first remark that the recent zero-pinning technique for orthogonal wavelet design cannot always guarantee the nonnegativity of the filter. We then propose to use sum of squares (SOS) decomposition to ensure its nonnegativity. The use of SOS decomposition also allows us to solve two optimization problems on the halfband filter via semidefinite programming. For a given length with pre-specified number of zeros at , we obtain halfband filters with maximal passband width. Design examples are provided.

We derive, via refinable super functions, the characterization of the approximation order of multi-scaling functions in both the time and frequency domains. It is shown that the approximation order is achieved if a linear operator, defined as the difference of the down-sampled convolution matrix and a matrix associated with the super function used, has a zero eigenvalue. The left eigenvectors associated with the zero eigenvalue define the combinations of scaling functions that produce the desired refinable super function. In the frequency domain, the approximation order condition is expressed in terms of the refinement masks of the multi-scaling functions and the refinable super function. It is shown that, implicit in this new characterization, there lie some well known results on approximation order. A matrix equality is derived that equates the presented frequency characterization and Strang's well known characterization of accuracy. It is shown that the approximation order of multi-scaling functions can always be achieved by a refinable, compactly supported super function.

In this work, the phase condition on the scaling filters of two wavelet bases which ensures that the corresponding wavelets are Hilbert transform pairs of each other is studied. An alternative and equivalent phase condition is derived. With the equivalent condition and Fourier series expansions, it is shown that the solution for which the corresponding scaling filters are offset from one another by a half sample is the only solution satisfying the phase condition which results in Hilbert transform pairs of wavelet bases.