We introduce a formalism that justifies and extends a heuristic method for algebraically decoupling deterministic features that recently proved useful for improving feature-based classification. Our new formalism is based on defining transformations inside manifolds, by following trajectories along the features' gradients. Through these transformations we define a normalization that, we demonstrate, allows for decoupling differentiable features. By applying this to sampling moments, we obtain a quasi-analytic solution for the orthokurtosis, a modification of the kurtosis that is not just decoupled from mean and variance, but also from skewness. After theoretically motivating feature decoupling for random data distributions, we illustrate with a regression problem example how decoupled features may perform significantly better than coupled ones.

Feature adjustment, understood as the process aimed at modifying at will global features of given signals, has cardinal importance for several signal processing applications, such as enhancement, restoration, style transfer, and synthesis. Despite of this, it has not yet been approached from a general, theory-grounded, perspective. This work proposes a new conceptual and practical methodology that we term Controlled Feature Adjustment (CFA). CFA provides methods for, given a set of parametric global features (scalar functions of discrete signals), (1) constructing a related set of deterministically decoupled features, and (2) adjusting these new features in a controlled way, i.e., each one independently of the others. We illustrate the application of CFA by devising a spectrally-based hierarchically decoupled feature set and applying it to obtain different types of image synthesis that are not achievable using traditional (coupled) feature sets.

Global features of signals are usually coupled, in the sense that not every combination of features' values is mathematically possible. The existence of forbidden combinations complicates the topology of the joint feature vector set, and precludes an interpretation of features as independent descriptors. Another consequence is that there is no easy and direct way of imposing a feature to the signal without perturbing others. Therefore, signal analysis, processing and synthesis applications based on global descriptors often resent from feature coupling. Here we first propose a sequential method for modifying arbitrary features independently of each other. Then we derive a decoupling method based on hierarchical normalization that yields new features with gradients mutually orthogonal everywhere in the signal space. Here we apply this method to the first ten marginal moments, emerging new sample statistics that, as we show, significantly improve texture classification.

In this paper, we describe and optimize a general scheme based on lifting transforms on graphs for video coding. A graph is constructed to represent the video signal. Each pixel becomes a node in the graph and links between nodes represent similarity between them. Therefore, spatial neighbors and temporal motion-related pixels can be linked, while nonsimilar pixels (e.g., pixels across an edge) may not be. Then, a lifting-based transform, in which filtering operations are performed using linked nodes, is applied to this graph, leading to a 3D (spatio-temporal) directional transform, which can be viewed as an extension of wavelet transforms for video. The design of the proposed scheme requires four main steps: 1) graph construction; 2) graph splitting; 3) filter design; and 4) extension of the transform to different levels of decomposition. We focus on the optimization of these steps in order to obtain an effective transform for video coding. Furthermore, based on this scheme, we propose a coefficient reordering method and an entropy coder leading to a complete video encoder that achieves better coding performance than a motion-compensated temporal filtering wavelet-based encoder, and a simple encoder derived from H.264/AVC that makes use of similar tools as our proposed encoder (reference software JM15.1 configured to use one reference frame, no subpixel motion estimation, and 16 × 16 inter and 4 × 4 intra modes).

The latest High Efficiency Video Coding standard (HEVC) provides a set of new coding tools to achieve a significantly higher coding efficiency than previous standards. In this standard, the pixels are first grouped into Coding Units (CU), then Prediction Units (PU), and finally Transform Units (TU). All these coding levels are organized into a quadtree-shaped arrangement that allows highly flexible data representation; however, they involve a very high computational complexity. In this paper, we propose an effective early CU depth decision algorithm to reduce the encoder complexity. Our proposal is based on a hierarchical approach, in which a hypothesis test is designed to make a decision at every CU depth, where the algorithm either produces an early termination or decides to evaluate the subsequent depth level. Moreover, the proposed method is able to adaptively estimate the parameters that define each hypothesis test, so that it adapts its behavior to the variable contents of the video sequences. The proposed method has been extensively tested, and the experimental results show that our proposal outperforms several state-of-the-art methods, achieving a significant reduction of the computational complexity (36.5% and 38.2% average reductions in coding time for two different encoder configurations) in exchange for very slight losses in coding performance (1.7% and 0.8% average bit rate increments).

The emerging high-efficiency video coding standard achieves higher coding efficiency than previous standards by virtue of a set of new coding tools such as the quadtree coding structure. In this novel structure, the pixels are organized into coding units (CU), prediction units, and transform units, the sizes of which can be optimized at every level following a tree configuration. These tools allow highly flexible data representation; however, they incur a very high computational complexity. In this paper, we propose an effective complexity control (CC) algorithm based on a hierarchical approach. An early termination condition is defined at every CU size to determine whether subsequent CU sizes should be explored. The actual encoding times are also considered to satisfy the target complexity in real time. Moreover, all parameters of the algorithm are estimated on the fly to adapt its behavior to the video content, the encoding configuration, and the target complexity over time. The experimental results prove that our proposal is able to achieve a target complexity reduction of up to 60% with respect to full exploration, with notable accuracy and limited losses in coding performance. It was compared with a state-of-the-art CC method and shown to achieve a significantly better trade-off between coding complexity and efficiency as well as higher accuracy in reaching the target complexity. Furthermore, a comparison with a state-of-the-art complexity reduction method highlights the advantages of our CC framework. Finally, we show that the proposed method performs well when the target complexity varies over time.

The motion estimation (ME) process used in the H.264/AVC reference software is based on minimizing a cost function that involves two terms (distortion and rate) that are properly balanced through a Lagrangian parameter, usually denoted as λmotion. In this paper we propose an algorithm to improve the conventional way of estimating λmotion and, consequently, the ME process. First, we show that the conventional estimation of λmotion turns out to be significantly less accurate when ME-compromising events, which make the ME process to perform poorly, happen. Second, with the aim of improving the coding efficiency in these cases, an efficient algorithm is proposed that allows the encoder to choose between three different values of λmotion for the Inter 16x16 partition size. To be more precise, for this partition size, the proposed algorithm allows the encoder to additionally test λmotion=0 and λmotion arbitrarily large, which corresponds to minimum distortion and minimum rate solutions, respectively. By testing these two extreme values, the algorithm avoids making large ME errors. The experimental results on video segments exhibiting this type of ME-compromising events reveal an average rate reduction of 2.20% for the same coding quality with respect to the JM15.1 reference software of H.264/AVC. The algorithm has been also tested in comparison with a state-of-the-art algorithm called context adaptive Lagrange multiplier. Additionally, two illustrative examples of the subjective performance improvement are provided.

Flicker is a common video-compression-related temporal artifact. It occurs when co-located regions of consecutive frames are not encoded in a consistent manner, especially when Intra frames are periodically inserted at low and medium bit rates. In this paper we propose a flicker reduction method which aims to make the luminance changes between pixels in the same area of consecutive frames less noticeable. To this end, a temporal low-pass filtering is proposed that smooths these luminance changes on a block-by-block basis. The proposed method has some advantages compared to another state-of-the-art methods. It has been designed to be compliant with conventional video coding standards, i.e., to generate a bitstream that is decodable by any standard decoder implementation. The filter strength is estimated on-the-fly to limit the PSNR loss and thus the appearance of a noticeable blurring effect. The proposed method has been implemented on the H.264/AVC reference software and thoroughly assessed in comparison to a couple of state-of-the-art methods. The flicker reduction achieved by the proposed method (calculated using an objective measurement) is notably higher than that of compared methods: 18.78% versus 5.32% and 31.96% versus 8.34%, in exchange of some slight losses in terms of coding efficiency. In terms of subjective quality, the proposed method is perceived more than two times better than the compared methods.

Flicker is a common video coding artifact that occurs especially at low and medium bit rates. In this paper we propose a temporal filter-based method to reduce flicker. The proposed method has been designed to be compliant with conventional video coding standards, i.e., to generate a bitstream that is decodable by any standard decoder implementation. The aim of the proposed method is to make the luminance changes between consecutive frames smoother on a block-by-block basis. To this end, a selective temporal low-pass filtering is proposed that smooths these luminance changes on flicker-prone blocks. Furthermore, since the low-pass filtering can incur in a noticeable blurring effect, an adaptive algorithm that allows for limiting the PSNR loss -and thus the blur- has also been designed. The proposed method has been extensively assessed on the reference software of the H.264/AVC video coding standard and compared to a state-of-the-art method. The experimental results show the effectiveness of the proposed method and prove that its performance is superior to that of the state-of-the-art method.

The basis functions of lifting transform on graphs are completely determined by finding a bipartition of the graph and defining the prediction and update filters to be used. In this work we consider the design of prediction filters that minimize the quadratic prediction error and therefore the energy of the detail coefficients, which will give rise to higher energy compaction. Then, to determine the graph bipartition, we propose a distributed maximum-cut algorithm that significantly reduces the computational cost with respect to the centralized version used in our previous work. The proposed techniques show improvements in coding performance and computational cost as compared to our previous work.

The latest H.264/AVC video coding standard achieves high compression rates in exchange for high computational complexity. Nowadays, however, many application scenarios require the encoder to meet some complexity constraints. This paper proposes a novel complexity control method that relies on a hypothesis testing that can handle time-variant content and target complexities. Specifically, it is based on a binary hypothesis testing that decides, on a macroblock basis, whether to use a low- or a high-complexity coding model. Gaussian statistics are assumed so that the probability density functions involved in the hypothesis testing can be easily adapted. The decision threshold is also adapted according to the deviation between the actual and the target complexities. The proposed method is implemented on the H.264/AVC reference software JM10.2 and compared with a state-of-the-art method. Our experimental results prove that the proposed method achieves a better trade-off between complexity control and coding efficiency. Furthermore, it leads to a lower deviation from the target complexity.

We present a new graph-based transform for video signals using wavelet lifting. Graphs are created to capture spatial and temporal correlations in video sequences. Our new transforms allow spatial and temporal correlation to be jointly exploited, in contrast to existing techniques, such as motion compensated temporal filtering, which can be seen as "separable" transforms, since spatial and temporal filtering are performed separately. We design efficient ways to form the graphs and to design the prediction and update filters for different levels of the lifting transform as a function of expected degree of correlation between pixels. Our initial results are promising, with improvements in performance as compared to existing methods in terms of PSNR as a function of the percentage of retained coefficients of the transform.

We propose a complete video encoder based on directional "non- separable" transforms that allow spatial and temporal correlation to be jointly exploited. These lifting-based wavelet transforms are applied on graphs that link pixels in a video sequence based on motion information. In this paper, we first consider a low complexity version of this transform, which can operate on subgraphs without significant loss in performance. We then study coefficient reordering techniques that lead to a more realistic and efficient encoder that the one we presented in our earlier work. Our proposed technique shows encouraging results as compared to a comparable scheme based on the DCT transform.

The H.264/AVC standard achieves a high coding efficiency compared to previous standards. However, this gain is accomplished at great computational cost, with mode decision being one of the most demanding subsystems. In this paper, a two-level classification-based approach to the inter mode decision problem is proposed. A first classifier detects SKIP/Direct modes, while a second one is able to decide whether to use a large (16 × 16, 16 × 8, and 8 × 16) or a small mode (8 × 8, 8 × 4, 4 × 8, and 4 × 4). The suggested classifiers are binary and linear, and the input features in the classifiers have been carefully selected. A novel cost function that pays more attention to the most critical samples during the classifier training process has been designed. The experimental results show an average computational savings of 60% of the total encoding time with respect to JM10.2 over a comprehensive variety of sequences and formats. This is achieved with negligible degradation in rate-distortion performance and compares favorably with state-of-the-art fast mode decision methods. Furthermore, the proposed method has been successfully assessed at different levels of complexity reduction.

The H.264/AVC standard achieves a high coding efficiency compared with previous standards. However, it does so at a very high computational cost, with motion estimation being one of the most demanding subsystems. In this paper a hierarchical classificationbased approach to the Inter Mode Decision (MD) problem is proposed. A first classifier detects SKIP/Direct modes while a second one is able to decide whether using a large (16×16, 16×8 and 8×16) or a small mode (8×8, 8×4, 4×8 and 4×4). The same procedure is applied at SubMacroblock level. The suggested classifiers are binary and linear. The input features that feed both classifiers have been carefully selected. A novel cost function that pays more attention to the most critical samples during the classifier training process has been designed. The results are very promising: a 64 % computational saving of the total encoding time with respect to JM10.2 is achieved with negligible degradation in the rate-distortion performance.

The H.264/AVC standard achieves a high coding efficiency compared to previous standards, but in exchange for a very high computational cost. This paper focuses on the mode decision subsystem, which is a critical one from the computational cost point of view. In particular, a hierarchical early-termination-based approach to the inter mode decision problem is proposed, in such a way that a cascade of early stops are considered, going from the most likely modes to the less likely ones. The main novelty of the proposed algorithm relies on the use of a regionalized hypothesis testing methodology for making the decisions on every potential early stop. The proposed method has been assessed on a large set of QCIF video sequences for several qualities, achieving a quite notable improvement in terms of total encoding time reduction in relation to a well-established mode decision method. Specifically, the time reduction is 43.43 % and 21.33 % for IPPP and IBBP patterns, respectively, without any significant quality loss.

The H.264/AVC standard achieves a high coding efficiency compared to previous standards. However, the encoder complexity results in very high computational cost due to motion estimation and macroblock mode decisions. In this paper we propose a fast mode decision for low computational complexity applications for which the rate distortion optimization mode decision becomes unacceptable. The proposed pruned mode decision method consists in a motion-cost based early termination algorithm and saves about 50% encoding time with negligible quality loss.