Rate control is a critical component for image and video compression Particularly under limited network bandwidth conditions, bitrate control is essential to ensure efficient image transmission by effectively allocation channel resources. In this research, since both Channel and Spatial have relationship with rate allocation, we first propose a joint Channel-wise and Spatial-wise Quantization scheme to determine optimal quantization parameters. Subsequently, we develop a quantization step estimation network to obtain parameters to efficiently allocate rate according to target rate. Experiments demonstrate that our algorithm significantly improve compressed image quality with minimal bitrate distortion and achieve accurate rate control with nearly 3% average bitrate error.

In light of the growing popularity of screen content video applications, there is a increasing demand for Screen Content Coding (SCC). The latest standard, Versatile Video Coding (VVC), exhibits exceptionally high coding efficiency, albeit accompanied by a considerable coding complexity. This complexity, in turn, restricts the widespread applicability of VVC SCC. To address this issue, this paper introduces a parallel based approach to enhance the coding speed of VVC SCC Intra Coding. Specifically, we established a large-scale database and then design distinct neural networks for Coding Units (CUs) of various sizes to predict candidate Coding Modes (CMs). Subsequently, we formulate a loss function based on CM distributions and Rate Distortion(RD) costs to train the designed models. Finally, we introduce a threshold selection scheme to balance coding efficiency and coding speed. Experimental results demonstrate that the proposed method improves coding speed by an average of 36.36%, with an average increase of 0.95% in Bjøntegaard Delta Bit Rate (BDBR).

Learning-based methods have proven successful in compressing geometric information for point clouds. For attribute compression, however, they still lag behind non-learning-based methods such as the MPEG G-PCC standard. To bridge this gap, we propose a novel deep learning-based point cloud attribute compression method that uses a generative adversarial network (GAN) with sparse convolution layers. Our method also includes a module that adaptively selects the resolution of the voxels used to voxelize the input point cloud. Sparse vectors are used to represent the voxelized point cloud, and sparse convolutions process the sparse tensors, ensuring computational efficiency. To the best of our knowledge, this is the first application of GANs to compress point cloud attributes. Our experimental results show that our method outperforms existing learning-based techniques and rivals the latest G-PCC test model (TMC13v23) in terms of visual quality.

Screen content image (SCI) compression faces challenges due to distinct characteristics such as sharp edges and repetitive structures. Existing learned image compression methods encounter two key issues: 1) insufficient frequency-aware processing, and 2) suboptimal entropy modeling for mixed-frequency components. To this end, we propose OMR-Net+, a novel SCI compression method that incorporates frequency-aware feature characteristics, including a frequency-aware refinement network (FARN) and a frequency-aware entropy model (FAEM). The proposed FARN uses an invertible neural network to preserve critical high-frequency details and a transformer-based model to reduce redundancy in low-frequency features. Additionally, the proposed FAEM provides tailored conditional probability estimation based on a parallel context model for high- and low-frequency features, respectively, to improve both coding performance and computational efficiency. Experimental results on the SCID and SIQAD datasets show that OMR-Net+ significantly outperforms the previous OMR-Net and other state-of-the-art methods in rate-distortion performance, demonstrating its potential for efficient SCI compression.

Versatile Video Coding (VVC) now supports Screen Content Coding (SCC) by integrating two efficient coding modes: Intra Block Copy (IBC) and Palette (PLT). However, the numerous modes and the Quad-Tree Plus Multi-Type Tree (QTMT) structure inherent to VVC contribute to a very high coding complexity. To effectively reduce the computational complexity of VVC SCC, we propose a fast Intra mode prediction algorithm for VVC SCC. More specifically, we first use the difference of minimum Sum of Absolute Transformed Differences (SATD) value of four Directional Modes (DMs) of Intra and the SATD value of the IBC-merge mode to determine whether to early skip Intra checking. Subsequently, we use a decision tree to determine whether to early terminate the checking after block differential pulse coded modulation (BDPCM). Finally, we employ a decision tree to determine whether to early skip multiple transform selection (MTS) and low frequency non-separable transform (LFNST) checking. The results demonstrate that our algorithm achieves an average encoding time reduction of 34.34% with a negligible Bjøntegaard delta bitrate increase of 0.46%.

Currently, screen content video applications are increasingly widespread in our daily lives. The latest Screen Content Coding (SCC) standard, known as Versatile Video Coding (VVC) SCC, employs screen content Coding Modes (CMs) selection. While VVC SCC achieves high coding efficiency, its coding complexity poses a significant obstacle to the further widespread adoption of screen content video. Hence, it is crucial to enhance the coding speed of VVC SCC. In this paper, we propose a fast mode and splitting decision for Intra prediction in VVC SCC. Specifically, we initially exploit deep learning techniques to predict content types for all CUs. Subsequently, we examine CM distributions of different content types to predict candidate CMs for CUs. We then introduce early skip and early terminate CM decisions for different content types of CUs to further eliminate unlikely CMs. Finally, we develop Block-based Differential Pulse-Code Modulation (BDPCM) early termination to improve coding speed. Experimental results demonstrate that the proposed algorithm can improve coding speed by $34.95 \%$ on average while maintaining almost the same coding efficiency.

In point cloud geometry compression, most octree-based context models use the cross-entropy between the one-hot encoding of node occupancy and the probability distribution predicted by the context model as the loss. This approach converts the problem of predicting the number (a regression problem) and the position (a classification problem) of occupied child nodes into a 255-dimensional classification problem. As a result, it fails to accurately measure the difference between the one-hot encoding and the predicted probability distribution. We first analyze why the cross-entropy loss function fails to accurately measure the difference between the one-hot encoding and the predicted probability distribution. Then, we propose an attention-based child node number prediction (ACNP) module to enhance the context models. The proposed module can predict the number of occupied child nodes and map it into an 8-dimensional vector to assist the context model in predicting the probability distribution of the occupancy of the current node for efficient entropy coding. Experimental results demonstrate that the proposed module enhances the coding efficiency of octree-based context models.

Due to multi-layer encoding and Inter-layer prediction, Spatial Scalable High-Efficiency Video Coding (SSHVC) has extremely high coding complexity. It is very crucial to improve its coding speed so as to promote widespread and cost-effective SSHVC applications. In this paper, we have proposed a novel Mode Selection-Based Fast Intra Prediction algorithm for SSHVC. We reveal the RD costs of Inter-layer Reference (ILR) mode and Intra mode have a significant difference, and the RD costs of these two modes follow Gaussian distribution. Based on this observation, we propose to apply the classic Gaussian Mixture Model and Expectation Maximization in machine learning to determine whether ILR is the best mode so as to skip the Intra mode. Experimental results demonstrate that the proposed algorithm can significantly improve the coding speed with negligible coding efficiency loss.

As the latest video coding standard, Versatile Video Coding (VVC) is highly efficient at the cost of very high coding complexity, which seriously hinders its widespread application. Therefore, it is very crucial to improve its coding speed. In this paper, we propose a learning-based fast split type (ST) prediction algorithm for VVC using a deep learning approach. We first construct a large-scale database containing sufficient STs with diverse video resolution and content. Next, since the ST distributions of coding units (CUs) of different sizes are significantly distinct, so we separately design neural networks for all different CU sizes. Then, we merge ambiguous STs into four merged classes (MCs) to train models to obtain probabilities of MCs and skip unlikely ones. Experimental results demonstrate that the proposed algorithm can reduce the encoding time of VVC by 67.53% with 1.89% increase in Bjøntegaard delta bit-rate (BDBR) on average.

To seamlessly adapt to time-varying network bandwidths, Quality Scalable High-Efficiency Video Coding (QSHVC) is developed. However, its coding process is overwhelmingly complex, and this seriously limits its wide applications in realtime environments. Therefore, it is of great significance to study fast coding algorithms for QSHVC. In this paper, we propose a novel probability-based All-Zero Block (AZB) early termination algorithm for QSHVC. We observe that the generated residual coefficients follow the Laplace distribution if a CU is accurately predicted. Based on this observation, we derive the sum of squared differences-based AZB decision condition. Second, the probability of each coding mode and coding depth being chosen as the best ones are combined with AZBs to derive the probability-based early termination condition. The experimental results show that the proposed algorithm can improve the average coding speed by 74.95% with a 0.26% increase in BDBR.

Detection of infrared dim and small targets with diverse and cluttered background plays a significant role in many applications. In this paper, we propose a deep low-rank and sparse patch-image network, termed as Deep-LSP-Net, to effectively detect small targets in a single infrared image. Specifically, by using the local patch construction scheme, we first transform the original infrared image into a patch-image, which can be decomposed as a superposition of the low-rank background component and the sparse target component. The target detection is thus formulated as an optimization problem with low-rank and sparse regularizations, which can be solved by the alternating direction method of multipliers (ADMM). We unroll the iterative algorithm into deep neural networks, where a generalized sparsifying transform and a singular value thresholding operator are learned by the convolutional neural networks (CNNs) to avoid tedious parameter tuning and improve the interpretability of the neural networks. We conduct comprehensive experiments on two public datasets. Both qualitative and quantitative experimental results demonstrate that the proposed algorithm can obtain improved performance in small infrared target detection compared with state-of-the-art algorithms.

Due to the diversity of terminal devices, Spatial Scalable High Efficiency Video Coding (SSHVC) is an efficient solution to meet this requirement. However, its coding process is very complex, which seriously prevents its wide applications. Therefore, it is very crucial to reduce coding complexity and improve coding speed. In this paper, we propose a Gaussian Distribution-based Mode Selection for Intra Prediction of SSHVC. We show that the rate distortion costs of Inter-layer Reference (ILR) mode and Intra mode are significantly different, and both follow a Gaussian distribution. Based on this discovery, we propose to use a Bayes decision rule to determine whether ILR is the best mode so as to skip Intra mode. Experimental results demonstrate that the proposed algorithm can significantly improve coding speed with negligible coding efficiency losses.

Scalable High-efficiency Video Coding (Scalable HEVC, SHVC) is the most recent scalable video coding standard, which is developed by the Joint Collaborative Team on Video Coding (JCT-VC). In order to meet the needs of various client terminals, network conditions, and user demands, a multi-layer encoding framework is employed to generate the scalable bitstream. In addition to the desirable scalabilities, SHVC also aims to achieve better coding efficiency than previous standards. Therefore, all the advanced coding tools ofHEVC are inherited, and several additional tools are specifically designed to support scalability. However, the superior coding efficiency is achieved at the cost of significantly increased computational complexity. Given the fact that an SHVC encoder is considerably more complex than an HEVC encoder, this makes a great demand of a fast SHVC encoder that incurs only a negligible degradation in compression efficiency. In order to address the above issue, several fast schemes have been suggested for SHVC intra and inter coding. However, temporal, spatial and inter-layer correlations have not been fully exploited. In addition, most existing algorithms aimed only at the improvement either on the intra coding process or the inter coding process. If all the available information including the texture complexity, the motion activity, as well as different types of correlation can be appropriately utilised, it is expected that the computational complexity can be reduced further. Therefore, a fast Coding Unit (CU) size decision algorithm is proposed for spatial SHVC intra coding to early terminate the CU size decision process. The energy distribution is exploited to classify the texture complexity, and the available information from the BL is used to narrow the coding depth level ranges for the LCUs in the EL. Unlikely intra modes are removed by considering the inter-layer, spatial and temporal correlations. Different candidate mode lists are constructed for the PUs of various sizes. In addition, the implementation after RMD is also improved. A fast Prediction Unit (PU) prediction mode decision method is also suggested for SHVC inter coding, in which the motion complexity and the inter-layer dependency of PU modes are jointly used to determine the best PU prediction mode for the PU in the EL. The proposed algorithms were implemented in the SHVC reference software SHM 12.0. Two-layers encoding structure was employed for spatial scalability, and the scalability factor was 2. Common test conditions were satisfied. The All Intra (AI) and Lowdelay configurations were employed to evaluate the proposed fast intra and inter coding algorithms. The quantisation parameters (QPs) for the BL and EL were (22,20), (27,25), (32,30) and (37,35). Experimental results showed that the proposed fast intra algorithm reduces the encoding time by 48% on average, while the proposed fast inter algorithm achieves an average time reduction of more than 36% over the unchanged SHM encoder, and the loss in Rate-Distortion (RD) performance is acceptable. It can be concluded that the proposed algorithms outperform the algorithms previously proposed.

Scalable High Efficiency Video Coding (SHVC) provides high compression efficiency at the expense of considerable computational complexity. In this paper, a fast algorithm is proposed to reduce the computational complexity of SHVC intra coding. The coding depth information, texture complexity, and spatio-temporal correlation are jointly used to achieve a faster depth decision process. The mode dependency between the base layer and the enhancement layer is combined with the temporal correlation to simplify the mode decision process. Experimental results demonstrate that the proposed scheme saves encoding time by up to 59% compared with the SHM12.0 encoder with negligible degradation in Rate Distortion (RD).

In High Efficiency Video Coding (HEVC), a quad-tree based Coding Unit (CU) partitioning scheme is employed, achieving a substantial improvement in coding efficiency compared with previous standards. The superior coding efficiency of HEVC is achieved at the expense of greatly increased complexity. A fast intra coding scheme consisting of fast CU depth decision and fast prediction mode decision is proposed to reduce the computational requirement. Classification of the homogeneity of video content using an adaptive double thresholds scheme is employed to reduce the number of Rate Distortion (RD) evaluations. The partition information of spatially neighbouring CUs is utilised to further narrow the depth range. The construction of the initial candidate list is improved for each Prediction Unit (PU). Then the prediction mode correlation between neighbouring quad-tree coding levels is considered to predict the most likely coding mode. The Hadamard cost of prediction modes is examined to further reduce the candidate modes. The computational complexity of HEVC intra coding is therefore reduced. Simulation results show that the proposed algorithm reduces encoding time by up to 71% compared with the HM 13.0 implementation, while having a negligible impact on rate distortion, with increases in bit-rate of 1.82%.

We introduce a novel image regularization termed as multiple degree total variation (MDTV). This type of regularization combines the first and second degree directional derivatives, thus providing a good balance between preservation of edges and region smoothness. In order to solve the resulting optimization problem, we proposed a fast majorize minimize algorithm. We demonstrate the utility of the MDTV regularization in the context of image denoising and compressed sensing. We compare the proposed method with standard TV, and the state of the art higher degree methods, including higher degree total variation (HDTV) and total generalized variation (TGV) based schemes. Numerical results indicate that MDTV penalty provides improved image recovery performance.

HEVC employs a quad-tree based Coding Unit (CU) structure to achieve a significant improvement in coding efficiency compared with previous standards. However, the computational complexity is greatly increased. We proposed a fast mode decision algorithm to reduce intra coding complexity. Firstly, an initial candidate list of intra modes is constructed for each Prediction Unit (PU). The prediction mode correlation between adjacent quad-tree coding levels and between temporal neighbouring frames is used to predict the most likely coding mode. The number of prediction mode that need to be evaluated in residual quad-tree (RQT) process is further reduced by taking the Hadamard cost of prediction mode into consideration. Simulation results show that the proposed algorithm saves encoding time by up to 51% compared with the HM 13.0 implementation, while having a negligible impact on rate distortion.

High Efficiency Video Coding (HEVC) incorporates a flexible quad-tree block partitioning scheme and up to 35 prediction modes for intra coding. This enables a significant improvement in coding efficiency compared with previous standards. The superior coding efficiency of HEVC is achieved at the expense of greatly increased complexity. A fast Coding Unit (CU) depth decision algorithm is proposed to reduce the computational requirement for intra coding. An adaptive double thresholds scheme is employed to classify the homogeneity of video content. The classification is used to reduce the number of Rate Distortion (RD) evaluations in the CU depth decision process. The partition information of the temporally co-located CU and the spatially neighbouring CUs is jointly utilised to further narrow the depth range that needs to be evaluated. The computational complexity of HEVC intra coding is therefore reduced. Simulation results show that the proposed algorithm reduces encoding time by up to 57% compared with the HM 13.0 implementation, while having a negligible impact on rate distortion, with PSNR losses of 0.01dB and increases in bit-rate of 0.31%.

An improved rate control algorithm for the Scalable Video Coding (SVC) extension of H.264/AVC is described. The rate control scheme applied to the Base Layer (BL) of SVC adopts the linear Mean Absolute Difference (MAD) prediction and quadratic Rate Distortion (RD) models inherited from H.264/AVC. A MAD prediction error always exists and cannot be avoided. However, some encoding results of the base layer can be used to inform the coding of the enhancement layers (ELs), thus benefitting from the bottom-up coding structure of SVC. This property forms the basis for the proposed rate control approach. Simulation results show that accurate rate control is achieved and, compared to the default rate control algorithm of SVC, namely the JVT-G012 rate control scheme, the average PSNR is increased by 0.27dB or the average bit rate is reduced by 4.81%.

In Scalable Video Coding (SVC), inter-layer prediction is utilized to improve the coding efficiency of the enhancement layers. However the rate control scheme in the Joint Scalable Video Model (JSVM) software lacks consideration of the implications of inter-layer prediction as it was designed for non-scalable video encoders. In this work, a novel rate control algorithm is proposed for when inter-layer prediction is employed in SVC. Firstly, a Rate-Quantization (R-Q) model for inter-layer prediction coding of the spatial enhancement layers is developed. Secondly, an optimized Mean Absolute Difference (MAD) prediction model for spatial enhancement layers is proposed, that considers the MAD from previous temporal frames and previous spatial frames together. Simulation results show that rate control accuracy is maintained to within 0.07% on average. Compared with the default rate control algorithm employed in SVC, namely the JVT-G012 rate control scheme, the proposed algorithm improves the average PSNR by up to 0.26dB or produces an average saving in bit rate of up to 4.66%.

A fast mode decision algorithm for efficient implementation of the scalable video coding (SVC) extension of H.264/AVC is described. SVC incorporates interlayer prediction, a new tool that exploits as much lower layer information as possible in order to improve the coding efficiency of the enhancement layer. However, it also greatly increases the computational complexity. A fast mode selection algorithm that exploits the correlation of a macroblock in the enhancement layer and both the colocated macroblocks in the base layer and neighboring macroblocks in the enhancement layer is proposed. The algorithm examines the level of picture details and motion activity, and utilizes the mode information of the base layer to make faster enhancement layer decisions and thus save coding time. Simulation results show that the proposed algorithm reduces encoding by up to 84% compared with the JSVM 9.18 implementation. This is achieved without any noticeable degradation in rate distortion.

In order to improve coding efficiency, a new inter-layer prediction mechanism is incorporated in the SVC extension of H.264/AVC. This utilizes information from the base layer to inform the process of coding the enhancement layer. However this increases the computational requirement. A fast mode decision algorithm that exploits the correlation of a macroblock in the enhancement layer and both the corresponding macroblock in the base layer and neighbouring macroblocks, is proposed. The algorithm also assesses the homogeneity of the picture content and uses the mode information of the base layer to make faster mode selections in the enhancement layer. The fact that larger partition sizes are more suitable for homogeneous regions, and vice versa, is also exploited. Empirical evaluation of the proposed algorithm shows that, for similar rate distortion performance, encoding time is reduced by up to 84% compared with the JSVM9.18 software implementation.