Panoramic images are critical for immersive VR/AR and 6DoF yet degraded by compression artifacts, projection distortion, and uneven sampling, with existing hybrid CNN-Transformer models struggling to reconcile fine details and structural consistency in panoramas; to address this, we propose Dynamic Channel Collaboration (DCC-Former) for panoramic enhancement, inspired by primate vision's hierarchical processing and three strategies: strengthening local feature representation via reparameterization and gating, enhancing global context with adaptive self-attention, and enabling cross-scale aggregation through cascaded multi-scale fusion, aligned with biological vision's ventral-dorsal stream division and fovea-periphery resource allocation to balance detail preservation, global consistency, and computational efficiency-extensive experiments on benchmark datasets demonstrate DCC-Former outperforms SOTA in restoration quality and inference efficiency, providing a practical-efficient paradigm for high-resolution panoramic enhancement.

This paper proposes a novel 3D neural network for RGB-D image compression, which treats depth as a temporal dimension of RGB and leverages 3D convolutions for unified cross-modal modeling. Specifically, the network employs a parallel Hybrid CNN and Gated SwinT-Mamba (HCGSM) module to jointly capture multi-scale local details and long-range global dependencies. Furthermore, it introduces a channel-wise autoregressive entropy model enhanced with a lightweight Feature Enhanced Attention (FEAtten) module to improve latent representation coding efficiency and preserve structural fidelity.

This paper proposes an image compression model based on high-low frequency decomposition and structural priors, as illustrated in Fig. 1. Specifically, we first employ Discrete Wavelet Transform (DWT) to decompose the image into high- and low-frequency components. Simultaneously, a Structural Prior Feature Extraction (SPFE) module is adopted to extract inherent structural information from medical images, which is incorporated into both the encoder and the entropy model at multiple scales as prior knowledge. Additionally, we design a Grayscale Feature Extraction (GFE) module by performing directional pooling and feature fusion on the image to further capture the gradually varying intensity characteristics commonly present in medical images.

This work proposed a non-reference Just Recognized Distortion (JRD) prediction framework for object detection task based on Generative Adversarial Network (GAN) image generation. Inspired by the concept of Just Noticeable Difference (JND), the JRD is used to describe the threshold of image distortion acceptable for machine vision tasks. Centered around JRD, the proposed framework primarily consisted of a JRD image generation network and a residual-guided JRD regression network. Among them, the JRD image generation network was trained in conjunction with a multi-scale discriminator in the form of GAN. Additionally, we had compiled a dataset comprising over 130,000 images for the YOLOv7 object detection task, and it was used to validate the effectiveness of our proposed framework. Experiments indicated that the framework can approximate the JRD for object detection task with notable accuracy. Consequently, adopting our proposed framework for image compression could reduce the bitrate by 50% while maintaining high accuracy in task.

The diversity of video content and unpredictability of distortions in user-generated content (UGC) videos pose a challenge for video quality assessment (VQA). Most existing methods are difficult to model complete visual perception loop to accurately capture video content and predict perceived quality. Thus, as shown in Figure 1 , this paper proposes a bidirectional hierarchical semantic extraction structure for VQA (BHSE-VQA), which simulates visual feedforward and feedback perception processes. Firstly, the feedforward and feedback multi-level network (FFMNet) based on the reverse hierarchy theory is designed to extract and adjust hierarchical semantic features on the bidirectional pathway. Then, considering the different effects of feature depth on visual perception results, this paper suggests a multi-level weight redistribution (MWR) strategy that makes use of the attention characteristics of the feedback outputs to realign the weights at each stage of the feedforward outputs. Finally, through a temporal attention fusion network (TAFNet), this paper further extracts the detailed features arising from feedforward and feedback perceptual differences and obtains quality scores. The experimental results show that the proposed model exceeds 0.845 on both SRCC and PLCC on YouTube-UGC database.

An audio-video collaborative JND estimation model for multimedia data was proposed in this paper by delving into the impact of audio signals on HVS. Firstly, we extract and analyze the temporal perceptual features of audio in terms of loudness, duration, and audio energy. Secondly, the spatial feature of audio is analyzed using audio-guided visual saliency. Based on the temporal and spatial perceptual features of audio, an audio adjustment factor is designed and integrated with a video-based JND model to propose the audio-video collaborative JND estimation model. Detailed performance results of different JND models are shown in Table. 1 . The simulation results validate that the proposed JND model significantly improves the JND estimation accuracy of multimedia data with excellent performance and stronger distortion concealment ability, with strong competitiveness among the state-of-the-art models.

There is larger compression potential for surveillance video coding due to the inherent super-redundancy in fixed camera scenarios. So, this paper propose a novel compression framework for indoor surveillance video.

Video quality assessment (VQA) for user generated content (UGC) videos plays important role in video compression and processing. Convolutional neural network (CNN) based quality assessment for UGC is the research focus with inspiring model accuracy increment in the past three years. However, regularly temporal-sampling with temporal feature loss, as well as fixed token selection strategy video transformer (ViT) with insufficient representational capacity of tokens, jointly degrade the accuracy of conventional ViT based quality assessment. Facing these two challenges, this article proposes an adaptive token-selection ViT (ATSViT) structure for UGCVQA. Accounting for the uneven distribution of spatiotemporal distortion-related features, this work proposes a timing block sampling (TBS) module to adaptively select video blocks and assemble them into content compacted subsequence for further processing. In addition, inspired by the mental filter theory in terms of visual information, we propose a stage-wise adaptive screening network (SSNet) in which (noise” features of tokens in the sense of perception are progressively detected and processed by imitating the behavior of perception process in the eye-brain system. Experimental results verify that the proposed VQA model achieves state-of-the-art (SOTA) accuracy, with the highest correlation with mean opinion scores (MOS).

The 3D extension of High Efficiency Video Coding (3D-HEVC) achieves excellent per-formance for 3D video coding while possessing significant computational complexity. To accelerate the time-consuming coding process of the depth map, a fast algorithm via XG-Boost is proposed in this paper. Specifically, a total of 14 specialized XGBoost models are used for different block sizes and viewpoint types to achieve early coding unit partition de-termination (ECP) and early prediction unit mode selection (EPM) to avoid executing the exhaustive traversal coding process. To promote the prediction accuracy of XGBoost mod-els, multi-domain correlations, including spatiotemporal, inter-view, and inter-component correlations are utilized and plenty of features are selected for model training. Evaluated on HTM-16.0 under random access configuration, the proposed ECP strategy can obtain 51.2% total encoding time saving with a 0.18% BDBR increase and the ECP+EPM can overall achieve 60.8% total encoding time saving with a 0.59% BDBR increase. The source code of our method is available at https://github.com/Joeyrr/_EPM.git.

Bi-prediction is a fundamental module of inter prediction in the blocked-based hybrid video coding framework. Block-based motion estimation(ME) and motion compensation(MC) with simple models are adopted in bi-prediction process. Unfortunately, this MEMC-based scheme can't guarantee the prediction performance when it comes to video with complicated motions. In this paper, a novel inter prediction scheme based on deep frame prediction network (DFP-net) is proposed to enhance bi-prediction accuracy especially in complicate scenes. Specifically, the proposed DFP-net is composed of multi-scale motion alignment, fusion of temporal and spatial correlation and frame synthesis module. The DFP-net can precisely extract and fuse motion features in various scales and completely exploit temporal and spatial correlation to generate the prediction frame in a data-driven manner. Moreover, the DFP-net is integrated into VTM-6.2 to provide an additional prediction frame for biprediction. Since the prediction generated by DFP-net is more similar with to-be-coded frame in the sense of temporal distance and texture, it can be added to reference list to improve the diversity of references. In this manner, the proposed bi-prediction scheme has surpassed VTM-6.2 on average 1.8% BD-rate saving.

In the latest video coding standard, Versatile Video Coding (H.266/VVC), a new quadtree with nested multi-type tree (QTMTT) coding block structure is proposed. QTMTT significantly improves coding performance, but more complex block partitioning structure brings greater computational burden. To solve this problem, a fast intra block partition pattern pruning algorithm is proposed using gray level co-occurrence matrix (GLCM) to calculate texture direction information of coding units, terminating the horizontal or vertical split of the binary tree and the ternary tree in advance. Experimental results show that the proposed algorithm achieves up to 53.57% encoding time saving on average with negligible quality loss under all-intra conguration.

Conventional intra prediction usually utilizes the top and left reconstructed reference samples of the current block to generate prediction pixels. However, with the distance between reference samples and the predicting pixel increasing, the correlation of them becomes weaker. The loss of the bottom-right corner of the current block is bigger than that of the top-left corner. To improve the situation above mentioned, a novel secondary intra prediction scheme is proposed for video coding in this paper. In this scheme, the block will be predicted twice and the first prediction is the same as the conventional intra prediction. After the first prediction, the boundary samples of the reconstructed block can be applied as the reference samples for the second prediction. In our second prediction, the current block is predicted with a designed distance-based bi-directional prediction method. The optimal prediction is determined through the rate-distortion optimization (RDO) strategy. Experimental results show that the proposed algorithm achieves about 3.4% bitrate savings on average comparing to HEVC reference software HM16.9.

The just noticeable distortion (JND) has been widely applied in perceptual image/video compression. Yet, the existing JND estimation models are not accurate enough, which results in the degradation of perceptual quality. In this paper, we propose a JND compensation based perceptual video coding (PVC) scheme to compress videos with better perceptual quality. Specifically, a block-level JND estimation model is proposed at first, which leads to this model can be employed in variable block-sizes based video coding directly. Then, considering the quantization distortion (QD), a QD&JND-based residue filter is proposed to suppress the residues in the encoder. To improve the quality of the reconstructed video, a QD&JND-based compensation scheme is developed in the decoder. Experimental results show that the QD&JND-based filter achieves bit rate saving significantly, and the QD&JND compensation scheme improves the perceptual quality of the compressed videos effectively.

With the rapid development of Internet, short videos draw more and more attentions nowadays. Due to the small scale of short videos, image-level coding scheme can be applied to improve compression efficiency. In this paper, we propose a densely connected unit based loop filter for short video coding in H.266/VVC, as Fig. 1 shows. In the proposed loop filter, the densely connected units are specially designed to extract feature maps, and fully decompose videos. By densely connection between layers, the designed units can reuse feature maps, and reduce the redundancy of features. Meanwhile, the proposed loop filter has the light weight, which can recover short videos at image level fast. Moreover, the proposed loop filtering network is embedded into VVC to replace the original sample adaptive offset (SAO) and adaptive loop filter (ALF), which saves bits cost and lowers the complexity of coding. The experimental results show that the proposed loop filter, compared with H.266/VVC, can achieve better performance in both AI and RA mode.

Convolutional neural network has witnessed remarkable progress in compressed video quality enhancement in high efficiency video coding (HEVC) standard. But most existing methods focus on single frame quality enhancement where copious temporal and spatial information is neglected. In this paper, we propose a spatial-temporal fusion convolutional neural network (STEF-CNN) to employ spatial and temporal information to improve the performance of in-loop filter in HEVC. Specifically, the STEF-CNN adopts a pre-denoising network which in advance processes the compressed videos frame by frame. The pre-denoising operation alleviates the impact of noise and blocking artifacts. Then the denoised frames are sent to temporal-spatial fusion module which picks out valuable temporal and spatial information. The fused frames are eventually fed to quality enhancement network which is based on residual learning and dense network. The STEF-CNN is capable of capturing abundant information from consecutive neighboring frames. Extensive experimental results demonstrate the effectiveness of the proposed method. The STEF-CNN achieves 11.53% BD-BR reduction in all-intra (AI) configuration and 10.20% BD-BR reduction in random-access (RA) configuration.

In order to achieve highly compact representation for videos, we propose an adaptive quantization based perceptual video coding (PVC) scheme in this paper. Because human only perceive the limited discrete-scale quality levels, the perceptual quantization is transformed into the problem of how to determine the maximum quantization parameter (Qp) under the same perceptual quality level. So, the relationship between perceptual quality level and quantization parameter is analyzed with the statistical way in this paper. The frame-level Qp value for each quality level is determined based on the maximum probability criterion. Then, the just noticeable distortion is estimated to guide the Qp adjustment in the coding unit level (CU-level). In summary, the perceptual quantization is achieved in both the frame level and the CU level according to characteristics of human visual system (HVS). Experimental results show that the proposed PVC scheme achieves substantial bitrate reduction with better subjective and objective quality in comparison with other PVC schemes.

The complexity of 3D-HEVC is fairly high due to quad tree structure and traversal searching in depth intra coding. In order to reduce complexity caused by coding unit (CU) size decision in rate distortion optimization (RDO) process, a fast algorithm based on adaptive QP convolutional neural network (AQ-CNN) structure is proposed in this paper. For each size of CU, the proposed structure automatically extracts deep feature information to terminate CU partition early. Specially, the AQ-CNN structure is suitable for different QPs because the QP has a great influence on CU partition and is connected into the CNN structure appropriately. Benefiting from the accurate prediction of CU partition label, the proposed algorithm reduces coding complexity sharply. Experimental results show that the proposed algorithm reduces the depth coding time by 69.4% with negligible BD-rate increase, and outperforms other recent algorithms in 3D-HEVC.

In video encoder, hard-decision quantization (HDQ) is well-suited for parallel processing, but suffers from non-negligible coding performance degradation compared with soft-decision quantization (SDQ). In this paper, by fully simulating the behavior of SDQ, a coefficient-adaptive offset model constructed by the deep learning approach is proposed to adjust the output of HDQ. Experiment results show that the proposed algorithm achieves promising RD performance and well-suited for hardware encoder implementation design.

3D video coding extension of High Efficiency Video Coding (3D-HEVC) adopts many high complexity approaches to improve the coding performance of the depth video, which leads to heavy computation. To alleviate the computation burden, a fast intra coding scheme for the depth map coding is proposed based on texture feature and spatial correlation. The coding unit (CU) sizes and intra prediction modes are selected differently due to different depth features. The CU block is divided into smooth block, texture block and edge block by using gray-level co-occurrence matrix (GLCM) and sobel operator. Firstly, for CU level, an early termination strategy of CU splitting for smooth CU is proposed to filter out unnecessary coding blocks. Then, for PU level, a fast candidate mode decision method (FCMDM) is proposed to reduce the redundant candidate modes based on PU's types and the prediction mode of neighbor PUs. Finally, a fast wedgelet pattern determination method based on K-Means is explored to reduce the complexity of depth modelling mode 1 (DMM1). Experimental results show that the proposed algorithm achieves an average time reduction of 40.71% for depth intra coding, with negligible drop in encoding quality. The proposed algorithm achieves better the time saving and coding performance compared with the state-of-the-art method.

In comparison with soft-decision quantization (SDQ), deadzone hard-decision quantization (HDQ) is well-suited for parallel processing, but suffering from non-negligible rate distortion (RD) performance loss. By fully simulating the behavior of SDQ, this paper proposes an adaptive deadzone offset model to improve traditional HDQ. Coefficient-wise offset model is elaborately design by offline analysis, forcing the improved HDQ to generate identical result as SDQ by maximizing the right judgment probability. In addition, offset models are adaptively built as functions of quantization step and component-wise parameter of DCT coefficients. The proposed model based HDQ achieves good RD performance and well-suited for hardware encoder implementation design.