Parameter-efficient transfer learning (PETL) has emerged as an effective strategy for fine-tuning large vision–language foundation models because it sharply reduces computational and memory overhead. However, existing PETL techniques underperform on dense prediction tasks that require fine-grained multimodal reasoning, such as referring image segmentation (RIS), owing to the lack of mechanisms that simultaneously strengthen local perception and enforce precise cross-modal alignment. We present a PETL framework with two lightweight and complementary adapters. The Global–Local Integrated Adapter (GLiA) enriches intra-modal features by coupling multi-scale depthwise-separable convolutions with a lightweight self-attention layer, capturing local context without sacrificing global dependencies. The Cross-Modal Alignment Adapter (CAA) explicitly aligns textual phrases with their corresponding visual regions, bridging the semantic gap between vision and language and enhancing multimodal reasoning. Experiments on three mainstream RIS benchmarks show that MiCA achieves the best accuracy while saving numerous updated parameters compared to the full fine-tuning and other PETL methods. Notably, with only 1.93% tunable backbone parameters, MiCA improves average accuracy by 0.8% across the three benchmarks compared to the baseline model.

A large-scale video quality dataset called the VideoSet has been constructed recently to measure human subjective experience of H.264 coded video in terms of the just-noticeable-difference (JND). It measures the first three JND points of 5-second video of resolution 1080p, 720p, 540p and 360p. Based on the VideoSet, we propose a method to predict the satisfied-user-ratio (SUR) curves using a machine learning framework. First, we partition a video clip into local spatial-temporal segments and evaluate the quality of each segment using the VMAF quality index. Then, we aggregate these local VMAF measures to derive a global one. Finally, the masking effect is incorporated and the support vector regression (SVR) is used to predict the SUR curves, from which the JND points can be derived. Experimental results are given to demonstrate the performance of the proposed SUR prediction method.

• A large-scale JND-based coded video quality dataset is presented. • The VideoSet contains 220 5-s sequences in four resolutions coded by H.264/AVC. • The subjective test procedure, JND data cleaning and properties are described. • The significance and implications of the VideoSet are discussed. • This work points out a clear path to data-driven perceptual coding. A new methodology to measure coded image/video quality using the just-noticeable-difference (JND) idea was proposed in Lin et al. (2015). Several small JND-based image/video quality datasets were released by the Media Communications Lab at the University of Southern California in Jin et al. (2016) and Wang et al. (2016) [3]. In this work, we present an effort to build a large-scale JND-based coded video quality dataset. The dataset consists of 220 5-s sequences in four resolutions (i.e., 1920 × 1080 , 1280 × 720 , 960 × 540 and 640 × 360 ). For each of the 880 video clips, we encode it using the H.264/AVC codec with QP = 1 , … , 51 and measure the first three JND points with 30 + subjects. The dataset is called the “VideoSet”, which is an acronym for “Video Subject Evaluation Test (SET)”. This work describes the subjective test procedure, detection and removal of outlying measured data, and the properties of collected JND data. Finally, the significance and implications of the VideoSet to future video coding research and standardization efforts are pointed out. All source/coded video clips as well as measured JND data included in the VideoSet are available to the public in the IEEE DataPort (Wang et al., 2016 [4]).

Perceptual video coding has the potential to provide the same visual quality at a lower bit-rate, compared with the traditional objective quality based scheme. Visual saliency represents the probability of human attention over frames, and it is used for allocating coding bits or controlling visual quality. In this paper, a HEVC compliant perceptual video coding scheme is proposed based on visual saliency. At first visual saliency map is attained to indicate the distribution of saliency. Then refined distortion allocating method is performed in CU level with adaptive QP which is adjusted by the average visual saliency. Besides, a fast CU mode decision algorithm suitable for perceptual video coding in HEVC is proposed to accelerate the encoder. In the fast algorithm, the average saliency is used to estimate texture complexity and movements in videos. Experimental results show that up to 22.52% bit-rate and 43.48% encoding time can be saved by our methods with negligible perceptual quality loss.

Intra coding has been significantly improved in HEVC over H.264/AVC with quad-tree based coding unit (CU) structure from size 64×64 to 8×8 and more prediction modes. However, these techniques cause a dramatic increase in computational complexity. In this paper, a novel intra coding algorithm is proposed consists of perceptual CU size decision algorithm and fast intra prediction mode decision algorithm. Firstly, based on the visual saliency detection, an adaptive and perceptual CU size decision method is proposed to alleviate intra encoding complexity. Furthermore, a fast intra prediction mode decision algorithm with step halving rough mode decision method is presented to selectively check the potential modes and effectively reduce the complexity of computation. Experimental results show that our proposed method reduces the computational complexity of the current HM to about 54.18% in encoding time with only 0.36% increases in BD rate and reasonable peak signal-to-noise ratio losses.

This paper presents a high-throughput and multi-parallel VLSI hardware architecture for the deblocking filter in the HEVC video coding standard. First, an implementation-friendly and fast boundary judgment method is proposed to avoid using the original recursion loop approach. Then a dedicated parallel VLSI architecture composed of four parallel filtering cores is presented based on the proposed boundary judgment method. With the parallel luma/chroma filtering and parallel vertical/horizontal edges filtering order, the proposed VLSI architecture can process filtering operations for one largest coding unit (LCU) with less filtering cycles than other conventional approaches. Furthermore, filtering efficiency is improved due to a novel ping-pang buffer architecture and the on-chip single-port SRAM with dedicated data arrangement in the memory modules. Experimental results demonstrate that the proposed deblocking filter architecture improves the performance by 28-89% at the expense of the slightly increased gate count compared to the previously known architecture in HEVC. The proposed architecture can reach a high operating clock frequency of 278 MHz with TSMC 90 nm library and meet the real time requirement of the deblocking filter for 8 K × 4 K video format at 123 frame/s.

Firstly, an implementation-friendly interpolation filter algorithm is proposed in this paper. It can save 19.6% processing time on average with negligible coding quality degradation. Then based on the proposed algorithm, an optimized interpolation filter VLSI architecture, composed of the reused data path of interpolation, efficient memory organization and the pipeline interpolation filter engine is presented to reduce the implement hardware area. The resulting design can achieve 240 MHz with only 37.2K gate count and support real-time interpolation filter operation of 3840×2160@47fps video application by using 90nm CMOS technology.

Like the previous video coding standard, DCT and quantization are also adopted in HEVC. Compared with H.264/AVC, HEVC employs larger transform blocks, which makes the all zero block detection algorithm designed for H.264/AVC inefficient for HEVC. An efficient all zero block detection algorithm aimed at HEVC, especially for 16×16 and 32×32 transform blocks, is proposed in this paper based on frequency characteristics of DCT. By analysing the distribution of residual energy in frequency domain, Hadamard transform is used to evaluate only a part of DCT coefficients which usually consume most of energy. To make the proposed algorithm efficient in complex sequences, a sum of absolute transformed difference based method is used to evaluate the maximum value of the rest of the coefficients. Experimental results show that more than 90% all zero blocks for 4×4, 8×8 and 16×16 transform blocks and about 80% for 32×32 transform blocks can be detected by the proposed algorithm. In addition, about 50% computational complexity in DCT/quantization can be reduced with negligible loss of video quality and compression efficiency.

This paper presents a novel VLSI hardware architecture for the real-time high-throughput implementation of the HEVC deblocking filtering. Based on the proposed implementation-friendly boundary judgment method, a dedicated multi-parallel architecture composed of four parallel filtering cores, parallel luma/chroma filtering and parallel vertical/horizontal edges filtering is presented. Experimental results demonstrate that the proposed architecture can greatly improve the performance at the expense of the slightly increased hardware cost compared to the previously known architecture in HEVC. The proposed architecture can also meet the real-time requirement of the deblocking filter for 8K×4K video format at 123fps under 278MHz clock rate.