Neural Video Compression has emerged in recent years, with condition-based frameworks outperforming traditional codecs. However, most existing methods rely solely on the previous frame's features to predict temporal context, leading to two critical issues. First, the short reference window misses long-term dependencies and fine texture details. Second, propagating only feature-level information accumulates errors over frames, causing prediction inaccuracies and loss of subtle textures. To address these, we propose the Long-term Spatio-Temporal Enhanced Context (L-STEC) method. We first extend the reference chain with LSTM to capture long-term dependencies. We then incorporate warped spatial context from the pixel domain, fusing spatio-temporal information through a multi-receptive field network to better preserve reference details. Experimental results show that L-STEC significantly improves compression by enriching contextual information, achieving 37.01% bitrate savings in PSNR and 31.65% in MS-SSIM compared to DCVC-TCM, outperforming both VTM-17.0 and DCVC-FM and establishing new state-of-the-art performance.

Recent advances in end-to-end video compression have shown promising results owing to their unified end-to-end learning optimization. However, such generalized frameworks often lack content-specific adaptation, leading to suboptimal compression performance. To address this, this paper proposes a content adaptive based motion alignment framework that improves performance by adapting encoding strategies to diverse content characteristics. Specifically, we first introduce a two-stage flow-guided deformable warping mechanism that refines motion compensation with coarse-to-fine offset prediction and mask modulation, enabling precise feature alignment. Second, we propose a multi-reference quality aware strategy that adjusts distortion weights based on reference quality, and applies it to hierarchical training to reduce error propagation. Third, we integrate a training-free module that downsamples frames by motion magnitude and resolution to obtain smooth motion estimation. Experimental results on standard test datasets demonstrate that our framework CAMA achieves significant improvements over state-of-the-art Neural Video Compression models, achieving a 24.95% BD-rate (PSNR) savings over our baseline model DCVC-TCM, while also outperforming reproduced DCVC-DC and traditional codec HM-16.25.

In point cloud compression, the quality of a reconstructed point cloud relies on both the global structure and the local context, with existing methods usually processing global and local information sequentially and lacking communication between these two types of information. In this paper, we propose a point-voxel interlaced network (PVINet), which captures global structural features and local contextual features in parallel and performs interactions at each scale to enhance feature perception efficiency. Specifically, PVINet contains a voxel-based encoder ($\mathcal {E}_{v}$) for extracting global structural features and a point-based encoder ($\mathcal {E}_{p}$) that models local contexts centered at each voxel. Particularly, a novel conditional sparse convolution is introduced, which applies point embeddings to dynamically customize kernels for voxel feature extraction, facilitating feature interactions from$\mathcal {E}_{p}$to$\mathcal {E}_{v}$. During decoding, a voxel-based decoder employs conditional sparse convolutions to incorporate point embeddings as guidance to reconstruct the point cloud. Experiments on benchmark datasets show that PVINet delivers competitive performance compared to state-of-the-art methods.

Nowadays, high-quality images are pursued by both humans for better viewing experience and by machines for more accurate visual analysis. However, images are usually compressed before being consumed, decreasing their quality. It is meaningful to predict the perceptual quality of compressed images for both humans and machines, which guides the optimization for compression. This issue remains underexplored. In this paper, we propose a unified approach to fill this gap. Specifically, we create a deep learning-based model to predict Satisfied User Ratio (SUR) and Satisfied Machine Ratio (SMR) of compressed images simultaneously. We first pre-train a feature extractor network on a large-scale SMR-annotated dataset with human perception-related quality labels generated by diverse image quality models, which simulates the acquisition of SUR labels. Then, we leverage and fuse the extracted multi-layer features to predict SUR and SMR with a more capable network architecture. We propose a Difference Feature Residual Learning (DFRL) module to learn more discriminative difference features. We also design a Multi-Head Attention Aggregation and Pooling (MHAAP) layer to aggregate difference features and reduce their redundancy. We further introduce an MLP-Mixer module to integrate global spatial and channel information for subsequent SUR and SMR regression. Experimental results indicate that the proposed model significantly outperforms state-of-the-art SUR and SMR prediction methods by up to 48% on several challenging datasets. Moreover, our joint learning scheme of human and machine perceptual quality prediction tasks is effective at improving the performance of both.

Recently, neural image compression (NIC) has made remarkable progress. Two key parts of NIC are the encoder-decoder and the entropy model. For the encoder-decoder, a larger effective receptive field (ERF) means a stronger transformation ability. Existing methods usually enlarge the ERF at the expense of complexity, which is intolerable. To address this issue, we propose a multi-scale depthwise separable dilated convolution (MSDSDC) to build the encoder-decoder. Specifically, we first construct a depthwise separable dilated convolution (DSDC) by using the depthwise separable strategy in dilated convolution to reduce its complexity. Subsequently, multi-scale features extracted by three DSDCs with varying dilation rates are fused to expand the ERF of the encoder-decoder, consequently enhancing its transformation capability. Besides, we design a multi-distribution mixture entropy model (MDMEM) to further enhance the flexibility of latent representation probability modeling. The experimental results demonstrate that our proposed method achieves the best balance between rate-distortion performance and complexity.

This paper introduces Hierarchical Image Steganography (HIS), a novel method that uses diffusion models to enhance the security and capacity of embedding multiple images into a single container. HIS assigns varying levels of robustness to images based on their importance, ensuring enhanced protection against manipulation. It adeptly exploits the robustness of the Diffusion Model and the reversibility of the Flow Model. Integrating Embed-Flow and Enhance-Flow improves embedding efficiency and image recovery quality, respectively, setting HIS apart from conventional multi-image steganography techniques. This innovative structure can autonomously generate a container image, thereby securely and efficiently concealing multiple images and text. Rigorous subjective and objective evaluations underscore HIS’s advantage in analytical resistance, robustness, and capacity, illustrating its expansive applicability in content safeguarding and privacy fortification.

3D Gaussian Splatting (3DGS) has emerged as a high-fidelity and efficient paradigm for online free-viewpoint video (FVV) reconstruction, offering viewers rapid responsiveness and immersive experiences. However, existing online methods face challenge in prohibitive storage requirements primarily due to point-wise modeling that fails to exploit the motion properties. To address this limitation, we propose a novel Compact Gaussian Streaming (ComGS) framework, leveraging the locality and consistency of motion in dynamic scene, that models object-consistent Gaussian point motion through keypoint-driven motion representation. By transmitting only the keypoint attributes, this framework provides a more storage-efficient solution. Specifically, we first identify a sparse set of motion-sensitive keypoints localized within motion regions using a viewspace gradient difference strategy. Equipped with these keypoints, we propose an adaptive motion-driven mechanism that predicts a spatial influence field for propagating keypoint motion to neighboring Gaussian points with similar motion. Moreover, ComGS adopts an error-aware correction strategy for key frame reconstruction that selectively refines erroneous regions and mitigates error accumulation without unnecessary overhead. Overall, ComGS achieves a remarkable storage reduction of over 159 × compared to 3DGStream and 14 × compared to the SOTA method QUEEN, while maintaining competitive visual fidelity and rendering speed. Project page: https://chenjiacong-1005.github.io/ComGS/.

Abstract Temporal inconsistency is the annoying artifact that has been commonly introduced in low-light video enhancement, but current methods tend to overlook the significance of utilizing both data-centric clues and model-centric design to tackle this problem. In this context, our work makes a comprehensive exploration from the following three aspects. First, to enrich the scene diversity and motion flexibility, we construct a synthetic diverse low/normal-light paired video dataset with a carefully designed low-light simulation strategy, which can effectively complement existing real captured datasets. Second, for better temporal dependency utilization, we develop a Temporally Consistent Enhancer Network (TCE-Net) that consists of stacked 3D convolutions and 2D convolutions to exploit spatial-temporal clues in videos. Last, the temporal dynamic feature dependencies are exploited to obtain consistency constraints for different frame indexes. All these efforts are powered by a Spatial-Temporal Compatible Learning (STCL) optimization technique, which dynamically constructs specific training loss functions adaptively on different datasets. As such, multiple-frame information can be effectively utilized and different levels of information from the network can be feasibly integrated, thus expanding the synergies on different kinds of data and offering visually better results in terms of illumination distribution, color consistency, texture details, and temporal coherence. Extensive experimental results on various real-world low-light video datasets clearly demonstrate the proposed method achieves superior performance to state-of-the-art methods. Our code and synthesized low-light video database will be publicly available at https://github.com/lingyzhu0101/low-light-video-enhancement.git .

Learned image compression has attracted a lot of attention in recent years. Currently, popular learned image compression methods usually exploit hyperprior and autoregressive models to facilitate probability estimation and reduce the redundancy of latent representation. These models ignore different image contents, and it is difficult to eliminate the spatial redundancy of image, resulting in the performance saturation. In this work, we propose a learned image compression method with large capacity and low redundancy of latent representation. We design two enhancement modules, i.e., the network capacity expansion module (NCEM) and the high-entropy content guided reconstruction module (HCGR), to construct network architectures with better rate-distortion performance than the existing hyperprior and autoregressive models. Experimental results show that our method can produce superior results compared to the state-of-the-art methods.

In the past few years, the large scale satellite constellation project represented by Starlink, oneweb, etc. has promoted a new round of large scale low earth orbit (LEO) satellite constellation networks (LS-LEOSCN) development wave. In LS-LEOSCN, multiple gateway stations need to be deployed to meet user needs, and their location greatly affects system capacity and other performance. Most of the factors considered are delay, hop count or capacity, but most of them do not consider the interference between the feeding links under the giant constellation. In this paper, a capacity evaluation method considering the interference between feeder links is proposed, and based on this capacity evaluation mechanism, a gateway station deployment optimization method based on genetic algorithm is proposed to maximize the system capacity in the case of considering the interference between feeder links. Compared with the randomly generated scheme, the capacity is increased by 6.9% in average.

Radar forward-looking imaging has important application value in target attack and terrain detection. However, traditional synthetic aperture radar (SAR) cannot perform high-resolution imaging of the area directly in front of the flight trajectory, due to the limitation of mechanism. This paper proposes a new airborne forward-looking imaging scheme based on maneuvering motion. The scheme firstly starts from the movement trajectory of the platform, gives the spatial geometric relationship of the forward -looking imaging, and establishes the echo signal model, further analyzes the azimuth resolution characteristics of the signal. On this basis, combined with the phase characteristics of the echo signal, a forward-looking imaging algorithm suitable for this scheme is derived. Finally, through simulation experiments, the target imaging results are analyzed to verify the effectiveness of the algorithm.

Quality enhancement of HEVC compressed videos has attracted a lot of attentions in recent years. In this article, we propose a robust multi-frame guided attention network (MGANet) to reconstruct high-quality frames based on HEVC compressed videos. In our network, we first use an advanced motion flow algorithm to estimate the motion information of input frames so as to guide the warping of adjacent frames. After performing the alignment, we find that large residuals still appear in the edge area of moving objects of the warped frames. Then, we design a temporal encoder based on a bi-directional convolutional long short term memory (ConvLSTM) with residual structure to further discover the variations between the current frame and its adjacent warped frames. Finally, we feed the extracted temporal information and a partitioned average image (PAI) to a multi-scale guided encoder-decoder subnet to reconstruct high-quality frames. Here, each PAI is generated according to the transform unit (TU) partitioning map that can be extracted directly from the coded bit-streams, thus enabling our network to focus on the TU boundaries while optimizing the global content. We present extensive experimental results to demonstrate the robustness of our method, especially for the high bit-rate coding case and large motion scenes. Due to the lightweight design structure, our proposed MGANet also has a very competitive inference time.

In this paper, a flow-guided temporal-spatial network (FGTSN) is proposed to enhance the quality of HEVC compressed video. Specifically, we first employ a robust motion estimation subnet via trainable optical flow module to estimate the motion flow between the target frame and its adjacent frames, and these adjacent frames are pre-warped guided by the predicted motion flow. Then, a temporal encoder is proposed to fuse the related information between the target frame and its pre-warped frames. Finally, a quality enhancement subnet with multi-scale encoder-decoder structure is designed to generate high quality frame by training the network in a multi-supervised fashion. Experimental results show the superior performance of our proposed FGTSN method for the reconstruction quality of HEVC compressed frames, much better than the state-of-the-art quality enhancement methods. In addition, our FGTSN method can also effectively mitigate the quality fluctuation of adjacent frames.

In this paper, we propose a novel salient object detection framework, which makes full use of the essential image compression. More specifically, we first compose an intuitive measure of compressibility from JPEG compression, namely bit-map. Then, depending on the relationship between bitmap and salient object, we generate the salient object window directly from bit-map without utilizing any features from the compressed image. Finally, the saliency map is calculated according to the salient object window and with a ranking algorithm. The proposed method achieves good performance as well as low complexity. The experimental results demonstrate the effectiveness of our proposed method compared with other existing approaches.

Multiple description coding (MDC) is an efficient source coding technique for error-prone transmission over multiple channels. In this paper, we focus on the design of a new polyphase down-sampling based MDC (NPDS-MDC) for image signals. The encoding of our proposed NPDS-MDC consists of three steps. First, we perform down-sampling on each N×N image block according to the quincunx down-sampling pattern. Second, we propose a new transform and apply it to the down-sampled pixels to produce the side descriptions. Third, we develop an error compensation algorithm to reduce the compression distortion occurring on the down-sampled pixels. In our scheme, the side decoding is performed posterior to image interpolation with reference to the down-sampled compressed pixels. Moreover, the central decoding is achieved by interlacing the side descriptions. We also propose a compression-constrained central deblocking algorithm to further improve the efficiency of the central decoding. The experimental results indicate that our proposed MDC scheme offers clearly superior performance, especially at high bit rates, as compared to the state-of-the-art methods for various types of images.

In this paper, we propose a quality enhancement network of versatile video coding (VVC) compressed videos by jointly exploiting spatial details and temporal structure (SDTS). The proposed network consists of a temporal structure fusion subnet and a spatial detail enhancement subnet. The former subnet is used to estimate and compensate the temporal motion across frames, and the latter subnet is used to reduce the compression artifacts and enhance the reconstruction quality of compressed video. Experimental results demonstrate the effectiveness of our SDTS-based method. The code of our proposed method is available at https://github.com/mengab/SDTS.

In this paper, we design a new multiple description coding scheme for image signals based on our proposed compression-guided optimization. Firstly, we propose a compression-constrained adaptive filtering method to produce two descriptions for the source image, where the proposed filtering algorithm works not only to guarantee a high-quality side decoding but also make a high-efficient central decoding. Secondly, we design a compression-dependent deblocking algorithm based on the transform coefficients which are decoded from both descriptions to improve the performance for the cental decoding. Experimental results demonstrate that our proposed method achieves impressive performance gains when it is applied to image signals.

In this paper, we build up a new block-based color image compression scheme based on our proposed transform domain down-sampling method and deep convolutional reconstruction algorithm. Specifically, our proposed down-sampling scheme aims to down-sample each N × N transform block into the N/2 × N/2 block for the saving of bit-cost. On the other hand, the proposed deep convolutional reconstruction algorithm is employed to reconstruct the down-sampled block for a full- resolution reconstruction. We apply our proposed methods to both the chrominance components to compress color images. Experimental results show that our proposed method achieves excellent results when used in practice.

MOTIVATION: Whole genome shotgun based next-generation transcriptomics and metagenomics studies often generate 100-1000 GB sequence data derived from tens of thousands of different genes or microbial species. Assembly of these data sets requires tradeoffs between scalability and accuracy. Current assembly methods optimized for scalability often sacrifice accuracy and vice versa. An ideal solution would both scale and produce optimal accuracy for individual genes or genomes. RESULTS: Here we describe an Apache Spark-based scalable sequence clustering application, SparkReadClust (SpaRC), that partitions reads based on their molecule of origin to enable downstream assembly optimization. SpaRC produces high clustering performance on transcriptomes and metagenomes from both short and long read sequencing technologies. It achieves near-linear scalability with input data size and number of compute nodes. SpaRC can run on both cloud computing and HPC environments without modification while delivering similar performance. Our results demonstrate that SpaRC provides a scalable solution for clustering billions of reads from next-generation sequencing experiments, and Apache Spark represents a cost-effective solution with rapid development/deployment cycles for similar large-scale sequence data analysis problems. AVAILABILITY AND IMPLEMENTATION: https://bitbucket.org/berkeleylab/jgi-sparc.

The ultra high-definition (UHD) video format, which has recently become popular, aims to provide high spatial resolution, high temporal frame rate, high sample bit-depth, and wide pixel color gamut. Despite the continued development of global network capacities, it inevitably causes the increased bandwidth cost of catering to the requirement of delivering UHD video services. To address such challenges, this paper presents an improved super coding unit (SCU) method for UHD video coding in High Efficiency Video Coding (HEVC). Initially, the medium coding unit (MCU) is proposed to avoid unnecessary brute-force coding unit (CU) partitions of SCU. Furthermore, the SCU is proposed to be encoded by Direct-MCU and SCU-to-MCU modes: the Direct-MCU mode is intended to better adapt to the texture-rich region, which guarantees the compression efficiency by avoiding extra-size CU partition; the SCU-to-MCU mode is designed for the homogeneous region of UHD content, which saves the encoding time by skipping fine-grained CU partition search. Moreover, a learning-based fast SCU decision approach is proposed to speed up the determination process of Direct-MCU and SCU-to-MCU, where three representative handcrafted features are extracted. Experimental results show that our method achieves an affordable complexity and excellent coding efficiency (up to 7.30% Bjøntegaard Delta rate savings) in UHD video coding compared to recent HEVC reference software.

In this paper, we propose a new HEVC in-loop filter based on a multi-channel long-short-term dependency residual network (MLSDRN). Inspired by the information storage and information update function of human memory cell, our MLSDRN introduces an update cell to adaptively store and select the long-term and short-term dependency information through an adaptive learning process. In addition, we leverage the block boundary information that recorded in the bit-streams to improve the filter performance, which also makes our MLSDRN to unequally treat the video content. Meanwhile, the multi-channel is introduced to solve the illumination discrepancy problem. We integrate the novel in-loop filter into HM reference software, and applying it to luma and chroma components, simulation results demonstrate that the proposed in-loop filter can save BD-rate reduction up to 15.9% with ALF off. For luma component, the novel in-loop filter achieves 6.0%, 8.1%, 7.4% BD-rate saving for all intra, low delay and random access configurations, respectively.

Transform domain down-scaling (TDDS) is traditionally implemented by dropping most of high-frequency components of the transformed block. Applying it to image compression can improve the compression efficiency by saving considerable bit-cost. Due to losing some necessary high-frequency information, the resulted image compressed by using the traditional TDDS-based coding often suffers a serious quality degradation. In this paper, we propose a compression-constrained TDDS and perform it on each N × N block to produce an N/2 × N/2 coefficient block for the compression. Our proposed TDDS not only guarantees a high reconstruction quality but also makes a low bit-cost for compression. We integrate it in practical image coding to build up our proposed compression scheme. Experimental results show that our proposed method demonstrates excellent coding performance when used to compress image signals.

This paper presents a DC coefficient estimation algorithm for intra-predicted residual blocks in the High-Efficiency Video Coding (HEVC) standard. Discarding the DC coefficient directly in each transform block leads to substantial bit-saving, but at the same time produces strong discontinuities between neighboring blocks. To overcome this problem, we propose an estimation algorithm for the DC coefficient, which solves an optimal offset in a closed-form to recover the corresponding block edges. Then, we embed this algorithm into HEVC in its rate-distortion optimized quantization and sign bit hiding steps. Furthermore, a flag is signaled to decide whether the DC estimation strategy is used for each transform block. Test results under the common test condition show that our algorithm achieves 1.5% and 1.6% BD-rate reduction on average for luma and chroma, respectively, under all intra configuration. In the meantime, our simulation results show that both encoding time and decoding time increase only slightly (about 10%, without any special optimization on programming our proposed algorithm). When testing the proposed DC estimation algorithm on inter coding configurations, including low delay with P pictures, low delay with B pictures, and random access, we can also achieve 0.5%-1.1% bit-rate savings on average, while nearly no extra encoding and decoding time is needed.

This paper proposes a DC coefficient estimation algorithm for intra-predicted residual blocks in the High Efficiency Video Coding (HEVC) standard. Discarding the DC coefficient in the current coding block leads to a substantial bit-saving but produces at the same time strong discontinuities between this block and its neighboring reconstructed blocks. To overcome this problem, we propose an estimation algorithm for the DC coefficient, which solves an optimal offset in a closed-form in the pixel domain to recover the corresponding block edges. Test results show that our algorithm achieves 1.0% and 1.4% BD-rate reduction on average for luma and chroma as compared with HM-16.6, respectively, when the sign-bit-hiding (SBH) technique is disabled. When SDH is set on, namely under the common test condition (CTC), the BD-rate reduction drops slightly to 0.7% and 1.1% for luma and chroma, respectively. In the meantime, the test results show that both encoding time and decoding time increase only slightly (about 10%, without any special optimization on programming our proposed algorithm).

High efficiency video coding (HEVC) defines 35 prediction modes in its intra prediction stage to signal the direction information of residual blocks. Traditionally, separable two-dimension (2-D) transforms (integer DCT and DST) are utilized in a similar manner as in the previous H.264/AVC standards. However, such 2-D transforms cannot yield the best energy compaction for a 2-D directional source where the dominating directional information is other than the horizontal or vertical one. In order to overcome this drawback, we build an elliptical model with directionality and design some non-separable transforms based on the Karhunen-Loeve transform in this paper. Specifically, we derive a non-separable transform in closed-form for each intra-prediction mode and replace the default transform in HEVC. Simulation results reveal that 1.7% and 2.0% on average and up to 7.7% and 8.1% BD-rate reduction can be achieved for luma and chroma component, respectively. In the meantime, the test results show that both the encoding time and decoding time increase only about 5%.

Advances in bioinformatics research continue to add complexity to the analyses and interpretation of biological data. Certain sequence database searches may take weeks to complete due to complicated data dependencies by dynamic programming. A reconfigurable coprocessor can remove this computational bottleneck and accelerate the operation. This paper presents a heterogeneous computing platform through Message Passing Interface (MPI) enabled enterprise computing infrastructure for high-throughput biological sequence analysis. The computing platform integrates heterogeneous computer architectures including conventional processors with Streaming Single Instruction Multiple Data Extensions 2 (SSE2) instructions, reconfigurable coprocessors, and legacy processors together into one system, and allows each to perform the task to which it is best suited. With appropriate computation and communication scheduling, the integrated heterogeneous computing infrastructure is designed to accommodate various types of accelerators to provide a High-Performance Computing (HPC) framework to support the most widely used life science applications.