With the exponential growth of video traffic and the continuous evolution of video coding standards, video transcoding has become essential for existing bitstreams to benefit from the advanced features of new video compression technologies. Typically, video transcoding involves decoding an existing bitstream and re-encoding the decoded sequence into a target format. A key challenge in transcoding is the inevitable presence of compression artifacts in the decoded sequences, which, if not properly addressed, can degrade transcoding efficiency by causing suboptimal bit allocation and disrupting core coding processes. In this article, a learned video transcoding framework (LVT) is proposed to optimize video transcoding, leveraging coding priors from the input bitstream to guide the transcoding process. In the framework, to mitigate the adverse effects of compression artifacts, a Coding Priors-Guided Spatial Feature Transform module is designed, which utilizes coding prior features to adaptively modulate intermediate features through spatial affine transformations, enhancing bit allocation and suppressing artifacts. Additionally, a Coding Priors-Guided Quality Adapter module is proposed to generate a compression degradation representation using coding priors, which dynamically interacts with intermediate features to enable the network to perceive and adapt to different levels of degradation in the input video. Furthermore, a Motion Vectors-Guided Flow Refinement module is proposed to reduce prediction errors caused by artifacts. It refines optical flow predictions by using motion vectors from the bitstream as auxiliary information. Extensive experiments demonstrate that our framework outperforms both existing traditional and learned video codecs in transcoding performance, achieving an average bitrate saving of 20.3% compared to the H.266/VVC reference software VTM under the practical YUV420 setting measured with PSNR.

This paper addresses the task of space-time video super-resolution (STVSR). Existing methods generally suffer from inaccurate motion estimation and motion compensation (MEMC) problems for large motions. Inspired by recent progress in physics-informed neural networks, we model the challenges of MEMC in STVSR as a mapping between two continuous function spaces. Specifically, our approach transforms independent low-resolution representations in the coarse-grained continuous function space into refined representations with enriched spatiotemporal details in the fine-grained continuous function space. To achieve efficient and accurate MEMC, we design a Galerkin-type attention function to perform frame alignment and temporal interpolation. Due to the linear complexity of the Galerkin-type attention mechanism, our model avoids patch partitioning and offers global receptive fields, enabling precise estimation of large motions. The experimental results show that the proposed method surpasses state-of-the-art techniques in both fixed-size and continuous space-time video super-resolution tasks. Code is publicly available at the URL https://github.com/hahazh/STVSR-NO.

Remote sensing image (RSI) super-resolution (SR) demands lightweight and efficient methods due to required rapid response in practical applications. Integrating RSIs with different resolutions for diverse applications also requires arbitrary-scale SR, making fix-scaled SR scale inflexible. Therefore, a lightweight SR algorithm capable of arbitrary-scale is necessary for RSIs. To address the above issue, a super-scale feature-based lightweight arbitrary-scale (SFLA) SR network is proposed in this paper. The network consists of two modules: 1) A super-scale feature extraction (SSFE) module that extracts features at both the initial low-resolution (LR) and an integer super-scale resolution, 2) A self-attention implicit function reconstruction (SIFR) module that utilizes multi-layer perceptron (MLP) network and self-attention mechanism for pixel-wise feature mapping to achieve superior SR results. Comparative experiments and ablation results demonstrate that the proposed SFLA algorithm effectively strikes a good balance between performance and complexity.

Low-light image enhancement (LLIE) aims to improve the illuminance of images due to insufficient light exposure. Recently, various lightweight learning-based LLIE methods have been proposed to handle the challenges of unfavorable prevailing low contrast, low brightness, etc. In this paper, we have streamlined the architecture of the network to the utmost degree. By utilizing the effective structural re-parameterization technique, a single convolutional layer model (SCLM) is proposed that provides global low-light enhancement as the coarsely enhanced results. In addition, we introduce a local adaptation module that learns a set of shared parameters to accomplish local illumination correction to address the issue of varied exposure levels in different image regions. Experimental results demonstrate that the proposed method performs favorably against the state-of-the-art LLIE methods in both objective metrics and subjective visual effects. Additionally, our method has fewer parameters and lower inference complexity compared to other learning-based schemes. Code will be made publicly available at the URL https://gitee.com/zhanghahaxixi/SCLM.

Object counting and localization for remote sensing images are effective means to solve large-scale object analysis problems. Nowadays, most counting methods obtain the number of objects by employing convolutional neural network (CNN) to regress a density map of objects. Even if these leading methods have achieved impressive performances, they simply focus on estimating the number of single-class objects, without providing location information and cannot support multiclass objects. To tackle these problems, a point-based network named Dense Object Prediction Network (DOPNet) is proposed for multiclass object counting and localization for remote sensing images. DOPNet differs from the conventional approach of predicting multiple density maps by incorporating category attributes into the predicted objects, enabling the accurate counting and localization of multiclass objects. Specifically, DOPNet adopts a multiscale architecture (MS) to provide dense predictions of object proposals. A scale adaptive feature enhancement module (SAFEM) is designed to predict scales of objects for the suppression of duplicate proposals. Given only point level annotations for training, a pseudo-box generation algorithm is designed to find the most suitable pseudo-box of each annotated object for the supervision of scale learning. Comprehensive experiments prove that DOPNet can achieve preferable performance on challenging benchmarks of counting while providing object locations. Code and pre-trained models are available athttps://github.com/Ceoilmp/DOPNet.

Exposure errors in images, including both underexposure and overexposure, significantly diminish images’ contrast and visual appeal. Existing deep learning-based exposure correction methods either require large networks or longer processing time for inference and are thus not applicable for embedded devices and real-time applications. To address these issues, a lightweight network is proposed in this paper to correct exposure errors with limited memory occupation and inference steps. It adopts the Laplacian pyramid to incrementally recover the color and details of the image through a layer-by-layer procedure. A structural re-parameterization structure is designed to both reduce model size for inference speed up and improve performance with a multi-branch learning structure. Extensive experiments demonstrate that our method achieves a better performance-efficiency trade-off than other exposure correction methods.

Given an image, crowd counting aims to estimate the amount of target objects in the image. With un-predictable installation situations of surveillance systems (or other equipments), crowd counting images from different data sets may exhibit severe discrepancies in viewing angle, scale, lighting condition, etc. As it is usually expensive and time-consuming to annotate each data set for model training, it has been an essential issue in crowd counting to transfer a well-trained model on a labeled data set (source domain) to a new data set (target domain). To tackle this problem, we propose a cross-domain learning network to learn the domain gaps in an unsupervised learning manner. The proposed network comprises of a Multi-granularity Feature-aware Discriminator (MFD) module, a Domain-invariant Feature Adaptation (DFA) module, and a Cross-domain Vanishing Bridge (CVB) module to remove domain-specific information from the extracted features and promote the mapping performances of the network. Unlike most existing methods that use only Global Feature Discriminator (GFD) to align features at image level, an additional Local Feature Discriminator (LFD) is inserted and together with GFD form the MFD module. As a complement to MFD, LFD refines features at pixel level and has the ability to align local features. The DFA module explicitly measures the distances between the source domain features and the target domain features and aligns the marginal distribution of their features with Maximum Mean Discrepancy (MMD). Finally, the CVB module provides an incremental capability of removing the impact of interfering part of the extracted features. Several well-known networks are adopted as the backbone of our algorithm to prove the effectiveness of the proposed adaptation structure. Comprehensive experiments demonstrate that our model achieves competitive performance to the state-of-the-art methods.

With the advancement in remote sensing satellite technology, object tracking in satellite videos has become an emerging research field. However, due to small object size, little appearance features, and poor distinguishability between targets and the background, traditional trackers with handcraft visual features achieve poor results in satellite videos. Deep neural networks have shown powerful potential for object tracking in ordinary videos but remain developing in satellite videos. In this letter, a Siamese network and a motion regression network are adopted to form a two-stream deep neural network (SRN) for satellite object tracking, which simultaneously utilizes appearance and motion features. Besides, a trajectory fitting motion (TFM) model based on history trajectories is also employed to further alleviate model drift. Comprehensive experiments demonstrate that the proposed method performs favorably compared with the state-of-the-art tracking methods.

Object counting has attracted a lot of attention in remote sensing image analysis. In density map based object counting algorithms, the ground truth density maps generated by fix-sized Gaussian kernels ignore the spatial features of the objects. In this paper, an Adaptive Density Map Assisted Learning algorithm (ADMAL) is proposed, which taps into spatial features of the objects from the beginning phase of ground truth density map generation. ADMAL consists of two networks: a Contexture Aware Density Map Generation (CADMG) network and a Transformer-based Density Map Estimation (TDME) network. The CADMG network is designed to generate a ground truth density map from each annotated point map. Comparing with Gaussian convolved density maps, the ground truth density maps generated by CADMG will be tailored according to the texture and neighborhood relationship among objects, which can promote the learning effect of the TDME network. TDME is the core network for object counting. The backbone of the TDME network adopts a Swin transformer structure, the self-attention mechanism of which possesses a larger receptive field for effective feature extraction in remote sensing images. Comprehensive experiments prove that the ground truth density map generated by CADMG can help various density map estimation networks achieve better training effects, among which TDME achieves the best performances. Moreover, the ADMAL algorithm can achieve preferable object counting performances for both satellite-based image and drone-based image. Code and pre-trained models are available at https://github.com/gcding/ADMAL-pytorch.

Car counting on drone-based images is a challenging task in computer vision. Most advanced methods for counting are based on density maps. Usually, density maps are first generated by convolving ground truth point maps with a Gaussian kernel for later model learning (generation). Then, the counting network learns to predict density maps from input images (estimation). Most studies focus on the estimation problem while overlooking the generation problem. In this paper, a training framework is proposed to generate density maps by learning and train generation and estimation subnetworks jointly. Experiments demonstrate that our method outperforms other density map-based methods and shows the best performance on drone-based car counting.

Video super-resolution (VSR) has attracted a lot of attention that converts a low resolution (LR) video into a high resolution (HR) one. The original LR video is typically produced either by the downscaling processing or low-resolution sensor. Considering that the resolution degradation or limitation makes different impacts on different low-frequency (LF) and high-frequency (HF) components of the LR video signal, we propose a Decomposition Oriented Video super-rEsolution (DOVE) method in this paper. More specifically, a three-stream VSR network is designed in which the proposed LF and HF stream is responsible for modeling LF and HF components in the feature space. Moreover, a multi-frame refinement stream takes features of coarsely aligned frames as input and generates finely aligned counterparts progressively to guide the learning of LF and HF streams at the intermediate feature level. Furthermore, non-local channel attention is devised to capture long-range dependencies on a global scale both in the channel domain. Experimental results indicate that separating the learning of LF and HF components helps better estimate the HR frame from LR frames and superior VSR performance is achieved when compared with that of recent state-of-the-art methods.

In this paper, an LSTM based rate-distortion (R-D) prediction method for low-delay video coding has been proposed. Unlike the traditional rate control algorithms, LSTM is introduced to learn the latent pattern of the R-D relationship in the progress of video coding. Temporal information, hierarchical coding structure information and the content of the frame which is to be encoded have been used to achieve more accurate prediction. Based on the proposed network, a new R-D model parameters prediction method is proposed and tested on test model of Versatile Video Coding (VVC). According to the experimental results, compared with the state-of-the-art method used in VVC, the proposed method can achieve better performance.

In this paper, we propose a new algorithm for High Efficiency Video Coding(HEVC) based on multi-objective particle swarm optimization (MOPSO) to enhance the visual quality of ROI while ensuring a certain overall quality. According to the R-λ model of detected ROI, the fitness function in MOPSO can be designed as the distortion of ROI and that of the overall frame. The particle consists of ROI's rate and other region's rate. After iterating through the multi-objective particle swarm optimization algorithm, the Pareto front is obtained. Then, the final bit allocation result which are the appropriate bit rate for ROI and non-ROI is selected from this set. Finally, according to the R-λ model, the coding parameters could be determined for coding. The experimental results show that the proposed algorithm improves the visual quality of ROI while guarantees overall visual quality.

In signal compression, distortion information is significant for rate distortion optimization. In this paper, we propose a convolutional neural network (CNN) to predict distortion information for H.265/HEVC. With the strong representation power of CNN, structural similarity (SSIM) maps indicating distortion information can be predicted directly in an end-to-end, pixel-to-pixel way. Different from traditional CNNs which focus on learning one-to-one mappings from input to output, we show that our CNN model can predict SSIM maps conditioned on quantization parameters (QPs), realizing one-to-many mappings. To construct our CNN network, QP labels are designed as conditions to feed the CNN model. We also apply symmetrical network architecture and multi-level feature fusion method to ensure our network can utilize both high-level semantic features and low-level structure features. The experiments on MS COCO database demonstrate the effectiveness of our CNN-based method for SSIM prediction.

360-degree video has been gaining its popularity as virtual reality (VR) technology develops in recent years. To compress these videos using standard video encoders, most of them are converted to a 2D image planar format with equirectangular projection (ERP). Compared with other encoders, High Efficiency Video Coding (HEVC) achieves significant improvements in coding efficiency. However, exhaustive computational complexity of HEVC makes it too time-consuming to compress 360-degree videos of high resolution and high frame rate for VR application. In this paper, a fast intra prediction algorithm is proposed according to the characteristics of ERP format video. Compared with original reference software HM 16.15, experimental results show that the proposed algorithm can achieve about 24.5% encoder time saving on average in All-Intra configuration with negligible quality loss.

In this paper, we propose a convolutional neural network (CNN)-based rate-distortion (R-D) modeling method for H.265/HEVC. A fully convolutional neural network (CNN) is designed to learn end-to-end, pixels-to-pixels mappings from the original images to the structural similarity (SSIM) maps indicating distortion. The rate information is predicted through a CNN with fully connected layers as well. When compared to traditional CNN methods, the proposed mappings to the distortion or rate information. The experiments demonstrate the feasibility of our CNN-based framework for rate-distortion modeling.

Band selection plays an important role in reducing the dimensionality of spectral response of hyperspectral images (HSIs) to avoid dimension disaster for land-cover classification. Compared with traditional dimension reduction methods, such as principal component analysis, independent component analysis, or linear discriminant analysis, band selection can help provide interpretability to later constructed models by preserving the physical meaning of selected features. In this letter, a group lasso-based band selection (GLBS) method is proposed for multilabel HSI classification. Using the group lasso algorithm, the two objectives of band selection and classification are implemented simultaneously. The performance of GLBS is fully investigated and compared with benchmark methods, and the experimental results demonstrate the superiority of GLBS.

Visual tracking plays a fundamental role in many applications, such as video surveillance, image compression and three-dimensional reconstruction. From the perspective of accuracy and complexity, correlation filter for target tracking has been proved to be one of the most efficient algorithms. However, it suffers from some difficulties when tracking complex objects with rotations, occlusions and other distractions. To improve its robustness, in this paper, we propose an adaptive background model (ABM) to realize real-time visual tracking with a high adaptivity and accuracy. We use the static background information to help tracking instead of only focusing on the target itself, especially when there are great appearance changes. Moreover, we use peak to side-lobe ratio to update the ABM. As shown in the experiments, our proposed method achieves effective performance in visual tracking with good tradeoff between computational complexity and accuracy.

The latest video coding standard, High Efficiency Video Coding (HEVC) can achieve up to 50% bit rate saving while maintain the same subjective quality compared to H.264/AVC. However, this great advance is obtained at the expense of significantly increased encoder complexity. In this paper, a two-step algorithm focusing on fast mode decision is proposed for HEVC intra prediction. Firstly, depth information of CU block is utilized to skip some unlikely selected modes, with the assumption that a brute-force search for a large CU is unnecessary. Secondly, the order of Most Probable Modes (MPM) and Rough Mode Decision (RMD) is adjusted, thus the amount of mode need be evaluated is further reduced. Experimental results show that the proposed algorithm can achieve about 31% encoder time saving on average while result in negligible BD-rate loss under the All Intra configuration compared with HM 16.0.

Accurate vehicular localization is important for various cooperative vehicle safety (CVS) applications such as collision avoidance, turning assistant, etc. In this paper, we propose a cooperative vehicular distance measurement technique based on the sharing of GPS pseudorange measurements and a weighted least squares method. The classic double difference pseudorange solution, which was originally designed for high-end survey level GPS systems, is adapted to low-end navigation level GPS receivers for its wide availability in ground vehicles. The Carrier to Noise Ratio (CNR) of raw pseudorange measurements are taken into account for noise mitigation. We present a Dedicated Short Range Communications (DSRC) based mechanism to implement the exchange of pseudorange information among neighboring vehicles. As demonstrated in field tests, our proposed technique increases the accuracy of the distance measurement significantly compared with the distance obtained from the GPS fixes.

In energy-limited wireless sensor networks, network clustering and sensor scheduling are two efficient techniques for minimizing node energy consumption and maximizing network coverage lifetime. When integrating the two techniques, the challenges are how to select cluster heads and active nodes. In this paper, we propose a coverage-aware clustering protocol. In the proposed protocol, we define a cost metric that favors those nodes being more energy-redundantly covered as better candidates for cluster heads and select active nodes in a way that tries to emulate the most efficient tessellation for area coverage. Our simulation results show that the network coverage lifetime can be significantly improved compared with an existing protocol.

The objective of this paper is two-fold. First, we compare the performance, through computer simulations, of some solutions for TCP packet reordering in wireless networks. Second, we present an alternative method to improve the connection goodput in wireless networks through link-layer retransmissions and applying the solutions to TCP packet reordering. Some link-layer retransmission approaches do not attempt to maintain in-order packet delivery. This leads to some segments, which belong to the same TCP flow, to arrive at their destination out of order. Thus, the problem of high channel error rates in wireless networks becomes the problem of packet reordering due to link-layer retransmissions. We performed a simulation study to evaluate the performance of four solutions for TCP packet reordering, namely, RR-TCP, TCP-DCR, TCP-DOOR, and TCP-PR, under the scenarios of an infrastructure-based wireless network and a multi-hop wireless network. We also compared them with SACK TCP and TCPW. Our simulation study reveals that TCP-PR outperforms all of the other five algorithms, enjoying a greater connection goodput and fewer false fast retransmissions.

Transmission control protocol (TCP) is the most popular transport layer protocol for the Internet. Due to various reasons, such as multipath routing, route fluttering, and retransmissions, packets belonging to the same flow may arrive out of order at a destination. Such packet reordering violates the design principles of some traffic control mechanisms in TCP and, thus, poses performance problems. In this paper, we provide a comprehensive and in-depth survey on recent research on packet reordering in TCP. The causes and problems for packet reordering are discussed. Various representative algorithms are examined and compared by computer simulations. The ported program codes and simulation scripts are available for download. Some open questions are discussed to stimulate further research in this area

In this paper, we propose an adaptive video transmission scheme to achieve unequal error protection in multichannel ad hoc networks. In our scheme, video data is divided into high priority (HP) and low priority (LP) portions, and mobile nodes have two channels for video transmission. A channel quality metric, busy time ratio (BTR), is employed to characterize the channel quality. The channel with better BTR metric is used for HP data, and the other for LP. Further, adaptive load control and delay-constrained queue management schemes are proposed to improve performance. Simulation results demonstrate the performance of video streaming is greatly enhanced in multichannel multi-hop ad hoc networks.

Opportunistic transmission scheduling schemes improve system capacity by taking advantage of independent time varying channels in wireless networks. In the design of such scheduling schemes, the fairness criterion plays an important role in the tradeoff of total system capacity and the achievable throughput of individual users. To meet different fairness demands with a unified opportunistic scheduling scheme, in this paper, we have extended the well known opportunistic scheduling scheme PFS into αPFS, which satisfies arbitrary fairness demands, varying from proportional fairness to maxmin fairness, through adjusting the parameter α. To further improve the achievable diversity gains of αPFS, we extend the αPFS scheme into an αPFS-P scheme. Performances of αPFS and αPFS-P are studied and compared. As demonstrated in the simulation results, both αPFS and αPFS-P can achieve adjustable fairness criteria, varying from proportional fairness to max-min fairness. Compared with αPFS, αPFS-P achieves higher diversity gains with degraded short term performance, which is still better than the performance of PFS.

The original power controlled multiple access (PCMA) protocol does not support service differentiation. In this paper, we extend PCMA to form a new media access control protocol supporting service differentiation in mobile ad hoc networks. To support QoS, we first introduce the in-station access category concept in 802.11e to PCMA. For service differentiation between access categories, our major contribution is to propose a sender-initiated busy tone based mechanism that allows a user to gain quick channel access. This quick access mechanism is only performed when the number of access failures exceeds a threshold. An access category with higher priority is assigned a lower threshold for easier channel access, and vice versa. Through analysis and simulation, we demonstrate that our protocol can provide better quality of service than 802.11e in terms of throughput, delay, loss, and fairness.

Shared channel contention-based MAC protocols, such as IEEE 802.11, are popular in ad hoc networks because of their ease of implementation. However, these contention-based MAC protocols do not coordinate between nodes at different hops within a multi-hop flow. This results in channel resource and node transmission power wastage and overall system throughput degradation. In this paper we present a novel distributed flow-based scheduling (DFBS) scheme that coordinates between neighbor links of a multi-hop flow. As demonstrated by the simulation results, DFBS achieves higher throughput and improves the transmission efficiency when traffic load is relatively high.