The combination of features from the convolutional layer and the fully connected layer of a convolutional neural network (CNN) provides an effective way to improve the performance of crime scene investigation (CSI) image classification. However, in existing work, as the weights in feature fusion do not change after the training phase, it may produce inaccurate image features which affect classification results. To solve this problem, this paper proposes an adaptive feature fusion method based on an auto-encoder to improve classification accuracy. The method includes the following steps: Firstly, the CNN model is trained by transfer learning. Next, the features of the convolution layer and the fully connected layer are extracted respectively. These extracted features are then passed into the auto-encoder for further learning with Softmax normalisation to obtain the adaptive weights for performing final classification. Experiments demonstrated that the proposed method achieves higher CSI image classification performance compared with fix weights feature fusion.

Video surveillance and its applications have become increasingly ubiquitous in modern daily life. In video surveillance system, video coding as a critical enabling technology determines the effective transmission and storage of surveillance videos. In order to meet the real-time or time-critical transmission requirements of video surveillance systems, the low-delay (LD) configuration of the advanced high efficiency video coding (HEVC) standard is usually used to encode surveillance videos. The coding efficiency of the LD configuration is closely related to the quantization parameter (QP) cascading technique which selects or determines the QPs for encoding. However, the quantization parameter cascading (QPC) technique currently adopted for the LD configuration in HEVC test model (i.e., HM) is not optimized since it has not taken full account of the reference dependency in coding. In this paper, an efficient QPC technique for surveillance video coding, referred to as QPC-SV, is proposed, considering all inter reference frames under the LD configuration. Experimental results demonstrate the efficacy of the proposed QPC-SV. Compared with the default configuration of QPC in the HM, the QPC-SV achieves significant rate-distortion performance gain with average BD-rates of -9.35% and -9.76% for the LDP and LDB configurations, respectively.

Abstract Face image super-resolution imaging is an important technology which can be utilized in crime scene investigations and public security. Modern CNN-based super-resolution produces excellent results in terms of peak signal-to-noise ratio and the structural similarity index (SSIM). However, perceptual quality is generally poor, and the details of the facial features are lost. To overcome this problem, we propose a novel deep neural network to predict the super-resolution wavelet coefficients in order to obtain clearer facial images. Firstly, this paper uses prior knowledge of face images to manually emphases relevant facial features with more attention. Then, a linear low-rank convolution in the network is used. Finally, image edge features from canny detector are applied to enhance super-resolution images during training. The experimental results show that the proposed method can achieve competitive PSNR and SSIM and produces images with much higher perceptual quality.

In this paper, we introduce a low-complexity fine-granularity scalable (FGS) video encoder that refines both residue and motion information in the quality layers. The current scalable video coding (SVC) draft shows that significant gains can be achieved when each enhancement layer undergoes the motion estimation/motion compensation (ME/MC) process with its own motion vector field (MVF). However, given the high computational cost of ME/MC, a motion-refined FGS scheme can be expensive to implement. The proposed scheme controls the macroblock (MB) mode allowed in the base layer and channels computational resources to refine motion in enhancement layers. Through a proper selection of Lagrangian factor for the generation of the first MVF, it is possible to design a low-complexity FGS encoder that has good overall coding performance. A simplified motion-refinement scheme is also adopted for selected MBs in enhancement layers by exploiting the correlation of MB-type information between successive layers to further reduce the complexity of the FGS encoder. Meanwhile, a framework of rate-distortion-complexity optimization is proposed for the FGS by considering the interpolation complexity during the ME/MC in each layer. The FGS decoder can be simplified through the reduction in the number of interpolations.

In H.264 video coding, there are a substantial number of 4x4 blocks becoming all-zeros after transformation and quantization. This is a waste of computational resources because these skipped blocks do not require forward transform and quantization. We proposed a very effective early detection of fast skipped block detections based on the theoretical derivation of H.264 integer transform and quantization. The experimental results show that the algorithm can detect 9.71%-43.35% more zero blocks than Yong's method.

It is well known that motion estimation is the most computationally intensive processing unit of the H.264 video encoder. Various fast motion estimation algorithms have been proposed to reduce its complexity. Generally, these approaches achieve speedup by reducing the number of candidate search points within the search window. In this paper, we propose a new method, which uses the Sum-of-Absolute-Differences mapping (SAD map) to dynamically cache the SAD values and then reuse them for different block sizes. Experimental results on standard video sequences verified that the proposed method is capable of increasing the encoder speed by up to 15% without any loss in PSNR value or increase in bit rate. Due to its generic nature, this method can be applied in any fast motion estimation methods although it is especially effective in the full search motion estimation method.

As one important component in H.264 video encoder, interpolation of half and quarter pixels is also computational intensive. Compared to integer pixel motion estimation, "finer" interpolation provides better block match. However, this good motion compensation performance is obtained at the expense of increased complexity. Based on our previous work, this paper presents an improved fast and adaptive interpolation method that further reduces the complexity of video encoding process. Experimental results on typical video sequences demonstrate that the proposed method is able to increase encoder speed ranging from 10% to 22% compared with our previous work without any PSNR loss or bit rate increase.

The H.264/AVC video coding standard aims to enable significantly improved compression performance compared to all existing video coding standards. In order to achieve this, a robust rate-distortion optimization (RDO) technique is employed to select the best coding mode and reference frame for each macroblock. As a result, the complexity and computation load increase drastically. This paper presents a fast mode decision algorithm for H.264/AVC intraprediction based on local edge information. Prior to intraprediction, an edge map is created and a local edge direction histogram is then established for each subblock. Based on the distribution of the edge direction histogram, only a small part of intraprediction modes are chosen for RDO calculation. Experimental results show that the fast intraprediction mode decision scheme increases the speed of intracoding significantly with negligible loss of peak signal-to-noise ratio.

The new video coding standard, H.264/MPEG-4 AVC, uses variable block sizes ranging from 4/spl times/4 to 16/spl times/16 in interframe coding. This new feature has achieved significant coding gain compared to coding a macroblock (MB) using fixed block size. However, this feature results in extremely high computational complexity when brute force rate distortion optimization (RDO) algorithm is used. This paper proposes a fast intermode decision algorithm to decide the best mode in intercoding. It makes use of the spatial homogeneity and the temporal stationarity characteristics of video objects. Specifically, spatial homogeneity of a MB is decided based on the MB's edge intensity, and temporal stationarity is decided by the difference of the current MB and it colocated counterpart in the reference frame. Based on the homogeneity and stationarity of the video objects, only a small number of intermodes are selected in the RDO process. The experimental results show that the fast intermode decision algorithm is able to reduce on the average 30% encoding time, with a negligible peak signal-to-noise ratio loss of 0.03 dB or, equivalently, a bit rate increment of 0.6%.

The paper presents a fast block INTER mode decision algorithm to improve significantly the time efficiency of the encoder in H.264. It makes use of the spatial homogeneity of a video object's textures and the temporal stationarity characteristics inherent in video sequences. Specifically, the homogeneity decision of a block is based on edge information, and MB differencing is used to judge whether the MB is time-stationary. Based on the above analysis, only parts of inter prediction modes are chosen for RDO (rate distortion optimization) calculation. Experimental results show that the new scheme is able to achieve a reduction of 30% encoding time on average, with a negligible average PSNR loss of only 0.03 dB and a mere 0.6% bit rate increase compared with the original H.264 reference software.

This paper presents a rate control scheme for H.264 by introducing the concept of basic unit and a linear prediction model. The basic unit can be a macroblock (MB), a slice, or a frame. It can be used to obtain a trade-off between the overall coding efficiency and the bits fluctuation. The linear model is used to solve the chicken and egg dilemma existing in the rate control of H.264. Both constant bit rate (CBR) and variable bit rate (VBR) cases are studied. Our scheme has been adopted by H.264.

The emerging H.264-AVC video coding standard aims to significantly improve compression performance compared to all existing video coding standards. In order to achieve this, a robust rate-distortion optimization (RDO) technique is employed to select the best coding mode and reference frame for each macroblock. As a result, the complexity and computation load increase drastically. This paper presents a fast mode decision algorithm for H.264 intra prediction based on local edge information. Prior to intra prediction, an edge map is created and a local edge direction histogram is then established for each sub-block. Based on the distribution of the edge direction histogram, only a small part of intra prediction modes are chosen for RDO calculation. Experimental results show that the last intra mode decision scheme increases the speed of intra coding significantly with negligible loss of PSNR.

Conventional rate control schemes focus on the problem of finding an optimal quantization value for P- and B-frames. No rate control is available for the encoding of I-frames. This could pose big problems due to the large number of bits an I-frame could generate, and due to the fact that the number of bits varies drastically from sequence to sequence, depending on their image complexity. This problem becomes severe especially at very low bit rate. Therefore, a mechanism to allocate data bits to an I-frame according to its complexity is indispensable in order to have constant coding quality. This work presents a mechanism to establish for I-frames a generic relationship between the quantization value, data bits, and their image complexity. Experimental results show that this generic relationship provides a fairly accurate estimation of quantization value for an I-frame at given data bits and image complexity, and is very useful in controlling the data bits generated by an I-frame.

The H.264 video coding standard can achieve considerably higher coding efficiency than previous standards. In order to achieve this, a robust rate-distortion optimization (RDO) technique is employed to select the best coding mode for each macroblock. As a result. the encoder complexity is increased considerably. This paper presents a directional field based fast intra mode decision algorithm to improve the encoder's efficiency. Prior to intra prediction, the directional field is calculated for all the block size to decide the dominant edge direction in the blocks. Based on the edge direction information, only a small number of prediction modes are chosen for RDO calculation. Experimental results show that the fast intra mode decision algorithm increases the speed of intra coding significantly with negligible loss of PSNR

Many rate control algorithms focus on the adjustment of quantisation values to retain a certain buffer level, and arbitrary frame-skipping is often needed to keep the buffer from overflow at very low bit-rates. A content adaptive rate control is proposed to optimise the balance between spatial and temporal quality via active frame-skipping. The occurrence of frame-skipping is jointly dependent on the temporal and spatial quality of the video, and on the fullness of the buffer. This helps to achieve a consistent spatial and temporal quality and to enhance the overall perceptual quality. Experimental results show that the new scheme is simple but very effective, with large average PSNR gains and consistently improved visual quality, and the improvement in perceptual quality is much more significant than that in average PSNRs.

In this paper, we propose a new video post-processing algorithm for reducing coding artifacts. The algorithm is based on regularized image restoration technique, in which a nonconvex regularization function is used to enforce smoothness constraints on the spatial image pixels for suppressing coding artifacts while preserving important image attribute. Temporal regularization is carried out along the motion trajectories. We apply half-quadratic regularization and piecewise constraint strategy to simplify the optimization problem. Explicit filtering formulae are obtained and a recursive filtering algorithm is developed. Experimental results demonstrate that the proposed method can significantly improve both subjective and objective quality of MPEG-4 coded video sequences in comparison with the postprocessing technique in MPEG-4.

This paper proposes an adaptive frame layer rate control scheme for H.264 by introducing a linear model to predict the mean absolute difference (MAD) of current frame by that of previous one. The target bit rate for each frame is computed by adopting a fluid flow traffic model and linear tracking theory. The corresponding quantization parameter is computed by using a quadratic rate-distortion model. The rate distortion optimization (RDO) is then performed for all macroblocks (MBs) in the current frame by the quantization parameter. Both constant bit rate (CBR) and variable bit rate (VBR) cases are studied. The average PSNR is improved up to 0.75 dB compared to an encoder with fixed quantization parameter.

We introduce a PDA-based live video streaming system on GPRS network based on MPEG-4 video compression standard. Due to the limited computational resources of PDA, all the key modules of MPEG-4 codec are efficiently implemented and optimized such as multithreading, buffer design, wireless communication, encoder and decoder. Several novel techniques are developed in the coding, streaming as well as the post- processing stages of the system.

With the recent development of third-generation communication technologies, encoding live video using a PDA and sharing it among friends has become a reality. However, the embedded processor inside a PDA is still not powerful enough and there are two major hurdles to overcome: (1) video coding needs to meet the rigorous constraint of the available computation capacity of a PDA; (2) In a PDA the computing power allocated to video coding may vary drastically (in bursts). In this paper, a new adaptive rate control algorithm is proposed for video coding over a PDA. This adaptive rate control scheme takes into account the time constraint of a PDA, and its bit allocation depends not only on the available data bits, but more importantly, on the available coding time. Experimental results show that, compared to the existing rate control scheme, the new algorithm can always achieve the maximum frame rate, maximize the utilization of the available bandwidth and computing power, increase the average PSNR, and improve the subjective perceptual quality of the reconstructed video.

This paper discusses some practical issues in implementing the MPEG-4 Q2 rate-control scheme, and proposes a number of ways to improve it. The improved algorithm has the following main features: (1) the bits allocated to each P-frame or B-frame are in proportion to its distance from the end this GOP. i.e., more bits are allocated to the frames that are nearer to their reference I-frame; (2) the target buffer level is a function of the frame position in the GOP, so that it will be achieved gracefully at the end of a GOP; and (3) the quantization value of an I-frame is decided based on its spatial complexity. Experimental results show that the improved rate-control scheme has significantly reduced the occurrence of frame skipping, increased the average PSNR by up to 0.6 dB, and improved the perceptual quality of the reconstructed video.

Frame skipping in low bit video coding could significantly reduce the visual quality of the reconstructed video. This paper analyzes the' main causes of frame skipping in current MPEG-4 frame rate control scheme, and presents a number of ways to reduce the occurrence of frame skipping. The main features of the new algorithm include an adaptive thresholding technique, an adaptive P-frame target bits allocation, and an adaptive I-frame target bits allocation. Experimental results show that the new rate control scheme has significantly reduced the risks of frame skipping without sacrificing the overall PSNR of the reconstructed video.

This paper presents new MRF (Markov random field) models in a bidirectional Bayesian framework for accurate motion and occlusion field estimation. With careful selection of the five free parameters required by the models, good experimental results have been obtained. The resultant computational speed is also 5.5 times faster compared with the conventional "iterated conditional mode" relaxation using the proposed fast bidirectional relaxation.

In block motion estimation, search patterns with different shape or size have a very important impact on search speed and distortion performance. In this paper, we propose a novel algorithm using a hexagon-based search (HEXBS) pattern for fast block motion estimation. The proposed HEXBS algorithm may find any motion vector with fewer search points than the diamond search (DS) algorithm. The speedup gain of the HEXBS method over the DS algorithm is more striking for finding large motion vectors. Experimental results substantially justify the fastest performance of the HEXBS algorithm compared with several other popular fast algorithms.

It is well known that accurate dense motion field can improve the video coding efficiency. This paper presents a novel Markov random field (MRF) model that estimates both the dense motion and uncovered background fields in image sequences, and the application of these estimates in H.263-based video coding framework.

This paper gives a description of our real-time software MPEG-4 video decoder. The performance of the decoder and the fast algorithms used for real-time decoding are discussed. Our techniques used to speed up four computationally intensive modules, inverse discrete cosine transform, context-based arithmetic decoding, padding, and variable length decoding, are also presented in this paper. Our decoding speed is improved so that software real-time video decoding is possible.

This paper proposes a new Markov random field (MRF) model for the detection of occluded regions in image sequences. Motion vectors are not defined in an occluded region, thus the regions with high motion compensated prediction error are commonly regarded as occluded regions. However, badly motion compensated pixels will also appear as occluded pixels, making it difficult to distinguish the true occluded pixels from the poorly motion compensated regions. The proposed MRF model addresses this problem by incorporating motion information into the occlusion model. This is derived from the observation that occlusion occurs when objects move. It is found that an accurate occlusion region can be detected and better motion estimation can be made with the new model using iterated conditional modes (ICM).