Vehicle taillight detection is essential to analyze and predict driver intention in collision avoidance systems. In this article, we propose an end-to-end framework that locates the rear brake and turn signals from video stream in real-time. The system adopts the fast YOLOv3-tiny as the backbone model and three improvements have been made to increase the detection accuracy on taillight semantics, i.e., additional output layer for multi-scale detection, spatial pyramid pooling (SPP) module for richer deep features, and focal loss for alleviation of class imbalance and hard sample classification. Experimental results demonstrate that the integration of multi-scale features as well as hard examples mining greatly contributes to the turn light detection. The detection accuracy is significantly increased by 7.36%, 32.04% and 21.65% (absolute gain) for brake, left-turn and right-turn signals, respectively. In addition, we construct the taillight detection dataset, with brake and turn signals are specified with bounding boxes, which may help nourishing the development of this realm.

In this paper, we address problems associated with free-energy-principle-based image quality assessment (IQA) algorithms for objectively assessing the quality of Screen Content (SC) and three-dimensional (3-D) synthesized images and also propose a very fast and efficient IQA algorithm to address these issues. These algorithms separate an image into predicted and disorder residual parts and assume disorder residual part does not contribute much to the overall perceptual quality. These algorithms fail for quality estimation of SC images as information of textual regions in SC images are largely separated into the disorder residual part and less information in the predicted part and subsequently, given a negligible emphasis. However, this is in contrast with the characteristics of human vision. Since our eyes are well trained to detect text in daily life. So, our human vision has prior information about text regions and can sense small distortions in these regions. In this paper, we proposed a new reduced-reference IQA algorithm for SC images based upon a more perceptually relevant prediction model and distortion categorization, which overcomes problems with existing free-energy-principle-based predictors. From experiments, it is validated that the proposed model has a better capability of efficiently estimating the quality of SC images as compared to the recently developed reduced-reference IQA algorithms. We also applied the proposed algorithm to judge the quality of 3-D synthesized images and observed that it even achieves better performance than the full-reference IQA metrics specifically designed for the 3-D synthesized views.

Objective image retargeting quality assessment aims to use computational models to predict the retargeted image quality consistent with subjective perception. In this paper, we propose a multiple-level feature (MLF)-based quality measure to predict the perceptual quality of retargeted images. We first provide an in-depth analysis on the low-level aspect ratio similarity feature, and then propose a mid-level edge group similarity feature, to better address the shape/structure related distortion. Furthermore, a high-level face block similarity feature is designed to deal with sensitive region deformation. The multiple-level features are complementary as they quantify different aspects of quality degradation in the retargeted image, and the MLF measure learned by regression is used to predict the perceptual quality of retargeted images. Extensive experimental results performed on two public benchmark databases demonstrate that the proposed MLF measure achieves higher quality prediction accuracy than the existing relevant state-of-the-art quality measures.

In recent years, single-image super-resolution (SR) reconstruction has aroused wide attention. Massive SR enhancement algorithms have been proposed. However, much less work has been down on the perceptual evaluation of SR enhanced images and the corresponding enhancement algorithms. In this work, we create a Super-resolution Reconstructed Image Database (SRID), which consists of images produced by two interpolation methods and six popular SR image enhancement algorithms at different amplification factors. Then, subjective experiment is conducted to collect the subjective scores by using the single-stimulus method. The performances of the SR image enhancement algorithms are then evaluated by the obtained subjective scores. Finally, the performances of the general-purpose no-reference (NR) image quality metrics are investigated on the SRID database. This study shows that it is difficult for the state-of-the-art NR image quality metrics to predict the quality of SR enhanced images.

In this paper, we propose a novel full-reference objective quality assessment metric for screen content images (SCIs) by structure features and uncertainty weighting (SFUW). The input SCI is first divided into textual and pictorial regions. The visual quality of textual regions is estimated based on perceptual structural similarity, where the gradient information is adopted as the structural feature. To predict the visual quality of pictorial regions in SCIs, we extract the structural features and luminance features for similarity computation between the reference and distorted pictorial patches. To obtain the final visual quality of SCI, we design an uncertainty weighting method by perceptual theories to fuse the visual quality of textual and pictorial regions effectively. Experimental results show that the proposed SFUW can obtain better performance of visual quality prediction for SCIs than other existing ones.

Document image quality assessment (DIQA) aims to predict the visual quality of degraded document images. Although the definition of “visual quality” can change based on the specific applications, in this paper, we use OCR accuracy as a metric for quality and develop a novel no-reference DIQA method based on high order image statistics for OCR accuracy prediction. The proposed method consists of three steps. First, normalized local image patches are extracted with regular grid and a comprehensive document image codebook is constructed by K-means clustering. Second, local features are softly assigned to several nearest codewords, and the direct differences between high order statistics of local features and codewords are calculated as global quality aware features. Finally, support vector regression (SVR) is utilized to learn the mapping between extracted image features and OCR accuracies. Experimental results on two document image databases show that the proposed method can accurately predict OCR accuracy and outperforms previous algorithms.

Research in human visual perception has found that the sense of natural scences cannot be conveyed only through lines and edges. It also needs the knowledge of texture regions within the image, which can be obtained through the analysis of higher derivatives. Inspired by the research from neuroscience that high order derivatives can capture the details of image structure, we propose a novel simple yet effective blind image quality assessment (IQA) metric based on high order derivatives (BHOD). In the proposed metric, we extract multi-scale structural features up to fourth order image derivatives, to obtain the image structural features. Support vector regression (SVR) is used to learn the mapping between feature space and subjective opinion scores. The proposed method is extensively evaluated on three image databases and shows highly competitive performance to state-of-the-art NR-IQA methods.

In this paper, we propose an effective no-reference image quality assessment (IQA) method based on local region statistics (NRLRS). The proposed method is built on the hypothesis that image distortions may alter the local region statistics which can be well characterized by the inter-pixel relationship. Hence, by extracting perceptual features that describe the inter-pixel patterns of a distorted image, we can effectively quantify the impact of image degradation. For this purpose, the perceptual gray-level differences between neighboring pixels are extracted and a Gaussian Mixture Model (GMM) codebook is constructed as the generative model of extracted features. The Fisher vector representation is then derived to describe image as their derivations from the GMM model. Finally, partial least square regression is used to map the Fisher encodings to quality scores. Experimental results indicate that the proposed method achieves better performance in quality prediction as compared to relevant full-reference and no-reference IQA methods.

In-loop filtering is an important technique in modern video coding standards. In this paper, we propose a transform-domain in-loop filter to further improve the compression performance of high efficiency video coding (HEVC) standard. The proposed method estimates block transform coefficients by adaptively fusing two prediction sources according to their uncertainties respectively. The first prediction is the block transform coefficients of compressed video frames, the uncertainty of which is related to quantization parameters. The second prediction is the weighted average of transform blocks in a neighborhood, and the weights are designed according to block similarity. Its uncertainty is estimated based on the coefficient variance. To optimize the filtering performance, the parameters utilized in the proposed in-loop filter are learned from compressed videos for each quantization parameter offline, and frame level flags are utilized to switch the proposed in-loop filter according to rate-distortion cost. Extensive experimental results show that the proposed in-loop filter can further improves the compression efficiency of HEVC.

In practice, images available to consumers usually undergo several stages of processing including acquisition, compression, transmission, and presentation, and each stage may introduce certain type of distortion. It is common that images are simultaneously distorted by multiple types of distortions. Most existing objective image quality assessment (IQA) methods have been designed to estimate perceived quality of images corrupted by a single image processing stage. In this letter, we propose a no-reference (NR) IQA method to predict the visual quality of multiply-distorted images based on structural degradation. In the proposed method, a novel structural feature is extracted as the gradient-weighted histogram of local binary pattern (LBP) calculated on the gradient map (GWH-GLBP), which is effective to describe the complex degradation pattern introduced by multiple distortions. Extensive experiments conducted on two public multiply-distorted image databases have demonstrated that the proposed GWH-GLBP metric compares favorably with existing full-reference and NR IQA methods in terms of high accordance with human subjective ratings.

Blind image quality assessment (BIQA) research aims to develop a perceptual model to evaluate the quality of distorted images automatically and accurately without access to the non-distorted reference images. The state-of-the-art general purpose BIQA methods can be classified into two categories according to the types of features used. The first includes handcrafted features which rely on the statistical regularities of natural images. These, however, are not suitable for images containing text and artificial graphics. The second includes learning-based features which invariably require large codebook or supervised codebook updating procedures to obtain satisfactory performance. These are time-consuming and not applicable in practice. In this paper, we propose a novel general purpose BIQA method based on high order statistics aggregation (HOSA), requiring only a small codebook. HOSA consists of three steps. First, local normalized image patches are extracted as local features through a regular grid, and a codebook containing 100 codewords is constructed by K-means clustering. In addition to the mean of each cluster, the diagonal covariance and coskewness (i.e., dimension-wise variance and skewness) of clusters are also calculated. Second, each local feature is softly assigned to several nearest clusters and the differences of high order statistics (mean, variance and skewness) between local features and corresponding clusters are softly aggregated to build the global quality aware image representation. Finally, support vector regression is adopted to learn the mapping between perceptual features and subjective opinion scores. The proposed method has been extensively evaluated on ten image databases with both simulated and realistic image distortions, and shows highly competitive performance to the state-of-the-art BIQA methods.

Blind image quality assessment (BIQA) aims to develop quantitative measures to automatically and accurately estimate perceptual image quality without any prior information about the reference image. In this paper, we introduce a novel BIQA metric by structural and luminance information, based on the characteristics of human visual perception for distorted image. We extract the perceptual structural features of distorted image by the local binary pattern distribution. Besides, the distribution of normalized luminance magnitudes is extracted to represent the luminance changes in distorted image. After extracting the features for structures and luminance, support vector regression is adopted to model the complex nonlinear relationship from feature space to quality measure. The proposed BIQA model is called no-reference quality assessment using statistical structural and luminance features (NRSL). Extensive experiments conducted on four synthetically distorted image databases and three naturally distorted image databases have demonstrated that the proposed NRSL metric compares favorably with the relevant state-of-the-art BIQA models in terms of high correlation with human subjective ratings. The MATLAB source code and validation results of NRSL are publicly online at http://www.ntu.edu.sg/home/wslin/Publications.htm.

The goal of Image Quality Assessment (IQA) is to design computational models that can automatically predict the perceived image quality consistent with human subjective ratings. In this paper, we propose a full reference IQA metric gradient weighted structural similarity (GW-SSIM) by incorporating the gradient information to the well-known IQA metric SSIM. Experimental results demonstrate that GW-SSIM can greatly improve the quality prediction accuracy and achieve the best performance among the SSIM-based methods by addressing SSIM's shortcomings. Additionally, incorporating the proposed gradient weighting (GW) map into peak-signal-to-noise ratio (PSNR) also makes it quite competitive to state-of-the-art IQA models, and this is meaningful since PSNR is still a widely adopted metric.

Previous feature learning based blind image quality assessment (BIQA) methods invariably require large codebook or codebook updating procedure to obtain satisfying performance. In this paper, we propose a novel general purpose BIQA method, local feature aggregation (LFA) model, which requires only a much smaller codebook without the need for codebook updating. The proposed model consists of three steps. Firstly, normalized local raw image patches are extracted as local features through a regular grid and a 100 codeword codebook is constructed by K-means clustering. Secondly, the soft weighted differences between local features and codewords are aggregated to the global quality aware representation. Finally, support vector regression (SVR) is utilized to learn the mapping between features and subjective opinion scores. The proposed method is evaluated on two large image databases and achieves comparable performance to state-of-the-art BIQA methods.

In this paper, we propose two novel Statistical Metric Fusion (SMF) methods for Image Quality Assessment (IQA) metric enhancement. First, local quality map is constructed from existing state-of-the-art IQA algorithm. After that several statistical indices are extracted from local quality map. Finally, the extracted statistical indices are fused by Supervised Statistical Metric Fusion (SMF-S) based on Support Vector Regression (SVR) and Unsupervised Statistical Metric Fusion (SMF-U) based on Reciprocal Rank Fusion (RRF) to obtain the final quality score, respectively. Experimental results on the largest public IQA database TID2013 have demonstrated that the two proposed SMF methods can generally enhance the quality prediction performance of the fused IQA metric in terms of high correlation with human opinion scores.