This paper proposes a map free lane following solution based on low-cost 2D laser scanners for Autonomous Service Vehicle to fill the gap between future driverless car and the lane keeping assistant. The applications of autonomous service vehicle include feeder bus in a local residential area, shuttle bus in a park or playground, sprinkler car, sweeper car, and transporter in airport or container terminal. As autonomous service vehicle is running only in a limited area and its speed is slow compared to normal vehicles, we can further simplify the problem regardless of the issues of road infrastructure detection/communication and V2I maps which prevent the popularization of driverless car, and to propose a unique map free solution. The features of our approach include: 1) an innovative configuration for two 2D laser scanners to detect the lane with sharp curve; 2) a fast and accurate lane detection algorithm based on 2D laser's raw date directly; 3) a reliable and smooth path planning based on local lane fitting and prediction; and 4) a self-built unique drive-by-wire system for electronic car. We successfully tested our vehicle with autonomous driving in the testing field. The experiments show that the vehicle's trajectory matched the planned path accurately.

For autonomous navigation system of intelligent vehicle, robust and stable control with accurate tracking ability is one of the key requirements. In this paper, we present a systematic controller design approach for autonomous vehicle navigation system. The proposed controller integrates dynamic vehicle model and online updated path model by quadratic programming (QP) cost function, which considers both tracking error and stability. A novel path planner based on the differential dynamic programming (DDP) with consideration of the kinematic feasibility is used. The path tracking accuracy has been improved by utilizing the proposed dynamic preview controller. Promising experimental results showed that the overall navigation system is robust and stable.

Virtual try-on of clothes has received much attention recently due to its commercial potential. It can be used for online shopping or intelligent recommendation to narrow down the selections to a few designs and sizes. In this paper, we present a mixed reality system for 3D virtual clothes try-on that enables a user to see herself wearing virtual clothes while looking at a mirror display, without taking off her actual clothes. The user can select various virtual clothes for trying-on. The system physically simulates the selected virtual clothes on the user's body in real-time and the user can see virtual clothes fitting on the her mirror image from various angles as she moves. The major contribution of this paper is that we automatically customize an invisible (or partially visible) avatar based on the user's body size and the skin color and use it for proper clothes fitting, alignment and clothes simulation in our virtual try-on system. We present three scenarios: i) virtual clothes on the avatar, ii) virtual clothes on the user's image and iii) virtual clothes on the avatar blended with the user's face image. We have conducted a user study to evaluate the effectiveness of these three solutions from the end user's perception of quality attributes, cognitive attributes and attitude towards using. The user study shows that among these three scenarios, the second one is most preferred by the users and for 50% of them the experience they had with our system was sufficient to make the purchase decision for the outfits they virtually tried-on.

This paper proposes a new method to automatically segment and re-color the clothing in an image sequence. Background and foreground are separated using background subtraction method with a Gaussian mixture model built for the static scene. Skin and face areas are detected and removed on the segmented foreground. A 2D histogram is constructed using the remaining pixels to model the probability distribution of the clothing chromaticity, which is then applied to each frame to find accurate clothing area using back projection method. The segmented clothing area is re-colored by mapping the hue value to a new one in the HSV color space to maintain the contrast of the clothing. Experiments on the real video captured with a monocular webcam are shown to demonstrate the effectiveness of the proposed algorithm for clothing segmentation and re-coloring.

It is possible to synthesis a high dynamic range (HDR) image by combining differently exposed low dynamic range (LDR) images of the same scene into one single image. For an HDR scene under low light condition, captured LDR images tend to be noisy, and the noise could usually be amplified during the synthesis process, causing severe degradation of the image quality. In order to reduce the noise during the HDR synthesis process, a de-noising scheme based on half-quadratic regularization is proposed for the synthesis of HDR images in this paper. By taking into consideration of two unique HDR image features, the proposed scheme effectively removes the noise, while the edges still being well preserved in the synthesized HDR images.

In this paper, we introduce a new collaborative image processing algorithm to enhance contours and surface details of a scene through combination of multi-light images, which capture the same scene with fixed view point but different lighting conditions. Firstly, a detail layer that contains all contents of the multi-light images is generated through a gradient domain method and a quadratic filter. A new shadow detection algorithm is introduced to remove the artifacts from the detail layer. Subsequently, a base layer is constructed by using one input image. To further increase the visibility of details in dark areas, a simple tool is presented to brighten all dark areas of the base layer. Finally, the detail layer is multiplied to the base layer to synthesize the enhanced image that contains the desirable details. This enhanced image can present an elaborate description of the real scene. The proposed scheme also gives some interactivities that allow users to easily change the appearance of the enhanced image according to their preferences.

This paper presents a new adaptive tone mapping for high dynamic range (HDR) images. In the proposed scheme, the luminance of an HDR image is decomposed into a base layer with large gradients and a detail layer with small gradients by using an adaptive half quadratic regularization method. The base layer is compressed by a novel global mapping to reduce the dynamic range while the detail layer can be amplified to enhance the local contrasts. With the proposed scheme, the generated low dynamic range (LDR) images look more natural and local details are also preserved very well.

In this paper, we propose a new image quality evaluation method, which is based on wavelet frequency sensitivity model and spatial non-uniform resolution sensitivity model with multiple region-of-interest. The visual quality index is given using probability summation model with weighted visual error factors. Experiments on the video database have shown that the proposed method can achieve very good correlation with subjective evaluation.

Objective video quality metrics reported in the literature have so far been focused on TV signals with large frame sizes, full TV frame rates, and high compressed bit rates. The aim of this paper is to provide an analysis of 4 objective video quality metrics tested on low bit rate videos with small frame sizes, various frame rates, and low bit rates. The video quality metrics used were the NTIA video quality metric, modified Watson's digital video quality metric, video structural similarity metric, and the perceptual video quality metric. The test videos consist of H.264 compressed videos with CIF and QCIF frame sizes, at various bit rates (24 kbps - 384 kbps) and frame rates (7.5 fps - 30 fps). Here, results and analysis derived from the comparison of the metrics are provided.

In general, low and very-low bitrate video communication systems cannot achieve the full frame rate (25 Hz for PAL or 30 Hz for NSTC), which brings temporal distortion. The influence of reduced frame rate on subjective quality evaluation is a very important topic for both perceptual quality metrics and communication system optimization. In this paper, we present our work on the numerical modeling of the influence. The work includes two parts: first we measured the detectability and annoyance of periodic frame dropping's effect on perceptual visual quality evaluation under different content and frame size conditions. Then, a simple and effective feature is proposed to represent the content of video in temporal quality evaluation. The high Pearson and Spearman correlation results between the MOS and proposed model, as well as the results of other two error metrics, confirm the success of the selected temporal content-quality feature.

This paper proposes a new full-reference objective metric for image quality assessment. The reference and distorted images are decomposed into a number of wavelet subbands, in which mean curvatures and perceived error of the wavelet coefficients of two images are computed and integrated to give overall quality index. Taking structural similarity and error visibility into account, the new method can achieve high consistency with subjective evaluation compared with other metrics. Experimental results have shown the effectiveness of the proposed metric.

In this paper, we present our work on developing perceptual quality metrics for low and very low bit-rate videos. At current stage, we've finished a number of subjective viewing experiments, and several video content-based models are built based on the experiments to measure the perceptual quality of frame-dropped and blurred video sequences. Experimental results show that the models can achieve very good performances.

This paper proposes a method for assessing blur caused by defocus from one image. The essential idea is to estimate the point spread function (PSF) from the line spread function (LSF), whereas the LSF is constructed from edge information. The procedure includes edge detection and localization, pixel interpolation, LSF determination and PSF extraction. This approach is independent of image contents, and the algorithm has fast speed as it works in spatial domain without complex Fourier transform or iterative computation. The experimental results validate the proposed method. It can be used for blind image quality evaluation in vision-based applications.

In this paper, we propose a new image quality evaluation method, which is based on foveation spatial error sensitivity model and wavelet error sensitivity model. The image quality is measured by isotropic contrast and the standard deviation of the errors of the wavelet coefficients in subbands with the weighting factors that are determined by two models. Experiments on a test database comprising JPEG and JPEG 2000 compressed images have shown that the proposed metric can achieve very good correlation with subjective evaluation.

The state-of-the-art objective video quality metrics reported in the literature have so far been focused on TV signals with large frame sizes, full TV frame rates, and high compressed bit rates. The aim of this paper is to present the performance of three widely-known objective video quality metrics but tested on multimedia videos with small frame sizes, various frame rates, and low bit rates. The video quality metrics used were the NTIA video quality metrics (the "general" and the "videoconferencing" models), a modified Watson's DVQ metric, and the VSSIM metric. The test videos consist of both H.264 and MPEG-4 compressed videos with GIF and QCIF frame sizes, at various bit rates (24 kbps-384 kbps) and frame rates (7.5 fps-30 fps). Here, results and conclusions derived from the comparison of the three metrics will be provided.

In this paper, we propose a new video quality evaluation method, which measures the perceived error between original and distorted images using wavelet just-noticeable-distortion (JND) profile. The error of the subband coefficient that is below the visual detection threshold is ignored while some errors that take place in the content-activity regions are masked. Luminance masking and temporal contrast masking are integrated to adjust the perceived error. Experiments on VQEG (video quality expert group) test sequences show that the proposed method can achieve good correlation with subjective evaluation.

This paper proposed an objective video quality metric designed for automatically assessing the perceived quality of digitally compressed multimedia videos using H.264 video compression. The rationale in proposing perceptual-based metric is because traditional measure, peak signal-to-noise ratio (PSNR), has been found to correlate poorly with subjective quality ratings, particularly at much lower bit rates. In this paper, computational models have been applied to emulate human visual perception based on a combination of local and global modulating factors. The proposed video quality metric has been tested on GIF and QCIF video sequences compressed using H.264 video compression technique at various bit rates (24-384 Kbps) and frame rates (7.5-30 Hz). Performance of the proposed metric with respect to subjective scores will be reported and a comparison with PSNR and also the video structural similarity method (being one of the best video quality metric for high bit rate videos recently reported in the literature) will also be provided in this paper

Many post-processing algorithms can efficiently reduce blocking artifacts but pay less attention to the sharpness of the image edges. In this paper, we propose a new post-processing method based on forward and backward anisotropic diffusion for reducing blocking, ringing effects and sharpening the edges. The proposed algorithm has been applied to the compressed video sequences for picture quality enhancement. Experimental results show that the proposed algorithm can efficiently improve the objective and subjective quality for different bit rate case.

We propose Gabor energy similarity (GES), a new full reference image quality assessment metric based on the measuring of Gabor energy features of images. It has been recognized that: 1) 2D Gabor filters can attain the theoretical conjoint resolution limit in spatial and frequency domain defined by Heisenberg's uncertainty principle; 2) it is widely reported that simple cells in visual cortex can be well modeled by 2D Gabor functions; and 3) the feature of Gabor energy is closely related to the model of complex cells in primary visual cortex that are very sensitive to minor changes in nature scene. Motivated by these facts we attempt to design a new image quality assessment by exploring the similarity in Gabor energy functions between the original and the distorted image. The images are firstly decomposed by a filter bank consists of 48 Gabor filters (6 directions, 4 spatial resolutions, and symmetric or anti-symmetric). Then the local energy feature is extracted by computing the modulus of the responses of symmetric and anti-symmetric kernel filters at each point. We finally propose an image quality metric based on averaged cross correlation between two images. Extensive experimental results are used to justify the effectiveness of the proposed GES.

The paper proposes an objective perceptual video quality metric to assess the perceived quality of digital videos automatically. Traditionally, peak signal-to-noise ratio (PSNR) has been used to represent the quality of a compressed video sequence. However, PSNR has been found to correlate poorly with subjective quality ratings, particularly at much lower bit rates and frame rates. Computational models are applied to emulate human visual perception based on block-fidelity, content richness fidelity, spatial-textural, colour, and temporal maskings. The proposed video quality metric has been tested on CIF and QCIF video sequences compressed at various bit rates and frame rates. It has been shown to give significantly better correlation to human perception than PSNR.

This paper proposed an objective colour perceptual video quality metric to automatically assess the perceived quality of colour digital videos. The rationale in proposing perceptual-based metric is because peak signal-to-noise ratio (PSNR), which has been used traditionally to represent the quality of a compressed video sequence, has been found to correlate poorly with subjective quality ratings, particularly at much lower bit rates and frame rates. In this paper, computational models have been applied to emulate human visual perception based on a combination of local and global modulating factors. The proposed video quality metric has been tested on CIF and QCIF colour video sequences compressed at various bit rates and frame rates. It has been shown to give significantly better correlations to human perception than PSNR.

Peak signal-to-noise ratio (PSNR) is commonly used as an objective quality metric in signal processing. However, PSNR correlates poorly with the subjective quality rating. In this paper, we propose a new metric using contrast signal-to-noise ratio (CSNR), which measures the ratio of the contrast information level of distorted signal to the contrast level of the error signal. The performance of the proposed method has been verified using a database of images compressed with JPEG and JPEG 2000. The results show that it can achieve very good correlation with the subjective mean opinion scores in terms of prediction accuracy, monotonicity and consistency.

Most distortion/artifacts in low bit-rate compressed videos are strong and obvious to human eyes, and may even change the spatio-temporal structure of visual objects. Therefore it is important to evaluate the compound perceptual impact of spatial and temporal changes. Recent neurophysiological researches have found that the high-level spatial information (shape and depth, etc.) and temporal information (motion, etc.) are processed separately in two visual pathways in human brains. In this paper, spatial and temporal distortions are evaluated separately in two modules, and are then combined in an integration module. Two subjective viewing experiments are performed. One experiment is to evaluate the subjective quality on temporal distortion only, and second experiment is to evaluate the low bit-rate videos with both spatial and temporal distortions. Experimental results and the associated analysis show that the proposed visual quality metric is in good alignment with human perception

We present a method for extracting local visual perceptual cues and its application for rate control of videophone, in order to ensure the scarce bits to be assigned for maximum perceptual coding quality. The optimum quantization step is determined with the rate-distortion model considering the local perceptual cues in the visual signal. For extraction of the perceptual cues, luminance adaptation and texture masking are used as the stimulus-driven factors, while skin color serves as the cognition-driven factor in the current implementation. Both objective and subjective quality evaluations are given by evaluating the proposed perceptual rate control (PRC) scheme in the H.263 platform, and the evaluations show that the proposed PRC scheme achieves significant quality improvement in block-based coding for bandwidth-hungry applications.

We present a motion-compensated residue signal preprocessing scheme in video coding scheme based on just-noticeable-distortion (JND) profile. Human eyes cannot sense any changes below the JND threshold around a pixel due to their underlying spatial/temporal masking properties. An appropriate (even imperfect) JND model can significantly help to improve the performance of video coding algorithms. From the viewpoint of signal compression, smaller variance of signal results in less objective distortion of the reconstructed signal for a given bit rate. In this paper, a new JND estimator for color video is devised in image-domain with the nonlinear additivity model for masking (NAMM) and is incorporated into a motion-compensated residue signal preprocessor for variance reduction toward coding quality enhancement. As the result, both perceptual quality and objective quality are enhanced in coded video at a given bit rate. A solution of adaptively determining the parameter for the residue preprocessor is also proposed. The devised technique can be applied to any standardized video coding scheme based on motion compensated prediction. It provides an extra design option for quality control, besides quantization, in contrast with most of the existing perceptually adaptive schemes which have so far focused on determination of proper quantization steps. As an example for demonstration, the proposed scheme has been implemented in the MPEG-2 TM5 coder, and achieved an average peak signal-to-noise (PSNR) increment of 0.505 dB over the twenty video sequences which have been tested. The perceptual quality improvement has been confirmed by the subjective viewing tests conducted.

With the fast development of visual noise-shaping related applications (visual compression, error resilience, watermarking, encryption, and display), there is an increasingly significant demand on incorporating perceptual characteristics into these applications for improved performance. In this paper, a very important mechanism of the human brain, visual attention, is introduced for visual sensitivity and visual quality evaluation. Based upon the analysis, a new numerical measure for visual attention's modulatory aftereffects, perceptual quality significance map (PQSM), is proposed. To a certain extent, the PQSM reflects the processing ability of the human brain on local visual contents statistically. The PQSM is generated with the integration of local perceptual stimuli from color contrast, texture contrast, motion, as well as cognitive features (skin color and face in this study). Experimental results with subjective viewing demonstrate the performance improvement on two PQSM-modulated visual sensitivity models and two PQSM-based visual quality metrics.

Both visual sensitivity and spatial selectivity determine the overall visibility threshold at each pixel in an image, according to the physiological and psychological evidence towards the human visual system (HVS). Visual sensitivity can be decided by an existing estimator for just-noticeable-distortion (JND). A computational model is proposed for incorporating a selectivity measure into the JND profile so that more effective noise shaping is possible in various applications. Experimental results with noise-embedded video sequences confirm that the introduction of spatial selectivity enhances the performance of the JND profile used in noise shaping.

A perceptual weighting model is proposed for effective rate control so as to enhance the perceptual coding quality of videophone. We exploit two categories of factors affecting the perception of the human visual system, stimulus-driven factors and cognition-driven factors. In order to achieve a simple, but effective, perceptual weighting model, we use luminance adaptation and texture masking as the stimulus-driven factors, while skin color serves as the cognition-driven factor in the videophone application. Both objective and subjective quality evaluations of videophone-like sequences in the H.263 platform validate the effectiveness of our perceptual weighting model.

The perceptual visual quality evaluation of Human Visual System (HVS) is very complex. It concerns almost all aspects of visual processing in vision path, from low-level neuron activities to high-level visual perception. Existing perceptual Visual Quality Metrics (VQMs) only considered several of the mechanisms of HVS and many others are ignored. In this paper, two global modulatory factors, visual attention and motion suppression, are modelled and combined to form a mathematic expression - Perceptual Quality Significant Level (PQSL). To a certain extent, it is believed that PQSL value reflect the processing ability of human brain on local visual contents. To evaluate their effects on visual quality evaluation, two VQMs are proposed. One is a MSE-like VQM based on PQSL-modulated JND profile, which was proposed in Z.K. Lu et al., (2004); the other VQM is based on Wang's visual quality assessment Zhou Wang et al., (2004) PQSL values are used to adjust the weights of his structural similarity index. Experimental results show that introducing of the global modulatory factors can improve the performance of current visual quality metrics.

Traditionally, peak signal-to-noise ratio (PSNR) has been used to represent the quality of a compressed video sequence. However, PSNR has been found to correlate poorly with subjective quality ratings, particularly at much lower bit rates and frame rates. This paper proposes an objective video quality metric to automatically measure the perceived quality of a stream of video images based on a combined measure of distortion-invisibility, block-fidelity, and content richness fidelity. The proposed method has been tested on CIF and QCIF video sequences compressed at low bit rates and frame rates and it is shown to give significantly better correlations to human perception than peak signal-to-noise ratio (PSNR).

We present a perceptually-adaptive pre-processing scheme for motion-compensated residue, based on just-noticeable-distortion (JND) profile. Human eyes cannot sense any changes below the JND threshold around a pixel due to their underlying spatial/temporal masking properties. From the viewpoint of signal compression, a smaller variance of signal results in less objective distortion of the reconstructed signal for a given bit-rate. In this paper, the JND profile is incorporated into a motion-compensated residue signal preprocessor for variance reduction, aimed at coding quality enhancement. A solution of adaptively determining the parameter for the residue pre-processor is also proposed. Experimental results show that both perceptual quality and objective quality are enhanced in coded video at a given bitrate.

JPEG 2000 image coding standard aims to achieve high compression performance. As the bit rate decreases, visual degradation caused by information loss on an image becomes more and more visible. We propose an adaptive nonlinear diffusion method to reduce ringing artifacts while preserving and enhancing image edges. The diffusion coefficient is the linear combination of three membership functions that correspond to the different gradient scales. The parameters are tuned using a stochastic optimization technique so that the diffusion coefficient can adapt to the given image data to achieve optimal restoration performance. Experimental results on JPEG 2000 compressed images have shown the effectiveness of the proposed nonlinear diffusion processes for ringing artifacts removal.

It is known that the human visual system (HVS) does not pay equal attention to each error and even region in judging picture quality. In this paper, we combine a perceptual model with an integrated detection of the spatially transitional regions in visual distortion evaluation to better match the HVS perception to visual quality. For decompressed images or video, the spatially transitional regions are the regions where major perceptually disturbing artefacts caused by edge impairments (mainly due to blurring and locations where the edge information is not adequately represented) and the presence of false edges (mainly due to blockiness and the presence of strong rippling effects of ringing) usually occur. Such regions are efficiently detected based on a single two-dimensional spatial high-pass filter in our work. Good correlation between the proposed method and the human perception has been demonstrated with the full set of 50-Hz video quality expert group test data.

The paper presents a new and general concept, PQSM (perceptual quality significance map), to be used in measuring visual distortion. It makes use of the mechanism that the HVS (human visual system) pays more attention to certain areas of visual signals due to one or more of the following factors: salient features in image/video; cues from domain knowledge; association of other media (e.g., speech or audio). PQSM is a 3D/4D array whose elements represent the relative perceptual-quality significance levels for the corresponding pixels/regions for images or video. Due to its generality, PQSM can be incorporated into any visual distortion metrics; it can improve effectiveness or/and efficiency of perceptual metrics or even enhance a PSNR-based metric. A three-stage PQSM generation method is also proposed, with an implementation of motion, luminance, skin-color and face mapping. Experimental results show that the scheme can significantly improve the performance of current image/video distortion metrics.

We propose a new spatial just noticeable distortion (JND) profile for color image processing. The JND threshold depends on various masking effects underlying existing in the human vision system (HVS). How to efficiently integrate different masking effects together is the key issue of modelling the JND profile. Based on recent vision research results, we model the masking effects in different stimulus dimensions as a nonlinear additivity model for masking (NAMM). It applies to all color components and accounts for the compound impact of luminance masking and texture masking to estimate the JND threshold in images. In our PSNR and subjective comparison to the related work, the proposed NAMM scheme provides a more accurate JND profile towards the actual JND bound in the HVS.

Peak signal-to-noise ratio (PSNR) is not a good measure of perceived picture quality, especially at low bit rates of coding. This paper proposes a new approach for computing perceptual distortion for visual signal in order to provide an objective measure for perceptual quality at low bit rate coding in typically mobile communications. The regions with three major perceptually disturbing artifacts, namely, damaged edge, blockiness and ringing, are detected as the basis of assessment. The correlation of the metric with human perception has been demonstrated with low bit rate CIF test data.

In this paper, we propose a new video post-processing algorithm for reducing coding artifacts. The algorithm is based on regularized image restoration technique, in which a nonconvex regularization function is used to enforce smoothness constraints on the spatial image pixels for suppressing coding artifacts while preserving important image attribute. Temporal regularization is carried out along the motion trajectories. We apply half-quadratic regularization and piecewise constraint strategy to simplify the optimization problem. Explicit filtering formulae are obtained and a recursive filtering algorithm is developed. Experimental results demonstrate that the proposed method can significantly improve both subjective and objective quality of MPEG-4 coded video sequences in comparison with the postprocessing technique in MPEG-4.

We explore a new perceptually-adaptive video coding (PVC) scheme in the motion-compensated prediction loop for hybrid video compression, aiming at better perceptual coding quality and operational efficiency. A new just noticeable distortion (JND) estimator for color images/video is first devised in image-domain based on our introduced nonlinear additivity model for masking (NAMM). Secondly, the image-domain JND profile is incorporated into hybrid video encoding via the JND-adaptive motion estimation (ME) and residue filtering. The scheme works with any prevalent video coding standards and various ME strategies. As an example of implementation, it is applied to the MPEG-2 TM5 coder and demonstrated to achieve average improvement of over 18% in ME efficiency, 0.6 dB in perceptual coding quality and most remarkably, 0.17 dB in the objective coding quality measure (PSNR).

We introduce a PDA-based live video streaming system on GPRS network based on MPEG-4 video compression standard. Due to the limited computational resources of PDA, all the key modules of MPEG-4 codec are efficiently implemented and optimized such as multithreading, buffer design, wireless communication, encoder and decoder. Several novel techniques are developed in the coding, streaming as well as the post- processing stages of the system.

In this paper, a method for measuring the perceptual image quality of JPEG-2000 coded images has been proposed. The image quality is characterized by the average edge-spread in the image, or more specifically the average extent of the slope's spread of an edge in the opposing gradients' directions. The proposed method is, in effect, a way of measuring the amount of blurring in the image. The effectiveness of such method is validated using subjective tests and the experimental results show that the proposed method can provide results that correlate relatively well with human subjective ratings.

This paper presents a hybrid method of fuzzy clustering using evolutionary search and self-organization, which has the advantages of both. Simulation experiments have shown that the convergence speed of the proposed method is faster than that of pure evolutionary algorithm.