Human beings always evaluate the perceptual quality of an image coupled with identifying the semantic content of images. This paper addresses the correlation issue between stereoscopic image quality assessment (SIQA) and semantic recognition. In contrast to the previous SIQA methods that relied on binocular quality-aware features of a stereoscopic image, our approach also extracts binocular semantic features using a pre-trained deep convolutional neural network (DCNN) on a large dataset like ImageNet dataset, as well as the manually designed binocular quality-aware features. It can solve the problem of limited SIQA dataset size and facilitate better prediction on the quality. Experimental results demonstrate that the binocular semantic features are a good predictor for the stereoscopic image quality. The proposed method outperforms the state-of-the-art SIQA methods on four benchmark SIQA datasets. Significantly, all Spearman rank-order correlation coefficients (SROCCs) between the predicted scores and the subjective scores on the four datasets exceed 0.95. The MATLAB source code of the proposed method is available athttps://github.com/kyohoonsim/Blind-Stereoscopic-Image-Quality-Evaluator-based-on-Binocular-Semantic-and-Quality-Channelshttps://github.com/kyohoonsim.

When human visual system (HVS) looks at a scene, it extracts various features from the image about the scene to understand it. The extracted features are compared with the stored memory on the analogous scene to judge their similarity[1]. By analyzing to the similarity, HVS understands the scene presented on eyes. Based on the neurobiological basis, we propose a 2D full reference (FR) image quality assessment (IQA) method, named mean and deviation of deep and local similarity (MaD-DLS) that compares similarity between many original and distorted deep feature maps from convolutional neural networks (CNNs). MaD-DLS uses a deep learning algorithm, but since it uses the convolutional layers of a pre-trained model, it is free from training. For pooling of local quality scores within a deep similarity map, we employ two important descriptive statistics, (weighted) mean and standard deviation and name it mean and deviation (MaD) pooling. The two statistics each have the physical meaning: the weighted mean reflects effect of visual saliency on quality, whereas the standard deviation reflects effect of distortion distribution within the image on it. Experimental results show that MaD-DLS is superior or competitive to the existing methods and the MaD pooling is effective. The MATLAB source code of MaD-DLS will be available online soon.

Most of the current blind stereoscopic image quality assessment (SIQA) algorithms cannot show reliable accuracy. One reason is that they do not have the deep architectures and the other reason is that they are designed on the relatively weak biological basis, compared with findings on human visual system (HVS). In this paper, we propose a Deep Edge and COlor Signal INtegrity Evaluator (DECOSINE) based on the whole visual perception route from eyes to the frontal lobe, and especially focus on edge and color signal processing in retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN). Furthermore, to model the complex and deep structure of the visual cortex, Segmented Stacked Auto-encoder (S-SAE) is used, which has not utilized for SIQA before. The utilization of the S-SAE complements weakness of deep learning-based SIQA metrics that require a very long training time. Experiments are conducted on popular SIQA databases, and the superiority of DECOSINE in terms of prediction accuracy and monotonicity is proved. The experimental results show that our model about the whole visual perception route and utilization of S-SAE are effective for SIQA.

Most previous 2-D and 3-D image quality evaluators were based on shallow architectures. Their shallow architectures cannot model phenomenon occurring in human visual systems sufficiently. Disparities between left and right views have been importantly used for 3-D image quality assessment (IQA), but single disparity, depth, or cyclopean maps made from the disparities cannot thoroughly reflect the depth sense. In this paper, we propose a blind stereoscopic image quality evaluator using stacked auto-encoders (SAE). The proposed method is based on two theories on initial stages of stereopsis. One is cyclopean channel theory and the other is binocular summation/difference channels theory. Especially, a cyclopean image that models the former theory is computed to consider binocular suppression, whereas summation and difference images that model the latter one are utilized to treat the depth sense. We train three SAEs for quality-aware features from the three images in an unsupervised manner. Through the SAEs, the features are transformed into more meaningful features, and they are used to train two regressors. The regressors are used to obtain a final predicted score. Experimental results conducted on popular 3-D IQA databases prove that the proposed algorithm outperforms state-of-the-art 3-D IQA methods.