Magnetic resonance (MR) imaging is vulnerable to a variety of artifacts, which potentially degrade the perceived quality of MR images and, consequently, may cause inefficient and/or inaccurate diagnosis. In general, these artifacts can be classified as structured or unstructured depending on the correlation of the artifact with the original content. In addition, the artifact can be white or colored depending on the flatness of the frequency spectrum of the artifact. In current MR imaging applications, design choices allow one type of artifact to be traded off with another type of artifact. Hence, to support these design choices, the relative impact of structured versus unstructured or colored versus white artifacts on perceived image quality needs to be known. To this end, we conducted two subjective experiments. Clinical application specialists rated the quality of MR images, distorted with different types of artifacts at various levels of degradation. The results demonstrate that unstructured artifacts deteriorate quality less than structured artifacts, while colored artifacts preserve quality better than white artifacts.

Unobtrusive recognition of the user's mood is an essential capability for affect-adaptive systems. Mood is a subtle, long-term affective state, often misrecognized even by humans. The challenge to train a machine to recognize it from, for example, a video of the user, is significant, and already begins with the lack of ground truth for supervised learning. Existing affective databases consist mainly of short videos, annotated in terms of expressed emotions rather than mood. In very few cases, we encounter perceived mood annotations, of questionable reliability, however, due to the subjectivity of mood estimation and the small number of coders involved. In this work, we introduce a new database for mood recognition from video. Our database contains 180 long, acted videos, depicting typical daily scenarios, and subtle facial and bodily expressions. The videos cover three visual modalities (face, body, Kinect data), and are annotated in terms of emotions (via G-trace) and mood (via the Self-Assessment Manikin and the AffectButton). To annotate the database exhaustively, we exploit crowdsourcing to reach out to an extensive number of nonexpert coders. We validate the reliability of our crowdsourced annotations by (1) adopting a number of criteria to filter out unreliable coders, and (2) comparing the annotations of a subset of our videos with those collected in a controlled lab setting.

A smart environment designed to adapt to a user's affective state should be able to decipher unobtrusively that user's underlying mood. Great effort has been devoted to automatic punctual emotion recognition from visual input. Conversely, little has been done to recognize longer-lasting affective states, such as mood. Taking for granted the effectiveness of emotion recognition algorithms, we propose a model for estimating mood from a known sequence of punctual emotions. To validate our model experimentally, we rely on the human annotations of two well-established databases: the VAM and the HUMAINE. We perform two analyses: the first serves as a proof of concept and tests whether punctual emotions cluster around the mood in the emotion space. The results indicate that emotion annotations, continuous in time and value, facilitate mood estimation, as opposed to discrete emotion annotations scattered randomly within the video timespan. The second analysis explores factors that account for the mood recognition from emotions, by examining how individual human coders perceive the underlying mood of a person. A moving average function with exponential discount of the past emotions achieves mood prediction accuracy above 60 percent, which is higher than the chance level and higher than mutual human agreement.

Although it has been reported that depth of field influences depth perception in nonstereo photographs, it remains unclear how depth of field affects depth perception under stereo viewing conditions. We showed participants stereo photographs with different depths of field using a Wheatstone stereoscope and a commercially available 3D TV. The depicted scene contained a floor, a background, and a measuring probe at different locations. Participants drew a floor plan of the depicted scene to scale. We found that perceived depth decreased with decreasing depth of field for shallow depths of field in scenes containing a height-in-the-field cue. For larger depths of field, different effects were found depending on the display system and the viewing distance. There was no effect on perceived depth using the 3D TV, but perceived depth decreased with increasing depth of field using the Wheatstone stereoscope. However, in the 3D TV case, we found that the perceived depth decreased with increasing depth of field in scenes in which the height-in-the-field cue was removed. This indicates that the effect of depth of field on perceived depth may be influenced by other depth cues in the scene, such as height-in-the-field cues.

In this research, a novel experimental setup was built to investigate human beings' sensitivity for lighting properties of a real scene using a gauge object or "probe". Light is a kind of medium that makes objects visible without being visible itself in empty space. The light field determines the appearance of objects and materials in it, and the appearance of objects is the main cue to the light field. Cuttle [2003] suggested to use probes (i.e., gauge objects) to reveal lighting characteristics: the sharpness of a light beam can be estimated from the shadow pattern of a peg on a disk and from the highlight pattern on a black shiny sphere, while the flow of light can be revealed from the shading pattern on a matte white sphere. In a workshop led by Madsen and Donn, white spheres were used as a light flow meter to visually assess spatial and form-giving characteristics of daylight [2006]. By putting white spheres into empty spaces of rendered artificial architecture, the judgments of the light flow improved. However, the shading pattern varied with different perspective views, which made the digital light-flow-meter perceptually ambiguous. Pont and Koenderink [2007] implemented the white sphere as their gauge object for estimating human beings' sensitivity to direction, diffuseness and intensity of illumination and found that human observers confuse illumination direction with illumination diffuseness (diffuseness-direction ambiguity). This explains the problem Madsen encountered and indicates that to implement the light-flow-meter as a quick and simple tool to visually assess the properties of light, alternative probes must be developed. Theoretically, the illuminance flow or texture contrast gradients over a rough sphere gives additional cues about its illumination [2004]. In this study we will address whether observers use such cues so that we can improve the probe by adding roughness. In addition, we will test how human beings perceive the global properties of the light field in a real scene instead of on photographs on a computer screen.

This paper reports the results of a psychometric experiment aimed at investigating the visibility and annoyance of packet-loss artifacts, with a focus on understanding to what extent their presence influences viewing behavior. The rationale behind this study is that packet loss artifacts might “distract” visual attention, creating visual saliency themselves, thereby becoming more visible. In turn, higher artifact visibility might impact their annoyance. Our study involved seven videos, compressed at a very high bitrate (thus free of spatial artifacts), impaired by discarding packets at different packet loss ratios. We tracked the observers' eye-movements while (1) they assessed the annoyance of impaired videos and (2) looked freely at pristine videos. Our results show that the viewing behavior significantly changes from pristine to impaired videos. This change is related to both properties of the video and to the annoyance of the artifacts presented.

Our previous research has demonstrated that adding natural scene saliency (NSS) obtained from eye-tracking data may improve an objective metric's performance in predicting perceived image quality. In this letter, we further investigate the image content dependency of this improvement. Results show that the variation in saliency between observers highly depends on image content, and that this variation predicts the extent to which a certain image may profit from adding saliency in the objective image quality assessment.

Fixation density maps (FDM) created from eye tracking experiments are widely used in image processing applications. The FDM are assumed to be reliable ground truths of human visual attention and as such, one expects a high similarity between FDM created in different laboratories. So far, no studies have analyzed the degree of similarity between FDM from independent laboratories and the related impact on the applications. In this paper, we perform a thorough comparison of FDM from three independently conducted eye tracking experiments. We focus on the effect of presentation time and image content and evaluate the impact of the FDM differences on three applications: visual saliency modeling, image quality assessment, and image retargeting. It is shown that the FDM are very similar and that their impact on the applications is low. The individual experiment comparisons, however, are found to be significantly different, showing that inter-laboratory differences strongly depend on the experimental conditions of the laboratories. The FDM are publicly available to the research community.

Based on “ground truth” eye-tracking data, earlier research [1] shows that adding natural scene saliency (NSS) can improve an objective metric's performance in predicting perceived image quality. To include NSS in a real-world implementation of an objective metric, a computational model instead of eye-tracking data is needed. Existing models of visual saliency are generally designed for a specific domain, and so, not applicable to image quality prediction. In this paper, we propose an efficient model for NSS, inspired by findings from our eye-tracking studies. Experimental results show that the proposed model sufficiently captures the saliency of the eye-tracking data, and applying the model to objective image quality metrics enhances their performance in the same manner as when including eye-tracking data.

3-D, and in particular stereoscopic, displays are believed to have great prospects. Due to imperfections in the optical and electronic components, however, some image distortions still exist in these displays. Among them, crosstalk is considered to be a very annoying one. The aim of the current research is to investigate crosstalk perception and its influence on image quality when rendering dynamic image sequences on two types of stereoscopic displays: a hold-type liquid-crystal display with patterned retarder and circular polarizing glasses, and an impulse-type digital light processing display with shutter glasses. Results show that the dynamic behavior of the displays has an impact on crosstalk perception. Perceived motion blur can mask crosstalk visibility, especially at low binocular parallax, which limits the crosstalk-induced image quality degradation.

People with some signs of binocular dysfunctioning can be susceptible to visual complaints associated with viewing stereoscopic content at large viewing distances. Two performance measurements enabled to distinguish people by their binocular status (BS) in previous research: the ratio of performance of the Wilkins rate of reading test (WRRT) between 2-D and 3-D, and the vergence facility. In an experiment, first, an extensive optometric screening was carried out to differentiate visually asymptomatic young adults with good BS (GBS) (N= 27) from those with a moderate BS (MBS) (N= 6). Second, participants had to perform the WRRT at short viewing distance followed by a questionnaire under different screen disparity settings. The results reveal that the ratio of the WRRT between 0 and -1.5 screen disparity is an appropriate indicator of participants with MBS in comparison with participants with GBS. In addition, the results show that 0.75°of screen disparity is already problematic for people with MBS. We conclude that the WRRT-ratio has potential as a BS test in consumer applications to provide individual settings for comfortable screen disparities based on viewers' BS.

Since the human visual system (HVS) is the ultimate assessor of image quality, current research on the design of objective image quality metrics tends to include an important feature of the HVS, namely, visual attention. Different metrics for image quality prediction have been extended with a computational model of visual attention, but the resulting gain in reliability of the metrics so far was variable. To better understand the basic added value of including visual attention in the design of objective metrics, we used measured data of visual attention. To this end, we performed two eye-tracking experiments: one with a free-looking task and one with a quality assessment task. In the first experiment, 20 observers looked freely to 29 unimpaired original images, yielding us so-called natural scene saliency (NSS). In the second experiment, 20 different observers assessed the quality of distorted versions of the original images. The resulting saliency maps showed some differences with the NSS, and therefore, we applied both types of saliency to four different objective metrics predicting the quality of JPEG compressed images. For both types of saliency the performance gain of the metrics improved, but to a larger extent when adding the NSS. As a consequence, we further integrated NSS in several state-of-the-art quality metrics, including three full-reference metrics and two no-reference metrics, and evaluated their prediction performance for a larger set of distortions. By doing so, we evaluated whether and to what extent the addition of NSS is beneficial to objective quality prediction in general terms. In addition, we address some practical issues in the design of an attention-based metric. The eye-tracking data are made available to the research community .

Visual attention models are typically designed based on human gaze patterns recorded through eye tracking. In this paper, two similar eye tracking experiments from independent laboratories are presented, in which humans observed natural images under task-free condition. The resulting saliency maps are analysed with respect to two criteria; the consistency between the experiments and the impact of the image presentation time. It is shown, that the saliency maps between the experiments are strongly correlated independent of presentation time. It is further revealed that the presentation time can be reduced without substantially sacrificing the accuracy of the convergent saliency map. The results provide valuable insight into the similarity of saliency maps from independent laboratories and are highly beneficial for the creation of converging saliency maps at reduced experimental time and cost.

A novel no-reference metric that can automatically quantify ringing annoyance in compressed images is presented. In the first step a recently proposed ringing region detection method extracts the regions which are likely to be impaired by ringing artifacts. To quantify ringing annoyance in these detected regions, the visibility of ringing artifacts is estimated, and is compared to the activity of the corresponding local background. The local annoyance score calculated for each individual ringing region is averaged over all ringing regions to yield a ringing annoyance score for the whole image. A psychovisual experiment is carried out to measure ringing annoyance subjectively and to validate the proposed metric. The performance of our metric is compared to existing alternatives in literature and shows to be highly consistent with subjective data.

Reduced-reference systems can predict in real-time the perceived quality of images for digital broadcasting, only requiring that a limited set of features, extracted from the original undistorted signals, is transmitted together with the image data. This paper uses descriptors based on the color correlogram, analyzing the alterations in the color distribution of an image as a consequence of the occurrence of distortions, for the reduced reference data. The processing architecture relies on a double layer at the receiver end. The first layer identifies the kind of distortion that may affect the received signal. The second layer deploys a dedicated prediction module for each type of distortion; every predictor yields an objective quality score, thus completing the estimation process. Computational-intelligence models are used extensively to support both layers with empirical training. The double-layer architecture implements a general purpose image quality assessment system, not being tied up to specific distortions and, at the same time, it allows us to benefit from the accuracy of specific, distortion-targeted metrics. Experimental results based on subjective quality data confirm the general validity of the approach.

An efficient approach toward a no-reference ringing metric intrinsically exists of two steps: first detecting regions in an image where ringing might occur, and second quantifying the ringing annoyance in these regions. This paper presents a novel approach toward the first step: the automatic detection of regions visually impaired by ringing artifacts in compressed images. It is a no-reference approach, taking into account the specific physical structure of ringing artifacts combined with properties of the human visual system (HVS). To maintain low complexity for real-time applications, the proposed approach adopts a perceptually relevant edge detector to capture regions in the image susceptible to ringing, and a simple yet efficient model of visual masking to determine ringing visibility. The approach is validated with the results of a psychovisual experiment, and its performance is compared to existing alternatives in literature for ringing region detection. Experimental results show that our method is promising in terms of both reliability and computational efficiency.

Current research on image quality assessment tends to include visual attention in objective metrics to further enhance their performance. A variety of computational models of visual attention are implemented in different metrics, but their accuracy in representing human visual attention is not fully proved yet. Thus, to provide more accurate evidence on whether and to what extent visual attention can be beneficial for objective quality prediction, the use of ¿ground truth¿ visual attention data is highly desired. In this paper, the data of an eye-tracking experiment are integrated in two objective metrics well-known in literature. Experimental results demonstrate that there is indeed a gain in performance including visual attention in objective metrics. The amount of gain in performance tends to depend on the type of objective metric and image distortion.

This paper investigates how saliency obtained from eye-tracking data can be integrated into objective metrics for JPEG compressed images. The objective metrics used in this paper are both based on features, locally extracted from the images and serving as input to a neural network for the overall quality prediction. We compare various weighting functions to combine saliency with these objective metrics, taking into account the possible distraction due to artifacts that might affect the quality judgment. Experimental results indicate that including saliency into objective metrics in an appropriate way can further enhance their performance.

A novel no-reference blockiness metric that can automatically and perceptually quantify blocking artifacts of DCT coding is presented. The proposed metric is built upon the specific structure information of the artifact itself combined with the properties of the human visual system (HVS) by means of a simple and efficient model of visual masking. Investigations are conducted to reduce the additional cost introduced by the human vision model, without compromising its overall prediction ability. The proposed metric is validated through comparing its performance to state-of-the-art HVS model based blockiness metrics with respect to accuracy, reliability and computational complexity.

This work applies neural-network technologies to the quality assessment of digital pictures processed by image-enhancement algorithms. The objective model uses a circular back-propagation (CPB) neural network to mimic human perception: the feed-forward structure maps input 'feature' vectors characterizing images into the associated quality ratings, obtained from human voters. "Objective" feature vectors describe images by measuring global statistical properties, which are worked out on a block-by-block basis. CPB networks can handle multidimensional data with non-linear relationships; at the same time, the neural model allows one to decouple the feature-selection task from the mapping-function set-up. Experimental results confirm the approach effectiveness, as the system provides a satisfactory approximation of the results of tests involving human viewers.

Several state-of-the-art noise reduction algorithms are ranked using both subjective evaluation and 20 objective measures proposed in the literature. It is found that no regression model based on these objective quality measures can replace subjective assessment.