Video Quality Assessment (VQA) metrics play a crucial role in modern video processing systems, yet their computational costs and environmental impact have received limited attention. This paper presents a comprehensive analysis of state-of-the-art VQA metrics across two standard datasets, including processing time, memory usage, and the ability to predict subjective scores. Results demonstrate that full-reference video metrics achieve superior accuracy compared to image-based metrics such as SSIM, but require up to more than 20 times more energy on average. No-reference metrics offer even higher accuracy but substantially larger energy usage. This difference in computational requirements has significant implications for energy consumption and environmental impact, particularly in large-scale deployments or high-resolution video processing scenarios.

This paper investigates the energy consumption of video encoding for high dynamic range videos. Specifically, we compare the energy consumption of the compression process using 10 -bit input sequences, a tone-mapped 8 -bit input sequence at 10 -bit internal bit depth, and encoding an 8 -bit input sequence using an encoder with an internal bit depth of 8 bit. We find that linear scaling of the luminance and chrominance values leads to degradations of the visual quality, but that significant encoding complexity and thus encoding energy can be saved. An important reason for this is the availability of vector instructions, which are not available for the 10-bit encoder. Furthermore, we find that at sufficiently low target bitrates, the compression efficiency at an internal bit depth of 8 bit exceeds the compression efficiency of regular 10-bit encoding.

This paper investigates the potential of exploiting change blindness (CB), a perceptual phenomenon where changes in a visual stimulus are not noticed by the observer, to enhance rendering efficiency and achieve higher compression gains in video encoding. We explore various distortion techniques, including foveation, cropping, and dimming, to introduce changes that can be noticed but typically are not, due to the limited bandwidth of our perceptual experience. Through a user study involving HEVC-encoded videos and a task designed to direct gaze, we assess the impact of these distortions on perceived video quality, bitrate reduction, and display luminance. Our findings suggest that significant bitrate and luminance reductions can be achieved without adversely affecting perceived quality, highlighting CB’s potential for reducing energy consumption and strain on streaming infrastructure. Despite observer variability and the ephemeral nature of CB, our results demonstrate that conscious attention plays a crucial role in the perception of video quality and that exploiting CB can lead to substantial efficiency gains in video coding.

What the human visual system can perceive is strongly limited by the capacity of our working memory and attention. Such limitations result in the human observer’s inability to perceive large-scale changes in a stimulus, a phenomenon known as change blindness. In this paper, we started with the premise that this phenomenon can be exploited in video coding, especially HDR-video compression where the bitrate is high. We designed an HDR-video encoding approach that relies on spatially and temporally varying quantization parameters within the framework of HEVC video encoding. In the absence of a reliable change blindness prediction model, to extract compression candidate regions (CCR) we used an existing saliency prediction algorithm. We explored different configurations and carried out a subjective study to test our hypothesis. While our methodology did not lead to significantly superior performance in terms of the ratio between perceived quality and bitrate, we were able to determine potential flaws in our methodology, such as the employed saliency model for CCR prediction (chosen for computational efficiency, but eventually not sufficiently accurate), as well as a very strong subjective bias due to observers priming themselves early on in the experiment about the type of artifacts they should look for, thus creating a scenario with little ecological validity.

We consider the problem of density-based unsupervised classification in hyperspectral data. Our focus is especially on methods based on K nearest neighbors (KNN) graph. In this paper, we propose some improvements of recently published methods in this vein, namely GWENN (Graph WatershEd using Nearest Neighbors) as well as a KNN version of Density Peaks Clustering. These improvements address (i) the structure of the KNN graph, which can be modified efficiently to emphasize the dependencies between objects, especially in high dimensional data sets; (ii) the choice of the pointwise density model; and (iii) the ability of these methods to handle variable NN graphs. The improved methods are compared in the context of pixel partitioning in hyperspectral images and are shown to give encouraging results, outperforming state-of-the-art methods like DBSCAN and FCM.

Visual masking is a well-studied phenomenon that has been exploited for signal compression, computer graphics and data hiding. Among the different types of visual masking, chromatic masking has received very little attention despite its importance and proven potential for the aforementioned applications. In this paper, we ask whether a deep neural network can learn to predict the detection thresholds in a chromatic masking paradigm. For that, a CNN model was trained and evaluated using a dataset made of 480 image patches for which chromatic thresholds were registered in terms of log-Gabor targets, as well as Root Mean Square (RMS) error. Experimental results show the superiority of the proposed approach.

This study investigates the inability of the human vision system to detect gradual changes in video quality, specifically chroma reduction. We present the results from a subjective study, in which participants compared video stimuli of three different types: pristine, mildly chroma-reduced background with a fixed rate, chroma reduced with a gradual rate from a pristine first frame to a fully achromatic background in the last one. Our results show that the second type yields quality ratings that are significantly lower than those of the gradually modified samples, which are on par with those of the pristine samples.

We present the results of a subjective experiment where we measured detection thresholds for 2°wide noise targets placed in 23 different natural scenes. Unlike previous studies on visual masking, we focus particularly on dissociating cases of low-level and high-level masking. That is, cases where the target is not perceived predominantly due to limits of either early or late vision. To that end, we exploit the change blindness paradigm and analyse detection rates, times and primed subjective ratings of target visibility. Our results are of significance for developing advanced models of human vision for signal quality/fidelity assessment, particularly in the context of compression.

Understanding how we perceive visual quality is important in a range of applications such as streaming or cross-media reproduction. Despite current perception models showing high correlations with recorded mean opinion scores, the factors influencing visual quality are still not fully understood, particularly when it comes to memory. We designed and carried out a study to compare quality assessment for two different levels of reliance on visual short-term memory. We found that assessments based mostly on memory tend to be more positive for compression, blur or gamut mapping distortions. Our results further highlight the role of memory in subjective quality assessment and visual masking.

The Hough transform is a well known technique for detecting straight lines within images, especially in the presence of noise, or where there is incomplete data (gaps or occlusions). When subjected to lens distortion, straight lines become curved, and indeed this can be used to identify and correct lens distortion. However, curved lines distort and blur the peaks within the Hough transform, making the lines more difficult to detect. However, by analysing the distortion within the Hough transform, it is possible to directly estimate the lens distortion parameters enabling the distortion to be corrected in real time. The proposed technique uses a Terasic DE1-SoC FPGA board (Cyclone V FPGA) to fit a parabola to the distorted peak using Hough's original transform, and from the parabola coefficients directly estimates the lens distortion parameter. This enables the following frame to be corrected in parallel with curve detection.

Despite recent advances in subjective image quality research, many fundamental questions are still unanswered. Although we understand early vision fairly well, little is known about late vision and how the two interact with each other. In this paper, we look at one particular limit in that interaction: high-level visual masking, which is best illustrated by the change blindness paradigm. We carried out a user study designed specifically to measure the influence of high-level masking by means of priming. Results suggest a significant influence of high-level masking in image fidelity assessment at the 95% confidence level for half of the participants, with an average magnitude over three times that of intra-observer variability.

Change blindness is a striking phenomenon which basically means that we can look without seeing. It originates from a faulty communication between early vision (the eye) and visual working memory (the brain). In this paper, we present evidence that this faulty communication needs to be accounted for in image fidelity assessment (also known as full-reference image quality assessment). We designed a user study to analyse participants opinions based on how much they have to rely on their visual working memory in order to give fidelity score. Results demonstrate that significantly more severe judgments were made when reliance on visual short-term memory was minimal, suggesting limitations in the observers' ability to notice image differences in the typical pairwise comparison setup. Furthermore, a comparison of the efficiency of six state-of-the-art image fidelity assessment models (so-called metrics) reveals that five of them perform significantly better at predicting results obtained when reliance on memory is minimal.

Aiming at understanding the role of short-term memory in subjective image quality assessment, we report and compare results from two pair-comparison methods: stimuli shown side-by-side versus stimuli shown one after the other. Our results suggest that there is a significant chance that an observer will make different quality assessments in the two setups.

We propose a new strategy to evaluate the quality of multi and hyperspectral images, from the perspective of human perception. We define the spectral image difference as the overall perceived difference between two spectral images under a set of specified viewing conditions (illuminants). First, we analyze the stability of seven image-difference features across illuminants, by means of an information-theoretic strategy. We demonstrate, in particular, that in the case of common spectral distortions (spectral gamut mapping, spectral compression, spectral reconstruction), chromatic features vary much more than achromatic ones despite considering chromatic adaptation. Then, we propose two computationally efficient spectral image difference metrics and compare them to the results of a subjective visual experiment. A significant improvement is shown over existing metrics such as the widely used root-mean square error.

We introduce a new strategy to evaluate the perceived quality of spectral image reproductions. We rely on the idea that spectral image difference can be computed by averaging scores of a color image-difference measure, computed for various viewing conditions. Two approaches are proposed to efficiently approximate this quality prediction by taking advantage of image-difference redundancies across illuminants. Our results suggest that, for the evaluated distortions, image-difference features extracted from the lightness component vary much less across illuminants than those from chroma and hue components. For our setup, we could reduce the numerical effort to compute the original prediction to 1.3%, without impairing accuracy.

We address the problem of unsupervised band reduction in hyperspectral remote sensing imagery. We propose the use of an information theoretic criterion to automatically separate the sensor's spectral range into disjoint subbands without ground truth knowledge. Our approach, named BandClust, preserves the physical sense of the spectral data and automatically provides relevant spectral subbands, i.e., of maximal informational complementarity. Experiments using real hyperspectral images are conducted to compare BandClust with four other unsupervised approaches. The comparison of the selected dimensionality reduction methods is performed via supervised classification using support vector machines and shows the potential of the proposed approach.

We present a new method for the visualization of spectral images, based on a selection of three relevant spectral channels to build a red–green–blue composite. Band selection is achieved by means of information measures at the first, second, and third orders. Irrelevant channels are preliminarily removed by means of a center-surround entropy comparison. A visualization-oriented spectrum segmentation based on the use of color matching functions allows for computational ease and adjustment of the natural rendering. Results from the proposed method are presented and objectively compared to four other dimensionality reduction techniques in terms of naturalness and informative content.

In this paper, a new color visualization technique for multi- and hyperspectral images is proposed. This method is based on a maximization of the perceptual distance between the scene endmembers as well as natural constancy of the resulting images. The stretched CMF principle is used to transform reflectance into values in the CIE L*a*b* colorspace combined with an a priori known segmentation map for separability enhancement between classes. Boundaries are set in the a*b* subspace to balance the natural palette of colors in order to ease interpretation by a human expert. Convincing results on two different images are shown.