Diffusion-based models have recently revolutionized image generation, achieving unprecedented levels of fidelity. However, consistent generation of high-quality images remains challenging partly due to the lack of conditioning mechanisms for perceptual quality. In this work, we propose methods to integrate image quality assessment (IQA) models into diffusion-based generators, enabling quality-aware image generation. We show that diffusion models can learn complex qualitative relationships from both IQA models' outputs and internal activations. First, we experiment with gradient-based guidance to optimize image quality directly and show this method has limited generalizability. To address this, we introduce IQA-Adapter, a novel framework that conditions generation on target quality levels by learning the implicit relationship between images and quality scores. When conditioned on high target quality, IQA-Adapter can shift the distribution of generated images towards a higher-quality subdomain, and, inversely, it can be used as a degradation model, generating progressively more distorted images when provided with a lower-quality signal. Under high-quality condition, IQA-Adapter achieves up to a 10% improvement across multiple objective metrics, as confirmed by a user preference study, while preserving generative diversity and content. Furthermore, we extend IQA-Adapter to a reference-based conditioning scenario, utilizing the rich activation space of IQA models to transfer highly specific, content-agnostic qualitative features between images.

Most neural-network-based image- and video-quality-assessment metrics exhibit state-of-the-art performance; recent research, however, exposes their vulnerability to adversarial attacks. These attacks manipulate input data to inflate metric scores without enhancing visual quality. Although current research predominantly focuses on white-box adversarial attacks, black-box attacks remain underdeveloped despite their practical relevance, such as targeting metrics that are nondifferentiable or whose weights are unavailable. This paper adapts 10 black-box adversarial attacks, originally designed for image classifiers, to evaluate their effectiveness against image- and video-quality-assessment metrics. We tested them on 18 quality metrics and demonstrated their susceptibility to adversarial manipulations. Nonetheless, the practical utility of these attacks is limited by their computational complexity. To address this problem, we incorporated universal adversarial perturbations (UAPs) into five black-box attacks. Our proposed UAP attacks, despite providing less attack strength than the originals, still effectively increase the metric scores and are viable for real-time applications. Our code is available at https://github.com/georgebychkov/black-box-iqa.

Modern neural-network-based Image Quality Assessment (IQA) metrics are vulnerable to adversarial attacks, which can be exploited to manipulate search engine rankings, benchmark results, and content quality assessments, raising concerns about the reliability of IQA metrics in critical applications. This paper presents the first comprehensive study of IQA defense mechanisms in response to adversarial attacks on these metrics to pave the way for safer use of IQA metrics. We systematically evaluated 30 defense strategies, including purification, training-based, and certified methods --- and applied 14 adversarial attacks in adaptive and non-adaptive settings to compare these defenses on 9 no-reference IQA metrics. Our proposed benchmark aims to guide the development of IQA defense methods and is open to submissions; the latest results and code are at https://msu-video-group.github.io/adversarial-defenses-for-iqa/.

We propose the LEHA-CVQAD (Large-scale Enriched Human Annotated) dataset, which comprises 6,240 clips for compression-oriented video quality assessment. 59 source videos are encoded with 186 codec-preset variants, ≈1.8M pairwise, and ≈1.5k MOS ratings are fused into a single quality scale; part of the videos remains hidden for blind evaluation. We also propose Rate-Distortion Alignment Error (RDAE), a novel evaluation metric that quantifies how well VQA models preserve bitrate-quality ordering, directly supporting codec parameter tuning. Testing IQA/VQA methods reveals that popular VQA metrics exhibit high RDAE and lower correlations, underscoring the dataset's challenges and utility. The open part and the results of LEHA-CVQAD are available at https://aleksandrgushchin.github.io/lcvqad/

Image Quality Assessment (IQA) models are increasingly relied upon to evaluate image quality in real-world systems --- from compression and enhancement to generation and streaming. Yet their adoption brings a fundamental risk: these models are inherently unstable. Adversarial manipulations can easily fool them, inflating scores and undermining trust. Traditionally, such vulnerabilities are addressed through data-driven defenses --- adversarial retraining, regularization, or input purification. But what if this is the wrong lens? What if robustness in perceptual models is not something to learn but something to design? In this work, we propose a provocative idea: robustness as an architectural prior. Rather than training models to resist perturbations, we reshape their internal structure to suppress sensitivity from the ground up. We achieve this by enforcing orthogonal information flow, constraining the network to norm-preserving operations --- and further stabilizing the system through pruning and fine-tuning. The result is a robust IQA architecture that withstands adversarial attacks without requiring adversarial training or significant changes to the original model. This approach suggests a shift in perspective: from optimizing robustness through data to engineering it through design.

Screen sharing is a common feature in video conferencing applications, especially for remote work and presentations. However, internet conditions such as limited bandwidth, packet loss, and compression can significantly reduce the visual quality of shared screen content. Most existing video quality metrics are designed for natural scenes and were not benchmarked on screen content. In this work, we present a large-scale subjective dataset of screen content videos captured from video conferencing apps. The dataset includes 1,600 distorted videos with corresponding subjective quality scores. Subjective scores were collected using crowdsourced pairwise comparisons. The dataset provides a valuable resource for developing and benchmarking video quality metrics tailored to screen content. The evaluation of objective metrics revealed that several general-purpose quality metrics outperform both full-reference and no-reference metrics. The dataset is available at the following link: https://videoprocessing.github.io/screen-content-dataset.

Nowadays, neural-network-based image- and video-quality metrics perform better than traditional methods. However, they also became more vulnerable to adversarial attacks that increase metrics' scores without improving visual quality. The existing benchmarks of quality metrics compare their performance in terms of correlation with subjective quality and calculation time. Nonetheless, the adversarial robustness of image-quality metrics is also an area worth researching. This paper analyses modern metrics' robustness to different adversarial attacks. We adapted adversarial attacks from computer vision tasks and compared attacks' efficiency against 15 no-reference image- and video-quality metrics. Some metrics showed high resistance to adversarial attacks, which makes their usage in benchmarks safer than vulnerable metrics. The benchmark accepts submissions of new metrics for researchers who want to make their metrics more robust to attacks or to find such metrics for their needs. The latest results can be found online: https://videoprocessing.ai/benchmarks/metrics-robustness.html.

No-reference image- and video-quality metrics are widely used in video processing benchmarks. The robustness of learning-based metrics under video attacks has not been widely studied. In addition to having success, attacks on metrics that can be employed in video processing benchmarks must be fast and imperceptible. This paper introduces an Invisible One-Iteration (IOI) adversarial attack on no-reference image and video quality metrics. The proposed method uses two modules to ensure high visual quality and temporal stability of adversarial videos and runs for one iteration, which makes it fast. We compared our method alongside eight prior approaches using image and video datasets via objective and subjective tests. Our method exhibited superior visual quality across various attacked metric architectures while maintaining comparable attack success and speed. We made the code available on GitHub: https://github.com/katiashh/ioi-attack.

Banding, a common video quality artifact, it refers to the noticeable gradient steps or abrupt transitions between adjacent colors or shades within an image or video, often as a resulting from low color depth or compression algorithms. Assessing banded video quality is vital, especially with the rise of high-quality screens. Several banding metrics have been proposed in recent years to quantify and evaluate the presence of banding artifacts in videos. However, the visibility of banding strongly depends on viewing conditions such as screen brightness, contrast, ambient lighting, and others. This results in inconsistencies in the correlation between banding metrics and the subjective quality perceived across various screens. In this study, we conduct subjective experiments under varied viewing conditions and demonstrate the limited reliability of banding metrics.

In this paper, we propose a new method of analysing the stability of modern deep image- and video-quality metrics to different adversarial attacks. The stability analysis of quality metrics is becoming important because nowadays the majority of metrics employ neural networks. Unlike traditional quality metrics based on nature scene statistics or other hand-crafter features, learning-based methods are more vulnerable to adversarial attacks. The usage of such unstable metrics in benchmarks may lead to being exploited by the developers of image and video processing algorithms to achieve higher positions in leaderboards. The majority of known adversarial attacks on images designed for computer vision tasks are not fast enough to be used within real-time video processing algorithms. We propose four fast attacks on metrics suitable for real-life scenarios. The proposed methods are based on creating perturbations that increase metrics scores and can be applied frame-by-frame to attack videos. We analyse the stability of seven widely used no-reference image- and video-quality metrics to proposed attacks. The results showed that only three metrics are stable against our real-life attacks. This research yields insights to further aid in designing stable neural-network-based no-reference quality metrics. Proposed attacks can serve as an additional verification of metrics’ reliability.

More and more visual-quality metrics are being developed to assess the quality of images, but little research has considered their limitations. In this paper, we demonstrate that image preprocessing before compression can artificially increase the quality scores provided by the popular metrics DISTS, LPIPS, HaarPSI, and VIF. We propose a series of neural-network preprocessing models that increase DISTS by up to 34.5%, LPIPS by up to 36.8%, VIF by up to 98.0%, and HaarPSI by up to 22.6% in the case of JPEG-compressed images. However, a subjective comparison of these preprocessed images showed that the visual quality either dropped or remained unchanged, indicating the limited applicability of these metrics. We used a ResNet-like lightweight CNN architecture for preprocessing and the differentiable DiffJPEG algorithm for compression.

Subjective image quality measurement plays a crucial role in the advancement of image-processing technologies. The primary goal of a visual quality metric is to reflect subjective evaluation results. Despite the rapid development of these metrics, their potential limitations have not been sufficiently explored. This research paper aims to address this gap by demonstrating how image preprocessing before compression can artificially inflate the quality scores of widely used metrics such as DISTS, LPIPS, HaarPSI, VIF, STLPIPS, ADISTS, MR-Perceptual, AHIQ, IQT, and CONTRIQUE. We present several CNN-based preprocessing models that significantly increase these metrics when images are JPEG-compressed. However, a subjective assessment (with 1027 participants) of preprocessed images reveals that the visual quality either decreases or remains unchanged, thereby challenging the universal applicability of these metrics. The detection of metric attacks integrated into image processing systems has emerged as a significant issue. Attack detection can be achieved by comparing subjective evaluation results with metric results. If these results are anticorrelated, it suggests that the metric has been attacked. However, the time-consuming nature of subjective evaluations and the need for numerous participants make them impractical for routine attack detection. To address this, we propose using other metrics to determine whether the target metric has been attacked. Our results show that attacking one metric affects the output of other metrics, offering a potential method for attack detection.

This paper discusses the challenges of evaluating deblurring-methods quality and proposes a reduced-reference metric based on machine learning. Traditional quality-assessment metrics such as PSNR and SSIM are common for this task, but not only do they correlate poorly with subjective assessments, they also require ground-truth (GT) frames, which can be difficult to obtain in the case of deblurring. To develop and evaluate our metric, we created a new motion-blur dataset using a beam splitter. The setup captured various motion types using a static camera, as most scenes in existing datasets include blur due to camera motion. We also conducted two large subjective comparisons to aid in metric development. Our resulting metric requires no GT frames, and it correlates well with subjective human perception of blur.

Modern codecs offer numerous settings that can alter the encoding process. Multiobjective video encoding optimization has been studied in various works, but the whole encoder's option space was never considered. In this paper, we present a method for multiobjective encoding optimization in terms of relative video bitrate and encoding speed over the entire option space for a given encoder.

In recent years, display intensity and contrast have increased considerably. Many displays support high dynamic range (HDR) and 10-bit color depth. Since high bit-depth is an emerging technology, video content is still largely shot and transmitted with a bit depth of 8 bits or less per color component. Insufficient bit-depths produce distortions called false contours or banding, and they are visible on high contrast screens. To deal with such distortions, researchers have proposed algorithms for bit-depth enhancement (dequantization). Such techniques convert videos with low bit-depth (LBD) to videos with high bit-depth (HBD). The quality of converted LBD video, however, is usually lower than that of the original HBD video, and many consumers prefer to keep the original HBD versions. In this paper, we propose an algorithm to determine whether a video has undergone conversion before compression. This problem is complex; it involves detecting outcomes of different dequantization algorithms in the presence of compression that strongly affects the least-significant bits (LSBs) in the video frames. Our algorithm can detect bit-depth enhancement and demonstrates good generalization capability, as it is able to determine whether a video has undergone processing by dequantization algorithms absent from the training dataset.

Video-quality measurement is a critical task in video processing. Nowadays, many implementations of new encoding standards - such as AV1, VVC, and LCEVC - use deep-learning-based decoding algorithms with perceptual metrics that serve as optimization objectives. But investigations of the performance of modern video- and image-quality metrics commonly employ videos compressed using older standards, such as AVC. In this paper, we present a new benchmark for video-quality metrics that evaluates video compression. It is based on a new dataset consisting of about 2,500 streams encoded using different standards, including AVC, HEVC, AV1, VP9, and VVC. Subjective scores were collected using crowdsourced pairwise comparisons. The list of evaluated metrics includes recent ones based on machine learning and neural networks. The results demonstrate that new no-reference metrics exhibit high correlation with subjective quality and approach the capability of top full-reference metrics.

This paper presents a software-based method for estimating the power consumption of video decoders on various Android devices. Using this method, we developed an automatic system that consists of the VEQE Android application to measure the power consumption of video decoders and a server to collect the metrics. The system allowed us to create power-consumption and decoding-speed dataset for video decoders operating on 236 devices, representing 147 models. The paper presents several charts: the top 30 models and video decoders in terms of power efficiency for playback and for decoding only, as well as video-decoder ratings by power consumption and decoding speed for a given device model.

The critical constraint of mobile devices is a limited battery life that is significantly reduced during video playback. The power efficiency of video playback mainly depends on the used compression standard, video-decoder, and device model. We propose a software-based method to estimate the power consumption of video-decoders on various Android devices. Experiments on two devices of the same model show a small variation of the power playback consumption and a lack of dependence between the power consumption and the battery level. We have implemented an automatic system that includes the VEQE Android application to measure the power consumption of decoders and a server to collect the power metrics. Our system has collected power-consumption and decoding-speed dataset for video-decoders of six standards (AV1, HEVC, VP9, H.264, VP8, and MPEG-4) operating on 285 devices, representing 147 models. We demonstrate some slices of the created dataset: the top 30 models and video-decoders in terms of power efficiency for playback and for decoding only, as well as video-decoder ratings by power consumption and decoding speed for a given device model.

Video codecs have several dozen parameters that subtly affect the encoding rate, quality and size of the compressed video. Codec developers, as a rule, provide standard presets that on average yield acceptable performance for all videos, but for a given video, certain parameters may yield more efficient encoding. In this paper, we propose a new approach to predicting video codec presets to increase compression efficiency. Our effort involved collecting a new representative video-sequence dataset from Vimeo.com. An experimental evaluation showed relative bitrate decreases of 17.8% and 7.9%, respectively for the x264 and x265 codecs with standard options, all while maintaining quality and speed. Comparison with other methods revealed significantly faster automatic preset selection with a comparable improvement in results. Finally, our proposed content-adaptive method predicts presets that archive better performance than codec-developer presets from MSU Codec Comparison 2020 [1].

Modern video codecs have many compression-tuning parameters from which numerous configurations (presets) can be constructed. The large number of presets complicates the search for one that delivers optimal encoding time, quality, and compressed-video size. This paper presents a machine-learning-based method that helps to solve this problem. We applied the method to the x264 video codec: it searches for optimal presets that demonstrate 9-20% bitrate savings relative to standard x264 presets with comparable compressed-video quality and encoding time. Our method is faster upto 10 times than existing solutions.

Despite the higher video-completion quality that recently proposed methods have enabled for a wide variety of cases, their computational complexity remains a major concern. These methods typically frame video completion as an optimization problem over the whole spatiotemporal domain-a problem that is expensive to solve both in time and space. In this paper we propose a lighter-weight multipass video-completion pipeline that replaces global spatiotemporal optimization with simpler frame-by-frame motion reconstruction and refinement. We achieve a processing speed of 2.6 seconds per frame on Full HD content while delivering nearly state-of-the-art completion quality for a wide range of dynamic scenes captured using a free-moving camera. To validate the performance of our proposed method, we conducted a subjective comparison of different video-completion results for 26 test sequences from the DAVIS data set.

This research aims to sufficiently increase the quality of visual-attention modeling to enable practical applications. We found that automatic models are significantly worse at predicting attention than even single-observer eye tracking. We propose a semiautomatic approach that requires eye tracking of only one observer and is based on time consistency of the observer's attention. Our comparisons showed the high objective quality of our proposed approach relative to automatic methods and to the results of single-observer eye tracking with no postprocessing. We demonstrated the practical applicability of our proposed concept to the task of saliency-based video compression.

In this paper we present a novel framework for Edge-Directed Interpolation (EDI) of still images. The problem is treated as finding maximum a posteriori estimates of each interpolated pixel type and intensity value. The pixel type may be one of the pre-defined edge directions or "non-edge". Instead of the separate steps of edge orientation detection and intensity interpolation, maximizing the joint probability density function of type and intensity provides a better fit to the local image structure. Such a technique allows an effective discrimination between edges and non-edges (uniform areas and texture), thus leading to the suppression of artifacts which are common to existing EDI methods. Objective and subjective comparisons with conventional EDI methods corroborate the advantages of the proposed one. Moreover, the locality and the low computational complexity of the method make it suitable for a hardware implementation. 1

In this paper we propose a novel super-resolution algorithm based on motion compensation and edge-directed spatial interpolation succeeded by fusion via pixel classification. Two high-resolution images are constructed, the first by means of motion compensation and the second by means of edge-directed interpolation. The AdaBoost classifier is then used to fuse these images into an high-resolution frame. Experimental results show that the proposed method surpasses well-known resolution enhancement methods while maintaining moderate computational complexity.