While adversarial perturbation of images to attack deep image classification models pose serious security concerns in practice, this paper suggests a novel paradigm where the concept of image perturbation can benefit classification performance, which we call amicable aid. We show that by taking the opposite search direction of perturbation, an image can be modified to yield higher classification confidence and even a misclassified image can be made correctly classified. This can be also achieved with a large amount of perturbation by which the image is made unrecognizable by human eyes. The mechanism of the amicable aid is explained in the viewpoint of the underlying natural image manifold. Furthermore, we investigate the universal amicable aid, i.e., a fixed perturbation can be applied to multiple images to improve their classification results. While it is challenging to find such perturbations, we show that making the decision boundary as perpendicular to the image manifold as possible via training with modified data is effective to obtain a model for which universal amicable perturbations are more easily found.

Recently, the vulnerability of deep image classification models to adversarial attacks has been investigated. However, such an issue has not been thoroughly studied for image-to-image tasks that take an input image and generate an output image (e.g., colorization, denoising, deblurring, etc.) This paper presents comprehensive investigations into the vulnerability of deep image-to-image models to adversarial attacks. For five popular image-to-image tasks, 16 deep models are analyzed from various standpoints such as output quality degradation due to attacks, transferability of adversarial examples across different tasks, and characteristics of perturbations. We show that unlike image classification tasks, the performance degradation on image-to-image tasks largely differs depending on various factors, e.g., attack methods and task objectives. In addition, we analyze the effectiveness of conventional defense methods used for classification models in improving the robustness of the image-to-image models.

This paper investigates the robustness of deep image super-resolution models using normalizing flow against adversarial attacks. Attack methods specific to flow-based super-resolution models are formulated, and the performance and influences of the attacks are analyzed. We show that flow-based super-resolution models are highly vulnerable to attacks, which are even more serious than other super-resolution models. Potential remedies to the vulnerability are also evaluated.

The video-based action recognition task has been extensively studied in recent years. In this paper, we study the structural vulnerability of deep learning-based action recognition models against the adversarial attack using the one frame attack that adds an inconspicuous perturbation to only a single frame of a given video clip. Our analysis shows that the models are highly vulnerable against the one frame attack due to their structural properties. Experiments demonstrate high fooling rates and inconspicuous characteristics of the attack. Furthermore, we show that strong universal one frame perturbations can be obtained under various scenarios. Our work raises the serious issue of adversarial vulnerability of the state-of-the-art action recognition models in various perspectives.

While deep learning-based single image super-resolution has progressed significantly, super-resolution of images containing blurring artifacts is still challenging. This paper proposes a unified model that can simultaneously perform deblurring and super-resolution for a given image, which is called Edge Attention Network (EAN). Our model employs an attention mechanism using the edge information in order to enhance the sharpness of the super-resolved output image. Experimental results demonstrate that our method outperforms existing super-resolution methods and separate application of deblurring and super-resolution.

Deep learning-based image processing algorithms, including image super-resolution methods, have been proposed with significant improvement in performance in recent years. However, their implementations and evaluations are dispersed in terms of various deep learning frameworks and various evaluation criteria. In this paper, we propose an integrated repository for the super-resolution tasks, named SRZoo, to provide state-of-the-art super-resolution models in a single place. Our repository offers not only converted versions of existing pre-trained models, but also documentation and toolkits for converting other models. In addition, SRZoo provides platform-agnostic image reconstruction tools to obtain super-resolved images and evaluate the performance in place. It also brings the opportunity of extension to advanced image-based researches and other image processing models. The software, documentation, and pre-trained models are publicly available on GitHub1.

Recently, deep learning-based lossy image compression methods have been proposed. However, their efficiency in terms of storage and computational costs has not been addressed adequately. In this paper, we propose efficient lossy image compression methods based on asymmetric autoencoder and decoder pruning. Experimental results demonstrate the effectiveness of our methods.

Successive image compression refers to the process of repeated encoding and decoding of an image. It frequently occurs during sharing, manipulation, and re-distribution of images. While deep learning-based methods have made significant progress for single-step compression, thorough analysis of their performance under successive compression has not been conducted. In this paper, we conduct comprehensive analysis of successive deep image compression. First, we introduce a new observation, instability of successive deep image compression, which is not observed in JPEG, and discuss causes of the instability. Then, we conduct a successive image compression benchmark for the state-of-the-art deep learning-based methods, and analyze the factors that affect the instability in a comparative manner. Finally, we propose a new loss function for training deep compression models, called feature identity loss, to mitigate the instability of successive deep image compression.

Automatic prediction of the aesthetic quality of a photograph has been an important research problem in image processing and computer vision. While assessing the aesthetic quality of a photograph by human is highly subjective, most existing studies have considered only objective (or general) opinion of multiple viewers. In this paper, we provide a comprehensive investigation of the issue of subjectivity in aesthetic quality assessment using a large-scale database containing photos, user ratings, and user comments. First, we analyze how the mean aesthetic quality level and the level of subjectivity have evolved over time. Second, we examine the feasibility of automatic prediction of the level of subjectivity based on visual features, and identify which features are effective for the prediction. Third, we analyze the users' comments given to photos to understand the sources of subjectivity of aesthetic quality rating. Our results show that several factors are simultaneously involved in determining the level of subjectivity of a photo, but it can be predicted with reasonable accuracy. We believe that our results provide insight toward personalized aesthetic photo applications.

Single-image super-resolution aims to generate a high-resolution version of a low-resolution image, which serves as an essential component in many image processing applications. This paper investigates the robustness of deep learning-based super-resolution methods against adversarial attacks, which can significantly deteriorate the super-resolved images without noticeable distortion in the attacked low-resolution images. It is demonstrated that state-of-the-art deep super-resolution methods are highly vulnerable to adversarial attacks. Different levels of robustness of different methods are analyzed theoretically and experimentally. We also present analysis on transferability of attacks, and feasibility of targeted attacks and universal attacks.

This paper presents a study that evaluates the performance of multi-view human activity recognition with videos having degraded quality. For the activity recognition models, a support vector machine-based approach using spatiotemporal features and a deep learning-based approach using convolutional and recurrent layers are built. We investigate the recognition performance of the two models with respect to the bitrate of the compressed videos and the peak signal-to-noise ratio of the videos corrupted by additive Gaussian random noise. We analyze the robustness of the models for the degraded videos.

In these days, a large number of videos is taken by various kinds of handheld devices, but many of them have poor aesthetic quality. In this paper, we present an automated video editing system that uses the shot length, camera motion, and color distribution as key aesthetic features. Given an amateur video, our system computes the original unrefined camera motion as homography and tries to remove some unreliable frames, which consequently splits the video into several shots. It then applies enhancement processes, including reconstruction of the overall camera motions and harmonization of color distributions. We apply our method to some amateur videos and evaluate the results through a subjective test. It is demonstrated that reducing the shot length in our method is a key point of editing that can lead enhanced satisfaction by viewers for the edited videos.

While most of the existing work in aesthetic image quality assessment focuses on the overall (or average) opinion of users, this paper raises the issue of subjectivity (or taste) of aesthetic quality. We argue that subjectivity differs among different images, and investigate what causes such difference. We first analyze statistics of the user ratings of photos in a photo contest website, DPChallenge, in the viewpoint of average and standard deviation values of the ratings. Then, more importantly, we analyze the users' comments in order to identify sources contributing to subjectivity. When considering the importance of personalization in photo applications, we believe that our findings will be a valuable first step in the relevant future research.