Exceptional advancements in various computer vision tasks, such as identifying and categorizing objects, have been realized through the use of deep learning models, with a particular emphasis on convolutional neural networks (CNNs). Yet, while these models deliver outstanding results, they remain vulnerable to adversarial examples, thereby raising questions about their safety and dependability. In this paper, we investigate the influence of the image characteristics on the efficacy of adversarial patch attack against an image classifier model. We analyzed such characteristics in the frequency domain, where the frequencies indicate the periodicity and information density that contribute to the efficacy of adversarial patches. Our results showed that low-frequency components had significant contribution to the effectiveness of adversarial patch attacks.

This study investigates the cultural dependence of a facial expression recognition (FER) system in an interactive agent by analyzing the performance of several recognition models in different cultural domains. A comprehensive cross-domain classification performance assessment reveals disparities in model performance across different cultural contexts, indicating challenges in cross-cultural FER. To further investigate these characteristics, several public datasets across regions and our cross-cultural dataset of facial expressions derived from Thai and Japanese TV shows are analyzed. By evaluating the capacity of existing FER models to interpret our newly collected data, we found significant variations in emotion interpretation across these cultural contexts, highlighting the necessity for culturally inclusive algorithms. These findings underscore the critical need for more consideration of cultural diversity in FER research, marking a crucial step toward more inclusive and culturally sensitive artificial intelligence technologies.

In recent years, Generative Adversarial Networks (GANs) have gained attention in image synthesis mapping from the latent space onto image space. Trained latent space carries the visual semantics for generated images. Past studies observed that arithmetic operation and linear interpolation in latent space could change the visible facial attributes, such as beards and glasses, in image space. In this work, the visual concepts in the latent space are observed, allowing to change the emotion attribute per facial expressions in the image space. We observed interpolation of a sample while disentangling the emotional attributes to edit the emotion-related facial expressions in the synthesized images. For the experiment, the Deep Convolution Generative Adversarial Networks (DCGANs) are utilized for image synthesis, and Extended Cohn Kanade (CK +) facial expression dataset is applied as the input. Our results showed that manipulating the latent space of the well-trained GANs can edit the emotional aspects of the image space. Moreover, editing facial expressions in the latent space is helpful for the recognition task to improve accuracy. Empirical results showed that the facial expressions classifier improved its performance in the recognition sadness class from 20% to 80% on the imbalance dataset.

Age progression of face images has been an important tool to search for missing children. Many studies on age progression were recently conducted by conditional Generative Adversarial Networks (cGAN) based methods. However, these methods cannot estimate facial aging from a child’s face in the early childhood stage, which exhibits drastic facial shape changes over time. Thus, the problem of age progression from young children remains challenging in the field. In this study, we propose a cGAN based two-stage age progression model considering heritable facial features from parents and child to generate candidates for age-progressed face images from a young child’s face image.

Video prediction is an active machine learning problem to use past information in a video sequence to acquire human-like understanding and then predicting future consequences of object states and actions. The existing prediction frameworks integrated the decomposition and disentanglement techniques to observe object interaction and use them to predict future video scenes. However, the previous works did not consider physical interaction among objects in the prediction. Thus, this research utilizes the physical reasoning concept to represent object dynamics in the real world and estimate future sequences enclosing object dependencies. This paper addresses the investigation of object interaction using stochastic video prediction with physical reasoning representation. We propose a self-supervised framework called Relational Prediction Auto-Encoder (RPAE). Extensive experiments demonstrate that the proposed RPAE can effectively improve the generation and prediction of the near future sequences. We also confirmed the predicted object dynamics by measuring the velocity of each object and its physical interaction in the experiments.

Disasters can have a variety of adverse effects on the terrain and drive people into chaos. Evacuation drills and disaster simulations are standard measures to mitigate this confusion and reduce the cognitive bias that hinders evacuation decisions of the residents. Although various disaster simulation systems are available, localized disaster simulators that can capture the surrounding environment information and synthesize disaster scenes from the victims' perspective are not widely available. Therefore, we propose to simulate a disaster scene from a simple visual image taken from the local area. This study uses conditional Generative Adversarial Network (cGAN) to synthesize flooding images by training generator networks to understand the relationship between original image, semantic segmentation, and corresponding flooding images. By manipulating the segmentation image, it is possible to change the position and type of the objects in the scene to control the generation outputs, allowing users to expand flooding areas or increase flood levels. As a result, the proposed method can generate a consistent flood image containing visual features such as building reflections and waves that look genuine from human perception. Finally, the experiment has shown a promising trend to raise awareness of people in the disaster crisis through flood images generated by our proposed method.

Motion analysis automatically captures, recognizes and predicts ongoing human activities, which can be widely applied to various useful domains such as security surveillance in public spaces, including shopping centers and airports. With the development of the technologies like 3D specialized markers, we could capture the moving signals from marker joints and create a huge set of 3D motion capture (MOCAP) data. We propose in this work a method to automatically extract the action features which can be used for action recognition. We create an depth architecture model by combining multilevel networks which can focus on the recognizing objects in detail. These networks can learn the extracted features and perform action recognition. This propose model not only can extract the semantic action features from 3D MOCAP data, but also can apply for the real-time action recognition.

In this paper, the Gabor filter is studied and further expanded for temporal facial expression analysis. Originally, the Gabor feature describes both spatial and frequency characteristics of 2D images. The prominent of the theorem has been validated in research communities for a decade due to its similarity to the human perception system. The performance of the filter in the existing research gives convincing results on recognizing the human emotions by using a still image. However, the previous research neglects the fact that the understanding of human facial expression of emotions is associated by the dynamic relation, which the motion of expression must be witnessed. Therefore, we propose the novel temporal features by deriving the dynamic of Gabor features in the temporal template representations. Then, we decompose the features onto discriminative subspace for estimating the emotion class.

Emergence of novel techniques devices e.g., MS Kinect, enables reliable extraction of human skeletons from action videos. Taking skeleton data as inputs, we propose an approach to extract the discriminative patterns for efficient human action recognition. Each action is considered to consist of a sequence of unit actions, each of which is represented by a pattern. Given a skeleton sequence, we first automatically extract the key-frames, and then categorize them into different patterns. We further use a statistical metric to evaluate the discriminative capability of patterns, and define them as local features for action recognition. Experimental results show that the extracted local descriptors could provide very high accuracy in the action recognition, which demonstrate the efficiency of our method in extracting discriminative unit actions.

We often need to consider the influence on human perception when we develop an image processing algorithm or design parameters for image processing. The perception of color noise is also important for understanding the human vision system (HVS). Although we can use the CSF to express the property of achromatic noise appearance, it is not so easy to be applied to colors. While the CSF can be explained by a primary color vision, a color appearance is governed by higher order mechanisms. In this paper, we show the quantitative difference between achromatic and chromatic noise appearance. To reveal the perception of color noise quantitatively, we have conducted subjective experiments with modeled achromatic and chromatic noises using the 2AFC method. According to the results, it is shown that the ratio of luminance noise sensitivity to color noise sensitivity is 100-102depends on their spatial frequencies and background colors.

With the development of the technology like 3D specialized markers, we could capture the moving signals from marker joints and create a huge set of 3D action MoCap data. The more we understand the human action, the better we could apply it to applications like security, analysis of sports, game etc. In order to find the semantically representative features of human actions, we extract the sets of action characteristics which appear frequently in the database. We then propose an Apriori-like algorithm to automatically extract the common sets shared by different action classes. The extracted representative action characteristics are defined in the semantic level, so that it better describes the intrinsic differences between various actions. In our experiments, we show that the knowledge extracted by this method achieves high accuracy of over 80% in recognizing actions on both training and testing data.

The perception of color noise is important for image processing and for understanding the human vision mechanism. To reveal the perception of color noise quantitatively, we have conducted subjective experiments with modeled color noises. We use vector color noises varied in a CIELAB ab-plane to study the perception properties of color noise in detail. Our results show that the perception of color noise depends on background colors, luminance, spatial frequencies and noise models. The perception of vector color noises shows bipolar shape. The bipolar shapes of high spatial frequency noises rotate a little clockwise than the ones of low spatial frequency. Noises with low spatial frequencies are perceived easier than noises with high spatial frequencies. Using these experimental results, we make a perception model for color noise levels. We also propose a denoising workflow by applying the perception model to a random sampling filter.

Compared to conventional synthesis methods of free viewpoint images that use only multi-view images, our research synthesizes high quality free viewpoint images by using multi-view depth information as well as the images. By recovering high resolution and high precision 3D shapes from multi-view information, high quality free viewpoint images are synthesizable. Our research captures the scene by acquiring its multi-view depth appearance information, using a laser range finder. By performing camera parameter estimation, multi-view 3D shape integration and depth estimation, high quality free viewpoint images for any scene are synthesized. The performance of our method has been investigated by experimental results.

The perception of color noise in a scene depends on surrounding colors and luminance. Knowing the nature of our perception more precisely, we can develop better image processing technique. To understand the perception of color noise quantitatively, we have conducted subjective and quantitative experiments using a modified grayscale method with modeled additive color noise. We also investigated how color noise is perceived, depending on (a) background color, (b) the color noise vector and (c) luminance. These modeled noises are Gaussian noises that form both an isotropic and an anisotropic vector in the CIELAB's ab-plane. The results show that an eye's chromatic aberration has a large influence on color noise perception. The color noises are easy to see when the average color is blue or when the average luminance is about L*≈35. We validate our experimental results by adding equally perceived color noises to an image.

This paper describes two novel facial expression recognition methods which are robust for head rotation within a certain angle range between -30 degrees and +30 degrees. We had proposed Eigenspace Method based on Class features of object (EMC) and Multiple Discriminant Analysis (MDA) for facial expression recognition. Our new methods, pEMC (parametric Eigenspace Method based on Class Features) and pMDA (parametric Multiple Discriminant Analysis), are extensions of EMC and MDA by using the parametric eigenspace technique. The parametric technique finds the manifold vector for recognition of rotated objects. Since EMC and MDA have the higher class separation, our new methods have both characteristics of parametric eigenspace and high classification of facial expression. pEMC and pMDA provide more 20 degree correct recognition than EMC regardless of the head pose.

This paper describes the dependency of noise perception on background color and luminance of noise quantitatively. We conduct subjective and quantitative experiments for three noise models, using a modified grayscale method. The subjective experiment results show the perceived color noise depends on the background color, but the perceived luminance noise does not. The most sensitive background colors for color noises are yellow and purple. The perceived noises against background gray level show the similar trend between noise models. When the background gray level is L*~25, we perceive the noise best. In addition, the perceived chromatic noise level is about 8 times smaller than the calculated color noise using CIELAB Euclidean distance.

This paper describes a new method of facial expression recognition based on independent component analysis (ICA) and eigen-space method. We had proposed eigen-space method based on class-features (EMC), and EMC was the outstanding method with classification accuracy superior to multiple discriminant analysis (MDA). Our new method, GEMC, is a generalization of EMC by using ICA technique. GEMC has discriminated the facial expression class in a precision 10 or more points higher than conventional methods (EMC, MDA and ICA) because of classification experiments.

In the facial expression recognition by analyzing feature-vectors with linear transformation, an accuracy of recognition is depending on expression-classes. The accuracy falls remarkably when feature vectors of expression-classes are linearly nonseparable in a feature space. This paper describes a new method of facial expression analysis and recognition by using nonlinear transformation for separating each expression-classes. Our new method, namely KEMC, consists of the nonlinear transformation defined by kernel functions for transforming higher dimensional space and EMC (eigenspace method based on class features). This paper also shows experimental results of facial expression classification by KEMC.

Even if facial expression analysis from 2D luminance images is the present mainstream, it has problems due to changes in facial pose and lighting. In this paper, we use 3D range images which do not maintain such problems for facial expression analysis. We first apply the subspace method to range and luminance images, and clarify their differences in image characteristics. Examining the validity of range images for facial expression analysis, we consider improvement in correct classification rates by integrating results from range and luminance images. We employ the linear combination for their integration and show experimental results.

The paper shows a method for evaluating the beauty of brush-drawn "kanji" characters, using emotional response information. This methodology is based on a multi-variate analysis (see Kan, T., "Multivariate Statistic Analysis", Gendai-Sugakusha Publishing Company, 1999) of picture features and develops a numerical value for beauty elements. The accuracy of the evaluation of beauty is dependent upon the characteristics of the evaluation models. Character image features are used to compute the beauty elements. Evaluation models are obtained by multi-variate analysis of beauty element models. We then use regression methods to combine these evaluation models into a single number representative of the value of a given image. We also evaluate the characteristics of beauty evaluation models because good evaluation models are indispensable for more accurately estimating the subjective mean opinion score (MOS). This evaluation model is highly accurate and estimates the MOS well.

The accuracy of optical flow estimation is much worth on the occluded and appeared objects. In this paper, we describe an extrapolation method for improving the accuracy of optical flow estimation based on the characteristics of constraint lines in the velocity space and the extraction of the occluded/appeared regions using cluster analysis.

This paper shows an extraction method of a symmetric object for eyeglass face analysis using an active contour model. A contour of eyeglasses hinders a face analysis and synthesis which treats contours of facial parts or partial regions of the face such as eyes, mouth, cheek, eyelid and so on. Methods of active contours extract object contour well. We show an active contour model which is adapted to extract a contour of symmetric object, especially we focus on extracting eyeglass frame using snakes. We study to obtain parameters of snakes by genetic algorithm. We show the good results of applying the snakes in an actual facial image, and synthesize an expressive face with eyeglasses as an application.

For image analysis of facial expressions, we deal with information which are not only the gray levels of pixels but also positions of feature points. We first obtain each result of facial expression identification by using each information and then show that there is the difference between their results. Due to their difference, we integrate both information to improve the results. We define integrated similarity measures by linear combination or belief integration using virtual belief space and show the experimental results of facial expression identification using the measure.

For estimation of optical flow, voting has been used in the process of detection of an intersection of constraint line in voting space. The intersections are often scattered because of the quantization of voting space, image noise, etc. The authors first analyze the intersection distribution, then they show a filtering of voting to converge the scattering by convolution with a weight function. Some experimental results of the method are also given.

This paper shows a picture quality evaluation model for JPEG coded color images. This methodology is obtained by a multi-variable analysis of picture distortion and it provides a numerical value of picture quality. The accuracy of the picture quality evaluation is dependent upon characteristics of the picture quality evaluation models. The error components e(x,y) are computed by subtracting color components (RGB, CIE L*a*b*, L*u*v*, etc.) of distorted compressed images from the original ones. The evaluation model is obtained by multi-variable analysis of distortion models. We then use regression methods to combine these picture quality evaluation models into a single number representative of the quality of a given image. We also evaluate the characteristics of picture quality evaluation models, because good picture quality evaluation models are indispensable to estimate the subjective mean opinion score (MOS) with a high accuracy. This model evaluates picture quality with high accuracy and estimates MOS well.

In the field of astronomy, there is a need for a method to get three-dimensional information of 'corona' from an X-ray solar image. From the observation satellite 'Yohkoh', we can obtain images from different view angles by the rotation of the sun. However, in the case of changing textures on the sun's surface, stereo images lose the stereoscopic effect. To display accurate stereoscopic images, we propose a method for making a correct correspondence between stereo image pairs which includes varying surface using morphological processing and affine transformation.

We propose the new quality evaluation method of coded video which is considering the time transition of video quality. The evaluated quality value is computed as the weighted mean of the frame quality by using the weighted function which is derived from the short-term characteristic of human memory. In this case, the time transition of video quality is obtained from the series of the frame quality of whole frame. The optimal weighted function for each coded video is found by using the Genetic Algorithm. The obtained video quality has a good agreement with MOS.

In this paper, we solve the topological problem of isosurfaces generated by the marching cubes method using the approach of combinatorial topology. For each marching cube, we examine the connectivity of polyhedral configuration in the sense of combinatorial topology. For the cubes where the connectivities are not considered, we modify the polyhedral configurations with the connectivity and construct polyhedral isosurfaces with the correct topologies.

This paper presents two methods for the analysis of facial individuality and expression; an eigenspace method based on class features (EMC) and multiple discriminant analysis (MDA). Those methods are used since they derive eigenvectors by which we may extract facial individuality or expression information from a given facial image. The facial individuality and expression analysis can be achieved by projecting the facial image onto the subspace spanned by a set of those eigenvectors. We apply EMC and MDA to the classification of facial images into 50 classes of individuals or into seven classes of facial expressions, and verify their effectiveness with some experimental results.

For facial image analysis, facial parts such as eyes, nose, and mouth are generally focused and used. When these facial parts are hindered by additional objects (eyeglasses, beard, injury, etc.), the feature extraction from facial image will not be accurate. In this paper, we focus on the eyeglass faces because they account for 40% of population in Japan, and present a method of the removal of eyeglass frame in facial images with eyeglasses using principal component analysts. Two approaches are discussed for removal of eyeglasses in facial image. The first method calculates basis vectors from many eyeglassless facial images and one eyeglass facial image, and reconstructs the facial image with the basis vectors which include no feature of eyeglass frame. The second method calculates basis vectors from a set of eyeglassless facial images, and reconstructs the facial image using the values of inner product of the basis vectors and an eyeglass facial image. The former obtains the images which restrain. The features of eyeglass frame while loses the facial individuality a little. The latter obtains a natural eyeglassless facial image.

This paper describes a reflection and transparency model of rose petals for photorealistic computer graphics. Our model for each rose petal is based on its microscopic scale structures, such as the dome-shaped and translucent cells. The model has some parameters whose values are estimated from measuring of reflected light's intensities on rose petals. The values of those parameters are adjusted for the faithful simulation of the optical phenomena on rose petals. Finally, images of a rose produced by using our model are shown.

A new methodology for the determination of an objective metric for still image coding is reported. This methodology is applied to obtain a picture quality scale (PQS) for the coding of achromatic images over the full range of image quality defined by the subjective mean opinion score (MOS). This PQS takes into account the properties of visual perception for both global features and localized disturbances. The PQS closely approximates the MOS, with a correlation coefficient of more than 0.92, as compared to 0.57 obtained using the conventional weighted mean-square error (WMSE). Extensions and applications of the methodology and of the resulting metric are discussed.

This paper considers an objective picture quality scale for color images (PQScolor). PQScolor approximates the mean opinion score satisfactorily, it takes into account the color perception by using color difference and the properties of visual perception for global features and for localized disturbances. There are two type systems, PQScolor1 is based on Godlove's color difference and PQScolor2 is based on H, V, C signal difference. The correlation coefficient between PQScolor (1, 2) and MOS is more than 0.9, which is very high compared to the value 0.34 obtained for the mean color difference scale.

Psychophysical considerations show that the block-shaped distortion peculiar to orthogonal transform coding (OTC) is ten times more objectionable than random noise distortion. Minimizing the blockshaped distortion is considered here by analyzing the process of generation of coding errors, and the required characteristics of the orthogonal transform function (OTF) are clarified. Computer simulations substantiate its validity. Graphical illustration and measures of the block-shaped distortion are proposed and substantiated. With the aid of these measures, one can evaluate the performance of OTC or a new OTF, if proposed, without conducting visual assessment tests.