Security vulnerabilities that arise during the training phase of machine learning models often emerge progressively and stealthily, making it difficult to fully analyze their causes and evolution through single-point performance evaluation alone. Existing security analysis frameworks primarily focus on one-off attack execution and quantitative metric reporting, which limits their ability to systematically interpret cumulative state changes and vulnerability manifestation patterns throughout the training process. To address this limitation, this study proposes an autonomous vulnerability analysis agent driven by a Large Language Model. The proposed framework employs an iterative analysis loop that integrates execution logs, step-wise analytical memory, and externally retrieved knowledge to track state transitions and anomalous signs during training, and to interpret the underlying mechanisms and causal relationships of vulnerability emergence. Experimental results across diverse training-phase attack scenarios show that the proposed agent is not restricted to a specific attack type, but can identify the distinct performance degradation mechanisms induced by different threat models and consistently infer their causal relationships based on execution logs and retrieved knowledge. In addition, quantitative reliability evaluations indicate that the generated reports are generally consistent with actual observations and support evidence-grounded, coherent analysis. Ultimately, the proposed framework provides an explainable and automated analytical foundation for security vulnerability analysis in the training phase of AI systems.

In artificial intelligence (AI), many legal conflicts have arisen, especially concerning privacy and copyright associated with training data. When an AI model's training data incurs privacy concerns, it becomes imperative to develop a new model devoid of influences from such contentious data. However, retraining from scratch is often not viable due to the extensive data requirements and heavy computational costs. Machine unlearning presents a promising solution by enabling the selective erasure of specific knowledge from models. Despite its potential, many existing approaches in machine unlearning are based on scenarios that are either impractical or could lead to unintended degradation of model performance. We utilize the concept of weight prediction to approximate the less-learned weights based on observations about further training. By repetition of 1) finetuning on specific data and 2) weight prediction, our work gradually eliminates knowledge about the specific data. We verify its ability to eliminate side effects caused by problematic data and show its effectiveness across various architectures, datasets, and tasks.

Recent significant advancements in diffusion models have revolutionized image generation, enabling the synthesis of highly realistic images with text-based guidance. These breakthroughs have paved the way for constructing datasets via generative artificial intelligence (AI), offering immense potential for various applications. However, two critical challenges hinder the widespread adoption of synthesized data: computational cost and the generation of peculiar images. While computational costs have improved through various approaches, the issue of peculiar image generation remains relatively unexplored. Existing solutions rely on heuristics, extra training, or AI-based post-processing to mitigate this problem. In this paper, we present a novel approach to address both issues simultaneously. We establish that both gradient descent and diffusion sampling are specific cases of the generalized expectation maximization algorithm. We hypothesize and empirically demonstrate that peculiar image generation is akin to the local minima problem in optimization. Inspired by optimization techniques, we apply naive momentum and positive-negative momentum to diffusion sampling. Last, we propose new metrics to evaluate the peculiarity. Experimental results show momentum effectively prevents peculiar image generation without extra computation.

Recent complicated problems require large-scale datasets and complex model architectures, however, it is difficult to train such large networks due to high computational issues. Significant efforts have been made to make the training more efficient such as momentum, learning rate scheduling, weight regularization, and meta-learning. Based on our observations on 1) high correlation between past eights and future weights, 2) conditions for beneficial weight prediction, and 3) feasibility of weight prediction, we propose a more general framework by intermittently skipping a handful of epochs by periodically forecasting near future weights, i.e., a Weight Nowcaster Network (WNN). As an add-on module, WNN predicts the future weights to make the learning process faster regardless of tasks and architectures. Experimental results show that WNN can significantly save actual time cost for training with an additional marginal time to train WNN. We validate the generalization capability of WNN under various tasks, and demonstrate that it works well even for unseen tasks. The code and pre-trained model are available at https://github.com/jjh6297/WNN.

Facial expressions are one of the most essential channels to communicate a person’s emotional state. In social interaction, the capability to accurately read subtle changes in facial expressions, which reveal emotional fluctuations, is critical for 1) comprehending others’ emotions in context and background situations, 2) identifying responsiveness to others’ emotions, and 3) developing social skills in human–computer interaction. In this paper, we first introduce automatic emotion change detection via facial expression that discovers timings or temporal locations in a video where facial expression significantly changes. We propose a weakly-supervised deep emotion change detection framework that does not require facial expression videos with expensive temporal annotations and instead learns static images for training. Incorporating these ideas, we performed extensive experiments to demonstrate fundamental insights into emotion change detection and the efficacy of our framework using three video datasets, i.e., CASME II, MMI, and our YoutubeECD. Furthermore, we modified our framework for temporal spotting, which is the most similar task to emotion change detection, and showed comparable results with state-of-the-art methods on CAS(ME)2, proving justification for the problem. Even though we only employed the AffectNet to train our framework rather than the CASME II, MMI, YoutubeECD, and CAS(ME)2, experimental results demonstrate its exceptional generalization capability in cross-dataset environments.

The development of artificial intelligence has led to significant progress in activity detection; however, a proper evaluation of the activity detection performance is not guaranteed and is still lacking in real-life environments. In this study, to verify the stability and usefulness of activity detection in a real-world setting, we analyzed the activity detection performance for 40 elderly people using a human care robot.

Deep learning, based on which many modern algorithms operate, is well known to be data-hungry. In particular, the datasets appropriate for the intended application are difficult to obtain. To cope with this situation, we introduce a new dataset called ETRI-Activity3D, focusing on the daily activities of the elderly in robot-view. The major characteristics of the new dataset are as follows: 1) practical action categories that are selected from the close observation of the daily lives of the elderly; 2) realistic data collection, which reflects the robot's working environment and service situations; and 3) a large-scale dataset that overcomes the limitations of the current 3D activity analysis benchmark datasets. The proposed dataset contains 112,620 samples including RGB videos, depth maps, and skeleton sequences. During the data acquisition, 100 subjects were asked to perform 55 daily activities. Additionally, we propose a novel network called four-stream adaptive CNN (FSA-CNN). The proposed FSA-CNN has three main properties: robustness to spatio-temporal variations, input-adaptive activation function, and extension of the conventional two-stream approach. In the experiment section, we confirmed the superiority of the proposed FSA-CNN using NTU RGB+D and ETRI-Activity3D. Further, the domain difference between both groups of age was verified experimentally. Finally, the extension of FSA-CNN to deal with the multimodal data was investigated.

Facial expression recognition (FER) is increasingly gaining importance in various emerging affective computing applications. In practice, achieving accurate FER is challenging due to the large amount of inter-personal variations such as expression intensity variations. In this paper, we propose a new spatio-temporal feature representation learning for FER that is robust to expression intensity variations. The proposed method utilizes representative expression-states (e.g., onset, apex and offset of expressions) which can be specified in facial sequences regardless of the expression intensity. The characteristics of facial expressions are encoded in two parts in this paper. As the first part, spatial image characteristics of the representative expression-state frames are learned via a convolutional neural network. Five objective terms are proposed to improve the expression class separability of the spatial feature representation. In the second part, temporal characteristics of the spatial feature representation in the first part are learned with a long short-term memory of the facial expression. Comprehensive experiments have been conducted on a deliberate expression dataset (MMI) and a spontaneous micro-expression dataset (CASME II). Experimental results showed that the proposed method achieved higher recognition rates in both datasets compared to the state-of-the-art methods.

This paper presents a recurrent learning-based facial attribute recognition method that mimics human observers' visual fixation. The concentrated views of a human observer while focusing and exploring parts of a facial image over time are generated and fed into a recurrent network. The network makes a decision concerning facial attributes based on the features gleaned from the observer's visual fixations. Experiments on facial expression, gender, and age datasets show that applying visual fixation to recurrent networks improves recognition rates significantly. The proposed method not only outperforms state-of-the-art recognition methods based on static facial features, but also those based on dynamic facial features.

Facial expression recognition is increasingly gaining importance in emerging affective computing applications. In practice, achieving accurate facial expression recognition is still challenging due to environmental variations. In this paper, we propose a color channel-wise recurrent facial feature learning. The proposed method adopts recurrent neural network to learn expression features sequentially along color channels. The proposed network preserves discriminative expression feature through a long short-term memory for the sequence of color spatial features. Comprehensive experiments have been conducted on the publically available CMU Multi-PIE dataset under illumination variations. Experimental results showed that the proposed method achieved higher recognition rates compared to the state-of-the-art methods.

Images acquired by a camera show lens blur due to imperfection in the optical system even when images are properly focused. Lens blur is non-stationary in a sense that the amount of blur depends on pixel locations in a sensor. Lens blur is also asymmetric in a sense that the amount of blur is different in the radial and tangential directions, and also in the inward and outward radial directions. This paper presents parametric blur kernel models based on the normal sinh-arcsinh distribution function. The proposed models can provide flexible shapes of blur kernels with a different symmetry and skewness to model complicated lens blur due to optical aberration in a properly focused images accurately. Blur of single focal length lenses is estimated, and the accuracy of the models is compared with the existing parametric blur models. An advantage of the proposed models is demonstrated through deblurring experiments.