This paper introduces Motion Personalization, a new task that generates personalized motions aligned with text descriptions using several basic motions containing Persona. To support this novel task, we introduce a new large-scale motion dataset called PerMo (PersonaMotion), which captures the unique personas of multiple actors. We also propose a multi-modal finetuning method of a pretrained motion diffusion model called PersonaBooth. PersonaBooth addresses two main challenges: i) A significant distribution gap between the persona-focused PerMo dataset and the pretraining datasets, which lack persona-specific data, and ii) the difficulty of capturing a consistent persona from the motions vary in content (action type). To tackle the dataset distribution gap, we introduce a persona token to accept new persona features and perform multi-modal adaptation for both text and visuals during finetuning. To capture a consistent persona, we incorporate a contrastive learning technique to enhance intra-cohesion among samples with the same persona. Furthermore, we introduce a context-aware fusion mechanism to maximize the integration of persona cues from multiple input motions. PersonaBooth outperforms state-of-the-art motion style transfer methods, establishing a new benchmark for motion personalization.

We propose a novel 3D gaze redirection framework that leverages an explicit 3D eyeball structure. Existing gaze redirection methods are typically based on neural radiance fields, which employ implicit neural representations via volume rendering. Unlike these NeRF-based approaches, where the rotation and translation of 3D representations are not explicitly modeled, we introduce a dedicated 3D eyeball structure to represent the eyeballs with 3D Gaussian Splatting (3DGS). Our method generates photorealistic images that faithfully reproduce the desired gaze direction by explicitly rotating and translating the 3D eyeball structure. In addition, we propose an adaptive deformation module that enables the replication of subtle muscle movements around the eyes. Through experiments conducted on the ETH-XGaze dataset, we demonstrate that our framework is capable of generating diverse novel gaze images, achieving superior image quality and gaze estimation accuracy compared to previous state-of-the-art methods.

In this paper, we propose a gait type classification method using various sensors in a smart insole. The measured data are normalized to the unit step of the same length in order to reduce the variation of the speed according to the measurement point and the situation even within the same gait type. From the normalized data of the individual sensors, the discriminant features useful for gait type classification are extracted by using the Null-Space Linear Discriminant Analysis (NLDA), one of the representative discriminant analysis methods. As a result of experiments on the data measured for the seven gait types, our method gives a good performance of gait type classification.

We propose a new confidence measure to evaluate the eye detection results and combine two different eye detectors. The confidence for the results of eye detection is measured by the distances from the test sample and the positive samples, where the distance is calculated in the composite feature space. By using the proposed confidence measure, we construct a hybrid detector by combining two different detectors, which are complementary to each other. The experimental results show that the proposed detector provides more accurate eye detection results and consequently results in better face recognition rates compared to when using an individual eye detector.

We first infer a 3-D face model from a single frontal image using automatically extracted 2-D landmarks and deforming a generic 3-D model. Then, for any input image, we extract feature points and track them in 2-D. Given these correspondences, sometimes noisy and incorrect, we robustly estimate the 3-D head pose using PnP and a RANSAC process. As the head moves, we dynamically add new feature points to handle a large range of poses. When the tracker gets lost, due to motion blur or occlusions, the system re-initializes by matching feature points to the reference frontal image feature points. Our system runs in real-time (>;15Hz) on a standard CPU with a GPU card. We present results on stored video and will present a live demo, showing excellent tracking under large motion, fast movement, occlusion and facial expression variations. We also show comparative results with the ground truth BU head tracking dataset.

We propose a new biased discriminant analysis (BDA) using composite vectors for eye detection. A composite vector consists of several pixels inside a window on an image. The covariance of composite vectors is obtained from their inner product and can be considered as a generalization of the covariance of pixels. The proposed composite BDA (C-BDA) method is a BDA using the covariance of composite vectors. We construct a hybrid cascade detector for eye detection, using Haar-like features in the earlier stages and composite features obtained from C-BDA in the later stages. The proposed detector runs in real time; its execution time is 5.5 ms on a typical PC. The experimental results for the CMU PIE database and our own real-world data set show that the proposed detector provides robust performance to several kinds of variations such as facial pose, illumination, eyeglasses, and partial occlusion. On the whole, the detection rate per pair of eyes is 98.0% for the 3604 face images of the CMU PIE database and 95.1% for the 2331 face images of the real-world data set. In particular, it provides a 99.7% detection rate for the 2120 CMU PIE images without glasses. Face recognition performance is also investigated using the eye coordinates from the proposed detector. The recognition results for the real-world data set show that the proposed detector gives similar performance to the method using manually located eye coordinates, showing that the accuracy of the proposed eye detector is comparable with that of the ground-truth data.

Shadows that occur on face images due to illumination variation can change the appearance of a face and degrade face recognition performance. In this paper, we propose a new shadow compensation method based on the Fourier analysis for handling illumination variation. The face image deteriorated by shadow is restored by using the auxiliary magnitude, which compensates for the magnitude of the Fourier transform distorted due to the illumination variation. The experimental results for the CMU-PIE and Yale B databases show that the proposed method results in improved recognition performance under illumination variation.

We propose a pixel selection method in a face image based on discriminant features for face recognition. By analyzing the relationship between the pixels in a face image and features extracted from face images, pixels that contain a large amount of discriminative information are selected, while pixels with less discriminative information are discarded. The proposed method orders the pixels based on the discriminative information in face recognition, instead of selecting salient a priori regions. Comparative experiments are performed using the FERET, CMU-PIE and Yale B databases. The experimental results show that the pixel selection results in improved recognition performance, especially under illumination variation.

Illumination and pose variations that occur on face images degrade the performance of face recognition. In this paper, we propose a novel approach for handling illumination and pose variations for face recognition simultaneously. We use the two-dimensional view-based face recognition method and the shadow compensation method to deal with both variations. We construct a subspace for each pose and use the relationship between facial feature points to identify the poses. Since most human faces are similar in shape, we can find the shadow characteristics that the illumination variation makes on a face depending on the direction of light. By using these characteristics, we can compensate for illumination variation in face images. The proposed method is simple and requires much less computational effort than the other methods based on 3D models, and at the same time, provides a comparable recognition rate.