Facial action unit (AU) intensity plays a pivotal role in quantifying fine-grained expression behaviors, which is an effective condition for facial expression manipulation. How-ever, publicly available datasets containing intensity annotations for multiple AUs remain severely limited, often featuring a restricted number of subjects. This limitation places challenges to the AU intensity manipulation in images due to disentanglement issues, leading researchers to resort to other large datasets with pretrained AU intensity estimators for pseudo labels. In addressing this constraint and fully leveraging manual annotations of AU intensities for precise manipulation, we introduce AUEditNet. Our proposed model achieves impressive intensity manipulation across 12 AUs, trained effectively with only 18 subjects. Utilizing a dual-branch architecture, our approach achieves comprehensive disentanglement of facial attributes and identity without necessitating additional loss functions or implementing with large batch sizes. This approach offers a potential solution to achieve desired facial attribute editing despite the dataset's limited subject count. Our experiments demonstrate AUEdit-Net's superior accuracy in editing AU intensities, affirming its capability in disentangling facial attributes and identity within a limited subject pool. AUEditNet allows conditioning by either intensity values or target images, eliminating the need for constructing AU combinations for specific facial expression synthesis. Moreover, AU intensity estimation, as a downstream task, validates the consistency between real and edited images, confirming the effectiveness of our proposed AU intensity manipulation method.

Learning-based gaze estimation methods require large amounts of training data with accurate gaze annotations. Facing such demanding requirements of gaze data collection and annotation, several image synthesis methods were proposed, which successfully redirected gaze directions pre-cisely given the assigned conditions. However, these methods focused on changing gaze directions of the images that only include eyes or restricted ranges of faces with low res-olution (less than$128\times 128$) to largely reduce interference from other attributes such as hairs, which limits application scenarios. To cope with this limitation, we proposed a portable network, called ReDirTrans, achieving latent-to-latent translation for redirecting gaze directions and head orientations in an interpretable manner. ReDirTrans projects input latent vectors into aimed-attribute embed-dings only and redirects these embeddings with assigned pitch and yaw values. Then both the initial and edited embeddings are projected back (deprojected) to the initial latent space as residuals to modify the input latent vec-tors by subtraction and addition, representing old status re-moval and new status addition. The projection of aimed at-tributes only and subtraction-addition operations for status replacement essentially mitigate impacts on other attributes and the distribution of latent vectors. Thus, by combining ReDirTrans with a pretrained fixed e4e-StyleGAN pair, we created ReDirTrans-GAN, which enables accurately redi-recting gaze in full-face images with$1024\times 1024$resolution while preserving other attributes such as identity, expres-sion, and hairstyle. Furthermore, we presented improvements for the downstream learning-based gaze estimation task, using redirected samples as dataset augmentation.

CNNs utilize lowpass frequency features, evidenced by max pooling operations that preserve dominant features. Addressing this, we used a one-layer FCN using both coarse and detailed components via DWT. We then developed UwU, a learnable wavelet-based unit that integrates the PR constraint relaxation, allowing feature map component fine-tuning. Distinctively, UwU’s coefficients are trainable, unlike prior studies. This is the first work that utilizes PR constraint relaxation to enhance CNNs. Our innovative techniques serve to enhance stride-convolution, pooling, and downsampling units in CNNs. We tested these improved units using the ResNet family architectures against traditional frequency and wavelet-based units. Performance metrics from CIFAR10, ImageNet1K, and the DTD show promising results. Particularly, while CIFAR10 results are on par with other methods, there’s a marked performance improvement on ImageNet1K and DTD. Further, integrating UwU into the ResNet18-based encoder of the CFLOW-AD system yields competitive anomaly detection results, particularly for hazelnut images in the MVTecAD dataset.

Center of pressure (COP) is an important measurement of postural and gait control in human biomechanical studies. A vision-based estimation of COP metrics offers a way to obtain these gold-standard metrics for the detection of balance and gait problems. In this paper, we propose an end-to-end framework to estimate the COP path length and the COP positions from the 3D skeleton, utilizing the spatial-temporal features learned by graph convolutional networks. We propose two single-task models for each metric and a multi-task approach jointly learning two metrics. To facilitate this line of research, we also release a novel 3D skeleton dataset containing a wide variety of action patterns with synchronized COP labels. The experiments on the dataset validate that our framework achieves state-of-the-art accuracies for both COP path length and COP position estimations, while the multitask approach could yield more accurate and robust performance on COP path length estimation compared to the single-task model.

Local image feature matching lies in the heart of many computer vision applications. Achieving high matching accuracy is challenging when significant geometric difference exists between the source and target images. The traditional matching pipeline addresses the geometric difference by introducing the concept of support region. Around each feature point, the support region defines a neighboring area characterized by estimated attributes like scale, orientation, affine shape, etc. To correctly assign support region is not an easy job, especially when each feature is processed individually. In this paper, we propose to estimate the relative affine transformation for every pair of to-be-compared features. This "tailored" measurement of geometric difference is more precise and helps improve the matching accuracy. Our pipeline can be incorporated into most existing 2D local image feature detectors and descriptors. We comprehensively evaluate its performance with various experiments on a diversified selection of benchmark datasets. The results show that the majority of tested detectors/descriptors gain additional matching accuracy with proposed pipeline.