3D face alignment from monocular images in challenging scenarios such as large poses and occlusions presents a huge challenge. To overcome this challenge, we propose a Multi-granularity Perturbation Relearning Network (MPRN), utilizing relearning attention to capture crucial features. Specifically, MPRN employs an attention mechanism to highlight effective features and further conducts relearning for attention to refine its accuracy. However, in extreme scenarios, the loss of key 3D facial information hampers the effective functioning of relearning attention. To this end, we construct multi-granularity perturbation graphs to infer the missing key 3D facial information and correspondingly guide the multiple times learning of attention module using perturbation graphs at various granularities. By doing this, our MPRN could effectively capture crucial 3D facial features in extreme scenarios, thereby achieving precise 3D face alignment. Experiments on the AFLW 2000-3D and AFLW datasets demonstrate the effectiveness of our MPRN.

Abstract This paper proposes a disentangled representation transformer network (DRTN) for 3D dense face alignment and reconstruction. Unlike traditional 3DMM-based approaches, the target parameters, such as shape, expression, and pose, are individually estimated without considering their direct influences on one another and then jointly optimized. Hence, DRTN aims to enhance the representation of facial attributes in a semantic sense by learning the correlation of different 3D facial attribute parameters. To achieve this, we present a novel strategy to design disentangled 3D face attribute representation, which decomposes the given facial attributes into identity, expression, and poses. Specifically, the 3D face parameter estimation in the regression network depends on the correlation of other face attribute parameters rather than being independent. The branching of the identity component aims to reinforce learning the expression and pose attributes by preserving the overall face geometry structure and identity. Accordingly, the expression and pose parts of the branch preserve the consistency of expression and pose attributes, respectively. Moreover, DRTN helps refine the reconstruction and alignment of facial details in large poses, mainly by coupling other facial attribute parameters. Extensive qualitative and quantitative experimental results on widely evaluated benchmarking datasets demonstrate that our approach achieves competitive performance compared to state-of-the-art methods.

3D face reconstruction from large pose images is a long-standing and challenging problem. Existing methods based on 3DMM parameter regression with strong constraints only consider reducing the error between 68 landmarks and their ground truth, which may lead to insufficient learning about the landmark. In this paper, we propose a deformable feature interaction network (DFIN) for 3D dense alignment and face reconstruction. We design shallow and deep modules to extract semantic information at different levels, perform feature interaction, and improve the ability of the network to perform local detail feature extraction. We use deformable convolution properties to increase the convolution kernel's acceptance domain to learn the effective face area and the weights of the facial feature distribution, and the output features are enriched. Moreover, we propose a graph structure reasoning method by introducing a facial geometric consistency loss. Specifically, we selected the leftmost and rightmost cheeks, the chin, the tip of the nose, and the landmarks of the eyes. We triangulate the landmarks of the chin, eyes, and cheeks and calculate the geometric relationship between the center of gravity of the triangle and the tip of the nose to constrain the entire face. Extensive experiments on AFLW2000-3D and AFLW datasets demonstrate the validity of our method. Source codes are available at https://github.com/wricked520/DFIN.

In this paper, we propose a multi-granularity feature interaction and relation reasoning network (MFIRRN) which can recover a detail-rich 3D face and perform more accurate dense alignment in an unconstrained environment. Traditional 3DMM-based methods directly regress parameters, resulting in the lack of fine-grained details in the reconstruction 3D face. To this end, we use different branches to capture discriminative features at different granularities, especially local features at medium and fine granularities. Meanwhile, the finer-grained branch network shares its information with the adjacent coarser-grained branch network to achieve feature interaction. Our model performs cross-granular information integration and inter-granular relationship reasoning to obtain prediction results. Extensive experiments on AFLW2000-3D and AFLW datasets demonstrate the validity of our method. The code is publicly available at https://github.com/leilimaster/MFIRRN.

In this paper, we propose a multi-attribute regression network (MARN) to investigate the problem of face reconstruction, especially in challenging cases when faces undergo large variations including severe poses, extreme expressions, and partial occlusions in unconstrained environments. The traditional 3DMM parametric regression method does not distinguish the learning of identity, expression, and attitude attributes, resulting in lacking geometric details in the reconstructed face. We propose to learn a face multi-attribute features during 3D face reconstruction from single 2D images. Our MARN enables the network to better extract the feature information of face identity, expression, and pose attributes. We introduce three loss functions to constrain the above three face attributes respectively. At the same time, we carefully design the geometric contour constraint loss function, using the constraints of sparse 2D face landmarks to improve the reconstructed geometric contour information. The experimental results show that our MARN has achieved significant improvements in 3D face reconstruction and face alignment on the AFLW2000-3D and AFLW datasets.