Learning-based garment prediction presents an appealing alternative to physics-based methods owing to its high efficiency. However, the predicted garments can exhibit noticeable penetrations into the body. Many collision-handling methods operate in the point domain, which has an inherent limitation in addressing the penetration issues with the mesh faces. In light of this, we propose a local Shape-Aware, Face-based collision handling approach (SAF) that can be applied to garment prediction networks to achieve real-time collision handling. Considering the nature of garments, we design a compact formulation to model the continuity of the garment surface, and utilize neural Signed Distance Fields (SDFs) to accomplish penetration resolution. Recognizing the significant impact of the body’s local shape on collision handling, we further propose using the angles between SDF gradients to characterize the sharpness of the body. Our approach can compensate for the inaccuracy of neural SDFs and preserve local smoothness and details. Extensive experiments demonstrate the outstanding performance and generalizability of our method.

Realistic garment simulation is critical for digital humans. However, noticeable penetrations still exist in current learning-based garment simulation techniques. To reduce penetrations in predicted garments, we resort to the garment geometry and neural Signed Distance Fields (SDFs) for effective collision handling. The key idea of our method is that we divide the garment into patches and model the local and global garment geometry through Intra- and Inter-Patch Correlations (IIPC), which can be easily learned through the powerful context-understanding ability of Transformers. The geometry information is then utilized to predict a per-vertex moving offset, according to which we move the penetrating vertices along the SDF’s gradient directions to solve collisions. Our module can be coupled with learning-based backbones to effectively solve penetrations while retaining real-time performance. Extensive experiments show that the proposed method excels the prior works significantly.

Reconstructing garments from monocular videos has attracted considerable attention as it provides a convenient and low-cost solution for clothing digitization. In reality, people wear clothing with countless variations and multiple layers. Existing studies attempt to extract garments from a single video. They either behave poorly in generalization due to reliance on limited clothing templates or struggle to handle the intersections of multi-layered clothing leading to the lack of physical plausibility. Besides, there are inevitable and undetectable overlaps for a single video that hinder researchers from modeling complete and intersection-free multi-layered clothing. To address the above limitations, in this paper, we propose a novel method to reconstruct multi-layered clothing from multiple monocular videos sequentially, which surpasses existing work in generalization and robustness against penetration. For each video, neural fields are employed to implicitly represent the clothed body, from which the meshes with frame-consistent structures are explicitly extracted. Next, we implement a template-free method for extracting a single garment by back-projecting the image segmentation labels of different frames onto these meshes. In this way, multiple garments can be obtained from these monocular videos and then aligned to form the whole outfit. However, intersection always occurs due to overlapping deformation in the real world and perceptual errors in monocular videos. To this end, we innovatively introduce a physics-aware module that combines neural fields with a position-based simulation framework to fine-tune the penetrating vertices of garments, ensuring robustly intersection-free. Additionally, we collect a mini dataset with fashionable garments to evaluate the quality of clothing reconstruction comprehensively. We release our code and data at https://github.com/SMY19999/IF-Garments.

Driver monitoring systems are becoming more common in modern cars, and they are crucial as autonomous vehicles depend on the driver’s continued attention. The increasing application of the deep learning techniques in in-car driver monitoring systems can be attributed to their success in estimating the human body position. In the 3-D human posture estimation, recent transformer-based methods have demonstrated remarkable effectiveness. However, as the number of joints increases, the computing cost to generate the joint-to-joint affinity matrix grows quadratically. To this end, this research develops a pretrained spatial-temporal transformer (PST-Transformer) model to facilitate the issue. In the pretrained phase, a masking module is used to randomly mask the joints. An autoencoder is employed to rebuild the distorted 2-D poses. During the training process, a temporal downsampling approach is advised to cut down on the duplicate data. To forecast the 3-D driving poses, an aggregator is paired with the fine tuned pretrained encoder. Prior to extracting 3-D spatial and temporal characteristics, the encoder in the PST-Transformer could learn the 2-D spatial-temporal relationships. To test the suggested approach, a new driving posture data set named human driving in vehicle (HDIV) is also created, which includes a variety of driving behaviors. Extensive experiments on the HDIV and the widely used Human3.6M data set show that our technique beats the state-of-the-art methods in terms of accuracy and computing complexity.