We present HuMoCon, a novel motion-video understanding framework designed for advanced human behavior analysis. The core of our method is a human motion concept discovery framework that efficiently trains multi-modal encoders to extract semantically meaningful and generalizable features. HuMoCon addresses key challenges in motion concept discovery for understanding and reasoning, including the lack of explicit multi-modality feature alignment and the loss of high-frequency information in masked autoencoding frameworks. Our approach integrates a feature alignment strategy that leverages video for contextual understanding and motion for fine-grained interaction modeling, further with a velocity reconstruction mechanism to enhance high-frequency feature expression and mitigate temporal over-smoothing. Comprehensive experiments on standard benchmarks demonstrate that HuMoCon enables effective motion concept discovery and significantly outperforms state-of-the-art methods in training large models for human motion understanding. We will open-source the associated code with our paper.

Laser powder bed fusion (LPBF) is an additive manufacturing process capable of producing intricate structures with high accuracy. Despite this capability, it struggles to achieve the required reliability for mass production—specifically the stability of a production run and repeatability across multiple runs. Parameter optimization, which adjusts process parameters to regulate a specific quantity of interest (QoI), is a crucial means of quality control. Existing methods, however, have not adequately addressed both random and systematic factors in the LPBF process. The stochastic nature of the process is often neglected under the assumption that identical parameter inputs will consistently yield the same QoI. This deviates from reality and is not intended to reduce potential variations in the QoI. Moreover, many studies do not incorporate the systematic neighboring effects between scan tracks into their optimization, so process reliability cannot be guaranteed. To address this issue, this study focuses on optimizing the probability distribution of the QoI. The key idea is not only to increase the likelihood of achieving the ideal QoI but also to reduce its variance. This is achieved by uncertainty-aware modeling and optimization of the LPBF process using machine learning. Specifically, the problem is formulated as maximizing the posterior distribution of scan parameters given an ideal QoI sequence and historical manufacturing data, yielding a large-scale constrained optimization problem. A stochastic, distributed, gradient-based method is proposed to solve this problem, where a coarse-to-fine strategy plays a critical role in accelerating convergence. A case study is then conducted to stabilize the melt pool volume by optimizing laser powers. The solutions are verified in a calibrated finite element-based simulation environment, in which the variations of the melt pool volume are effectively reduced both within a single run and across multiple runs. The implementation of our method is available at https://github.com/qihangGH/uncertainty_aware_param_optim_for_AM.

The widespread adoption of laser powder bed fusion (LPBF) additive manufacturing is hampered by process unreliability problems. Modeling the melt pool behavior in LPBF is crucial to develop process control methods. While data-driven models linking melt pool dynamics to specific process parameters have shown appreciable advancements, existing models often oversimplify these relationships as deterministic, failing to account for the inherent instability of LPBF processes. Such simplifications can lead to overconfident and unreliable predictions, potentially resulting in erroneous process decisions. To address this critical issue, we propose a probabilistic data-driven approach to melt pool modeling that incorporates process noise and uncertainty. Our framework formulates a problem that includes distribution approximation and uncertainty quantification. Specifically, the Gaussian distribution with higher order priors, aided with variational inference and importance sampling, is used to approximate the probability distribution of melt pool characteristics. The uncertainty inherent in both LPBF process data and the modeling approach itself are then decomposed and approximated by using Monte Carlo sampling. The melt pool model is improved further by using a novel grid-based representation for the neighborhood of a fusion point, and a neural network architecture designed for effective feature fusion. This approach not only refines the accuracy of the model but also quantifies the uncertainty of the predictions, thereby enabling more informed decision-making with reduced risk. Two potential applications, including LPBF process planning and anomaly detection, are discussed. The implementation of our model is available athttps://github.com/qihangGH/probabilistic_melt_pool_model. Note to Practitioners—Modeling the melt pool behavior in laser powder bed fusion (LPBF) processes is pivotal for enhancing its quality control. However, a problem is that most existing data-driven melt pool models learn melt pool behavior with a deterministic function, which predicts the same outputs if its inputs are the same. This deviates from the reality and neglects the uncertainty in LPBF processes. As a consequence, the quality control methods based on such melt pool models lack required reliability. In response to these challenges, this work proposes to model melt pool behavior by using probability distributions with deep learning techniques, which can quantify the uncertainty in both LPBF process data and data-driven models. Aided with an elegantly designed representation for the neighborhood of a fusion point as model input, and a neural network architecture that fuses multi-modal data, the proposed model achieves accurate melt pool size prediction results. More importantly, this work quantifies and decomposes the prediction uncertainty. By accounting for noise and parameter variations, the probabilistic modeling models developed herein offer a more robust foundation for LPBF quality control than the existing ones. They can be readily applied by practitioners to perform improved process planning, defect prognosis, and real-time anomaly detection tasks.

A radiance field is an effective representation of 3D scenes, which has been widely adopted in novel-view synthesis and 3D reconstruction. It is still an open and challenging problem to evaluate the geometry, i.e., the density field, as the ground-truth is almost impossible to obtain. One alternative indirect solution is to transform the density field into a point-cloud and compute its Chamfer Distance with the scanned ground-truth. However, many widely-used datasets have no point-cloud ground-truth since the scanning process along with the equipment is expensive and complicated. To this end, we propose a novel metric, named Inverse Mean Residual Color (IMRC), which can evaluate the geometry only with the observation images. Our key insight is that the better the geometry, the lower-frequency the computed color field. From this insight, given a reconstructed density field and observation images, we design a closed-form method to approximate the color field with low-frequency spherical harmonics, and compute the inverse mean residual color. Then the higher the IMRC, the better the geometry. Qualitative and quantitative experimental results verify the effectiveness of our proposed IMRC metric. We also benchmark several state-of-the-art methods using IMRC to promote future related research. Our code is available at https://github.com/qihangGH/IMRC.

Recent advancements in models linking natural language with human motions have shown significant promise in motion generation and editing based on instructional text. Motivated by applications in sports coaching and motor skill learning, we investigate the inverse problem: generating corrective instructional text, leveraging motion editing and generation models. We introduce a novel approach that, given a user’s current motion (source) and the desired motion (target), generates text instructions to guide the user towards achieving the target motion. We leverage large language models to generate corrective texts and utilize existing motion generation and editing frameworks to compile datasets of triplets (source motion, target motion, and corrective text). Using this data, we propose a new motion-language model for generating corrective instructions. We present both qualitative and quantitative results across a diverse range of applications that largely improve upon baselines. Our approach demonstrates its effectiveness in instructional scenarios, offering text-based guidance to correct and enhance user performance.

Additive manufacturing (AM) can build up complex parts in a layer-by-layer manner, which is a kind of novel and flexible production technology. The special manufacturing capability of AM shows great application potential in various fields. However, an open-loop control method cannot guarantee the reliability and repeatability of an AM process. Defects often occur to deteriorate product quality and lead to material and time waste, which hinders the development of AM industry. In this regard, a lot of efforts have been made to make an AM process more controllable. This work proposes an AM control framework that divides the related studies into three feedback loops, including the in-situ monitoring of process defects, fault diagnosis of 3-D printers, and closed-loop control of an AM process. These three loops constitute the inspection and control of AM from the machine level to product level. Specifically, the measurement requirements for monitoring techniques, defect detection, fault diagnosis, and closed-loop control are summarized. The challenges and future trends in realizing a more reliable and repeatable AM process are discussed. Note to Practitioners—This survey is motivated by urgent need to solve product quality problems in additive manufacturing (AM) caused by open-loop control. Three feedback loops can be established to solve them. The first one is defect detection that inspects part quality during fabrication. The second one is the fault diagnosis of a 3-D printer that monitors the health and operation conditions of its actuators. The last one is closed-loop control that improves AM process reliability and repeatability by regulating process variables in real time. These three loops are all based on the feedback signals of in-situ monitoring systems. This paper reviews the related studies and provides guidance for establishing the monitoring systems, performing defect detection and fault diagnosis, and designing closed-loop control systems, which helps realize more reliable and repeatable AM.

Multiview 3-D reconstruction holds considerable promise across a wide applications in social manufacturing. Conducting in-depth research on precise and robust multiview 3-D reconstruction holds the potential to significantly empower the domain of social manufacturing. Recently, there has been a burgeoning interest in the domain of neural implicit surfaces learning through volume rendering for the purpose of multiview reconstruction without 3-D supervision. Conventional approaches often overlook explicit multiview geometry constraints, resulting in shortcomings in generating consistent surface reconstructions and recovering fine details. To solve this, we propose geometry-guided neural implicit surface (GG-NeuS), a geometry-guided neural implicit surfaces learning method for multiview surface reconstruction. Our model places a stronger emphasis on maintaining geometry consistency, significantly enhancing the quality of reconstruction. First, we enforce multiview geometry constraints on the surface points by locating the zero-level set of signed distance function (SDF). Second, we incorporate normal cues, predicted by general-purpose monocular estimators, to substantially recover fine geometric details. Additionally, we introduce a voxel-based surface reconstruction methodology that strikes an optimal balance between training time and reconstruction quality. Through comprehensive qualitative and quantitative experiments and analyses, we demonstrate thatGG-NeuSsuccessfully reconstructs fine-grained surface details and achieves superior surface reconstruction quality than state-of-the-art approaches.

Digital light processing (DLP) is a popular additive manufacturing technology that uses light irradiation to fabricate 3-D devices via a projector to achieve laser-sensitive resin curing. However, the performance and reliability of DLP can be affected by internal defects such as printing errors and the accumulation of residual stress. Existing defect detection methods rely on monitoring the printed parts, which leads to resource wastage and struggles to effectively handle imbalanced defect data. In this article, we propose a defect detection method called dual neural network, which involves detecting defects in materials before the printing process to prevent resource wastage and serious consequences. Specifically, to handle the highly imbalanced class distribution problem in online DLP defect detection, dual neural network utilizes a domain learner and balance learner to effectively balance the information of the minority class and learn the generalization knowledge from the imbalanced defect dataset. Experimental results demonstrate the effectiveness of our proposed method, which has also been applied to real-world production equipment successfully.

The quality of the dataset is critical to the performance of neural networks for error prediction in 3-D printing. In order to enlarge the dataset, we propose a customized two-stage framework, cascaded cross-modality generative adversarial networks (CCMGANs), for generating dental crown meshes in an unsupervised manner. At the first stage, a displacement map-guided generative adversarial network (GAN) is used to generate coarse meshes with diverse shapes. At the second stage, fine-grained details are added to the coarse meshes using an image-based GAN. Unlike previous work that integrates a differentiable renderer into the mesh deformation process directly, we adopt a two-step strategy. First, we use a depth image refinement module to achieve the domain transformation from the rendered depth images of the generated meshes to those of the real ones. Then, we propose a mesh refinement module to optimize the coarse meshes in an image-supervised manner. To alleviate the self-intersection problem, we propose a loss to penalize the distances of point pairs in self-intersection regions. Experimental results show that our method is able to generate highly realistic meshes and outperforms the state-of-the-art point cloud generation method TreeGCN in terms of the metrics FDD, MMD-CD, MMD-EMD, and COV-EMD. Furthermore, we utilize the generated data to augment the original dataset, and demonstrate that the generated data can effectively improve the accuracy of the error prediction task in 3-D printing.

A neural radiance field (NeRF) enables the synthesis of cutting-edge realistic novel view images of a 3D scene. It includes density and color fields to model the shape and radiance of a scene, respectively. Supervised by the photometric loss in an end-to-end training manner, NeRF inherently suffers from the shape-radiance ambiguity problem, i.e., it can perfectly fit training views but does not guarantee decoupling the two fields correctly. To deal with this issue, existing works have incorporated prior knowledge to provide an independent supervision signal for the density field, including total variation loss, sparsity loss, distortion loss, etc. These losses are based on general assumptions about the density field, e.g., it should be smooth, sparse, or compact, which are not adaptive to a specific scene. In this paper, we propose a more adaptive method to reduce the shape-radiance ambiguity. The key is a rendering method that is only based on the density field. Specifically, we first estimate the color field based on the density field and posed images in a closed form. Then NeRF's rendering process can proceed. We address the problems in estimating the color field, including occlusion and non-uniformly distributed views. Afterwards, it is applied to regularize NeRF's density field. As our regularization is guided by photometric loss, it is more adaptive compared to existing ones. Experimental results show that our method improves the density field of NeRF both qualitatively and quantitatively. Our code is available at https://github.com/qihangGH/Closed-form-color-field.