3-D language Gaussian splatting has recently shown strong potential for open-vocabulary scene understanding and robotic manipulation. However, most existing methods require dense multiview observations to achieve accurate geometry reconstruction and reliable semantic alignment, which limits their applicability in scenarios where only sparse-view observations are available. In this work, we propose SparseGrasper, a framework for language-guided zero-shot robotic grasping under sparse-view conditions. SparseGrasper constructs a 3-D Gaussian language field from as few as three RGB images, enabling joint reasoning over geometry and semantics without the need for dense observations. To improve representation learning under sparse observations, we introduce a dual feature distillation module that fuses local object features with global contextual cues. We further design a language-guided grasp pose generation strategy that incorporates semantic grounding into grasp candidate selection, encouraging grasps that are both semantically relevant and geometrically feasible. Real-world experiments on a 7-DoF robotic manipulator validate that SparseGrasper effectively performs language-guided grasping of diverse, previously unseen objects from sparse observations.

The power industry relies on efficient inspection systems to ensure stability and safety. While deep learning has advanced automated inspection, its reliance on custom modules for specific tasks can impact efficiency. Knowledge distillation (KD) offers a balanced solution, but the complex textures and structures of power equipment challenge conventional KD methods, which often fail to capture essential local semantic and topological relationships. To address this, we proposeTopNet, a novel topology-preserved KD framework for power scene detection tasks. Specifically, we model the teacher’s knowledge as a graph, where nodes encode local fine-grained features and edges capture global topological relationships. Based on this, we introduce node feature distillation and edge feature distillation to transfer local–global structural knowledge, which can enhance the student’s ability to perceive objects. Furthermore, we also introduce aggregated feature distillation to incorporate and transfer contextual semantic knowledge. Comprehensive experiments are conducted on two different benchmark datasets to demonstrate that TopNet achieves state-of-the-art detection performance with high efficiency, offering a robust solution for automated power equipment inspection.

Real-world transmission line inspections (RTLIs) ensure power stability and safety. Deep learning (DL) models have become prevalent approaches for performing RTLI tasks. However, the high computational demands and substantial parameter requirements of DL models limit their real-world applicability. This article introduces a novel approach, a multiscale uncertainty-aware knowledge-based network, which is designed to balance the accuracy and efficiency in RTLI tasks. Specifically, we propose an uncertainty-aware knowledge distillation method that incorporates pixel-level uncertainty into the knowledge transfer process, mitigating the impact of noisy knowledge derived from extra background information contained in ground truths. In addition, our method integrates a multiscale relationship distillation technique, thus enhancing the transfer of multiscale information between the teacher and student models. Consequently, RTLI tasks can be efficiently accomplished using the well-learned lightweight student model. Comprehensive experiments conducted on a real-world dataset collected via uncrewedaerial vehicles demonstrate the efficacy of our proposed approach in terms of achieving high detection accuracy with reduced computational costs.

Free-form surfaces have been widely used in industrial design and manufacturing. For the requirements of measurement efficiency and precision, robots and optical scanners are applied to measure free-form surface parts increasingly. Due to the complex geometry shapes and occlusions of these parts, how to plan accessible viewpoints of a scanner to achieve the expected coverage rate is a challenging task. This paper presents a novel viewpoint planning method based on the visibility cone space explorer (VP-VCSE) for robotic measurement systems with 7 degrees of freedom (7-DOF). A digital twin for the robotic measurement system is implemented to provide core services for robotic measurement tasks, including sensor simulation and collision detection. To generate initial candidate viewpoints, a novel mesh segmentation algorithm based on the hybrid mixture model is proposed, which is convenient to handle the triangular mesh of the target object. Visibility computation for a target object in given viewpoints is the key to dealing with the occlusion problem. For this purpose, a general visibility model of a structured-light scanner is presented to compute visible areas accurately. In order to reduce occlusions, a visibility cone space explorer is designed to search optimal candidate viewpoints considering inverse kinematics and physical collisions simultaneously. The viewpoint planning problem is formulated as a set covering optimization problem and a next-best-view operator is introduced to improve the efficiency of the genetic algorithm for searching the resultant viewpoint set, guaranteeing the expected coverage rate and data overlap rate. The simulation and experiment results for four different test models show that the proposed algorithm outperforms the existing methods in terms of the uncovered rate and the minimum number of viewpoints.Note to Practitioners—This paper addressed a viewpoint planning problem for the robotic measurement system with a binocular structured light 3D scanner mounted on the end effector of the robot, where a robot and a turntable cooperate to complete the measurement tasks. The goal is to find a minimal number of viewpoints that provides full coverage of the target surfaces. Although many studies have addressed this problem, there is little discussion about strategies to improve coverage rate when the target object has complex occlusions. This paper suggested a valuable practice to construct a visibility cone space to adjust viewpoint to reduce occlusions and improve the overall coverage rate. Simulation and experimental results demonstrated the feasibility and effectiveness of the proposed approach. This paper showed how to deal with various constraints that a feasible viewpoint needs to satisfy in the viewpoint generation, viewpoint adjustment, and viewpoint selection phase. Moreover, this paper provided a solution for developing the visualization, simulation, and interaction of a digital twin for the 7-DOF robotic measurement system. All core services for robotic measurement tasks are implemented based on a set of open source libraries, which provides a convenient learning and research software platform for practitioners. In future research, we will study how to improve the intelligence and cooperation of the robotic measurement system through deep learning or reinforcement learning techniques.

The edge detection technique is an essential step for aircraft skin milling in aviation manufacturing. Most of the current detection methods focus on traditionally defined edge extraction tasks but disregard the crucial systematic requirement of edge milling. In this article, we proposed a novel edge detection framework for automatic edge milling of aircraft skins. First, an edge probability detector is proposed by the spatial tangent continuity to provide the essential reference. Second, we propose a hierarchical branch searching method to hierarchically strip the desired milling edges from the raw point cloud, which consists of the following three graded progressive steps: branch backbone generation, branch extension, and branch pruning. We demonstrate the performance of the proposed method on both synthetic models and aircraft skin workpieces. The proposed method outperforms the other baselines and shows accurate edges for the edge milling task.

Diffusion probabilistic models have achieved remarkable success in text guided image generation. However, generating 3D shapes is still challenging due to the lack of sufficient data containing 3D models along with their descriptions. Moreover, text based descriptions of 3D shapes are inherently ambiguous and lack details. In this paper, we propose a sketch and text guided probabilistic diffusion model for colored point cloud generation that conditions the denoising process jointly with a hand drawn sketch of the object and its textual description. We incrementally diffuse the point coordinates and color values in a joint diffusion process to reach a Gaussian distribution. Colored point cloud generation thus amounts to learning the reverse diffusion process, conditioned by the sketch and text, to iteratively recover the desired shape and color. Specifically, to learn effective sketch-text embedding, our model adaptively aggregates the joint embedding of text prompt and the sketch based on a capsule attention network. Our model uses staged diffusion to generate the shape and then assign colors to different parts conditioned on the appearance prompt while preserving precise shapes from the first stage. This gives our model the flexibility to extend to multiple tasks, such as appearance re-editing and part segmentation. Experimental results demonstrate that our model outperforms recent state-of-the-art in point cloud generation.

Multirobot systems have shown great potential in dealing with complicated tasks that are impossible for a single robot to achieve. One essential problem encountered in cooperatively working of the multirobot systems is the unknown initial transformation relationships from hand to eye, base to base, and flange to tool. In this article, the problem of multicoordinates calibration for a dual-robot system is formulated to a matrix equation AXB = YCZ. A novel approach for simultaneously solving the unknowns in equation AXB = YCZ is proposed, which is composed of a closed form method based on the Kronecker product and an iterative method which converts the calculation of a nonlinear problem to an optimization problem of a strictly convex function. The closed form method is used to quickly obtain an initial estimation for the iterative method to improve the efficiency and accuracy of iteration. In addition, a series of conditions on the solvability of the problem are proposed to guide the operators to select appropriate robot attitudes during the calibration process. To show the feasibility and superiority of the proposed iterative method, two other calibration methods are chosen to be compared to the proposed method through simulation and practical experiments. The comparison results verify the superiority of the proposed method in accuracy, efficiency, and stability.

Expanding new functionalities efficiently is an ongoing challenge for single-turn task-oriented dialogue systems. In this work, we explore functionality-specific semi-supervised learning via self-training. We consider methods that augment training data automatically from unlabeled data sets in a functionality-targeted manner. In addition, we examine multiple techniques for efficient selection of augmented utterances to reduce training time and increase diversity. First, we consider paraphrase detection methods that attempt to find utterance variants of labeled training data with good coverage. Second, we explore sub-modular optimization based on n-grams features for utterance selection. Experiments show that functionality-specific self-training is very effective for improving system performance. In addition, methods optimizing diversity can reduce training data in many cases to 50% with little impact on performance.

Free-form surface matching that aligns measured points with a design model is a common problem in manufacturing automation. In this paper, an iterative variance-minimization matching (VMM) method is proposed to address measured points that have measuring defects, such as uneven/open point distributions and measuring noise. The basic idea is that the objective function is defined as the variance of the closest distance from each measured point to the design model, and the measuring defects are considered by incorporating an average distance item into the objective function. Using the defined average distance item, a strategy for analyzing the effect of measuring defects on VMM and existing methods is presented. It is shown that the VMM method does not easily become trapped in a local optimum when measuring defects exist. To consider convergence speed and convergence stability, a new distance based on the first-order point-to-point distance and point-to-tangent distance is developed and used in the objective function. To demonstrate the availability of the proposed method, quadratic convergence and positive definiteness are theoretically analyzed. The proposed method is efficient and insensitive to measuring defects and is useful for shape matching tasks involving free-form surface features. Note to Practitioners-This paper is motivated by the problem of matching measured points with a design model to automate manufacturing processes such as geometric inspection, workpiece localization, and allowance distribution. Measured points are obtained by applying a scanning device where measuring defects usually appear. Existing matching methods suffer from the drawback that the measured points may incline toward dense data and become trapped in a local optimum, due to measuring defects. To address this practical issue, this paper proposes a new method called variance-minimization matching (VMM), in which the objective function is optimized to weaken the effect of measuring defects. By examining the differences between VMM and existing methods, it is found that VMM can achieve quadratic convergence speed. Most importantly, the method is insensitive to uneven/open point distributions. In summary: 1) this method allows us to improve the matching accuracy in the presence of measuring defects; 2) there is no need to obtain a high-quality scan of the entire workpiece, potentially reducing scanning difficulty and improving scanning efficiency; and 3) the requirement of uniform sampling for measured points is reduced.

In the first part of this paper, a free-form surface matching method called variance-minimization matching (VMM) was proposed to address uneven/open point distributions and measuring noise. The convergence property and sensitivity to measuring defects were theoretically studied. In the second part of this paper, a series of experiments are presented to verify the feasibility of the proposed method in free-form surface matching. The experiments are divided into four sets: a measuring defects experiment, a noise experiment, a convergence experiment, and an artificial experiment. In the first set of experiments, the existing methods are prone to becoming trapped in a local optimum affected by uneven/open point distributions, which shows that measured points incline toward dense areas. However, in VMM, there is little inclination regardless of the increase in the number of measuring defects. In the second set of experiments, sensitivity to varying noise is tested. The results show that VMM helps prevent unstable sliding in the presence of Gaussian noise. In the third set of experiments, we compare convergence speed and convergence stability under different initial positions. It is verified that VMM exhibits the quadratic convergence. Finally, a set of artificial experiments is implemented, revealing that the proposed method is appropriate for use in automated manufacturing processes such as geometric inspection and allowance distribution. Note to Practitioners-Measuring defects usually occur when using a scanning device to obtain the measured points of a workpiece. Weakening the effect of measuring defects on matching results is critical to promoting manufacturing automation. This paper proposes a new method called variance-minimization matching (VMM) that considers measuring defects. In the first part of this paper, the modeling and theoretical analysis of VMM were introduced. In the second part of this paper, simulated experiments are performed to verify the feasibility of VMM in addressing uneven/open point distributions, measuring noise, and large initial positions. Next, artificial experiments employing VMM in geometric inspection and allowance distribution are presented. The proposed method also applies to other automated manufacturing processes, such as workpiece localization, deformation analysis, and complex parts repair.

Visually-guided robot grinding is a novel and promising automation technique for blade manufacturing. One common problem encountered in robot grinding is hand-eye calibration, which establishes the pose relationship between the end effector (hand) and the scanning sensor (eye). This paper proposes a new calibration approach for robot belt grinding. The main contribution of this paper is its consideration of both joint parameter errors and pose parameter errors in a hand-eye calibration equation. The objective function of the hand-eye calibration is built and solved, from which 30 compensated values (corresponding to 24 joint parameters and six pose parameters) are easily calculated in a closed solution. The proposed approach is economic and simple because only a criterion sphere is used to calculate the calibration parameters, avoiding the need for an expensive and complicated tracking process using a laser tracker. The effectiveness of this method is verified using a calibration experiment and a blade grinding experiment. The code used in this approach is attached in the Appendix.