Accurate prediction of material removal depth and averaged surface roughness is crucial for evaluating the performance of robotic belt grinding (RBG). Nevertheless, the machining parameters of RBG across different spaces exhibit various data distributions, which often results in prediction shifts on unseen machining parameters when using conventional approaches. In this article, we introduce a pairwise domain adaptation-assisted dual-task learning (PW-DA-DTL) method for copredicting material removal depth and averaged surface roughness with regard to new RBG machining parameter spaces. The multigate mixture-of-experts method is employed as the foundational framework for dual-task learning, effectively capturing and modeling the relationships between material removal depth and average surface roughness by leveraging their inherent task interdependencies. The pairwise domain adaptation strategy is put forward to simultaneously enhance sample diversity and mitigate cross-domain data distribution discrepancy between the existing and new RBG machining parameter spaces. Comparative experiments are presented to demonstrate the effectiveness and superiority of the proposed PW-DA-DTL method.

In featureless point clouds, such as cylinder-ruled surfaces or shapes with lots of flattened areas, we observe that some classic iterative closest point variants, including point-to-point, point-to-plane, and symmetric metrics, are trapped in local minima. To explain the above phenomenons, we derive that the upper bounds error of landscapes (UBEL) of point-to-plane and symmetric metrics are nearly zero in the above scenes, resulting in slow convergence or local minima. To address the above challenges, we introduce a constrained registration method, which integrates geometry distance constraints (GDC) and weighted enhanced distance metric. Specifically, WED combines point-to-point and point-to-plane metrics, resulting in a steeper UBEL than point-to-plane and symmetric; GDCs constrain the transformation matrix into a feasible subset to escape local minima. Moreover, we introduce a dynamic slack of constraint algorithm to improve the stability of the linear perturbation in the constrained registration problem. Simulations and experiments are conducted on typical featureless objects, including a turbine blade, a cylinder-ruled surface, an outlet guide vane, and a rotor engine, to verify the effectiveness and efficiency of the presented registration framework.

We present a generative dialogue system capable of operating in a full-duplex manner, allowing for seamless interaction. It is based on a large language model (LLM) carefully aligned to be aware of a perception module, a motor function module, and the concept of a simple finite state machine (called neural FSM) with two states. The perception and motor function modules operate in tandem, allowing the system to speak and listen to the user simultaneously. The LLM generates textual tokens for inquiry responses and makes autonomous decisions to start responding to, wait for, or interrupt the user by emitting control tokens to the neural FSM. All these tasks of the LLM are carried out as next token prediction on a serialized view of the dialogue in real-time. In automatic quality evaluations simulating real-life interaction, the proposed system reduces the average conversation response latency by more than threefold compared with LLM-based half-duplex dialogue systems while responding within less than 500 milliseconds in more than 50% of evaluated interactions. Running an LLM with only 8 billion parameters, our system exhibits an 8% higher interruption precision rate than the best available commercial LLM for voice-based dialogue.

Manufacturing systems envisioned for factories of the future will promote human-centricity for close collaboration in a shared working environment towards better overall productivity within the context of Industry 5.0. Robust and accurate recognition and prediction of human intentions are crucial to reliable and safe collaborative operations between humans and robots. For this purpose, this paper proposed a safety-aware human-centric collaborative assembly approach driven by function blocks, human action recognition for intention detection, and collision avoidance for safe robot control. Within the context, a deep learning-based recognition system is developed for high-accuracy human intention recognition and prediction, and an assembly feature-based approach driven by function blocks is presented for assembly execution and control. Thus, assembly features and human behaviours during assembly are formulated to support safe assembly actions. Skeleton-based human behaviours are defined as control inputs to an adaptive safety-aware scheme. The scheme includes collaborative and parallel mode-based pre-warning and obstacle avoidance approaches for a human-centric collaborative assembly system. The former is to monitor and regulate robot control modes when working in parallel with humans, and the latter uses a position-based approach to control robot actions by adaptively adjusting obstacle avoidance trajectories in a dynamic collaborative environment. The findings of this paper reveal the effectiveness of the developed system, as experimentally validated through an engine-assembly case study.

This work develops a dynamic compliant force control (DCFC) strategy for the robotic belt grinding system to suppress the vibrations and over-grinding phenomenon. First, the vibration mechanism is investigated, and the corresponding vibration models before and after contact are constructed, both of which can decompose the vibrations into three components: free, accompanying and forced vibrations. Next, the extra compliant hardware is equipped to the grinder to realize the dynamic adjustment of equivalent damping. The DCFC strategy considering mechanical compliance accompanied by the dynamic closed-loop control of the grinder damping is presented based on the empirical wavelet transform and multi-scale permutation entropy. Moreover, the cutting fluctuation ratio index is proposed to evaluate the severity of the over-grinding together with over-grinding time. Experiments demonstrate that compared with the general force control, the DCFC strategy can reduce the vibration amplitude from 3.01 mm/s$^2$to 0.97 mm/s$^2$, and the over-grinding time/cutting fluctuation ratio from 1.05 s/24.71% to 0.54 s/13.18%, consequently enhancing the grinding stability and quality.Note to Practitioners—This work is motivated by the need to maintain the grinding stability and suppress the over-grinding phenomenon for the robotic belt grinding system. The vibrations and over-grinding phenomenon caused by the weak rigidity and poor precision of the robot always result in incomplete grinding of the workpiece that needs partially manual regrinding. The proposed DCFC strategy enables a dynamic compliant contact force between the grinding tool and the robot, thus effectively suppressing the over-grinding phenomenon and enhancing the consistency of grinding quality, which is particularly suitable for considering material removal consistency in cut-in and cut-out areas. This method can be implemented by the user as described or be acquired as a standalone device, but some extra hardware needs to be equipped with the grinding tool to use this method.

The ability to synthesize long-term human motion sequences in real-world scenes can facilitate numerous applications. Previous approaches for scene-aware motion synthesis are constrained by pre-defined target objects or positions and thus limit the diversity of human-scene interactions for synthesized motions. In this paper, we focus on the problem of synthesizing diverse scene-aware human motions under the guidance of target action sequences. To achieve this, we first decompose the diversity of scene-aware human motions into three aspects, namely interaction diversity (e.g. sitting on different objects with different poses in the given scenes), path diversity (e.g. moving to the target locations following different paths), and the motion diversity (e.g. having various body movements during moving). Based on this factorized scheme, a hierarchical framework is proposed, with each sub-module responsible for modeling one aspect. We assess the effectiveness of our framework on two challenging datasets for scene-aware human motion synthesis. The experiment results show that the proposed framework remarkably outperforms previous methods in terms of diversity and naturalness.

We revisit human motion synthesis, a task useful in various real-world applications, in this paper. Whereas a number of methods have been developed previously for this task, they are often limited in two aspects: 1) focus on the poses while leaving the location movement behind, and 2) ignore the impact of the environment on the human motion. In this paper, we propose a new framework, with the interaction between the scene and the human motion taken into account. Considering the uncertainty of human motion, we formulate this task as a generative task, whose objective is to generate plausible human motion conditioned on both the scene and the human’s initial position. This framework factorizes the distribution of human motions into a distribution of movement trajectories conditioned on scenes and that of body pose dynamics conditioned on both scenes and trajectories. We further derive a GAN-based learning approach, with discriminators to enforce the compatibility between the human motion and the contextual scene as well as the 3D-to-2D projection constraints. We assess the effectiveness of the proposed method on two challenging datasets, which cover both synthetic and real-world environments.

Reducing inconsistencies in the behavior of different versions of an AI system can be as important in practice as reducing its overall error. In image classification, sample-wise inconsistencies appear as "negative flips": A new model incorrectly predicts the output for a test sample that was correctly classified by the old (reference) model. Positive-congruent (PC) training aims at reducing error rate while at the same time reducing negative flips, thus maximizing congruency with the reference model only on positive predictions, unlike model distillation. We propose a simple approach for PC training, Focal Distillation, which enforces congruence with the reference model by giving more weights to samples that were correctly classified. We also found that, if the reference model itself can be chosen as an ensemble of multiple deep neural networks, negative flips can be further reduced without affecting the new model’s accuracy.

We present a unified and flexible framework to address the generalized problem of 3D motion synthesis that covers the tasks of motion prediction, completion, interpolation, and spatial-temporal recovery. Since these tasks have different input constraints and various fidelity and diversity requirements, most existing approaches only cater to a specific task or use different architectures to address various tasks. Here we propose a unified framework based on Conditional Variational Auto-Encoder (CVAE), where we treat any arbitrary input as a masked motion series. Notably, by considering this problem as a conditional generation process, we estimate a parametric distribution of the missing regions based on the input conditions, from which to sample and synthesize the full motion series. To further allow the flexibility of manipulating the motion style of the generated series, we design an Action-Adaptive Modulation (AAM) to propagate the given semantic guidance through the whole sequence. We also introduce a cross-attention mechanism to exploit distant relations among decoder and encoder features for better realism and global consistency. We conducted extensive experiments on Human 3.6M and CMU-Mocap. The results show that our method produces coherent and realistic results for various motion synthesis tasks, with the synthesized motions distinctly adapted by the given action labels.

This Research Work in Progress Paper aims to develop holistic competencies in engineering students through course-level teaching and learning activities. Information is the core of many engineering fields, such as communications engineering, signal processing, and computational intelligence. Despite its ambient existence in our everyday lives, most of the engineering students are not aware of the connection between information and their professional role as an engineer in the informational world. We seek to engage engineering students in understanding their professional and ethical responsibility, and promote their awareness of the impact of engineering solutions in a global and societal context. Our study is grounded in the intersection between two theoretical frameworks: (1) Holistic Competencies and (2) William Perry's scheme of intellectual and ethical development. Having students' learning evidence against William Perry's framework and identified students' cognitive changes along their course study, we suggest that holistic competencies can be fostered through cultivating students' multifaceted understanding of information and supplementing with real cases where social media serve as a channel for students to exchange ideas to address various global challenges.

In this work, we aim to generate long actions represented as sequences of skeletons. The generated sequences must demonstrate continuous, meaningful human actions, while maintaining coherence among body parts. Instead of generating skeletons sequentially following an autoregressive model, we propose a framework that generates the entire sequence altogether by transforming from a sequence of latent vectors sampled from a Gaussian process (GP). This framework, named Convolutional Sequence Generation Network (CSGN), jointly models structures in temporal and spatial dimensions. It captures the temporal structure at multiple scales through the GP prior and the temporal convolutions; and establishes the spatial connection between the latent vectors and the skeleton graphs via a novel graph refining scheme. It is noteworthy that CSGN allows bidirectional transforms between the latent and the observed spaces, thus enabling semantic manipulation of the action sequences in various forms. We conducted empirical studies on multiple datasets, including a set of high-quality dancing sequences collected by us. The results show that our framework can produce long action sequences that are coherent across time steps and among body parts.

Dynamics of human body skeletons convey significant information for human action recognition. Conventional approaches for modeling skeletons usually rely on hand-crafted parts or traversal rules, thus resulting in limited expressive power and difficulties of generalization. In this work, we propose a novel model of dynamic skeletons called Spatial-Temporal Graph Convolutional Networks (ST-GCN), which moves beyond the limitations of previous methods by automatically learning both the spatial and temporal patterns from data. This formulation not only leads to greater expressive power but also stronger generalization capability. On two large datasets, Kinetics and NTU-RGBD, it achieves substantial improvements over mainstream methods.

Fashion landmarks are functional key points defined on clothes, such as corners of neckline, hemline, and cuff. They have been recently introduced [18]as an effective visual representation for fashion image understanding. However, detecting fashion landmarks are challenging due to background clutters, human poses, and scales. To remove the above variations, previous works usually assumed bounding boxes of clothes are provided in training and test as additional annotations, which are expensive to obtain and inapplicable in practice. This work addresses unconstrained fashion landmark detection, where clothing bounding boxes are not provided in both training and test. To this end, we present a novel Deep LAndmark Network (DLAN), where bounding boxes and landmarks are jointly estimated and trained iteratively in an end-to-end manner. DLAN contains two dedicated modules, including a Selective Dilated Convolution for handling scale discrepancies, and a Hierarchical Recurrent Spatial Transformer for handling background clutters. To evaluate DLAN, we present a large-scale fashion landmark dataset, namely Unconstrained Landmark Database (ULD), consisting of 30K images. Statistics show that ULD is more challenging than existing datasets in terms of image scales, background clutters, and human poses. Extensive experiments demonstrate the effectiveness of DLAN over the state-of-the-art methods. DLAN also exhibits excellent generalization across different clothing categories and modalities, making it extremely suitable for real-world fashion analysis.

Visually-guided robot grinding is a novel and promising automation technique for blade manufacturing. One common problem encountered in robot grinding is hand-eye calibration, which establishes the pose relationship between the end effector (hand) and the scanning sensor (eye). This paper proposes a new calibration approach for robot belt grinding. The main contribution of this paper is its consideration of both joint parameter errors and pose parameter errors in a hand-eye calibration equation. The objective function of the hand-eye calibration is built and solved, from which 30 compensated values (corresponding to 24 joint parameters and six pose parameters) are easily calculated in a closed solution. The proposed approach is economic and simple because only a criterion sphere is used to calculate the calibration parameters, avoiding the need for an expensive and complicated tracking process using a laser tracker. The effectiveness of this method is verified using a calibration experiment and a blade grinding experiment. The code used in this approach is attached in the Appendix.