Multi-task imitation learning is a pressing issue for robotics applications. However, the inaccuracy and inefficiency of imitation learning frequently limits current approaches. To cope with the problem, this paper proposes Modules Generation with Adaptive Discriminator-Feedback Mechanism with Diffusion Model (MGAD-FMDM), a novel general modular method for robotic multi-task imitation learning. A transferable adaptive discriminator constitutes the first component whose function is to automatically generate potential primitive modules, which enables robots to improve the imitation accuracy. A universal feedback mechanism for guiding policy updates represents the second development. By employing a diffusion model, the feedback mechanism achieves a two-step process of first recognizing the expert demonstrated distribution, then evaluating the fit between imitated trajectories and expert demonstrations, thereby enhancing learning efficiency. Extensive experiments on practical robot tasks indicate that the proposed MGAD-FMDM achieves superior performance over state-of-the-art methods, including the improvements of 2.22%-34.85% in Spearman’s rank correlation, 14.29%-100.00% in success rate, 5.43%-64.35% in earth mover’s distance, and 21.74%-30.10% in convergence speed. The video can be obtained at: https://github.com/yueli0827/Robotic-Multi-Task-Imitation-Learning.

This paper explores the application of large language models (LLMs) in nursing and elderly care, focusing on AI-driven patient monitoring and interaction. We introduce a novel Chinese nursing dataset and implement incremental pre-training (IPT) and supervised fine-tuning (SFT) techniques to enhance LLM performance in specialized tasks. Using LangChain, we develop an interactable nursing assistant capable of real-time care and personalized interventions. Experimental results demonstrate significant improvements, paving the way for AI-driven solutions to meet the growing demands of healthcare in aging populations.

Imitation learning is a significant precondition for robotics automation. However, traditional approaches of Behavioral Cloning (BC), Inverse Reinforcement Learning (IRL) and Generative Adversarial Imitation Learning (GAIL) are frequently accompanied by the problems of poor learning success and the over-reliance on expert samples. To remedy these problems, this paper proposes a novel robotic trajectory imitation method, named TGOD-SD. First, the TGOD generates several distinguishable trajectories for imitation through noreward Reinforcement Learning. Then, the agent searches the most appropriate trajectory that matching the target trajectory by Sinkhorn Distance (SD). Finally, the proposed TGOD-SD is verified in real universal robot UR5. TGOD-SD can directly follow the expert demonstration, and also succeed the similar robotic tasks. Quantitative and qualitative evaluation illustrates that TGOD-SD achieves remarkable learning success rate compared with the state-of-the-art robot imitation learning methods. In addition, TGOD-SD achieves robot imitation learning from a single expert demonstration, effectively reducing the dependence on the expert demonstrations. The code and video can be obtained at: https://github.com/Nursing-Robot-Laboratory

In the context of computer vision and human-robot interaction, forecasting 3D human poses is crucial for understanding human behavior and enhancing the predictive capabilities of intelligent systems. While existing methods have made significant progress, they often focus on predicting major body joints, overlooking fine-grained gestures and their interaction with objects. Human hand movements, particularly during object interactions, play a pivotal role and provide more precise expressions of human poses. This work fills this gap and introduces a novel paradigm: forecasting 3D whole-body human poses with a focus on grasping objects. This task involves predicting activities across all joints in the body and hands, encompassing the complexities of internal heterogeneity and external interactivity. To tackle these challenges, we also propose a novel approach: C3HOST, cross-context cross-modal consolidation for 3D whole-body pose forecasting, effectively handles the complexities of internal heterogeneity and external interactivity. C3HOST involves distinct steps, including the heterogeneous content encoding and alignment, and cross-modal feature learning and interaction. These enable us to predict activities across all body and hand joints, ensuring high-precision whole-body human pose prediction, even during object grasping. Extensive experiments on two benchmarks demonstrate that our model significantly enhances the accuracy of whole-body human motion prediction. The project page is available at https://sites.google.com/view/c3host.

The significance of telerobotic trajectory segmentation has been demonstrated on a range of skill training and robotic automation tasks. However, the trajectories of telerobot are characterized by complexity and high dimensionality, making it difficult to segment them accurately. Existing methods are often plagued by feature inefficiency, and the clustering methods are also difficult to properly model the trajectories. In addition, over-segmentation also affects the accuracy of the clustering-based segmentation methods. To address above problems, this article presents a new unsupervised approach that automatically segments multimodal trajectory data of a telerobot and solves the problem of over-segmentation through a postpromoting procedure. First, we present an unsupervised visual feature-extraction network, that is, a dense connection spatial convolution network (DCSC) to generate more discriminative features for clustering. The dense convolution and spatial convolution facilitates information flow, enhances feature propagation, and avoids manual annotation. Next, we develop an unsupervised trajectory segmentation method that is called multimodality clustering with Chinese restaurant process (MC-CRP). The model utilizes data from different modalities and segments the trajectory through hierarchical clustering. MC-CRP obtains more accurate results in a short period of time. To further improve the precision of trajectory segmentation, we merge over-segments based on predefined similarity measurements. Extensive experiments on the publicly available data set JIGSAWS show that the presented approach achieves 70.1% silhouette coefficient, 25.1% normalized mutual information, and 71.4% segmentation accuracy. These metrics demonstrate that the presented segmentation approach provides deeper insight into the trajectory features and improve the accuracy of segmentation more efficiently than others.

Reinforcement Learning (RL) with Skill Diversity (SD) is an appealing method of imitating expert trajectory. Numeral papers have presented methods of SD in several imitation learning scenarios. However, it is still a gap between SD and the practical implementation of robot trajectory imitation. The poor performance of matching algorithm often leads to imitation failures as well. To bridge the gaps and remedy the drawbacks, this paper proposed a robotic trajectory imitation method SD-PDMD with no-reward function reinforcement learning. A RL frame for robot trajectory imitation with SD is constructed to generate potential imitation trajectories, and then an optimal similarity algorithm based on predefined similarity measurements (PDMD) is proposed to match the expert trajectory with the most similar generated trajectory. Extensive experiments illustrate that the proposed SD-PDMD can effectively complete the robot trajectory imitation task, the performance of PDMD for similarity matching is also better than the traditional Euclidean Distance with an improvement of 5.7%-9.2%. The code and video can be obtained at: https://github.com/Nursing-Robot-Laboratory/SD-PDMD

Multi-size object detection is a technical difficulty which impeding the development of the intelligent nursing robot. To cope with the problem, this paper proposes a Pyramid Transformer model to detect the objects with different sizes in nursing scenario. Pyramid Transformer consists of three parts including Transformer Module, Pyramid Structure and Convolution Module. Transformer Module can improve the performance of large object detection with Multi-head Attention mechanism, and Pyramid Structure enables the model to make prediction with feature maps of different sizes which benefits the detection of small objects. Convolution Module is employed to reduce hardware requirements, and it makes Pyramid Transformer could run and implement on a single graphics card. The experiments show that the mean average precision reaches 72.7% which makes improvement over other models. This shows that the proposed Pyramid Transformer model is practical and effective for object detection of the nursing robot. The dataset can be got at https://github.com/NotFar1997/NSI-dataset.

To cope with the difficulty of training and eval-uation for nursing telerobot operator. This paper proposes a training-evaluation method for operator with unsupervised trajectory segmentation. To evaluate the dexterity and proce-dural knowledge of the operators objectively, we propose a new unsupervised model TSC-CRP that can automatically segment trajectory from nursing robotic training sessions. By comparing the segmented sub-trajectories and the standard sub-trajectory process, the method can provide objective evaluation and meaningful feedback without the intervention from experts. Experiments show that TSC-CRP has higher segmentation accuracy than other unsupervised methods, and it can identify the operators with different skill levels. In practical, the proposed training-evaluation system allows to provide an in-depth analysis of operator action to assess their skills precisely.

To improve the efficiency of surgical trajectory segmentation for robot learning in robot-assisted minimally invasive surgery, this paper presents a fast unsupervised method using video and kinematic data, followed by a promoting procedure to address the over-segmentation issue. Unsupervised deep learning network, stacking convolutional auto-encoder, is employed to extract more discriminative features from videos in an effective way. To further improve the accuracy of segmentation, on one hand, wavelet transform is used to filter out the noises existed in the features from video and kinematic data. On the other hand, the segmentation result is promoted by identifying the adjacent segments with no state transition based on the predefined similarity measurements. Extensive experiments on a public dataset JIGSAWS show that our method achieves much higher accuracy of segmentation than state-of-the-art methods in the shorter time.