The growth of the Internet of Things (IoT) has reshaped the way devices, systems, and applications connect, leading to an enormous surge in data generation across various domains. This expansion, paired with the exponential increase in IoT devices, requires advanced data analysis capabilities to manage the multimodal sensory data collected by the massive IoT devices in real time, such as sensor outputs, visual data, audios, and videos. To address this challenge, large generative artificial intelligent (AI) models are designed, showing promise in processing multimodal data. However, deploying these models on IoT devices is constrained by limited computational power, memory, and energy resources, preventing full realization of their potential for real-time IoT systems. To address these limitations, we propose an innovative end-edge collaborative model framework between end nodes and edge servers, designed to balance computational load and optimize resource use. This approach transmits both extracted features and residual mapping data from end nodes to edge servers, allowing for spectrum efficient data handling across the network. Our work formulates an optimization strategy to enhance mean average precision (mAP) by adjusting task distribution, bandwidth, and data quantization in response to real-time network and device conditions. Comprehensive simulations demonstrate the proposed approach’s superiority over conventional centralized edge model computing and distributed end model computing frameworks, achieving enhanced efficiency across various communication rates in real time.

In this paper, a real-time obstacle avoidance approach and a trajectory planning method are proposed to avoid collisions to move an object jointly by a human and an omnidirectional mobile robot. Different from many existing approaches of local trajectory planning, the proposed method called Multiscale Local Perception Region Approach (MLPRA) is specially designed for obstacle avoidance of omnidirectional wheeled robots with direction constraints of human guidance, which can respond fast to dynamic obstacles and guarantee the safety of the robot. To solve the problem that the position of the human hand is difficult to obtain by visual sensing due to visual occlusion, a simple mechanism that can indirectly measure the change of the position of the human hand is designed, and further a method to follow the human’s intention based on this mechanism is proposed. Finally, the simulation environment on the Gazebo simulation platform is built to verify the feasibility and effectiveness of our proposed methods. Experimental results show that after embedding proposed methods into the omnidirectional mobile robot, obstacles can be effectively avoided in co-transporting processes.Note to Practitioners—The motivation of this paper is focusing on obstacle avoidance of omnidirectional mobile robots in human-robot co-transporting, application scenarios concentrated in factories and logistics warehouses, such as the mobile robot collaborates with a human (a robot teleoperated by human) transporting a table. The existing obstacle avoidance algorithm can be regarded as a local trajectory planning problem, which cannot directly adapt to co-transporting tasks in real-time. Considering the specific task if the robot cannot cooperate with human in real-time, the carried object will fall. In this paper, a local trajectory planning method is proposed, regarding the obstacle generating virtual repulsive force, regarding the human action on the robot as the virtual gravitational force, and the mobile robot under two virtual forces not only follows human action in co-transporting, but also avoids obstacles. The method shows quick calculation and good real-time performance. The method proposed is evaluated by co-transporting in Gazebo and experiments based on omnidirectional mobile robots, and ensured that the object being carried does not fall and robot can avoid collisions in the unknown environments without positioning systems.

In this paper, an adaptive fixed-time control approach is designed for a robot with dynamic uncertainty in the presence of input quantization by using the Broad learning system (BLS). The proposed BLS-based control algorithm is constructed by fusing the BLS with the radial basis function neural network, which is improved in terms of node selection rule with a self-adjusting Gaussian function center and enhancement layer. A hysteresis quantizer is applied to the requirement of a low transmission rate. For the nonlinearity occurring in the quantized input, a novel adaptive fixed-time method is developed such that 1) the adverse effect of quantization nonlinearity is removed in a finite interval; 2) the BLS-based approximation technique can improve the approximation accuracy, which enhances the robustness of the closed-loop system; and 3) via the Lyapunov stability method, the fixed-time convergence of the closed-loop system is proved. Finally, numerical simulations and experiments validate the effectiveness of the proposed control scheme.

This study introduces a novel finite time fault tolerant controller integrating nonsingular terminal sliding mode (NTSM) and reinforcement learning (RL) strategies for manipulator systems with actuator faults. Leveraging an actor-critic network architecture, the RL algorithm facilitates the computation of the cost function and the approximation of unknown nonlinear dynamics. The inherent properties of NTSM mitigate the effects of parameter uncertainties, thereby enhancing system robustness. Furthermore, an adaptive law is crafted to counteract the deleterious effects of actuator faults. Through the direct Lyapunov function approach, it is demonstrated that the closed-loop system achieves semi-global practical finite-time stability. This control strategy diminishes the dependence on precise model accuracy and augments the system's fault tolerance. The viability of the proposed algorithm is corroborated by simulation results, and its efficacy is further validated through experiments conducted on the 6-DOF Kinova Jaco 2 platform.

Pouring from one beaker to another is crucial in the preparation of coatings within material synthesis. In this study, a collaborative robot is utilized to imitate the behavior of experimenters in order to accomplish pouring tasks across various scenarios. Given that these tasks involve complex position-attitude relationships and the presence of obstacles, traditional rigid programming methods are hardly employed. Instead, learning from demonstration is incorporated to transfer experimenters’ pouring skills, which encompasses three phases: teaching, learning, and reproduction. We propose a vision-based dynamic movement primitives approach to generalize the skill based on visual feedback. Utilizing real-time visual information feedback regarding liquid level and beaker size, the teaching trajectory, which involves coupled position and attitude relationships, is generalized to adapt dynamically to differing experimental requirements. In experiments, we utilized the Kinect V2 camera and the Kinova Jaco2 manipulator to assess the efficacy of the proposed method.

This article presents an intelligent robotic system developed for experiments in the materials science laboratory, specifically focusing on coating preparation via layer-by-layer self-assembly techniques and hydrophobic detection. The system integrates two collaborative robotic arms, enhanced with dynamic movement primitives (DMPs), to mimic human manipulation skills and bolster the robots’ imitation capabilities. Additionally, a mobile robotic arm facilitates autonomous operations. A key component is an independently designed optical detection device capable of measuring water droplet angles. Coupled with a compatible simulation platform, the system can perform virtual experiments and generate trajectories for obstacle avoidance, and in which generative adversarial imitation learning (GAIL) in simulating robot trajectories. This article details the system’s construction, process design encompassing the robotic systems, optical detection device, simulation, and visualization platform. It also explores the vast potential of future AI-driven laboratories in materials science, biology, medicine, and chemistry.

This article proposes an adaptive fault-tolerant control (AFTC) approach based on a fixed-time sliding mode for suppressing vibrations of an uncertain, stand-alone tall building-like structure (STABLS). The method incorporates adaptive improved radial basis function neural networks (RBFNNs) within the broad learning system (BLS) to estimate model uncertainty and uses an adaptive fixed-time sliding mode approach to mitigate the impact of actuator effectiveness failures. The key contribution of this article is its demonstration of theoretically and practically guaranteed fixed-time performance of the flexible structure against uncertainty and actuator effectiveness failures. Additionally, the method estimates the lower bound of actuator health when it is unknown. Simulation and experimental results confirm the efficacy of the proposed vibration suppression method.

In this article, we focus on human-to-cobot dual-arm handover operations for large box-type objects. The efficiency of handover operations should be ensured and the naturalness as if the handover is going on between two humans. First of all, we study the human-human dual-arm large box-type object natural handover process to guide this research. Then, for efficiency, we combine the probabilistic approach with the online learning algorithm to predict the beginning of the handover task and handover positions. The online updating probabilistic models can deal with not only human givers' regular motion patterns but also their irregular motion patterns. Then, to guarantee that human givers can perform handover operations naturally, we apply the probabilistic robot skill learning method kernelized movement primitives (KMPs) to adapt the learned receiving skills and fulfill some constraints for safety based on online predicted results. Furthermore, we give special attention to the dual-arm grasp strategy and control design to guarantee a stable grasp. In addition, we equip this handover system on a Baxter cobot and extend its grippers to make it more suitable for dual-arm handover operations. The experimental results show that the proposed handover system can solve human-to-cobot dual-arm handover operations for large box-type objects naturally and efficiently.

In this article, neural network (NN)-based sliding mode control schemes are proposed for an n-link robotic manipulator with system uncertainties, input deadzone, and external perturbations. A novel error-shifting function is proposed to release initial conditions. NNs are employed to approximate the unknown parameters of both system uncertainties and input deadzone. To update the sliding mode scheme, two advanced sliding mode surfaces with error-shifting function and barrier function are proposed to reduce the dependency of prior information and to realize a finite time convergence result, collectively. It should be pointed out that the proposed methods do not require initial states to satisfy the prescribed constraint caused by the barrier function and can be applied under unknown initial conditions. Furthermore, finite-time convergence for both tracking errors and NN weights is guaranteed. The effectiveness of the proposed schemes is demonstrated by simulation and experiments on the KINOVA robot.

Human-robot co-transportation allows for a human and a robot to perform an object transportation task cooperatively on a shared environment. This range of applications raises a great number of theoretical and practical challenges arising mainly from the unknown human-robot interaction model as well as from the difficulty of accurately model the robot dynamics. In this article, an adaptive impedance controller for human-robot co-transportation is put forward in task space. Vision and force sensing are employed to obtain the human hand position, and to measure the interaction force between the human and the robot. Using the latest developments in nonlinear control theory, we propose a robot end-effector controller to track the motion of the human partner under actuators' input constraints, unknown initial conditions, and unknown robot dynamics. The proposed adaptive impedance control algorithm offers a safe interaction between the human and the robot and achieves a smooth control behavior along the different phases of the co-transportation task. Simulations and experiments are conducted to illustrate the performance of the proposed techniques in a co-transportation task.

In this article, subject to time-varying delay and uncertainties in dynamics, we propose a novel adaptive fixed-time control strategy for a class of nonlinear bilateral teleoperation systems. First, an adaptive control scheme is applied to estimate the upper bound of delay, which can resolve the predicament that delay has significant impacts on the stability of bilateral teleoperation systems. Then, radial basis function neural networks (RBFNNs) are utilized for estimating uncertainties in bilateral teleoperation systems, including dynamics, operator, and environmental models. Novel adaptation laws are introduced to address systems' uncertainties in the fixed-time convergence settings. Next, a novel adaptive fixed-time neural network control scheme is proposed. Based on the Lyapunov stability theory, the bilateral teleoperation systems are proved to be stable in fixed time. Finally, simulations and experiments are presented to verify the validity of the control algorithm.

This article proposes a Bayesian method to acquire the estimation of human impedance and motion intention in a human-robot collaborative task. Combining with the prior knowledge of human stiffness, estimated stiffness obeying Gaussian distribution is obtained by Bayesian estimation, and human motion intention can be also estimated. An adaptive impedance control strategy is employed to track a target impedance model and neural networks are used to compensate for uncertainties in robotic dynamics. Comparative simulation results are carried out to verify the effectiveness of estimation method and emphasize the advantages of the proposed control strategy. The experiment, performed on Baxter robot platform, illustrates a good system performance.

This article focuses on the tracking control issue of robotic systems with dynamic uncertainties. To enhance tracking accuracy in a robotic manipulator with uncertainties, an adaptive fuzzy full-state feedback control is proposed. In view of output-feedback control with unknown states, a high-gain observer is employed to estimate unknown states. Considering the particular requirement that output of systems should be constrained in some practical working fields, we further design adaptive fuzzy full-state and output-feedback control schemes with output constraint to ensure that output maintains in constrained regions. By applying the Lyapunov theory, it is guaranteed that closed-loop systems are semiglobally uniformly ultimately bounded (SGUUB). Tangent-type barrier Lyapunov function is used for the controller design with output constraint and ensure stability. Finally, the effectiveness of our proposed methods is shown through both simulation examples and experimental results, comparative experiments in Baxter robot are proposed for evaluating the practicability of our proposed methods in actual applications.

In this article, an admittance-based controller for physical human-robot interaction (pHRI) is presented to perform the coordinated operation in the constrained task space. An admittance model and a soft saturation function are employed to generate a differentiable reference trajectory to ensure that the end-effector motion of the manipulator complies with the human operation and avoids collision with surroundings. Then, an adaptive neural network (NN) controller involving integral barrier Lyapunov function (IBLF) is designed to deal with tracking issues. Meanwhile, the controller can guarantee the end-effector of the manipulator limited in the constrained task space. A learning method based on the radial basis function NN (RBFNN) is involved in controller design to compensate for the dynamic uncertainties and improve tracking performance. The IBLF method is provided to prevent violations of the constrained task space. We prove that all states of the closed-loop system are semiglobally uniformly ultimately bounded (SGUUB) by utilizing the Lyapunov stability principles. At last, the effectiveness of the proposed algorithm is verified on a Baxter robot experiment platform. Note to Practitioners-This work is motivated by the neglect of safety in existing controller design in physical human-robot interaction (pHRI), which exists in industry and services, such as assembly and medical care. It is considerably required in the controller design for rigorously handling constraints. Therefore, in this article, we propose a novel admittance-based human-robot interaction controller. The developed controller has the following functionalities: 1) ensuring reference trajectory remaining in the constrained task space: a differentiable reference trajectory is shaped by the desired admittance model and a soft saturation function; 2) solving uncertainties of robotic dynamics: a learning approach based on radial basis function neural network (RBFNN) is involved in controller design; and 3) ensuring the end-effector of the manipulator remaining in the constrained task space: different from other barrier Lyapunov function (BLF), integral BLF (IBLF) is proposed to constrain system output directly rather than tracking error, which may be more convenient for controller designers. The controller can be potentially applied in many areas. First, it can be used in the rehabilitation robot to avoid injuring the patient by limiting the motion. Second, it can ensure the end-effector of the industrial manipulator in a prescribed task region. In some industrial tasks, dangerous or damageable tools are mounted on the end-effector, and it will hurt humans and bring damage to the robot when the end-effector is out of the prescribed task region. Third, it may bring a new idea to the designed controller for avoiding collisions in pHRI when collisions occur in the prescribed trajectory of end-effector.

This paper proposed an adaptive neural admittance control strategy for collision avoidance in human-robot collaborative tasks. In order to ensure that the robot end-effector can avoid collisions with surroundings, robot should be operated compliantly by human within a constrained task space. An impedance model and a soft saturation function are employed to generate a differentiable reference trajectory. Then, adaptive neural network control with position constraint, based on integral barrier Lyapunov function (IBLF), is designed to achieve precise tracking while guaranteeing constrained satisfaction. Utilizing Lyapunov stability principles, we prove that semi-globally uniformly bounded stability is guaranteed for all states of the closed-loop system. At last, the effectiveness of the proposed algorithm is verified on a Baxter robot experimental platform. Collisions with surroundings can be avoided in human-robot collaborative tasks.