Efficiently coordinating multiple robotic arms is vital for secure and optimal operation in a shared workspace. This requires not only successful task completion but also minimizing collision risks from overlapping movements. Introducing real-time motion modulation adds an extra layer of challenge to this coordination task. In this article, we introduce a novel framework for real-time multiarm coordination, offering two main contributions: First, based on fuzzy model-based movement primitives, we propose a method for real-time trajectory modulation by learning from single demonstrations. This capability allows robots to modulate their motions online to reach arbitrary new desired places smoothly without necessitating extra demonstrations from users. Second, our framework incorporates a real-time multiarm coordination strategy that seamlessly integrates the trajectory modulation method with an extended reactive approach. This strategy empowers multiple robotic arms operating within a shared workspace to dynamically regulate their movements and execute tasks simultaneously in a human-desired manner while reactively avoiding mutual collisions. In the experiments, we utilize a group of robotic arms working in a shared workspace to validate the effectiveness of our framework and to make comparisons with state-of-the-art methods.

Environmental collision is a challenging issue in human–robot collaboration. This article proposes a novel fuzzy cluster-based framework for robots to have reactive responses to various environmental collision scenarios. This framework makes four contributions: First, a fuzzy cluster-based environmental collision detection algorithm is developed to efficiently classify the collision area and noncollision (free) area of the environment. Second, based on the collision detection algorithm, a$p$-norm approximation-based collision avoidance algorithm is proposed to enable robots to avoid environmental collisions with guaranteed stability. Third, by extending the collision avoidance algorithm, an environmental collision adaptation algorithm is proposed to allow robots to adapt to environmental collisions with intelligently regulated contact force. Fourth, a teleoperation controller is designed to strengthen haptic force rendering and enhance the operator's perception of collisions. Going beyond existing methods, the proposed framework allows teleoperated robots to have real-time responses to collisions in quasi-static environments without suffering from local optima, where the environments can be unstructured, nonconvex, and detected with noisy outliers. In addition, this framework is simple in implementation because the proposed collision avoidance and collision adaptation algorithms work as several linear quadratic programming constraints that can be flexibly used by inverse kinematics solvers. Several experiments using 7-degree-of-freedom robots are conducted to test and compare the proposed framework with existing methods, demonstrating the effectiveness of our work.

Manipulability ellipsoid on the Riemannian manifold serves as an effective criterion to analyze, measure, and control the dexterous performance of robots. For asymmetric bilateral telerobotics, due to the different structures of leader and follower robots, it is difficult or even impossible for the operator to manually regulate the manipulability of the remote follower robot. Thus, it is desired that the follower robot can automatically regulate its manipulability to assist the operator in remote different task executions, like humans regulating their own postures to enhance manipulability and adapt to different task scenarios. This article proposes a novel framework for manipulability transfer from human to robot. In this framework, we develop a Type-2 fuzzy model-based imitation learning method to encode and reproduce manipulability ellipsoids from demonstrations. This method can achieve high performance in accuracy and computational efficiency. In addition, it supports learning from a single demonstration. Then, we combine this method with a Riemannian manifold-based quadratic programming control algorithm such that the robot manipulability can fast track the desired manipulability profile. This framework is applied to telerobotics, in which a bilateral teleoperation controller is designed that enables the robot to follow the operator's command and simultaneously self-regulate its manipulability to perform the task adaptively. Meanwhile, the operator can receive force feedback relating to the manipulability regulation. Evaluations using comparative studies and practical experiments with a 3-degrees of freedom (DoF) haptic device and 7-DoF robots are presented to show the effectiveness of the proposed framework.

This article studies group-wise point set registration and makes the following contributions: "FuzzyGReg", which is a new fuzzy cluster-based method to register multiple point sets jointly, and "FuzzyQA", which is the associated quality assessment to check registration accuracy automatically. Given a group of point sets, FuzzyGReg creates a model of fuzzy clusters and equally treats all the point sets as the elements of the fuzzy clusters. Then, the group-wise registration is turned into a fuzzy clustering problem. To resolve this problem, FuzzyGReg applies a fuzzy clustering algorithm to identify the parameters of the fuzzy clusters while jointly transforming all the point sets to achieve an alignment. Next, based on the identified fuzzy clusters, FuzzyQA calculates the spatial properties of the transformed point sets and then checks the alignment accuracy by comparing the similarity degrees of the spatial properties of the point sets. When a local misalignment is detected, a local re-alignment is performed to improve accuracy. The proposed method is cost-efficient and convenient to be implemented. In addition, it provides reliable quality assessments in the absence of ground truth and user intervention. In the experiments, different point sets are used to test the proposed method and make comparisons with state-of-the-art registration techniques. The experimental results demonstrate the effectiveness of our method. The code is available at https://gitsvn-nt.oru.se/qianfang.liao/FuzzyGRegWithQA.

This article studies the fuzzy cluster-based point set registration (FuzzyPSReg). First, we propose a new metric based on Gustafson–Kessel (GK) fuzzy clustering to measure the alignment of two point clouds. Unlike the metric based on fuzzy c-means (FCM) clustering in our previous work, the GK-based metric includes orientation properties of the point clouds, thereby providing more information for registration. We then develop the registration quality assessment of the GK-based metric, which is more sensitive to small misalignments than that of the FCM-based metric. Next, by effectively combining the two metrics, we design two FuzzyPSReg strategies with global optimization. 1)FuzzyPSReg-SS, which extends our previous work and aligns two similar-sized point clouds with greatly improved efficiency. 2)FuzzyPSReg-O2S, which aligns two point clouds with a relatively large difference in size and can be used to estimate the pose of an object in a scene. In the experiment, we use different point clouds to test and compare the proposed method with state-of-the-art registration approaches. The results demonstrate the advantages and effectiveness of our method.

Imitation learning is an important direction in the area of robot skill learning. It provides a user-friendly and straightforward solution to transfer human demonstrations to robots. In this article, we integrate fuzzy theory into imitation learning to develop a novel method called type-2 fuzzy model-based movement primitives (T2FMP). In this method, a group of data-driven type-2 fuzzy models are used to describe the input–output relationships of demonstrations. Based on the fuzzy models, T2FMP can efficiently reproduce the trajectory without high computational costs or cumbersome parameter settings. Besides, it can well handle the variation of the demonstrations and is robust to noise. In addition, we develop extensions that endow T2FMP with trajectory modulation and superposition to achieve real-time trajectory adaptation to various scenarios. Going beyond existing imitation learning methods, we further extend T2FMP to regulate the trajectory to avoid collisions in the environment that is unstructured, nonconvex, and detected with noisy outliers. Several experiments are performed to validate the effectiveness of our method.

This study presents a new point set registration method to align 3D range scans. In our method, fuzzy clusters are utilized to represent a scan, and the registration of two given scans is realized by minimizing a fuzzy weighted sum of the distances between their fuzzy cluster centers. This fuzzy cluster-based metric has a broad basin of convergence and is robust to noise. Moreover, this metric provides analytic gradients, allowing standard gradient-based algorithms to be applied for optimization. Based on this metric, the outlier issues are addressed. In addition, for the first time in rigid point set registration, a registration quality assessment in the absence of ground truth is provided. Furthermore, given specified rotation and translation spaces, we derive the upper and lower bounds of the fuzzy cluster-based metric and develop a branch-and-bound (BnB)-based optimization scheme, which can globally minimize the metric regardless of the initialization. This optimization scheme is performed in an efficient coarse-to-fine fashion: First, fuzzy clustering is applied to describe each of the two given scans by a small number of fuzzy clusters. Then, a global search, which integrates BnB and gradient-based algorithms, is implemented to achieve a coarse alignment for the two scans. During the global search, the registration quality assessment offers a beneficial stop criterion to detect whether a good result is obtained. Afterwards, a relatively large number of points of the two scans are directly taken as the fuzzy cluster centers, and then, the coarse solution is refined to be an exact alignment using the gradient-based local convergence. Compared to existing counterparts, this optimization scheme makes a large improvement in terms of robustness and efficiency by virtue of the fuzzy cluster-based metric and the registration quality assessment. In the experiments, the registration results of several 3D range scan pairs demonstrate the accuracy and effectiveness of the proposed method, as well as its superiority to state-of-the-art registration approaches.

In order to better handle the coupling effects when controlling multiple-input multiple-output (MIMO) systems, taking the decentralized control structure as the basis, this paper proposes a sparse control strategy and a decoupling control strategy. Type-1 and type-2 Takagi-Sugeno (T-S) fuzzy models are used to describe the MIMO system, and the relative normalized gain array-based criterion is employed to measure the coupling effects. The main contributions include: 1) compared to the previous studies, a manner with less computational cost to build fuzzy models for the MIMO systems is provided, and a more accurate method to construct the so-called effective T-S fuzzy model (ETSM) to express the coupling effects is developed; 2) for the sparse control strategy, four indexes are defined in order to extend a decentralized control structure to a sparse one. Afterward, an ETSM-based method is presented that a sparse control system can be realized by designing multiple independent single-input single-output (SISO) control loops; and 3) for the decoupling control strategy, a novel and simple ETSM-based decoupling compensator is developed that can effectively compensate for both steady and dynamic coupling effects. As a result, the MIMO controller design can be transformed to multiple noninteracting SISO controller designs. Both of the sparse and decoupling strategies allow to use linear SISO control algorithms to regulate a closely coupled nonlinear MIMO system without knowing its exact mathematical functions. Two examples are used to show the effectiveness of the proposed strategies.

Virtual reality (VR) is regarded as a useful tool for teleoperation systems and provides operators with immersive visual feedback on the robot and the environment. However, without any haptic feedback or physical constructions, VR-based teleoperation systems normally suffer from poor maneuverability, and operational faults may be caused in some fine movements. In this article, we employ mixed reality (MR), which combines real and virtual worlds, to develop a novel teleoperation system. A new system design and control algorithms are proposed. For the system design, an MR interface is developed based on a virtual environment augmented with real-time data from the task space with the goal of enhancing the operator's visual perception. To allow the operator to be freely decoupled from the control loop and offload the operator's burden, a new interaction proxy is proposed to control the robot. For the control algorithms, two control modes are introduced to improve the long-distance movements and fine movements of the MR-based teleoperation system. In addition, a set of fuzzy-logic-based methods are proposed to regulate the orientation, position, velocity, and force of the robot to enhance the system's maneuverability and address potential operational faults. A barrier Lyapunov function and a backstepping method are leveraged to design the control laws and simultaneously guarantee the system's stability under state constraints. Experiments conducted using a six-degree-of-freedom robotic arm prove the feasibility of the system.

This article proposes a new single master bimanual teleoperation system with an efficient position, orientation, and force regulation strategy. Unlike many existing studies that solely support motion synchronization, the first contribution of the proposed work is to propose a solution for orientation regulation when several slave robots have differing motions. In other words, we propose a solution for self-regulated orientation for dual-arm robots. A second contribution in the article allows the master with fewer degrees of freedom (DoF) to control the slaves (with higher DoF), while the orientation of the slaves is self-regulated. The system further offers a novel force regulation that enables the slave robots to have a smooth and balanced robot-environment interaction with proper force directions. Finally, the proposed approach provides adequate force feedback about the environment to the operator and assists the operator in identifying different motion situations of the slaves. Our approach demonstrates that the forces from the slaves will not interrupt the operator's perception of the environment. To validate the proposed system, experiments are conducted using a platform consisting of two 7-DoF slave robots and one 3-DoF master haptic device. The experiments demonstrated good results in terms of position, orientation, and force regulation.

This paper develops an innovative multilateral teleoperation system with two haptic devices on the master side and a newly designed reconfigurable multi-fingered robot on the slave side. A novel nonsingular fast terminal sliding-mode algorithm, together with varying dominance factors for cooperation, is proposed to offer this system's fast position and force tracking, as well as an integrated perception for the operator on the reconfigurable slave robot (manipulator). The Type-2 fuzzy model is used to describe the overall system dynamics, and accordingly a new fuzzy-model-based state observer is proposed to compensate for system uncertainties. A sliding-mode adaptive controller is designed to deal with the varying zero drift of the force sensors and force observers. The stability of the closed-loop system under time-varying delays is proved using Lyapunov-Krasovskii functions. Finally, experiments to grasp different objects are performed to verify the effectiveness of this multilateral teleoperation system.

Interaction measure determines decentralized and sparse control configurations for a multivariable process control. This paper investigates interval type-2 Takagi-Sugeno fuzzy (IT2TSF) model based interaction measures using two different criteria, one is controllability and observability gramians, the other is relative normalized gain array (RNGA). The main contributions are: first, a data-driven IT2TSF modeling method is introduced; second, explicit formulas to execute the two measures based on IT2TSF models are given; third, two interaction indexes are defined from RNGA to select sparse control configuration; fourth, the calculations to derive sensitivities of the two measures with respect to parametric variations in the IT2TSF models are developed; and fifth, the discussion to compare the two measures is presented. Three multivariable processes are used as examples to show that the results calculated from IT2TSF models are more accurate than that from their type-1 counterparts, and compared to gramian-based measure, RNGA selects more reasonable control configurations and is more robust to the parametric uncertainties.