A new planning and control scheme for multirobot coordination is presented. First, the event-based planning and control theory is introduced. The most important step is the design of an event-based motion reference for the multirobot system. It drives the system to achieve the best possible coordination. Hybrid position/force controllers which are able to perform a large class of tasks are designed based on the combination of general task space with the well-known nonlinear feedback linearization technique. To improve the force control performance, the dynamics of joint motors have been considered in the force control. For a given task, a task projection operator can be found for each robot with the consideration of redundancy management. It projects the feedback linearized model to the actual task space. A distributed computing architecture is proposed to implement this scheme in a parallel computation. The event-based coordination scheme was experimentally implemented and tested for the coordinated control of two 6 DOF PUMA 560 robots with very good results.

A new intelligent fusion scheme for human operator command and autonomous planner/controller in telerobotic system has been developed based on the event-based planning and control theory. It provides a unified model to integrate a human operator control command with action planning and control of autonomous operation. As a result, the telerobotic system can perform tasks which cannot be done by either human operator or autonomous planner/controller alone. This scheme lays down a foundation for planning and control of a general robotic system involving human operators, and provides a natural and efficient way to fuse the human intelligence with the machine intelligence. The scheme is implemented and tested on a PUMA 560 robotic system. The experimental results of obstacle avoidance, and hybrid force/position control with a commanded force are presented.

This paper describes a display and control methodology for combining (or fusing) different multidimensional sensor data to guide the performance of telerobotic contact or near-contact tasks successfully in both manual and supervised automatic modes of control. Success is defined as a mapping of control goals or subgoals into a multidimensional data space. Several implemented examples are presented and illustrated. The methodology can also be extended to virtual reality simulation of telerobotic task scenarios through proper physical modeling of sensors.

A new function-based control sharing scheme for robotic systems has been developed based on the event-based planning and control theory. It provides a unified model to integrate a human operator control command with action planning and control of autonomous operation. As a result, the robotic systems can perform tasks which cannot be done by either human operator or autonomous planner/controller alone. This scheme lays down a foundation for planning and control of a general robotic system involving human operators, and provides a natural and efficient way to fuse the human intelligence with the machine intelligence. The scheme is implemented and tested on a PUMA 560 dual-arm system. The experimental results of obstacle avoidance, hybrid force/position control with a commanded force as well as dual-arm coordinated teleoperation are presented.

Using semi-algebraic description of task level configuration space (TLCS) and calculus of variations, trajectory plans avoiding the otherwise invisible, TLCS obstacles, arising from the kinematic constraints due to the joint limits are synthesized. This computationally tractable trajectory planning consists of shortest and smooth geometric path planning satisfying kinematic constraints, and event-based motion planning. As opposed to the conventional algorithmic approaches to the path planning, this appealing closed form solution of the path and motion planning, and real time control is easily applicable to computer integrated manufacturing applications. The validity of this synthesis along with a real time control is demonstrated through simulation and experiment on a PUMA 560 manipulator.>

A novel event based motion planning method, together with nonlinear feedback algorithm is applied to the controlled transition between unconstrained and constrained motion of a rigid robotic manipulator. The motion planner is driven by a task related scalar motion reference. This reference is a function of the actual manipulator position in the unconstrained directions and a function of the interaction force in the constrained directions. Impact is considered a discrete event, separating different operation modes. The dynamics of the arm is decoupled with respect to a path-relative representation of manipulator outputs. The feedback control law directly servos these outputs. The event based planning and control scheme was experimentally implemented for impact control and compared with impedance control and explicit force control.>

This paper describes a distributed computing and control architecture for experiments in dual-arm sensor-based planning and control. The system comprises of two PUMA 560 robots with state-of-the-art joint-level motion controllers. These controllers have accurate joint position and velocity measurement, an accurate joint calibration system, and six-axis wrist force/torque sensors with optical fiber communication with the controller. A high-power Silicon Graphics workstation, through a shared memory interface with the joint-level controllers, provides a task-level command and control capability. A variety of control schemes can be easily implemented on this system. A servo rate of 1000 Hz, without interpolation, has been achieved for task-level servo schemes with force feedback. The experimental results presented here demonstrate the performance capabilities of the system.>

This paper briefly describes an advanced teleoperator (ATOP) system and its control station where a variety of computer-based operator interface devices and techniques are integrated into a functional setting, accommodating a primary operator and secondary operators. Computer graphics is a key operator interface component in the control station where new types of manual interface devices also are employed. The results of some generic and application task experiments are summarized, including the performance of a simulated remote satellite servicing task, carried out under four to eight seconds communication time delay, using satellite TV and Internet computer communication links. In conclusion, the paper highlights the lessons learned so far.>

Based on our newly developed event-based planning and control theory, a new action planning and control scheme for the telerobotic system is presented. It provides a unified model to integrate a human operator control command (supervisory command) with motion planning and control for automatic operation. As a result, supervisory command and autonomous motion planning and control are naturally combined together such that the autonomous motion plan can be adjusted and modified by human operator during its execution. This scheme lays down a foundation for planning and control of a general robotic system involving human operators. The scheme is implemented and tested on a PUMA 560 dual-arm system. A Spaceball is used by the human operator as a command generator to input the supervisory command. The experimental results of both single arm teleoperation and dual-arm coordinated teleoperation are presented.>

Successful execution of a planned task using robotic manipulators, requires that the task satisfies both the kinematic and dynamic abilities of the manipulator. The executability of a task is generally unknown a priori. In this paper, the dynamic ability of a manipulator for motions constrained by the holonomic constraints and by the loading in the object maneuver are considered. Simulation and experimental results for holonomically constrained motions are presented to illustrate the usefulness of the presented formulation in analyzing the task executability and preventing manipulator damage for some tasks, especially, those involving environmental interaction.>

Recent advances in real-time 3-D computer graphics and hierarchically structured graphical user interfaces (GUI) enable development of very efficient graphics-based operator interfaces for telerobotic operations. This paper addresses potential benefits of such advanced graphics interfaces to telerobotic servicing and inspection, emphasizing operator efficiency and system safety. Specifically, the authors report on calibrated 3-D graphics displays and GUIs developed and applied to an advanced teleoperation system and a remote surface inspection system at the Jet Propulsion Laboratory.

A highly effective predictive/preview display technique for telerobotic servicing in space under several seconds communication time delay has been demonstrated on a large laboratory scale. The technique is based on a high-fidelity calibration procedure that enables a high-fidelity overlay of 3-D graphics robot arm and object models over given 2-D TV camera images of robot arm and objects. To generate robot arm motions, the operator can confidently interact in real time with the graphics models of the robot arm and objects overlaid on an actual camera view of the remote work site. The technique also enables the operator to generate high-fidelity synthetic TV camera views showing motion events that are hidden in a given TV camera view or for which no TV camera views are available. The positioning accuracy achieved by this technique for a zoomed-in camera setting was about +or-5 mm, well within the allowable +or-12 mm error margin at the insertion of a 45cm long tool in the servicing task.>

The problem of rigid object handling by multiple robot arms is investigated. The primary goal is to make the object exhibit a prescribed behavior while in contact with a fully known environment. Point contacts are assumed between the object and the arms. The aspect of task definition to achieve decoupling and linearizing control laws is discussed. Control laws are first formulated at the object level to provide decoupled force and position servo loops. It is then used to form control laws for the individual arms. Redundancies exist at the object and arm levels. The object level redundancy is used to achieve secondary goals in object handling. The arm level redundancies are the zero dynamics and can be controlled by redundant inputs. Full use of the available inputs are used to control the system as a whole. Numerical simulations for a dual-arm situation illustrate the validity of the approach.>

In this paper, we focus on the role and im- portance of experiments in the development of tele- operation systems. Experiments conducted in the JPL Advanced Teleoperation Laboratory and les- sons we have learned from them are described. We also describe novel procedures and techniques that facilitate the conduct of experiments and these are presented in this paper. Also reported are problems faced in a laboratory that develops new teleoperation technology while concurrently validating the tech- nology with experiments.

Abstruct- This paper presents new robot aim1 planiiiiig and control schemes compatible with sensing based planning and control. These developiiieiits are a central component of robot arm coiitrol intelligence. The sclieines iise pliase space (velocity versus position) representation of robot arm motion in tlie task space, and apply time and energy optimization tecliiiiqiics to deteriiiiuc non-time based trajectories for giveii geometric path in tlie task space. A new pliasc space crror defiiiitioii aiid coiripiitatioii liavc been introtliiced aiid combined witli kiiowii nonlinear fcetlback control law, whicli linearizes and decoiiples the control in tlie task space. Tlic! system stability lias been proveii. Tlie new phase space plaiiiiiiig aiid coiitrol schemes were experiiiieiitally iiiipleiiieiited and tested on the 3-DOF positioii coiitrol of a PUMA 560 robot arm with very good results. I. INTRODIICTION

Shared compliant control has been incorporated into an advanced six-degree-of-freedom force-reflecting telemanipulation system. With this system the authors have investigated the effect of time delay on human telemanipulation task performance. Time delays of between 2 and 4096 ms were introduced between master and slave arms, and high-precision peg-in-hole tasks were performed by six test operators with two modes of control: kinesthetic force feedback (KFF) and shared compliant control (SCC). Task performance was quantified in terms of the completion time (CT) and the sum of square forces (SOSF). The experimental results demonstrate the superiority of SCC over KFF for time-delayed telemanipulation. SCC has significantly lower rates of increase than KFF in both CT and SOSF with time delay. Only SCC enabled task performance at delays above 1 s. Constant force maintenance tasks were also performed to investigate the effect of time delay on the stability of force reflection. SCC also has beneficial effects on telemanipulation without time delay.>

A manipulator motion generator algorithm is described. It is based on observations of task space trajectory profiles produced by human operators manually through a position hand controller. The algorithm, the harmonic motion generator (HMG), is formulated in the velocity-position phase space and employs harmonic (sinusoidal) base functions in that space. The reported performance results include combinations of this HMG algorithm with force-torque sensor based active compliance control and with Cartesian servo. The results are encouraging.>

Parallel computation of manipulator forward dynamics is investigated. Considering the three classes of algorithms for the problem, the authors show that the O(n/sup 3/) algorithms are the most efficient for parallel computation. Parallel algorithms for computation of inertia matrix, the bias vector, and the linear system solution are developed which can be efficiently implemented on a unique architecture, a triangular array of n(n+1)/2 processors with a nearest neighbor interconnection. This architecture is highly suitable for VLSI and WSI implementation.>

The discrete time theory that feedback linearizes and output decouples the coupled nonlinear dynamics of the PUMA 560 robot arm is systematically developed. The effect of sampling on the performance of the arm has been studied and shown through experiments. Certain restrictions that are necessary on the sampling time of the system are shown. The results obtained have been experimentally verified. Since a number of researchers have addressed the problem of loss of feedback linearizability under sampling, it was important to develop theories and obtain results in discrete time that take into consideration the effects due to sampling. This problem is addressed.>

A method for achieving redundant arm kinematic control based on parameterization is presented. The method, referred to as the parameterization method, parameterizes properly chosen redundant joints, and obtains a close form of parameterized inverse kinematic solution to accomplish position-based kinematic control. The parameterization of redundant joints is based on: (1) selecting a point in the solution manifold of redundant joints that is closest to one of the reference points, where the reference points represent desirable arm configuration in the parameter space; and (2) adjusting the selected parameter values in the parameter space for the optimization of performance specified by a potential field over the joint space. The proposed method provides the computational efficiency and numerical stability, as well as the capability of trajectory planning in the joint space for redundant arm kinematic control. The case study demonstrates that the parameterization method can be successfully applied to the position-based kinematic control of the eight-degree-of-freedom redundant arm.>

The trajectory tracking performance of the nonlinear feedback controller based on differential geometric control theory was experimentally studied. The performance is compared to three other methods: independent joint proportional-derivative (PD) control, independent joint PD control with feedforward torque computations, and the computed torque method. In the nonlinear feedback controller, servo is in the task space. In the other three methods servo is in the joint space. Path planning is in the task space. Circular and straight line paths are considered. A sampling frequency of 200 Hz is used. The effect of sampling frequency on tracking error is studied for the nonlinear feedback control scheme. The performance of these methods was tested on a PUMA 560 arm. The trajectory tracking performance of the robot arm under the nonlinear feedback controller is comparable to that of the computed torque control schemes.>

The authors focus on the investigation of maximum applicable force/torque by multiple robot arms in coordinated motion, i.e., the dynamic workspace analysis of multiple cooperating robot arms. Simple procedures to solve the dynamic workspace analysis problem for multiple cooperating robot arms with or without the internal force/torque constraint are presented. These procedures model the cooperating robot arms as a closed mechanical chain and are based on the theory of linear transformations and the properties of mechanics. For low degree-of-freedom robot arms, the maximum force/torque problem can be solved by geometrically using the proposed procedures, as illustrated by examples.>

A nonlinear feedback robot controller that incorporates the robot manipulator dynamics and the robot joint motor dynamics is proposed. The manipulator dynamics and the motor dynamics are coupled to obtain a third-order-dynamic model, and differential geometry control theory is applied to produce a linearized and decoupled robot controller. The derived robot controller operates in the robot task space, thus eliminating the need for decomposition of motion commands into robot joint space commands. Computer simulations are performed to verify the feasibility of the proposed robot controller. The controller is further experimentally evaluated on the PUMA 560 robot arm. The experiments show that the proposed controller produces good trajectory tracking performances and is robust in the presence of model inaccuracies. Compared with a nonlinear feedback robot controller based on the manipulator dynamics only, the proposed robot controller yields conspicuously improved performance.>

An enhanced teleoperation technique for time-delayed bilateral teleoperator control is discussed. The control technique selected for time delay is based on the use of a high-fidelity graphics phantom robot that is being controlled in real time (without time delay) against the static task image. Thus, the motion of the phantom robot image on the monitor predicts the motion of the real robot. The real robot's motion will follow the phantom robot's motion on the monitor with the communication time delay implied in the task. Real-time high-fidelity graphics simulation of a PUMA arm is generated and overlaid on the actual camera view of the arm. A simple camera calibration technique is used for calibrated graphics overlay. A preliminary experiment is performed with the predictive display by using a very simple tapping task. The results with this simple task indicate that predictive display enhanced the human operator's telemanipulation task performance significantly during free motion when there is a long time delay. It appears, however, that either two-view or stereoscopic predictive displays are necessary for general three-dimensional tasks.>

The nonlinear feedback method of robot control has been experimentally implemented on two PUMA 560 robot arms. The feasibility of the proposed controller, which is already known to be viable, is stressed. The servomechanism operates in task space, and the nonlinear feedback takes care of the necessary transformations to compute the necessary joint currents. A discussion is presented of the implementation with details of the experiments performed. The performance of the controller is encouraging but was limited to 100 Hz sampling frequency and to derived velocity information at the time of the experimentation. The setup of the lab, the software aspects, results, and the control hardware architecture that has recently been implemented are discussed.>

Hamilton's principle is used to derive the dynamic model for a large class of flexible robot arms. The resultant dynamic model consists of a system of partial differential-integral equations and the dynamic boundary conditions associated with it. Some properties of the model are observed and its application to control is discussed. This model represents an infinite-dimensional nonlinear dynamic system and yet can be turned into a finite-dimensional system that could be obtained by modal expansion, if it is desired. This provides more flexibility for control purposes as well as for the analysis of the system.>

An efficient method of computation of the manipulator inertia matrix is presented. Using spatial notations, the method leads to the definition of the composite rigid-body spatial inertia, which is a spatial representation of the notion of augmented body. The previously proposed methods, the physical interpretations leading to their derivation, and their redundancies are analyzed. The proposed method achieves a greater efficiency by eliminating the redundancy in the intrinsic equations as well as by a better choice of coordinate frame for their projection. In this case, removing the redundancy leads to greater efficiency of the computation in both serial and parallel senses.>

Parallel and parallel/pipeline algorithms for computation of the manipulator inertia matrix are presented. An algorithm based on the composite rigid-body spatial inertia method, which provides better features for parallelization, is used for the computation of the inertia matrix. Two parallel algorithms are developed which achieve the time lower bound in computation. Also described is the mapping of these algorithms with topological variation on a two-dimensional processor array, with nearest-neighbor connection, and with cardinality variation on a linear processor array. An efficient parallel/pipeline algorithm for the linear array was also developed and has significantly higher efficiency.>

A class of parallel and parallel/pipeline algorithms for computation of the manipulator inertial matrix is presented. An algorithm based on the composite rigid-body spatial inertia method, which results in less data dependency and hence better parallelization efficiency, is used for computation of the inertia matrix. Two parallel algorithms are developed which achieve the time lower bound of O((log/sub 2/ n))+O(1) in the computation with O(n/sup 2/) processors. The architectural features required for perfect mapping of these algorithms and their communication complexity are analyzed. The performance of the algorithms when mapped on two- and one-dimensional (linear) processor arrays with nearest-neighbor connection is investigated. Mapping on the linear array results in new algorithms with a computational complexity of k/sub 1/n(log/sub 2/n)+k/sub 2/(log/sub 2/n)+k/sub 3/. A parallel/pipeline algorithm is also presented which achieves the computation time of k/sub 1/n+k/sub 2/(log/sub 2/ n)+k/sub 3/ on the linear array. An architecture-oriented approach is used in the design of the algorithms.>

Symbolic inversion of the Jacobian matrix for spherical wrist arms is investigated. It is shown that, taking advantage of the simple geometry of these arms, the closed-form solution of the system Q=J/sup -1/X, representing a transformation from task space to joint space, can be obtained very efficiently. The solutions for PUMA and Stanford arms and a six-revolute-joint coplanar arm, along with all singular points, are presented. The solution for each joint variable is found as an explicit function of the singular points which provided a better insight into the effect of different singular investigated points on the motion and force exertion of each individual joint. For the arms investigated, the computation cost of the solution is the same order as the cost of forward kinematic solution and it is significantly reduced if a forward kinematic solution is already obtained. A comparison with previous methods shows that this method is the most efficient to date.>

Based on a differential geometric feedback linearization technique for nonlinear time-varying systems, a dynamic force control method for robot arms is developed. It uses active force-moment measurements at the robot wrist. The controller design fully incorporates the robot-arm dynamics and is so general that it can be reduced to pure position control, hybrid position/force control, and pure force control. The controller design is independent of the tasks to be performed. Computer simulations show that the controller improves the position error by a factor of ten in cases in which position errors generate force measurements. A theorem on linearization of time-varying system is also presented.>

A space telerobot system may contain forty or more motors under computer control. A Universal Computer Control System (UCCS) is under development at the Jet Propulsion Laboratory (JPL) for all motor elements of a space telerobot. The basic hardware architecture and software design of UCCS are described together with the rich motor sensing, control and self-test capabilities of this new all-computerized motor control system. UCCS is integrated into a multibus computer environment with direct interface to higher level control processors, uses PWM power amplifiers, and one unit can control up to sixteen different motors simultaneously at a high I/O rate. UCCS performance capabilities are illustrated by a few data. UCCS is now used with customized software at JPL for telerobot research, development and future demonstrations.

A new device for human operator control of a robotic gripper has been developed and preliminary evaluation has been performed. The JPL Force Reflecting Hand Trigger system features: an instrumented index finger trigger with load cell detection of finger force. A servo controlled, lead screw driven backdrive capability by which the trigger's position can be made to follow that of the remotely controlled gripper. And a novel feedback mechanism by which clamping force or some other signal can be fed back via a swiveling motion, also servo controlled, of the trigger surface (force reflection). This system has undergone preliminary testing in which the amount of force reflection is varied and dynamic force tracking response is observed.

In this paper the dynamic equation describing two cooperating robot arms which simultaneously are working on the same object is established by considering the whole system as a closed kinematic chain. This formulation has the advantage of automatically handling the coordination and load distribution between the robot arms through the dynamic equation. It can also be generalized to a multi-robot arm system. The control problem is solved as a design problem for a linear system by first applying a nonlinear feedback and nonlinear coordinate transformation to linearize the nonlinear dynamic equation, and then employing the linear optimal control theory to design a robust controller in the task space. This new dynamic control method establishes a direct control response to task space commands facilitating dynamics based task encoding in robotics.

A modular and expandable distributed microcomputer hardware and software system is described matching the natural needs of real-time mechanization of manipulator control in space teleoperation. The natural needs include: (a) distribution of real-time control computing power between control station and remote robot arms equipped with smart end effectors and (b) the use of interchangeable control techniques like (i) generalized force-reflecting hand control, (ii) sensor-referenced automatic control and (iil) supervisory control with interface to high level task planning control. The distributed microcomputer control system is also handling computer graphics displays and the control of TV cameras. The paper describes the major components of the modular control software system including inter-processor communication and control mode selection. A simulation example is quoted to verify control hardware/software performance and to aid human operator control performance studies.

A new control method is presented for the coordinated control of two robot arms based on exact system linearization by appropriate non-linear feedback. The two arms are assumed to be working on the same object simultaneously. The control method uses a dynamic coordinator acting on relative position and velocity error and/or on relative force-torque errors between the two arms. Simulation results are presented for the coordinated control of two PUMA 560 robot arms using an optimal dynamic coordinator. For completeness, the paper also presents (i) a new revised set of inertia parameters for all six links of PUMA 560 robot arm, (ii) the form of nonlinear feedback and diffeomorphic state transformation which linearizes the control of PUMA 560 working in all six degrees of freedom, and (iii) the reduced numeric form of all dynamic equations for PUMA 560 working in three (positioning) degrees of freedom.

A new dynamic control method is discussed for robot arms. It is based on nonlinear feedback and diffeomorphic (state) transformation which externally (or exactly) linearizes the whole system and provides simultaneous output decoupling. The non-linear feedback is augmented with optimal error correcting controller, which operates on the task error level. Several computer simulations were performed to evaluate the performance of the new dynamic control method. The simulation results are discussed in detail. The simulations show that the new dynamic control method provides very good tracking performance even in cases when the dynamic model parameters of the robot arm are only known with 10-30% error. The functional characteristics of the nonlinear feedback law and optimal error correcting controller are discussed briefly.

This paper presents the conceptual design, analysis, synthesis and software organization of an advanced teleoperator control system with sensory feedback. The design requirements for the system are discussed in detail and an implementation strategy is presented. The resulting system features maximum autonomy of the local hand controller and remote manipulator subsystems, along with kinematic and dynamic coordination between these subsystems. The final design emphasizes cooperation and interaction between the human operator and the computers in control of the sensor-based manipulator system. The hardware and software modules being used to implement the system at JPL are described.