A discrete event controller synthesis methodology for the successful convergence of assembly tasks is presented. The modelling of an assembly process as a hybrid dynamic system has been shown to be a very effective strategy to incorporate both the continuous and discrete natures of the interaction between the workpiece and its environment. Prior works have presented separate controller synthesis methodologies for both velocity- and force-controlled systems and here we present the next step, a hybrid force/velocity controller. Force control is a natural paradigm for assembly as it is fundamentally compliant and velocity control is used to ensure that frictional forces are overcome during sliding motions. Experimental results for a four degree of freedom assembly task are presented, demonstrating the robustness of the system to very large positioning and orientation errors.

Presents an approach to the management of perception in a multi-sensor robotic system. The approach is based around an adaptive sensory perception controller which has two significant characteristics: (1) it performs real-time selection of process monitors based on the minimisation of the expected cost of perception with constraints on the uncertainty of perception, and (2) it performs online adaptation of the perception models on which the monitor selection is based. The approach is shown to be useful and effective in experiments involving a range of sensing modalities which may typically be encountered in robotic applications.

Models of perception are an important element in the control of sensory perception in autonomous systems. The performance of a perception controller will depend on how well the models reflect the time-varying performance characteristics of sensors and data processing algorithms. A novel approach to achieving high quality models through real-time adaptation is presented. Models reflecting observation uncertainty are adapted in accordance with online sensor performance using a radial basis function approach modified to allow real-time operation.

This paper presents a novel approach to constructing a task level plan for an assembly task from demonstration. A hybrid dynamic system is chosen as an attractive way to model assembly tasks. We propose a framework where the mappings required within the hybrid dynamic system model are extracted from a demonstration. We extract the event path planner mapping, which determines a task level plan. Demonstrated paths are broken down into their base elements, called transitions. A path plan is then constructed from transitions that were well demonstrated. We test the approach by having a robot execute an assembly task using constructed task plans. The framework produces excellent results because: 1) the robot can perform selected paths better than when it simply copies the demonstrator, and 2) it allows flexible path selection so the robot can be given a disposition in how it executes the task.

Presents the fundamentals of a hybrid dynamic framework applied to constrained manipulation. The formalised modelling and control synthesis are given. Additionally, a method of task-level adaptation, suitable for the online training of assembly lines, is detailed. Importantly, the method is applied and demonstrated on an industrial-standard assembly with very tight tolerances. A gear mechanism for a starter motor is successfully assembled. Furthermore, the online adaptation method is used to train the robots and improve the assembly of the gear mechanism. The results are very positive, encouraging further development of the hybrid dynamic framework.

A systematic approach to the modeling of deformable rodlike objects is presented. Various rodlike objects such as cords and wires are manipulated in many manufacturing processes. In such processes, it is important for successful manipulation to evaluate their shapes on a computer in advance because their shapes can be changed easily and their deformation often shows hysteresis properties. In this paper, we develop an analytical method to model the shape of deformable rodlike objects including hysteresis properties. First, we investigate the mechanism of hysteresis. Second, the potential energy of a rodlike object and the geometric constraints imposed on it are formulated. The shape of the object can be derived by minimizing the potential energy under the geometric constraints. Thirdly, a procedure to compute the shape of a deformed rodlike object is developed by applying a non-linear programming technique. Finally, we show some numerical examples with hysteresis using our proposed method.

Human integration is essential in systems where autonomous control alone would not be successful. In this paper, we present a framework for integrating a human supervisor into an otherwise autonomous control system. To facilitate integration, discrete event systems theory is adopted to model human interactions. It is via this interaction model that human commands can be combined with commands from an automated control system. For control synthesis, a method based on constraints is being used to generate velocity commands from the autonomous task level controller. The constraints are also utilized to limit human input so that erroneous human input is minimized. The methods are demonstrated by experiments.

We discuss the relationship between a sensory perception controller and traditional data fusion techniques. The perception controller selects process monitors in real-time, based on the expected uncertainty and cost. We experimentally compare its operation to traditional data fusion methods and show how to combine the two for improved performance.

We present an approach to managing the quality and cost of perception in a multi-sensor robotic system. The approach involves prediction of monitoring errors of low-performance process monitors using a weighted least squares algorithm, and detection of instances when high-performance process monitoring is necessary. Two significant characteristics of the approach are (1) dynamic, real-time management of process monitors, and (2) the ability to deliver high quality information while keeping the cost of perception low.

For a visually impaired person, walking without assistance from something or someone is difficult. We use a robotic cane to demonstrate the operation of a shared control framework in order to assist a user in walking through an obstacle rich environment. As the device would normally avoid all obstacles it would be difficult for the user to approach chairs or closed doors. It would also be difficult for the user to reach an overall goal as the robotic aid does not "know" what the goal is. Therefore shared control is important so that the user can override or compromise with the autonomous command so that the user is helped not impaired by the device. Here, a shared control framework including a model of human interaction is demonstrated by experiments.

We present an approach to the control of sensory perception. It involves real-time selection of process monitors based on the expected value of the monitoring operations. Appropriate definitions of the value of monitoring are a key element of the approach. The sensory perception control methodology is evaluated through simulation of an environment mapping task.

Presents a framework for robot programming by human demonstration. The framework builds a high level robot controller using information extracted from the demonstration. The high level robot controller is broken down into three component parts, each fulfilling a different function during execution. The paper focuses on the construction of an event path planner, which determines the optimal event path of a task. The approach varies the optimal path according to what characteristics of the demonstrations are stressed, thus giving the robot a selected disposition. The approach was implemented on a simple navigational task. The event path planner selected appropriate paths and could change its selection according to what characteristics were desired in the selected path.

The problem of controlling sensory input and perception for use in mobile navigation is addressed in this paper. The proposed solution offers several advantages compared to existing methods in the literature. First, the proposed solution is based on a stochastic dynamic programming algorithm which guarantees cost-efficiency of the real-time sensory perception controller (SPC) by solving a constrained optimisation problem. Second, a new and unique discrete event model of mobile navigation is presented. The model is task-independent and can be used in a wide range of navigation problems. Third, the discrete events become a natural common representational format for the sensors, which further extends the applicability of the proposed solution. Fourth, the sensing aspect of discrete event systems has often been neglected in the literature. In this paper we present a unique approach to online discrete event identification.

A new, complete discrete event framework for modelling and control of mobile robots is presented. Discrete event modelling is the natural paradigm for systems where motion is constrained by the presence of obstacles. As the mobile robot encounters obstacles the system dynamics change dramatically. These changes are explicitly represented by discrete events. The discrete event framework supports the use of existing path planning techniques as well as reactive approaches and considers the dynamics of the robot. A number of discrete event models for mobile robots have been proposed but these use rather arbitrary definitions for the discrete states. Instead, we rigorously define the discrete state from the currently active constraints upon the robot. Experimental results demonstrating the application of the discrete event model for path planning and control of a nonholonomic mobile robot in indoor office navigation tasks are presented.

In this paper we address the problem of controlling sensory perception for use in discrete event feedback control systems. The process of controlling sensory perception has two main objectives: 1) to collect perceptual information to identify events with high levels of confidence and 2) to keep the sensing costs low. Several event recognition techniques have been developed by the authors, where each of the event recognisers produces confidence levels of recognised events. For a discrete event control system running in normal operation, the confidence levels are typically large and only a few event recognisers are needed. Then, as the event recognition becomes harder, the confidence levels will decrease and additional event recognisers are utilised by a sensory perception controller (SPC). Hence, the overall system has the ability to actively control the use of sensory input and perception to achieve high performance discrete event recognition. Two applications are presented in the paper: 1) robotic assembly and 2) mobile navigation. Both applications fit well in the discrete event framework and demonstrate the benefits of sensory perception control.

Human integration into discrete event control systems is considered by utilising potential fields for control synthesis. Human integration into a control system is of importance in. Many situations, including those in which limited sensor information about the environment or the task is available. The framework presented allows the sharing of human commands with commands from an automated control system. Potential fields are being used as a tool to generate velocity commands from an autonomous task level controller as well as allowing the human to interact. The potential fields are also utilised to constrain human input such that human input error is minimised. The ideas presented here are supported by experiments.

A new discrete event controller synthesis methodology for the successful convergence of assembly tasks is presented. The modelling of an assembly process as a hybrid dynamic system has been shown to be a very effective strategy to incorporate both the continuous and discrete natures of the interaction between the workpiece and its environment. Prior works have presented controller synthesis methodologies for velocity-controlled systems (McCarragher and Asada, 1995) and here we follow a similar structure for force-controlled systems. Force control is a natural paradigm for assembly as it is fundamentally compliant. Compliance reduces sensitivity to positioning errors, which, are the most common, source of failure in assembly. A non-trivial example is given which demonstrates the effectiveness of this method.

A new technique for selecting, in real time, different sensing techniques for a robotic system has been developed. The proposed method is based on stochastic dynamic programming, which provides an effective solution to multi-stage decision problems. At each stage in the decision process a sensor selection controller has the option of consulting a new process monitoring technique to improve the knowledge of the task or terminating the decision process without any further information gathering. The sensor selection controller has been successfully implemented for the real-time control of a planar robotic assembly task in a discrete event control framework. One of the monitoring methods used is based on hidden Markov models, where the average recognition rate was 87%. The rate of 87% was chosen to show the effectiveness of the dynamic sensor selection method. The experiments show that the method performs better than any individual process monitor. A successful event recognition rate of 97% with an average CPU time of 0.38 seconds is achieved when two force monitors and one position monitor are available to the sensor selection controller.

A unified method for the planning and control of tasks involving flexible parts is applied to two case studies: the bending of sheet metal and the insertion of a beam into a hole with friction. The method is formulated on a hybrid position/force framework so as to exploit the advantages brought about by load elasticity. The unifying design guideline is the maintenance of deformations on the load so that a region of reduced stiffness is presented to the force controller. The main focus of this paper is the experimental results obtained from the implementation of the tasks.

Shared control between a human supervisor and an autonomous discrete event control system is the focus of this paper. Two task must be considered simultaneously: 1) restricting the human input; and 2) determining the autonomous control commands. For these purposes, methods utilising either state dependent constraints or potential fields are considered. Each of these methods has advantages over the other in providing the desired control function. This paper contrasts the methods as well as presenting experimental results demonstrating the effectiveness of each of the methods in a variety of tasks.

A new discrete event controller synthesis methodology for the successful convergence of assembly tasks is presented. Discrete event control has been shown to be a very effective strategy to incorporate both the continuous and discrete natures of an assembly task. A discrete event controller synthesis methodology for force-controlled systems is highly desirable because force control is naturally compliant and compliance reduces sensitivity to positioning errors. In this paper we present a number of experimental results for the new controller synthesis method, including a complete assembly task. These experiments demonstrate the effectiveness of the combination of discrete event control and force control for assembly tasks, successfully completing the assembly with positioning errors of up to 50 mm and orientation errors of up to 30/spl deg/.

A method for combining dynamic force and static position measurements for the monitoring of assembly is presented. A multilayer perceptron (MLP) network is used as a classifier where the individual network outputs correspond to contact state transitions occuring during the assembly process. When a contact state transition occurs, the MLP output with the largest value is chosen. The recognised contact state is sent to a discrete event controller which guides the workpiece through a series of contact states to the final desired configuration. The MLP has been successfully implemented on a Motorola 68040 based VxWorks board with successful recognition rates of 94.4% and 92.0% on a training set and an independent test set, respectively.

The problem of coordinating two arms for the manipulation of flexible materials is studied through the bending of a flexible beam. The solution proposed in this paper is based on a hybrid position/force approach. The choice of controlled variables in each direction is determined by minimum-effort optimality criteria. The bending effort is defined as the overall bending moment and force applied by the grippers. The control of forces exploits the material's elasticity, which is an impedance for the manipulators, The force controller then behaves as an admittance, in the form of an accommodation (inverse damping) law. Experimental results show the efficacy of the approach in achieving task requirements.

Programming robots by human demonstration is an effective method for task-level robot programming. Robot programming for contact tasks such as assembly, where both motion and forces need to be controlled, requires an understanding of the interaction with the environment and cannot effectively be accomplished by playing back recorded trajectories. Furthermore, since human demonstration is used to provide the motion and force trajectories, it is also important to include aspects of human movement control in the model of interaction with the environment. In this paper, we present a method of understanding and modeling contact tasks based on end-point stiffness using data obtained by human demonstration. The concept of end-point stiffness has been used both for explaining human movement control and for the control of robots. Experimental results demonstrate the effectiveness of the method in understanding and interpreting the data obtained by human demonstration of contact tasks. 1 Introd...

A discrete event controller (DEC) synthesis methodology for the successful convergence of assembly is presented. The modelling of an assembly process as a hybrid dynamic system (HDS) has been shown to be a very effective strategy to incorporate the interaction between the workpiece and its environment. In addition, this modelling has allowed the derivation of realistic conditions on the DEC for the convergence to the full assembly, known as p-convergence. In this paper the authors propose a DEC synthesis methodology resulting in an optimal p-convergence by combining the maintaining and enabling conditions proposed by McCarragher and Asada and dynamic programming. The applicability of the proposed synthesis methodology is then demonstrated for a dual peg insertion task.

Discrete event recognition based on force measurements in the frequency-domain, is presented. The force signals arise from interaction between the workpiece and the environment in a planar assembly task. The discrete events are modeled as hidden Markov models (HMMs), where the models are trained off-line with the Baum-Welch re-estimation algorithm. After the HMMs have been trained, we use them online in a robotic system to recognise discrete events as they occur. Event recognition with an accuracy as high as 98% was accomplished in 0.5-0.6 s with a relatively small training set.

A new approach to skill acquisition in assembly is proposed. An assembly skill is represented by a hybrid dynamic system where a discrete event controller models the skill at the task level. The output of the discrete event controller provides the reference commands for the underlying robot controller. This structure is naturally encoded by a hidden Markov model (HMM). The HMM parameters are obtained by training on sensory data from human demonstrations of the skill. Currently, assembly tasks have to be performed by human operators or by robots using expensive fixtures. Our approach transfers the assembly skill from an expert human operator to the robot, thus making it possible for a robot to perform assembly tasks without the use of expensive fixtures.

Assembly operations that involve flexible parts differ greatly from rigid-part assembly due to the capacity of the parts to comply with motions of the manipulators by yielding to the contact forces. This natural compliance can be exploited in the design of tasks involving the manipulation of flexible loads. Namely, using knowledge about the force vector field generated by conservative restoring forces of the load, a trajectory for the manipulator end-effector can be specified. The idea is illustrated through two case studies: an analysis of the problem of bending a flexible sheet, and experiments on an insertion task where a flexible beam has to be inserted in a hole with friction in the direction of insertion.

A new task-level adaptive controller which learns online is presented for the discrete event control of robotic tasks. Using a discrete event model of the assembly task, velocity constraints are derived from which desired velocity commands are obtained. Due to modelling errors and uncertainties, the velocity commands may result in a suboptimal, unwanted contacts between the workpiece and the environment. A task-level adaptive control scheme based on the occurrence of discrete events is used to change the model parameters from which the velocity commands are determined. Using Lyapunov stability theory, the adaptive control scheme is proved to converge to the actual model parameters. The method is applied to a starter motor assembly in an industrial setting. The results are excellent. The robot adapts its motion online to successfully assemble the starter motor. Moreover, the robot improves its motion with each subsequent trial. This online learning ability demonstrates the power of the discrete event approach to robotic manipulation.

Many robotic tasks are easily described using discrete event dynamic systems. However, the robot sensory and control systems operate in the continuous domain, leading to the problem of associating states of the continuous system with the states and events (changes in state) in the discrete task space. This paper presents a new approach to discretizing sensory data, based on discriminant functions and clustering techniques, for applications in robotic process monitoring and in interpreting human sensory data. The discriminant functions are learned from real sensory data, and hence the approach has the advantages of being adaptive, and also of taking into account various task parameters such as friction. Most importantly, the approach can be adapted quickly to different tasks by simply learning a new set of discriminant functions from sensory data corresponding to the task. Experimental results are presented to demonstrate the effectiveness of this approach.

A novel approach to the modeling and control of six degree-of-freedom (DOF) assembly tasks using polygonal pacts is presented. The assembly process is modeled as a discrete event system. Three key features of the assembly process are shown to be essential for a comprehensive understanding and effective control of the assembly task. It is also shown that discrete event modeling highlights these key aspects of the assembly process. Next the assembly process is broken down into its task primitives, or fundamental states of contact. It is shown that with only two primitives, the entire assembly process can be represented. These two task primitives are a surface-vertex contact and an edge-edge contact. The significance of this development is demonstrated. Petri nets are then used to give the discrete event model its necessary mathematical description for robotic control applications. The control methodology, again based on the two primitives, establishes a powerful framework which is robust and flexible in the control of assembly. Finally, simple experiments are discussed in which a successful 6-DOF assembly was completed.

A new approach to the planning and analysis of the successful convergence of assembly is presented. The convergence is referred to as p-convergence, since it is derived within a probability framework. The assembly is modeled as a hybrid dynamic system (HDS) accounting for the presence of the discrete change of contact and the continuous movement during the process. The continuous property is then employed to represent an assembly DES as a Markov chain such that p-convergence can be proposed. To derive conditions for p-convergence of the full assembly, methods for estimating the transition probability matrix are derived for the perfect case, the tracking error case, and the case of parameter uncertainty. It is shown that p-convergence for an assembly DES requires a control effort which is significantly more realistic for practical implementation than current methods presented in the literature.

Sufficient conditions for the success of the robotic control of assembly are presented. The assembly is modelled as a discrete event system using Petri nets. The abstraction of assembly to the discrete event system highlights the changes in the state of contact as the key points to control in the assembly process. Using this abstraction, sufficient conditions on the velocity commands are developed which guarantee successful parts mating. A method to derive robot velocity commands satisfying these conditions is derived. The method is then applied to a dual peg-in-the-hole problem.>

This paper presents an analysis of force data that was generated by a human demonstration of a complex and asymmetrical insertion task. The human was blindfolded. And only had force sensors (i.e. fingers) available. The process is modelled using hybrid dynamics which highlighted the changes in contact as the essential parts of the assembly process. Qualitative reasoning is then applied to the force signals to identify those portions of the force signals that corresponded to either a gain or a loss of contact. The emphasis of the paper is to understand the force signals generated by the human subject. It is shown that qualitative reasoning proves to be a simple, efficient and effective means to understand the force signal by identifying the changes in contact state. Qualitative reasoning is also shown to be suitable for automated industrial assembly.>

This paper presents an analysis of force and position data that were generated by a human demonstration of a complex and asymmetrical insertion task. The goal of this paper is to understand human sensing ability so as to apply similar techniques to robotics. The assembly process is modelled using hybrid dynamics which separates discrete states of contact and highlights the changes in contact as the essential parts of the process. The human was blindfolded and only had tactile sense available. Position, velocity and force data were recorded as generated by the human. Both stiffness and mechanical impedance are used to attempt to interpret the human generated data. Surprisingly, both of these methods are inadequate to describe any significant portions of the data. It is argued that classical approaches to the understanding of human manipulation skills are insufficient, and more advanced testing and analysis is needed to effectively understand human force sensing abilities.>

A task-level adaptive controller is presented for the discrete event control of robotic assembly tasks. Using a Petri net model of the assembly task, velocity constraints are derived from which desired velocity commands are obtained. Due to modeling errors and uncertainties, the velocity commands may result in a suboptimal, unwanted contacts between the workpiece and the environment. Simple vector addition is used to adapt the velocity commands so as to avoid the unwanted contacts. Gram-Schmidt orthogonalization is used to ensure that the adaptation does not violate any geometric constraints. An example is given with experimental results, demonstrating the viability and success of the adaptation method.

paper takes a new approach to robotic assembly, treating assembly as a discrete event system. A discrete event in assembly is defined as a change in contact state reflecting a change in a ge- ometric constraint. The discrete event modelling is ac~complished using Petri nets. The problems of task-level planning .and syn- thesis are addressed. Using the Petri net modelling, a discrete event controller (DEC) is developed. The DEC is a task-level controller that directs the assembly process to the desired end state. The DEC outputs both a desired velocity and it desired force command. The desired velocity commands insure consis- tency with geometric constraints while simultaneouslly directing the system toward the next desired discrete state. The desired force commands attempt to maintain the current state of contact. Task-level planning is accomplished using dynamic programming to deterimine the optimal sequence of discrete states for success- ful assembly. This desired sequence of discrete states is then used as a reference input to the DEC. 1 Iintroduction In discrete automation, assembly is one of the most error-prone, unreliablle processes. There is an increasing need for efficient methods of increased robustness and reliability in :the face of tolerancing errors and uncertainties in the workpieces and the environment. The control of assembly, including task level tra- jectories and on-line task level control, is a key component to improving the success of robotic assembly. The goal of this paper is to present a new approach to the control of robotic assembly tasks, treating assembly as a discrete event dynamic system. The abstraction to discrete event modelling highlights the necessary transitions for successful assembly. Additionally, the abstraction allows for planning on a task level rather than the cumbersome process of exact trajectory planning. Various methods have been proposed for the control of assem- bly processes. (Whitney, 19821 derives quasi-static con.dit,ions for a successful insertion operations. (Mason, 19821 (Raibert and Craig, 19811 describe a hybrid position/force control for various tasks. Other work has included using force feedback as a source of task preformance information (Hannaford, 19871. However, unlike most force-feedback applications, the state of contact changes during an assembly process. These discrete changes in state should be the focal point of the control procedure In order t,o incorporate the state changes, we will model assembly as a discrete event dynamic system. The discrete state vector will be the discrete states of contact in the assembhly process. The discrete events (occurring at discrete instances in time) are the since they indicate significant changes in the system dynamics. changes in contact state. Typically, discrete event dynamic sys- tem models arise from certain aspects of manufacturing systems and data network protocols. In contrast to these applications, this paper presents the discrete event modelling and control of robotic assembly. The difficulty of modelling and designing discrete event control systems has long been recognized in the literature. Moreover, there is apparently no unifying theory for the control of discrete event systems. Numerous methods have been proposed for the modelling and analysis of discrete event systems, each having different characteristics. Some of these include formal languages (Ramadge, 19891, finite state machines (Tadmor and Maimon, 19891, and Petri nets (Peterson, 19811. The goal of this paper is to present a new approach to robotic assembly, treating assembly as a discrete event dynamic system. The discrete event modelling is accomplished using Petri nets. The abstraction of assembly to Petri net modelling highlights the key transitions necessary for successful insertion. Additionally, the abstraction allows for planning on a task level rather than the cumbersome process of exact trajectory planning. Using the Petri net modelling, a discrete event controller (DEC) is developed. The DEC is a task-level controller that directs the assembly process to the desired end state. Two condi- tions are derived that perform this function. First, a closed form solution is given for the desired velocity commands. Desired ve- locity commands insure consistency with geometric constraints while simultaneously directing the system toward the next de- sired discrete state. Second, an equation is derived the specifies the desired force commands, The desired force commands are the forces that attempt to maintain the current state of contact. Lastly, an optimal sequence of discrete events (changes of con- tact) leading to successful insertion is determined using dynamic programming, thus completing the specificaiions for the DEC.