An easily retrofittable stereo vision based system for quick and temporary measurement of a mining shovel's swing angle is presented. The stereo camera is mounted externally to the upper swingable shovel house, with a clear view of the shovel's lower carbody. As the shovel swings from its 0° swing angle position, the camera revolves with the shovel house, seeing differing views of the carbody. In real-time, the camera position is tracked, which in turn is used to calculate the swing angle. The problem was solved using the Simultaneous Localization and Mapping (SLAM) approach in which the system learns a map of 3D features on the carbody while using the map to determine the camera pose. The contribution includes a locally maximal Harris corner selection technique and a novel use of 3D feature clusters as landmarks, for improving the robustness of visual landmark matching in an outdoor environment. Results show that the vision-based sensor has a maximum error of +/- 1° upon map convergence.

Eye-gaze tracking devices are typically used to estimate the point-of-gaze (POG) of a subject on a 2D surface such as a computer screen. Using model based methods for POG estimation we have developed a system based on the vergence of the eyes which can be used to estimate the POG on a real-world volumetric display.

The precision of point-of-gaze (POG) estimation during a fixation is an important factor in determining the usability of a noncontact eye-gaze tracking system for real-time applications. The objective of this paper is to define and measure POG fixation precision, propose methods for increasing the fixation precision, and examine the improvements when the methods are applied to two POG estimation approaches. To achieve these objectives, techniques for high-speed image processing that allow POG sampling rates of over 400 Hz are presented. With these high-speed POG sampling rates, the fixation precision can be improved by filtering while maintaining an acceptable real-time latency. The high-speed sampling and digital filtering techniques developed were applied to two POG estimation techniques, i.e., the high-speed pupil-corneal reflection (HS P-CR) vector method and a 3-D model-based method allowing free head motion. Evaluation on the subjects has shown that when operating at 407 frames per second (fps) with filtering, the fixation precision for the HS P-CR POG estimation method was improved by a factor of 5.8 to 0.035 degrees (1.6 screen pixels) compared to the unfiltered operation at 30 fps. For the 3-D POG estimation method, the fixation precision was improved by a factor of 11 to 0.050 degrees (2.3 screen pixels) compared to the unfiltered operation at 30 fps.

In this paper we present a novel signal processing algorithm accompanied with an array structure for audio signal localization and surveillance in three dimensional space. This method, which has similarities to the eye structure, consists of a novel hemispherical microphone array with microphones on the shell and one microphone in the sphere center. A signal processing scheme utilizes parallel creation of a special closeness function for each microphone direction on the shell. Contrary to traditional algorithmic sound source localization techniques, our method is based on some simple parallel mathematical calculations in the time domain; therefore it can be easily implemented on a custom designed integrated circuit.

Eye-gaze as a form of human machine interface holds great promise for improving the way we interact with machines. Eyegaze tracking devices that are non-contact, non-restrictive, accurate and easy to use will increase the appeal for including eye-gaze information in future applications. The system we have developed and which we describe in this paper achieves these goals using a single high resolution camera with a fixed field of view. The single camera system has no moving parts which results in rapid reacquisition of the eye after loss of tracking. Free head motion is achieved using multiple glints and 3D modeling techniques. Accuracies of under 1 ° of visual angle are achieved over a field of view of 14x12x20 cm and over various hardware configurations, camera resolutions and frame rates.

This paper describes a vision-based system for 3-D localization of a mobile robot in a natural environment. The system includes a mountable head with three on-board charge-coupled device cameras that can be installed on the robot. The main emphasis of this paper is on the ability to estimate the motion of the robot independently from any prior scene knowledge, landmark, or extra sensory devices. Distinctive scene features are identified using a novel algorithm, and their 3-D locations are estimated with high accuracy by a stereo algorithm. Using new two-stage feature tracking and iterative motion estimation in a symbiotic manner, precise motion vectors are obtained. The 3-D positions of scene features and the robot are refined by a Kalman filtering approach with a complete error-propagation modeling scheme. Experimental results show that good tracking and localization can be achieved using the proposed vision system.

This paper describes a vision-based system for 3D localization and tracking of a mobile robot in an unmodified environment. The system includes a mountable head with three on-board stereo CCD cameras that can be installed on the robot. There the main emphasis is on the ability to estimate the geometric information of the robot independently from any prior scene knowledge, landmark or extra sensory device. Distinctive scene features are identified using a novel algorithm and their 3D locations are estimated with a stereo algorithm. Using multi-stage feature tracking and motion estimation in a symbolic manner, precise motion vectors are obtained. The 3D positions of the scene features are updated by a Kalman filtering process. Experimental results show that robust tracking and localization can be achieved using our vision system.

Corner extraction is an important task in many computer vision systems. The quality of the corners and the efficiency of the detection method are two very important aspects that can greatly impact the accuracy, robustness and real-time performance of the corresponding corner-based vision system. In this paper we introduce a fast corner detector based on local binary-image regions. We verify the performance of the proposed method by measuring the repeatability rate under various illumination, scale and motion conditions. Our experimental results show that while the quality of the features is comparable with other conventional methods, ours delivers a faster performance.

An indirect, self-calibrating, easy to install, and robust joint angle sensing method for heavy-duty manipulators is presented in this paper. This method is suitable for the harsh working environment of these machines where conventional contact-type angle sensors cannot be deployed, or problems are associated with their use. The approach is based on processing the outputs of a pair of biaxial accelerometers placed very close to the joint axis on the adjacent links. In the proposed technique, joint angles are obtained without integrating the accelerometer outputs to avoid measurement error accumulation over a long period of time. Two calibration procedures are also described for accelerometers to ensure the accuracy of their measurements. The experimental results attest to the efficiency and accuracy of the new angle sensing mechanism.

This paper presents a vision-based tracking system suitable for autonomous robot vehicle guidance. The system includes a head with three on-board CCD cameras, which can be mounted anywhere on a mobile vehicle. By processing consecutive trinocular sets of precisely aligned and rectified images, the local 3D trajectory of the vehicle in an unstructured environment can be tracked. First, a 3D representation of stable features in the image scene is generated using a stereo algorithm. Next, motion is estimated by trading matched features over time. The motion equation with 6-DOF is then solved using an iterative least squares fit algorithm. Finally, a Kalman filter implementation is used to optimize the world representation of scene features.

A novel yet simple approach for experimental determination of the link (mass and inertia-related) parameters and friction coefficients is developed for a typical excavator arm. The parameters are needed for indirect measurement of the external forces, compensation for the link weights in the operator's hand in a force-feedback teleoperation setup, impedance control of the arm, simulation of the manipulator dynamics, and model-based fault detection. Treating the machine arm as an open kinematic chain, its dynamic equations are presented symbolically. The static torque equations are derived from these equations and the gravitational parameters are defined accordingly. A new method for decoupled estimation of the gravitational parameters from static experiments is presented. Furthermore, the arm dynamics are expressed in a form which is linear in the inertia and friction-related parameters. The results obtained show that the identified model predicts the joint torques, in both static and dynamic conditions, with a very good accuracy.

Presents the design of a cardiac tissue stabilizing device with fixating surgical arm, developed for open-heart cardiac surgery to stabilize anastomosis sites during off-pump coronary artery bypass grafting surgeries. The stabilization device is a "C"-shaped hollow ring that holds the heart using vacuum pressure. The arm is a seven link, 18-DOF, redundant, passive, serial linkage. As it attaches directly to the patient on the chest retractor, the arm has a stable mechanical ground protecting the patient from injury should they move during surgery. Measured forces and torques from two pigs' hearts were scaled to human magnitudes to determine stabilizer lifting and arm braking requirements.

The problem of robot visual navigation is bound up with the problem of selecting and extracting salient features or landmarks which can guide the robot safely through its environment. In addition for vision based manipulation tasks for mobile robots we must register the position and orientation of objects to be manipulated. In general identifying the ground plane and distinguishing non-planar obstacles is important for indoor navigation. The large scale industrial robots we study are used in such tasks as forest management or hazardous waste clean up. For these machines identification and registration of vertical cylinders representing trees to be avoided or 45 gal. drums to be manipulated is critical. Instead of laborious reconstruction of surfaces to find these critical plane and cylinder shape primitives we propose that the subspace defined by the disparity gradients of visible surfaces can be used to extract them efficiently. We demonstrate the use of these features, which are easily extracted in real time using a high speed pipeline processor.

A nonparametric learning algorithm is used to build a robust mapping between an image obtained from a mobile robot's on-board camera, and the robot's current position. The mapping uses 19,200 unprocessed pixel values (160 by 120 pixel image). Because the learning algorithm is nonparametric, it uses the learning data obtained from these raw pixel values to automatically choose a structure for the mapping without human intervention, or any a priori assumptions about what type of image features should be used. The learning data consisting of a series example image inputs and corresponding position values, is collected in a calibration phase where the robot randomly traverses its intended workspace. This process of building visual localization maps for mobile robots is completely general and can be applied to any implementation which uses on-board cameras. We demonstrate the feasibility of this approach on a mobile platform performing in a robotics laboratory workspace. This workspace is visually cluttered, with humans and other objects continually moving within the robot's environment. The mapping learned in this environment is robust to these dynamic visual features and consistently reports timely localization information (at greater than 7 Hz) to within acceptable limits.

In nonautomated unstructured environments, a machine such as a hydraulic log loader has no model of either itself or the world, and the operator has to control it at a joint-mode level. The operator can be more productive by controlling the machine in task coordinates, whereby the implement direction and speed are first decided by the deflection of a 3D joystick handle. Then, the appropriate control signals to the hydraulic actuators are determined by a computer control system. In this article the application of fuzzy logic to the development of a real-time hydraulic positioning control system for such machines is demonstrated. The control requirements of hydraulic actuation are met by development of (i) a PD-type fuzzy controller with simple set of membership functions and specialized rules and, (ii) a novel offset compensator for removing the steady-state regulating and tracking errors. Experimental results, performed on a Caterpillar 215B-based log loader, are presented.

We propose a framework for programming robotic tasks using human-to-robot skill transfer. We assume that there exists a human expert who can accomplish a task in an unstructured environment by using various sensor displays and controls. The human expert performs the desired task a number of times while his/her input/output pairs are being recorded by the robot. The robot then uses this recorded data to construct a mapping between these sensor inputs and actuator outputs. This mapping must be general enough to allow the robot to accomplish the same task, in similar but not identical, dynamic, unstructured environments. This paper presents a testbed for human-to-robot skill transfer which is based on the teleoperated control of a small mobile robot working in an unstructured environment. The skill which is transferred from human-to-robot is loosely based on the tree tending task, a task which was chosen for its inherently unstructured nature. The SPORE approximation is proposed as a means for creating the robot's mapping from sensor inputs to actuator outputs.

Gravitational and friction terms play an important role in achieving high performance control of heavy-duty hydraulic machines, such as excavators. In this paper, a new approach for decoupled estimation of the gravitational parameters is presented. Static experiments are carried out with an instrumented computer-controlled mini excavator to estimate the gravitational parameters. Load pins are used for indirect measurement of the joint torques from cylinder reaction forces. It is investigated via experiments that bucket payload can be estimated with a 5% accuracy. Furthermore, direct measurement of the actuators' friction shows that considerable amount of static friction exists inside the cylinders that cannot be neglected.

This paper describes the design of a one-person hydraulic motion simulator with six degrees of freedom. An inverted, ceiling-mounted Stewart platform design enables the use of narrower hydraulic cylinders than required with floor-mounted designs, and significantly reduces hydraulic flow requirements and hence the cost. The paper details the platform geometry, shows the effectiveness of Newton's method in solving the direct kinematics, and describes a tested hydraulic system design. Issues of control and safety are also addressed. It is expected that this motion simulator will provide 9.8 m/s/sup 2/, 400/spl deg//s/sup 2/ accelerations and 1 m/s, 30/spl deg//s speeds to a 250 kg payload with 1 m, 45/spl deg/ displacements from a nominal center. Preliminary experiments at low-pressure have been encouraging.>

A new method, termed the offset modification method (OM method), for solving the manipulator inverse kinematics problem is presented. The OM method works by modifying the link offset values of a manipulator until it is possible to derive closed-form inverse kinematics equations for the resulting manipulator (termed the model manipulator). This procedure allows one to derive a set of three nonlinear equations in three unknowns that, when numerically solved, give an inverse kinematics solution for the original manipulator. The OM method can be applied to manipulators with any number of degrees of freedom, as long as the manipulator satisfies a given set of conditions (Theorem 1). The OM method is tested on a 6-degree-of-freedom manipulator that has no known closed-form inverse kinematics equations. It is shown that the OM method is applicable to real-time manipulator control, can be used to guarantee convergence to a desired endpoint position and orientation (if it exists), and allows one to directly choose which inverse kinematics solution the algorithm will converge to (as specified in the model manipulator closed-form inverse kinematics equations). Applications of the method to other 6-DOF manipulator geometries and to redundant manipulators (i.e. greater than 6 DOF geometries) are discussed.>

This paper presents a method for motion control of a class of heavy-duty hydraulic machines in a resolved teleoperation mode. A model-based sensor-based algorithm is proposed which is applied in conjunction with the closed-loop components. The algorithm is a feedforward load compensating scheme which uses the measured hydraulic line pressures along with an appropriate portion of the hydraulic model to control the joint velocities. The scheme incorporates a logic system which, according to the states relevant to the machine task, decides on the minimum portion of the hydraulic model to be used. Inclusion of this algorithm eliminates most of the effects of loading, coupled actuation and single joint power limitations, which allows easier implementation of the closed-loop part. A knowledge of some hydraulic parameters is the only requirement of this scheme. No knowledge about the structure or loading is necessary. I. INTRODUCTION There exist many industrial tasks that are performed in di- verse and unstructured surroundings. Examples of these tasks can be found in the forest, mining and construction industries These industries utilize many heavy-duty machines (such as excavators, feller bunchers or log loaders) that have linked me- chanical structures, actuated through coupled and complex hydraulic systems. The present control of these machines is based on individually actuating each link by an operator which requires significant visual feedback, judgement and skill. These machines do not benefit from computer-assisted con- trols; however, they have the potential to be automated. This paper presents a novel application of teleoperation to such industrial machines. In teleoperation, the operator's commands are passed to a computer which is interfaced to the machine and performs the actual control. At least three benefits can be achieved. Firstly, the human's operation of the machine can be coordi- nated. Secondly, the computer can check for incipient machine failures. Thirdly, repetitive operations can be automated un- der supervision of the human.

Accurate and efficient inverse kinematics transformations are required to accurately specify the Cartesian position and orientation of robotic manipulators. A novel algorithm is proposed which is accurate near singular points and is also stable at singular points and for targets which are out of the manipulator's reach. The algorithm uses the error-damped pseudoinverse to obtain stable joint corrections near singularities. Various strategies are presented to describe and handle the particular situations which arise near singularities.>

A novel iterative inverse transform solution for robotic manipulators, termed functional joint control (FJC), is proposed. It is applicable to an important class of robots that includes many industrial robots. FJC is numerically well-behaved around singularities. The rate of convergence of FJC is comparable to other iterative solutions. The algorithm can be decomposed naturally for a multiprocessor implementation, with up to one processor for each joint of the manipulator. Building on the basic FJC algorithm, higher-level heuristics are used to handle joint limits and some redundant manipulators.>

A control strategy is presented for robots that do not have accurately known mechanical structures or have inaccuracies caused by bending, slip, or backlash. In the system, the manipulator's endpoint is monitored in a servo loop, so that inaccuracies in the structure can be compensated. An adaptive transformation from task to robot-oriented coordinates has been used online, without prior modeling or calculation. The strategy developed has been simulated for a two-degree-of-freedom robot. Results are compared with those obtained using the inverse Jacobian as part of the control system.>

In a multidegree of freedom manipulator there can exist the problem of how to coordinate all the actuators to produce movement in a space of lower dimension. In this paper statistical decision procedures are used to select an elbow position of an externally powered arm aid to deal with this problem. The decision makers are derived from a set of exoskeletal goniometer movement recordings of typical daily tasks. Both pattern recognition and regression techniques are employed. Pattern recognition is used to decide which class of tasks the human is currently performing based on the desired end-point position and the position of the three hand axes: wrist supination, wrist flexion, and prehension. A regression function with end-point position arguments is then used to estimate a parameter called the ``elbow angle.'' The combination of the elbow angle and the desired end-point position is transformed geometrically to obtain the correct actuator positions of the assistive device. Both decision schemes are implemented in a real-time algorithm for the computer control of a modified Rancho electric arm.