Planetary surface science operations performed by robotic space systems frequently require pointing cameras at various objects and moving a robotic arm end effector tool toward specific targets. Earlier NASA Mars Exploration Rovers did not have the ability to compute actual coordinates for given object coordinate frame names and had to be provided with explicit coordinates. Since it sometimes takes hours to more than a day to get final approval of certain calculated coordinates for command uplink via the Earth-based mission operations procedures, a highly desired enhancement for future rovers was to have the onboard automated capability to compute the coordinates for a given frame name. The Mars Science Laboratory (MSL) rover mission is the first to have a centralized coordinate transform database to maintain the knowledge of spatial relations. This onboard intelligence significantly simplifies communication and control between Earth-based human mission operators and the robotic rover on Mars by supporting higher level abstraction of commands using object and target names instead of coordinates. More specifically, the spatial relations of many object frames are represented hierarchically in a tree data structure, called the frame tree. Individual frame transforms are populated by their respective modules that have specific knowledge of the frames. Through this onboard centralized frame tree database, client modules can query transforms between any two frames and support spacecraft commands that use any frames maintained in the frame tree. Various operational examples in the MSL mission that have greatly benefitted from this onboard centralized frame tree database are presented.

This paper presents an in-depth performance analysis and validation of a correlation based stereo vision system being used as part of the ongoing 2003 Mars Exploration Rover flight mission. Our analysis includes the effects of correlation window size, pyramidal image down-sampling, vertical misalignment, focus, maximum disparity, stereo baseline, and range ripples. A key element of validation is to determine the stereo localization error both analytically and experimentally. We study both down-range and cross-range error and verify that while camera calibration inaccuracy contributes to both, stereo correlation error affects only the former. Error contributions of subpixel interpolation, vertical misalignment, and foreshortening on stereo correlation are examined carefully. A novel method using bricks with reflective metrology targets and a mast-mounted stereo camera system enabled experimental measurements of the stereo disparity error. The standard deviation of the down-range disparity error was measured at sigma=0.32 pixel for high-resolution 1024times768 camera images. The result is critical in evaluating accurate rover navigation and instrument placement within given error budgets

Presents our experience in developing a methodology and tool at the Jet Propulsion Laboratory (JPL) for software-implemented fault injection (SWIFI) into a parallel-processing supercomputer which is being designed for use in next-generation space exploration missions. The fault injector uses software-based strategies to emulate the effects of radiation-induced transients occurring in the system hardware components. JPL's SWIFI tool set, which is called JIFI (JPL's Implementation of a Fault Injector), is being used in conjunction with an appropriate system fault model to evaluate candidate hardware and software fault tolerance architectures, to determine the sensitivity of applications to faults, and to measure the effectiveness of fault detection, isolation and recovery strategies. JIFI has been validated to inject faults into user-specified CPU registers and memory regions with a uniform random distribution in location and time. Together with verifiers, classifiers and run scripts, JIFI enables massive fault injection campaigns and statistical data analysis.

A computer vision assisted semi-automatic virtual reality (VR) calibration technology has been developed that can accurately match a virtual environment of graphically simulated 3D models to the video images of the real task environment. We have developed a new 20-variable weighted least-squares algorithm that updates both camera and object models simultaneously for given two camera views of two mating objects. This simultaneous update enables accurate model matching even with rough, approximate initial camera calibrations. The developed semi-automatic VR calibration supports automated intermediate updates, eliminating nearly all operator interaction except for initial coarse matching. In our quasi-static supervisory telerobotic applications, intermediate VR calibrations are performed intermittently at a few robot stopping poses only, as a cost-effective and safer approach compared to real-time visual servoing. Extensive experimental results comparing alignment errors under various viewing conditions are presented.

CDMA is a very attractive technique for PCS because of its potentially large capacity and well-known ability to resolve multipath fading. The next step will be to enhance the performance of CDMA system efficiently and optimally. Two new frequency planning schemes are proposed. One uses a frequency reuse factor of 3 and the other uses a cell partitioning. To show the advantages of two new frequency planning schemes, the capacity of CDMA systems is compared. First of all, the capacity of the current CDMA system is analyzed with the consideration of power control. Then, the performance of the CDMA systems using new frequency planning schemes is obtained. The results show that the current frequency reuse factor of 1 is not always optimal, even though it is the simplest frequency planning to deploy. The new frequency planning schemes provide more capacity and are more adaptive in the cases of non-uniformly distributed traffic loads.

New developments of edge matching based semiautomatic virtual reality calibration are presented to provide more reliable and accurate telerobotic servicing. Three basic algorithms employed are (1) weighted average local edge detector, (2) weighted least squares for the edge-based camera calibration and object localization, and (3) robust matching to remove outliers. A key new 20-variable least-squares algorithm is developed that updates both the camera and the object models simultaneously for given two camera views of two mating objects. This simultaneous update reduces relative positioning errors considerably, and is essential to achieve high alignment precision. The developed system has been successfully tested for an ORU (Orbital Replacement Unit) insertion task within a /spl plusmn/1/4 inch and /spl plusmn/3/spl deg/ alignment precision.

A virtual reality calibration technique of matching a virtual environment of simulated graphics models in 3-D geometry and perspective with actual camera views of the remote site task environment has been developed to enable high-fidelity preview/predictive displays with calibrated graphics overlay on live video. Reliable and accurate calibration is achieved by operator-interactive camera calibration and object localization. Both rely on nonlinear least-squares algorithms combined with linear ones. Since the object pose becomes known through object localization, the computer-generated trajectory mode can be effectively used in our new approach. This newly developed calibration technique has been successfully utilized in a recent JPL/GSFC (Jet Propulsion Laboratory/Goddard Space Flight Center) telerobotic servicing demonstration. The positioning alignment accuracy achieved by this technique from four camera views was 0.51 cm on the average for a tool insertion in the servicing task.>

Recent advances in real-time 3-D computer graphics and hierarchically structured graphical user interfaces (GUI) enable development of very efficient graphics-based operator interfaces for telerobotic operations. This paper addresses potential benefits of such advanced graphics interfaces to telerobotic servicing and inspection, emphasizing operator efficiency and system safety. Specifically, the authors report on calibrated 3-D graphics displays and GUIs developed and applied to an advanced teleoperation system and a remote surface inspection system at the Jet Propulsion Laboratory.

A highly effective predictive/preview display technique for telerobotic servicing in space under several seconds communication time delay has been demonstrated on a large laboratory scale. The technique is based on a high-fidelity calibration procedure that enables a high-fidelity overlay of 3-D graphics robot arm and object models over given 2-D TV camera images of robot arm and objects. To generate robot arm motions, the operator can confidently interact in real time with the graphics models of the robot arm and objects overlaid on an actual camera view of the remote work site. The technique also enables the operator to generate high-fidelity synthetic TV camera views showing motion events that are hidden in a given TV camera view or for which no TV camera views are available. The positioning accuracy achieved by this technique for a zoomed-in camera setting was about +or-5 mm, well within the allowable +or-12 mm error margin at the insertion of a 45cm long tool in the servicing task.>

Two schemes of force reflecting control, position-error based force reflection and low-pass-filtered force reflection, both combined with shared compliance control, were developed for dissimilar master-slave arms. These schemes permitted high force reflection gains, which were not possible with a conventional scheme when the slave arm was much stiffer than the master arm. The experimental results with a peg-in-hole task indicated that the force reflecting control schemes combined with compliance control, as presented, resulted in best task performances. As a related application, a simulated force reflection/shared compliance control teleoperation trainer was developed that provided the operator with the feel of kinesthetic force virtual reality.>

Shared compliant control has been incorporated into an advanced six-degree-of-freedom force-reflecting telemanipulation system. With this system the authors have investigated the effect of time delay on human telemanipulation task performance. Time delays of between 2 and 4096 ms were introduced between master and slave arms, and high-precision peg-in-hole tasks were performed by six test operators with two modes of control: kinesthetic force feedback (KFF) and shared compliant control (SCC). Task performance was quantified in terms of the completion time (CT) and the sum of square forces (SOSF). The experimental results demonstrate the superiority of SCC over KFF for time-delayed telemanipulation. SCC has significantly lower rates of increase than KFF in both CT and SOSF with time delay. Only SCC enabled task performance at delays above 1 s. Constant force maintenance tasks were also performed to investigate the effect of time delay on the stability of force reflection. SCC also has beneficial effects on telemanipulation without time delay.>

Orbital replacement unit (ORU) changeout experiments were performed with the JPL/NASA telerobot testbed system for three different control modes: force reflecting teleoperation (FR), shared compliant control (SCC), and autonomous control (AC). In FR, the operator was provided with kinesthetic force feedback, feeling contact forces through a force-reflecting hand controller. In SCC, the operator controlled a compliant robot, where the robot compliance was implemented by an autonomous compliant control force feedback. In AC, telemanipulation tasks were performed autonomously by using a sequence of macros. Experimental results indicate that SCC greatly reduces the magnitudes and durations of contact forces as compared to FR. AC took the least time to complete the ORU changeout task and, with appropriate operator intervention capabilities, it could be very pleasant to use in some applications with well-structured task environments.>

An enhanced teleoperation technique for time-delayed bilateral teleoperator control is discussed. The control technique selected for time delay is based on the use of a high-fidelity graphics phantom robot that is being controlled in real time (without time delay) against the static task image. Thus, the motion of the phantom robot image on the monitor predicts the motion of the real robot. The real robot's motion will follow the phantom robot's motion on the monitor with the communication time delay implied in the task. Real-time high-fidelity graphics simulation of a PUMA arm is generated and overlaid on the actual camera view of the arm. A simple camera calibration technique is used for calibrated graphics overlay. A preliminary experiment is performed with the predictive display by using a very simple tapping task. The results with this simple task indicate that predictive display enhanced the human operator's telemanipulation task performance significantly during free motion when there is a long time delay. It appears, however, that either two-view or stereoscopic predictive displays are necessary for general three-dimensional tasks.>

Two cooperative control schemes for telerobot visual displays are addressed. In the first scheme, on-the-screen visual enhancements such as reference lines indicating the vertical height of the robot hand, a stick figure model of the robot hand, and its projection on the horizontal grid plane are constructed by the interactive cooperation between the human operator and the telerobotic system, and then superimposed on the video screen. Experimental results with a five-degree-of-freedom robot and a frame grabber indicate that superimposition of visual enhancements on the video screen greatly improves telemanipulation task performance. In the second scheme, the position and orientation of an object on video screens are determined interactively; these then assist the telerobotic system in executing the human operator's task-level commands autonomously.>

Time delays of between 2 ms and 4096 ms were introduced between master and slave manipulators in advanced 6-DOF (degrees-of-freedom) force-reflecting telemanipulation system. The effects of the imposed delay were quantified in terms of completion time (CT) and sum-of-squared-force (SOSF) with six test operators and two modes of control: kinesthetic force feedback (KFF) and shared compliant control (SCC). KFF applies forces and torques proportional to those sensed by the slave robot to the human operator through the master. SCC uses the force/torque information entirely at the slave side to implement a variation on impedance control in parallel with the operator's position commands. Experiments measured effects of human operator grasp variation on stability of contact with KFF. CT and SOSF performance in a peg-in-hole task decreased linearly with imposed time delay using KFF, and decreased at a lower rate with SCC. SCC enabled task performance for 2 and 4 second delays, critical values for low-earth-orbit applications.>

It is argued that autonomous robotics is not yet feasible, and that instantaneous human control may be difficult because of communication delay and expensive because of labor costs. Therefore, it is argued, human supervisory control with cooperation between man and a partially autonomous distant robot is the most viable engineering solution. Model control of image processing enables efficient visual displays. Cooperative control in paired manipulators required rapid controller signal information to be passed over a cross-filter in a heuristic study.>

Position and rate control are the two common manual control modes in teleoperations. Human operator performance using the two modes is evaluated and compared. Simulated three-axis pick-and-place operations are used as the primary task for evaluation. First, ideal position and rate control are compared by considering several factors, such as joystick gain, joystick type, display mode, task, and manipulator work space size. Then the effects of the manipulator system dynamics are investigated by varying the natural frequency and speed limit. Experimental results show that ideal position control is superior to ideal rate control, regardless of joystick type or display mode, when the manipulation work space is small or comparable to the human operator's control space. Results also show that when the manipulator system is slow, the superiority of position control disappears. Position control is recommended for small-work-space telemanipulation tasks, while rate control is recommended for slow wide-work-space telemanipulation tasks.

A visual display system serves as an important man-machine interface for efficient teleoperations. However, careful consideration is necessary to display three-dimensional information on a two-dimensional screen effectively. A teleoperation simulator is constructed with a vector display system, joysticks, and a simulated cylindrical manipulator to evaluate various display conditions quantitatively. Pick-and-place tasks are performed, and mean completion times are used as a performance measure. Two experiments are performed. First, effects of variation of perspective parameters on human operator's pick-and-place performance with monoscopic perspective display are investigated. Then visual enhancements of monoscopic perspective display by adding a grid and reference lines are investigated and compared with visual enhancements of stereoscopic display. The results indicate that stereoscopic display does generally permit superior pick-and-place performance, while monoscopic display can allow equivalent performance when it is defined with appropriate perspective parameter values and provided with adequate visual enhancements. Mean completion time results of pick-and-place experiments for various display conditions shown are observed to be quite similar to the normalized rms error results of mannal tracking experiments reported previously.

Optimal presentation of three-dimensional information on a two-dimensional display screen requires careful design of the projection to the display surface. Monoscopic perspective projection alone is usually not sufficient to represent three-dimensional spatial information. It can, however, be improved by the adjustment of perspective parameters and by geometric visual enhancements such as reference lines and a background grid. Stereoscopic display is another method of providing three-dimensional information to the human operator. Two experiments are performed with three-axis manual tracking tasks. The first experiment investigates the effects of perspective parameters on tracking performance. The second experiment investigates the effects of visual enhancements for both monoscopic and stereoscopic displays. Results indicate that, though stereoscopic displays do generally permit superior tracking performance, monoscopic displays can allow equivalent performance when they are defined with optimal perspective parameters and provided with adequate visual enhancements.