Sarcopenia, characterized by progressive loss of muscle mass and function, significantly impacts the quality of life in aging populations. Early detection and personalized intervention are crucial yet challenging due to the limited accessibility and scalability of traditional diagnostic methods. Building upon our previous work on gait-based assessment, this study presents a novel framework for early sarcopenia-prone risk detection using the five-time sit-to-stand (5TSTS) test, embodying Healthcare Industry 5.0’s vision of mass personalization with human-centered technology. Utilizing the Internet of Things (IoT)-enabled wearable inertial measurement units (IMUs) and advanced analytics, our system segments 5TSTS into four biomechanically significant submotions [standing up (StU), standing transition (StT), sitting down (SiD), and sitting transition (SiT)]. This granular segmentation allows mass personalization in diagnostic evaluations by capturing individual-specific biomechanical profiles via wavelet-based feature extraction and machine learning (ML) techniques. Our framework employs big data analytics tools, including the extreme gradient boosting (XGBoost)-based feature selection and support vector machine synthetic minority oversampling technique (SVMSMOTE), to handle class imbalance and optimize individualized predictive accuracy. Tested on data from 52 elderly participants (aged 65–84 years), the system achieves outstanding personalized classification accuracy—up to 97.97% for multiclass risk stratification and 99.28% for binary (healthy versus sarcopenia-prone) classification—highlighting its potential for precise, patient-specific clinical decision-making. Furthermore, the wireless capability of our IoT-enabled wearable IMUs, coupled with minimal setup requirements, facilitates seamless data integration into cloud-based healthcare systems. This integration supports real-time remote monitoring and personalized health management. By leveraging advanced sensing, analytics, and connectivity technologies, our approach significantly advances personalized, accessible, and scalable sarcopenia-prone risk assessment, thereby contributing directly to the vision of Healthcare Industry 5.0.

Prosthetics, exoskeletons, and rehabilitation devices that seamlessly respond to the user's motion intent are essential for improving the quality of life for individuals with physical disabilities. However, existing motion intent recognition systems based on Electromyography (EMG) often experience significant performance degradation over time and limb positions. To address this, this paper proposed a Hierarchical EMG-FMG Fusion (HEFF) framework that incorporates Force Myography (FMG) as a complementary modality to enhance robustness in long-term and multi-position motion intent recognition. The HEFF framework introduced three new strategies: (1) Complementary Pyramid Fusion, a neural network architecture that effectively integrates the complementary characteristics of EMG and FMG; (2) Position Compensation and Pairing, to increase the generalizability of the model by producing augmented training data mimicking the temporal and positional variability; and (3) Feedback-based Baseline Correction, to dynamically counteract FMG baseline drift. The robustness of the system was evaluated through 10 distinct motions performed in 3 different limb positions, retested after an interval of approximately 7 days. Notably, the training data were only collected in a single limb position on the first day. The real-time experiments with 15 subjects showed that the HEFF framework maintained 91.89% classification accuracy in the primary limb position and 85.39% across multiple unseen positions. According to offline analysis, HEFF improved classification accuracy by up to 34.99% across various limb positions compared to the conventional fusion method. These results highlight HEFF's robustness across various conditions, being well-suited for assistive technology applications.

For pneumatic artificial muscles, it is always considered the more maximum contraction ratio the better. While for human joint assisting applications, PAMs with configurable maximum contraction rate are more suitable because of advantageous safety and adaptability. A PAM based on planar-to-specific-wave body shape morph is proposed in this work. Shape-morphing-based braided artificial muscles (SBAMs) have uniqueness of initial elasticity and maximum contraction ration programmability, which meet the favors of human joint assisting applications. The basic structure and working mechanism of contraction in SBAMs will be explained, and their mathematical model will also be established. According to the experimental results, a SBAM prototype generates a force more than 140 times its weight under an easily accessible pressure of 150 kPa. A mannequin wearing the SBAM enables actively flexes its elbow over 120 °.

Soft artificial muscle actuators have gained attention in robotics for their remote control, fast response, and high compliance. However, replicating the intricate and efficient motions of natural muscles remains a challenge. Existing designs often lack the hierarchical and anisotropic properties of muscle sarcomeres, limiting their ability to achieve biomimetic movements. We developed a novel Biomimetic Magnetic Artificial Actuator (BMAA) inspired by muscle sarcomeres. Using a soft magnetic composite material arranged in a hierarchical structure, the actuator mimics the arrangement of actin and myosin filaments. External magnetic fields enable precise control of contraction and relaxation, emulating natural muscle motion. The driver can achieve the motion performance of muscle like motion characteristics, and its driving ability is verified by reptile experiment and elbow experiment. The actuator demonstrates significant deformation, fast response, and excellent controllability, enabling complex and precise movements. This research advances the development of biomimetic soft actuators, offering potential applications in soft robotics, biomedical devices, and artificial muscles, and paving the way for more versatile and intelligent machines.

The growing complexity of robotic teleoperation systems necessitates the integration of multiple feedback modalities, including video, audio, force, tactile, and temperature feedback. The concept of supermedia is utilized to describe the aggregation of these feedback streams. By integrating multiple media forms, supermedia can offer a more comprehensive interactive experience for robot teleoperation systems. However, existing transmission protocols struggle to maintain synchronization among these diverse feedback streams, particularly in demanding network environments. In this paper, we present the Tele-Robotic Control Protocol (TRCP), a novel network transmission protocol specifically designed for supermedia-enhanced robotic teleoperation systems. TRCP incorporates an event reference mechanism that coordinates multiple feedback streams based on robot state rather than traditional time-based sampling. It also employs multi-queue management for the independent handling of different feedback types and integrates an adaptive adjustment mechanism that optimizes transmission parameters in response to real-time network conditions. The effectiveness of TRCP is demonstrated through a cross-continental teleoperation experiment between the University of Glasgow and the University of Hong Kong. TRCP achieves superior feedback synchronization and real-time responsiveness, significantly enhancing both task success rates and operator performance.

Age-related sarcopenia weakens balance in older adults, highlighting the need for effective assistive robots. How-ever, existing assistive technologies often suffer from mechanical, kinematic, and control incompatibilities with the human body. Here, we present a wearable assistive robotic system that integrates muscle-like actuation and control algorithms to directly compensate for impaired muscle force generation at the physiological level. To address mechanical incompatibility, the system employs artificial muscle actuators that replicate the contraction dynamics of human muscles. By mimicking the natural direction of muscle force generation and the anatomical anchor points of muscle attachment, it also resolves kinematic incompatibility. Furthermore, the system uses real-time electromyography (EMG) signals for neuromuscular sensing and implements a muscle-like recruitment and rate coding algorithm to control twisted fiber soft actuators, thereby addressing control incompatibility through human-in-the-loop signal integration. A comprehensive evaluation using the Functional Reach Test (FRT) with ten older adults demonstrated a 16.6% average increase in total ankle joint torque and a statistically significant improvement in forward reach distance (from 66.4 ± 17.4 cm in the OFF condition to 77.2 ± 16.6 cm in the ON condition). These findings highlight the system’s potential to mitigate age-related muscle decline and establish a novel muscle-like actuation–motion–control paradigm for wearable assistive robotics.

Internet-based robotic teleoperation systems face significant challenges due to unpredictable Internet latency, leading to instability, reduced transparency, and synchronization issues. Existing methods exhibit fundamental limitations when addressing random delay impacts, as they typically rely on bounded delay assumptions, predictable delay patterns, or statistical models that fail in actual Internet environments where delays are inherently random and unpredictable. This paper introduces the Tele-Robotic Control Protocol (TRCP), a specialized Internet transport protocol engineered specifically for robotic teleoperation applications. Unlike conventional information-oriented protocols, TRCP implements a paradigm shift to control-oriented design through an event-based control approach that completely removes delay impact rather than merely reducing it. The protocol features a content-based validation mechanism that analyzes actual command appropriateness within the current operational context, providing unprecedented safety guarantees for teleoperation in dynamic environments and ensuring TRCP can safely support Internet-based closed-loop control, which conventional protocols fundamentally cannot achieve. Additionally, TRCP incorporates an optimized header format and bidirectional stream architecture for efficient data exchange. Theoretical analysis confirms system stability through Petri net modeling, while comprehensive experimental validation across diverse teleoperation systems. Results confirm that TRCP significantly enhances the performance of Internet-based teleoperation systems.

Sarcopenia is a progressive condition characterized by age-related losses in muscle mass and strength, and irreversible in its advanced stages. While sarcopenia negatively impacts daily living, accurately, quickly and economically assessing its effects can be challenging due to individual variability in activity levels. This study introduced a novel approach for assessing the risk of sarcopenia-prone using machine learning and wearable Internet of Things (IoT) sensors. A total of 53 community-dwelling older adults aged 65+ underwent gait analysis using dual sensors. Nineteen gait features were extracted from each cycle and used to train classification algorithms to categorize participants as healthy, risk level 1, risk level 2, or risk level 3 for sarcopenia. Binary classification of healthy versus sarcopenic-prone achieved 97.41% accuracy on average, while four-class classification averaged 94.67%. Notably, the research discovered worsening gait symmetry with increasing sarcopenia-prone severity. These results indicate IoT sensor-assessed gait may serve as a sensitive indicator for daily sarcopenia-prone screening. Accurate assessment of sarcopenia-prone individuals can be achieved through only a 4-m walking test, significantly reducing the burden for older adults. This approach offers a cost-effective, convenient, and accurate method for early sarcopenia risk detection and intervention, potentially improving quality of life for older adults. This system could also aid in creating widely applicable monitoring products for assessing sarcopenia risk, supporting IoT, and thereby enabling early identification and intervention for individuals at risk of this condition.

Accurate and prompt speed grasp intention recognition is crucial in online human-robot interaction (HRI). However, dynamic grasping relying on complete motion for high recognition accuracy will lead to an unavoidable delay in real-time prediction. To address this issue, we propose a Spatial Gating and Temporal Weighting Early Grasp Prediction (STEGP) method that utilizes incomplete dynamic grasping data from sliding windows to reduce the time delay for reliable robot teleoperation. The proposed method comprises a synergy-based feature extraction module, a spatial gating classification module, and a time-decay weighting fusion prediction module. The spatial-temporal mechanism with gating units effectively classifies sequential movements, achieving performance comparable to that of Transformers but being much easier to implement. Integrating a time-decay weighting frame enables reliable early prediction even with incomplete data. gMLP is chosen for the classification of hand dynamic grasping because of its high accuracy, realizing 93.83% accuracy for 33 grasping categories. The prediction tests demonstrated 85.4% accuracy, with the first 25% grasp completion across 28 subjects. Online robotic teleoperation grasp experiments achieved a 57.4% reduction in time delay and a 93.3% success rate.

Soft pneumatic actuators (SPAs) are widely used in robotic applications due to their inherent compliance and outstanding mechanical performance. However, a tradeoff between load capacity and deformation capacity is required when selecting material hardness for SPAs. Too hard material usually results in limited deformation, such as small extension, bending, and twisting, while too soft a material decreases robustness. This study introduces coil-reinforced flat tube actuators (CFTAs) that exhibit excellent flexibility and high load capacity by braiding flat tubes in coil springs. By adjusting the braiding pattern of the flat tube, the CFTAs can realize extending, in-plane bending, and out-of-plane helical bending motions. In addition, analytical models are proposed to predict the deformation behavior of the CFTAs and verified by experiments. The bending type CFTA deforms with an excellent curvature (0.516 mm−1) under 160 kPa input pressure, five times larger than the reported SPAs on the same scale. The CFTAs show high design flexibility by programming the flat tube pattern and using multiple coiled springs for wide application scenarios. Based on the CFTAs, this study shows wearable upper limb robots, a soft entanglement gripper, and a rob-climbing robot. CFTAs provide design insight for applications requiring dexterous and versatile deformations.

Assist-as-needed (AAN) assistance can promote active voluntary participation in rehabilitation and motor function recovery of post-stroke patients. However, different patients have personalized damaged regions and recovery states, causing difficulties to obtain adaptive and customized assistance in robot-assisted rehabilitation. This paper presents a mirror Adaptive Assist-As-Needed (AAAN) scheme, including two modules of Multi-Sensors Fused Estimation (MSFE) and Online Incremental Mirror Adaptation (OIMA), to encourage the subjects to actively participate in rehabilitation. Specifically, the first MSFE module can obtain the needed assistance based on the functional capability of the post-stroke patients via the data fusion of biological and motional signals using Kalman Filter. The second OIMA module fine-tunes the control torques estimated by MSFE to adapt the muscle fatigue and stiffness varieties of the affected limb based on the motion and physiological reference of the mirror healthy limb. The results demonstrate that the AAAN strategy can realize the transparent mode for healthy subjects and promote post-stroke patients to rehabilitate the affected limb with active participation using EMG signals 90.5% similar to those of the mirror healthy limb. The proposed method can be expected to greatly enhance power assistance and rehabilitation outcome of post-stroke patients using exoskeletons by provoking active participation. Note to Practitioners—For robotic rehabilitation, it is crucial to provide suitable assistances that can maximize the participation of post-stroke patients, which can promote the recovery outcome of therapies. The main purpose of this work is to achieve the adaptive assist-as-needed control strategy for upper limb rehabilitation tasks in two steps. Firstly, the elbow joint torques of a post-stroke patient are estimated by data fusion of motion and electromyography (EMG) signals using Kalman Filter, which can make up for the shortcomings of the individual signals, such as poor reliability and low sensitivity. Secondly, the joint motion and EMG signals of the mirror healthy limb are used as the reference to calculate the adaptive needed assistance to rehabilitate the affected limb. The preliminary experiments with healthy and post-stroke subjects demonstrate that this approach can obtain stable motion with the natural physiological states of subjects and enhance active voluntary participation in rehabilitation. In the future study, it will be investigated how to accelerate the adaptation of new patients based on the knowledge of the learned individuals using machine learning methods, such as lifelong learning and incremental learning.

Robot motion control aims to generate control inputs for a robotic system to track a planned trajectory. Feedback provided by sensors plays an essential role in motion control by improving system performance when external disturbances and/or initial errors exist. However, feedback signals, such as images are often of a large size, which imposes a heavy computational burden on the system. In this paper, a new robot motion control scheme is proposed based on compressive feedback to improve feedback rate. The controller is designed in non-vector space using compressive feedback. As an application, visual servoing is formulated under the proposed framework by considering a feedback image as a set, instead of a traditional feature vector. Experiments are conducted to validate the proposed scheme.

Spatio-temporal information is key to resolve occlusion and depth ambiguity in 3D human pose estimation. Previous methods have focused on either temporal contexts or local-to-global architectures that embed fixed-length spatiotemporal information. To date, there have not been effective proposals to simultaneously and flexibly capture varying spatiotemporal sequences and effectively achieves real-time 3D human pose estimation. In this work, we improve the learning of kinematic constraints in the human skeleton: posture, local kinematic connections, and symmetry by modeling local and global spatial information via attention mechanisms. To adapt to single- and multi-frame estimation, the dilated temporal model is employed to process varying skeleton sequences. Also, importantly, we carefully design the interleaving of spatial semantics with temporal dependencies to achieve a synergistic effect. To this end, we propose a simple yet effective graph attention spatio-temporal convolutional network (GAST-Net) that comprises of interleaved temporal convolutional and graph attention blocks. Experiments on two challenging benchmark datasets (Human3.6M and HumanEva-I) and YouTube videos demonstrate that our approach effectively mitigates depth ambiguity and self-occlusion, generalizes to half upper body estimation, and achieves competitive performance on 2D-to-3D video pose estimation. Code, video, and supplementary information is available at: http://www.juanrojas.net/gast/

Bio-syncretic robots made up of living biological systems and electromechanical systems may have the potential excellent performance of natural biological entities. Therefore, the study of the bio-syncretic robots has got lots of attention in recent years. The 3D skeletal muscles have been used widely, due to the considerable contraction force and the controllability. However, the low differentiation quality of the C2C12 in the tissues hinders the broad application in the development of the skeleton muscle actuated bio-syncretic robots. In this work, an approach based on circular mould and rotary electrical stimulation to build high-quality muscle rings, which can be used to actuate various bio-syncretic robots, has been proposed. Firstly, the advantage of the proposed circular mould for the muscle rings culture has been shown by simulation. Then, the muscle rings have been fabricated with different moulds using the experiment-optimized compositions of the biological mixture. After that, the muscle rings in the circular moulds with different electrical stimulations have been cultured, to show the superiority of the proposed rotary electrical stimulation. Moreover, the contractility of the muscle rings have been measured under the different electrical pulses stimulation, for the study of the control property of the muscle rings. This work may be meaningful not only the development of bio-syncretic robots actuated by 3D muscle tissues but also the muscle tissue engineering.

Micro-robots have a great application prospect in the biomedical field due to the feature of small size. To solve the issues of energy supply and bio-compatibility of micro-robots, bio-syncretic micro-robots composed of biological materials and electromechanical systems have been studied widely. The skeletal muscle is a potential material to develop bio-actuator for the bio-syncretic robots on account of the great contraction force and the controllability. However, the low differentiation quality of C2C12s and the control of the bio-syncretic robots are the two of the main challenges for the development of the bio-syncretic robots based on the skeleton muscle. In this paper, an approach based on circularly distributed multiple electrodes (CDMEs) was proposed to improve the differentiation of C2C12 myoblast cells and characterize the electro-responsive beating behavior of myotubes for the development of bio-syncretic robots. Three groups of C2C12 blasts were used to fulfill the differentiation experiments without electrical stimulation and with electrical stimulation using parallel electrodes and CDMEs respectively, for evaluating the effect of CDMEs on C2C12 differentiation. It was demonstrated that electrical field through CDMEs can improve the differentiation quality of C2C12 blasts into myotubes in terms of intensity, length, and widths. Then, the effect of electrical stimulation on the beating behaviors of myotubes was also investigated with CDMEs, and it was shown that the beating amplitudes of myotubes were significantly affected by the frequencies, amplitude and direction of electrical stimulation with respect to the myotubes, which is fundamental for the control of the micro-robot based on skeletal muscle cells. The proposed approach is useful for not only the development of the bio-syncretic robots, but also the study of muscle tissue engineering.

The robotic grasp detection is a great challenge in the area of robotics. Previous work mainly employs the visual approaches to solve this problem. In this paper, a hybrid deep architecture combining the visual and tactile sensing for robotic grasp detection is proposed. We have demonstrated that the visual sensing and tactile sensing are complementary to each other and important for the robotic grasping. A new THU grasp dataset has also been collected which contains the visual, tactile and grasp configuration information. The experiments conducted on a public grasp dataset and our collected dataset show that the performance of the proposed model is superior to state of the art methods. The results also indicate that the tactile data could help to enable the network to learn better visual features for the robotic grasp detection task.

This article develops a set space visual servoing method that is quiet different from state-of-the-art approaches. Our approach does not require complex image processing techniques for the extraction, matching and tracking of image features. Instead, it only requires a simple matching algorithm and builds visual errors in set space. Each error is mainly related to one degree of freedom of the camera; therefore, we can design a decoupled control law. This control law is robust and does not require calibrated inner parameters of the camera. Our approach has been validated in 4-degree-of-freedom (DOF) visual servoing simulations with common image patterns and 6-DOF visual servoing experiments with specific image patterns. These visual servoing tasks are properly achieved even when partial occlusions occur.

Bio-syncretic robots, consisting of living biological materials and traditional electromechanical systems, have attracted lots of attention due to the potentialities of self-sensing, self-actuation and self-repairing with intrinsic safety and high energy conversion efficiency. However, most of current researches focus on the movement of the devices, and have ignored the study on the control of actuation unit “cells”, which is as important as motors for traditional electromechanical robots. In this work, the effects of cell culturing time, seeding concentration and functional drugs (cytochalasin and adrenalin) on contractile frequency and force strength of cardiomyocytes have been studied using scanning ion conductance microscope (SICM) and arrays of micro-pillars made of PDMS. This work will lay the foundation for the further study of quantitatively control of bio-syncretic robots actuated by cardiomyocytes and is also meaningful for the development of cytology, medicine, and clinical science.

In atomic force microscopy (AFM)-based nanomanipulation, the tip position uncertainties still exist due to the parameter inaccuracies in the open-loop compensation of the piezo scanner, the noise in the closed-loop control and thermal drift. These spatial uncertainties are very challenging to be directly estimated owing to the lack of real-time feedback, and its effects are more significant in performing an automatic nanomanipulation/assembly task than macro world manipulations. In this paper, we propose a stochastic framework for feature-based localization and planning in nanomanipulations to cope with these uncertainties. In the proposed framework, some features in the sample surface are identified to calculate their positions in statistics, and detected by using the AFM tip as the sensor itself through a local scan-based motion. In the localization, the Kalman filter is used through incorporating the tip motion model and the local scan-based observation model to estimate the on-line tip position in the task space. The simulation and experiments about tip positioning are carried out to illustrate the validity and feasibility of the proposed algorithm. Then, positioning tip for effective nanomanipulation is presented by using several experiments. Finally, a carbon nanotube is followed to show that the proposed method can provide a great potential for improving the position accuracy.

Nanorobot has the potential to automate the manipulation and observation processes at molecular scale. Current drug discovery process is labor and cost intensive. Thus, a strong demand exists to automate this process. Here, we developed an Atomic Force Microscopy (AFM) based nanorobot for in situ sensing molecular interactions for drug discovery. The AFM tip and sample substrate are functionalized by the molecules of interest. Via measuring the interactive binding forces between these two molecules using the AFM based nanorobot, we are able to test the effectiveness of drug candidates on attenuating or enhancing the molecular interactions involved in cell signaling pathways. To measure the single molecular interactions precisely, we developed a new substrate coating method. The new method functionalizes the substrate much more evenly compared with the previous method. We further optimized settings during measuring the interactive binding force, such as coating with bovine serum albumin (BSA) and the level of indentation of AFM tip onto the substrate. With these progresses, this nanorobot measures single molecular interactions automated. We further used this nanorobot to test a small peptide, which was designed to attenuate the interactive binding force between focal adhesion kinase (FAK) and protein kinase B (AKT), two molecules involved in cell adhesion.

Cellular mechanical properties as the main physical performance characteristics have been actively studied in the past years for the study of cytobiology and the development of medicine. In this study, by combining Hertz model, a novel strategy is proposed to simultaneously measure the cellular mechanical properties including cellular mass, elasticity and viscosity, based on the principle of forced vibration stimulated by simple harmonic force, with piezoelectric transducer (PZT) as vibrator and Atomic Force Microscope (AFM) as detector. The corresponding theoretical model was derived and the simulation was realized based on the proposed model. The experiments of indentations and vibrations with myoblasts and myotubes were implemented to calculate the three mechanical parameters of cells according to the proposed strategy. The results validated the proposed approach. This work would be useful for the development of cytology, medicine, previously diagnose, specific therapy and so on.

Atomic force microscopy (AFM) is a powerful measurement instrument, which has been widely implemented in various fields. To enhance the maneuverability, an AFM can be modified and its cantilever can be controlled as a robotic end-effector. To precisely control the nanorobots, their scanners inherent hysteresis, which is the main disadvantage, should be compensated sufficiently. Mostly, hysteresis compensators used in AFM control system only employ the symmetric models, which cannot well represent the asymmetric hysteresis phenomena of piezo-scanners. However, under many cases, the smart material based scanners possess asymmetric hysteresis slightly or seriously, which is far more complex than the regular symmetric case; in addition, for commercial or customized AFMs, the scanners are usually controlled without feedback. These drawbacks tackle further improvement of positioning accuracy of AFM systems. To effectively describe and further reduce the hysteresis of general cases, we propose a new type of generalized Prandtl-Ishlinskii (PI) operator based superposition model, named unparallel PI (UPI) model. The flexible edge of the UPI operator can be tilted freely within a defined range, which enables it to capture asymmetric hysteresis efficiently. To cancel the hysteresis effect in an open-loop system, three inverse compensation approaches are proposed, compared, and the corresponding stability is analyzed. Experiments with AFM based nanorobot verified the proposed approaches with convincing performance.

In this paper, two techniques have been presented for slippage detection. They are independent of sensor signal type and are promising for general use on tactile array sensors. The first method is based on frequency analysis of the correlation coefficient sequence of sensor array data sampled as time evolves. The main idea is that a slippage causes heavier fluctuation in the sensor signal distribution and values than a static case. The second approach employs 2-D cross correlation to detect displacements of the sliding object from tactile images, which makes it possible to estimate the slippage velocity using commercially available sensors rather than custom hardware. Experiments have been implemented to evaluate the proposed approaches. It can be seen that the first method is capable of detecting both translational and rotational slippage. Also, it works well in dynamic environments. Additionally, the ability of the second method to detect slippage velocity has been confirmed in the experiment.

Small insects can use vision to successfully land on arbitrary surfaces. For miniature robots with a size of a few centimeters, it is challenging to achieve similar capabilities due to their small sizes. To address this problem, we present a new vision based landing algorithm using a non-vector space control method, which formulates the system dynamics in the space of sets—different from the traditional vector space with a linear structure. Using a stabilization controller in the non-vector space, we can control the landing posture of a small robot with an active tail. Preliminary experimental results demonstrate the effectiveness of the proposed approach. The research presented in this paper, together with the idea of compressive feedback—when only a compressed image is used for feedback—can potentially achieve real time onboard vision based control for miniature robots.

A new planning and control scheme for natural language control of robotic operations using the perceptive feedback is presented. Different from the traditional open-loop natural language control, the scheme incorporates the high-level planning and low-level control of the robotic systems and makes the high-level planning become a closed-loop process such that it is able to handle some unexpected events in the robotics system and the environment. The experimental results on a natural language controlled mobile manipulator clearly demonstrate the advantages of the proposed method.

Mobile manipulators provide more advantages and flexibility in a wide range of applications than standard manipulators by introducing mobility. However, adding mobile platforms to standard manipulators, especially nonholonomic mobile platforms, introduces new challenges to the system modeling and control. Most existing methods for mobile manipulators do not consider the performance difference between the mobile platform and the manipulator and therefore cannot handle the uncertain and unexpected events happened in both the mobile platform and the manipulator. This paper introduces a planning and control method in a perceptive reference frame for a nonholonomic mobile manipulator to efficiently handle uncertain and unexpected events. The experimental results on a nonholonomic mobile manipulator demonstrate the effectiveness and advantages of the designed method.

Robot calibration is widely used in the manufacture industry to enhance and achieve a higher level of accuracy on industrial robot manipulators. Currently, there are many reliable calibration systems able to perform the calibration task. Those systems, however, are far from being a user friendly calibration tool; they are time consuming, very expensive, and usually requires a lot of human interaction. Therefore, we proposed a new calibration system able to overcome those problems with promising results. However, to take the system to the next level, and create an automated calibration process, we need to create a control system capable of guiding the robot's tool center point to a multi-position alignment. Throughout this paper the control approach needed to achieve automation of the entire system is presented and discussed. Simulations and experimental results demonstrated the feasibility of the overall calibration system including hardware, software and control algorithms.

In this paper, we present the design and development of a miniature robot that is able to run and jump. This robot can use wheeled locomotion to travel on the flat ground. When it encounters a large obstacle compared to its size, it can stand up and leap over the obstacle. The robot has a mass of 25 grams and a maximum size of 9 centimeters. Experimental results show that with a take-off angle 80°, the robot can jump up to 1.44 meter in height and 0.59 meter in distance. Moreover, it has on-board energy, control, and communication abilities, which enables tetherless or autonomous operation. With the multi-modal locomotion abilities, the robot is expected to have many applications ranging from environmental monitoring, search and rescue, to military surveillance.

Standard manipulators are restrained in many applications due to their limited working ranges. Adding mobile platforms, in particular nonholonomic mobile platforms, can expediently enlarge their working ranges but also introduces new challenges. The problem of the existing control methods for nonholonomic mobile manipulators is that they leave out the consideration of the differences between the mobile platform and the manipulator such as the dynamics differences and working condition differences. This may consequently result in some unnecessarily large errors for the motion tracking in the implementation. To address this problem, this paper proposes a new practical control method using the adaptive motion distribution and coordination between the mobile platform and manipulator to minimize the errors of the motion control and also automatically handle some unexpected events. The effectiveness and advantages of the proposed method were demonstrated through both simulation and experimental results.

In this paper, we present a non-vector space stochastic control method for nano robotic manipulations. Non-vector space control employs sets as the states and formulates dynamics equations in the space of sets. This method provides a natural description of physical phenomena such as images. When applied to nanomanipulations, it excludes the use of external position sensors, and the positioning precision can be as good as the imaging resolution. In this paper, we investigate the stochastic control problem for non-vector space dynamic systems when the sets are corrupted by random noises. We use Kalman filtering to estimate the true set for feedback control. The stability of the method is investigated. We apply the method to atomic force microscope (AFM) system to improve position accuracies of nanomanipulations. Simulation and experimental results have clearly validated the proposed method.

Robots often have limited knowledge and need to continuously acquire new knowledge and skills in order to collaborate with its human partners. To address this issue, this paper describes an approach which allows human partners to teach a robot (i.e., a robotic arm) new high-level actions through natural language instructions. In particular, built upon the traditional planning framework, we propose a representation of high-level actions that only consists of the desired goal states rather than step-by-step operations (although these operations may be specified by the human in their instructions). Our empirical results have shown that, given this representation, the robot can reply on automated planning and immediately apply the newly learned action knowledge to perform actions under novel situations.

This paper describes an approach for a robotic arm to learn new actions through dialogue in a simplified blocks world. In particular, we have developed a three-tier action knowledge representation that on one hand, supports the connection be-tween symbolic representations of lan-guage and continuous sensorimotor repre-sentations of the robot; and on the other hand, supports the application of existing planning algorithms to address novel situ-ations. Our empirical studies have shown that, based on this representation the robot was able to learn and execute basic actions in the blocks world. When a human is engaged in a dialogue to teach the robot new actions, step-by-step instructions lead to better learning performance compared to one-shot instructions. 1

Though the structured light measurement system has been successfully applied to the profile measurement of diffuse objects, it is still a challenge to measure shiny objects due to the mix of both specular and diffuse reflections. To this end, we propose a robust encoding and decoding method in this paper. First, the monochromatic stripe patterns are utilized to eliminate the effect of texture and color of objects. Second, an intensity mask, dynamically adjusting the intensity of a projected pattern, is applied to avoid overexposure without any pre-knowledge of the workpiece. Thus, it is more flexible and efficient, compared with the existing methods. Third, to solve the internal reflection of the shiny part, an extrapolation model, combined with the intensity mask, is developed to detect the stripe edge for pattern decoding, resulting in accurate and robust 3D reconstruction. Compared with traditional polarization based methods, it does not need to readjust for a new part. The experimental results show that the proposed method is capable of measuring various parts without surface pretreatment.

Conventional scanning probe microscopy (SPM)-based nanomanipulations always have to face scanner accuracy problems such as hysteresis, nonlinearity, and thermal drift. Although some scanners consist of internal position sensors, the sensitivity is not high enough to monitor high-resolution nanomanipulations. Additionally, once the scan size decreases to a nanolevel such as less than 100 nm, the noise brought by sensors is large enough to affect the performance of the closed-loop motion control system. In this paper, a non-vector space control strategy based on compressive feedback is proposed in order to improve the accuracy of SPM-based nanomanipulations. In this approach, local images (or compressive data) are used as both the reference input and feedback for a non-vector space closed-loop controller which considers the local image (or compressive data) as a set. The controller is designed in non-vector space, and it requires no prior information on features or landmarks which are widely used in traditional visual servoing. In this paper, the atomic force microscopy is used as an example of SPM to implement the non-vector space control strategy for nanomanipulations. The motivation of designing such a non-vector space controller is to solve the accuracy problem in nanomanipulation. Without this technique, the SPM-based nanomanipulations, such as nanomeasurement and nanosurgery, are difficult to conduct, with accuracy controlled under several nanometers. In order to illustrate the contributions and potential applications of this non-vector controller, at the end of this paper, an application of carbon nanotube local electrical property characterization based on a non-vector space motion control is shown to clearly verify the concept. Compared with other research in the local electrical property characterization, the non-vector space controller can ensure that the measurement accuracy (position error) is controlled within a few nanometers, which also ensures the reliability of measurement results. Additionally, this non-vector space control method can be implemented into any kind of SPM to realize a real-time control for nanomanipulation such as nanofabrication and nanoassembly.

In this paper, we present the design and experimentation of a miniature robot that can jump, run, and perform aerial maneuvering. Specifically, this robot can use wheeled locomotion to run on the ground. Encountering an obstacle, it can jump up to overcome the obstacle. After leaping into the air, the robot can control its body angle using its tail for aerial maneuvering. To the best of our knowledge, this is the first miniature (maximum size 6.5 centimeters) and lightweight (28.0 grams) robot that having all the three capabilities. Furthermore, this robot is equipped with on-board energy, sensing, control, and wireless communication capabilities, which enables the tetherless operation. It can be potentially employed for mobile sensor networks in environments with obstacles.

One of the major limitations for Atomic Force Microscopy (AFM)-based nanomanipulation is that AFM only has one sharp tip as the end-effector, and can only apply a point force to the nanoobject, which makes it extremely difficult to achieve a stable manipulation. For example, the AFM tip tends to slip-away during nanoparticle manipulation due to its small touch area, and there is no available strategy to manipulate a nanorod in a constant posture with a single tip since the applied point force can make the nanorod rotate more easily. In this paper, a robotic nano-hand method is proposed to solve these problems. The basic idea is using a single tip to mimic the manipulation effect that multi-AFM tip can achieve through the planned high speed sequential tip pushing. The theoretical behavior models of nanoparticle and nanorod are developed, based on which the moving speed and trajectory of the AFM tip are planned artfully to form a nano-hand. In this way, the slip-away problem during nanoparticle manipulation can be get rid of efficiently, and a posture constant manipulation for nanorod can be achieved. The simulation and experimental results demonstrate the effectiveness and advantages of the proposed method.

ABSTRACT Although there has been substantial progress for multi‐robot exploration of an unknown area, little attention has been given to communication, especially bandwidth constraints in time‐sensitive and bandwidth‐consuming tasks such as search and surveillance. In such tasks, video/audio streams of a newly explored area should be sent back to the base station in a timely manner. To address this issue, we propose connectivity and bandwidth‐aware exploration (CBAX), which is an efficient iteration based real‐time exploration. CBAX divides the problem into frontier node placement, relay node placement with routing path selection, and matching of each robot with its target position. Moreover, we model bandwidth‐constrained relay node placement into a new variant of the Steiner minimum tree problem and present our solution. While reducing the exploration time, CBAX maintains the network's connectivity and ensures the aggregated data flows are under the link capacity in transmission. Simulation shows that CBAX outperforms two recent exploration schemes qualitatively by demonstrating major improvement in terms of non‐overflow transmission time and fully connected transmission time. With enhanced communication quality, CBAX still reduces the exploration time, on average, by 40% and 15%, respectively. In moderately dense scenarios, CBAX even decreases time by 50% and 25%. Copyright © 2011 John Wiley & Sons, Ltd.

In teleoperation studies, most researchers have been researching on the stability and telepresence. Few of them have studied the influence of the operational status of the teleoperator on the telerobotic systems. As a matter of fact, improper and incorrect operations of the teleoperator may decrease the teleoperation efficiency and even result in some serious safety problems even if the stability and telepresence are both guaranteed. Thus, this paper investigates a method to online identify the quality of the teleoperator and then integrate it into the planning and control of the telerobotic system. The method can help improve the performance of the system including efficiency and safety. It is also implemented on a mobile manipulator and the experimental results illustrate the effectiveness of the designed method.

AFM based nanomanipulations have been successfully applied in various areas such as physics, biology and so forth in nano scale. Traditional nanomanipulations always have to approach the problems such as hysteresis, nonlinearity and thermal drift of the scanner, and the noise brought by the position sensor. In this research, a compressive feedbacks based non-vector space control approach is proposed for improving the accuracy of AFM based nanomanipulations. Instead of sensors, the local image was used as the feedback to a non-vector space controller to generate a closed-loop control for manipulation. In this paper, there are four research topics: First, local scan strategy was used to get a local image. Second, since the feedback is an image, a non-vector space controller was designed to deal with the difficulty in vector space such as calibration and coordinate transformation. Third, in order to further decrease the time spent on local scan, compressive sensing was introduced to this system. Finally, to overcome the disadvantage that compressive sensing costs time on reconstructing the original signal, we directly use the compressive data as the feedback. Both theoretical analysis and experimental results have shown that the system has a good performance on AFM tip motion control. Therefore, the non-vector space control method can make visual servoing easier, and the compressive feedback could make a high speed real-time control of nanomanipulation possible. In addition, this new method can be applied to nano-assembly, nano-imaging and nanomanipulation.

Teleoperation systems extend the ability of human power to manipulate objects in distant locations and have a wide range of applications in many areas. For decades, most researchers focused on the stability and telepresence of the teleoperation systems. Few of them have studied the influence of the teleoperation condition variables on the telerobotic operations, such as the quality of teleoperator, task dexterity and network quality. In fact, these variables may seriously affect the telerobotic operations and system performance even if the system stability and telepresence are both perfectly guaranteed. Thus, this paper investigates the method to online identify these condition variables and then employs them to enhance the telerobotic operations with multiple objectives. The designed method was implemented on a mobile manipulator and the experimental results demonstrated its effectiveness.

Redundant nonholonomic mobile manipulators have wide range applications in civilian and military areas. The high redundancy provides high operation flexibility but also introduces redundancy resolution problems. The existing methods mainly focus on the off-line redundancy resolution, and most of them are based on single objective optimization. However, in order to efficiently accomplish a specific task, the dynamic environment information, task constraints and requirements should be considered. This paper investigates a new method which employs the onboard sensor information to resolve the redundancy online by realizing multiple objectives optimization. The effectiveness of this method is demonstrated by simulation results.

This paper together with the accompanied video presents our improved single motor actuated miniature jumping robot. The robot has a maximum size 6.5 centimeter and a mass 23.5 gram. It can jump towards a desired direction and jump continuously. With a take-off angle 75°, the average jumping height is 0.9 meter. In the video, the detailed robot design is illustrated, and experiments in various situations are presented. These scenarios suggest potential applications of such miniature jumping robots such as surveillance, environmental monitoring, or locomotion in environments with obstacles.

With respect to ground level, wireless communication signal strength increases with the elevation of communicating wireless sensor network devices, within practical bounds. Jumping sensors are mobile sensors that provide relocation capabilities and a temporary increase in elevation can be utilized for improving communication. This paper provides a comprehensive multidimensional analysis for jumping sensors. It studies the main factors that impact the Received Signal Strength (RSS) in sensor communication, and performs a comparative analysis between theoretical and experimental results. Sensor elevation from ground level is a key factor, which is often neglected, and plays an important role for successful wireless communication. The impact of jump height manipulation on a jumping sensor to the packet transmission goodput is presented. An airborne two-way communication scheme is defined and studied with experiments. The results indicate the effectiveness of utilizing the change in elevation of a jumping sensor to increase communication range. Since energy is at premium in sensor nodes, the operational energy cost of a jumping sensor prototype is studied in detail. A jumping sensor network is simulated with the parameters learned from the experimental results and the jumping sensor prototype analysis. Simulation results show the enhancement in connectivity and the feasibility of a jumping sensor network.

A humanoid neck design with low motion noise yields a cable-driven parallel manipulator to imitate two axis rotational motion of a human neck. The fixed base and moving platform of the manipulator are connected by four cables and a column compression spring. The four cables are actuated separately, while the spring can support weight on the moving platform. Although similar manipulators exist in literature, the analysis for them is scarce. The difficulty is that while traditional parallel manipulator usually has a rigid kinematic chain as the spine, this manipulator has a flexible spring. With the spring's lateral bending motion, new approach must be adopted to solve the kinematics. In this paper, we propose a method which combines the kinematics with the statics to solve them simultaneously. We parameterize the posture of the moving platform with four parameters and consider one of them as parasitic motion. Using the spring bending equation, we can obtain the parasitic motion and solve the inverse position problem. The analysis in this paper provides a novel way to analyze parallel manipulator with a spring spine, and it can be applied to other manipulators with flexible spines.

Robotic teleoperation constitutes a key technique for networked manufacturing system. Since Internet is noisy and performance of telerobotic systems strongly depends on communication conditions, control strategies which could be used to distribute computation and communication resource properly and require less data transmission are required. Motion description language (MDL) is an appropriate solution. In this paper, comparison study between MDL-based and the traditional direct control method under randomly varying time-delay condition has been pursued which show that the MDL-based method has the ability to guarantee the stability of the whole telerobotic system under randomly varying time-delay condition. Furthermore, MDL-based method provides an uniform mathematical description on discrete control on the master site and continuous execution on the slave site. Simulation results indicate that the proposed method is promising for Networked Manufacturing development.

A miniature robot with continuous jumping ability is presented in this paper. The robot has a dimension about 6cm×8cm×2cm and weighs 20 grams. To achieve continuous jumping, various mechanisms are needed including the jumping mechanism, energy store and release mechanism, self-righting mechanism, and jumping direction changing mechanism. The design and analysis for those mechanisms are elaborated in this paper. Moreover, implementation and experimental results are also presented. It is shown that the robot can jump higher than 55cm with a 75° takeoff angle. The robot can be used as mobile sensors and deployed in the areas of rugged terrain and natural obstacles which are not suitable for sensors with wheels.

In this paper, we focus on developing a new constant-voltage-driver (CVD) based electrotactile display system. The system owns the feature of on-line identifying the bio-impedance parameters of fingertip skin for individual tactile preference tuning and haptic rendering enhancement. The identification method relies on a discrete-time extended least squares (ELS) iterative approach with forgetting factor (FF), which serves as an adaptive law to on-line estimate parameters of the first-order bio-impedance model of fingertip skin when the stimulation current control is performing in z-domain. Experimental results demonstrate the performance of the developed CVD electrotactile display, as well verify the effectiveness of the proposed on-line identification method. That is, it is capable of quickly estimating parameters of the fingertip skin bio-impedance model. The significance of our work is that it will greatly improve the tactile/haptic rendering performance of electrical stimulation (electrotactile) systems and benefits the development of electrotactile based haptic rendering devices and applications.

Multiple robots can be teleoperated by a single operator to provide enhanced capacity and efficiency on accomplishing complicated tasks. The design of this kind of systems is challenging because simultaneously tele-controlling multiple robots exceeds the ability of one single operator. The intelligence and autonomy of the robots need to be integrated into the system. Besides, the communication between the operator and the multi-robot system and the communication among the multiple robots are both subject to communication constraints like time delays and packet losses. This paper designs a non-time based method to realize the single-operator-multi-robot system with random communication delay. The system is designed based on the non-time based teleoperation method and a proposed perceptive coordination method. The random delay problem and the problem of simultaneously tele-controlling multiple robots by a single operator are resolved. Experiments implemented on a multi-robot system illustrated the effectiveness of the design.

Telerobotic systems have been researched for decades due to their extensive applications in many civilian and military areas. Most research mainly focused on either the stability or telepresence of the telerobotic systems. Few studies have investigated the effects of the confidence of the teleoperator on the performance of the teleoperation. The confidence of the teleoperator is of significant importance to the efficiency and safety of the telerobotic systems. This paper proposes a concept named quality of teleoperator (QoT) to represent the confidence of the decisions and commands generated by the teleoperator. The value of QoT is computed based on a set of mental states of the teleoperator. Based on the QoT, a control adjustment mechanism is designed to enhance the efficiency and safety of the telerobotic systems. Experiments were implemented on a manipulator to demonstrate the effectiveness of the proposed method.

A MDL-based control method for tele-robotic systems over internet is presented in this paper. Internet-based tele-robotic systems are characterized by the fact that the operator and the remote robotic system are connected by Internet. Since limited bandwidth, random time delay and other transmission problems deteriorate performance of tele-robotic systems severely, control strategies which can be used to settle transmission problems are expected. The obvious characteristic of the new method introduced in this paper is that control commands have a linguistic flavor, data transferred between operator and remote robot can be reduced accordingly. The framework presented in this paper has the potential of reducing operator's working pressure and enhancing the performance of the system. Furthermore, the existing motion description language modal is expanded to deal with the packet loss and disorder problem. The main contribution of this paper is the development of a novel framework for tele-robotics and its implementation in a real tele-manipulator system.

This paper presents our recently developed humanoid neck system that can effectively mimic motion of human neck with very low motion noises. The feature of low motion noises allows our system to work like a real human head/neck. Thus the level of acoustic noises from wearable equipments, such as donning respirators or chemical-resistant jackets, induced by human head motion can be simulated and investigated using such a system. The objective of this investigation is to facilitate using head-worn communication devices for the person who wears the protective equipment/uniform that usually produces communication-noise when the head/neck moves. Our low motion-noise humanoid neck system is based on the spring structure, which can generate 1 Degree of Freedom (DOF) jaw movement and 3DOF neck movement. To guarantee the low-noise feature, no noise-makers like gear and electro-driven parts are embedded in the head/neck structure. Instead, the motions are driven by seven cables, and the actuators pulling the cables are sealed in a sound insulation box. Furthermore, statics analysis of the system has been processed completely. Experimental results validate the analysis, and clearly show that the head/neck system can greatly mimic the motions of human head with an A-weighted noise level of 30 dB or below.

A new approach for adaptive torque distribution of 4-Powered Caster Vehicle (4-PCV) is presented on complex terrain without any additional sensor. The objective is that torques applied to wheels are dynamically redistributed based on the real time conditions of the whole wheel-ground interactions in order to track the desired trajectory. A novel approach based on the redundant actuated wheels is proposed to identify the status of the vehicle and the wheel slip ratio by only observing the velocity feedback from motors encoders. A dynamic model considering the wheel-ground interaction is described. Based on the slip ratio of the wheel joints and the null space of the operational space, control strategies are employed to redistribute the torques applied to the wheel joints so that each wheel can be self-adapted to meet a complex wheel-ground condition to eliminate slippage with high rate. Simulation results show the effectiveness of the proposed estimation approach and the performance of the torque distribution schemes.

While there has been substantial progress for multi-robot exploration of an unknown area, little attention has been given to communication, especially bandwidth constraints in time-sensitive and bandwidth-consuming tasks such as search and surveillance. In such tasks, video/audio streams of a newly explored area should be sent back to the base station in a timely manner. To address this issue, we propose Connectivity and Bandwidth Aware eXploration (CBAX), which is an efficient iteration based real-time exploration. CBAX divides the problem into frontier node placement, relay node placement with routing path selection, and the match of each robot with its target position. Moreover, we model bandwidth-constrained relay node placement into a new variant of the Steiner Minimum Tree problem and present our solution. While reducing the exploration time, CBAX maintains the network's connectivity and ensures the aggregated data flows are under the link capacity in transmission. Simulation shows that CBAX outperforms two recent exploration schemes qualitatively by demonstrating major improvement in terms of non-overflow transmission time and fully-connected transmission time. With enhanced communication quality, CBAX still reduces the exploration time, on average, by 40% and 15% respectively. In moderately dense scenarios, CBAX even decreases time by 50% and 25%.

The objective of this paper is to propose a robust one shot structured light pattern for real time 3D shape inspection system. To reduce the influence of inspected part reflectance property and ambient light, the pattern is constructed by using monochromatic light. The corner of the chessboard is utilized as the primitive of the pattern since it can provide highly accurate position. Additionally, the orientation of the corner is used to encode the primitive of the pattern. Compared with ordinary two dimensional patterns, the pattern is developed in one dimension, along the epipolar line so that the search of the corresponding pixels between the projector and the camera is speeded up. Last, experiments were conducted to evaluate the robustness and accuracy of the inspection system using the proposed pattern. The results demonstrate that the system has high accuracy performance.

Three dimensional optic measurement system's accuracy is highly related with the field of inspection. Increasing of field inspection costs increasing camera / projector pixel area on the test surface. Small surface changes within one pixel area cannot be directly detected, which will lower the system accuracy. A pixel-to-pixel strategy is developed to solve this problem. Increasing field of inspection also costs a longer standoff distance. The random image noise from the environment, uncertainties functions by lens distortion and resolution variation are all amplified. Therefore, a more complicated calibration model for each pixel is proposed to calibrate the system. In traditional structured light vision systems, a single sensor usually detects around 10,000 - 50,000 mm2, and the 3D vision sensor in this paper needs to detect around 2,400,000 mm2. Larger detection range gives more challenge to finish the calibration tasks. This paper proposes a clear calibration procedure to a large field of inspection structured light system. Last the comparison with the CMM measured results is used to prove that the calibration tasks have been successfully achieved.

Mobile sensors with jumping ability provide several advantages compared with the traditional wheeled sensors such as ability to move in rugged terrain. A controllable jumping robot for this purpose is described in this paper. The robot has dimension about 9.5cm × 9cm × 3cm and weighs 54.1 grams. It can perform the jumping process continuously. This paper focuses on the mechanisms to achieve such a continuous jumping ability, including the jumping mechanism, energy store and release mechanism, and self-righting mechanism. Detail implementation and experimental results are also given in this paper. It is shown that with a 75° takeoff angle, the robot can jump about 20cm in height.

Research in rehabilitation engineering shows that electrodes can produce tactile sensations with appropriate electrical signals tailored to stimulate the multiple tactile receptors located under the fingertip skin. However, electrical stimulation that is suitable in terms of current or voltage level for tactile sensations experienced from an individual cannot be guaranteed for every user; the dynamic range is largely dependent on the user. An identification method is then necessary for characterizing the parameters of the skin-electrode interface circuit model as to improve rendering consistency and comfort for every user regardless of skin condition. In this paper, we focus on developing a custom-built electro-tactile display terminal for data collection and an identification method to determine the individual bio-impedance parameters based on the well-known skin-electrode interface circuit model: the Cole-Cole circuit model. The goal of this work is to develop an online identification and stimulation current control approach for enhancement of electro-tactile-based rendering in applications such as rehabilitation, sensory substitution, telepresence, etc.

One of the prerequisite conditions for making a successful manipulation is that the relative position between the AFM tip and the objects can be sensed and controlled accurately. While this prerequisite is grandly hampered by the PZT nonlinearity and thermal drift. Although the PZT nonlinearity can be compensated to a certain extent through mounting a position sensor on the PZT scanner, this method leads to a higher system noise and a higher cost. In addition, this method can not handle the positioning error caused by thermal drift due to the lack of sensing ability to the displacement between the AFM tip and the sample stage. This paper propose a newly developed strategy to solve these problems. Its pivotal idea is the tip position is localized based on the sensing information to sample features, not PZT driving voltage or sensor signal. In this way, the positioning error aroused from PZT nonlinearity and thermal drift can be effectively suppressed. Experimental results demonstrate the advantage and effectiveness of the proposed method.

This paper describes an industrial robot joint offset calibration method called the virtual line-based single-point constraint approach. Previous methods such as using CMM, laser trackers or cameras are limited by the cost or the resolution. The proposed method relies mainly upon a laser pointer attached on the end-effector and single position-sensitive detector (PSD) arbitrarily located on the workcell. The automated calibration procedure (about three minutes) involves aiming the laser lines loaded by the robot towards the center of the PSD surface from various robot positions and orientations. The intersections of each pair of laser lines eventually should converge to the same point after compensating the joint offsets. An optimization model and algorithm have been formulated to identify the robot offset. For the highly precise feedback, a segmented PSD with a position resolution of better than 0.1 ¿m is employed. The mean accuracy of robot localization is up to 0.02 mm, and the mean error of the parameter identification is less than 0.08 degrees. Both simulations and experiments implemented on an ABB industrial robot verify the feasibility of the proposed method and demonstrated the effectiveness of the developed calibration system. The goal of fast, automated, low-cost, and high precision offset calibration are achieved.

This paper describes our updated system for industrial robot joint offset calibration. The system consists of an IRB1600 industrial robot, a laser tool attached to the robot's end-effector, a portable position-sensitive device (PPD), and a PC based controller. By aiming the laser spot to the center of position-sensitive-detector (PSD) on the PPD with different robot configurations, the developed system ideally implements our proposed calibration method called the virtual line-based single-point constraint approach. However, unlike our previous approach, the calibration method is extended to identify the offset parameters with an uncalibrated laser tool. The position errors of the PPD and the sensitivities of error in the PSD plane to the variation of joint angles are analyzed. Two different robot configuration patterns are compared by implementing the calibration method. Both simulation and real experimental results are consistent with the mathematical analysis. Experimental results with small (10−3−10−2) mean and standard deviation of parameters error verify the effectiveness of both the sensitivity analysis and the developed system.

Hopping sensors are a type of low cost mobile sensors that are small in size, have limited capability and imprecise movement. However, their unique method of movement makes them suitable for rugged terrains. Sensors may fail when deployed in a rugged terrain or in an obstacle-abundant environment. Therefore, redundant sensors may be identified and relocated to the sensor holes. This paper addresses the problem of relocating such capability-constrained sensors in an obstructive environment. We propose an enhanced quorum-grid solution with binary splitting message forwarding (BSMF), which is decentralized and can detect both existing and newly appearing obstructions in the supplier and consumer cells matching process. Furthermore, a grid-based movement model is introduced for the hopping sensors. Simulation shows that our scheme significantly reduces the communication overhead and achieves relatively constant total energy consumption with varying amount of obstructions.

The non-contact 3D sensing technology, though achieved many success in a variety of applications, needs an automation system to expand its applications to automotive industries for 3D shape inspection. The reason is the difficulty for an operator to find an optimal solution by the manual control of sensor viewpoints. The problem of this industrial application is the capability for the sensing system to simultaneously satisfy all requirements of competence, efficiency, and cost. A robot-aided 3D sensing system can provide such a solution. A CAD-guided robot view planner can automatically generate viewpoints. Measurement accuracy can be satisfied in a certain range. However, the unpredictable image noises still need to be compensated for better measurement performance. In this paper, a feedback planning system is designed and applied to the CAD-guided robot sensor planning system. The feedback controller can automatically evaluate the accuracy of obtained point clouds and generate new viewpoints. This feedback-based inspection system had been successfully implemented in filling holes of a point cloud, caused by shadows and light reflections. Such a system had been implemented on an ABB industrial robot for a 3D measurement of an automotive glass and a pillar. This paper introduces the developed planning system and our current results.

The objective of this paper is to propose a wind- shield surface shape and optical parameters inspection system. In this paper, a white board works as a diffuse planar light source projecting structured light patterns because physical properties of specular surface do not allow us to directly apply the triangulation-based techniques used in ordinary diffuse surface inspection. The board is placed at two different positions and the distorted structured light patterns are observed by a fixed camera so that the incident vector for every point on the windshield surface is determined using corresponding points in the board at two different positions. Likewise, the reflection vector is determined by camera calibration. Hence, the 3D shape and normal of the windshield surface are obtained by the intersection of the incident and reflection vectors. The normal of the surface denotes the optical reflective property of the windshield. Last, accuracy and consistence experiments are conducted. The experiment results demonstrate the efficiency and accuracy of our system.

Nanomanipulation with Atomic force microscopy (AFM) is one of the fundamental tools for nano-manufacturing. The control of the nanomanipulation system requires accurate feedback from the piezoelectric actuator and high frequency response of the control system. We designed and implemented two distinct control schemes by using real-time Linux. The aim is to study various factors in the control of the AFM based nanomanipulation system. By integrating the original controller with the external Linux real-time controller, we achieved a stable system with high response frequency. Finally this multiple input single output (MISO) system is validated to be an effective and efficient tool for the controlling of the nanolithography operation through a haptic device.

We present the design and implementation of a new jumping robot for mobile sensor network. Unlike other jumping robots, the robot is based on a simple two-mass-spring model. After we throw it on ground, it can stabilize itself and then jump once. The detailed mechanism design including the load holding and self-stabilization are presented. Jumping heights and distances with different robot weights are measured and compared with calculated values from the two-mass-spring model.

One of the main roadblocks to Atomic Force Microscope (AFM) based nanomanipulation is lack of real time visual feedback. Although the model based visual feedback can partly solve this problem, its unguaranteed reliability due to the inaccurate models in nano-environment still limits the efficiency of AFM based nanomanipulation. This paper introduce a Real-time Fault Detection and Correction (RFDC) method to improve the reliability of the visual feedback. By utilizing Kalman filter and local scan technologies, the RFDC method not only can real-time detect the fault display caused by the modeling error, but also can on-line correct it without interrupting manipulation. In this way, the visual feedback keeps consistent with the true environment changes during manipulation, which makes several operations being finished without a image scanning in between. The theoretical study and the implementation of the RFDC method are elaborated in this paper. Experiments of manipulating nano-particles have been carried out to demonstrate the effectiveness and efficiency of the proposed method.

For 3D dimensional inspection systems, point clouds measured on each viewpoint need to be processed for quality evaluation. Three steps are usually included in this process: filtering, registration, and error map generation. For quality control, small defects like dints and dents have to be kept in the point cloud. Therefore, a filtering algorithm is required to automatically remove outliers and keep dints/dents. Many filtering algorithm smooth the point cloud for better display, however, since the measured point cloud is used to represent the shape of the part, modification of any point's coordinates is not allowed because that will modify the error map. A point cloud filtering algorithm is developed using a link clustering algorithm to identify and remove outliers. Point cloud filtering is especially important in an iterative closest point (ICP)-based robot hand- eye calibration method because outliers will bring calibration errors into the calculated transformation matrix. With this technique, the cleaned point clouds can be directly transformed to a world frame for registration. This registration method has two advantages compared to feature-based registration methods: 1) the entire inspection process can be automatically executed, 2) avoid holes in point clouds caused by artificial markers. For error map generation, a point-to-plane distance is used in this paper which calculates the distance of a point to its closest triangle. The introduced automated inspection system had been implemented on a PUMA robot system. Experimental results are described in this paper.

The variable time delay and the packet loss degrade the performance of Internet based teleoperation system seriously, even make the system unstable. To overcome the trouble, an idea that using CARIMA model of the linearized slave robot to design a controller based on the Generalized Predictive Control (GPC) method is proposed in this paper. We place the GPC controller at the remote site. First of all, The CARIMA model of the linearized slave robot is derived. Secondly, a GPC controller is designed at slave site to generate the redundant control information to diminish the influence of the packet loss and the large time delay in the internet to the system. Moreover, in order to solve the problem caused by the variable time delay, the reference information signed with time stamp is used and feedback to the operator, so the operator can predict the next round trip time delay(RTT) according to the preceding RTT we got. Finally, stability condition is achieved. Simulation results show that these strategies can dynamically compensate for the variable time delay and reduce the performance degradation induced by packet loss.

In this paper, we present a new mathematical model for designing micro electrode geometry and electric field for carbon nanotubes application. The micro electrode design is based on the assumption that the electrical potential at any point (x,y,z) created by a micro electrode of interest is defined by a polynomial that obeys Laplacepsilas equation. By substituting this polynomial into Laplacepsilas equation the corresponding equipotentials can therefore be determined, and these in turn can be used to define the required micro electrode boundaries for use in carbon nanotube deposition, manipulation, and implementation using dielectrophoresis for electronics and sensing application. Simulation and preliminary experimental results are presented for the developed models.

In this paper we introduce a new point cloud registration strategy for 3D surface inspection, named object-oriented method. Traditionally, the registration methods use a fitting based algorithm, which is a recursive process and too time consuming to do quality inspection. Moreover, the fitting based algorithms require that the measured data and the CAD model have same shapes. Photogrammetry and ICP-based fitting algorithms are commonly used in this area. However, both of them cannot be applied directly to online point cloud registration for quality inspection. Since it requires: 1.) no markers on the object surface; 2.) a very limited inspection period of time for each part on an assembly line; and 3.) the measured work piece can not be assumed as same as the CAD model. The online registration is currently demanded by 3D surface inspection in automotive glasses manufacturing industry. The object-oriented registration method is developed based on matching the invariant geometric features between two data sets instead of the artificial markers and the recursive fitting algorithms. Therefore the registration speed is much higher. A datum filter is developed to extract the invariant features from the measured point cloud, based on the information that the CAD model carries. The curvature information and moment values are used as characteristic features to design the filter. The registration matrix is calculated by the invariance parts in both data sets. The new online registration method has been applied to the quality inspection in the windshield manufacturing industry. Experimental results are presented in this paper to demonstrate the effectiveness of the developed strategy.

This paper addresses the problem of how to navigate a miniature Crawler robot in a typical engineered structure inspection application-aircraft rivet inspection. First, a novel vision-assisted localization algorithm is developed to find the heading and position of the Crawler robot. Second, a new algorithm is developed to solve the path planning problem so that the Crawler robot can navigate through all the rivets. Experimental results validate the localization and path planning algorithms. This inspection system can be extended to other similar engineered structure inspection applications.

Network surveillance systems provide extended perception and distributed sensing capability in monitored environments through real time monitoring of the target area and target objects using multiple networked sensors. The development of wireless communication and sensing technology make it possible to deploy networked surveillance systems in various environments. We consider a surveillance network where the sensors are static. The task of tracking targets in a surveillance network is challenging because of the following reasons: (1) the location of the sensors need to be optimally deployed. (2) The view of the sensors need to be optimized so that at a given time the targets are shown with a discernable resolution for feature identification. (3) It is important to devise stable control algorithms for accomplishing the surveillance task. When the target moves, it is important to switch the sensing task between sensors to maintain the visibility of the target with adequate resolution. This paper presents a novel method to deploy static sensors given a target region and a dynamic programming method to optimally switch sensors when the target moves. Finally, simulation results demonstrate the efficacy of the proposed approach for tracking targets over an area.

The task of visual surveillance involves pervasively observing multiple targets as they move through a field of sensor nodes. Mutational analysis and shape based control have been proposed to overcome the limitations of current feature (point) based visual servoing and tracking techniques generally employed to provide an optimal solution for the surveillance task. Hausdorff tracking paradigm for visual tracking of multiple targets using a single sensor has been proposed for accomplishing the surveillance task. However, Hausdorff tracking incorporates some redundancy in the actuation mechanism. This paper exploits this redundancy in the camera motion in order to accomplish various sub-tasks which can be assigned to the system, such as minimization of consumed energy maintaining manipulability etc. The complete task can then be expressed in a multi-objective constrained optimization framework and can be solved, i.e., the input to the camera can be derived, using various methods such as physical programming, nonlinear programming, weighted sum method, etc. In this paper, we use the physical programming method based on the various advantages such as ease of expressing multiple objectives in a physically significant manner. Experimental results are provided which show the advantages of using the physical programming approach over the weighted sum method for constructing the task criterion for multi-objective optimization problems.

Accuracy is essential to surface quality control when a range sensor is applied to measure the 3D shape of an automotive body part. A sensor's viewing pose, including location and orientation, is related to measurement accuracy. It is usually difficult to find an optimal solution by manual control of sensor viewpoints. A CAD-guided robot view planner developed previously can automatically generate viewpoints. Measurement accuracy can be satisfied in a certain range. However, the unpredictable image noises, especially in regions with low intensity contrast, cannot be compensated by the CAD-guided robot view planner. In another aspect, measurement accuracy is evaluated all over the part surface. The local accuracy of a small patch may exceed the measurement tolerance. In this paper, feedback design is applied to the CAD-guided robot sensor planning system. The feedback controller can evaluate the accuracy of obtained point clouds, identify problem regions, and generate new viewpoints. This process is recursively executed until the measurement accuracy reaches to a tolerant value. This feedback-based inspection system had been implemented in previous work to fill holes of a point cloud, which are caused by shadows and light reflections. In this paper, the feedback controller is specifically designed to improve the measurement accuracy. Experimental results show the success of applying this feedback system for dimensional inspection of an automotive body part.

This paper introduces our recently developed high precision robot localization system employing position sensitive detectors (PSD). A lateral effect PSD is an ideal position sensitive device with a resolution of approx. 0.5 mum. Such a device can be used to enhance the precision capabilities of robot localization or robot calibration. The developed PSD guided localization system is comprised of a structured laser pointer system attached to the robot end-effector and a PSD fixture. It is capable of feeding back the movement information of the TCP relative to the PSD frame. This is achieved by detecting the position of the laser beams emitted from the laser pointers onto the PSD surface. The system was well designed, mapped, and constructed and hence allows it to perform precision localization of the robot. Simulations and preliminary experiments demonstrated the effectiveness of the developed PSD guided localization system including hardware, software, and algorithms. Therefore, it is capable of improving the localization performance of robots as well as promoting the application of precision robot calibration.

This paper presents a novel automated manufacturing process for mass production of nano devices from advanced material such as carbon nanotubes (CNTs). CNTs have been found to be a promising and advanced material for nanoelectronics due to their size and excellent mechanical and electrical properties. Conventional electronic devices replaced by CNT are possibly to be miniaturized, and CNT based nano devices can provide better performance. Therefore, researchers have focused on developing different methods to manufacture CNT based nano devices in recent years. Since the size of CNTs is in nano scale, traditional robotic manipulation cannot be applied. Electrical assembly of CNT based devices is one of the promising methods to manipulate CNT to desired position. Building nano device by a single CNT is challenging, therefore, most people have concentrated on manipulating bundled CNTs. However, the electrical properties of bundled CNTs are difficult to control. As a result, device cannot benefit from quantum properties of a single CNT. Therefore, an automated process for manufacturing single CNT based nano devices is necessary for this application. A CNT deposition system is developed to manipulate a single CNT across microelectrodes precisely and repeatedly by using dielectrophoresis (DEP). Moreover, certain types of CNTs can be selected by using a micro chamber to filter other unnecessary types of CNTs. The system can potentially be used to fabricate an array of CNT based devices, and a fast and feasible batch nano assembly of consistent nano devices can be achieved.

Atomic Force Microscope (AFM) has been used as a manipulation tool for a decade. The problem of lacking real time visual feedback still limits its efficiency and hinders its wide application. Although the model based visual feedback can partly solve this problem, due to the complexity of nano environment, it is difficult to use a model to accurately describe the object’s behavior. The modeling error will give the operator a false feedback and lead to a failed manipulation. In this paper, a strategy for visual feedback error on-line detection and correction is proposed to solve this problem. As the real time force information is a key factor for this strategy, an adaptable end effector is employed to accurately measure the interaction force between the probe and the nano-objects, and the system error is also compensated to improve the accuracy of interaction force measurement. Based on the true real time force information, an extended Kalman filter is developed to online detect whether there is a false feedback. Once a false feedback is detected, an optimal searching pattern is generated to get the real manipulation result in a short time. With the assistance of this strategy, the false visual feedback can be realtime detected and corrected without interrupting manipulation. Complex manipulation task can be finished without being interrupted by a new image scan. Experiments of manipulating nano-particles are performed to verify the effectiveness of this strategy, which demonstrated the improved efficiency of the AFM based nano-assembly system.

Carbon nanotubes (CNTs) are good candidates for many electronics and sensing applications. These applications will require moving (deposit) and orienting carbon nanotubes to specific location, and separating CNTs with semiconducting and metallic band structure. In this paper, a new mathematical model describing precisely the rotational motion of carbon nanotubes in viscous medium (acetone) is presented. This new mathematical model correctly assumes that carbon nanotubes form a line shape after undergoing AC electric field compare to existing model that assumes carbon nanotubes to be a spherical particle. The new mathematical model is based on a newly developed model for applied torque due to electrorotation. It is also a method for controlled assembly of the CNTs on microstructures that have the plausibility to be scaled to wafer- level manufacturing. Simulation results are presented for the developed models. The developed model can be used to detect the position of carbon nanotubes and further implemented in the separation of semiconducting and metallic band structure carbon nanotubes.

This paper presents our development of a network-enabled human/robot cooperative interface for tele- diagnostics of breast pathology. To enhance telepresence capabilities, our human/robot interface is comprised of an anthropomorphic arm/hand equipped with haptic and tactile sensing, ultrasound imaging capabilities, a physician interface capable of rendering both haptic and tactile information, and two-way audio and video. This cooperative and multimedia interface was tested via the Internet and enables tele-diagnostics. Preliminary experiments had demonstrated the effectiveness of the developed human/robot tele-diagnostic interface. That is, it will greatly enhance and improve the diagnostic performance as well as promote the tele-diagnostics of breast pathology in remote areas a reality.

A distributed processing and communication approach is applied to networked surveillance systems providing an extended perception and sensing capability in monitored environments. By pervasive we mean that the entire surveyed area is covered by a heterogeneous collection of fixed and mobile sensors. The sensor network uses a cooperative tracking technique that allows for the deployment of mobile sensors based on data provided by other sensors in the network. Also, an image-based tracking technique is used for tracking when the target is in view. The task of tracking multiple targets in a distributed surveillance network is a challenging problem because of the following reasons: (1) multiple targets need to be monitored and tracked continuously and must remain in view of at least one of the sensors; (2) the view of the sensors needs to be optimized so that the targets are observed with a discernible resolution for feature identification; (3) it is important to devise stable control algorithms for accomplishing the surveillance task; (4) assigning tracking tasks to sensors must consider load balancing and efficient use of all sensors. This paper presents a distributed communication and processing model that allows for a fast deployment of sensor nodes and implementation of ad hoc tracking in a multi-target surveillance scenario. Also, experimental results demonstrate the efficacy of the proposed approach for tracking multiple targets over a large area with fixed and mobile sensors

Motion control is investigated for a nonholonomic mobile manipulator using hybrid joints, which could be switched to either active (actuated) or passive (underactuated) mode. The mobile platform is driven with two independent wheels and the arm has hybrid joints in the horizontal plane. The dynamic constraints are shown to be a first-order nonholonomic for the mobile base and a second-order nonholonomic for the link when one of hybrid joints is underactuated. Two Different motion control methods for mobile underactuated manipulator with one passive joint are proposed. Simulation and experimental studies show the effectiveness of the proposed methods

A robotic device with haptic, tactile, and ultrasound capabilities, which can acquire and render information of breast pathology was developed. A physician interface that can examine the human breast remotely and accurately, using such a robotic device was also developed. Such a robotic device can be used to do screening or focused breast exams for patients in remote areas without convenient access to physicians. Because of in-situ ultrasound imaging, examination by the robotic device may prove to be more accurate than examination by the physicians own hand. In addition, the robotic device can also be used to train healthcare professionals in breast pathology

This paper introduces a novel robotic range sensor planning system, which is developed for 3D dimensional inspection of automotive body parts. For active, triangulation-based range sensors, shadow and reflection causes problems when measure a metal part with glossy and discontinuous surface. To solve these problems, a feedback based dynamic view planning system is proposed, which not only generate viewpoints from a CAD model of a part, but also recursively add viewpoints according to the measured information. The planning process stops only if the obtained point clouds meet the pre-determined requirements. General framework of the system is introduced, and the experimental results are also presented

In this paper, a new pneumatic end effector system for micro/nano fluidic handling, nanomanufacturing, and micro/nano manipulation is presented. The new micro pneumatic end-effector system consists of a DC micro-diaphragm pump and compressor, one region of flexible latex tube, a Polyvinylidene Fluoride (PVDF) sensor for in-situ measurement of micro force, and a micro steel tip. The micro steel tip of the new pneumatic end effector system has an internal diameter (ID) of 20 mum used for handling nano entities such as carbon nanotubes, DNA, micro/nano particles as well as for microfluidic handling and droplet control. The DC micro-diaphragm pump is automatically controlled via a voltage driver interfaced with a computer in order to effectively and efficiently control suction force and pressure during microfluidic handling and droplet control in nano manufacturing. The new pneumatic end effector system with force sensing can significantly improve the success rate for handling/depositing micro/nano entities in the case of carbon nanotubes. The experimental results show the success rate of placing carbon nanotubes between electrodes can reach close to 80%. Ultimately, the technology will provide a critical and major step towards the development of automated manufacturing process for batch assembly of micro devices, manufacturing of nano devices, microfluidic droplet control, and drug delivery

Pervasive surveillance can be defined as continuous monitoring and tracking of multiple targets in a large monitored region so that they do not leave the field of view (FOV) of the sensors observing them. Despite the limited sensing capability and range of the individual sensors, the surveillance network can track targets over a large region based on transferring the target tracking task. The challenge for such large scale networked systems is to design an efficient and scalable modeling and analysis tool and devise stable control algorithms for accomplishing the surveillance task. Mutational analysis and shape based control have been proposed to overcome the limitations of current feature (point) based visual servoing techniques, however, they fail to capture the discrete switching nature of the surveillance task of tracking the target using multiple sensors. This paper presents a mutational hybrid model for such pervasive surveillance networks which retains the advantages of using mutational equations while also being able to model the discrete switching between various sensors. We also present an example pervasive surveillance scenario modeled using the proposed method and experimental results verifying the proposed approach. © 2006 IEEE.

This paper presents the development of a tactile-haptic interface for quantifying surface texture properties such as roughness, veins, and softness of automobile interior. This interface will assist the designer to quantify and classify the comfort index of automobile interior design for quality control, as well as provide customers a sensory experience of surface textures so as to determine the desired surface characteristics for each individual customer. To realize this purpose, a high resolution optical tactile sensor for accurately capturing interior surface textures is developed. Based on the intensity information of the achieved 2-D tactile image, the contact area map and the 3-D shape of the surface texture is recovered. In addition, a high performance haptic electrotactile array device was fabricated for rendering the tactile sensation of surface texture in the fingertip of the designer/customer. Calibration and experimental results verified the high performance of the developed tactile haptic interface. This interface can be further developed to be an effective and efficient knowledge based evaluation system for improving automobile interior design

This paper aims at investigating and characterizing force behavior and mechanical properties of living drosophila embryos using an in situ modeled PVDF (polyvinylidene fluoride) piezoelectric microforce sensing tool with a resolution in the range of sub-muN. Drosophila embryo is one of the most studied organisms in biological research, medical research, genetics and developmental biology, and has implications in the cure of human diseases. In order to achieve high efficiency and accuracy during microinjection of genetic material into a drosophila embryo, it is absolutely necessary to allow close monitoring of the magnitude and direction of microinjection forces acting on the embryo during injection. In this paper, a microrobotic biomanipulation platform integrating a two-axis (2-D) modeled PVDF micro-force sensor is used to implement force sensing during microinjection of living drosophila embryos. Micro injection forces and membrane deformation of embryos in different stages of embryogenesis are found. Ultimately, the technology will provide a critical and major step towards the development of automated biomanipuation for batch microinjection of living embryos in genetics

Structural health monitoring and inspection is very important for many civil, mechanical and aerospace systems. It is a critical step in maintaining and improving the structural integrity of these systems. In this paper, we propose to develop an automated, intelligent inspection system for these engineered structures, which employs a team of intelligent climbing robots and a command robot to collaboratively carry out the inspection task. To support autonomous navigation, a Miniature Active Camera (MaCam) module is designed, which can be used in the pose calibration of the robot. The robot pose error model is introduced. Based on that, the path planning problem for a single robot is studied and a hierarchical algorithm is developed to generate a path that satisfies the navigation requirements of the robot. Both the error model and the path planning algorithm is verified in experiments.

Point cloud registration and sensor calibration are two critical technical issues concerning robot-mounted, area sensor systems. Iterative closest point (ICP)-based algorithms developed in the past are commonly used for point cloud registration. However, due to its least squared fitting nature, registration quality depends on how closely the measured part matches its nominal definition, typical in the form of a CAD model in modern times. To eliminate the dependency of registration quality on part closeness to the CAD model, we present, in this paper, a more robust approach based in a series of coordinate transformations. Geometric features and surface gradients are accounted for to improve the registration performance. To achieve robot/sensor hand-eye calibration, an ICP-based method is used. The reason is that this calibration step typically utilizes standard parts or gauges machined for the purpose of calibration, as such they are known shapes that match their CAD models with much tighter tolerances. This offers us an unique opportunity to apply an ICPbased tool to find the transformation matrix from the robot end effector to an area sensor mounted onto it. The discussed method was successfully implemented and tested in a feedback-based, robot-mounted area sensor system developed for manufacturing quality control of 3D freeform surfaces

For AFM based robotic nanomanipulation system without displacement sensor, one of the key technical problems is to realize high accurate positioning of the AFM probe. To solve the problem, based on the hysteresis and nonlinear characteristics analysis of AFM PZT actuator, a new actuating method called "actuating method based on reappearing the scanning trajectory" is presented to actuate the PZT actuator. Then two kinds of probe positioning errors, namely kinematics coupling errors due to the tube actuator's bend motion and probe tip's positioning errors caused by cantilever deflections, are compensated to further improve the probe's positioning accuracy. Nanopatterning experiments are performed to verify the effectiveness of the new actuating method and the compensation methods of probe positioning errors

Cast aluminum wheels must be deburred and polished before chrome plating operation. Manual deburring and polishing is tedious and monotonous. Industrial robots are attracting more and more attention for the deburring applications. However, Each wheel path program was taking eight to ten weeks to create because the position and orientation of the deburring tool must be accurately identified based on the curvature of the wheel. To solve this problem, we developed a practical 6 DOF robot path generation method using hybrid force and visual servoing methodology. The force servoing keeps the robot tool continuously contacting with the wheel surface and the visual servoing controls the robot tool to follow a marked tool path on the wheel while the position and orientation are controlled and recorded. A robot path is then generated from the recorded data. Experiments are performed and the experimental results demonstrate that the developed algorithm can be used to automatically generate accurate 6 DOF robot paths for deburring aluminum wheels in less than an hour. This significantly reduces the robot programming time. The developed technology can also be used for robot path generation in many other manufacturing applications, such as welding and strip painting

An atomic force microscope (AFM) explores the topography of a sample surface using a micro-sized flexible cantilever, which works as a flexible robot arm. The flexible cantilever is controlled to keep vibrating when an AFM works in the tapping mode. The cantilever is modeled as a flexible beam instead of a point mass system in this paper. The nonlinear interaction force between the tip and sample surface is also modeled. A simulation environment is developed to analyze the dynamics of cantilevers using the flexible beam model. Simulation results confirm that the flexible beam model can represent the system more accurately than the point-mass model. It has been shown that lower modes are more sensitive to changes of surface topography or surface materials when the cantilever is driven to vibrate at a higher harmonic mode. At the same time, this simulation environment also provides a more accurate way to validate the design of a new AFM probe and AFM controller than simulation packages which use the point-mass model

Nanoassembly using atomic force microscopy (AFM) is a promising technique for nanomanufacturing. Most AFM-based nanoassembly schemes are implemented either manually using haptic devices or in an interactive way between the users and the atomic force microscope images. These schemes are time consuming and inefficient. Therefore, the computer-aided design (CAD)-guided automated nanoassembly using AFM is desirable for nanomanufacturing. In this paper, a general framework for CAD-guided automated nanoassembly using AFM is developed. Based on the CAD model of a nanostructure, the manipulation paths for both nanoparticles and nanorods are generated automatically. A local scanning method is developed to compensate for the random drift that may cause the failure of the nanoassembly. The experimental results demonstrate that the developed general framework can be employed to manufacture nanostructures efficiently. The research work opens a door to the CAD-guided automated nanomanufacturing using AFM. Note to Practitioners-Atomic force microscope (AFM)-based nanoassembly will lead to potential breakthroughs in manufacturing new revolutionary industrial products because many potential nanostructures and nanodevices are asymmetric, which cannot be manufactured using self-assembly only. In order to increase the efficiency and accuracy of AFM-based nanoassembly, automated computer-aided-design (CAD)-guided nanoassembly is desirable to manufacture nanostructures and nanodevices. Based on the CAD model, the environment model and the model of the nanoobjects, collision-free paths are generated to control the AFM tip to manipulate nanoobjects. A local scanning method is developed to obtain the actual position of each nanoobject to compensate for the random drift. Since the building materials of nanostructures and nanodevices may include nanoparticles, nanorods, nanowires, nanotubes, etc., automated path planning algorithms are developed for both nanoparticles and nanorods. The experimental results show that the developed general framework can be used to manufacture nanostructures more efficiently.

The use of nanomaterials in nanotechnological applications, and developments in nanoscale manipulation and assembly, are discussed. There are many nanoscale materials with unique mechanical, electrical, optical, and chemical properties which have a variety of potential applications in nanodevices, nanosensors, and nanoelectromechanical systems (NEMS). The ability to manipulate the nanomaterials in a controllable manner is very critical to make these nanomaterials useful in nanotechnology. Nanomanipulation and nanoassembly are one of the important challenges in realizing the miniaturization of devices and machines potentially down to atomic and molecular sizes. The development of an automated microspotting system for rapid dielectrophoretic fabrication of bundled carbon nanotube sensors, provides a new method for using microfluidic and dielectrophoretic forces for nanoassembly.

Available bandwidth (AB) defined as a minimum spare capacity of links constituting a network path is an important QoS characteristic of the path. We propose to improve a whole range of "probe-rate" AB measurement tools that send sequences of measurement packets (called "trains") across the network path. If transmission times of packets in the train, called one way delays (OWD), show an increasing trend as packet sequence number in the train increases, then AB is believed to be lower than the rate at which the train was sent. In contrast, an absence of a trend indicates that AB is higher than the rate of the train. We propose an algorithm for efficient OWD trend detection and compare it to widely used OWD trend detection tests.

For Internet based real-time teleoperation systems, random time delay can cause instability in the closed loop control system and hence hinder task accomplishment. Event based control systems have been proposed to overcome the instability caused by the random time delay. High latency at the transport layer can still impede effective and reliable execution of tasks with high dexterity requirements. Network QoS based dynamic resource allocation has been proposed to increase the efficiency and reliability of task execution. However, these approaches only try to mitigate or overcome the effects of random time delay and do not address the cause of latency issues in the communication channel. This paper addresses the efficiency and reliability requirements for supermedia enhanced teleoperated systems by reducing the end-to-end transmission latency through the use of overlay networks. The proposed system reduces the transmission latency by using multiple, disjoint paths in overlay networks. The proposed system facilitates reliable and efficient task completion for tasks with high dexterity requirements. Experimental validation of the proposed teleoperated system using the PlanetLab Network is provided for the task of teleoperating a mobile manipulator system.

For the aircraft structure inspection, this paper introduces a micro biped robot with inspection probe and wireless vision, analyzes the robot locomotion modes and dynamic models, and studies the motion control algorithm. Considering the movement flexibility caused by five degrees of freedom, a hierarchy structure is presented for the robot motion control system. For the long distance locating problem, a relay locating approach is presented to solve robot locating and to estimate the orientation of the robot by using vision and distance information from encoder and CAD model. The movement orientation can be adjusted rivet by rivet in the inspection process. The experimental results show that the control algorithm works well, and orientation estimation algorithm provides an acceptable orientation precision for continuous rivet inspection.

Nanomanufacturing using Atomic Force Microcopy has been widely investigated. Most of nanomanipulation schemes go through the scan-design-manipulation-scan cycle manually which is time consuming and inefficient. Automated AFM tip path planning is desirable for nanomanufacturing, but does not receive much attention. In this paper, a CAD guided automated nanomanufacturing system is presented. Based on the CAD model of a nanostructure, the manipulation paths for both nanoparticles and nanorods are generated automatically. A local scanning method is developed to compensate for the random drift that may cause the failure of the nano-assembly. The experimental results demonstrate that the developed algorithm can be employed to manufacture nanostructures efficiently. The research work opens a door to the CAD guided automated nanomanufacturing.

a human-symbiotic safe mobile manipulator with nonholonomic constraint is described to realize human safety, impact force absorbing and task fulfillment. The robot consists of an arm covered with soft materials and hybrid joints, which can be put into active or passive mode as needed. In an unexpected or expected collision with human, the arising impulse force is attenuated effectively by the physical model formed with the hybrid joint and the soft material. Owing to the displacement movement of the link when the joint is passive, a recovery control algorithm has been developed for the end-effector to maintain its desired task position after the collision. Simulation and experiment results confirm that the proposed physical model is suitable for robot working in the human-robot symbiotic environment and the control method are useful for robot’s task fulfillment.

Atomic force microscopy is a powerful and widely used imaging technique that can visualize single molecules both in air and solution. Using the AFM tip as the end effector, it can be modified into a nano-robot which can manipulate objects in nanoscale. By functionalizing the nano-robot end effector with antibodies, atomic force microscopy is able to identify specific types of receptors on cells’ membrane in an image much as fluorescent tags do in optical microscopy but with higher resolution. After the single receptors have been identified, it becomes possible to manipulate these biological macromolecules in their physiological environment. This new technology open a promising way to study the function of biological macromolecules individually.

This paper addresses the path planning problem of a mobile sensor, which is a micro Crawler robot equipped with a Eddy Current probe, for aircraft rivet inspection. Due to the specific movement characteristic of the Crawler robot, the path, or the rivet sequence should enable the Crawler robot to realize the localization-and-reposition process. Therefore a final path is subject to the following three requirements: i) the distance between any two consecutive rivets on the path should be less than a threshold distance; ii) the number of turns should be minimized and iii) the overall distance should be minimized. In this paper, a novel algorithm is developed to generate the required path. This algorithm first identifies the minimum set of line segments partitioning all the rivets and then connects the line segments to minimize the overall distance. Simulation results are provided to validate the proposed algorithm.

This paper introduces a fully automated dimensional inspection (ADI) system using a digital area sensor. Instead of measuring a part surface point-by-point using a Coordinate Measuring Machine (CMM), an ADI system can measure a part surface patch-by-patch. Therefore, the inspection time can be significantly reduced. Many existing sensor planners were developed for one or two cameras. In the proposed ADI system, the sensor planner is developed for a digital area sensor, which is made by a CCD camera and a digital projector. In application, sensor viewpoints are estimated automatically. On each viewpoint, thousands of points on part surfaces can be measured in only several seconds.

Nanomanipulation using Atomic Force Microscope (AFM) has been extensively investigated for many years. However, control of tip position during nanomanipulation is still a major issue because of the deformation of the cantilever caused by manipulation force. The softness of the conventional cantilevers also cause the failure of the manipulation of relatively large and sticky nano-object because the tip can easily slip over the nano-object. In this paper, an active atomic force microscopy probe is used to solve these problems by changing the cantilever’s flexibility or rigidity through different control strategies in imaging and manipulation modes respectively. During imaging mode, the active probe is controlled to bend up with respect to the interaction force between the tip and samples, thus making the tip response faster and increase the imaging speed. During manipulation mode, the active probe is controlled to bend down with respect to the interaction force between tip and the samples; thus increasing its nominal rigidity to avoid tip slipping over object. A detailed model of the active probe is presented in this paper and the controller designed based on the proposed active probe model is also implemented on the augmented reality system, which is an AFM based nanomanipulation system with both real-time visual and haptic feedback. The simulation results for the control strategies and the preliminary experimental results for the AFM based nanmomanipulation verified the validity of the model and effectiveness of the controller.

In supermedia enhanced Internet based teleoperation systems, the data flowing between the operator and the robot include robotic control commands, video, audio, haptic feedback and other media types. The difference between an Internet based teleoperation system and other Internet applications are that (1) there are many media types involved in teleoperation systems and each of them has a particular quality of service (QoS) requirement; and (2) some media types are very latency sensitive. Overlay networks have been proposed to improve the QoS of teleoperation applications. However efficient use the overlay network resources and the distribution of these resources optimally to all supermedia streams remains an important problem. This paper aims to provide a framework of QoS management for teleoperation systems over overlay networks. The validity and performance of the system is evaluated using the PlanetLab overlay network.

Automatic chopper gun trajectory generation for spray forming is highly desirable for today's automotive manufacturing. Generating chopper gun trajectories for free-form surfaces to satisfy the given requirements is still highly challenging due to the complex geometry of free-form surfaces and the spray gun model. A CAD-guided chopper gun trajectory generation system for both uniform and nonuniform material distribution of free-form surfaces has been developed in our previous work. A material distribution model is also presented. To verify the developed algorithms, experiments were performed. In this paper, the experimental results are presented and compared with the simulation results. The results demonstrate that the developed trajectory generation system can be applied to generate trajectories for free-form surfaces such that the material distribution requirements can be satisfied. Also, the material distribution model can be used to compute the material distribution for free-form surfaces. This trajectory generation method can also be applied to generate trajectories for many other CAD-guided robot trajectory planning applications.

This paper reports a case study of using single 3D stereoscopic visual feedback for real-time teleoperation of dexterous tasks. In traditional teleoperation systems, real-time visual feedbacks of multiple monoscopic views of the robot workspace are provided for remote operator. However, it is difficult for the operator to control remote robot to perform dexterous tasks by looking at multiple video feedbacks at the same time. During teleoperation, remote operators usually find multiple 2D visual feedbacks confusing, especially when performing dexterous tasks that require accurate positioning and orientating of robot end-effectors. In this paper, we propose to provide single real-time 3D stereoscopic visual feedback for remote operators so that they perceive remote robot workspace with the sense of depth. This sense of 3D empowers remote operators to accurately position and orient robot end-effector with confidence. Experiments have been conducted to reveal the usefulness of real-time 3D stereoscopic video feedback over multiple monoscopic video feedback in real-time teleoperation.

Networked surveillance systems provide an extended perception and distributed reasoning capability in monitored environments through the use of multiple networked sensors. The challenge for such large scale networked systems is to design an efficient modeling and analysis tool and devise stable control algorithms for accomplishing the surveillance task. Current feature (point) based visual servo and tracking techniques generally employed do not provide an optimal solution for the surveillance task. This paper presents a mutational analysis approach for shapes, and shape based control to model and design mechanisms for such active surveillance systems. The techniques of image based Hausdorff tracking and cooperative Hausdorff tracking are introduced. Finally, experimental results demonstrate the efficacy of the proposed approach for tracking targets over a large area.

So far, in-situ PVDF (polyvinylidene fluoride) films bonded to the surface of flexible cantilever structure act as the micro-force sensors, they are mostly modelled using quasi-static relationships. However, such sensors are usually a significantly compliant and easily deformable structure in order to reach highly sensitive performance in micromanipulation. As a result, this may be reasonable to consider bandwidth measurement and high frequency response for achievement of high accuracy, and thus a dynamic analysis of such sensors become essentially necessary. In this paper, a cantilever beam based micro-force sensor was designed based on the infinite dimensional system model (distributed parameter model) using the Bernouli-Euler formulation. Furthermore, in order to enable an engineering implementation, we used a zero frequency term to effectively replace the high order modes of the dynamic sensing model in the prescribed frequency range. The corrected model can efficiently minimize the effect of removed higher order modes, and then the micro-force measurement can be obtained accurately with this corrected in-bandwidth dynamic model. Preliminary simulation and experimental results both verified the performance of the developed dynamic micro-force sensor and the effectiveness of the corrected model.

This paper aims at developing an active force sensing technology for micromanipulation and microassembly using in-situ piezoelectric polyvinylidene fluoride (PVDF) films symmetrically bonded to the surface of a flexible cantilever beam structure. The designed micro-force sensing beam has both sensing and actuating layers. The sensing layer can detect the deformation signal due to the external micro-force acting at the sensor tip, the signal is then fed back to the actuating layer through a servoed transfer function or servo controller, as a result, a counteracting bending moment generated by the actuating layer can be used to balance the deformation of sensor beam in real time. Once balanced, the sensor tip will maintain in the equilibrium position as if the sensor stiffness is virtually improved, yielding accurate motion control of the sensor tip. Especially, the micro-force can be obtained by calculating the balance force through the counteracting servo voltage applied to the actuating layer. The developed active structure greatly enlarge dynamic range of micro-force sensor and enhance the manipulability during micromanipulation/microassembly when the sensor is mounted at the end-effector. Preliminary calibration and experimental results both verified the performance of the developed active micro-force sensor and the effectiveness of the models.

This paper presents new calibration methods for a robotic area sensing system developed for industrial manufacturing inspection. A pixel-to-pixel calibration scheme is introduced to obtain standoff and baseline distance of the developed area sensor. In the three-dimensional (3D) space, the area sensor also has an offset angle in a plane parallel to the reference plane. Calibration of this offset angle improves the measurement accuracy. This pixel-to-pixel scheme is particularly useful for calibrating a 3D optical area sensor with off-the-shelf lenses. An exploding vector method was also developed to transform the measured depth map to a 3D point cloud. A robotic area sensing system was developed by integrating an area sensor prototype and a robot sensor planning system. Calibration of the integrated system includes robot calibration, part calibration, and robot hand-eye calibration. Calibration error of the robot system is related to the area sensor positioning accuracy in the robotic system. Experimental results show a successful practice of the developed robotic area sensing system. Measurement performance of the developed system can be improved if advanced lenses are adopted.

Atomic force microscope (AFM) based nanomanipulation has been extensively investigated for many years. However, the efficiency and accuracy of the AFM based nanomanipulation is still a major issue due to the nonlinearities and uncertainties in nanomanipulation operations. The deformation of the cantilever caused by manipulation force, in our experience, is one of the most major nonlinearities and uncertainties. It causes difficulties in precisely controlling the tip position, which will cause the tip to miss the position of the object. In order to solve this problem, the traditional approach is to use a rigid cantilever. However, this will significantly reduce the sensitivity of the force feeling during the manipulation, which is essential for achieving an efficient and reliable nanomanipulation. An active AFM probe is used to solve this problem by directly controlling the cantilever's flexibility or rigidity during manipulation. An infinite dimensional model of the active probe is developed. Control of the active probe employing an optimal LQR control law is also implemented. The experimental results have verified the theoretical model and demonstrated that the precise position control and high sensitive interaction force measurement can be achieved simultaneously.

In a bilateral teleoperated system, the signal transmissions between the operator and the slave manipulators have different QoS requirements in comparison to traditional network traffic. Running teleoperated systems over wireless networks poses more challenges in comparison to wired networks. The media streams involved differentiate themselves from other media types in that they require both reliable and smooth delivery. Reliable delivery requires the transport service to have TCP style semantics. By being smooth, the transport service should be able to deliver the control and sensing data with bounded and reduced latency and its variation. For example, we have conducted numerous teleoperated experiments using our system. We have found in some of our applications that if the end-to-end latency variance becomes larger than 0.3 second, the operator has difficulty maintaining smooth control of the slave manipulator. However, our simulations show that using TCP, the end-to-end latency variance can be as much as 2.5 seconds in an ad hoc wireless network. This paper proposes an improved transport service for remote sensing and control (TRSC). The service reduces the end-to-end latency and latency variance (jitter) for real-time reliable media in mobile ad hoc networks by using forward error correction encoding and multiple network paths. Simulation using NS2 shows the approach performs well under different wireless scenarios.

Spray forming is an emerging manufacturing process. The automated tool planning for this process is a nontrivial problem, especially for geometry-complicated parts consisting of multiple freeform surfaces. Existing tool planning approaches are not able to deal with this kind of compound surface. This paper proposes a tool-path planning approach which optimizes the tool motion performance and the thickness uniformity. There are two steps in this approach. The first step partitions the part surface into flat patches based on the topology and normal directions. The second step determines the tool movement patterns and the sweeping directions for each flat patch. Based on the above two steps, optimal tool paths can be calculated. Experimental tests are carried out on automotive body parts and the results validate the proposed approach. Note to Practitioners-This paper was motivated by the problem of automatically planning tool paths for spray forming using Programmable Powdered Preforming Process (P4) technology. However, the proposed approach can be applied to other surface manufacturing applications such as spray painting, spray cleaning, rapid tooling, etc. Existing tool planning approaches are not able to handle complicated, multi-patch surfaces. This paper proposes a methodology to partition complicated surfaces into easy-to-handle patches and generate tool paths with optimized thickness uniformity and tool motion performance. We tested the approach using simulation on sample automotive body parts and proved its feasibility. However, this approach requires that the parts to be sprayed belong to the sheet-metal type so that the part geometry can be analyzed on a plane. In our future research, we will run physical tests on actual parts and investigate the deposition effects on the thickness uniformity.

During imaging and nanomanipulation with a sample-scanning AFM based nanomanipulator, because of bend motion of tube scanner, two important errors will be generated, namely scanning size error and cross coupling error, and they are destructive to both image quantitative analysis and nanomanipulation accuracy. To minimize the errors, a kinematics model of the scanner is presented, which shows that lateral and vertical displacements at any point on sample depend on its onset to tube axis, applied voltage and sample thickness besides scanner geometric parameters and piezoelectric constant. According to the model, the two errors are quantitatively analyzed. Imaging and nanolithography experiments verify the kinematics model and errors calculation formulas, and some methods are also proposed for minimizing the errors.

During the control of teleoperation via Internet, an operator cannot obtain the real-time data in a timely and accurate fashion. This is due to the uncertain time delays, which are unavoidable in the Internet-based communication and cause instability and poor system performance. This paper deals with the modelling and control of Internet-based teleoperation systems using the hybrid dynamic system approach. A supervisory controller is implemented to overcome the uncertain time delays. This method provides an efficient way to model the concurrence and complexity of the Internet-based robotic teleoperation.

This paper presents a fuzzy navigation and motion control system for miniature climbing robots. After the introduction of robot structure, a navigation approach based on a fuzzy multi-sensor data fusion scheme is proposed, which integrates task scheduling, sensing, planning and real-time execution in a unified task reference framework and makes the task synchronization much easier. For a highly nonlinear system as the miniature climbing robot, a fuzzy motion controller with gravity compensation is designed to reduce energy consumption and improve the robot performance at different situations. Experimental results prove the validity of the proposed method.

Automated chopper gun trajectory planning for spray forming is highly desirable for today's automotive manufacturing. Generating chopper gun trajectories for free-form surfaces to satisfy thickness requirements is still highly challenging due to the complexity of problems. Automated trajectory planning for constant material distribution has been widely studied. However, achieving a variational (non-uniform) material distribution has not been addressed. A CAD-guided chopper gun trajectory planning system with non-uniform material distribution for free-form surfaces is presented. A multi-objective constrained optimization problem is formulated. The simulation results have shown that this system achieves performance required by production applications. This trajectory planning algorithm can also be applied to generate trajectories for other CAD-guided robot trajectory planning applications.

Assembly of nano-structures involves manipulation of nanoparticles, nano-rods, nanowires and nanotubes. Modelling the behavior of a nano-rod or a nanotube pushed by an AFM tip is much more complex than that of a nano-particle because in the case of the nano-particle usually only translation occurs while for the nano-rod and nanotube both translational and rotational motion occurs during manipulation. In this work, the behavior of nano-rods under pushing is theoretically analyzed and the interaction among tip, substrate and nano-rods has been modelled. Based on these models, the real-time interactive forces are used to update the AFM image. The real-time visual display combined with the real-time force feedback provides an augmented reality environment in which the operator not only can feel the interaction forces but can also observe the real-time changes of the nano-environment. The new developed augmented reality system capable of manipulating not only nanoparticles but also nano-rods makes nano-assembly using AFM based nanomanipulation system feasible and applicable.

Using AFM as a nanomanipulation tool has been discussed for more than a decade. However, its lack of real-time visual feedback during manipulation has hindered its wide application. Fortunately, this problem has been overcome by our recently developed augmented reality system. Not only can this new system provide real-time force feedback but real-time visual feedback during nanomanipulation. In order to build this augmented reality system, a lot of parameters are used to model the force interaction, measure the forces, and control the tip motion. In this paper, calibration of all parameters used in the augmented reality system is discussed in details.

This paper aims at developing a force-guided micromanipulation technology with in-situ PVDF beam force sensing and hybrid force/position control based on an infinite dimensional system model. By using the designed PVDF force sensing cantilever composite structure with high sensitivity, the micro contact force/impact signal and its derivative can be extracted and processed. As the sensor structure installed at the end of micromanipulator is a soft beam, when manipulation is performed, the cantilever beam is necessary to be considered as a distributed parameter flexible link, then we developed a hybrid micro contact force/position control scheme on the basis of an infinite dimension system model. Experimental results verify the performance of the developed micro force sensing and hybrid control scheme. Ultimately the technology will provide a critical and major step towards the development of automated manufacturing processes for batch assembly of micro devices.

Spray forming is an emerging manufacturing process. The automated tool planning for this process is a nontrivial problem, especially for geometry-complicated parts consisting of multiple freeform surfaces. Existing tool planning approaches are not able to deal with this kind of compound surfaces. This paper proposes a tool path planning approach which considers the tool motion performance and the thickness uniformity. There are two steps in this approach. The first step partitions the part surface into flat patches based on its topology and normal directions. The second step determines the tool movement patterns and the sweeping directions for each flat patch. Based on that, optimal tool paths can be calculated. Experimental tests are carried out on automotive body parts and the results validate the proposed approach.

This paper presents a modeling and analysis method for the motion planning and control of mobile robot systems in a hybrid fashion. Robotic systems obtain environmental information from perceptive sensors and respond to the perceptions to execute the task through decision and control process. The hybrid perceptive model of the robotic system has one continuous and two discrete layers, and it also has continuous and discrete perceptive references. The discrete layers enable the robot system to plan and modify original path through switching. Stability can be guaranteed in switching. The hybrid perceptive references can not be stopped by blocking the continuous reference, including obstacles. The model is verified by experiments.

This paper presents a distributed model for cooperative multiple mobile robot systems. In a multiple robot system, each mobile robot has sensing, computation and communication capabilities. The mobile robots spread out across certain area and share sensory information through an ad hoc wireless network. The multiple mobile robot system is therefore a mobile sensor network. In this paper, Voronoi diagram and Delaunay triangulation are introduced to model the area coverage and cooperation of mobile sensor networks. Based on the model, this paper discusses a fault tolerant algorithm for autonomous deployment of the mobile robots. The algorithm enables the system to reconfigure itself such that the area covered by the system can be enlarged. The proposed formation control algorithm allows the mobile sensor network to track moving target and sweep a larger area along specified paths.

This paper presents a fuzzy system approach that incorporates sensing, control and planning to enable micro wall climbing robots navigating in unstructured environments. Based on a fuzzy multi-sensor data fusion scheme, a novel strategy for integrating task scheduling, sensing, planning and real-time execution is proposed so that task scheduling, action planning and motion control can be treated in a unified framework, and the design of task synchronization becomes much easier. A hybrid method is employed to create robot's gaits by switching the robot between continuous motion modes with the help of a finite state machine driven by the signals from robot sensors. For a highly non-linear system as the micro wall climbing robot, a fuzzy motion controller is designed to improve the robot's performance at different situations. Experimental results prove the validity of the proposed method.

This paper proposes an integrated nano-liter bio-material spotting system. The system can be used for DNA gene-chip microarray, protein microarray and other chemical microarray fabrication. The system features a fixed print head, which significantly reduces spotting position and spot size uniformity variations. The system also features a unique spotting pin and print head alignment system design.

This paper surveys methods for drug delivery and discusses micro/nano-devices for controlled drug delivery. The concept of a micro-robot for controlled drug delivery and micro-needles for Chinese acupuncture are also presented. The proposed micro-needles can reduce pain and provide an alternative for drug delivery to the Chinese acupuncture treatment, these micro-needles are fabricated using standard micro-fabrication techniques.

The development of nanomanufacturing technologies will lead to potential breakthroughs in manufacturing of new industrial products. Nanomanufacturing by manipulating nanoparticles using an atomic force microscope is desirable to manufacture asymmetric nanodevices and nanostructures. The complexity of nanomanufacturing requires positioning, manipulating and assembling nanoparticles. Typical manual nanomanipulation is time-consuming and inefficient. Automated path planning is desirable for nanomanufacturing, but does not receive much attention. In this paper, a general framework is developed to manufacture nanostructures and nanodevices. An automated tool path planning algorithm is presented. Simulations are performed to test the generated paths. The generated paths are also implemented to manipulate nanoparticles to manufacture nanostructures automatically. The simulation and experimental results are consistent. The general framework can also be extended to manipulate other nanoobjects.

Significant research has been conducted in the field of Internet-based teleoperation. However, there is a lack of objective performance measures beyond completion time, which is dependent on several external factors to the system. This paper develops the concept of event-synchronization for supermedia enhanced Internet based teleoperation systems. Supermedia is the term used to refer to the different feedback streams; for example, video, haptic, temperature and others. This performance measure or system property is not affected by external factors; such as, the human operator and the communication characteristics. In addition, the design, which is based on Petri net theory, of systems satisfying this property is detailed. The experimental results obtained using a mobile manipulator bilateral teleoperation system, are given.

In this paper, a human-symbiotic robot is described to realize human safety, impact force absorbing and task fulfillment. The robot consists of an arm covered with soft materials and hybrid joints, which can be put into active or passive mode as needed. In an unexpected or expected collision with human, the arising impulse force is attenuated effectively by the physical model formed with the hybrid joint and the soft material. Owing to the displacement movement of the link when the joint is passive, a recovery control algorithm has been developed for the end-effector to maintain its desired task position after the collision. Simulation results confirm that the proposed physical model is suitable for robot working in the human-robot symbiotic environment and the control method is useful for robot's task fulfillment.

DNA electronics circuits require an efficient way to accurately position and individually manipulate DNA molecules. The recent development of atomic force microscopy (AFM) seems to be a promising solution. We have recently developed an AFM based augmented reality system. This new system can provide both real-time force feedback and real-time visual feedback during nanomanipulation. We have shown that nano-imprinting and manipulation of nano-particles and nano-rods can be easily performed under assistance of the augmented reality system. In this research, the system's ability is extended to manipulation of DNA molecules. Using a polynomial fitting method, the deformation of DNA molecules is displayed in real time in the augmented reality system during manipulation. Indeed, DNA molecules adopt many different structures including kinks, bends, bulges and distortions. These different structures and inappropriate physical contacts may result in the controversy of DNA conductivity reported over the last decade. The AFM based nanomanipulation system can be used either as a nanolithography tool to make small gap electrodes or a nanomanipulation tool to elongate, deform and cut DNA molecules. The measurement of the conductivity of DNA molecules in their different shapes and structures is a promising method to find conclusive evidences, which verify the electrical conductivity of DNA molecules.

Robots are developed to perform myriad kinds of complex tasks. To enhance the system performance, tasks should be optimally distributed among robots or subsystems of a single robot. Applied to mobile manipulators, optimal task distribution can be obtained by implementing an optimal redundancy resolution approach. A lot of work has been done in this area with a priori-specified tasks. However, for tele-operated mobile manipulator systems, tasks are generated on-line by the operator, and task requirements vary significantly and are not known a priori. This renders static task distribution schemes unsuitable for achieving optimal performance and mandates the use of online modification of optimal task distribution algorithms. This paper proposes a new optimal task distribution method for tele-operated mobile manipulators. In this method, task dexterity indices, which describe the task requirements are generated online. Based on these indices, the criterion function for optimal performance is constructed using physical programming. The solution obtained by this algorithm is more suitable for varying tasks, and has a better defined physical meaning. The effectiveness of the proposed method is verified by experimental results.

This paper aims at advancing micromanipulation technology with in situ polyvinylidene fluoride (PVDF) piezoelectric force sensing during microassembly and packaging process. Based on the previously developed PVDF 1-D sensor, by employing the parallel beam structure, a novel 2-D force sensor with relatively high natural frequency and sensitivity is optimally designed. The sensor can detect micro force and force rate signals, which can be fed back so as to greatly, improve the reliability of microassembly. Preliminary calibration and experimental results on assembly of surface MEMS devices both verified the performance of the new 2-D sensor that demonstrates a high sensitivity and a resolution in the range of /spl mu/N. Ultimately the technology would provide a critical and major step towards the development of automated micro-manufacturing processes for batch assembly of micro devices.

This paper proposes a reliable and efficient multi-robot coordination algorithm to accomplish an area exploration task given that the communication range of each robot is limited. This algorithm is based on a distributed bidding model to coordinate the movement of multiple robots. Two measures are developed to accommodate the limited-range communications. First, the distances between robots are considered in the bidding algorithm so that the robots tend to stay close to each other. Second, a novel coding mechanism is introduced in the map representation to reduce the exchanged data volume when new communication links are formed. Simulation results show the effectiveness of the use of the nearness measure in the coordination algorithm as well as the new coding mechanism to reduce the amount of map data exchanged. By handling the limited communication range we can make the coordination algorithms more practical in multi-robot applications.

The timed Petri net model introduces the possibility of applying a set of mathematical results, mainly based on the use of max-plus algebra, for performance analysis. This paper aims to build a new scheduling method for a flexible manufacturing work-cell by merging the timed Petri net model and max-plus algebra. The results can be computed as functions of a certain set of decision parameters. These functions can be used to schedule, plan and control the flexible manufacturing work-cell, so that it can real-timely adapt itself to machine faults.

This paper presents a fuzzy control system that incorporates sensing, control and planning to improve the performance of the wall-climbing microrobots in unstructured environments. After introduction of the robot system, a task reference method is proposed which is based on a fuzzy multisensor data fusion scheme. The method provides a novel mechanism to efficiently integrate task scheduling, action planning and motion control in a unified framework. A robot gait generation method is described which switches the robot locomotion between different motion modes with the help of a finite state machine driven by sensory information. A fuzzy motion controller is designed to improve control performance and reduce power consumption by the suitable selection of fuzzy sets and inference methods, as well as the definition of corresponding membership functions and control rule bases. A fuzzy logic compensator is developed to compensate the gravitational effects according to different robot configurations and task situations. Experimental results prove the validity of the proposed methods.

Surface manufacturing is widely used in industry. Automatic CAD-guided tool planning has many applications in surface manufacturing, such as spray painting, spray forming, rapid tooling, cleaning and polishing. According to the material quantity requirements, these tasks can be categorized into two groups: the material uniformity problem and coverage problem. A general framework is developed to automatically generate trajectories of a free-form surface for these tasks. A given task is first transformed into one of the groups. Based on the CAD model of a free-form surface, constraints and tool model, a trajectory for a free-form surface is generated. Velocity optimization is discussed to optimize the material quantity. Simulations are performed to verify the developed framework. This framework can also be extended to other applications.

Automatic trajectory generation for spray painting is highly desirable for today's automotive manufacturing. Generating a trajectory for a surface with only one patch is widely studied to satisfy the paint thickness constraints. However, a complex surface has to be divided into several patches to satisfy the paint thickness and paint gun orientation constraints. Trajectory generation for a surface with multiple patches has not been addressed yet. In this paper, optimization processes are developed to optimize the paint thickness on a surface which consists of several patches. Optimization results are presented. Simulations are performed to verify the optimized parameters. Verification results is consistent with the optimization results.

The coordination of multi-robots is required in many scenarios for efficiency and task completion. Combined with teleoperation capabilities, coordinating robots provide a powerful tool. Add to this the Internet and now it is possible for multi-experts at multi-remote sites to control multi-robots in a coordinated fashion. For this to be feasible there are several hurdles to be crossed including Internet type delays, uncertainties in the environment and uncertainties in the object manipulated. In addition, there is a need to measure and control the quality of tele-coordination. This paper proposes a measure for the quality of tele-coordination, referred to as the coordination index, and details the design procedure that ensures a system performs at a required index. The theory developed was tested by bilaterally tele-coordinating two mobile manipulators via the Internet. The experimental results confirmed the theory presented.

Heat generation is one of the major harming factors in the reaming process of the internal bone fixation surgery. In order to minimize the temperature rise of the bone during surgery, a reaming system with temperature sensation is developed to allow the real time tracking of the heat generation caused by rotational sliding friction. We have conducted experiments to calibrate and demonstrate the stability and sensitivity of the wireless transmitted temperature data. We have also analyzed the possible factors causing the temperature change of the bone cavity during surgery theoretically and performed experiments to validate these factors. An experiment was also conducted to demonstrate the sufficient penetrating power of the wireless temperature signals from a sensor inside a bone replicate.

The use of atomic force microscope (AFM) as a nanomanipulator has been evolving for various kinds of nanomanipulation tasks. Due to the bow effect of the piezo scanner of the AFM, the AFM space is different from the Cartesian space. In this paper, different 3-D nanomanipulation tasks using AFM such as nanolithography, pushing and cutting are discussed. 3-D path planning are performed directly in the AFM space and the 3-D paths are generated based on the 3-D topography information of the surface represented in the AFM space. This approach can avoid the mappings between the AFM space and Cartesian space in planning. By following the generated motion paths, the tip can either follow the topography of the surface or move across the surface by avoiding collision with bumps. Nanomanipulation using this method can be considered as the "true" 3-D operations since the cantilever tip can be controlled to follow any desired 3-D trajectory within the range of AFM space. The experimental study shows the effectiveness of the planning and control scheme.

With the rapid development of information technology, Internet has evolved from a simple data-sharing media to an amazing information world where people can enjoy different kinds of services. Recently, the use of the Internet has been expanded to the field of automation, i.e. using the Internet as a tool to control equipment located at remote sites. This paper presents a cooperative robot system consisting of a robot hand and a mobile robot carrying a stereo vision, which can be tele-operated by operators at different sites via the Internet. To overcome the instability and reliability problem caused by the random time delay of the Internet communication, we adopt an event as the reference for controller design of the system. A vision-based method is adopted to maintain interactions among the operations. Results obtained in teleoperation experiments among Hong Kong, the mainland China, and USA will be demonstrated to confirm the usefulness and effectiveness of the developed method and system.

This paper presents an analysis and design method for human/robot integrated systems, especially fro the coordination of human and robot formations based on a group of distributed mobile manipulators. The key for the human/robot integrated system is to create a common motion reference that can be understood by both human and the robots in the formation. First, the perceptive reference frame is introduced and the characteristics of perceptive frame are compared with time based reference frame. Next, the stability of the system based on perceptive reference frame is investigated. The applications of perceptive reference frame to multi-agents coordination in a formation, human/mobile manipulators coordination are then discussed. Based on the perceptive reference frame, the human can be naturally integrated into the robot formation. Human intelligence can therefore be integrated with the mobility and dexterous manipulation capability of the mobile manipulators to undertake complex tasks. The robot formation is therefore able to reconfigure and thus cope with unexpected events. Experiments have been used to verify the theoretical results.

Automatic CAD-guided optimal tool planning has many application in surface manufacturing, such as spray painting, spray forming, rapid tooling, cleaning and polishing. A general framework for CAD-guided optimal tool planning is developed to automatically generate optimal trajectories for these tasks. A material distribution model is developed. A multi-objective constraint optimization problem is formulated. An optimal tool trajectory with an optimal time and material quantity deviation is generated. Simulations are performed to verify the generated trajectory. The results show that the developed optimal tool planning algorithm can be applied to generate optimal tool trajectories. This framework can also be extended to other applications.

In real-time Internet based teleoperation, different robotic tasks have different dexterity requirements during task progress and thus different network resources are required for safe and reliable task accomplishment. In order to control remote manipulators efficiently and smoothly via the Internet, dynamic bandwidth allocation is crucial to successful accomplishment of robotic tasks controlled by remote operator. In this paper, a novel bandwidth allocation mechanism is developed based on the online measured task dexterity index of current dexterous tasks so that operators can control remote manipulators efficiently and smoothly even under poor network quality. Experiments have been conducted to demonstrate the effectiveness of the presented resource (bandwidth) allocation algorithm in Internet based teleoperation system.

Previous studies on nanomanipulation using Atomic Force Microscope (AFM) go through the scan-design-manipulation-scan cycle, in which an operator does not have any real-time visual feedback during manipulation. In this paper, a simple model of tip-substrate-object interactive forces has been presented for the qualitative and quantitative analysis. Based on this model, the real-time tip-substrate-object interactive forces are used to update the AFM images in order to provide the operator with real-time visual feedback. The real-time visual display combining with real-time force feedback provides an augmented reality environment, in which the operator not only can feel the real-time 3-D interaction forces but also observe the real-time changes of the nano-environment.

This paper aims at developing the force-guided microassembly technology with in-situ PVDF piezoelectric force sensing and control. By using the designed force sensors with the effective signal processing techniques, the micro contact force/impact signal and its derivative can be extracted and processed. Furthermore, based on a new sensor-referenced control scheme, micro mirrors can be reliably assembled by regulating the micro contact force. Experimental results verify the performance of the developed micro force sensing and control system. Ultimately the technology will provide a critical and major step towards the development of automated manufacturing processes for batch assembly of micro devices.

Additive manufacturing processes such as spray coating, spray painting and rapid tooling important steps in many products' life cycle. Robotic manipulators are widely adopted to carry out these processes. If done by human operators, the tool planning for these applications is usually time-consuming and the generated tool plans are prone to inaccuracy and errors. This research develops a fully-automated, CAD-guided tool planning system which eliminates the human involvement. Meanwhile, this system can generate optimized tool plans in the sense of robot motion performance. The critical part of this tool planning system is the partitioning of part surfaces into multiple easy-to-handle patches. In this paper, a decomposition-based approach is developed, which models the surface partitioning problem in geometric domain as an integer programming problem in algebraic domain. Experimental tests and evaluation carried out on automotive parts validate this new approach.

This paper describes a gait generation and control approach of a bipedal climbing robot with under-actuated mechanism. The special mechanical structure enables the robot to perform exploration tasks using "crawling", "pivoting" or "climbing" gaits. Multiple sensors are synthesized to generate successful gaits using a finite state machine. Experiments are conducted which demonstrate the effectiveness of proposed approach.

This paper introduces new planning and control methods for supermedia-enhanced real-time telerobotic operations via the Internet. Supermedia is the collection of video, audio, haptic information, temperature, and other sensory feedback. However, when the communication medium used, such as the Internet, introduces random communication time delay, several challenges and difficulties arise. Most importantly, random communication delay causes instability, loss of transparency, and desynchronization in real-time closed-loop telerobotic systems. Due to the complexity and diversity of such systems, the first challenge is to develop a general and efficient modeling and analysis tool. This paper proposes the use of Petri net modeling to capture the concurrency and complexity of Internet-based teleoperation. Combined with the event-based planning and control method, it also provides an efficient analysis and design tool to study the stability, transparency, and synchronization of such systems. In addition, the concepts of event transparency and event synchronization are introduced and analyzed. This modeling and control method has been applied to the design of several supermedia-enhanced Internet-based telerobotic systems, including the bilateral control of mobile robots and mobile manipulators. These systems have been experimentally implemented in three sites test bed consisting of robotic laboratories in the USA, Hong Kong, and Japan. The experimental results have verified the theoretical development and further demonstrated the stability, event transparency, and event synchronization of the systems.

In order to execute complicated tasks involving contact with the environment, the robot manipulator must have force control capabilities. Many strategies for performing constrained motion control have been developed. The major drawback of these strategies is that they can only provide for the control of the interaction forces along fixed task space directions. However, many applications necessitate interaction force control along arbitrary and changing directions. In this paper, a new hybrid force/position controller, implemented in a moving reference frame, which can control the interaction force along arbitrary directions is proposed. Using the proposed controller the robot manipulator can interact with workpieces whose shape is not pre-specified. Thus tasks like milling, deburring, polishing and surface tracking, that involve constrained motion control of the robot manipulator can be performed on a wide variety of workpieces without explicit knowledge of the shape of the workpiece. The design of this new hybrid force/position controller, implemented in the moving hand coordinate frame, is presented and the controller is shown to be stable. Simulation studies for the task of tracking unknown surfaces are also presented.

Automatic trajectory generation for spray painting is highly desirable for today's automotive manufacturing. Generating paint gun trajectories for free-form surfaces to satisfy paint thickness requirements is still highly challenging due to the complex geometry of free-from surfaces. In this paper, a CAD-guided paint gun trajectory generation system for free-form surfaces has been developed. A paint thickness verification method is also provided to verify the generated trajectories. The results of experiments and simulations show that the trajectory generation system achieves satisfactory performance. This trajectory generation system can also be applied to generate trajectories for many other CAD-guided robot trajectory planning applications.

This paper concentrates on transparency and synchronization of supermedia in Internet based teleoperation. Supermedia is used to describe the collection of all the feedback streams in teleoperations, such as haptic, video, audio, temperature and others. Transparency and synchronization are introduced and analyzed from the event-based control perspective. The concepts of event-transparency and event-synchronization for event-based control telerobotic systems are developed and their implications are studied. To illustrate those concepts and their benefits, the teleoperation of a mobile manipulator via the Internet, where haptic, video and temperature information is fed back to the operator, is discussed. Experimental results will verify the event-transparency and event-synchronization of this event-based telerobotic system.

Despite the enormous research efforts in creating new applications with MEMS, the research efforts at the backend such as packaging and assembly are relatively limited. We present our ongoing development of a polyvinylidene fluoride (PVDF) multi-direction micro-force sensing system that can be potentially used for force-reflective manipulation of micro-mechanical devices or micro-organisms over remote distances. Thus far, we have successfully demonstrated 1D and 2D sensing systems that are able to sense force information when a micro-manipulation probe-tip is used to lift a micro mass supported by 2 /spl mu/m/spl times/30 /spl mu/m/spl times/200 /spl mu/m polysilicon beams. Hence, we have shown that force detection in the 50 /spl mu/N range is possible with PVDF sensors integrated with commercial micro-manipulation probe-tips.

This paper presents a QoS based efficiency improving scheme for Internet-based teleoperation. One of the widely used QoS parameters for showing network status is network delay. With the help of event-based control technique, the stability of the teleoperation systems is guaranteed under random network delay. This paper investigates how to improve the efficiency of teleoperated tasks based on the network quality by introducing a Command Negotiator and a robot controller gain adjustment scheme using the measured QoS parameters into the proposed QoS based teleoperation systems. The presented methods improve the efficiency and system responses of the teleoperation systems when the network quality is poor. Moreover, a teleoperation experiment is presented to demonstrate the effectiveness of the proposed controller gains adjustment scheme.

A model identification method for unknown environments has been developed. By the interactions between a mobile manipulator and the unknown object, a nonholonomic cart, sensory information has been collected to estimate the model parameters of the cart, which are used to control the cart. Since the raw data are contaminated by noise they can not be modeled statistically, a wavelet based least square method is proposed to estimate these parameters for the cart. The raw signal has been decomposed into a certain bandwidth to generate a series of new signals, which are used to estimate the parameters. The new signal, which has the minimal estimation residual in least square sense is adopted as the best estimation. The error convergence of the estimation approach is given. The experimental results indicate that the estimation accuracy can be significantly improved by the use of the proposed method.

This paper presents an integrated task planning and control approach for manipulating a nonholonomic cart by mobile manipulators. The task considered in this study is to manipulate a nonholonomic cart to perform certain tasks such as pushing the cart along a straight line, making a turn at a corner, or tracking a sine wave. The cart manipulation task fully integrates the motion and force planning of the cart, and the planning and control of the mobile manipulators. The motion of the mobile manipulator and cart is coordinated by a common motion reference. The cart manipulating control has been implemented based on the decoupled mobile manipulation model and the force planning of the cart. The proposed integrated task planning and control approach enables the mobile manipulator complete complicated tasks by regulating its output force. The approach has been tested on a mobile manipulator consisting of a Nomadic XR4000 and a Puma 560 robot arm. The experimental results demonstrate the efficacy of this approach in the mobile manipulation of a nonholonomic cart.

This paper presents theoretical and experimental results on supermedia enhanced human/machine cooperative control of robot formations. Supermedia is the collection of all feedback streams rendered for the operator; such as video, haptic, temperature and others. The core idea is to utilize machine intelligence for the control of a robot formation. However, once this intelligence is insufficient to cope with unexpected events, human intervention is an option. To accomplish this, without the need for replanning, perceptive planning and control theory is utilized. This would allow the cooperation of human and machine for the control of the robot formation. To increase the flexibility and efficiency of such systems, commands of different levels of complexity can be issued. This gives rise to a hierarchical command structure, which can be described by a hierarchical perceptive frame that is modeled using automata and languages.

The evolution of human-to-computer input devices lags far behind the evolution of processing power. In this paper, we present work on merging MEMS force sensors and existing wireless technologies to develop a novel multifunctional interface input system, the Micro Input Devices System (MIDS), which could potentially replace the mouse, the pen, and the keyboard as input devices to the computer. Moreover, initial experimental results indicate that further exploration of this technology could eventually produce a new control-input device for grasping robotic manipulators. We have thus far developed a prototype MIDS that consists of two MIDS rings, each packaged with commercial MEMS acceleration sensors to sense multi-axes motion, and a MIDS wrist watch that communicates with the rings and transmits data wirelessly to interface with a CPU. The system has been demonstrated to perform click and drawing motions successfully. A self-calibration method was also developed to resolve ambiguities in sensed motion for the MEMS sensors.

Calibration is indispensable for automatic micromanipulation. In this paper, using the height difference between different focus planes detected from the microscope and the coordinates of the probe measured from encoders, systematic methods to calibrate the relative position and orientation between the tools and the microscope are developed. Using the least square error (LSE) method to process the data measured, the influence of various positioning errors can be reduced. Experiments are performed to calibrate a micromanipulation system using the proposed methods. The calibration results are validated by cross verification through further experiments.

For the visual servoing tasks, it is required to calibrate accurately the homogeneous transformation matrix between the robot base frame and vision frame besides the intrinsic parameters of the vision system. In this paper, based on an important observation, that is, the unknown transformation matrix between the robot base frame and vision frame can be separated from the visual Jacobian matrix, and by virtue of decomposition of rotation matrix, we design a novel adaptive position-based visual servo controller for manipulators when the transformation matrix is not calibrated. It is proved with a full dynamics of the system by the Lyapunov approach that the motion of the manipulator approaches asymptotically to the desired trajectory. Simulations and experimental results both demonstrate the performance of this new controller.

This paper addresses the robot path planning problem for automotive part inspection using structured light method. This problem is rendered as a traveling salesman problem (TSP). A new approach is developed to solve the TSP into its sub-optimality quickly. Instead of solving a large size TSP, this approach utilizes the clustered nature of the viewpoints and converts the TSP into a clustered traveling salesman problem (CTSP). A new algorithm, which favors the inter-group paths, is proposed to solve the CTSP. Experimental results on various automotive parts validate the new algorithm.

This paper describes the motion planning of a bipedal crawling robot with an under-actuated mechanism and multiple locomotion modes. A hybrid configuration space is proposed to incorporate the continuous configuration space with discrete motion status space imposed by the kinematic constraints. Under the hybrid configuration space framework, a motion planning method is developed which consists of a global planner and a local planner to generate a collision-free path and a feasible motion sequence to travel along the path. A cost function is defined based on the motion status information to guide the search for an optimal path. Simulation and experimental results have verified the theoretical development.

Robotic operations carried out via the Internet face several challenges and difficulties. These range from human-computer interfacing and human-robot interaction to overcoming random time delay and task synchronization. These limitations are intensified when multi-operators at multisites are collaboratively teleoperating multirobots to achieve a certain task. In this paper, a new modeling and control method for Internet-based cooperative teleoperation is developed. Combining Petri net model and event-based planning and control theory, the new method provides an efficient way to model the concurrence and complexity of the Internet-based cooperative teleoperation. It also provides an efficient analysis tool to study the stability, transparency and synchronization of the system. Furthermore, the new modeling and control method enables us to design an Internet-based cooperative telerobotic system that is reliable, safe and intelligent. This new method has been experimentally implemented in a three site test bed consisting of robotic laboratories in the USA, Hong Kong and Japan. The experimental results have verified the theoretical development and further demonstrated the advantages of the new modeling and control method.

The development of a medical surgical tool, packaged with micro sensors for transmission of supermedia information over the Internet is described in this paper. We define supermedia as a set of communication media, which encompasses acoustic, force, visual, audio, temperature, tactile, and chemical (e.g., taste and smell) information, and which can be physically experienced by a communicator. In this project, we specifically develop supermedia capability for a bone-reaming system that is used for intramedullary fixation procedure of fractured bone treatments. Thus far, transmission of temperature, force, and pressure information from MEMS sensors over the Internet has been demonstrated. Force-reflective control over the Internet using force information from a micro tip has also been shown. We have also packaged a MEMS pressure sensor inside a bone reaming guide-rod and proved that pressure variations inside a long cavity that simulated the environment inside a bone can be monitored, even with the guide-rod rotating up to 600 rpm. This paper describes our experimental methods and gives the experimental results for these accomplishments.

A CAD-guided camera planning system for dimensional inspection of automotive parts has been developed. This system utilizes the CAD information of inspected parts and the camera model to plan camera viewpoints that satisfy certain task constraints. To overcome some limitation inherited in the existing method, a new CAD data representation is adopted to combine the advantages of parametric surface and triangulation. A graph-based surface merging algorithm is proposed to reduce the number of flat patches, and eventually, the number of viewpoints. Tests on different real part models show that the surface merging algorithm is effective. This algorithm can also benefit many other CAD-guided robot path planning applications.

In this paper, a unified dynamic model for integrated mobile platform and on-board manipulator is developed. The mobile manipulator is considered as a redundant robot in the model. It provides a efficient and convenient framework to design a mobile manipulator controller as well as its action plans. Combing the event-based planning and control method with the nonlinear feedback technique, a task level action controller is designed. An online kinematic redundancy resolution scheme has also been developed. The system stability has been proven in the normal operation as well as in the case of appearance of unexpected obstacles. Furthermore, a robotic task involving both position and output force control of mobile manipulator can be easily planned. The proposed unified model approach has been implemented and tested on a mobile manipulator consisting of a Nomadic XR4000 and a Puma 560 robot arm. A cart pushing task is used to demonstrate the efficiency and effectiveness of the proposed approach.

Now a number of advanced control systems take advantage of networks such as Ethernet, Fieldbus and Internet as their communication channel. Transmission time delay of these networks may lead unstable problems to system. In this paper, the prediction of network delay using neural network is presented first. Then the analysis of the control oriented quality of service (CO-QoS) specially with the Internet-based teleoperation is considered and fuzzy inference is used to construct a quantification of CO-QoS. At last, the simulation results show that this method provides a practical way for the judgment of the network's characteristics.

Sensor planning is critical in many inspection systems. In typical part dimensional inspection systems, eye-in-hand robots are usually adopted. One practical problem that needs to be addressed is that the viewpoints, or correspondingly, the robot hand-tip positions and orientations, should be within the robot's reachability space. Furthermore, it is desirable that the robot can reach these viewpoints "comfortably". In this sense the robot kinematics constraint should be considered in sensor planning. In the paper, an approach is developed to integrate the kinematics constraint into sensor planning. This approach models the problem as an integer programming problem. With this approach, the computation burden of the robot placement problem can be avoided. Experiment and simulation results demonstrate the effectiveness of our approach.

The paper proposes, an integrated sensing and control framework for autonomous mobile manipulators. First, the mobile manipulator is considered as a redundant robot, while a unified dynamic model for an integrated mobile platform and on-board manipulator are developed. Combining the event-based planning and control method with a nonlinear feedback technique, a task level action controller is designed and an online kinematic redundancy resolution scheme is developed. Secondly, a force/torque sensor and a laser range sensor are used for the mobile manipulator to interact with objects and the environment. A nonholonomic cart pushing task has demonstrated the advantages of the integrated sensing and control approach of a mobile manipulator. The sensing and control approaches are tested on a mobile manipulator consisting of a Nomadic XR4000 and a Puma 560 robot arm.

This paper presents the modeling and control of our second generation prototype miniature crawler robot which was targeted to applications in constrained environments. The mechanical design and the drive mechanism of the robot are first discussed A kinematic model is then derived and the motion planning is analyzed. A description of the Texas Instrument DSP-based embedded controller is presented. Finally, experimental results are presented for evaluation of the robot performance.

The ability to manipulate and control biological cells with reflective-force information is a key technology necessary for many new applications in bio-MEMS, but is currently lacking in all cellular manipulators. We report our preliminary experimental work in using an ionic conducting polymer film (ICPF) to develop a biological cellular robotic gripper with force sensing capability. ICPF actuators are able to give large deflection with small input voltage (/spl sim/5V) in aqueous environments, and also able to give relatively large output voltage due to deflection by mechanical forces. Thus, ICPF actuators are investigated as possible cellular force-reflection controlled manipulators in our work. Individual multi-finger grippers with dimensions of 200/spl mu/m /spl times/ 200/spl mu/m /spl times/ 3000/spl mu/m for each finger were realized. We report on the design, fabrication procedures, and operating performance of our ICPF actuators. Further development in the reduction of size of these actuators will enable effective force-feedback control of underwater micro objects and lead to new frontiers in cellular manipulation.

The use of the Internet is no longer limited to the transmission of data. In the past few years many successful attempts have been made to use the Internet as a command transmission media; through which control can be sent to remote systems and feedback can be obtained. But with this media come several limitations: delay, lost packets and disconnection. All of these limitations may cause instability in the system especially if the system loop is closed. All the previous work addressing these problems assumed several conditions; for example, time delay is constant or has an upper bound, control is not in real-time. A new real-time control approach is presented that deals with these limitations without any assumptions made regarding delay. The approach is based on event-based control, which was implemented on a mobile robot over the Internet. The commands sent to the robot are velocity and the feedback is force based on the environment. It will be shown that this approach results in a stable system. In addition, a new force feedback generation method is used.

A general method of controller design is developed in this paper for the purpose of formation keeping and reconfiguration of multiple autonomous vehicles. Controllers are designed to keep multiple vehicles in a required formation, and to coordinate the vehicles in the presence of environmental changes. The method is applicable to a large variety of autonomous agents. Examples and simulations are given for ground vehicles, robotic arms and satellites.

Structured light is one of the well-known methods in part dimensional inspection that have been successfully employed in various applications in the past decades. In this method, the positioning of the camera is very critical, which affects the accuracy and efficiency of the whole inspection system. Here we develop a CAD-guided camera positioning system to aid the 3-D part inspection. The geometric information in the CAD model of the inspected part and the camera model are used to plan camera configurations that satisfy certain task constraints. The overall system we propose can be applied to the 3-D inspection of parts with free-form surfaces, such as automobile door panels. Experiments on different parts show satisfying results.

This paper presents a hybrid system approach in the design of a singularityless task level controller. To achieve a singularityless motion control in the neighborhood of singularity, the hybrid system approach is used to integrate the task level controller and joint level controller. First, a hybrid system model is developed for the singularityless task level controller. A max-plus dynamic model is used to integrate the discrete switching control and continuous motion control in the controller. Based on this model, a smooth trajectory and control command for the hybrid system can be obtained. The Lyapunov theory was used to prove the stability of the singularityless controller. The new singularityless task level controller has been experimentally implemented and tested on a PUMA 560 robot manipulator. Experimental results have been employed to verify the theoretical conclusions, thus clearly demonstrating the advantages of the new task level control method.

The e-world, also known as the Internet, has added a new dimension to many of the traditional concepts in industrial applications and everyday life. The use of robots has dramatically expanded the potential of e-services. Individuals with particular expertise can perform highly accurate and fairly complicated tasks remotely via the Internet. This increase in the human reachability is faced by several obstacles. Reliable and efficient robot facilitated services via the Internet face several challenges. These range from human-computer interfacing and overcoming random time delay to task synchronization and human-robot interaction. These limitations intensify when many operators in many sites are involved. This paper provides new theoretical and experimental results on these challenges. Specifically, multisite cooperative control of an Internet based mobile manipulator is presented. The two main characteristics of this system are Internet based real-time closed loop control and coordinated operation. In addition, it is shown that despite random time delay the stability and synchronization of the system were achieved using event-based control.

In this paper, a CAD-guided robot path generator is developed for the spray painting of compound surfaces commonly seen in automotive manufacturing. Instead of widely used parametric representation of surfaces, a planar facet scheme is used to approximate the painting surfaces. As a result, a new combinatorial gun path planning algorithm is proposed. In this algorithm, big patches are formed by stitching small surfaces together. Therefore, the path planning is solved based on the global characteristics of tire part, and the resulting spray gun paths are well-behaved in the sense of time, coverage, and wastage. The proposed algorithm has been implemented and tested using ROBCAD/sup TM//Paint.

The dynamic workspace and the motion plans for a micro biped walking robot are studied in the paper. The dynamic model and controller of an under-actuated walking robot system are developed. Two optimal navigation algorithms for the motion of the robot in the same plane without obstacles are developed, and optimality is proved. The motion of the robot is divided into normal motion status and transition status. The complete scheme to perform the two status and this real time implementation are presented. The dynamic simulation results and 2D-rendering of the robot navigation is given.

This paper presents a novel approach for solving a challenging problem in the intelligent control of robotic manufacturing systems, i.e., the integration of low-level system sensing and control with high-level system behavior and perception. First, a newly developed event-based planning and control method will be introduced. It will then be extended to a robotic manufacturing system via a hybrid system approach. The tasks of a robotic manufacturing system usually consist of multiple segments of robotic actions, which involve both continuous and discrete types of actions. The max-plus algebra model has been proposed to model such a system. Combined with the event-based planning and control methods, both discrete and continuous actions can be planned and controlled based on the max-plus algebra model. More important, the interactions between discrete and continuous actions can be formulated analytically. A typical parts-sorting task in a robotic manufacturing system is used to illustrate the proposed approach. The experimental results clearly demonstrate the advantages of this method.

A design method for non-time based tracking controller of mobile robots is presented. The new design method converts a controller designed by traditional time-based approaches to a non-time based controller using any given action reference. The stability condition of the non-time based tracking controller is developed and theoretically proved. The analysis and design methods are exemplified by a mobile robot tracking control problem. The controller has been implemented and tested in a Nomadic XR4000 mobile robot. The experimental results demonstrate the advantages of proposed method.

A modular sensor, which may jointly include photo and acceleration sensors with feedback mechanisms to improve performance, is designed to be integrated on a single CMOS chip. We describe some components of the modular sensor and the feedback control mechanism, which is synthesized from the available and permissible electronic building blocks. We then discuss the performance of the controlled sensors necessary to demonstrate reliability and repeatability of the modular sensors. These modular sensors are intended for a hierarchical and distributed control strategy by placing multiple modular sensors in key locations or throughout the surface of a robot in order to facilitate and reduce the computational effort in a global control strategy for trajectory planning, obstacle avoidance, and navigation.

This paper explores a new method for action synchronization and control of telerobotic systems. The significance of the method is that it can effectively deal with the random time-delay existing in the communication channel, such as the Internet. In addition, the result is independent of human operator. First, a novel non-time referenced action control scheme is introduced. Instead of using time as an action reference, a new sensor-based action reference is developed. As a result, the communication delay will have little impact on the operations, in particular the stability of the system. Furthermore, the stability of the system is not dependent on the model of the human operator in the system. All these will lead to the development of the theoretical foundation on the stable control of robotic systems involving human operators. In this paper, the stability condition of robotic system involving both random communication time delay and unknown dynamics of the operator has been developed. It provides the output stability as well as the state stability.

Develops a general method of controller design for the formation keeping of multiple autonomous vehicles. The purpose of the algorithm is to find controllers to keep multiple vehicles in a required formation, and to coordinate the vehicles in the presence of environmental change. Some useful coordination strategy such as movement with a leader, simultaneous movement, and series connections of formations are demonstrated by simulations.

A team of interdisciplinary researchers are involved in the development of a miniature biped robot that can traverse flat inclined surfaces such as walls and ceilings (R.L. Tummala et al., 1999). Unlike conventional biped robots, only limited degrees-of-freedom are available in the design. Thus the kinematics and workspace analyses become important for motion planning and control. The paper briefly discusses the mechanical structure and drive mechanism of the miniature robot. The kinematic model of the robot is then developed, and its workspace is analyzed in detail. Simulation results of the robot motion are also presented.

This paper presents a novel approach for solving the challenging problem in intelligent control of manufacturing systems. The proposed max-plus algebra model combined with event-based planning and control provides a mechanism to efficiently integrate task scheduling, sensing, planning and real-time execution so that the task scheduling which usually deals with discrete types of events, as well as action planning which usually deals with continuous events, can be treated systematically in a unified analytical model. The unique feature of this approach is that interactions between discrete and continuous events can be considered in a unified framework. This feature allows the manufacturing system to intelligently cope with unexpected events and uncertainties so that the efficiency and reliability of the task schedule and action plan can increase significantly. A robotic manufacturing system is used to illustrate the proposed approach. The experimental results demonstrate the advantages of the proposed approach.

Explores a new dimension in the area of human/robot cooperation: heterogeneous human/robot cooperations. The heterogeneity of human and robot functions can be characterized by the fact that humans are intelligent while robots are fast, powerful and accurate. The important issue, in light of human/robot heterogeneity, is how to plan and control a robotic operation such that the human and the robot can cooperate in a complementary manner. In the paper, the specific problems to be discussed include developing a perceptive action reference frame to integrate human/robot heterogeneous functions; and modeling, analyzing and designing an integrated human/robot system with respect to the perceptive reference frame. These lead to the development of theoretical results on stability of human/robot cooperations, and special methods for planning and control of human/robot integrated systems. The implementations and experimental results presented demonstrate the advantages of the proposed methods.

This paper discusses the planning and control problems of a robot manipulator for a class of constrained motions. The task under consideration is to control a robot such that a tool grasped by the end-effector of the robot follows a path on an unknown surface with the aid of a single camera vision system. To accomplish the task, we propose a new planning and control strategy based on multisensor fusion. Three different sensors-joint encoders, a wrist force-torque sensor and a vision system with a single camera fixed above a workspace-are employed. We decouple control variables into two sub-spaces: one is for force control and the other for control of constrained motion. We also develop a new scheme by means of sensor fusion to handle the uncertainties in an uncalibrated workspace. The contact surface is assumed to be unknown and the precise position and orientation of the camera with respect to the robot is also unknown.

The analytical and robust integration of low-level system sensing and simple control with high-level system behavior and perception is critical technology in the intelligent control of modern manufacturing systems. The paper presents a novel approach by using a max-plus algebra model and a newly designed event-based planning and control scheme to provide an advanced mechanism to efficiently integrate manufacturing systems, so that task scheduling, which usually deals with discrete events, as well as action planning/control, which usually deals with continuous dynamics, can be treated in a unified analytical model. More importantly, the interactions between discrete event systems and continuous dynamics are achieved so that manufacturing systems are allowed to intelligently cope with unexpected system faults and uncertainties during well-scheduled tasks. A robotic manufacturing system is used to illustrate the proposed approach. The experimental results clearly demonstrate the advantages of the proposed approach.

The real-time planning and control problems of a robot manipulator in an unstructured workspace are considered. Our goal is to control a robot to follow an unknown path. Difficulties arise when the robot is required to work in an unstructured workspace. Based on multisensor fusion, a novel strategy for integrating real-time planning and control is proposed to deal with the uncertainties in the environment. In the proposed scheme, a new hybrid position/force control strategy is utilized to maintain the contact between the robot and the unknown surface, while a new planner is designed to complete the path-following task without a priori knowledge of the path. In the paper, to achieve intelligent robot manipulation in an unstructured workspace, multi-sensor fusion is employed both for force-torque and visual sensors with complementary observed data compared to the traditional fusion schemes with redundant data.

A new design method for nontime based tracking controller is presented. The key to the nontime based control method is the introduction of a suitable action or motion reference variable other than time, which is directly related to the desired and measurable system output. It enables the construction of control systems with integrated planning capability, in which planning becomes real-time closed-loop process. The new design method converts a controller designed by traditional time-based approach to a nontime based controller using action reference. It significantly simplifies the design procedure. The design method is exemplified by a unmanned vehicle tracking control problem. The design procedure and simulation results demonstrate the advantages of proposed method.

The essence of our scheme is to extend our earlier (Ghosh et al., 1996) proposed multi-sensor based observation scheme to a situation where the location of the mobile platform is unknown. A new self-calibrated manipulation scheme is proposed to determine the position and orientation of the part with respect to the coordinate system attached to the base of the robot manipulator. The novelties of the proposed approach can be summarized as (i) multi-sensor fusion scheme based on complementary data for the purpose of part localization, (ii) part tracking and grasping control based on parallel tracking, and (iii) self-calibration of the location of the mobile platform using visual data and feature points on the end-effector. The principle advantages of the proposed scheme are described as follows. (i) It renders possible reconfiguring a manufacturing workcell without recalibrating the robot based coordinate frame. This significantly reduces the setup time of the workcell. (ii) It significantly reduces the requirement on the image processing speed. Experimental study of the proposed scheme has been carried out and the results are reported in this paper.

The main problem that we address in this paper is how a robot manipulator is able to track and grasp a part placed arbitrarily on a moving disc conveyor aided by a single CCD camera and fusing information from encoders placed on the conveyor and also from encoders on the robot manipulator. The important assumption that distinguishes our work from what has been previously reported in the literature is that the position and orientation of the camera and the base frame of the robot is a priori assumed to be unknown and is 'visually calibrated' during the operation of the manipulator. Moreover the part placed on the conveyor is assumed to be nonplanar, i.e. the feature points observed on the part is assumed to be located arbitrarily in R/sup 3/. The novelties of the proposed approach in this paper includes a (i) multisensor fusion scheme based on complementary data for the purpose of part localization, and (ii) self-calibration between the turntable and the robot manipulator using visual data and feature points on the end-effector. The principle advantages of the proposed scheme are the following. (i) It renders possible to reconfigure a manufacturing workcell without recalibrating the relation between the turntable and the robot. This significantly shortens the setup time of the workcell. (ii) It greatly weakens the requirement on the image processing speed.

Presents a paradigm for task scheduling and action planning/control of a robotic system. Based on the proposed max-plus algebra model, a robotic task involving both discrete and continuous actions can be scheduled, planned and controlled in a perceptive reference frame. Therefore, task scheduling, which usually deals with discrete type of events, as well as action planning, which usually deals with continuous events, can be treated systematically in a unified framework. More importantly, the unique feature of this approach is that interactions between discrete and continuous events can be considered in the same framework. As a result the efficiency, robustness and reliability of the task schedule and action plan can increase significantly. A typical assembly task in a dual-robot manufacturing work-cell is used to illustrate the proposed approach. The experimental results clearly demonstrate the advantages of the proposed approach.

This paper presents a modular control architecture for multi-arm telerobotic systems. It provides a platform for the implementation of various robot control and planning schemes, as well as a framework for quickly transferring academic research to real-world applications. The advantage is exemplified by the transfer of an intelligent planning and control scheme for telerobotic systems to Sandia National Laboratories. The intelligent planning and control scheme accepts the human supervisory control and autonomous control into one seamless unit. It combines the strength of both human and autonomous control to achieve a safe, robust, and reliable operation. More importantly, the results presented in this paper demonstrate the importance of developing an efficient mechanism for transferring research and technology developed in the academic environment to the national laboratories and eventually into practical applications. It could be a crucial step towards the cross-development of robotic technology in our university, industrial, and national laboratories. The modular controller presented in this paper surely is an important step in this direction.

In this paper we introduce a new approach to visually manipulate, with the aid of a robot manipulator, a part placed randomly on a rotating turntable. The highlight of our approach is that the camera and the robot end effector are both assumed to be uncalibrated. We only assume that the height of the robot end effector is known. Our approach utilizes virtual rotation of the camera via image processing not previously introduced in the literature. Finally, the tracking scheme is implemented by planning the error and gradually forcing it to zero while maintaining the torque controls within acceptable limits. This way we demonstrate a new visually guided analytical tracking scheme.

In this paper, a long outstanding problem, robust impact control and force regulation with guaranteed stability is solved by adding a novel positive acceleration, feedback to the feedback loop in the direction of impact and force control for robot manipulators working in an unknown environment. A quick and stable transition can be accomplished with a nonzero impact velocity, and the transient force response can be controlled with minimum impact forces and minimum bouncing. In addition, a switching control strategy is designed to guarantee the stability of the impact control. Unexpected bouncing can be eliminated after finite switches. The output force and motion can be regulated simultaneously with high accuracy after contact is established. Rigorous stability analysis based on the Liapunov-like method is presented for the proposed control scheme. The control scheme was implemented and tested on a 6 DOF PUMA 560 robot arm. The experimental comparisons with other force control strategies were given to demonstrate the advantage of the proposed method. The result obtained in this paper can be applied to any system requiring impact control and force regulation.

A new planning and control scheme for multirobot coordination is presented. First, the event-based planning and control theory is introduced. The most important step is the design of an event-based motion reference for the multirobot system. It drives the system to achieve the best possible coordination. Hybrid position/force controllers which are able to perform a large class of tasks are designed based on the combination of general task space with the well-known nonlinear feedback linearization technique. To improve the force control performance, the dynamics of joint motors have been considered in the force control. For a given task, a task projection operator can be found for each robot with the consideration of redundancy management. It projects the feedback linearized model to the actual task space. A distributed computing architecture is proposed to implement this scheme in a parallel computation. The event-based coordination scheme was experimentally implemented and tested for the coordinated control of two 6 DOF PUMA 560 robots with very good results.

A new intelligent fusion scheme for human operator command and autonomous planner/controller in telerobotic system has been developed based on the event-based planning and control theory. It provides a unified model to integrate a human operator control command with action planning and control of autonomous operation. As a result, the telerobotic system can perform tasks which cannot be done by either human operator or autonomous planner/controller alone. This scheme lays down a foundation for planning and control of a general robotic system involving human operators, and provides a natural and efficient way to fuse the human intelligence with the machine intelligence. The scheme is implemented and tested on a PUMA 560 robotic system. The experimental results of obstacle avoidance, and hybrid force/position control with a commanded force are presented.

A new sensor-referenced control method for robot impact control and force regulation with environmental uncertainties is developed in this paper. The robot dynamic model is feedback linearized and decoupled for the free motion mode, the phase transition, mode as well as the constrained motion mode. After introducing an impact model, together with the proposed positive acceleration feedback control scheme, a stable contact can be made with a nonzero impact velocity. During the impact, large impact forces and bouncing can be avoided. Moreover, the output force can be regulated after contact is established. The scheme was implemented and tested on a 6 DOF PUMA 560 robot arm. The comparison with other control methods for robot phase transition is presented. The experimental results clearly demonstrate the advantages of the proposed method.

Discusses the problem of using visual and other sensors in the manipulation of a part by a robotic manipulator in a manufacturing workcell. The authors' emphasis is on the part localization problem involved. The authors introduce a new sensor-fusion approach which fuses sensory information from different sensors at various spatial and temporal scales. Relative spatial information obtained from processing of visual information is mapped to absolute taskspace of the robot through fusing of information from an encoder. Data obtained this way can be superimposed upon data obtained from displacement based vision algorithms at coarser time scales to improve overall reliability. Tracking plans reflecting sensor fusion are proposed. The localization of a part by spatial sensor fusion is experimentally demonstrated to be able to give required fast and accurate part localization.

A new function-based control sharing scheme for robotic systems has been developed based on the event-based planning and control theory. It provides a unified model to integrate a human operator control command with action planning and control of autonomous operation. As a result, the robotic systems can perform tasks which cannot be done by either human operator or autonomous planner/controller alone. This scheme lays down a foundation for planning and control of a general robotic system involving human operators, and provides a natural and efficient way to fuse the human intelligence with the machine intelligence. The scheme is implemented and tested on a PUMA 560 dual-arm system. The experimental results of obstacle avoidance, hybrid force/position control with a commanded force as well as dual-arm coordinated teleoperation are presented.

A multi-sensor based planning and control scheme for robotic manufacturing is presented in this paper. The proposed approach fuses sensory information from various sensors at different temporal and spatial scales in an event-based planning and control scheme. By combining the measurement of an encoder sensor, relative spatial information obtained from processing of visual measurement is mapped to the absolute task-space of the robot, delayed data obtained from a displacement based vision algorithm that represent absolute part position measurement is brought to up to date. A four-step approach to planning and control of a robotic manipulator is discussed. An event-driven tracking and control scheme that is based on multi-sensor information is given. The approach is illustrated by considering a manufacturing workcell where the manipulator is commanded to pick up a part on a disc conveyor under the guidance of computer vision.

This paper describes a distributed computing and control architecture for experiments in dual-arm sensor-based planning and control. The system comprises of two PUMA 560 robots with state-of-the-art joint-level motion controllers. These controllers have accurate joint position and velocity measurement, an accurate joint calibration system, and six-axis wrist force/torque sensors with optical fiber communication with the controller. A high-power Silicon Graphics workstation, through a shared memory interface with the joint-level controllers, provides a task-level command and control capability. A variety of control schemes can be easily implemented on this system. A servo rate of 1000 Hz, without interpolation, has been achieved for task-level servo schemes with force feedback. The experimental results presented here demonstrate the performance capabilities of the system.>

Based on our newly developed event-based planning and control theory, a new action planning and control scheme for the telerobotic system is presented. It provides a unified model to integrate a human operator control command (supervisory command) with motion planning and control for automatic operation. As a result, supervisory command and autonomous motion planning and control are naturally combined together such that the autonomous motion plan can be adjusted and modified by human operator during its execution. This scheme lays down a foundation for planning and control of a general robotic system involving human operators. The scheme is implemented and tested on a PUMA 560 dual-arm system. A Spaceball is used by the human operator as a command generator to input the supervisory command. The experimental results of both single arm teleoperation and dual-arm coordinated teleoperation are presented.>

Abstruct- This paper presents new robot aim1 planiiiiig and control schemes compatible with sensing based planning and control. These developiiieiits are a central component of robot arm coiitrol intelligence. The sclieines iise pliase space (velocity versus position) representation of robot arm motion in tlie task space, and apply time and energy optimization tecliiiiqiics to deteriiiiuc non-time based trajectories for giveii geometric path in tlie task space. A new pliasc space crror defiiiitioii aiid coiripiitatioii liavc been introtliiced aiid combined witli kiiowii nonlinear fcetlback control law, whicli linearizes and decoiiples the control in tlie task space. Tlic! system stability lias been proveii. Tlie new phase space plaiiiiiiig aiid coiitrol schemes were experiiiieiitally iiiipleiiieiited and tested on the 3-DOF positioii coiitrol of a PUMA 560 robot arm with very good results. I. INTRODIICTION