A deformable, flexible sheet that is held by multiple robots can be used for manipulating and transporting an object. For such multi-robot collaborative transporting system, forward kinematics provide object's potential positions where it might remain stationary on the sheet. Some of these forward kinematics solutions are however demonstrated unstable under object's position or robot formation perturbations. This paper presents stability criteria of the kinematics solutions of the object on the deformable sheet held by multi-robot system. We capture the sheet deformation and object position using the virtual variable cable model. A constrained quadratic problem is formulated to obtain the forward kinematics solution for a given multi-robot configuration. Linear dependence of active constraints and stability multipliers are used to assess the stability of the kinematics solutions. Two stability criteria are proposed and analyzed under object position or robot formation perturbations. These criteria are related to the properties and conditions of the stability multipliers. We present an efficient computational algorithm to determine stable kinematics, which are a small fraction of feasible solutions. Experimental results and case studies are presented to validate and demonstrate the effectiveness and efficiency of the analyses and algorithms.

With the rapid development of rehabilitation robotics, there is a pressing need for efficient and accurate gait prediction methods. However, due to the complexity and variability of individual gait characteristics and external disturbances, accurately predicting gait in real time remains a significant challenge. This paper proposes an innovative Bayesian-inference-based method for real-time gait prediction while a subject walks with a lower-limb exoskeleton. Periodic gait information is represented using von Mises basis functions, and the weight parameters serve as real-time updated state variables. The error-subspace transform Kalman filter (ESTKF) is applied for gait trajectory prediction. A fully connected neural network (FCNN) is used to estimate the walking speeds in real time based on predicted trajectories. Comparative experiments based on an open-source database prove the advantages of ESKTF compared with other Bayesian filters. Walking experiments are conducted to estimate phase and speed in real time, and to predict the joint angle, total joint torque, and lower-limb muscle surface electromyography (sEMG) values. Experimental results validate the method's prediction performance across different speeds and demonstrate its resilience to external interference.

Model-based robot control requires an accurate dynamics model and a machine learning-based method can extract robot dynamics from collected motion data by simulation and experiment. A Gaussian process (GP) has been used as one of the learning methods to obtain robot dynamics. To avoid large training datasets for learning robot dynamics, we propose an active training data selection strategy. The data sampling criteria are to minimize the probability density difference between the actual model and the GP-based estimate. Using such a criterion, the active training data strategy identifies where to sample the next data point for model training. We demonstrate the proposed active learning strategy with a 3-link robot arm in both fully actuated and underactuated modes. With the selected dataset containing 150 data points, the integrated probability density error compared with the entire dataset (over 30, 000 data points) is less than 0.3. The experimental results confirm that the GP-based control performance is greater than that under the model-based control.

Construction workers regularly perform walking locomotion on level and inclined surfaces. It is critical to detect walking activity and estimate body postures in real time for monitoring workers’ safety and health conditions. This article presents a machine learning-based framework for real-time activity detection and posture estimation during human walking on level and sloped terrains using a single wearable inertial measurement unit (IMU). The framework integrates recurrent neural networks with Gaussian process dynamical models to achieve accurate predictions of walking activity, floor slope angles, and workers’ turning angles and full-body limb joint angles estimation in real time. The proposed design offers a streamlined, cost-effective solution with significant advantages over multi-sensor systems. Extensive experiments of different walking activities on level and sloped surfaces are conducted to validate and demonstrate the design. The proposed algorithm detects gait activities with 96% accuracy, the estimated human limb joint angle errors are within 11 deg, the predicted turning angles have an error less than 16 deg and the end-to-end detection latency is within 21 ms using only one single IMU attached to the human shank.

Construction workers experience serious safety and health risks in hazardous working environments. Quiet stance and kneeling are among the most common postures performed by construction workers during their daily work. This paper first analyzes lower-limb joint influence on neural balance control strategies using the frequency behavior of the intersection point of ground reaction forces. To evaluate the impact of elevation and wearable knee exoskeletons on postural balance and welding task performance, we design virtual- and mixed-reality (VR/MR) to simulate elevated environments and welding tasks. A linear quadratic regulator-controlled triple- and double-link inverted pendulum model is used for balance strategy quantification in stance and kneeling, respectively. Extensive multi-subject experiments are conducted to evaluate the usability of wearable exoskeletons in destabilizing construction environments. The quantified balance strategies capture the significance of the knee joint during balance control of quiet stance and kneeling gaits. Results show that center of pressure sway area is reduced up to 62% in quiet stance and 39% in kneeling gait for subjects tested in high-elevation worksites with knee exoskeleton assistance. The balance and multitask evaluation confirm and provide guidance on exoskeleton design to mitigate the fall risk in construction. Note to Practitioners—Construction workers commonly perform tasks that require prolonged quiet stance or kneeling gaits on high elevations. Worker balance can be undermined by chronic knee injuries, musculoskeletal disorders, and destabilizing visual perturbations caused by occupational activities. Wearable knee exoskeletons have evolved as promising interventions to reduce knee joint stress across a variety of work gaits in construction. Emerging technologies such as virtual- and mixed-reality (VR/MR) provide an enabling tool to study underlying balance strategies to complete tasks in dynamic environments. The VR/MR- generated immersive elevated welding environment are leveraged to examine the effects of threatening visual stimuli, wearable exoskeletons, and construction tasks on worker balance and skill performance. Intersection point height frequency analysis is used to quantify the neural balance strategy during various testing scenarios. We particularly explore the often-neglected role of the knee joint to facilitate research on knee exoskeleton assisted balance in stance and kneeling gaits. The experimental results provide insight into the efficacy of knee exoskeletons in improving worker’s stability and task performance in construction. The results highlight the need for a holistic approach to exoskeleton design to ensure that developed solutions can be safely and successfully integrated into the workplace environment.

Legged robots have demonstrated high efficiency and effectiveness in unstructured and dynamic environments. However, it is still challenging for legged robots to achieve rapid and efficient locomotion on deformable, yielding substrates, such as granular terrains. We present an enhanced resistive force model for bipedal walkers on soft granular terrains by introducing effective intrusion depth correction. The enhanced force model captures fundamental kinetic results considering the robot foot shape, walking gait speed variation, and energy expense. The model is validated by extensive foot intrusion experiments with a bipedal robot. The results confirm the model accuracy on the given type of granular terrains. The model can be further integrated with the motion control of bipedal robotic walkers.

External and internal convertible (EIC) form-based motion control (i.e., EIC-based control) is one of the effective approaches for underactuated balance robots. By sequentially controller design, trajectory tracking of the actuated subsystem and balance of the unactuated subsystem can be achieved simultaneously. However, with certain conditions, there exists uncontrolled robot motion under the EIC-based control. We first identify these conditions and then propose an enhanced EIC-based control with a Gaussian process data-driven robot dynamic model. Under the new enhanced EIC-based control, the stability and performance of the closed-loop system are guaranteed. We demonstrate the GP-enhanced control experimentally using two examples of underactuated balance robots.

The human foot arch plays a significant role in bearing weight, keeping balance, and walking efficiently. In this study, we present a pneumatic arch support insole (PASI) and a foot arch stiffness-based dynamic plantar support control to reduce the metabolic cost of walking. We first obtain the foot arch quasi-stiffness estimation over the gait phase by conducting multiple subject experiments. The design and modeling of the PASI and foot-insole interactions are then discussed. A model predictive control scheme is presented to dynamically regulate the foot arch stiffness over the gait phase by using the active PASI. We validate the system experimentally and conduct multi-subject walking tests. The results show that dynamic plantar support control of the foot arch stiffness reduces the metabolic cost by 7.99% compared to regular walking. In contrast, passive support without dynamic regulation increases the metabolic cost by 6.60%. The new pneumatic insoles and dynamic support method demonstrate promising potential for everyday and medical applications.

Aggressive vehicle maneuvers such as those performed by professional racing drivers achieve high agility motion at the edge of handling limits. These aggressive maneuvers can be used to design human-inspired active safety features for next-generation “accident-free” vehicles. We present a motion planning and control design for autonomous aggressive vehicle maneuvers. The motion planner takes advantages of the sparse stable trees and the enhanced rapidly exploring random tree (RRT*) algorithms. The use of the sparsity property helps to reduce the computational cost of the RRT* method by removing non-useful nodes in each iteration and therefore to rapidly converge to the optimal solution. The proposed motion control design allows the vehicle to operate outside the stability region to accomplish a safe, agile maneuver. A safety region is computed to augment the stability region and the motion control is built on a modified nonlinear model predictive control method. We implement the proposed planner and controller and demonstrate the autonomous aggressive maneuvers on a 1/7-scale racing vehicle platform. Comparison with human expert driver and other existing methods is also presented to demonstrate the performance and robustness.Note to Practitioners—Motion planning and control of human driver-inspired aggressive vehicle maneuvers is a challenging task because of high-agility, unstable fast vehicle motions. This paper is motivated by addressing this challenge in autonomous driving technologies. Instead of restricting vehicle motions within a stability region that is taken by existing methods, we augment the conservative stability region to a safety region with guaranteed performance. To improve the computational efficiency of sampling-based motion planners, we take advantage of sparsity and also integration of a nonlinear predictive control method to compute feasible vehicle motion in searching space. The stability of the vehicle motion controller and sub-optimality of the motion planner are analyzed and guaranteed. Using a scaled vehicle testbed, we validate and compare the proposed motion planning and control design with other existing methods and human expert driver. The experimental results demonstrate the superior performance than the other methods and comparable with human expert driving skills.

We are pleased to present this Special Issue of IEEE Transactions on Automation Science and Engineering (TASE), featuring 12 extended articles selected from the technical program of the 2022 IEEE International Conference on Automation Science and Engineering (CASE 2022). CASE 2022 was primarily held in Mexico City from August 20 to 24, 2022, with a concurrent satellite site in Chengdu, China. As an offspring of TASE, CASE is the flagship conference of the IEEE Robotics and Automation Society, providing a premier international forum for automation researchers and practitioners to present and discuss their work. CASE 2022 marks the first time that the conference was held in Latin America, a significant milestone for both local and international researchers, academics, and practitioners. CASE 2022 is also the first time to have a satellite site besides the main event and venue in the COVID-19 pandemic era. The theme of CASE 2022 is Artificial Intelligence (AI) Automation.

We present a simultaneous sensor-based inspection and footprint coverage (SIFC) planning and control design with applications to autonomous robotic crack mapping and filling. The main challenge of the SIFC problem lies in the coupling of complete sensing (for mapping) and robotic footprint (for filling) coverage tasks. Initially, we assume known target information (e.g., cracks) and employ classic cell decomposition methods to achieve complete sensing coverage of the workspace and complete robotic footprint coverage using the least-cost route. Subsequently, we generalize the algorithm to handle unknown target information, allowing the robot to scan and incrementally construct the target map online while conducting robotic footprint coverage. The online polynomial-time SIFC planning algorithm minimizes the total robot traveling distance, guarantees complete sensing coverage of the entire workspace, and achieves near-optimal robotic footprint coverage, as demonstrated through empirical experiments. For the demonstrated application, we design coordinated nozzle motion control with the planned robot trajectory to efficiently fill all cracks within the robot's footprint. Experimental results illustrate the algorithm's design, performance, and comparisons. The SIFC algorithm offers a high-efficiency motion planning solution for various robotic applications requiring simultaneous sensing and actuation coverage.

Tracking control of underactuated balance robots needs to estimate balance profiles, that is, balance equilibrium manifold (BEM) of the unactuated subsystems. We present a learning-based approach to obtain the balance manifold for underactuated balance robots. We first establish the relationship between the BEM and the zero dynamics of the underactuated balance robots. The analysis shows that the BEM is a close approximation of the equilibria of the zero dynamics under perfectly tracking control. A Gaussian process learning-based method is proposed to estimate and obtain the BEM and zero dynamics, avoiding the direct inversion of the physics-based robot dynamic model. We demonstrate the analysis and applications experimentally on a rotary inverted pendulum and a bipedal robot.

The control of underactuated balance robots is aimed at performing both the external (actuated) subsystem trajectory tracking and internal (unactuated) subsystem balancing tasks. In this article, we propose a learning-based control design for underactuated balance robots. The key idea integrates a model predictive control method to design the desired internal subsystem trajectory and perform the external subsystem tracking task, while an inverse dynamics controller is used to stabilize the internal subsystem to its desired trajectory. The control design is based on Gaussian process (GP) regression models that are learned from experiments without requiringa prioriknowledge about the robot dynamics or the demonstration of successful stabilization. GP regression models also provide estimates of modeling uncertainties of the robotic systems, and these estimations are used to enhance control robustness to modeling errors. The learning-based control design is analyzed with guaranteed stability and performance. The proposed design is demonstrated by experiments on a Furuta pendulum and an autonomous bikebot.

Motion planning is a fundamental and critical task in modern autonomous systems. Conventionally, motion planning is built on uniform sampling that causes long planning procedure. Recently, built upon the powerful learning and representation abilities of deep neural network (DNN), neural motion planners have attracted a lot of attention because of the better biased sampling strategy learned from data. However, the existing NN-based motion planners are facing several limitations, especially the insufficient exploit of critical spatial information and the high computational cost incurred by neural network models. To overcome these limitations, in this paper we propose IMG-SMP, an algorithm and hardware co-design framework for neural sampling-based motion planner. At the algorithm level, IMG-SMP is an end-to-end neural network that can efficiently capture and process the critical spatial correlation to ensure high planning performance. At the hardware level, by properly rescheduling the computing scheme, the dataflow of IMG-SMP architecture can eliminate the unnecessary computations without affecting planning quality. The IMG-SMP hardware accelerator is implemented and synthesized using CMOS 28nm technology. Evaluation results across different planning tasks show that our proposed hardware design achieves order-of-magnitude improvement over CPU and GPU solutions with respect to planning speed, area efficiency and energy efficiency.

The COVID-19 pandemic has not only posed a significant threat to health, life, economy, and the whole society but also led to numerous new theoretical and practical challenges for automation science and engineering. The goal of this Special Issue is to bring together researchers and practitioners into a forum to show the state-of-the-art research and applications in responding to the challenges and opportunities of automation science and engineering to the pandemic, by presenting efficient scientific and engineering solutions, addressing the needs and difficulties for integration of new automation methodologies and technologies, and providing visions for future research and development.

Accurate objective, automated limb lameness detection and pose estimation play an important role for animal well-being and precision livestock farming. We present a wearable sensor-based limb lameness detection and pose estimation for horse walk and trot locomotion. The gait event and lameness detection are first built on a recurrent neural network (RNN) with long short-term memory (LSTM) cells. Its outcomes are used in the limb pose estimation. A learned low-dimensional motion manifold is parameterized by a phase variable with a Gaussian process dynamic model. We compare the RNN-LSTM-based lameness detection method with a feature-based multi-layer classifier (MLC) and a multi-class classifier (MCC) that are built on support vector machine/K-nearest-neighbors and deep convolutional neural network methods, respectively. Experimental results show that using only accelerometer measurements, the RNN-LSTM-based approach achieves 95% lameness detection accuracy and also outperforms the feature-based MLC or MCC in terms of several assessment criteria. The pose estimation scheme can predict the 24 limb joint angles in the sagittal plane with average errors less than 5 and 10 degs under normal and induced lameness conditions, respectively. The presented work demonstrate the successful use of machine learning techniques for high performance lameness detection and pose estimation in equine science.Note to Practitioners—Automation technologies are increasingly used for precision agriculture but few have focused on monitoring individual animals in open field for precision livestock farming. Limb lameness detection and pose estimation in open field is labor-intensive, unsafe for farmers, and inefficient. The presented machine learning-enabled, wearable inertial sensor-based design provides an effective and efficient approach for horse limb lameness detection and pose estimation applications. We present an RNN-LSTM for lameness detection and an integrated manifold learning model is used to predict the horse limb joint angles in walk and trot gaits under normal and induced lameness conditions. We also present a systematic analysis and experiments to demonstrate the impacts of the wearable sensor locations and signal information on lameness detection and pose estimation performance. Several other machine learning-based lameness detection methods are also presented and compared. The extensive multi-horse testing results are presented to demonstrate the superior accuracy and higher performance than other types of machine learning methods. One attractive feature of the proposed design lies in its high performance and fast computational capability for potential real-time applications in open field.

In this article, we develop a robust sliding-mode nonlinear predictive controller for brain-controlled robots with enhanced performance, safety, and robustness. First, the kinematics and dynamics of a mobile robot are built. After that, the proposed controller is developed by cascading a predictive controller and a smooth sliding-mode controller. The predictive controller integrates the human intention tracking with safety guarantee objectives into an optimization problem to minimize the invasion to human intention while maintaining robot safety. The smooth sliding-mode controller is designed to achieve robust desired velocity tracking. The results of human-in-the-loop simulation and robotic experiments both show the efficacy and robust performance of the proposed controller. This work provides an enabling design to enhance the future research and development of brain-controlled robots.

Real-time lower limbs motion or gait measurement is an important part in human-robotic interaction for the control of robotic walkers and rehabilitation devices. Laser range finder or infrared sensor that is mounted on the device has been widely used in applications. Although these sensors can provide accurate horizontal motion information of lower limbs during human walking, it is still difficult to measure the angular motion of lower limbs due to their functional principles. Using inertial measurement units (IMU) can measure the angular motion of lower limbs, but it requires a large amount of IMU units for measurements of all lower limb segments. In this study, a novel method is developed for real-time monitoring lower limbs (shanks and thighs) motion in human walking using just two shank-mounted IMUs. A pose prediction model based on multiple linear regression and Kalman filter is proposed. The root-mean-square error (RMSE) of the thigh orientation and the knee joint angle estimation in sagittal plane are 6.1 ± 1.3 and 6.8 ± 1.4 degs, respectively. The RMSE of the ankle, knee, and hip position estimation are 4.2 ± 1.3, 4.2 ± 1.1 and 3.5 ± 0.9 cm, respectively.

Modern agriculture relies on accurate and timely data. Currently, most of this data is gathered using remote sensing, which uses a combination of satellite and aerial imagery. However, ground robots are needed to fill in the gaps for finer ground-level data and the execution of physical tasks such as sample collection. The scales at which crops are produced preclude the inspection of each and every plant, thus requiring the selection of a smaller number of inspection targets. In this paper, we solve this multi-robot inspection problem using a novel task allocation algorithm. The algorithm derives its utility function from a model based on Gaussian process machine learning with a kernel that is learned from previous data. The algorithm also considers the physical limitations of moving within crop rows by dividing the plot into geodesic Voronoi regions based on robot locations. Simulation studies are performed to validate the method.

Load-carrying capability is an essential criterion in legged robots' practical application. This paper proposes an unpowered hydraulic auxiliary system to improve the legged robot's loading capability and energy efficiency. For humans, it has been widely hypothesized that intra-abdominal pressure can reduce potential injurious compressive force imposed on spinal discs when a person lifts heavy objects. Inspired by this human biomechanical phenomenon, we design a novel loading-carrying strategy using hydraulic cylinders, valves and accumulators. Different from ordinary powered hydraulic systems, this design provides continuous support force to share the load applied on knee joint actuator without consuming extra energy. The bent-leg theoretical model is constructed to validate the design and analysis. A bipedal hydraulic-assisted electric leg prototype (HyELeg) is fabricated and tested for squatting and walking gaits. The results show that with hydraulic assistance, the prototype can save energy by 45.9% for squatting with a load of 125% of its own body weight, and the walking performance is enhanced by 16.9% in energy efficiency with carrying a load of 67.5% of its own body weight.

Inertial measurement units (IMU) have been used for gait analysis in many clinical studies, as a more convenient, low cost and less restricted alternative to the laboratory-based motion capture systems or instrumented walkways. Spatial-temporal gait parameters such as gait cycle duration and stride length calculated from the IMUs were often used in these studies for evaluating the impaired gait. However, the spatial-temporal information provided by IMUs is limited, and sometime suffers incomplete and less effective evaluation. In this study, we develop a novel IMU-based method for clinical gait evaluation. Nine gait variables including three spatial-temporal parameters and six kinematic parameters are extracted from two shank-mounted IMUs for quantifying patient's gait deviations. Based on those parameters, an IMU-based gait normalcy index (INI) is derived to evaluate the overall gait performance. Eight inpatient subjects with gait impairments caused by n-hexane neuropathy and ten healthy subjects were recruited. The proposed gait variables and INI were examined on the inpatients at three to five time instants during the rehabilitation process until being discharged. A comparison with healthy subjects and statistical analysis for the changes of gait variables and INI demonstrated that the proposed new set of gait variables and INI can provide adequate and effective information for quantifying gait abnormalities, and help understanding the progress of gait and effectiveness of therapy during rehabilitation process.

We present stability and control analysis of a rider- bicycle system under human steering and body movements. The dynamic model of rider-bicycle interactions is first constructed to integrate the rider's body movement with the moving bicycle platform. We then present human balance control strategies based on human riding experiments. The closed-loop system stability is analyzed and discussed. Quantitative influences of the bicycle physical parameters, the human control gains, and the time delays are also analyzed and discussed. Extensive experiments are conducted to validate the human control models and demonstrate human balance performance using the bikebot, an instrumented bicycle platform. The presented modeling and analysis results can be potentially used for further development of bicycle-assisted rehabilitation for postural balance patients.

Control of underactuated balance robot requires external subsystem trajectory tracking and internal unstable subsystem balancing with limited control authority. We present a learning-based control approach for underactuated balance robots. The tracking and balancing control is designed the controller in fast- and slow-time scales. In the slow-time scale, model predictive control is adopted to plan desired internal state profile to achieve external trajectory tracking task. The internal state is then stabilized around the planned profile in the fast-time scale. The control design is based on a learned Gaussian process (GP) regression model without need of a priori knowledge about the robot dynamics. The controller also incorporates the GP model predicted variance to enhance robustness to modeling errors. Experiments are presented using a Furuta pendulum system.

A simultaneous robotic footprint and sensor coverage planning scheme is proposed to efficiently detect all the unknown targets with range sensors and cover the targets with the robot's footprint in a structured environment. The proposed online Sensor-based Complete Coverage (online SCC) planning minimizes the total traveling distance of the robot, guarantees the complete sensor coverage of the whole free space, and achieves near-optimal footprint coverage of all the targets. The planning strategy is applied to a crack-filling robotic prototype to detect and fill all the unknown cracks on ground surfaces. Simulation and experimental results are presented that confirm the efficiency and effectiveness of the proposed online planning algorithm.

Slip is one of the leading causes of fall-related injuries among occupational and elderly population. Existing research supports proactive slip and fall prevention approaches, while active strategies remain underdeveloped. Development of active slip-induced fall prevention systems requires fast, effective slip detection. This paper aims to develop a novel, real-time slip detection and estimation algorithm during human walking. The slip estimation is built on a slip dynamic model for biped walkers with the integration of the human locomotion constraints. The slip detection uses a set of wearable inertial sensors attached on the lower limbs. A slip indicator is introduced to detect the slip shortly after the heel-strike event. We present an extended Kalman filter-based slip estimation to characterize the slipping distance. One attractive property of the algorithm is its fast, accurate slip onset detection and slipping distance estimation with low-cost, nonintrusive sensing features. Experiments are conducted to validate and demonstrate the performance of the proposed slip detection and estimation scheme.

Inspired by the inchworm locomotion mechanism, this paper presents our recently developed self-deformable capsule-like robot. The robot has the actuated deformation capability that relies on a novel rigid elements-based morphing structure (REMS) and its soft actuation mechanisms. When the robot deforms, it generates the crawling locomotion behavior and thus friction waves between the robot and contact surface to facilitate the inchworm-like crawling movement. The paper starts reviewing the deformable properties of natural biological entities like capsules, presents state of the art of the current capsule-like robots, and details the bio-inspired design of the self-deformable capsule-like robot by describing the model of robot kinematics and its locomotion mechanism. Both simulation and experimental results validate the excellent performance of this capsule-like robot. The developed self-deformable capsule-like robot has the advantage of crawling on varied surfaces and it also has the capabilities to crawl in a variety of narrow pipes based on the deformation elicited locomotion nature of the robot.

To fully take advantage of the enormous potential of functional nanodevices, it is crucial to automate their manufacture with efficient steering and manipulation of multiple nanowires with controlled orientations to specific spatial locations. In this paper, we present motion planning and control algorithms for simultaneously steering multiple nanowires in liquid suspension. The motion planning and control is designed for a microfluidic device that is actuated by a simple generic set of electrodes. We first present a motion control algorithm to simultaneously steer multiple nanowires along different desired trajectories under controlled electrophoretic forces. A two-stage motion planning algorithm is then presented to generate the desired trajectory for each individual nanowire. Both numerical simulation and experimental results are presented to demonstrate the performance of the motion planning and control design using electric fields to simultaneously steer and manipulate multiple nanowires.

We present trajectory tracking and balance control of an autonomous bikebot. The bikebot is a single-track autonomous mobile robot that is designed to study unstable physical human-robot interactions. The controller is built on the property of the external-internal convertible (EIC) structure for the bikebot dynamics. We present two types of the designs and analyses of the control systems and also demonstrate their performance through extensive experiments. The comparison results with the human riding experiments show that the rider's motor skills generate the similar strategies as the proposed EIC-based balance control.

We present a drift-free pose estimation scheme for rigid body and its supporting platform by fusing only two gyroscopes and the relative complementary measurements. The fusion design not only provides robust relative attitude estimation between the rigid body and the platform, but also is capable of identifying partial global absolute attitudes without capturing any absolute attitude information. The pose estimation is built on a special design of the coupled kinematic model with the relative measurements between the rigid body and its supporting platform. We compare the fusion design with an alternative kinematic model and the posterior Cramer-Rao bound analyses are presented to show the completely different estimation performances. An extended Kalman filter (EKF) implementation of the fusion design is presented for the bicycle riding application.

Slip is the major cause of falls in human locomotion. We present a new bipedal modeling approach to capture and predict human walking locomotion with slips. Compared with the existing bipedal models, the proposed slip walking model includes the human foot rolling effects, the existence of the double-stance gait and active ankle joints. One of the major developments is the relaxation of the nonslip assumption that is used in the existing bipedal models. We conduct extensive experiments to optimize the gait profile parameters and to validate the proposed walking model with slips. The experimental results demonstrate that the model successfully predicts the human recovery gaits with slips.

Dynamic modeling of human bipedal walking is important for studying human locomotion and designing assistive and rehabilitation robotic devices. Physical principle-based models and data-driven learning models are two main methods to obtain human walking dynamics. We present analysis and connections between these two different modeling approaches. Mapping and correspondence are established between the reduced analytical dynamics (i.e., zero dynamics) of the bipedal walking and the learned latent dynamics from walking kinematic data. We present these mapping functions, their properties and applications. Experiments are presented to demonstrate and illustrate the analysis and results.

Underwater mobile sensor networks such as Autonomous Underwater Vehicles (AUVs) or robots are envisioned to enable applications for oceanographic data collection, environmental and pollution monitoring, offshore exploration, and distributed tactical surveillance. These applications require running compute- and data-intensive algorithms that go beyond the capabilities of the individual AUVs that are involved in a mission. To execute these task-parallel algorithms in resource- and time-constrained environments, dynamic and reliable collaboration between local networked robots (e.g., AUVs) and remote public Clouds is needed. To this end, the heterogeneous sensing, computing, communication, and storage capabilities of local and remote resources are exploited to form a “loosely coupled” mobile Cloud, and a novel resource provisioning engine that dynamically takes decisions on “what” and “where” the tasks should be executed in the mobile Cloud is introduced. Comparison of benefits of collaboration between local and Cloud resources with purely local and centralized approaches are presented through exhaustive computer simulations.

Automated manipulation of nanowires and nanotubes would enable the scalable manufacturing of nanodevices for a variety of applications, including nanoelectronics and biological applications. In this paper, we present an electric-field-based method for motion control, planning, and manipulation of nanowires in liquid suspension with a simple, generic set of electrodes. We first present a dynamic model and a vision-based motion control of the nanowire motion in dilute suspension with a set of$N\times N$controllable electrodes. Since the motion planning of a nanowire from one position to the target location is NP-hard, two heuristic algorithms are presented to generate near-optimal motion trajectories. We compare the heuristic motion planning algorithms with other existing algorithms such as the rapidly exploring random tree (RRT) and$A^{\ast}$algorithms. The comparisons show that the proposed heuristic algorithms obtain near-optimal minimum time trajectories. Finally, we demonstrate a single, integrated process to position, orient, and deposit multiple nanowires onto the substrate. Extensive experimental and numerical results are presented to confirm the motion control and planning algorithms.

We present a high level landmark-based visual navigation approach for a monocular mobile robot. We utilize heterogeneous features, such as points, line segments, lines, planes, and vanishing points, and their inner geometric constraints as the integrated high level landmarks. This is managed through a multilayer feature graph (MFG). Our method extends local bundle adjustment (LBA)-based framework by explicitly exploiting different features and their geometric relationships in an unsupervised manner. The algorithm takes a video stream as input, initializes and incrementally updates MFG based on extracted key frames; it also refines localization and MFG landmarks through the LBA. Physical experiments show that our method can reduce the absolute trajectory error of a traditional point landmark-based LBA method by up to 63.9%.

We present the development of the gyroscopic-balanced control of an autonomous bikebot. The bikebot is an actively controlled bicycle-based robotic platform with a gyro-balancer developed to study human dynamic postural balance motor skills through unstable physical human-robot interactions. We also present a dynamic model and analysis for stationary bikebot. A nonlinear balancing controller is designed to stabilize the underactuated stationary bikebot on an orbital trajectory around the unstable equilibrium point that is coupled with another orbit of the actuated gyro-balancer. We then demonstrate the analysis and control design with experimental validations. Finally, we present a set of human riding experiments to show how the bikebot can be used to perturb and excite human sensorimotor feedback loop for dynamic postural balance motor skills.

Bridge deck inspection is conducted to identify bridge condition deterioration and, thus, to facilitate implementation of appropriate maintenance or rehabilitation procedures. In this paper, we report the development of a robotic system for bridge deck data collection and analysis. The robotic system accurately localizes itself and autonomously maneuvers on the bridge deck to collect visual images and conduct nondestructive evaluation (NDE) measurements. The developed robotic system can reduce the cost and time of the bridge deck data collection. Crack detection and mapping algorithm to build the deck crack maps is presented in detail. The electrical resistivity (ER), impact-echo (IE) and ultrasonic surface waves (USW) data collected by the robot are analyzed to generate the corrosion, delamination and concrete elastic modulus maps of the deck. The presented robotic system has been successfully deployed to inspect numerous bridges.

We report the development of a human whole-body pose estimation scheme with application to rider-bicycle interactions. The estimation scheme is built on the fusion of measurements of a monocular camera on the bicycle and a set of small wearable gyroscopes attached to the rider's upper- and lower-limb and the trunk. A single feature point is collocated with each wearable gyroscope and also on the segment link where the gyroscope is not attached. An extended Kalman filter is designed to fuse the vision-inertial measurements to obtain accurate whole-body poses. The estimation design also incorporates a set of constraints from human anatomy and the physical rider-bicycle interactions. We demonstrate and compare the performance of the estimation design through multiple subjects riding experiments.

We develop a localization method to enable a team of mobile robots to search for multiple unknown transient radio sources. Because of signal source anonymity, short transmission durations, and dynamic transmission patterns, robots cannot treat the radio sources as continuous radio beacons. Moreover, robots do not know the source transmission power and have limited sensing ranges. To cope with these challenges, we pair up robots and develop a cooperative sensing model using signal strength ratios from the paired robots. We formally prove that the joint conditional posterior probability of source locations for the m-robot team can be obtained by combining the pairwise joint posterior probabilities, which can be derived from signal strength ratios. Moreover, we propose a pairwise ridge walking algorithm (PRWA) to coordinate the robot pairs based on the clustering of high-probability regions and the minimization of local Shannon entropy. We have implemented and validated the algorithm under both the hardware-driven simulation and physical experiments. Experimental results show that the PRWA-based localization scheme consistently outperforms the other four heuristics.

We develop a decentralized algorithm to coordinate a group of mobile robots to search for unknown and transient radio sources. In addition to limited mobility and ranges of communication and sensing, the robot team has to deal with challenges from signal source anonymity, short transmission duration, and variable transmission power. We propose a two-step approach: first, we decentralize belief functions that robots use to track source locations using checkpoint-based synchronization, and second, we propose a decentralized planning strategy to coordinate robots to ensure the existence of checkpoints. We analyze memory usage, data amount in communication, and searching time for the proposed algorithm. We have implemented the proposed algorithm and compared it with two heuristics. The experiment results show that our algorithm successfully trades a modest amount of memory for the fastest searching time among the three methods.

Underwater autonomous gliders (UAG) such as Slocum electric gliders provide an effective platform and tool for marine and coastal scientists to conduct prolonged missions over weeks or months. However, localization of these gliders underneath ocean surface is challenging due to the lack of inexpensive, effective underwater positioning sensors. In this paper, we present a dynamic model-based localization scheme to improve the positioning estimation of UAG. The new localization approach is built on an experimentally validated dynamic model and fused with onboard depth and yaw angle sensor measurements. We compare and validate the proposed scheme with dead reckoning and the ground truth.

We report system and algorithm developments that utilize a single mobile robot to simultaneously localize multiple unknown transient radio sources. Because of signal source anonymity, short transmission durations, and dynamic transmission patterns, the robot cannot treat the radio sources as continuous radio beacons. To deal with this challenging localization problem, we model the radio source behaviors using a novel spatiotemporal probability occupancy grid that captures transient characteristics of radio transmissions and tracks posterior probability distributions of radio sources. As a Monte Carlo method, a ridge walking motion planning algorithm is proposed to enable the robot to efficiently traverse the high-probability regions to accelerate the convergence of the posterior probability distribution. We also formally show that the time to find a radio source is insensitive to the number of radio sources, and hence, our algorithm has great scalability. We have implemented the algorithms and extensively tested them in comparison with two heuristic methods: a random walk and a fixed-route patrol. The localization time of our algorithms is consistently shorter than that of the two heuristic methods.

We develop a localization method enabling a team of mobile robots to search for multiple unknown transient radio sources. Due to signal source anonymity, short transmission durations, and dynamic transmission patterns, robots cannot treat the radio sources as continuous radio beacons. Moreover, robots do not know the source transmission power and have limited sensing ranges. To cope with these challenges, we pair up robots and develop a sensing model using the signal strength ratio from the paired robots. We formally prove that the sensed conditional joint posterior probability of source locations for the m-robot team can be obtained by combining the pairwise joint posterior probabilities, which can be derived from signal strength ratios. Moreover, we propose a pairwise ridge walking algorithm (PRWA) to coordinate the robot pairs based on the clustering of high probability regions and the minimization of local Shannon entropy. We have implemented and validated the algorithm under hardware-driven simulation.

We present balancing control analysis of a stationary riderless motorcycle. We first present the motorcycle dynamics with an accurate steering mechanism model with consideration of lateral movement of the tire/ground contact point. A nonlinear balance controller is then designed. We estimate the domain of attraction (DOA) of motorcycle dynamics under which the stationary motorcycle can be stabilized by steering. For a typical motorcycle/bicycle configuration, we find that the DOA is relatively small and thus balancing control by only steering at stationary is challenging. The balance control and DOA estimation schemes are validated by experiments conducted on the Rutgers autonomous motorcycle. The attitudes of the motorcycle platform are obtained by a novel estimation scheme that fuses measurements from global positioning systems (GPS) and inertial measurement units (IMU). We also present the experiments of the GPS/IMU-based attitude estimation scheme in the paper.

In this paper, we analyze optimal scheduling of a tree-like multicluster tool with single-blade robots and constant robot moving times. We present a recursive minimal cycle time algorithm to reveal a multi-unit resource cycle for multicluster tools under a given robot schedule. For a serial-cluster tool, we provide a closed-form formulation for the minimal cycle time. The formulation explicitly provides the interaction relationship among clusters. We further present decomposition conditions under which the optimal scheduling of multicluster becomes much easier and straightforward. Optimality conditions for the widely used robot pull schedule are also provided. An example from industry production is used to illustrate the analytical results. The decomposition and optimality conditions for the robot pull schedule are also illustrated by Monte Carlo simulation for the industrial example.

In semiconductor manufacturing, finding an efficient way for scheduling a multicluster tool is critical for productivity improvement and cost reduction. This two-part paper analyzes optimal scheduling of multicluster tools equipped with single-blade robots and constant robot moving times. In this first part of the paper, a resource-based method is proposed to analytically derive closed-form expressions for the minimal cycle time of two-cluster tools. We prove that the optimal robot scheduling of two-cluster tools can be solved in polynomial time. We also provide an algorithm to find the optimal schedule. Examples are presented to illustrate the proposed approaches and formulations.

The nine The ten papers in this special issue focus on three areas: equipment automation, operations automation, and modeling.

A mobile robot with a limited sensing range is deployed to search for a stationary target that intermittently emits short duration signals. The searching mission is accomplished as soon as the robot receives a signal from the target. We propose the expected searching time (EST) as a primary metric to evaluate different robot motion plans under different robot configurations. To illustrate the proposed metric, we present two case studies. In the first case, we analyze two common motion plans: a slap method (SM) and a random walk (RW). The EST analysis shows that the SM is asymptotically faster than the RW when the searching space size increases. In the second case, we compare a team ofnhomogeneous low-cost robots with a super robot that has the sensing range equal to that of the summation of thenrobots. Our analysis shows that the low-cost robot team takes Θ(1/n) time, while the super robot takes Θ(1/√n) time asn→ ∞. Our metrics successfully demonstrate their ability in assessing the searching performance. The analytical results are also confirmed in simulation and physical experiments.

We report our system and algorithm developments that enable a single mobile robot equipped with a directional antenna to simultaneously localize multiple unknown transient radio sources. Due to signal source anonymity, short transmission durations, and dynamic transmission patterns, the robot cannot treat the radio sources as continuous radio beacons.We model the radio source behaviors using a novel spatiotemporal probability occupancy grid (SPOG) that captures transient characteristics of radio transmissions and tracks the spatiotemporal posterior probability distribution of the radio transmissions. As a Monte Carlo method, we propose a ridge walking motion planning algorithm that enables the robot to efficiently traverse the high probability regions to accelerate the convergence of the posterior probability distribution. We have implemented the algorithms and the experiment results show that our method consistently outperforms methods such as a random walk or a fixed-route patrol mechanism.

Skid-steered mobile robots are widely used because of the simplicity of mechanism and high reliability. However, understanding of the kinematics and dynamics of such a robotic platform is challenging due to the complex wheel/ground interactions and kinematic constraints. In this paper, we attempt to develop a kinematic and dynamic modeling scheme to analyze the skid-steered mobile robot. We model wheel/ground interaction and analyze the robot motion stability. As an application example, we present how to utilize the kinematic and dynamic modeling and analysis for robot localization and slip estimation using only low-cost strapdown inertial measurement units (IMU). The extended Kalman filter (EKF)-based localization scheme incorporates the kinematic constraints. The performance of the EKF-based localization and slip estimation scheme are presented. The estimation methodology is tested and validated on a robotic testbed.

Skid-steered mobile robots are widely used because of their simple mechanism and high reliability. Understanding the kinematics and dynamics of such a robotic platform is, however, challenging due to the complex wheel/ground interactions and kinematic constraints. In this paper, we develop a kinematic modeling scheme to analyze the skid-steered mobile robot. Based on the analysis of the kinematics of the skid-steered mobile robot, we reveal the underlying geometric and kinematic relationships between the wheel slips and locations of the instantaneous rotation centers. As an application example, we also present how to utilize the modeling and analysis for robot positioning and wheel slip estimation using only low-cost strapdown inertial measurement units. The robot positioning and wheel slip-estimation scheme is based on an extended Kalman filter (EKF) design that incorporates the kinematic constraints for accuracy enhancement. The performance of the EKF-based positioning and wheel slip-estimation scheme are also presented. The estimation methodology is tested and validated experimentally on a robotic test bed.

We report our algorithmic development of the pframe problem that addresses the need of coordinating a set of p networked robotic pan-tilt-zoom cameras for n, (n ≫ p), competing polygonal requests. We assume that the p frames have almost no overlap on the coverage between frames and a request is satisfied only if it is fully covered. We then propose a Resolution Ratio with Non-Partial Coverage (RRNPC) metric to quantify the satisfaction level for a given request with respect to a set of p candidate frames. We propose a latticebased approximation algorithm to search for the solution that maximizes the overall satisfaction. The algorithm builds on an induction-like approach that finds the relationship between the solution to the (p — 1)-frame problem and the solution to the p-frame problem. For a given approximation bound ε, the algorithm runs in O(n/ε3+p2/ε6) time. We have implemented the algorithm and experimental results are consistent with our complexity analysis.

Cluster tools are widely used as semiconductor manufacturing equipment. While throughput analysis and scheduling of single-cluster tools have been well-studied, research work on multicluster tools is still at an early stage. In this paper, we analyze steady-state throughput and scheduling of multicluster tools. We consider the case where all wafers follow the same visit flow within a multicluster tool. We propose a decomposition method that reduces a multicluster tool problem to multiple independent single-cluster tool problems. We then apply the existing and extended results of throughput and scheduling analysis for each single-cluster tool. Computation of lower-bound cycle time (fundamental period) is presented. Optimality conditions and robot schedules that realize such lower-bound values are then provided using ldquopullrdquo and ldquoswaprdquo strategies for single-blade and double-blade robots, respectively. For an -cluster tool, we present lower-bound cycle time computation and robot scheduling algorithms. The impact of buffer/process modules on throughput and robot schedules is also studied. A chemical vapor deposition tool is used as an example of multicluster tools to illustrate the decomposition method and algorithms. The numerical and experimental results demonstrate that the proposed decomposition approach provides a powerful method to analyze the throughput and robot schedules of multicluster tools.

Modeling and scheduling of cluster tools are critical to improving the productivity and to enhancing the design of wafer processing flows and equipment for semiconductor manufacturing. In this paper, we extend the decomposition methods in the work of Dwande et al. (2005) for multi-cluster tools with buffer/process modules (BPMs). The computation of the lower-bound cycle time (fundamental period) is presented. Optimality conditions and robot schedules that realize such lower-bound values are then provided using "pull" and "swap" strategies for single-blade and double-blade robots, respectively. The impact of BPMs on throughput and robot schedules is studied. It is found that such an impact depends on the BPM processing time and the cycle times of the decomposed clusters on both sides of BPMs. A chemical vapor deposition (CVD) tool is used as an example of multi-cluster tools to illustrate the proposed method, analysis, and algorithms. The numerical and experimental results demonstrate the effectiveness and efficiency of the algorithms.

Skid-steered mobile robots have been widely used for terrain exploration and navigation. In this paper, we present an adaptive trajectory control design for a skid-steered wheeled mobile robot. Kinematic and dynamic modeling of the robot is first presented. A pseudo-static friction model is used to capture the interaction between the wheels and the ground. An adaptive control algorithm is designed to simultaneously estimate the wheel/ground contact friction information and control the mobile robot to follow a desired trajectory. A Lyapunov-based convergence analysis of the controller and the estimation of the friction model parameter are presented. Simulation and preliminary experimental results based on a four-wheel robot prototype are demonstrated for the effectiveness and efficiency of the proposed modeling and control scheme

Localization and wheel slip estimation of a skid- steered mobile robot is challenging because of the complex wheel/ground interactions and kinematics constraints. In this paper, we present a localization and slip estimation scheme for a skid-steered mobile robot using low-cost inertial measurement units (IMU). We first analyze the kinematics of the skid-steered mobile robot and present a nonlinear Kalman filter (KF)- based simultaneous localization and slip estimation scheme. The KF-based localization design incorporates the wheel slip estimation and utilizes robot velocity constraints and estimates to overcome the large drift resulting from the integration of the IMU acceleration measurements. The estimation methodology is tested and validated experimentally with a computer vision- based localization system.

In semiconductor assembly and test manufacturing (ATM), a station normally consists of multiple machines (maybe of different types) for a certain operation step. It is critical to optimize the utilization of ATM stations for productivity improvement. In this paper, we first formulate the bottleneck station scheduling problem, and then apply ant colony optimization (ACO) to solve it metaheuristically. The ACO is a biological-inspired optimization mechanism. It incorporates each ant agent's feedback information to collaboratively search for the good solutions. We develop the ACO-based scheduling framework and provide the system parameter tuning strategy. The system implementation at an Intel chipset factory demonstrates a significant machine conversion reduction comparing to a traditional scheduling approach.

We report a new trajectory tracking and balancing control algorithm for an autonomous motorcycle. Building on the existing modeling work of a bicycle, the new dynamic model of the autonomous motorcycle considers the bicycle caster angle and captures the steering effect on the vehicle tracking and balancing. The trajectory tracking control takes an external/internal model decomposition approach. A nonlinear controller is designed to handle the vehicle balancing. The motorcycle balancing is guaranteed by the system internal equilibria calculation and by the trajectory and system dynamics requirements. The proposed control system is validated by numerical simulations, and is based on a real prototype motorcycle system

We report our development of a vision-based motion planning system for an autonomous motorcycle designed for desert terrain, where uniform road surface and lane markings are not present. The motion planning is based on a vision vector space (V2-Space), which is an unitary vector set that represents local collision-free directions in the image coordinate system, V2-Space is constructed by extracting the vectors based on the similarity of adjacent pixels, which captures both the color information and the directional information from prior vehicle tire tracks and pedestrian footsteps. We report how V2-Space is constructed to reduce the impact of varying lighting conditions in outdoor environments. We also show how V2-Space can be used to incorporate vehicle kinematic, dynamic, and time-delay constraints in motion planning to fit the highly dynamic requirements of the motorcycle. The combined algorithm of the V2-Space construction and the motion planning runs in O(n) time, where n is the number of pixels in the captured image. Experiments show that our algorithm outputs correct robot motion commands more than 90% of the time

Cluster tools are widely used as semiconductor manufacturing equipment. While throughput analysis and scheduling of single-cluster tools have been well-studied, the corresponding research on multi-cluster tools is still at the early stage. This paper analyzes steady-state throughput and scheduling of multi-cluster tools. A decomposition method is utilized to reduce a multi-cluster tool problem to multiple single-cluster tool problems. Existing research on the throughput and scheduling results is then applied to each single-cluster tool. For an M-cluster tool, an O(M) throughput calculation and robot scheduling algorithm is presented. A chemical-mechanical planarization (CMP) polisher is used as an example of the multi-cluster cluster tools to illustrate the proposed decomposition method and algorithms.