The biased scene graph generation problem arises from the inherent long-tailed distributions of predicates, which are challenging to handle effectively with a single network. In this paper, we introduce a novel framework called peer learning, designed to address the issue of unbiased scene graph generation (USGG) through a divide-and-vote approach. To address the long-tailed problem, our framework operates in three steps. Firstly, we partition the heavily long-tailed distribution into subsets of more balanced sub-distribution groups, including head, body, and tail classes with a predicate sampling module. Next, we establish a peer network consisting of multiple peers, where each peer receives a combination of sub-distributions. This division enables peers to focus on different aspects of the scene graph generation task. Then, a novel peer learning loss function is introduced to cultivate the learning process among peer networks. Lastly, we employ the voting strategies for making final predictions within the peer network, boosting the influence of the majority’s opinion while downplaying the minority’s perspective. To illustrate the applicability of the proposed framework in intelligent transportation systems (ITSs), we further conduct qualitative evaluations on traffic scene understanding tasks. The results demonstrate that peer learning markedly enhances the reliability of interpreting complex traffic scenarios. Experimental results on the Visual Genome and Open Images V6 datasets further verify the effectiveness of our proposed model. These results highlight that the peer learning framework is well-suited for addressing the challenges of unbiased scene graph generation, offering practical benefits for ITS applications such as traffic analysis and monitoring. The code is available at: PL.

Image classification plays a pivotal role across diverse robotic applications, yet challenges persist when models are deployed in real-world scenarios. These models often fail to detect out-of-distribution (OOD) samples, classes not included in their training. This makes OOD detection a significant challenge for safe and effective real-world use. While existing techniques, like max logits, aim to leverage logits for OOD identification, they often disregard the intricate interclass relationships that underlie effective detection. This paper presents an innovative class relevance learning (CRL) method tailored for OOD detection. Our method establishes a comprehensive class relevance learning framework, strategically harnessing interclass relationships within the OOD pipeline. This framework significantly augments OOD detection capabilities. Extensive experimentation on diverse datasets, encompassing generic image classification datasets (Near OOD and Far OOD datasets), demonstrates the superiority of our method over state-of-the-art alternatives for OOD detection. The code is available on GitHub at: CRL.

In our daily lives, emotions are extremely important. However, predicting and regulating emotion is still a critical problem to be solved in the research of the human-computer interaction (HCI). In this study, we explore this problem using a regulation method. A novel multimodal affective computing algorithm is proposed and implemented in an emotion regulation system. The selection of music stimuli and the modeling of dynamic emotions are done using emotional Markov chains. This regulation system can monitor the user’s emotion and play music, selected by the regulation policy until the user can maintain the desired emotion. Our system was verified by two experiments. In the first experiment, by predicting the participants’ affective states, we tested the precision of our multimodal affective computing system. In the second experiment, we tested the regulation algorithm embedded in a closed-loop regulation system by comparing it with playing music without feedback. The results suggest that participants can regulate and maintain the desired affective state by using the emotion regulation system.

In this paper, we extend the blind-spot based self-supervised denoising by using affinity learning to remove noise from affected pixels. Inspired by inpainting, we introduce a novel Mask Guided Residual Convolution (MGRConv) to learn a neighboring image pixel affinity map that gradually removes noise and refines blind-spot denoising process. We show that mask convolution plays an important role in blind-spot denoising since it is theoretically aligned with $\mathcal{J} - invariance$, which blind-spot based self-supervised denoising frameworks are built upon. The theoretical analysis further shows the motivation behind using more adaptive mask convolutions. Our MGRConv not only enables dynamic mask learning without external trainable parameters, but also preserves appropriate mask constraints by sigmoid activation and residual summation. Our MGRConv is a balance between partial convolution and learnable attention maps, and boosts denoising performance better than other inpainting convolutions with similar or even less parameters, memory, and training/inference time. Extensive experiments show that our proposed plug-and-play MGRConv can assist blind-spot based denoising networks to reach promising results on both existing single-image based and dataset based benchmarks.

Natural image matting aims to precisely separate foreground objects from backgrounds using alpha mattes. Fully automatic natural image matting without external annotations is challenging. Well-performed matting methods usually require accurate labor-intensive handcrafted trimap as an extra input while the performance of automatic trimap generation method, e.g., erosion/dilation manipulation on foreground segmentation, fluctuates with segmentation quality. Therefore, we argue that how to produce a high-quality trimap using coarse segmentation is a major issue in automatic matting. In this paper, we present a two-stage trimap-free natural image matting pipeline that does not need trimap and background as input. Specifically, guided by a coarse segmentation, Trimap Generation Network (TGN) estimates a trimap where the coarse segmentation can be produced by segmentation/salient object detection/matting approaches, which enables more flexibility for matting to adapt into different scenarios. Then, with an estimated trimap as guidance, our Sampling Propagation Attention Matting Network (SPAMattNet) estimates an alpha matte. Different from previous propagation-based matting networks, inspired by traditional sampling/propagation matting approaches, we propose Sampling Propagation Attention (SPA) for matting network to incorporate sampling and propagation procedures in deep learning based manner for network explainability and performance improvement. It explicitly investigates local spatial and global semantic relationships to reconstruct alpha features. To better harvest sampling/propagation and local/global information, a Cross-Fusion Contextual Module (CFC) is introduced to aggregate features from different sources. Extensive experiments are conducted to show that our matting approach is competitive compared to other state-of-the-art methods in both trimap-free and trimap-needed aspects on several challenging matting benchmarks.

This paper introduces an intelligent system which composes music following the users' instructions. Current auto-matic music generation models are lack of stability. Meanwhile, they cannot satisfy the preference of different people. To overcome these challenges, we train a Transformer-based neural network to generate short music segments using a dataset. A user can compose music pieces by interacting with a well-trained generator. Our system collects the user's feedback during the interactions, and fine-tunes the neural network to optimize the generator. After a large number of interactions, our system can learn the musical taste of the user and customize a personal automatic music composer for him or her. Our work enhances the application value of generative models significantly, which enables people to compose music with the assistance of artificial intelligence.

The width of a neural network matters since increasing the width will necessarily increase the model capacity. However, the performance of a network does not improve linearly with the width and soon gets saturated. In this case, we argue that increasing the number of networks (ensemble) can achieve better accuracy-efficiency trade-offs than purely increasing the width. To prove it, one large network is divided into several small ones regarding its parameters and regularization components. Each of these small networks has a fraction of the original one's parameters. We then train these small networks together and make them see various views of the same data to increase their diversity. During this co-training process, networks can also learn from each other. As a result, small networks can achieve better ensemble performance than the large one with few or no extra parameters or FLOPs, i. e., achieving better accuracy-efficiency trade-offs. Small networks can also achieve faster inference speed than the large one by concurrent running. All of the above shows that the number of networks is a new dimension of model scaling. We validate our argument with 8 different neural architectures on common benchmarks through extensive experiments.

Hand gesture recognition plays an essential role in the human-robot interaction (HRI) field. Most previous research only studies hand gesture recognition in a short distance, which cannot be applied for interaction with mobile robots like unmanned aerial vehicles (UAVs) at a longer and safer distance. Therefore, we investigate the challenging long-range hand gesture recognition problem for the interaction between humans and UAVs. To this end, we propose a novel attention-based single shot multibox detector (SSD) model that incorporates both spatial and channel attention for hand gesture recognition. We notably extend the recognition distance from 1 meter to 7 meters through the proposed model without sacrificing speed. Besides, we present a long-range hand gesture (LRHG) dataset collected by the USB camera mounted on mobile robots. The hand gestures are collected at discrete distance levels from 1 meter to 7 meters, where most of the hand gestures are small and at low resolution. Experiments with the self-built LRHG dataset show our methods reach the surprising performance-boosting over the state-of-the-art method like the SSD network on both short-range (1 meter) and long-range (up to 7 meters) hand gesture recognition tasks.

Accurate perception of the surrounding scene is helpful for robots to make reasonable judgments and behaviours. Therefore, developing effective scene representation and recognition methods are of significant importance in robotics. Currently, a large body of research focuses on developing novel auxiliary features and networks to improve indoor scene recognition ability. However, few of them focus on directly constructing object features and relations for indoor scene recognition. In this paper, we analyze the weaknesses of current methods and propose an Object-to-Scene (OTS) method, which extracts object features and learns object relations to recognize indoor scenes. The proposed OTS first extracts object features based on the segmentation network and the proposed object feature aggregation module (OFAM). Afterwards, the object relations are calculated and the scene representation is constructed based on the proposed object attention module (OAM) and global relation aggregation module (GRAM). The final results in this work show that OTS successfully extracts object features and learns object relations from the segmentation network. Moreover, OTS outperforms the state-of-the-art methods by more than 2% on indoor scene recognition without using any additional streams. Code is publicly available at: https://github.com/FreeformRobotics/OTS.

Brain-computer interface (BCI)-based robotic telepresence provides an opportunity for people with disabilities to control robots remotely without any actual physical movement. However, traditional BCI systems usually require the user to select the navigation direction from visual stimuli in a fixed background, which makes it difficult to control the robot in a dynamic environment during the locomotion. In this paper, a novel SSVEP-based BCI stimuli system is proposed for robotic telepresence. The novel system utilized the live video streamed from the robot onboard camera as the input. By altering and flickering the detected objects in the scene with different frequencies predefined based on their relative positions on the screen, the robot can be navigated based on the user’s attention in a dynamic manner. In order to better differentiate multiple objects (more than the number of frequencies predefined), the task-related component analysis (TRCA) model was trained with a priori offline experimental data to select the front objects with priority. Experiments were conducted to validate the proposed system. Using the system, four human subjects are able to control a humanoid robot to navigate through multiple objects to reach the desired goal. The success rate reaches 87.5% in average.

Scene recognition is a fundamental task in robotic perception. For human beings, scene recognition is reasonable because they have abundant object knowledge of the real world. The idea of transferring prior object knowledge from humans to scene recognition is significant but still less exploited. In this paper, we propose to utilize meaningful object representations for indoor scene representation. First, we utilize an improved object model (IOM) as a baseline that enriches the object knowledge by introducing a scene parsing algorithm pretrained on the ADE20K dataset with rich object categories related to the indoor scene. To analyze the object co-occurrences and pairwise object relations, we formulate the IOM from a Bayesian perspective as the Bayesian object relation model (BORM). Meanwhile, we incorporate the proposed BORM with the PlacesCNN model as the combined Bayesian object relation model (CBORM) for scene recognition and significantly outperforms the state-of-the-art methods on the reduced Places365 dataset, and SUN RGB-D dataset without retraining, showing the excellent generalization ability of the proposed method. Code can be found at https://github.com/FreeformRobotics/BORM.

In this paper, a novel soft inflatable arm is proposed for telepresence robots. The new proposed structure of the arm is achieved by a very common and low-cost inflatable material and it is very light, weighing only about 50 g. However, it can realize agile movement by driving six tiny cables installed in the shoulder and elbow joints. The soft inflatable arm can work by pumping air at a very low pressure (7.32 ± 3.45 kPa) and allows direct and soft human contact without any external force sensors. This paper proposes joint compressed models, joint kinematic models, and kinematic models for the whole inflatable robot arm. These models can be easily applied to multijoint arms. In addition, a new redundancy resolution method is also developed for the inflatable arm, which makes it easier to control resolution and is less complex than other traditional approaches. Numerous experiments have been conducted, including performances of accuracy, repeatability, motion trajectories, human-safe interaction, and remote interaction. Results are satisfactory and validate the expected performance of the proposed robotic arm.

In this paper, a longitudinal wheel-slip controller for four wheel independent steering and drive (4WISD) vehicles is proposed to suppress longitudinal wheel slip in varying road conditions. Different from conventional methods that consider single driving source and zero steering angle, the proposed controller considers all independent traction sources from each driving wheel and omnidirectional steering command so as to eliminate slip detection errors in 4WISD vehicles. The proposed controller requires low cost sensing equipment, including merely wheel speed sensor and accelerometer, which makes the system practical to be utilized. The proposed wheel-slip controller can be applied to vehicles with arbitrary quantity of driving wheels and different steering configurations such as traditional two-front-wheel steering and two-rear-wheel steering. Numerical simulation results are presented to demonstrate the efficiency of the proposed longitudinal wheel-slip controller.

In this paper, a novel soft inflatable arm is proposed for telepresence robots. It is capable of imitating human arms to realize remote interaction. The new proposed arm using a very common and low cost inflatable material, and it is very light, which weight is only about 50 grams, but can well realize agile movement by driving three tiny cables installed in shoulder joint and elbow joint, respectively. Meanwhile, the proposed cable driven mechanism also allows connecting numbers of joints easily. The soft inflatable can work just by pumping air with very low pressure (7.32 ± 3.45 kPa), and allows human directly and safely contact without any external sensors. Moreover, to solve the challenge problems of soft joint deformation, the kinematic modeling of the joint with deformation compensation is also developed. Experimental results show that the soft inflatable arm can agilely move for remote interaction. The workspace and velocity are also close to an adult's arm movement space and normal motion speed.

In this paper, the kinematic modeling and control for a multi-joint inflatable robot arm with cable-driven mechanism are proposed. The soft inflatable robot arm is capable of imitating human arms to realize remote interaction. The weight of the arm is only about 50 grams, and collision safe. To solve the challenge problems of kinematics of the soft inflatable arm, new approaches are proposed, including redundant rigid arm and soft inflatable joint models. The approaches have a good advantage of applying to muti-joint arms. As our knowledge it is the first time to solve the kinematics of muti-joint soft inflatable arm in Three-Dimensional coordinate space. Numerous experiments have been conducted, including movement space and positioning accuracy. The workspace and velocity are close to an adult's arm movement space and normal motion speed.

Known as “the art of strokes”, Chinese calligraphy expresses its aesthetic through the strokes. A calligraphy learner practise the strokes and compose a calligraphic character by the strokes thereafter. Following the same process, the calligraphy robot, Callibot [2] needs to extract the strokes from a character. Therefore, we propose an approach to extract strokes using the geometric properties on the contour(s) of a character. A key discovery is that if two strokes intersect, the contour is concave; otherwise it is convex. The curvature vector defined in [1] is used to locate the vertexes whose interior angles are greater than 180° (these vertexes are named as C-points). C-points separate the contours into sub-contours. The corresponding sub-contours then form the basic strokes (i.e. dot stroke, horizontal stroke, vertical stroke, left-falling stroke and right-falling stroke). The experimental results show that this approach is feasible of extracting strokes from characters. This research is also useful for Chinese character recognition and calligraphic styles classification.

In this paper, mechanical design and implementation of a low-cost and lightweight inflatable robot finger are proposed. The proposed soft inflatable robot finger is different from traditional designs. It uses a common and low cost inflatable material and can be easily and massively manufactured. The proposed soft inflatable finger only weighs 0.8 grams, but can well realize swift movement which is actuated by low pressure air. Numerous analyses and experiments have been conducted for key parameters selection of the mechanical design. The performances of the proposed finger including flexing and extending have also been evaluated, and results are satisfactory.

Chinese calligraphy is a unique form of art in the world, whose aesthetic is mainly created by the proper manipulation of the brush. However, it is impossible for a person to figure out the 6-D motion of the brush from calligraphy images, if he has no experience of writing calligraphy. In this paper, we propose a Learning from Demonstration approach for our calligraphy robot, Callibot, to acquire calligraphy skills. We first propose a new stroke parametrization approach. Then we apply Locally Weighted Linear Regression to map from the stroke parameters to the trajectory of the brush. The training data are obtained from several demonstrations. Thereafter, Callibot is capable of writing a new stroke, if the stroke's parameters are given. The resulting motion is as natural as human writing. Experimental results prove the feasibility of our proposed approach. This approach is independent of the robot and is compatible with any robot with six or more degrees of freedom. This approach can be further integrated with our previous research, i.e. stroke extraction, so that Callibot will be able to replicate calligraphy from images.

This paper1presents a novel shape representation approach, Finite Element Contour (FEC), based on studies of shape analysis from the perspective of Finite Element Method (FEM). We assume that an edge of a contour can be modeled as a bendable beam element. Linking finite number of beam elements end to end along the contour, we obtain a closed-loop Finite Element Contour model as an approximate physical model of the original shape. By this model, we can calculate its natural frequency, which is one of mechanical properties that directly related to the geometric shape, and employed it as the shape representation. FEC shape feature possesses translation and rotation invariant properties naturally. We also realized scale and unique affine normalization in few simple steps based on intrinsic physical properties of shape from FEM viewpoint. Experimental results validated that the proposed FEC feature is capable of identifying shape in object recognition task. It also can describe shape deformation. In the well-known MPEG 7 shape retrieval task, the enhanced FEC approach obtains Bullseye score 87.11%.

In this paper, a traction/braking force distribution algorithm for omni-directional all-wheel-independent-drive vehicles is proposed as a tool to enhance driving stability. In the proposed algorithm, the amount of the traction or braking force on each driving wheel can be determined so as to generate a desired tangential force, yaw moment and centripetal force independently. The algorithm considers omni-directional steering command and is capable of handling both traction and braking force commands. The algorithm is applicable on vehicles with at least three independent driving wheels. Simulations have been conducted to illustrate the use of the proposed force distribution method in enhancing vehicles stability.

As one of the most unique types of art in Chinese culture, nowadays Chinese calligraphy is attracting increasing interests from researchers. It will be a big step if we have a robot to write Chinese calligraphy, especially in various styles, and it will form a bridge to combine science with art directly. However, there are so many different types and styles in Chinese calligraphy, and to distinguish them is the most basic quality to a green hand, but it is a big challenge for a robot to do so. For the lack of exploration about this, we conduct a lot of experiments to help the robot to accomplish it automatically. We first propose a parametric representation of calligraphic characters, and then adopt the Mahalanobis distance for similarity measurement and classification. The average accuracies of classifying types and styles of the Chinese calligraphy are 96.36% and 95.61% respectively. During the experiments, some interesting phenomena are discovered through similarity measure. Meanwhile, the parametric representation also has some potential applications, such as defining aesthetic grading standards of calligraphy and synthesizing calligraphy. Based on our research, the calligraphy robot can tell which style of calligraphy it sees for mimicking.

This paper presents a newly-designed robot named “Rubbot” dedicated to climbing on soft flexible clothes. Equipped with novel grippers which grip and rub on clothes, Rubbot is able to climb on flexible clothes and control how much fabric to grasp by feedback from infrared sensor. Rubbot also has a frame which has three passive folders which adjust the climbing posture of Rubbot. This not only makes Rubbot quite functional with clothes of different thicknesses and curved surfaces, but also makes Rubbot's motion more flexible. A theory of the deformation of cloth is then presented based on an analysis of creases created while Rubbot is climbing, this leads to a more reliable method to climb flexible surfaces. Finally experiments have verified that Rubbot is effective on flexible surfaces, as it can climb on 95% of the surfaces human clothes and still perform well on non-rigidly backed cloth.

Hand posture is a natural and effective human robot interaction way. In this paper, an user-independent hand posture recognition system using depth and color images captured from an RGB-D camera is presented. To recognize hand posture against complicated background conditions, we propose a novel method for automatic and accurate hand posture segmentation which detects the hand with Chamfer matching, tracks the hand with Kalman filter and segments the hand with region growing algorithm only in the depth space. A new hand posture descriptor invariant to scale, shift and in-plane rotation is constructed with the combination of local contour Fourier descriptor and global Bag-of-Features (BoF) descriptor based on Scale Invariance Feature Transform (SIFT). The sparse representation-based classification (SRC) is applied to perform the hand posture recognition task in the system. Experiments with a self-built large scale hand posture database collected online show the robustness and effectiveness of the proposed system.

The Canadarm2, also named Space Station Remote Manipulator System (SSRMS), is a 7-joint redundant manipulator. Without spherical wrists, the singularity analysis and avoidance of these manipulators are very difficult. In this paper, a method is presented to analytically identify its singular configurations based on the elementary transformation of Jacobian matrix. Firstly, we constructed a general kinematics model to describe them in a united manner. Correspondingly, the differential kinematics equation and the modified form are derived. Secondly, the singularity conditions are isolated and collected in a 3×4 sub-matrix by several times row transformation of the modified Jacobian matrix, which is partitioned into a block-triangle matrix. Finally, all the singularity configurations are determined by analyzing the rank degeneracy conditions of the 3×4 sub-matrix. The proposed method isolates the singularity conditions, and collects them in a 3×4 sub-matrix, largely reducing the computation workload.

In this paper, a deterministic yaw moment controller for four wheel independent steering and drive vehicles is proposed to enhance driving stability and controllability. Different to conventional methods that track a desired yaw rate, the proposed controller stabilizes a vehicle by additionally tracking the heading angle of a vehicle which is more efficient and robust. The heading angle of a vehicle is obtained by a novel method which is based on centripetal force detection. It eliminates the prerequisite knowledge of the characteristics between wheels and road surface which are time varying and difficult to be measured in real time. The proposed system only requires low cost sensing equipment such as wheel speed sensor and accelerometer that makes the system practical to be utilized. The proposed heading angle detection method can be generally applied to any kind of vehicle. The deterministic yaw moment controller is also applicable to any type of four wheel independent drive vehicles.

This paper presents a novel climbing robot called Clothbot which has high maneuverability on flexible clothes. It has a novel gripper consisting of two parallel wheels that can grip continuously and stably on various kinds of clothes. Clothbot also has an omni-directional tail of two DOFs so that it can change its center of gravity to control the moving direction on complex and undeterminate clothes. Consequently, Clothbot is able to access most positions of the clothes by moving straight and turning around with only four motors. It is compact, small and light-weighted but has a load capacity six times its own weight. A series of experiments validate its high performance on flexible clothes.

Collision avoidance of industrial robot arms in varying environment is a challenging task which has been a tough problem for decades. It often requires a large number of sensors and high computational power. Moreover, since the sensors are often mounted on the surface of robot arms, they may affect the appearance of the robot arms and may be vulnerable to damage. This video presents a cost-effective invisible sensitive skin that can cover a large area without utilizing a large number of sensors and it is built inside the robot arm. By using only 5 contactless capacitive sensors and specially designed antennas, collision avoidance of a 6-DOF industrial robot arm is attained.

This paper proposes a novel path planning method for a robot to climb on the deformable clothes surface. Based on the deformable characteristic of the clothes, the tension force of clothes is analyzed and the model of tension degree is established. A clothes climbing robot called Clothbot is composed of a two-wheeled gripper and a 2 Degrees of Freedom (DOF) tail. Based on the locomotion of this robot, the weights of tension degree and the locomotion characteristic are added into the A* algorithm. Combined with the two weights applied, the optimal path to the target for the Clothbot is obtained. The Clothbot has been developed to evaluate the algorithm. The simulation and the experiments have verified the feasibility of this method. In addition, The error state of the movement of the robot which is called side tumbling has been corrected by the motion of the 2-DOF tail.

This paper proposed a novel Tri-star wheeled robot called “Tribot”, which targets on high obstacle performance in unstructured environments, especially at the performance for climbing vertical obstacles. Tribot equips with six Tri-star wheels and each wheel can be driven independently. The chassis of the Tribot is divided into two parts which are connected by an articulated mechanism, making the Tribot has a remarkable obstacle performance to adapt changing environments mechanically, without any interpolate complex control. Numerous experiments have been conducted for vertical obstacle performance tests. Although the diameter of the wheel of the Tribot is only 220 mm, the robot can climb over vertical obstacle of 450 mm high, twice more of the wheel diameter. All results show that Tribot has excellent vertical climbing performance in unstructured environments.

In this paper, we present a novel Movable Gripper (MovGrip) which targets on climbing non-enclosable rectangular trusses such as bridges and space stations. It is designed with a transformation mechanism which provides the features of parallel grippers, rotatory grippers, and active wheels. For truss climbing, MovGrip acts as a parallel gripper which allows the change in gripping width according to different size of trusses. Since MovGrip is equipped with active wheels, fast climbing motion can therefore be realized. Moreover, MovGrip can be transformed into a mobile platform which is suitable for the navigation on ground. It weighs only 500 grams with a climbing speed of 3 cm/s. To maintain the climbing stability, we steer the rotation axis of wheels. By this, a directional pulling force (from MovGrip to truss surface) can be distributed from the drive force of wheels which pulls MovGrip towards the truss while climbing. Experimental results show that tilting of MovGrip can be auto-adjusted based on the proposed design. Also, it is shown that the load carrying capability of MovGrip is approximately 1 kg which is 2 times of its weight.

This paper proposed an autonomous tree climbing algorithm for a novel tree climbing robot named Treebot. Making a robot realize an environment and climb on a tree autonomously is a challenging task as the shape of tree is complex and irregular. To our best knowledge, this is the first paper dealing with the autonomous climbing problem in an unknown tree environment. The proposed method is aimed to use minimal sensing resources to achieve autonomous climbing. It reconstructs the shape of tree by using tactile sensors and guides the robot to climb along an optimal path. Numerous experiments have been carried out and the results are satisfactory.

This paper proposed a novel tree climbing robot "Treebot" that has high maneuverability on an irregular tree environment and surpasses the state of the art tree climbing robots. Treebot's body is a novel continuum maneuver structure that has high degrees of freedom and superior extension ability. Treebot also equips with a pair of omni-directional tree grippers that enable Treebot to adhere on a wide variety of trees with a wide range of gripping curvature. By combining these two novel designs, Treebot is able to reach many places on trees including branches. Treebot can maneuver on a complex tree environment, but only five actuators are used in the mechanism. As a result, Treebot can keep in compact size and lightweight. Although Treebot weighs only 600 grams, it has payload capability of 1.75 kg which is nearly three times of its own weight. On top of that, the special design of the gripper permits zero energy consumption in static gripping. Numerous experiments have been conducted on real trees. Experimental results reveal that Treebot has excellent climbing performance on a wide variety of trees.

In this paper, an novel tree gripping mechanism has been proposed for miniature tree-climbing robots. It is capable of attaching on a wide variety of trees with a wide range of gripping curvature. In addition, it is lightweight and simple in control. The gripper is simple in control as it is actuated by one actuator only. In addition, the omni-directional gripping ability also simplifies the use of the gripper as there is no extra actuator needed for orientation control. The special mechanism and optimized settings make the gripper able to attach on different sizes of tree tightly. The mechanism also allows zero energy consumption in static gripping. Numerous on-tree experiments of the proposed mechanism have been conducted and the results are satisfied.

In this paper, we propose an autonomous tree climbing strategy for a novel tree climbing robot that is named Treebot. The proposed algorithm aims to guide Treebot in climbing along an optimal path by the use of minimal sensing resources. Inspired by inchworms, the algorithm reconstructs the shape of a tree simply by the use of tactile sensors. It reveals how the realization of an environment can be achieved with limited tactile information. An efficient nonholonomic motion planning strategy is also proposed to make Treebot climb on an optimal path. This is accomplished by the prediction of the future shape of the tree. The study that is presented in this paper also includes the formulation of Treebot kinematics and an analysis of the workspace of Treebot on different shapes of a tree. Numerous experiments have been conducted to evaluate the proposed autonomous climbing algorithm and to unveil the ability of Treebot.

This paper proposes a path and motion planning algorithm for a tree climbing problem. This problem is challenging as the shape of tree is complex and irregular. To our best knowledge, this is the first paper dealing with the path planning problem on natural tree environment. Different from conventional motion planning approach that requires constructing a complex configuration space, this paper divides the planning problem into two parts, i.e., path and motion planning problem so as to reduce the dimension of the problem. An intuitive method to represent a climbing space is proposed that highly simplifies the path planning problem. With the use of a dynamic programming algorithm, an optimal path to reach a target position can be acquired in linear time. In addition, an efficient motion planning algorithm for a tree climbing robot named TreeBot is developed to make TreeBot follow the planned path.

In this paper, we study the stability region for a set of intelligent controllers developed by learning human expert control skills using support vector machines (SVMs). Based on the discrete-time system Lyapunov theory, a Chebychev points based estimation approach is proposed to evaluate the stability region, a key property of this set of SVM-based human learning controllers. One of such learning controllers has been implemented in vertical balance control of a dynamically stable, statically unstable single wheel mobile robot - Gyrover. The experimental results validate the proposed scheme for estimation of the stability region.

The promising electric vehicle (EV) technology is a direction to tackle the global non-renewable energy problem. However, the efficiency to use the electric energy still needs deliberate research. Traditional EV has no choice to manage its energy flow, because it has only one traction motor. With the robotic research in 4 wheel independent drive (4WID), the driving task of the single traction motor can be shared by 4 independent in-wheel motors. By exploring the motor efficiency map, we propose the energy management strategy based on optimal driving torque distribution (ODTD). The total input power of the 4 motors can be minimized while the driving performance is still maintained, and electric energy consumption can be reduced compared with traditional single motor driving EV. Simulation results validate the proposed strategy. The energy management strategy can also be applied to multi-driving-wheel mobile robots.

The failures of GEO (Geostationary Orbit) spacecrafts will result in large economic cost and other bad impacts. In this paper, we propose a space robotic servicing concept, and present the design of the corresponding system. The system consists of a 7-DOF redundant manipulator, a 2-DOF docking mechanism, a set of stereo vision and general subsystems of a spacecraft platform. This system can serve most existing GEO satellites, not requiring specially designed objects for grappling and measuring on the target. The serving tasks include: (a) visual inspecting; (b) target tracking, approaching and docking; (c) ORUs (Orbital Replacement Units) replacement; (d) un-deployed mechanism deploying; (e) extending satellites lifespan by replacing its own controller. As an example, the servicing mission of a malfunctioned GEO satellite with three severe mechanical failures is presented and simulated. The results show the validity and flexibility of the proposed system.

Battery equalization, aiming at keeping the state of charge of inside cells in the same level, is of great importance to maximize the capacity of whole battery pack and keep cells away from overcharge and overdischarge damage. In this paper, based on the analysis of bi-directional Cuk converter, we have proposed a fuzzy controller to adaptively tune the equalizing current. The inputs of fuzzy controller are selected as the difference in state of charge, the average of state of charge and the total internal resistance. The overall performance of the proposed equalizer is evaluated by multi-indexes such as equalizing speed, efficiency and cell protection. Simulations are conducted based on a well established 6Ah Li-ion battery provided in Advisor. The results under various initial conditions show that the proposed equalizer has the ability to balance the equalizing speed and efficiency. Any pair of cells with difference in state of charge less than 0.3 can be equalized within one hour and with the energy efficiency around 0.95.

Battery fast charging is a crucial issue in both research and application to realize and promote the mass commercialization of electric vehicles, especially pure electric vehicles. However, due to the strong nonlinear properties of batteries, the charging process should take into consideration various factors such as state of charge (SoC), temperature, and charging current, so as to assure the safety, reduce charging time, and enhance charging efficiency. In this paper, we propose a fast charging strategy under the model predictive control framework. Two models are employed to predict SoC and temperature under a sequence of future charging currents. SoC predictor is based on RC equivalent circuit and temperature predictor is based on thermal conduction and convection. The prediction of battery future states allows optimization of the control sequence, with the objectives to follow a predetermined SoC trajectory and to minimize battery temperature rising. Genetic algorithm are introduced to solve the constrained multi-objective optimization problem. The results using Advisor platform demonstrate the availability and efficacy of the proposed framework and prove that it has the ability to reduce charging time and heat generation simultaneously.

In this paper, an omni-directional steer-by-wire interface for four wheel independent steering vehicle is presented. The proposed steering interface is an extension of a traditional steering interface that provides three steering inputs. By combination of which, driver can control the vehicle in traditional way or omni-directionally without any mode switching operation. The reservation of the conventional steering behavior makes driver easy to adapt the novel steering interface. The force feedback controller is designed to synchronize the extended steering interface and the orientations of wheels so as to improve vehicle handling. Hardware-in-the-loop simulations are conducted to verify the hardware prototype and examine the proposed algorithms.

This paper proposes an optimal traction force distribution for omni-directional four wheel independent steering (4WIS) and four wheel independent drive (4WID) vehicle. The proposed force distribution algorithm is aimed to enhance the vehicle stability with minimum cost. The algorithm avoids the use of any feedback information of vehicle motion such as linear velocity as this information is difficult to measure accurately and the price of the measuring equipment is very high. As a result, the implementation cost can be reduced and at the same time avoid improper force distribution due to the inaccurate measured information. Moreover, the proposed algorithm does not involve any parameter tuning. It makes the algorithm easy to implement. The proposed algorithm can also be applied to any steering types of 4WID vehicle such as typical two wheel steering (2WS) as 4WIS is the general case of any steering configuration. Simulation results reveal that the performance of the proposed force distribution is superior to the uniform force distribution which is commonly used in 4WID vehicle.

In this paper, we aim to study and classify gait patterns among flat walking, descending stairs, and ascending stairs using inertial measurement unit (IMU) including triaxial accelerometers and gyroscopes. Six subjects were invited to gather gait data of flat walking, descending stairs, and ascending stairs wearing the shoe-integrated system with free speeds. The design of the classifier for identifying gait patterns based on continuous kinematic signals is composed of three steps. In the first step, we separate gait signals of the six sensors in the same period into gait segments which are further used as the units for pattern feature analysis. Secondly, based on discrete wavelet transform (DWT), the average sum of squares of wavelet coefficients of each segment for anteroposterior acceleration, vertical acceleration, and sagittal plane angular rate are demonstrated and selected as the common features for gait pattern classification. At the last step, the fuzzy logic based classifier is proposed according to the distribution of the common features of different gait patterns. Experimental results demonstrate the proposed methodology is efficient for classifying gait patterns during humans' daily activity.

State-of-charge (SOC) estimation is the most difficult problem in battery management system, which is one of the key component of electric vehicle and hybrid electric vehicle. Suffered from the non-zero mean noise and uncertain model parameters in practice, the conventional current integral and Kalman filter estimation methods can not achieve the required accuracy, even causing nonconvergent results. The essential difficulties to apply current integral and Kalman filter to solve SOC estimation problem in colored noise and time-variant battery system are analyzed. H∞filter, an estimator designed to handle the estimation problem in noised and uncertain situation, is then applied to calculate SOC online. The simulation experiment based on a typical battery model verifies the availability and efficiency of the proposed method.

In this paper we introduce a shoe-integrated system for human abnormal gait detection. This intelligent system focuses on detecting the following patterns: normal gait, toe in, toe out, oversupination, and heel walking gait abnormalities. An inertial measurement unit (IMU) consisting of three-dimensional gyroscopes and accelerometers is employed to measure angular velocities and accelerations of the foot. Four force sensing resistors (FSRs) and one bend sensor are installed on the insole of each foot for force and flexion information acquisition. The proposed detection method is mainly based on Principal Component Analysis (PCA) for feature generation and Support Vector Machine (SVM) for multi-pattern classification. In the present study, four subjects tested the shoe-integrated device in outdoor environments. Experimental results demonstrate that the proposed approach is robust and efficient in detecting abnormal gait patterns. Our goal is to provide a cost-effective system for detecting gait abnormalities in order to assist persons with abnormal gaits in the developing of a normal walking pattern in their daily life.

Postural kyphosis as one of the most common kinds of kyphosis is usually diagnosed in adolescents and young adults. Long-term kyphosis will not only affect the persons' appearance, but also result in thoracic deformity accompanied by pain. In this paper, we introduce a cost-effective shoe-integrated system which mainly consists of 8 force sensing resistors (FSRs) for gathering the pressure information under the 8 bony prominences. Based on the gathered plantar pressure information, the methodology of cascade neural networks with node-decoupled extended Kalman filtering (CNN-NDEKF) is applied for training the model of detecting the gait pattern associated with postural kyphosis. Experimental results demonstrate that the proposed approach is efficient. This device is of particular significance to provide feedback in the application of postural kyphosis rectification.

Condition monitoring and fault diagnosis in modern manufacturing automation is of great practical significance. It improves quality and productivity, and prevents damage to machinery. In general, this practice consists of two parts: 1)extracting appropriate features from sensor signals and 2)recognizing possible faulty patterns from the features. Through introducing the concept of marginal energy in signal processing, a new feature representation is developed in this paper. In order to cope with the complex manufacturing operations, three approaches are proposed to develop a feasible system for online applications. This paper develops intelligent learning algorithms using hidden Markov models and the newly developed support vector techniques to model manufacturing operations. The algorithms have been coded in modular architecture and hierarchical architecture for the recognition of multiple faulty conditions. We define a novel similarity measure criterion for the comparison of signal patterns which will be incorporated into a novel condition monitoring system. The sensor-based intelligent system has been implemented in stamping operations as an example. We demonstrate that the proposed method is substantially more effective than the previous approaches. Its unique features benefit various real-world manufacturing automation engineering, and it has great potential for shop floor applications.

Human gait is a kind of dynamic biometrical feature which is complex and difficult to imitate, it is unique and more secure than static features such as password, fingerprint and facial feature. Analyzing people walking patterns, their "step-prints", can lead to the recognition of personal identity. In this paper, we propose to design, build, calibrate, analyze, and use wearable intelligent shoes; then focus on classifying the wearers into authorized ones and unauthorized ones by modeling their individual gait performance. Firstly the intelligent shoes for collecting and modeling human gait to measure an unprecedented number of parameters relevant to gait are presented. Then we introduce cascade neural networks with node-decoupled extended Kalman filtering (CNN-NDEKF) from the paper by Nechyba and Xu (1997) to apply for modeling and classifier generation. Finally, the experimental results of learning algorithms and comparison are described and verify that the proposed method is valid and useful for human identification.

In this paper, we present an innovative design of a solder paste inspection device which can be practically integrated into existing solder paste printing machines. Since solder paste inspection systems usually occupy a large space in vertical direction, we designed a mirror box that can re-direct the transmission of fringe pattern. In this way, a new parallel solder paste inspection device with a significant reduction in the vertical constraint is developed. We also developed a hybrid weighting algorithm that applied the distance and fringe contrast to acquire the height of solder pastes. Furthermore, we developed an algorithm that generates the 2-D image from the fringe pattern images during the 4-steps algorithm. It gives benefit (time for solder paste inspection) to traditional approach that uses some special lighting systems to create the 2-D image. Experimental results show our device can inspect the 20mm times 20mm PCB area within 2 seconds and the maximum standard deviation for the average height is 3 mum.

Because of the inherent advantages of increased fuel economy, reduced harmful emissions and better vehicle performance, hybrid electric vehicles (HEV) powered by internal combustion engine (ICE) and energy storage, are being given more and more attention. In this paper, we present a novel approach to the problem of power control strategy for series hybrid electric vehicles (SHEVs). We define 3 different SHEV operation modes and a cost function. After the support vector machine (SVM) training process, we generate a classifier to determine which operation mode should be chosen during driving cycles based on the road situation data, battery state of charge (SOC) data and vehicle speed data. The approach does not need models of SHEV devices, costs less computationally and is more efficient. These distinguished advantages make the approach more practicable in real-time operation. Simulation study proves the feasibility of the approach.

In this paper, we presents a novel approach for tracking and catching operation of space robots using learning and transferring human control strategies (HCS). We firstly use an efficient support vector machine (SVM) to parametrize the model of HCS. Then we develop a new SVM-based learning structure to better implement human control strategy learning in tracking and capturing control. The approach is fundamentally valuable in dealing with some problems such as small sample data and local minima, and so on. Therefore this approach is efficient in modeling, understanding and transferring its learning process. The simulation results attest that this approach is useful and feasible in generating tracking trajectory and catching objects autonomously.

Aimed at the more and more serious problems of energy and pollution, Hybrid Electric Vehicle (HEV) is one of the best practical applications for transportation with high fuel economy and low emission. Since the power control strategy has a critical effect on the performance of HEV, genetic algorithm is introduced to optimize the strategy parameters for fuel economy and emissions in this paper. Compared with two main strategies, Thermostatic and DIRECT, the computation procedures of genetic algorithm are discussed, and simulation study based on the model of series hybrid electric vehicle is given to illustrate the optimization validity of the genetic algorithm.

Any motion of robotic manipulator will disturb its base in space due to the dynamic coupling. Such a disturbance will produce the serious impact between the manipulator hand and the object. Moreover, the disturbance will affect the communication with the ground and power supply for the space robot. On the other hand, compensating the disturbance using the attitude control system will consume large fuel which is limit in space. Therefore, a novel approach based on genetic algorithms (GA) is developed to find a global optimal path of a space robotic manipulator in joint space in order to minimize the disturbance to the base of space robot. The planning procedure is performed with respect to all constraints, such as joint angle constraints, joint velocity constraints, joint angular acceleration and torque constraints, and so on. We use GA to search the optimal joint inter-knot parameters in order to realize the minimum disturbance. These joint inter-knot parameters mainly include joint angle and joint angular velocities. We use an illustrative example to verify that GA-based optimal path planning method has satisfactory performance and real significance in engineering

Taking as reference the concept of agent in the artificial intelligence, this paper proposes a new multi-agent approach which can be employed in the surveillance for a people group in public places rather than a single person. The agent embodies the state and logic relationship between the person which the agent represents and the others in the same group. It does not merely stand for such individual information of persons as given by the existing surveillance systems. The results of experiments show that by using our multi-agent approach to compute and analyze the relationship between a number of agents, we can well perform real-time surveillance for the group and the related events so as to enhance the applicability and intelligence level of surveillance systems

As a typical multi-objective optimization problem, parameter optimization of HEV power control strategy must deal with the conflict between objectives, as fuel consumption and emissions. Classical methods define the HEV parameter optimization as a single objective problem to minimize the fuel consumption. In this paper, the multi-objective genetic algorithm (MOGA) is generalized for parameter optimization of power control strategy of series hybrid electric vehicle. Using a single unified formulation, a number of design objectives can be simultaneously optimized through searching in the parameter space. Compared with two main strategies, as Thermostatic and single-objective genetic algorithm (SOGA), the computation procedures of MOGA are discussed. Simulation results based on the model of series hybrid electric vehicle illustrate the optimization validity of MOGA

The goal of this paper is to monitor in-process on-board data as a means of indicating product quality and to be able to respond quickly to unexpected process disturbances. Due to pending environmental legislation and market requirements, lead-free soldering is widely used by the electronics industry. As the margin between the higher melting temperatures of lead-free solders and the heat-resistant temperatures of electronic components becomes narrower than lead solders, a more precise control of the temperature is required. Traditionally, the control processes of the on-board temperature are open loop because it is difficult to monitor the temperature on a PCB board. In this paper, we establish a method to determine the real process temperatures at any point on a PCB board in the furnace. We develop the method based on support vector machines (SVM) with multiple-input single-output strategies to learn relationship between the temperatures near the PCB board and the on-board temperature. The method is the only one which has been commercially utilized to predict the on-board temperature because of its low cost and high accuracy

In this paper, we introduce an intelligent glasses, which can automatically translate multiple languages in real-time, called wearable translation robot. This paper proposes the concept of the system and demonstrates the advantages of the device over other existed systems. The paper presents the system architecture and the functions of components in the system. We then focus on the most crucial technical component, text detection. The paper proposes a novel algorithm based on the fundamental characteristics of all the characters in common use called CIC-based text detection algorithm. We show the effectiveness of the proposed methods, and define some future works.

In this article, we discuss the dynamics characteristics of contact and impact when the hand of space manipulator captures the free-flying target (FFT). We establish the dynamics model of contact and impact between a space manipulator and FFT. The pre-impact, post-impact effect and condition of the space robot system and the FFT system are analyzed when there are any differences between the speed of the end-effector of the space manipulator and that of the contact and impact point on the surface of FFT. We present the relationship between the speed varieties of the space base and that of the FFT. If the impact force is kept constant, the speed varieties of the space base are different when the space robot system is at different configuration. Those methods can be used to analyze the contact and impact problem of the space robot.

This paper introduces a real-time video surveillance system which detects human abnormal behaviors. We present two approaches to such a problem. The first one employs principal component analysis for feature selection and support vector machine for classification of human behaviors. The proposed feature selection method is based on the border information of four consecutive blobs. The second approach computes optical flow to obtain the velocity of each pixel for determining whether a human behavior is normal or not. Both algorithms are successfully implemented in crowded environments for detecting the human abnormal behaviors, such as (1) running people in a crowded environment, (2) bending down movement while most are walking or standing, (3) a person carrying a long bar and (4) a person waving hand in the crowd. Experimental results demonstrate the two methods proposed are robust and efficient in detecting human abnormal behaviors.

In this paper, we developed a sensor-integrated shoe as an information acquisition platform to sense the foot motion. The system is small, portable and wearable. The platform is mainly composed of four parts including a sensing module, a computing module, a wireless communication module, and a data visualization module. Based on this platform, we developed a novel input device called Shoe-Mouse, which can be used by people who have difficulties in using their hands to operate computers or devices. We evaluated the performance of Shoe-Mouse, and the initial experimental results demonstrated the function. The platform can be also used for applications such as gait recognition, human identification, and motion monitoring.

Robot tracking and intercepting fast-maneuvering object is a classical and important issue. Many research results were published in recent years. Most of them employed model-based methods which require robot's model in advance. However, it is difficult and time-consuming to obtain robot's mathematical model. In this paper, we present a novel approach which needs no mathematical model. The proposed approach is based on learning tracking strategy from human beings. With human's demonstrations, the robot can learn and abstract human tracking and intercepting skill using cascade neural network. Preliminarily simulation results attest the feasibility of this novel approach. Furthermore, experiment is done on a real-time human face tracking system and the results verify the validity and efficiency of the approach.

In practice, the environments in which mobile robots operate are usually modelled in highly complex forms, and as a result autonomous navigation can be difficult. A novel navigation learning methodology is presented to abstract and transfer the human sequential navigational skill to a robotic wheelchair by showing the platform how to respond in different local environments along a demonstrated, designated route using a lookup-table representation. This method utilizes limited on-board range sensing information to concisely model local unstructured environments, with respect to the robot, for navigation along the learned route in order to achieve good performance with low on-line computational demand and low-cost hardware requirements. Experimental study demonstrates the feasibility of this method and some interesting characteristics of navigation and its associated localization and environmental modelling problems. Analysis is also conducted to investigate performance evaluation, advantages of the approach, choices of lookup-table inputs and outputs, and potential generalization of this study.

Surveillance of public places has become a world-wide concern in recent years. The ability to classify human behaviors in real-time is fundamental to the success of intelligent surveillance systems. The recognition of different human walking trajectory patterns is an important step towards the achievement of this goal. In this research, we utilize the approach of Longest Common Subsequence (LCSS) in determining the similarity between different types of walking trajectories. In order to establish the position and speed boundaries required for the similarity measure, we compare the performance of a number of approaches, including fixed boundary values, variable boundary values, learning boundary by support vector regression, and learning boundary by cascade neural networks. The LCSS similarity approach is also compared with a similarity measure based on hidden Markov model. We found that the boundary establishing method based on learning by support vector regression gives the best results using real-life data during testing.

This paper studied convergence conditions for a class of intelligent controllers. We formulated conditions to verify that the learned closed-form control system is strongly stable under perturbations (SSUP). We developed an approach to evaluate the convergence quality of this class of controllers with representation of support vector machine. It has been implemented in a balance control of a dynamically stable, statically unstable single wheel robot. The experimental results verified the proposed convergence conditions and the theory upon which it is based.

Condition monitoring and fault diagnosis in modern engineering practices is of great practical significance for improving the quality and productivity, preventing the machinery from damages. In general, this practice consists of two parts: extracting appropriate features from sensor signals and recognizing possible faulty patterns from the features. In order to cope with the complex manufacturing operations and develop a feasible system for real-time application, we proposed three approaches. By defining the marginal energy, a new feature representation emerged, while by real-time learning algorithms with support vector techniques and hidden Markov model representations, a modular software architecture and a new similarity measure were developed for comparison, monitoring, and diagnosis. A novel intelligent computer-based system has been developed and evaluated in over 30 factories and numerous metal stamping processes as an example of manufacturing operations. The real-time operation of this system demonstrated that the proposed system is able to detect abnormal conditions efficiently and effectively resulting in a low-cost, effective approach to real-time monitoring in manufacturing. The related technologies have been transferred to industry, presenting a tremendous impact in current automation practice in Asia and the world.

The real-time fault detection and diagnosis are critical for healthy operation of electromechanical systems, of which the complex characteristics affect the performance of current shop floor fault diagnosis methods. Aiming to overcome the drawbacks, this paper presents a new fault diagnosis method using a newly developed method, support vector machines (SVM). First, the basic theory of SVM is briefly introduced and new intelligent fault diagnosis system is presented. Next, three common SVM algorithms - v-SV, Lagrangian, and hyper-kernel - are employed for the proposed multiple faults diagnosis system. In comparison, the trade-offs among these three methods are discussed resulting in a general guideline of selecting appropriate learning algorithm for various applications. Then, the methods are applied for diagnosing vibration signals of a typical electromechanical system, elevator door. The real-time tests on 10 faulty conditions demonstrate that the proposed method is effective and efficient. In addition, the method requires only few training samples and permits fast calculation, giving it a big potential in real-world applications.

Human action understanding is crucial to the success of many human-machine interfaces based on vision. In this research, we apply artificial intelligence and statistical techniques towards observation of people, leading to modeling of their actions, and understanding of their intentions. In order to actualize the paradigm of learning from demonstration, a tracking system that is capable of locating the head and hand positions of moving humans has been developed. We propose to classify the motion trajectories of humans in the scene by using support vector classification. Since the data size of human motion trajectories is large, we apply principal component analysis (PCA) and independent component analysis (ICA) for data reduction. We have successfully applied the developed technique on two different applications: action recognition of table tennis players, and detection of human fighting motions.

The purpose of this work is to design a piecewise human learning control strategy for the autonomous control of dynamically stable systems in the following two cases. One case is in a single control process, the learning model is built up by combining some local neural networks. The other is that a desirable control target consists of some small control tasks which can be realized by human learning controllers individually. By estimating the stability region, we can guarantee the successful switch between two connected control pieces.

We are developing a novel robot concept called the wearable robot. Wearable robots are mobile information devices capable of supporting remote communication and intelligent interaction between networked entities. In this paper, we explore the possible functions of such a robotic network and will present a distributed network architecture based on service components. In order to support the interaction and communication between the components in the wearable robot system, we have developed an intelligent network architecture. This service-based architecture involves three major mechanisms. The first mechanism involves the use of a task coordinator service such that the execution of the services can be managed using a priority queue. The second mechanism enables the system to automatically push the required service proxy to the client intelligently based on certain system-related conditions. In the third mechanism, we allow the system to automatically deliver services based on contextual information. Using a fuzzy-logic-based decision making system, the matching service can determine whether the service should be automatically delivered utilizing the information provided by the service, client, lookup service, and context sensors. An application scenario has been implemented to demonstrate the feasibility of this distributed service-based robot architecture. The architecture is implemented as extensions to the Jini network model.

Changing facial expressions is a natural and powerful way of conveying personal intention, expressing emotion and regulating interpersonal communication. Automatic estimation of human facial expression intensity is an important step in enhancing the capability of human-robot interfaces. In this research, we have developed a system which can automatically estimate the intensity of facial expression in real-time. Based on isometric feature mapping, the intensity of expression is extracted from training facial transition sequences. Then, intelligent models including cascade neural networks and support vector machines are applied to model the relationship between the trajectories of facial feature points and expression intensity level. We have implemented a vision system which can estimate the expression intensity of happiness, anger and sadness in real-time.

We are developing a network architecture for our novel robot concept of wearable robot. Wearable robots are mobile information devices capable of supporting remote communication and intelligent interaction between networked entities. In this paper, a service-based wearable robot network architecture that involves extensions to the Jini network model is presented. We discuss three extensions to the original Jini model. The first extension involves the incorporation of a task coordinator service such that the execution of the services can be managed using a priority queue. The second extension enables the system to automatically push the required service proxy to the client intelligently based on certain system-related conditions. In the third extension, we allow the system to automatically deliver the services based on contextual information. Using a fuzzy-logic-based decision making system, the matching service can determine whether the service should be automatically delivered utilizing the information provided by the service, client, lookup service and context sensors. An application scenario has been implemented to demonstrate the feasibility of this distributed service-based robot architecture.

In this paper, we discuss the problem of how human control strategy can be represented as a parametric model using a Support Vector Machine (SVM), and how an SVM-based controller can be used to effectively control a dynamically stable system. We formulate the learning problem as a support vector regression and develop a new SVM learning structure to better implement human control strategy learning in control. The approach is fundamentally valuable in dealing with problems that normally dynamically stable robots experience, such as small sample data and local minima, and therefore is extremely useful in abstracting human controller for dynamic systems. The experimental study on the SVM approach with respect to other approaches clearly demonstrated the superiority of the SVM approach in terms of fidelity, efficiency and effectiveness in implementation.

In this paper, we study the interpolation approach in reducing the problem of small training sample sizes severely affecting the learning control performance of artificial neural networks when the dimension of the input variables is high. We use the local polynomial fitting approach to individually rebuild the time-variant functions of system states. Based on these functions, we can effectively produce new unlabelled training samples. We show that by using additional unlabelled samples, the learning control performance can be improved and, therefore, the overfitting phenomenon can be mitigated. Furthermore, experimental results verified these claims.

Surveillance of public places has become a world-wide concern. The ability to identify abnormal human behaviors in real-time is fundamental to the success of intelligent surveillance systems. The recognition of abnormal and suspicious human walking patterns is an important step towards the achievement of this goal. In this research, we have developed an intelligent visual surveillance system that can classify normal and abnormal human walking trajectories in outdoor environments by learning from demonstration. It takes into account both the local and global characteristics of the observed trajectories and be able to identify their normality in real-time. By utilizing support vector learning and a similarity measure based on hidden Markov models, the developed system has produced satisfactory results on real-life data during testing.

In this paper, we study the input selection in reducing the problem of the high dimension of input variables severely affecting the learning control performance of artificial neural networks. We first locally transform a nonlinear mapping problem into a nearly linear one by using the first-order derivatives of it. Then, we performed a local measure of the sensitivity of each of the model inputs (state variables) with respect to model outputs (human control inputs) under the least square error standard. Finally, based on voting, we defined a determination-rule to decide the importance order of the system state variables globally. By abstracting a human expert skill for controlling a dynamically stabilized robot: Gyrover, we validated the proposed approach.

We developed a semi-autonomous control for a dynamically stable robot, Gyrover, by combining machine intelligence and human operating behaviors into a shared control environment. In this system, the entire control task is shared between the autonomous module and the human operator: the robot itself maintains local balancing, while the operator is responsible for the global navigation. The autonomous module consists of two unique and essential behaviors: lateral balancing and fall recovery. These behaviors are modeled by a machine learning algorithm. We developed a method enabling the system to make a reasonable decision in shared control, and addressed the implementation issues in the paper. Experiments demonstrated that this shared control scheme provides an efficient way to control a dynamically stable system, such as Gyrover.

The single-wheel gyroscopically-stabilized robot, Gyrover, is dynamically stable but statically unstable, with both first-order and second-order nonholonomic constraints. In this paper, based on the dynamic model of the robot, we first study the two classes of nonholonomic constrains associated with the system. We then propose control laws for the stabilization in Cartesian space and tracking line segments, while keeping its balance laterally.

In practice, the environments in which mobile robots operate are usually modeled in highly complex geometric representations, and as a result real-time autonomous navigation can be difficult. Such difficulty is even exacerbated for robots with limited but more realistic on-board computational resources since this paradigm of environmental modeling requires enormous computational power. Inspired from human daily life experience, we propose in this paper a new direction for practical robotics navigation system with locally sensed non-geometric environmental modeling. With human-guided demonstrations, the robot can learn and abstract human navigational skill in the form of reactive sensor-motor mapping to navigate in the demonstrated route with simultaneous obstacle avoidance, localization, path and trajectory planning. Learning in a cascade neural network with node-decoupled extended Kalman filtering is adopted as the basis for such reactive mapping. Preliminarily experimental results show the feasibility of this practical approach.

Many disabled people do not have the dexterity necessary to use a joystick or other hand interface for controlling a standard robotic wheelchair. In many cases, they suffer from diseases which have damaged most of the nervous and muscular system in their body, but leave the brain and eye movement unimpaired. To this end, we developed an interface which enables the user to guide a robotic wheelchair by eye-gaze, using a minimal number of electrodes attached on the head. The device can measure the electro-oculographic potential of the eye-gaze movement together with the electromyographic signals from the jaw muscle motion. By coupling these simple actions, one is able to navigate a wheelchair solely by the eye and jaw movements, which provides an aid to mobility for severely disable people.

The single wheel, gyroscopically stabilized robot, Gyrover, is dynamically stable but statically unstable, with both first-order and second-order nonholonomic constraints. In this paper, based on the dynamic model of the robot, we first study the two classes of nonholonomic constraints associated with the system. We then propose control laws for balance control in different cases.

In this paper, we describe the design of the hardware and software for a virtual bronchoscope with force feedback. A haptic interface allows surgeons to feel the reaction force of virtual pneumonic surgery as if they were touching the area directly. We present novel algorithms for haptic force rendering, and examine its ability to display force. The rendering algorithms have been interfaced with a force-reflecting device. This virtual haptic bronchoscope is of significance in training inexperienced doctors in pneumonic diagnosis and surgery.

A dual neural network is presented for the bi-criteria kinematic control of redundant manipulators. To diminish the discontinuity of minimum infinity-norm solutions, the kinematic-control problem is formulated in the bi-criteria of the infinity and Euclidean norms. Physical constraints such as joint limits and joint velocity limits are also incorporated simultaneously into the proposed kinematic control scheme. The single-layer dual neural network model with a simple structure is developed for bi-criteria redundant resolution of redundant manipulators subject to robot physical constraints. The dual neural network is shown to be globally convergent to optimal solutions in the bi-criteria sense, and is demonstrated to be effective in controlling the PA10 robot manipulator.

We model human sensations in virtual reality applications using cascade neural networks. In the modeling process, the dimension of inputs presented to the humans and the sensation systems may be very high. In this research we propose using the independent component analysis (ICA) to achieve input reduction. We obtain human sensation data from a full-body motion virtual reality interface - "motion-based movie". A fixed-point ICA algorithm is applied to achieve feature extraction and input selection for reducing the dimension of the environmental stimulus data. The fidelity of the sensation models trained using the reduced inputs is verified by the hidden Markov model based similarity measure. The performance of input reduction using ICA is compared with that using the principal component analysis. Experimental results showed that the input selection scheme based on ICA is capable of improving the modeling performance of the computational sensation systems and reducing the input dimension by 60%.

Models of human control strategy (HCS), which accurately emulate dynamic human behavior, have far reaching potential in areas ranging from robotics to virtual reality to the intelligent vehicle highway project. A number of learning algorithms, including fuzzy logic, neural networks, and locally weighted regression exist for modeling continuous human control strategies. These algorithms, however, may not be well suited for modeling discontinuous human control strategies. Therefore, we propose a new stochastic, discontinuous modeling framework, for abstracting human control strategies, based on hidden Markov models (HMM). In this paper, we first describe the real-time driving simulator which we developed for investigating human control strategies. Next, we demonstrate the shortcomings of a typical continuous modeling approach in modeling discontinuous human control strategies. We then propose an HMM-based method for modeling discontinuous human control strategies. The proposed controller overcomes these shortcomings and demonstrates greater fidelity to the human training data. We conclude the paper with further comparisons between the two competing modeling approaches and we propose avenues for future research. © 2001 John Wiley & Sons, Inc.

In this paper, we present a method of modeling human strategy in controlling a light source in a dynamic environment. We take a simple example of how to control the light source to avoid a shadow and maintain appropriate illumination conditions on the target area of attention to illustrate the procedure and method. This work is valuable to various applications of automatic light control from surgical room and space applications to inspections.

A single wheel, gyroscopically stabilized robot was developed to provide a dynamic stability for rapid locomotion. It is a sharp-edged wheel actuated by a spinning flywheel for steering and a drive motor for propulsion. The spinning flywheel acts as a gyroscope to stabilize the robot and it can be tilted to achieve steering. In this paper, we present a path following controller for the robot. We first describe the robot motion by a set of configurations using the path curvature. We present a controller for tracking any desired straight line without falling over. For the controller, we first design the linear and steering velocities for driving the robot to the desired straight line through controlling the path curvature. The controller then applies the linear state feedback to stabilize the robot to the predefined lean angle such that the resulting steering velocity of the robot converges to the given steering velocity.

This paper studies the tracking control problem of mobile manipulators with consideration of the interaction between the mobile platform and the manipulator. A global tracking controller is proposed based on the dynamics of the defined tracking error and the extended Barbalat's lemma. The proposed controller ensures that the full state of the system asymptotically track the given desired trajectory globally in the presence of the system coupling. Extensive simulations presented in the paper show the effectiveness of the proposed approach.

We present a derivation for a spline smoother which takes into account local velocity information. This smoother is well suited for finding a best-fit trajectory from multiple example trajectories and is thus useful in applications such as programming by demonstration and online gesture recognition for teleoperation. Currently available smoothers are designed to consider only position information and not local velocity information, and are thus less suited for smoothing trajectories over time of dynamic systems.

The dynamics of a single wheel robot rolling without slipping on an inclined plane are investigated. The motion of a single wheel robot is analyzed using Lagrangian dynamics with no assumption that the robot is constrained to remain vertical. We linearized the dynamic model around the position perpendicular to the surface and proposed a state feedback controller for preventing the robot falling over. The backstepping control was designed to stabilize the robot following a straight path with a general heading angle. The feasibility and efficiency of the method are then validated by simulation study.

This paper aims to study human-machine integration in the human sensation aspect. We propose using cascade neural networks to model human sensation during the interaction, between humans and machines. The fidelity of the sensation models is verified using a hidden Markov model (HMM)-based similarity measure scheme. We applied this modeling technique in a full-body motion virtual reality interface-"motion-based movie". The sensation levels of the human participants in this application were modeled effectively by the cascade neural networks and the fidelity of the models were revealed by the HMM similarity measure scheme.

The dynamics of a rolling disk and a single wheel robot on an incline are derived and established respectively. The condition of rolling up is addressed. If the condition of rolling up is violated, a methodology of tracking is proposed. The system is stabilized around the position perpendicular to the surface. Simulation results are also provided.

We address the problem of transferring human control strategies (HCS) from an expert model to an apprentice model. The proposed algorithm allows us to develop useful apprentice models that incorporate some of the robust aspects of the expert HCS models. We first describe our experimental platform, a real-time graphic driving simulator, for collecting and modeling human control strategies. Then, we discuss an adaptive neural network learning architecture for abstracting HCS models. Next, we define a hidden Markov model (HMM) based similarity measure which allows us to compare different human control strategies. This similarity measure is combined subsequently with simultaneously perturbed stochastic approximation to develop our proposed transfer learning algorithm. In this algorithm, an expert HCS model influences both the structure and the parametric representation of the eventual apprentice HCS model. Finally, we describe some experimental results of the proposed algorithm.

In this paper, we present a method of modeling human strategy in controlling light source in dynamic environment. We take a simple example of how to control the light source to avoid a shadow and maintain appropriate illumination condition on the target area of attention to illustrate the procedure and method. The work is valuable to various applications of automatic light control from surgical room and space applications to inspections.

Gyrover is a single wheel, gyroscopically stabilized robot. It is a single wheel connected to a spinning flywheel through a two-link manipulator at the wheel bearing. The nature of the system is nonholonomic, nonlinear and underactuated. In this paper, we first develop a dynamic model and decouple the model with respect to the control inputs. We then study the effect of the flywheel dynamics on stabilizing the single wheel robot via simulation and experiment study. Finally, we design a linear state feedback control law that stabilizes the single wheel robot toward/in different lean angles, so as to control the precession rate. Simulation and experiment study validated the proposed controller as well as the developed dynamic model.

We present a method to model human operator's strategy in controlling a dynamically stabilized robot, Gyrover, which is a single-wheel gyroscopically stabilized robot. We first select the relevant state variables for training from kinematic and dynamic equations. Then, we defined a measure of the sensitivity of each of the state variables with respect to operator's control input by a sensitivity function in order to reduce the number of the state variables required in the model. We experimentally implemented the method and demonstrated that the robot can be automatically controlled using the learned human control model. The work is of significance in abstracting operator's skill for controlling a dynamically stabilized system in generating an automatic control input.

This paper studies the problem of modeling and control of multiple cooperative underactuated manipulators handling a rigid object. We reveal holonomic property of such a system by presenting a smooth feedback controller subject to two conditions: 1) there are not fewer active joints than the degrees of freedom of the object; and 2) the Jacobian matrix with respect to passive joints is not singular. This controller is an extension of the PD plus gravity compensation scheme and its asymptotic stability is guaranteed by the LaSalle theorem. Furthermore, we develop a trajectory tracking controller that yields asymptotic convergence of position errors and bounded interaction forces simultaneously. The performance of the proposed controllers has been investigated by simulations on two 6-DOF underactuated manipulators and by experiments on the cooperative underactuated manipulator system developed at CMU.

In this paper, we present a method for high-level control of robots whose low-level software is based on dynamically reconfigurable, reusable real-time software modules. Our approach is to use an embedded interpreter for a general-purpose programming language to direct the operation of the low-level modules toward meeting the task-level goals of the robot. To this end, we present RSK, a virtual-machine kernel implementing a scheme interpreter capable of hard real-time operation, and employing a method of code execution we call "message-based evaluation" (MBE). MBE is a novel combination of a traditional code execution model and a message-passing architecture, which simplifies the process of writing code for managing the robot's reconfigurable subsystem.

We develop a dynamic model of a gyroscopic wheel, an important component of Gyrover, a single-wheel robot developed at Carnegie Mellon University. The Gyrover robot consists of a single wheel, and is actuated through a spinning flywheel attached through a two-link manipulator at the wheel bearing. The flywheel can be tilted to achieve steering, and can be driven forwards and backwards to accelerate the robot. This paper focuses on developing a 3D model of the wheel part of the Gyrover. We first describe the Gyrover robot. We then develop the dynamic model of the wheel through the Lagrangian constrained generalized formulation. Finally, we implement the resulting equations of motion and present simulation results for the unactuated Gyrover in the different gravitational environments of Earth, the Moon, and Mars.

Models of human control strategy (HCS), which accurately emulate dynamic human behavior, have far reaching potential in areas ranging from robotics to virtual reality to the intelligent vehicle highway project. A number of learning algorithms, including fuzzy logic, neural networks, and locally weighted regression exist for modeling continuous human control strategies. These algorithms, however, may not be well suited for modeling discontinuous human control strategies. Therefore, we propose a new stochastic discontinuous modeling framework, for abstracting human control strategies, based on hidden Markov models. In this paper, we first describe the real-time driving simulator which we have developed for investigating human control strategies. Next, we demonstrate the shortcomings of a typical continuous modeling approach in modeling a discontinuous human control strategy. We then propose an HMM-based method of modeling discontinuous human control strategies, and show that the proposed controller overcomes these shortcomings and demonstrates greater fidelity to the human training data. We conclude the paper with further comparisons between the two competing modeling approaches.

In the last few years, modeling dynamic human control strategy (HCS) is becoming an increasingly popular paradigm in a number of different research areas, such as the intelligent vehicle highway system, virtual reality and robotics. Usually, these models are derived empirically, rather than analytically, from real human input-output control data. As such, there is a great need to develop adequate performance criteria for these models, as few guarantees exist about their theoretical performance. It is our goal in this paper to develop several such criteria. In this paper, we first collect driving data from different individuals through a real-time graphic driving simulator. We then model each individual's control strategy through the flexible cascade neural network learning architecture. Next, we develop two performance measures for evaluating the resulting HCS models, one dealing with obstacle avoidance, the other with tight-turning behavior. Finally, we evaluate the relative skill of different HCS models through the proposed performance criteria.

We propose in this work the first model-based robust control method for a team of underactuated manipulators jointly manipulating a load. The method is based on feedback linearization of the nonlinear dynamic coupling between the torques applied at the actuated joints and the Cartesian acceleration of the load, combined with a variable structure controller. Singularities in the control method are addressed, and a sufficient condition for a singularity-free controller implementation is obtained. Simulation and experimental results are presented to validate the theory presented.

Despite the large amount of research currently directed toward programming robots by demonstration, a significant problem with this method of human-to-robot skill transfer has not yet been addressed: developing representations of human performances which isolate the intrinsic dimensions of the performances (and thus the skills which guide them) within high-dimensional, raw human performance data. In this paper we propose the use of three methods for representing high-dimensional human performance data within lower-dimensional spaces: principal component analysis (PCA), nonlinear principal component analysis (NLPCA), and sequential nonlinear principal component analysis (SNLPCA). We compare the appropriateness of these methods for modeling a simple human grasping operation.

The underactuated free floating robot in space is a nonlinear system where velocity and acceleration constraints are both nonintegrable, therefore it is a second-order nonholonomic system. Some of the existing nonholonomic control methods will not be directly applicable to such systems as it is extremely difficult, if not impossible, to find the control Lie brackets. In this paper, by investigating the system dynamics in depth, we propose a simple velocity-based method to control the unactuated joints and a multistep composite strategy to implement orientation tracking tasks. The proposed algorithm is of significance in controlling of space robots when some joints fail to function, or they are intentionally set to be passive for energy efficiency and safety purposes.

Modeling the dynamic human control strategy (HCS) is becoming an increasingly popular paradigm in a number of different research areas, ranging from robotics to intelligent vehicle highway systems. Usually, HCS models are derived empirically, rather than analytically, from real human input-output data. While these empirical models offer an effective means of transferring intelligent behaviors from humans to robots and other machines, the models are not explicitly optimized with respect to potentially important performance criteria. We therefore propose an iterative algorithm for optimizing an initially stable HCS model with respect to an independent, user-specified performance criterion. We first collect driving data from different individuals through a real-time graphic driving simulator. Next, we describe how we model each individual's control strategy through flexible cascade neural networks. Once we have initially stable HCS models, we propose simultaneously perturbed stochastic approximation (SPSA) to optimize these models with respect to a chosen performance criterion. Finally, we describe and discuss some experimental results with the proposed algorithm.

We develop a dynamic model of the steering and actuation mechanism of Gyrover, a single-wheel robot which can be considered as a single wheel, actuated through a spinning flywheel attached through a two-link manipulator at the wheel bearing and a drive motor. The spinning flywheel acts as a gyroscope to stabilize the robot, and at the same time it can achieve steering. We develop a dynamic model, investigate its motion equation, and nonholonomic constraints, and present a simulation study. The work is significant in understanding this type of dynamically stable but statically unstable system, and in developing automatic control of the system.

Modeling dynamic human control strategy (HCS), or human skill in response to real-time sensing is becoming an increasingly popular paradigm in many different research areas. We propose a stochastic similarity measure, based on hidden Markov model analysis, capable of comparing and contrasting stochastic, dynamic, multidimensional trajectories. We first derive and demonstrate properties of the similarity measure for stochastic systems. We then apply the similarity measure to real-time human driving data by comparing different control strategies among different individuals. We show that the proposed similarity measure out performs the more traditional Bayes classifier in correctly grouping driving data from the same individual. Finally, we illustrate how the similarity measure can be used in the validation of models which are learned from experimental data, and how we can connect model validation and model learning to iteratively improve our models of HCS.

We propose a method to drive an underactuated manipulator among obstacles in its workspace. The method allows for collision-free trajectories to be generated from the dynamic equations of the manipulator. When the passive joints are locked, these trajectories lie on surfaces parallel to the axes of the active joints. When the passive joints are free, the trajectories lie on surfaces determined by the nonholonomic constraints imposed by the lack of actuation at the passive joints. By switching the joint brakes on and off, we obtain a sequence of trajectories that connect the start and the goal configurations. A robust controller is utilized to ensure that the manipulator follows the pre-planned trajectories closely despite modeling errors and external disturbances. Simulation and experimental studies demonstrate the validity of the proposed theory.

In this paper, we discuss the problem of how a free-floating space manipulator (SM) can be mapped to a conventional, fixed-base manipulator which preserves its dynamic and kinematic properties, and thus is called dynamically equivalent manipulator (DEM). The DEM concept allows one to use a conventional manipulator system to simulate a free-floating space manipulator connected to a space station, spacecraft, or satellite, without complicated experimental set-ups. This paper presents the theoretical development of the DEM concept, and demonstrates its dynamic and kinematic equivalence to the SM.

A single manipulator with passive joints is most likely nonholonomic systems, but multimanipulator systems may not. This paper investigates this issue by presenting a smooth feedback stabilization controller when the number of passive joints is not more than that of motion constraints associated to cooperations. This controller is a variation of the classical PD plus gravity compensation scheme and its asymptotic stability is guaranteed by LaSalle theorem. On the basis of this controller, we further discuss holonomy and nonholonomy conditions of multi-manipulator systems with passive joints. In addition, we propose a trajectory tracking controller which gives rise to asymptotic convergence of position errors and bounded interaction forces. Finally, we demonstrate asymptotic convergences of the proposed controllers with simulation study.

Modeling dynamic human control strategy (HCS), or human skill through learning is becoming an increasingly popular paradigm in many different research areas, such as intelligent vehicle systems, virtual reality, and space robotics. Validating the fidelity of such models requires that we compare the dynamic trajectories generated by the HCS model in the control feedback loop to the original human control data. To this end we have developed a stochastic similarity measure-based on hidden Markov model (HMM) analysis-capable of comparing dynamic, multi-dimensional trajectories. In this paper, we first derive and demonstrate properties of the proposed similarity measure for stochastic systems. We then apply the similarity measure to real-time human driving data by comparing different control strategies for different individuals. Finally, we show that the similarity measure outperforms the more traditional Bayes classifier in correctly grouping driving data from the same individual.

This work examines force modeling and software-based commutation for closed-loop control of planar linear motors, motivated by the need for a robust and versatile planar robot for precision assembly. The approach taken is to make measurements of the static and dynamic force capabilities of the motor as directly as possible, and determine the applicability of simple models commonly used. Measurements of force ripple, linearity with current, force reduction with skew angle, and eddy current damping forces are presented. The high-frequency current changes required for high-speed motion are shown to make the system sensitive to both the latency and update rate of the commutator and to limit the force generation capabilities at high velocities. Although this effect is caused by multiple sources, it is shown that it is well modeled as a scalar with units of time.

We propose the concept of the dynamically equivalent manipulator (DEM) of a free-floating space manipulator (SM) system. The dynamically equivalent manipulator can be physically built and used as an experimental testbed for the study of the dynamic performance and task execution of space robots. As it is a fixed-base manipulator, there is no need to resort to complex mechanisms to simulate the space environment. In this paper, we discuss two important issues associated with the DEM concept. First, we demonstrate the property of conservation of angular momentum and verify the validity of the DEM under free-flying conditions (i.e., when the SM base attitude is controlled via reaction wheels). Next, we investigate the effect of model uncertainty in the space manipulator and how it maps as errors in the parameters of the DEM. We derive explicit expressions for the error mapping and present a case study.

To successfully interact with and learn from humans in cooperative modes, robots need a mechanism for recognizing, characterizing, and emulating human skills. In particular, it is our interest to develop the mechanism for recognizing and emulating simple human actions, i.e., a simple activity in a manual operation where no sensory feedback is available. To this end, we have developed a method to model such actions using a hidden Markov model (HMM) representation. We proposed an approach to address two critical problems in action modeling: classifying human action-intent, and learning human skill, for which we elaborated on the method, procedure, and implementation issues in this paper. This work provides a framework for modeling and learning human actions from observations. The approach can be applied to intelligent recognition of manual actions and high-level programming of control input within a supervisory control paradigm, as well as automatic transfer of human skills to robotic systems.

Considers the problem of controlling an underactuated manipulator with less actuators than passive joints. The control methodology consists of dividing the passive joints in several groups, and of controlling one group at a time via its dynamic coupling with the actuators. Among the many possible control sequences for a given robot, we choose the optimal one based on the dynamic programming method. The optimization is based on a control cost defined as the reciprocal of the coupling index, a measure of the dynamic coupling available between the active and the passive joints of the manipulator. The detailed theory, computational procedures, simulation results, and experimental results are presented.

We are developing a novel concept for mobility, and studying fundamental research issues on dynamics and control of the mobile robot. The robot, called Gyrover, is a single-wheel vehicle with an internal gyroscope that provides mechanical stabilization and steering capability. This configuration conveys significant advantages over multi-wheel, statically stable vehicles, including good dynamic stability and insensitivity to attitude disturbances; high manoeuvrability; low rolling resistance; ability to recover from falls; and amphibious capability. In this paper we present the design, analysis and implementation of the robot, as well as the associated research issues and potential applications.

We have developed a gesture recognition system, based on hidden Markov models, which can interactively recognize gestures and perform online learning of new gestures. In addition, it is able to update its model of a gesture iteratively with each example it recognizes. This system has demonstrated reliable recognition of 14 different gestures after only one or two examples of each. The system is currently interfaced to a Cyberglove for use in recognition of gestures from the sign language alphabet. The system is being implemented as part of an interactive interface for robot teleoperation and programming by example.

Modeling dynamic human control strategy, or human skill, in response to real-time sensing is becoming an increasingly popular paradigm in many research areas. These models are learned from experimental data, and as such can be characterized despite the lack of a good physical model. Unfortunately, learned models presently offer few, if any, guarantees in terms of model fidelity to the source data. As such, we propose an independent, post-training model validation procedure based on hidden Markov models (HMMs). The proposed method generates a stochastic similarity measure comparing system trajectories for the source process and the learned models. Using this method, we are able to verify model fidelity. We demonstrate the proposed method in the validation of neural-network models for real-time human driving skill.

We present a novel mobile manipulator concept called the dual-use mobile detachable manipulator, or (DM)/sup 2/, for early construction and maintenance tasks in lunar stations. The robot consists of a wheeled rover, or mobile base and a detachable manipulator arm. The arm is symmetric, with a gripper at each end. When the arm attaches to the mobile base by grasping a handle with one of its grippers, the robot becomes a mobile manipulator and can perform exploration tasks such as collecting soil samples, surveying the lunar surface, and transporting tools and supplies. When the robot nears a lunar center structure such as a manufacturing center or a fuel tank, the manipulator arm can detach from the base and walk hand-over-hand, by grasping a series of handles on the structure, to perform tasks such as structure inspection, parts delivery, and simple assembly tasks. The paper discusses the concept and its advantages, the system under development, and its software architecture.

Underactuated manipulators are a class of robotic mechanisms where passive joints are present. By controlling only the motion of the active joints, it is possible to control the entire system. Our goal is to develop control schemes using both classical nonlinear and modem learning techniques for underactuated manipulators. To examine the validity of the approaches, we developed an experimental setup known as U-ARM, or underactuated robot manipulator. In this paper we present the hardware development, dynamic parameters, control software and experimental results of real-time control of the U-ARM.

Much work in recent years has focused on transferring human skill to robots by abstracting that skill into a machine-understandable, computational model. Such skill models, however, can be used not only for transferring human control strategy to robots, but also for helping less-skilled human operators improve their performance. The authors propose a two-step approach for transferring skill from human expert to human apprentice. An expert's relevant control strategies or skills are first abstracted into a sensory-based computational model. Afterwards, this trained computational model is used to generate on-line advice for less-skilled operators who need to improve their skill. This advice can take advantage of many different sensor modalities, thereby potentially improving both the quality and speed of learning for the apprentice. Furthermore, this approach allows for the efficient transfer of skill from a single expert to many apprentices, as well as from many experts to a single apprentice. In this paper, the authors first describe a flexible neural-network-based method for modeling human control strategy and provide motivation for its use. The authors then present a case study for teaching control strategy from one person to another in this two-step approach of transferring skill.

The self-mobile space manipulator (SM/sup 2/) has evolved to adapt to the new pre-integrated I-beam structure of the Space Station Freedom (SSF). In this paper, we first briefly overview the update of the robot configuration and testbed. The new robot is capable of projecting cameras anywhere interior or exterior of SSF, and will be an ideal tool for inspecting connectors, structures, and other facilities on SSF. Experiments have been performed under two gravity compensation systems and a full-scale model of a segment of the SSF. This paper then presents a real-time shared control architecture that enables the robot to coordinate autonomous locomotion and teleoperation input for reliable walking on SSF. Autonomous locomotion can be executed based on a CAD model and off-line trajectory planning, or can be guided by a vision system with neural network identification. Teleoperation control can be specified by a real-time graphical interface and a free-flying hand controller. SM/sup 2/ will be a valuable assistant for astronauts in inspection and other EVA missions.>

This paper presents a method for developing a gesture-based system using a multidimensional hidden Markov model (HMM). Instead of using geometric features, gestures are converted into sequential symbols. HMMs are employed to represent the gestures and their parameters are learned from the training data. Based on "the most likely performance" criterion, the gestures can be recognized by evaluating the trained HMMs. We have developed a prototype to demonstrate the feasibility of the proposed method. The system achieved 99.78% accuracy for a 9 gesture isolated recognition task. Encouraging results were also obtained from experiments of continuous gesture recognition. The proposed method is applicable to any multidimensional signal representation gesture, and will be a valuable tool in telerobotics and human computer interfacing.>

In this paper, we discuss the problem of how human skill can be represented as a parametric model using a hidden Markov model (HMM), and how an HMM-based skill model can be used to learn human skill. HMM is feasible to characterize a doubly stochastic process--measurable action and immeasurable mental states--that is involved in the skill learning. We formulated the learning problem as a multidimensional HMM and developed a testbed for a variety of skill learning applications. Based on "the most likely performance" criterion, the best action sequence can be selected from all previously measured action data by modeling the skill as an HMM. The proposed method has been implemented in the teleoperation control of space station robot system, and some important implementation issues have been discussed. The method allows a robot to learn human skill in certain tasks and to improve motion performance.

Robotic manipulators will play a significant role in the maintenance and repair of space stations and satellites, and other future space missions. Robot path planning and control for the above applications should be optimum, since any inefficiency in the planning may considerably risk the success of the space mission. This paper presents a global optimum path planning scheme for redundant space robotic manipulators to be used in such missions. In this formulation, a variational approach is used to minimize the objective functional. It is assumed that the gravity is zero in space, and the robotic manipulator is mounted on a completely free-flying base (spacecraft) and the attitude control (reaction wheels or thrust jets) is off. Linear and angular momentum conditions for this system lead to a set of mixed holonomic and nonholonomic constraints. These equations are adjoined to the objective functional using a Lagrange multiplier technique. The formulation leads to a system of differential and algebraic equations (DAEs). A numerical scheme for forward integration of this system is presented. A planar redundant space manipulator consisting of three arms and a base is considered to demonstrate the feasibility of the formulation. The approach to optimum path planning of redundant space robots is significant since most robots that have been developed for space applications so far are redundant. The kinematic redundancy of space robots offers efficient control and provides the necessary dexterity for extra-vehicular activity that exceeds human capacity.>

A shared control system is a modular real-time system which is designed to execute complex tasks through the intelligent coordination of task modules. A state machine is used to control task sequencing and, due to the automatic switching, the accuracy and reliability with which tasks are executed is greatly improved. Tasks consist of sets of independent, modular and reusable subtasks whose outputs are combined to create the robot control. This system has proved itself useful for rapid development of reliable high-level, multiple sensor-based manipulation and control tasks. Additionally, an extensible neural network-based visual servoing system, semi-compliant Cartesian trajectory-following heuristics, and a real-time graphical user interface have been developed. The shared control system was developed for the Self-Mobile Space Manipulator to handle a range of tasks associated with locomotion, manipulation, and material transportation on Space Station Freedom.

A model-based control scheme for robot manipulators employing a variable structure control law has been found to perform well, provided that the design parameters are carefully chosen. A refinement of the system model of this original scheme in which the actuator dynamics is taken into consideration is studied. Practical experiments are carried out on a commercial revolute-joint robot manipulator.

To perform simulations of partial or microgravity environments on earth requires some method of compensation for the earth's gravitational field. The paper discusses an active compensation system that modulates the tension in a counterweight support cable in order to minimize state deviation between the compensated body and the ideal weightless body. The system effectively compensates for inertial effects of the counterweight mass, viscous damping of all pulleys, and static friction in all parts of the gravity compensation system using a hybrid PI (proportional plus integral)/fuzzy control algorithm. The dynamic compensation of inertia and viscous damping is performed by PI control, while static friction compensation is performed by the fuzzy system. The system provides a very precise gravity compensation force, and is capable of non-constant gravity force compensation in the case that the payload mass is not constant. The only additional hardware requirements needed for the implementation of this system on a passive counterweight balance system are: a strain gauge tension sensor, and a torque motor with encoder.

Inverse kinematics is computationally expensive and can result in significant control delays in real time. For a redundant robot, additional computations are required for the inverse kinematic solution through optimization schemes. Based on the fact that humans do not compute exact inverse kinematics, but can do precise positioning for heuristics, an inverse kinematic mapping using fuzzy logic is developed. The implementation of the scheme has demonstrated that it is feasible for both redundant and nonredundant cases, and that it is very computationally efficient. The result provides sufficient precision, and transient tracking error can be controlled based on a fuzzy adaptive scheme proposed in the paper.

Self-Mobile Space Manipulator (SM/sup 2/) is a simple, 5-DOF (degree-of-freedom), 1/3-scale, laboratory version of a robot designed to walk on the trusswork and other exterior surfaces of Space Station Freedom. It will be capable of routine tasks such as inspection, parts transportation, and simple maintenance procedures. The authors have designed and built the robot and gravity compensation system to permit simulated zero-gravity experiments. They have developed the control system for the SM/sup 2/ including control hardware architecture and operating system, control station with various interfaces, hierarchical control structure, multiphase control strategy for step motion, and various low-level controllers. The system provides operator-friendly real-time monitoring, and robust control for 3D locomotion movements of the flexible robot.>

The authors discuss adaptive control of a space robot system with an attitude-controlled base on which the robot is attached. An adaptive control scheme in joint space is proposed. Since most tasks are specified in inertia space, instead of joint space, the authors discuss the issues associated to adaptive control in inertia space and identify two potential problems, unavailability of the joint trajectory (since mapping from inertia space trajectory is dynamics-dependent and subject to uncertainty), and nonlinear parameterization in inertia space. For a planar system, the linear parameterization problem is investigated, the design procedure of the controller is illustrated, and the validity and effectiveness of the proposed control scheme are demonstrated.>

We have developed a light-weight space manipulator, Self-Mobile Space Manipulator (SM'), in the Robotics Institute at Carnegie Mellon University. SM' is a 7-degreeof-freedom (DOF), 1/3-scale, laboratory version of a robot designed to walk on the trusswork and other exterior surfaces of Space Station Freedom, and to perform manipulation tasks that are required for inspection, maintenance, and construction. Combining the mobility and manip ulation functions in one body as a mobile manipulator, SM2 is capable of routine tasks such as inspection, parts transportation, object lighting, and simple assembly procedures. The system will provide assistance to astronauts and greatly reduce the need for astronaut extra-vehicular activity (EVA). This paper discusses the robot hardware development, gravity compensation system, control structure and teleoperation functions of the SM2 system, and demonstrates its capabilities of locomotion and manipulation in space applications.

An adaptive control scheme combining the variable structure and model reference methods is presented. With the variable structure technique, all known parameters of the robot system are fully used while the unknown parameters are adaptively adjusted. The overall control system maintains the basic structure of the computer torque controller, but incorporates adaptive components in the system. The method removes the requirement for persistent excitation, essential to traditional adaptive schemes for satisfactory operation. The control algorithm ensures the robustness of the controlled system with respect to disturbance, since the tracking error always converges to zero theoretically, rather than to an ill-defined residual set as in other adaptive schemes. Using this method, the transient response can be prescribed in advance. Thus, all the outstanding issues in adaptive control are directly treated. Simulation analysis for a two-degree-of-freedom robot is conducted to compare the method with the classical model reference method and computed torque method.>

Problems of modeling and controlling the 3D motion of a flexible space robot is discussed. By concentrating the mass at the joint and neglecting the link mass, the model of the flexible robot is derived based on Lagrange dynamics and rigid and flexible coordinates transformation. For further frequency analysis, the obtained linear model is modified so that state variables of the model are independent. The model has been verified by simulation results and experiments using the laboratory robot. The resultant model is simple and easy to be executed efficiently in the real-time control. Control problems are then discussed based on the derived model. A control scheme for adaptation to variations of dynamics due to configuration changes is presented and has been implemented in a space flexible robot.>

A compliant wrist combining passive compliance and a displacement sensor has been developed for a robot manipulator to be used in assembly operations. The wrist provides the necessary flexibility to accommodate transitions as the robot makes contact with the workpiece, to correct positioning error, and to avoid high impact forces in automatic assembly. Sensing from the device makes it possible to actively control the contact forces or to compensate the positioning error during motion and contact. The design features of two prototypes of the device are described. A hybrid position force control scheme using the device and incorporating the passive compliance in the design is presented. Two basic primitives in the assembly process, edge tracking and insertion operation with the compliant wrist, are investigated. A fuzzy controller is presented to assign velocity instead of evaluating force zones in insertion. The experimental results show that the system provides a feasible and economical solution to the provision of necessary compliance in automated assembly and manufacturing.>

A compliant wrist instrumented between a robot and its end effector provides a necessary compliance for assembly operations, and the displacement and force information generated from the wrist sensor can be utilized to actively control the end effector. The authors discuss the position compensation for the detection of the compliant wrist due to the gravity load and other external forces at the unconstrained space, and the force control as the robot is constrained with the environment. The system stability problem and dynamic performance are investigated for the different control laws, wrist parameters, and environment models. By analysis and simulation, some meaningful conclusions are obtained. The results are useful for design of the compliant wrist device and determination of the compensator law in the feedback loop.>