The 3D Bin Packing Problem (3D-BPP) is one of the most demanded yet challenging problems in industry, where an agent must pack variable size items delivered in sequence into a finite bin with the aim to maximize the space utilization. It represents a strongly NP-Hard optimization problem such that no solution has been offered to date with high performance in space utilization. In this paper, we present a new reinforcement learning (RL) framework for a 3D-BPP solution for improving performance. First, a buffer is introduced to allow multi-item action selection. By increasing the degree of freedom in action selection, a more complex policy that results in better packing performance can be derived. Second, we propose an agnostic data augmentation strategy that exploits both bin item symmetries for improving sample efficiency. Third, we implement a model-based RL method adapted from the popular algorithm AlphaGo, which has shown superhuman performance in zero-sum games. Our adaptation is capable of working in single-player and score based environments. In spite of the fact that AlphaGo versions are known to be computationally heavy, we manage to train the proposed framework with a single thread and GPU, while obtaining a solution that outperforms the state-of-the-art results in space utilization.

One of the most important and fundamental topics in autonomous navigated vehicle research is the lane change maneuver for obstacle avoidance or overtaking maneuver. In the literature, the lane change maneuver path for car-like vehicles has widely been generated with geometrically smooth segments by solving boundary conditions under given constraints. This paper proposes a new method of continuous curvature path generation for the issue of lane change maneuver for obstacle avoidance while solving the clothoids composition problem using an efficient algorithmic procedure. Conventional approaches resorting to mathematical or engineering optimization without considering human-side activity and response may fail to deliver driving performance that is favorable to humans. The novelty of the proposed method lies in its adoption of a human driving pattern, which is non-symmetric and composed of two different modes ofavoidanceandrecoveryduring the maneuver, and utilizes the property given by an appropriate iterative algorithm which takes into account all the constraints in order to solve the problem. As compared to conventional methods, the proposed method not only provides overall safety for obstacle avoidance, but also exhibits efficiency of increased comfort and human like steering motion during the lane change maneuver. The proposed path planning method is compared to other methods in order to validate its efficiency for safe and smooth obstacle avoidance maneuver.

The real-time pick and place of 3D industrial parts randomly filed in a part-bin plays an important role for manufacturing automation. Approaches based solely on the conventional engineering discipline have been shown limitations in terms of handling multiple parts of arbitrary 3D geometries in real-time. In this paper, we present a machine learning approach to the real-time bin picking of randomly filed 3D industrial parts based on deep learning with/without hybridizing conventional engineering approaches. The proposed hybrid approach, first, makes use of deep learning-based object detectors configured in a cascaded form to detect parts in a bin and extract features of the parts detected. Then, the part features and their positions are fed to the engineering approach to the estimation of their 3D poses in a bin. On the other hand, the proposed sole deep learning approach is based on, first, extracting the partial 3D point cloud of the object from its 2D image with the background removed and then transforming the extracted partial 3D point cloud to its full 3D point cloud representation. Or, it may be based on directly transforming the object 2D image with its background removed to the 3D point cloud representation. The experimental results demonstrate that the proposed approaches are able to perform a real-time multiple part bin picking operation for multiple 3D parts of arbitrary geometries with a high precision.

The first three industrial revolutions came about as a result of mechanization, electricity, and information technology (IT), respectively. Now, the introduction of the Internet of Things and Services into the manufacturing environment is fostering a 4th industrial revolution (Industry 4.0), where the smart optimization and computerization of all the actors and phases of the manufacturing process, including the conceptualization and design of a product, as well as its production and transaction, are taking a leading role.

For smart manufacturing, an automated robotic assembly system built upon an autoprogramming environment is necessary to reduce setup time and cost for robots that are engaged in frequent task reassignment. This article presents an approach to the autoprogramming of robotic assembly tasks with minimal human assistance. The approach integrates “robotic learning of assembly tasks from observation” and “robotic embodiment of learned assembly tasks in the form of skills.” In the former, robots observe human assembly operations to learn a sequence of assembly tasks, which is formalized into a human assembly script. The latter transforms the human assembly script into a robot assembly script in which a sequence of robot-executable assembly tasks are defined based on action planning supported by workspace modeling and simulated retargeting. The assembly tasks, in the form of the robot assembly script, are then implemented via pretrained robot skills. These skills aim to enable robots to execute difficult tasks that involve inherent uncertainties and variations. We validate the proposed approach by building a prototype of the automated robotic assembly system for a power breaker and an electronic set-top box. The results verify that the proposed automated robotic assembly system is not only feasible but also viable, as it is associated with a dramatic reduction in the human effort required for automating robotic assembly.

The early detection of a driver's intention prior to the initiation of actual maneuvering is to offer effective means of assisting the driver in times of safety. Conventional approaches in predicting a driver's intention include framing the motion parameters of the car and driver as well as the eye gaze sequence of the driver into the Hidden Markov Model (HMM) for analysis. However, their power of describing driver behavior is not only limited but also their performance, indicated by the recognition rate relative to the amount of preceding time in early detection, is not satisfactory and needs further improvement. This paper presents an approach for early detection of a driver's intention by modeling and analyzing driver behavior by using structural pattern recognition based on context-free and context-sensitive grammars. We specifically structured a sequence of the driver's eye fixations as well as of the vehicle speed, steering angle and signaling into a sequence of symbolic vectors to form sentences representing a specific driver behavior. It turns out that the proposed approach resulted in an average of 70.5% and 80% recognition rates at the respective 2 and 1 second preceding time to the actual initiation of maneuvering behavior. This performance can be compared to 56.8% and 69% by the state-of-the-art conventional results.

Fast and accurate pose estimation of non-textured 3D objects based on 3D point cloud is emerged as important for the application of vision to industrial automation. There have been proposed a number of global and local 3D geometric features that can be extracted from 3D point cloud. However, they are either too inefficient or too inaccurate to be viable. In this paper, we presents a fast and accurate method of 3D pose estimation based on transforming 3D point cloud into 2D image, where the azimuth and elevation angles of as well as the distance to 3D points are converted to 2D pixels and their image intensities, respectively. The experiments show that the proposed method provides faster and more accurate 3D pose estimation, two orders of magnitude and over 25% of improvements in processing time and accuracy, respectively, compared to a top performance conventional 3D geometric feature based approach.

In this paper, we propose a deep network based on PointNet to estimate the orientations and predict the object classes of 3D oriented objects using their partial model point clouds. More specific, our network exploits the advantages of PointNet to extract the global features of two kinds of point cloud: 1) 3D partial model orientation point cloud which is a part of a 3D object in an observed orientation and 2) full object model point cloud of the 3D object in the reference orientation which is referred to specify the orientations. We then associate the partial model point cloud global features with the corresponding reference global features by an association subnetwork, in which the association network takes the partial model global features as the input and output the corresponding reference feature reconstruction. We use this global feature reconstruction as aligned global features to infer the object classes of the partial model point cloud. To predict the orientation of an oriented point cloud from its partial model point cloud, we use the concatenation of partial model global features and the reference feature reconstruction as an optimal orientation features for network learning with orientation targets. Using the orientation dataset with partial model point clouds based on 3D ModelNet, our experiments have shown the better object classification performance comparing to the vanilla PointNet and the robustness of our proposed network in orientation estimation.

For the success of visually-guided robotic errand service, it is critical to ensure dependability under various ill-conditioned visual environments. To this end, we have developed Adaptive Bayesian Recognition Framework in which in-situ selection of multiple sets of optimal features or evidences as well as proactive collection of sufficient evidences are proposed to implement the principle of dependability. The framework has shown excellent performance with a limited number of objects in a scene. However, there arises a need to extend the framework for handling a larger number of objects without performance degradation, while avoiding difficulty in feature engineering. To this end, a novel deep learning architecture, referred to here as FER-CNN, is introduced and integrated into the Adaptive Bayesian Recognition Framework. FER-CNN has capability of not only extracting but also reconstructing a hierarchy of features with the layer-wise independent feedback connections that can be trained. Reconstructed features representing parts of 3D objects then allow them to be semantically linked to ontology for exploring object categories and properties. Experiments are conducted in a home environment with real 3D daily-life objects as well as with the standard ModelNet dataset. In particular, it is shown that FER-CNN allows the number of objects and their categories to be extended by 10 and 5 times, respectively, while registering the recognition rate for ModelNet10 and ModelNet40 by 97% and 89.5%, respectively.

Localization of a human or a robot in indoor environment prior to initial human-robot interaction is the motivation of this paper. The utilization of Wi-Fi received signal strength (RSS) by adopting fingerprinting technique is used to compute the location estimation. However, Wi-Fi RSS instability incurred by mutable channel characteristics hampers a wide-spread adoption of RSS based location fingerprinting to real world applications. To overcome RSS instability problem, we propose a new method based on the concept of “invariant RSS statistics”, where it is defined as the RSS samples collected at each calibration location under minimal random spatiotemporal disturbances. The invariant RSS statistics thus collected forms the reference pattern classes in the space formed by Wi-Fi sources. Fingerprinting is done by identifying the reference pattern class that maximally supports the RSS readings collected at an unknown location for available Wi-Fi sources. The support of RSS readings is defined here as the sum of the likelihood probabilities of individual RSS readings. Experimental results show that the proposed method provides superior performance to conventional ones with the success rate higher by 17%, the printing resolution finer by 30% and, naturally, no performance degradation in time without recalibration. Lastly, we present a design guideline that relates statistically the different levels of the random spatiotemporal disturbances inducing RSS instability to the minimum number of Wi-Fi sources required for achieving a certain class separation degree under the given number of calibration locations to be identified.

A method of extracting major lines from a 2D image is presented. The novelty lies in that major lines are recruited from the maximally generated yet well-thinned zero threshold Canny edge links based on the, so called, Sobel highlights as the guide for recruitment. The Sobel highlights introduced here represent the scores accumulated at individual pixels that measure their significance of forming line segments along Sobel edge orientations. The proposed method offers several advantages over conventional ones: it is 1) more powerful to extract cursive lines, in particular, with a larger curvature, 2) more effective to represent a line as a whole with less intermittent discontinuities, and 3) more straightforward to use as it relies on no sensitive edge thresholding. Experimentations verify that the proposed method outperforms LSD, a top performance major line detector currently available, by about 20% in the coverage of ground truth major lines as well as in the coverage per major line: an intermittent discontinuity indicator. The extraction cycle time is about 54 msec using the Intel i7-2600 CPU and 4G RAM under Window 7.

International audience

This paper proposes a Kinect-based calling gesture recognition scenario for taking order service of an elderly care robot. The proposed scenarios are designed mainly for helping non expert users like elderly to call service robot for their service request. In order to facilitate elderly service, natural calling gestures are designed to interact with the robot. Our challenge here is how to make the natural calling gesture recognition work in a cluttered and randomly moving objects. In this approach, there are two modes of our calling gesture recognition: Skeleton based gesture recognition and Octree based gesture recognition. Individual people is segmented out from 3D point cloud acquired by Microsoft Kinect, skeleton is generated for each segment, face detection is applied to identify whether the segment is human or not, specific natural calling gestures are designed based on skeleton joints. For the case that user is sitting on a chair or sofa, correct skeleton cannot be generated, Octree based gesture recognition procedure is used to recognize the gesture, in which human segments with head and hand are identified by face detection as well as specific geometrical constrains and skin color evidence. The proposed method has been implemented and tested on “HomeMate”, a service robot developed for elderly care. The performance and results are given.

Object search is a basic function of a service robot. Previous approaches focus on cases where objects are lying out in the open, which transform object search into active visual search. However, in real life, objects may be behind cupboards occluded by other objects, instead of conveniently lying on a table by themselves. We present a searching method that uses object information, spatial constraints and Bayes probability calculation to handle problems such as occlusion while searching in a clutter environment.

The purpose of this study is to understand human's visual information processing mechanisms. We emphasize the importance of "context" for attracting visual attention as an effect of top-down process. We created indices for saliency index by degree of saliency using Itti's saliency model and for context index by object importance from pre-experimental survey for analyzing the present data. In terms of bottom-up process, the effect of bottom-up saliency was insignificant and correlation between the saliency map and human fixation map was rather low. We compared the context index within both salient and non-salient area that top-down processing is a strong attentional guidance. If the contextually meaningful objects and visually salient area was overlapped, it robustly attracts attentions. Furthermore, the counts of fixations on objects and the order in sequence of eye-movement showed that the subjects tended to fixate more often and more frequently toward more importantly considered objects.

An energy-efficient support vector machine (EE-SVM) learning control system considering the energy cost of each training sample of biped dynamic is proposed to realize energy-efficient biped walking. Energy costs of the biped walking samples are calculated. Then the samples are weighed with the inverses of the energy costs. An EE-SVM objective function with energy-related slack variables is proposed, which follows the principle that the sample with the lowest energy consumption is treated as the most important one in the training. That means the samples with lower energy consumption contribute more to the EE-SVM regression function learning, which highly increases the energy efficiency of the biped walking. Simulation results demonstrate the effectiveness of the proposed method.

An unscented Kalman filter (UKF)-based predictable support vector regression (SVR) learning controller is proposed to improve the flexibility of biped walking robots. After estimating the biped states of the next moment using a UKF, an SVR learning controller with the predicted biped states is implemented to ensure the zero moment point (ZMP) stability. Using the predicted biped states, the SVR learning controller can predictably adjust the posture of the trunk timely and properly to adapt to the dynamic posture of the whole body. The flexibility of biped robots is enhanced by the proposed method, which is promising for realizing the stable biped walking in unstructured environments. Simulation and experimental results demonstrate the superiority of the proposed methods.

This study explores how human users respond to coordinated and uncoordinated gestures of a robot as an information deliverer. A between-subject experiment was conducted using the Wizard of Oz method, with 63 participants randomly assigned to one of four conditions (voice-only vs. no-gesture vs. coordinated gesture vs. uncoordinated gesture) taking an artwork class in a museum-like setting. The robot was explaining the information of the artworks with modalities accordingly designed to each condition. Results showed that the coordinated gesture was not aiding information delivery. However, there were notable relations between the coordinated gesture and intimacy, homogeneity, and involvement. These results have theoretical implications for cognitive load of working memory and practical implications for designing and deploying dynamic humanoid robots for museum tour guide.

In this paper, we present a new method for decoding pixel correspondences in structured light based 3D reconstruction, refer to here as Ray-Tracing codec. The key idea of Ray-Tracing codec is to correctly define the region boundaries in real number, for each layer of the Hierarchical Orthogonal Code (HOC) based on an accurate boundary estimator, and to inherit the correct region boundaries between layers sharing common boundaries. Furthermore, each region in lower layer is traced back to the upper layer for the correct correspondence between regions. This is an improvement over existing HOC decoding algorithms as the wrong decoded pixel correspondences can be greatly reduced. The experimental results have shown that the proposed Ray-Tracing codec significantly enhances the robustness and precision in depth imaging, compare with HOC and other well-known conventional approach. The proposed approach opens a greater feasibility of applying structured light based depth imaging to a 3D modeling of cluttered workspace for home service robots.

In real applications, sometimes, visual recognition may need to rely on incomplete or ambiguous features for a unique decision. Furthermore, the detected features may suffer a lot of uncertainties due to environment changes. In order to solve the problem with ambiguities and uncertainties in one computational framework, we propose a probabilistic 3D object recognition approach using both positive and negative evidences in cluttered environment. First of all, initial feature are selected as parallel and perpendicular line pairs to generate pose hypotheses as the multiple interpretations. Secondly, given a 3D polyhedral object model and the estimated pose, positive and negative evidences can be identified as additional information for probability computation of the multiple interpretations. More specifically, given the estimated pose, followed by visibility test, positive evidence is the feature that should be appeared around the pose, and negative evidence is the feature that should not be appeared due to self-occlusion. Where the probability is computed using Bayesian principle in terms of both likelihood and unlikelihood. The experimental results support the potential of the proposed approach in the real environment.

We present a novel plane extraction algorithm based on coplanar line pairs for poorly textured scenes. Coplanar line pairs are extracted from intersecting line pairs and matched between two views, and their correspondences act as geometric primitives for 3D planar patch extraction. Each correspondence generates a plane hypothesis by estimating planar homography using the compatibility constraint with the fundamental matrix. These candidates are merged into disjoint planar regions using a greedy graph-clustering algorithm. In a graph, each candidate consists of a node and is connected to the others by edges weighted by transfer errors of the corresponding convex hulls. Compared to previous approaches, we do not assume known camera geometry or scene structures such as piece-wise planarity or the presence of rectilinear structure. In addition, uncertainty modeling for coplanar line pairs is incorporated into matching and reconstruction for accurate computation of the transfer errors reflecting 3D plane geometry.

Successful recognition and pose estimation of logs and trees as well as workspace modeling in the forest environment is essential for extensive automation of the harvesting and logging tasks of forestry machines. However, the free form features of logs, few reliable textural features, large edge extraction errors, and segmentation faults caused by the barks on the surface of the logs present clear challenges for recognition and classification. To solve these problems, robust algorithms able to recognize and estimate poses of a variety of objects even under poor and partial inputs need to be developed. In this paper we focus on the most relevant task of recognizing and estimating postures of a bunch of logs located on the ground with varying orientation and distance. Experiments carried out with the help of a structured light camera demonstrate the feasibility of the proposed algorithm.

This talk introduces “HomeMate,” a cognitive consumer robot targeted for serving the elderly with such services as errands, medicine delivery and video chatting in a home environment. HomeMate features, in particular, cognitive recognition for interacting with humans as well as for searching for target objects to be delivered. The cognitive recognition developed for HomeMate is intended for achieving human-like dependability in interaction and recognition based on a series of self-generated cognitive behaviors. They include the selection of features optimal for the given visual environment, the proactive collection of evidences for reducing uncertainties and ambiguities in recognition, the probabilistic fusion of evidences for reaching a high confidence of decisions, the incorporation of context into probabilistic information fusion, the interactive conversation with humans for verifying facts or receiving assistance, etc. The details of implementation and experimentation of HomeMate will be presented with the demonstration of the power of cognitive recognition toward opening consumer market applications.

In this paper, we present a novel method based on clustering for identifying 3D line from point clouds, called “self-organizing fuzzy k-means algorithm”. The algorithm automatically finds the optimal number of cluster and self organizes the clusters based on inter/intra-cluster distances and cluster's performance evaluation. The self-organizing fuzzy k-means is applied in 3D line identification from point clouds. We use the point clouds provided by Stereo camera and 2D images. The 3D point clouds of each line is clustered by clustering algorithm, then we perform eigen-analysis on clusters and estimate the final 3D lines; the 3D lines can be cut off into several segments. In addition, to increase the accuracy of detection, the error evaluation is invoked to analyze the error of the 3D candidate lines. Our algorithm was evaluated on the real test scenes, which content noisy point clouds, and shows the high performance and robust results.

This paper presents a novel line matching method based on the intersection context of coplanar line pairs especially working in poorly textured indoor scenes. To overcome the matching ambiguity of single line segments, intersecting line pairs in 2D images are utilized for line matching. Coplanarity of the intersecting line pairs and their corresponding intersection context discriminate true intersecting line pairs from false intersecting ones in 3D world. Compared to previous approaches, the proposed method can match line segments and estimate camera geometry simultaneously not knowing camera geometry in advance or not considering topological relations of all line segments. Comparison studies and experimental results prove the accuracy and speed of the proposed method in real world situations.

This paper presents a novel method of wide-baseline image matching based on the intersection context of coplanar line pairs especially designed for dealing with poorly textured and/or non-planar structured scenes. The line matching in widely separated views is challenging because of large perspective distortion and the violation of the planarity assumption in local regions. To overcome the large perspective distortion, the local regions are normalized into the canonical frames by rectifying coplanar line pairs to be orthogonal. Also, the 3D interpretation of the intersection context of the coplanar line pairs helps to match the non-planar local regions by adjusting the region of interest of the canonical frame according to the different types of 3D non-planar structures. Compared to previous approaches, the proposed method offers efficient yet robust wide-baseline line matching performance under unreliable detection of end-points of line segments and poor line topologies or junction structures. Comparison studies and experimental results demonstrate the accuracy of the proposed method for various real world scenes.

In home service robot applications, robots often deal with textured objects. The variation of surface's reflectivity may decrease the accuracy of depth measurement of structured light 3D camera. In this paper, we propose a new method to correct the boundary of structured light pattern in order to increase the accuracy of depth measurement. The algorithm first estimates the reflection index of the surface. The light stripe is deconvoluted to remove the smooth effect of the camera lens. Finally, the edge is normalised by the reflection index, and the boundary is estimated from the intersection between the corrected versions of the edge and its inverse. The experimental results show that with the proposed boundary estimator, there are significant improvements in term of surface smoothness of textured objects.

This paper presents a method of light pattern blur estimation for its application to automatic projector focus control in structured light 3D camera. To overcome shortcoming of the conventional methods for estimating blur of light pattern, we propose to normalise the light pattern image incorporating reference images. Which are made available by a structured light 3D camera. The reference black and white patterns are utilised in such a way as to eliminate the dependence of intensity variation around an edge on surface reflectivity. Then the unknown blur radius is estimated from the first derivative of normalised edge. Experimental results on both synthetic and real images verify that the proposed method is indeed independent of surface reflectivity and is accurate and sensitive enough to the small change of blur. Finally, based on the proposed light pattern blur estimation, a structured light 3D camera that is equipped with automatic multi-stage focus control in a beam projector has been built to demonstrate its capability of capturing the 3D workspace of large depth variations.

Projector-camera system has been researched for years and it has become more and more popular in 3D vision and many other applications. However, with pre-determined position of projector's lens, projector based researches were not able to deal with the scenes that have objects with different distances from the projector. In this paper, we present a projector autofocusing technique based on local blur information of the image that can overcome above limitation. The algorithm is implemented on projector-camera system, in order to focus the pattern which is projected by projector on all objects in the scene sequentially. The proposed algorithm first obtains a blur-map of the scene on the image by using a robust local blur estimator, and then the region of interest is decided by thresholding the obtained blur-map. Since the main light source is provided by projector, the proposed auto-focusing algorithm achieves a good performance with different light conditions.

A Gray code and its variants are popular code-patterns for a structured light system. An n-bit Gray code is a kind of binary code whose adjacent code-strings differ only in one bit position. We introduce a specified Gray code, called an 'antipodal Gray code.' Since the n-bit antipodal Gray code has the additional property that the complement of any code-string appears exactly n steps away in the list, the spatial frequency (the width between the white and black stripes) of the antipodal Gray code-pattern is similar along frames. In this paper, we describe the limitations of structured light and the criteria for robust codes. We evaluate the original and antipodal Gray codes, and the experimental results show that the antipodal Gray code provides more robust and accurate results than original Gray codes.

A real-time solution for estimating and tracking the 3D pose of a rigid object is presented for image-based visual servo with natural landmarks. The many state-of-the-art technologies that are available for recognizing the 3D pose of an object in a natural setting are not suitable for real-time servo due to their time lags. This paper demonstrates that a real-time solution of 3D pose estimation become feasible by combining a fast tracker such as KLT [7] [8] with a method of determining the 3D coordinates of tracking points on an object at the time of SIFT based tracking point initiation, assuming that a 3D geometric model with SIFT description of an object is known a-priori. Keeping track of tracking points with KLT, removing the tracking point outliers automatically, and reinitiating the tracking points using SIFT once deteriorated, the 3D pose of an object can be estimated and tracked in real-time. This method can be applied to both mono and stereo camera based 3D pose estimation and tracking. The former guarantees higher frame rates with about 1 ms of local pose estimation, while the latter assures of more precise pose results but with about 16 ms of local pose estimation. The experimental investigations have shown the effectiveness of the proposed approach with real-time performance.

A sequence of images in multiple views rather than a single image from a single view is of great advantage for robust visual recognition and pose estimation of 3D objects in noisy and visually not-so-friendly environments (due to texture, occlusion, illumination, and camera pose). In this paper, we present a particle filter based probabilistic method for recognizing an object and estimating its pose based on a sequence of images, where the probability distribution of object pose in 3D space is represented by particles. The particles are updated by consecutive observations in a sequence of images and are converged to a single pose. The proposed method allows an easy integration of multiple evidences such photometric and geometric features as SIFT, color, 3D line, 2D square, etc. The integration of multiple evidences, including photometric and geometric features, in space and time makes the proposed method robust to various not-so-friendly visual environments. The experimental results with a single stereo camera show the validity of the proposed method in an environment containing both textured and texture-less objects.

In this paper, we present a novel approach for sensor-guided robotic execution of everyday tasks, which is amenable to be integrated in current mobile manipulators and humanoid robots. We consider a robot which is observing simultaneously his hand and the object to manipulate, by using an external camera (i.e. robot head). Task-oriented grasping algorithms are used in order to plan a suitable grasp on the object according to the task to perform. A new vision/force coupling approach [1] is used in order to, first, guide the robot hand towards the grasp position and, second, perform the task taking into account external forces. Experimental results on a real robot are presented which validate our approach.

We present two robust and novel algorithms to model a 3D environment using both intensity and range data provided by an off-the-shelf stereo camera. The main issue we need to address is that the output of the stereo system is both sparse and noisy. To overcome this limitation,, we detect planar patches in the environment by region segmentation in 2D and plane extraction in 3D. The extracted planar patches are used not only to represent the workspace, but also to fill holes in range data. We also suggest a new planar patch based scan matching algorithm to register multiple views, and to incrementally augment the description of the 3D workspace in a sequence of scenes. Experimental results on real data show that planar patch segmentation and 3D scene registration for environment modeling can be robustly achieved by the proposed approaches.

In recent years, a lot of companies and laboratories start to announce their various prototype service robots. It does not exist that a killer application of a service robot in a service robot market yet, however it is expected that customers demand a robot having more variety peculiarity when a robot is supplied widely. In order words, in the future, robot users probably want the robot which is fitted for their individual tastes. In the present paper we shall see the robot's variation of behavioral personality from the experiments. These results show that by adjusting the parameters related to robot perception, the robot can be varied in its personality for user. The robot is implemented by "cognitive robotic engine (CRE)" for the dependable integration of human-robot interaction (HRI) components.

We present a real-time 3D camera based on IR (infrared) structured light suitable for robots working in home environment. First, we implemented a HOC (hierarchical orthogonal coding) based FPGA board. The HOC gives robust depth images because the signal separation coding provides not only the separation of overlapped codes, but also a robust decision on pixel correspondence with error correction. The FPGA module can handle high computational cost of HOC based signal separation coding. Second, we implemented a compact optic system of the camera to project and receive IR structured light. The invisible IR pattern light provides users inconvenient in the home environment. Various objects and workspaces which has continuous and/or non-continuous surface are tested and, sensitivity to illumination change and processing time are analyzed. The experiment results show the robust performance for surface smoothness, color, and materials of objects used in home environment. The proposed approach opens a greater feasibility of applying structured light based depth imaging to a 3D modeling of cluttered workspace for home service robots

For robust 3D object recognition in the environment having diverse variances, it is necessary to increase the certainty by using consecutive scenes rather than using a single scene and combining different features. This paper proposes a novel 3D object recognition and pose estimation approach based on combining photometric feature (SIFT) and geometric feature (3D lines) in a sequence of scenes. In order to utilize the consecutive scenes, we use the particle filtering method and all particles which represent the possible pose of object are generated by each feature. These particles are to be spread out where the object is considered to exist, and the probability of each particle is obtained through matching test with each feature in the scene. Then the particle sets derived from SIFT and 3D lines are fused and it gives a pose of the object estimated. For the sake of computational efficiency, this recognition system is performed in a hierarchical process. In this paper, we also introduce a simple method to decide the next best view position based on results of recognition. Lastly we have proved through experiments that the proposed methods are feasible

Presented in this paper is an approach to realtime spatial reasoning for manipulative robotic tasks. When a service robot is requested to manipulate an object, it should determine the directions along which it can access and remove the object. The potential accessible directions for the object are retrieved from the object database. Then, spatial reasoning with the surrounding environment and the gripper geometry is invoked to verify the directions. The verification process mainly utilizes the visibility test of the commodity graphics hardware. Then, the directions along which both of the object and gripper are translated without colliding with the surrounding obstacles are computed using Minkowski sum and cube map of the graphics hardware. The access and removal directions are passed to the potential field path planning algorithm to determine the robot arm's full path for accessing, removing and delivering the object. The experimental results show the feasibility of using graphics hardware for manipulative robotic tasks and further its performance gain in real-time manipulation.

A "cognitive robotic engine (CRE)" that generates perceptual and action behaviors to select and collect an optimal set of evidences has been introduced previously by the authors. CRE aims at enabling a robot to be capable of dependable and robust recognition and decision under a high level of uncertainty and ambiguity in perception. This paper improves the performance of CRE based on the following expansion: 1) the provision of an evidence structure separately from the internal perceptual processes represented by a precedence graph, such that the contribution of individual evidences to the certainty of the premise pertaining to the given robotic mission can be more clearly defined, and 2) the establishment of a search process for action behaviors based on the overall contribution of the chosen action behaviors to the certainty of the premise pertaining to the given robotic mission. CRE is applied to the two robotic missions, caller identification and caller following, and is evaluated by actual experimentation. The experimental results show the expanded version of CRE results in improvement not only in dependability but also in stability

This paper presents an effective and real-time approach for detecting walls in indoor environment. This approach relies on the fact that the rear of the opaque walls is not visible. Thus, to detect the walls in an indoor environment a set of hypothetical walls, based on the ceiling edges or ground level edges, are considered; and their validity is checked using point cloud, generated by a sensor. A certainty factor is calculated for each detected wall, which is updated continuously based on the newly gathered sensory information. Furthermore, the certainty of the walls can be updated using other source of information for better and more reliable wall detection. The novelty of this approach is in its capability to handle environments, with texture-less walls, in real-time. The algorithm has been implemented in simulation, and tested in real environment and has shown effective, reliable and real-time performance

An approach to identifying a caller or callers by a service robot is presented for a natural interaction with people in a home/office environment. The problem addressed specifically in this paper is how to successfully identify a caller in a cluttered environment with large uncertainties involved in the sensed audio-visual cues. The proposed approach is based on a proposition that the dependability of perceptual recognition may come unlikely from "the effort to make individual sensing perfect", but likely from "the effort to self-generate perceptual behaviors of integrating individual sensing that lead to mission accomplishment, no matter how imperfect and uncertain individual sensing may be". We implement the above proposition in terms of a novel robotic architecture, referred to here as "cognitive robotic engine (CRE)." CRE implemented for the case of a robot identifying a caller in a crowded and noisy environment, including its experimental results, are shown

There has shown a significant interest in a high performance of, at the same time, a compact size and low cost of, 3D sensor, in reflection of a growing need of 3D environmental sensing for service robotics. One of the important requirements associated with such a 3D sensor is that sensing does not irritate or disturb human in any way while working in close and continuous contact with human. Furthermore, such a 3D sensor should be reliable and robust to the change of environmental illumination as service robots are required to work day and night. This paper presents a 3D IR camera with variable structured light that is human friendly and robust enough for application to home service robots. Infrared is chosen as the sensing medium in order to meet the requirement of human friendliness and robustness to illumination change. A Digital Mirror Device (DMD) is employed to generate and project variable patterns at a high speed for real-time operation. In implementation, we emphasize the integration of modular components to support real-time sensing and compactness in size. A number of real-world experimentations are conducted, including a human face, a statue, and a plastic model. The experimental results have demonstrated that the implemented 3D IR Camera is robust to illumination change, in addition to its advantage of human friendliness.

This paper presents an original approach to coding the light patterns for robust depth imaging based on structured light. We have discovered that the degradation of precision and robustness, seen in most conventional approaches to structured light, comes mainly from the overlapping of multiple codes in the signal received at a camera pixel, where the overlapped codes are from the neighbouring and/or, even, distant pixels of the projecting mirror array. Considering the criticality of separating the overlapped codes to precision and robustness, we propose a novel signal separation code, referred to here as “Hierarchical Orthogonal Code (HOC),” for depth imaging. HOC provides not only the separation of overlapped codes, but also a robust decision on pixel correspondence with error correction based on a contextual likelihood among the sets of separated codes from neighbouring camera pixels. The experimental results have shown that the proposed HOC significantly enhances the robustness and precision in depth imaging, compared to the best known conventional approaches. The proposed approach opens a greater feasibility of applying structured light based depth imaging to a 3D modelling of cluttered workspace for home service robots.

This paper presents a novel approach to accessibility analysis for manipulative robotic tasks. The workspace is captured using a stereo camera, and heterogeneously modeled with the recognized plane features, recognized objects with complete solid models, and unrecognized 3D point clouds organized with a multi-resolution octree. When the service robot is requested to manipulate a recognized object, the local accessibility information for the object is retrieved from the object database. Then, accessibility analysis is done to verify the object accessibility and determine the global accessibility. The verification process utilizes the visibility query, which is accelerated by graphics hardware. The experimental results show the feasibility of real-time and behavior-oriented 3D modeling of workspace for robotic manipulative tasks, and also the performance gain of the hardware-accelerated accessibility analysis obtained using the commodity graphics card.

This paper presents a novel approach to real-time 3D modeling of workspace for manipulative robotic tasks. First, we establish the three fundamental principles that human uses for modeling and interacting with environment. These principles have led to the development of an integrated approach to real-time 3D modeling, as follows: 1) It starts with a rapid but approximate characterization of the geometric configuration of workspace by identifying global plane features. 2) It quickly recognizes known objects in workspace and replaces them by their models in database based on in-situ registration. 3) It models the geometric details on the fly adaptively to the need of the given task based on a multi-resolution octree representation. SIFT features with their 3D position data, referred to here as stereo-sis SIFT, are used extensively, together with point clouds, for fast extraction of global plane features, for fast recognition of objects, for fast registration of scenes, as well as for overcoming incomplete and noisy nature of point clouds. The experimental results show the feasibility of real-time and behavior-oriented 3D modeling of workspace for robotic manipulative tasks.

A robot soccer match poses an interesting problem in planning: the planning of coordinated motion of individual players as a team, against the opponent players the next move of which is not known a priori, in such a way as to achieve the common goal, i.e. winning the match. The outcome of a soccer match depends largely on how well the motions of individual players of a team are coordinated against those of the opponent. The technical challenge lies in formulating such coordination strategy together with computational formalism that allows a real-time search for an optimal coordinated motion of individual players. In this paper, we propose the concept of the "controllable zone" of a player or a team, representing the area or the set of field positions that the player or the team can occupy before the opponents at time t. A systematic way of computing various forms of controllable zone is presented. Based on the formalism of controllable zone, it is possible to identify whether a team should be in an offensive or a defensive mode, and which strategy among shooting, capturing, dribbling, and passing is desirable for the team at time t. The coordinated motions of individual players are then determined in such a way as to optimize the strategic scores associated with controllable zones. The proposed paradigm is applied to a real environment for animated simulation: 3/spl times/3 FIRA World Cup micro robot soccer match, to evaluate its performance.

Reports the discovery of a new type of singularity that is as significant as other singularities like kinematic singularity, algorithmic singularity, and semi-singularity. This singularity, named semi algorithmic singularity (SAS), occurs when a redundant manipulator, faced with inequality constraints arising from kinematic limits such as joint angle limits and obstacles, proceeds to conduct optimization of a performance measure. Through mathematical analysis, we have proved its existence and proposed an analytic matrix function to identify its presence. In conjunction, we have exposed its relationship with the other singularities above. More specifically, SAS is similar to the algorithmic singularity in that it is an artificial singularity coming from an endeavor for performance optimization. They are different in that the former occurs in one direction in the C-space, whereas the latter occurs bidirectionally. Besides, we have also found an analogy in the real singularities that while the semi-singularity is unidirectional in the workspace, the kinematic singularity is bi-directional. Through a simulation for a three DOF planar manipulator we have visualized the existence of SAS and its relationship with the other singularities.

The design and fabrication of a piezoelectric microactuator on a silicon diaphragm is presented for application of micro ink jet. We developed the micro-devices for the fluid transference that are fabricated using thick film technologies for ceramic materials and Si micromachining. The performance of droplet ejection is shown from the optimal design parameters. It is closely related to the precise machining of flow channel structures and the piezoelectric actuator capacity. The features of micro-fabrication and ink drops are described.

This paper describes a vision based robust lane keeping system which warns the driver of inattentively drifting off the lane in such a way as to keep the vehicle in the lane. The overview of the lane keeping system mounted on the automobile is presented. A novel sensor fusion algorithm is proposed to estimate the vehicle pose and lane geometry data so as to detect an unintended lane departure. The performance criteria of decision making strategies of the lane keeping system are proposed as: 1) false alarm rate, and 2) alarm triggering time. Two heuristic decision making strategies of the lane keeping system are provided with experimental results evaluating the performance of each strategy.

We propose a geometric data fusion (GDF) method using a perception-net which can provide error reducing, uncertainty management, and maintaining consistency. We propose a perception-net to design a state estimator for dynamic systems and apply the proposed geometric data fusion method to obtain the optimal estimate, propagate uncertainties and utilize the system knowledge. We present comparisons between the proposed estimator and the conventional estimators. It is also shown that the additional priori information on the system can be easily utilized in the proposed estimator to improve the performance. Through illustrative examples, it is verified that the proposed estimator presents better performances than existing filters and improves performances via utilizing system knowledge.

An extension to operational space (EXOS) is presented for the explicit representation of the null-space (NS) dynamics and its interaction with the operational-space dynamics. First, the EXOS Jacobian is formed by augmenting the Jacobian matrix with a minimum number of its NS vectors. Based on the EXOS Jacobian, free of algorithmic singularity, the kinematics, statics, and dynamics of a redundant manipulator are derived in a compact form. In particular, the resulting EXOS dynamics is able to identify the inner dynamic structure. Its efficacy and efficiency have been demonstrated through comparative analysis and simulation.

Describes two decision making strategies of the lane departure warning system for a vehicle and provides experimental results to assess the performance of each strategy. The overview of the lane departure warning system mounted on the vehicle is presented. A lateral offset (LO) based strategy and a time to lane crossing (TLC) based strategy are proposed to be used in the warning module which detects unintended lane departure so as to warn the driver. The performance criteria of the warning strategies are defined as: 1) false alarm rate, 2) alarm triggering time. The proposed strategies are incorporated in our lane departure warning system to be tested in real expressway experiments. The experimental results for the performance optimization of both strategies are discussed and compared with each other.

Proposed is an algorithm to estimate vehicle pose and road curvature by geometrically fusing sensor data from camera image, velocity meter, and steering wheel encoder. To achieve computational efficiency in processing in a real time sequence, we propose a method to model the lane on the road as a series of connected rectangular plates. We propose an algorithm, the so called "Perception-Net", where not only variables denoting the vehicle pose and the road curvature, but also the corresponding uncertainties are propagated in forward and backward directions in such a way to satisfy the given constraint condition, maintain consistency, reduce the uncertainties, and guarantee robustness. An experimental result is also presented.

The design of an assembly line for a mechanical assembly has to satisfy all the geometrical and the physical constraints presented in the assembly such as blocking relation and contact forces. Given a desired final assembly, determining an appropriate assembly sequence which satisfies not only the blocking constraints but also the force constraints needs a proper model to directly provide the information needed to reason about assembly. We introduce a representation, the directional force graph, which explicitly describes how parts interact in the assembly. This representation is used to determine disassemblable parts that are naturally broken apart from an assembly by a decomposition force. A decomposition force is an external force applied to a held part(s) against a fixture part(s). The proposed approach provides a mechanism to explicitly incorporate the real-world constraints of assembly directions, fixturing, and tooling requirements into planning. We describe a complete planning algorithm and report the results.

A systematic method for reducing uncertainties and calibrating biases involved in sensing systems is presented. The method is based on representing a sensing system by a perception net and applying a geometric method for combining multiple data and constraints. The proposed method is applied to the self-calibration of the eye-hand system equipped for a JPL/NASA planetary rover. Experimental results are shown.

Proposes a vision based lane departure warning system which warns the driver of drifting off the lane inattentively. An estimation algorithm is proposed to obtain the vehicle pose and lane geometry data so as to detect an unintended lane departure. The lane departure alarm is triggered by a warning algorithm which makes decisions on giving an alarm based on the data from the estimation algorithm. The performance criteria of the lane departure warning system are defined as false alarm rate and alarm triggering time. The system is implemented in a test automobile and tested in real expressway experiments optimizing the performance. The experimental results demonstrate the applicability of the proposed system.

The emergence of micro sensors and actuators is pushing forward the revolution of intelligent systems technology. Intelligent systems technology allows a dramatic change in the way that machines, tools, and systems serve humans. This paper reviews recent R&D and commercial status of inertial sensors using silicon micromachining technologies and discusses the requirement for commercialization of the research results.

The problem of two teams of micro-robots playing a soccer game constitutes an excellent testbed where various aspects of machine intelligence such as visual perception, path-planning, behavior coordination as well as learning need to be integrated. The paper describes the set of algorithms that govern the behavior, of a team of three microrobots playing a soccer game under MIROSOT regulations. These algorithms are structured in three levels: strategy, path-planning and motor control. At the strategy level the motions of individual team players are coordinated against those of the opponent. This is achieved based on the novel mathematical concept of 'controllable zone'. The arrangement of the players is optimized in such a way as to maximize the probability of possessing and shooting the ball. This concept leads to a formal mathematical framework to represent behavior-coordinated strategies for shooting, capturing and passing the ball. At the path-planning level a pseudo-optimal path is generated in real time to minimize the time required by the player to perform the desired behavior and the probability to bump into an obstacle. Finally, the motor control level computes the necessary motor commands to keep the robot on the right track. Experimental results of the system are shown.

The objective of this research is to monitor the vehicle motion against its external and internal constraints and to remove the existing safety risk, if any, based on the human-machine cooperative vehicle control. We begin by solving the problem of automatic lane following of a vehicle, where a vision system with a single camera installed inside the vehicle is used for identifying the vehicle pose with reference to the lane that the vehicle is following. To perform this task, the road with lanes was modeled by three connected plates. The perception net based method is proposed to identify lane and to estimate vehicle pose under all weather and lighting conditions as well as variations of the road conditions. As a result, the lane to follow under large noise as well as variants embedded in the street images can be identified. The 3D geometric relationship between the lane and the vehicle is also determined.

An optimal projection of stereo images onto an HMD screen is presented, such that the viewer can observe a real-world object or local scene of his/her interest in 3D, with the feeling of desired depth and scaling, under the minimum distortion. Taking the object or local scene of viewer's focal point as that of his/her interest, the proposed 3D viewing is intended to march the characteristics of human vision. The optimal projection is done by determining the scaling factor and horizontal displacement of the original stereo images as a function of the convergence angle of the stereo camera. In this paper, we put an emphasis on the verification of the proposed optimal 3D viewing method through experimentation.

The assemblability, which is the probability of successfully assembling the product parts, is directly related to the product cost. Especially, we consider the assembly failures due to tolerances. Because clearances provide the adjustability to part poses in an assembly, the assemblability analysis must be analyzed not only in terms of tolerances but also clearances. In this paper, we propose an analytic approach to the assemblability analysis. Represented by ellipsoids in a deviation space, the propagations of tolerances and clearances are calculated using Jacobian and sweep operations. The simulation results are given using a 2D assembly. The expected contribution of this paper is that the proposed method allows to compute the assemblability of a product in real time.

A new paradigm of event-based robotic and telerobotic control, referred to as the reference adaptive impedance control, is presented for partially constrained robotic and telerobotic tasks under uncertain or unknown environmental constraints. The reference adaptive impedance control determines the desired next state adaptively to follow the optimal path from the current state, where the optimal path is generated with the environmental constraints updated through time. The embedded impedance control provides a mechanism for collecting more complete information on the environment. The reference adaptive impedance control also serves as a powerful tool for implementing shared control in teleoperation. A case study of robotic and telerobotic soil trenching is given with peg-hole insertion experiment.

We present an augmented space method to solve the assemblability of products where tolerances may prevent parts assembling successfully. The accumulated tolerances of two serial chains of the parts are modeled by a hyper-ellipsoid in an augmented space. By the same token, the accumulated nominal clearances of two serial chains are modeled by a hyper-ellipsoid. Complete algorithms for solving the assemblability, tolerance, and clearance of parallel chains in the augmented space are described. This augmented space method gives a great computational advantage over simulation based methods, which is a necessary requirement for the real time assemblability analysis. Simulation results are given.

In this paper, jump impact controller and observer derived previously for the manipulator collision control problem under the conditions that there are force sensor delay and uncertainties in the environment dynamics and location is modified with better intuitive appeal and understanding. It turns out that the solution is a combination of integral action and gain scheduling which provide a valuable design freedom or tool to achieve the desired robustness and performance in accordance with the uncertainties involved. An example that demonstrate this design process with simulation results are presented.

A new method of constructing 3D maps based on the relative magnification and blurring of a pair of images is presented, where the images are taken at two camera positions of a small displacement. The method, referred to as depth from magnification and blurring, aims at generating a precise 3D map of local scenes or objects to be manipulated by a robot arm with a hand-eye camera. The method uses a single standard camera with telecentric lens, and assumes neither active illumination nor active control of camera parameters. The proposed depth extraction algorithm is simple in computation. Fusing the two disparate sources of depth information, magnification and blurring, the proposed method provides more accurate and robust depth estimation. Experimental results demonstrate the effectiveness of the proposed method.

We present an efficient path planning approach which represents a path by a series of via points connected by elastic strings that are subject to displacement due to collisions with obstacles as well as constraints pertaining to the domain to which path planning is applied. Obstacle regions are represented by a potential field created by a multilayered neural network. A fast simulated annealing approach is used for local minima problems from the potential field. The automatic generation and removal of via points is incorporated in the path planning approach to ensure collision-free planning regardless of the complexity of the environment (e.g., convex, concave or complicated obstacle regions). Our path planning approach is flexible due to the automatic generation and removal of via points based on the complexity of the domain of the path planning process, efficient due to the use of the necessary via points for the path representation at all times, and massively parallel due to the parallel computation of motions of via points with only local information.

In this paper a method for collision detection and collision avoidance for dual arm robots is proposed. The collision detection, a fast geometric approach based on link intersection, detects the collision and constructs the collision region for given trajectories. The time delay planning determines the minimum necessary time delay to avoid collisions using a potential field representation for collision regions. The planning is based on a flexible, massively parallel approach. It effectively avoids any local minima problems due to the potential field representation and can be used regardless of the complexity of the collision region (e.g., concave or complicated regions).

The capability of robotic systems to deal with uncertainties, biases, and errors automatically is crucial for the tasks defined in an unstructured environment. In this paper, we present a method of automatically reducing uncertainties and calibrating possible biases involved in sensed data and extracted features by a system based on the geometric data fusion. The perception net, as a structural representation of the sensing capabilities of a system, connects features of various levels of abstraction, referred to here as logical sensors, with their functional relationships such as feature transformations, data fusions, and constraints to be satisfied. The net maintains the consistency of logical sensors based on the forward propagation of uncertainties as well as the backward propagation of constraint errors. A novel geometric data fusion algorithm is presented as a unified framework for computing forward and backward propagations through which the net achieves the self-reduction of uncertainties and self-identification of biases. The effectiveness of the proposed method is validated through simulation by applying it to a mobile robot self-localization problem.

We present an approach for evaluating the assemblability of a product based on tolerances and adjustable displacements. An adjustable displacement is a functionally permitted free space between two mating features, which can be used for compensating tolerances. The assemblability of a product is computed in term of a multi chain by incrementally solving the parallel chains. We consider both the functionality and assembly sequence constraint in the evaluation. The result of the computation is a statistical measure of the product assemblability which can be used by the designer to evaluate and to optimize the tolerance allocation. In addition, this measure can be used for evaluating assembly sequences. The algorithms and simulation results are given.

A statistical measure of the cost of assembling two mating features is presented. It is based on how well the geometry of the task compensates, during the assembly, the uncertainties associated with the mating features: manufacturing tolerances and the pose uncertainty. The latter is due either to the accumulation of tolerances and clearances between mating features that are previously assembled or to the positioning errors of an assembly robot. The proposed measure allows a systematic evaluation of the assembly cost for a given assembly sequence, thus providing a method for evaluating assembly sequences and selecting an optimal one from the assembly point of view. The influence of different sources of uncertainties and the assembly procedure can also be evaluated with the proposed cost index. The paper includes an example to illustrate the use of the proposed cost measure.

A new method of extracting depth from blurring and magnification of objects or local scene is presented. Assuming no active illumination, the images are taken at two camera positions of a small displacement, using a single standard camera with telecentric lens. Thus, the depth extraction method is simple in structure and efficient in computation. Fusing the two disparate sources of depth information, magnification and blurring, the proposed method provides more accurate and robust depth estimation. This paper describes the process of various experimentations performed to validate this concept and describes the present work that has been done in that field. The experimental result shows less than 1% error for an optimal depth range. The ultimate aim of this concept would be the construction of dense 3D maps of objects and real time continuous estimation of depth.

A method of automatically reducing uncertainties and calibrating possible biases involved in sensed data and extracted features by a system based on the geometric data fusion is proposed. The perception net, as a structural representation of the sensing capabilities of a system, connects features of various levels of abstraction, referred to here as logical sensors, with their functional relationships such as feature transformations, data fusions, and constraints to be satisfied. The net maintains the consistency of logical sensors based on the forward propagation of uncertainties as well as the backward propagation of constraint errors. A novel geometric data fusion algorithm is presented as a unified framework for computing forward and backward propagations through which the net achieves the self-reduction of uncertainties and self-calibration of biases. The effectiveness of the proposed method is validated through simulation.

A new paradigm of event-based robotic control, called reference adaptive impedance control, is presented for partially constrained robots under uncertain or unknown environmental constraints. The reference adaptive impedance control determines the desired next state adaptively in such a way as to follow the optimal path from the current state, where the optimal path is generated with the environmental constraints updated through time. The embedded impedance control provides a mechanism for collecting more complete information on the environment. An application to obstacle avoidance trajectory planning is given with computer simulations.

An assembly task, whether robotic or manual, may involve adjusting the positions of already assembled parts when a new part is assembled, due to tolerance and position uncertainties of the parts. Such an adjusting operation can increase the cost of the product because of the adjusting time, fixturing costs, additional operations, etc. In this paper, a statistical approach to evaluate the cost of assembly is proposed. The cost is measured according to the number of objects that must be moved in order to successfully assemble the product. The approach uses a statistical analysis of the accumulated tolerances, the local clearances, and the possible adjustable zones due to clearances between the already assembled parts, to calculate the minimum number of objects that must be adjusted to complete the assembly. Special attention is given to the assembly of parallel chains. The complete procedure and simulation results are given.

A new impact/force controller known as jump impact controller (JIC) is designed and evaluated experimentally. JIC is derived by representing dynamics of collision in terms of a jump linear system where the jumps of system dynamics can be described by a state-dependent stochastic process. Thus, JIC provides robustness to the uncertainties in the environment dynamics as well as the location of the collision surface. The experimental results not only verify the robustness of JIC but also help explain and interpret how robustness is associated with the JIC theory in terms of such known concepts as approach velocity and control system bandwidth.

In this paper, we present a method for representing and discovering skills by a robot based on observed data. First, the capabilities of a robot to transform a situation from one to another based on available control actions are extracted from the data, and represented in the situation-action space as a feasible situation transition manifold (FSTM). The multi-resolution globally competitive and locally cooperative algorithm is formulated to self-organize the FSTM in terms of an union of hyper-ellipsoidal subregions in various sizes and shapes. An optimal sequence of transitions from the initial to the goal situations is searched for as a skill, under the constraint imposed by the FSTM. The search of an optimal sequence is based on the novel bidirectional dynamic path planning algorithm formulated based on the potential field method. The proposed methodology is applied for the discovery of a nonholonomic motion planning skill of a car-like robot and for a telemanipulation skill.

We propose that the manipulator impact control problem be approached from a stochastic optimal control perspective. The reason is that not only is such an approach able to model uncertainties in contact environment, force sensing, as well as manipulator dynamics, the controller obtained is optimally robust in terms of performance. This result is verified by analyses and simulations.

A method of optimal 3D viewing based on adaptive displays of stereo images is presented for advanced telemanipulation. The method provides the viewer with the capability of accurately observing a virtual 3D object or local scene of his/her choice with minimum distortion. The viewer is allowed to define a virtual 3D object or local scene at a desired depth as a scaled version of a real 3D object or local scene which he/she wants to focus on. The key result is an algorithm which determines in real-time the optimal parameter values associated with stereo imaging and image projection on a video screen. The selected parameter values implement the optimal stereo viewing adapted to the viewer's focus of attention. Simulation results are shown.

In this paper, we present statistical tolerance and clearance analysis for the assembly. Tolerances work against the assemblability of a product since they can propagate and accumulate in the product, making it more difficult or impossible to assemble. Clearances work for the assemblability since they can be used to compensate for tolerances. The poses of parts in an assemble may be adjusted within the permitted clearances to assemble the parts. Monte-Carlo method is used in the analysis, with Gaussian distribution, Gaussian-Sigmoid distribution, and Chi-Square error reduction scheme to approximate tolerances and clearances. Then, algorithms to compute the propagation of tolerances and clearances are proposed. Our proposed work is expected to make the following contributions: 1) to help the designers to evaluate products for assemblability, 2) to provide a new perspective to tolerance problems, and 3) to provide a tolerance analysis tool which can be incorporated into a CAD or solid modeling system.

This paper presents a new approach to representation and recognition of handwritten numerals. The approach first transforms a two-dimensional (2-D) spatial representation of a numeral into a three-dimensional (3-D) spatio-temporal representation by identifying the tracing sequence based on a set of heuristic rules acting as transformation operators. A multiresolution critical-point segmentation method is then proposed to extract local feature points, at varying degrees of scale and coarseness. A new neural network architecture, referred to as radial-basis competitive and cooperative network (RCCN), is presented especially for handwritten numeral recognition. RCCN is a globally competitive and locally cooperative network with the capability of self-organizing hidden units to progressively achieve desired network performance, and functions as a universal approximator of arbitrary input-output mappings. Three types of RCCNs are explored: input-space RCCN (IRCCN), output-space RCCN (ORCCN), and bidirectional RCCN (BRCCN). Experiments against handwritten zip code numerals acquired by the U.S. Postal Service indicated that the proposed method is robust in terms of variations, deformations, transformations, and corruption, achieving about 97% recognition rate.

The main objective of assembly planning is to determine a sequence of assembling a product with respect to its geometric and resource constraints. Recent strides toward concurrent engineering have called for a tighter integration of assembly planning with design, because often during assembly planning, one realizes that a significant amount of assembly cost can be cut by redesigning the product itself. Instead of "retroactively" redesigning a product upon assembly planning, a concurrent engineering platform must allow its users to evaluate designs, and "proactively" explore different alternatives at various levels of abstractions so that one can zoom in on "promising" design paths and design right within the first few design cycles. This paper presents such an integrated system in which assembly planning becomes an essential part of the design exploration process. An algorithm for assembly planning during design for such an environment is developed. Designing for assembly is more effective and correct because, for a particular sequence of assembly, different design alternatives can be created and evaluated.

A novel optical proximity sensor system, HexEYE, capable of measuring the distance to, the orientation of, and the curvature at a local area of object surface has been designed and evaluated for robotic application. This paper presents: 1) the implementation of distance mapping from raw data based on a multilayer backpropagation neural network; 2) the hardware and software implementation of a single sensing unit; and 3) the experimental evaluation under various surface conditions. A prototype of a single sensing unit has been built with commercially available parts.

The specification of tolerances is an integral part of product design since tolerances directly affect the assemblability, as well as the functionality, manufacturability, and cost-effectiveness of the product. However, there has been little research on the assemblability evaluation based on tolerance propagation which can play an important role in assembly planning. Assembly planners in the past have assumed nominal dimensions for product components for generating the feasible assembly sequences. These sequences, however, may not be assemblable in practice due to propagations and accumulations of tolerances. In this paper, we present a new approach to assemblability evaluation based on tolerance propagation. For the assemblability evaluation, we introduce an important concept of clearance. Clearance represents the possible free space between two mating features. In the assemblability evaluation, clearances are used to compensate for tolerances.

A new paradigm of sensor planning is presented based on a hierarchically distributed perception net (HDPN) proposed as a general sensing architecture. In the proposed parametric sensor planning, the uncertainties are propagated in HDPN, and the sensing parameters of HDPN are iteratively modified so that HDPN ultimately generates the desired accuracy of outputs at a minimum sensing cost. The structural sensor planning aims at self-organizing an optimal configuration of HDPN by exploiting redundant sensing. Simulation study and experiment are conducted by applying the proposed parametric sensor planning method for the accurate self-localization of a mobile robot operating in a known environment with multiple range sensors. The proposed paradigm provides a formal, yet general and efficient method of representing and solving a sensor planning problem for an integrated sensor system.

The topology of self-motion manifolds for redundant manipulators with kinematic constraints is discussed. The semi-singularity is a fundamental concept for this analysis, and in this paper we present its analytical properties. A space discretization scheme is proposed, which is driven by the self-motion topology and embeds the kinematic map in its representation. A 3-DOF planar manipulator illustrates the methodology.

This paper presents a novel impact controller which is robust to the uncertainties in environment dynamics and location of the collision surface. By modeling the impact dynamics as a state dependent jump linear system and applying a modified version of the stochastic maximum principle of a state dependent jump linear system, the controller thus obtained optimizes, in the mean square sense, the approach velocity, the force transient during impact and the steady state force error after contact is established. Compared with impedance controllers via simulation whose data is obtained from experimental data the results indicate that not only is the performance of the jump impact controller superior in terms of overshoot and steady state error its robustness is quite remarkable too.

In this paper we present a statistical approach to assemblability evaluation based on tolerance and clearance propagations. Tolerance affect assemblability of an assembly sequence, however, clearance can compensate for tolerance. Assembly planners in the past have assumed nominal dimensions for product components for generating feasible assembly sequences. These sequences, however, may not be assemblable in practice due to propagations and accumulations of tolerances. With this concept of clearance, approaches to compute statistical tolerance and clearance propagations for serial, parallel, and multi-loop chains are proposed. The proposed method can be a valuable tool for CAD/CAM since early detection of assembly problems can significantly reduce the product cost.

A new paradigm of sensor planning is presented based on a hierarchically distributed perception net (HDPN) proposed as a general sensing architecture. In the proposed parametric sensor planning, the uncertainties are propagated in HDPN, and the sensing parameters of HDPN are iteratively modified so that HDPN ultimately generates the desired accuracy of outputs at a minimum sensing cost. An experiment is conducted by applying the proposed parametric sensor planning method for the accurate self-localization of a mobile robot operating in a known environment. The proposed paradigm provides a formal, yet general and efficient method of representing and solving a sensor planning problem for an integrated sensor system.

This paper presents a new approach for representing and registering skills based on state transition graphs. First, we generate clusters of feasible state transitions from the experimental data such that the union of these clusters represents the feasible state transition region of the system. Globally competitive and locally cooperative (GCLC) algorithm is formulated for self-organizing clusters and state transitions accurately through interpolation. A state transition graph is then constructed by representing possible transitions between clusters. Skills are considered embedded in this state transition graph and can be extracted by searching for an optimal path from the current to the goal states. The reactive nature of skills can be implemented by forming priority indices for sibling branches for local decision with possible merging and splitting of clusters to ensure consistency in prioritization. Experimental results are shown.>

This paper presents an in-depth analysis on the kinematic features of parallel manipulators, such as deficiency, singularity, and redundancy. Parallel manipulators under consideration are of a general form, with no restriction on their structure. Three orthogonal instantaneous motion spaces (IMSs) of a parallel manipulator are identified including deficient, uncontrollable, and controllable IMSs. Based on the IMSs, the deficiency and singularity of a parallel manipulator are devised and analyzed and their remedies are devised. Three types of redundancies of a parallel manipulator are introduced including the redundancies in serial chain, joint actuation, and parallelism. The three types of redundancies are shown to overcome the singularity of a parallel manipulator, as well as to improve either its dexterity or load capacity.>

A new method for designing an optimal time-delayed teleoperator control system based on Smith's principle is presented. First, an optimal delay-free teleoperator control system is designed with the human dynamics of visual and force trackings in the control loop and with the concept of telemonitoring force feedback. Then, an optimal time-delayed teleoperator control system is obtained by applying Smith's principle to the designed delay-free teleoperator control system and by modifying system parameters to achieve robustness to modeling errors. The proposed method provides an analytic design of a closed loop control structure for conventional teleprogramming facility and of a visual and kinesthetic man-machine interface required for time-delayed teleoperation. Simulation results are shown.>

This paper explores the intrinsic self-motion manifold topology structure in the presence of kinematic constraints. The objective is to design an algorithms structure to address redundancy resolution and path planning for redundant manipulators in the global sense. The proposed top-down algorithm structure provides a global view of the search space with the help of a connectivity graph in the top layer where path options are obtained for further exploration in the lower layers. A 3-DOF planar manipulator is used to illustrate the methodology and a real application is based on the AAI Arm.>

A method for designing a robust time-delayed teleoperator control system based on H/sub /spl infin// optimization is presented. The proposed teleoperator control system deals with the robustness of teleoperation especially during the contact phase. In the approach phase where the slave end-effector is assumed to be in free space such that no significant modeling uncertainties are present, an optimal time-delayed teleoperator control system designed based on Smith's principle may suffice. However, in the contact phase where the uncertainties of contact time and environmental dynamics are of major concern, a time-delayed teleoperator control system should be designed to be robust to modeling uncertainties. Here, we apply the H/sub /spl infin// methodology for the design of such a robust time-delayed teleoperator control system. Simulation results show the validity of the proposed method.>

This paper presents an in-depth analysis on the kinematic features of parallel manipulators, such as deficiency, singularity, and redundancy. Parallel manipulators under consideration are of a general form, with no restriction on their structure. Three orthogonal instantaneous motion spaces (IMSs) of a parallel manipulator are identified, including deficient, uncontrollable, and controllable IMSs. Based on the IMSs, the deficiency and singularity of a parallel manipulator are described and analyzed, and their remedies are devised. Three types of redundancies of a parallel manipulator are introduced, including the redundancies in serial chain, joint actuation, and parallelism. The three types of redundancies are shown to overcome the singularity of a parallel manipulator, as well as to improve either its dexterity or load capacity.>

In this paper we initially present an in-depth analysis of the semi-singularity in terms of its definition, properties and its relationship to the self-motion topology structure in the presence of kinematic constraints. Next, the topology structure is explored as a basis for the design of an algorithm structure suitable to address redundancy resolution and path planning for redundant manipulators in the global sense. The proposed top-down algorithm structure provides a global view of the search space with the help of a connectivity graph in the top layer where path options are obtained for further exploration in the lower layers under a parallelizable framework. A 3-DOF planar manipulator is used to illustrate the methodology and a case study is based on the AAI Arm. The methodology is also appropriate for the implementation of the telemonitoring concept for the kinematic control of redundant manipulators.>

This paper presents a method for solving inverse mapping of a continuous function learned by a multilayer feedforward mapping network. The method is based on the iterative update of input vector toward a solution, while escaping from local minima. The input vector update is determined by the pseudo-inverse of the gradient of Lyapunov function, and, should an optimal solution be searched for, the projection of the gradient of a performance index on the null space of the gradient of Lyapunov function. The update rule is allowed to detect an input vector approaching local minima through a phenomenon called "update explosion". At or near local minima, the input vector is guided by an escape trajectory generated based on "global information", where global information is referred to here as predefined or known information on forward mapping; or the input vector is relocated to a new position based on the probability density function (PDF) constructed over the input vector space by Parzen estimate. The constructed PDF reflects the history of local minima detected during the search process, and represents the probability that a particular input vector can lead to a solution based on the update rule. The proposed method has a substantial advantage in computational complexity as well as convergence property over the conventional methods based on Jacobian pseudo-inverse or Jacobian transpose.

This paper presents a method for the automatic generation of assembly sequences from a liaison graph representation of an assembly through the recursive decomposition of assembly into subassemblies. In order to increase the planning efficiency, the proposed assembly planning system automatically identifies and avoids those decomposition that incur physically infeasible assembly operation. This is achieved by merging those parts that can not be mutually separable at the current stage of assembly planning due to interconnection infeasibility as well as functional dependency. The above merging process transforms the original liaison graph into an abstract liaison graph with smaller number of nodes. Then, weights are assigned to each liaison of abstract liaison graph based on the stability and structural connectivity associated with liaison such that these weights are used to extract the tentative subassemblies. To select preferred subassemblies, the extracted tentative subassemblies are evaluated based on the subassembly selection indices defined in terms of mobility, structural preference, stability, and parallelism. Furthermore, by adjusting the assembly coefficients of subassembly selection indices according to given assembly environment, an optimal assembly sequence can be generated. The application of the proposed planning system to the tabletop vise assembly is illustrated as an example.>

Investigates whether there exists some universal trend such that an actuator utilizing the same kind of energy source but having a small size will have an intrinsic advantage in terms of power-to-weight ratio (PWR). The results indicate that this trend does not exist in general. In fact it is found that the maximum PWR of most electric motors is independent of motor size even though their force-to-weight ratio (FWR) may vary, except that the PWR of striated muscles of vertebrates increases with muscle size. Therefore, the advantage of integrated actuation must come from other aspects such as flexibility in controlling shape, velocity and force, robustness, etc.

This paper presents a unified and efficient method of solving the inverse kinematics for six degree-of-freedom serial connections of two three degree-of-freedom serial and parallel manipulators. Based on the kinematic analysis of three degree-of-freedom serial and parallel manipulators, the forward kinematics are formulated for six degree-of-freedom serial connections, including parallel-parallel, serial-parallel, parallel-serial, and serial-serial types. A unified method of solving the inverse kinematics for the serial connections is established: for a given task velocity, the inverse kinematics of two arms are obtained separately, thereby significantly reducing a computational cost. This is made possible by projecting a given velocity onto the infeasible velocity space of one arm independently. Through computational analysis, the proposed method is shown to be effective for the serial connections having at least one parallel manipulator.

Force-based reasoning, for identifying a cluster of parts that can be decomposed naturally by the applied force, plays an important role in selecting feasible subassemblies and analyzing subassembly stability in assembly planning. This is because force-based reasoning provides a new set of constraints which are distinctive from those from geometric reasoning. Such constraints not only enhance the planning efficiency but also allows stability analysis in terms of subassembly deformation due to the mating forces. Specifically, force-based reasoning can be applied during subassembly identification to eliminate unstable subassemblies that are deformable during the mating operation.

The paper investigates the topology of self-motion manifolds for serial redundant manipulators with revolute joints in the presence of joint limits. It is known that the preimages of singular taskpoints divide the configuration space into regions where self-motion manifolds are homotopic, Initially, the authors describe how self-motion manifolds rupture as one moves from one region to the next. The influence of joint limits on those topologies is investigated next. This analysis led to the discovery of the semi-singularity, a new type of singularity introduced by the presence of joint limits in redundant manipulators. A 3-DOF planar robot is used to illustrate the phenomena and a case study explores the concepts in more detail. This is an important analysis for global redundancy resolution and path planning because it describes the connectivity as well as the general form of self-motion manifolds.

This paper presents a method of automatically constructing a model or a set of constraints from domain principles for solving the dynamic behavior of a mechanical system through qualitative simulation. It emphasizes the following issues: the extraction of a set of necessary and sufficient constraints which provides the minimum uncertainty associated with qualitative simulation, from the fundamental principles and laws of physics based solely on the physical description of a given mechanical system; and the modification of constraints through time by detecting and identifying “system discontinuities” due to collisions, separations, and other critical states associated with each of the object primitives. The first is accomplished by describing a mechanical system by a collection of object and inter-connection primitives, which allows direct instantiating of all the relevant physics laws from the knowledge base. Then, the final set of constraints having the minimum complexity and uncertainty is extracted from the relevant physics laws, based on an A* algorithm with heuristics providing the problem solving expertise. The second is accomplished by monitoring whether the states of any individual sub-systems evolve to system discontinuities represented by intra/inter subsystem critical states.

A real-time teleoperator control system designed to achieve desired performance and robustness under shared compliance control and short time delays of up to a few seconds is evaluated. In the design, telemonitoring force feedback is introduced as a new form of kinesthetic coupling; dynamic characteristics of the master and slave arms are actively modified based on generalized impedance control according to local design criteria; the human dynamics involved in generating a control command based on visual and force stimuli are incorporated into the controller design; and to balance the robustness/performance trade-off, system performance is optimized subject to a known maximum time delay. Simulation results suggest that the system is superior to conventional systems in terms of performance and robustness under short time delays and human control errors. Experimental evidence is presented to supports the advantages of the proposed telemonitoring force feedback. An experimental method for further validating the human dynamic model is described.>

An approach for tracing, representation, and recognition of a handwritten numeral in an offline environment is presented. A 2D spatial representation of a numeral is first transformed into a 3D spatiotemporal representation by identifying the tracing sequence based on a set of heuristic rules acting as transformation operators. Given the dynamic information of the tracing sequence, a multiresolution critical-point segmentation method is proposed to extract local feature points, at varying degrees of scale and coarseness. A neural network architecture, the hierarchically self-organizing learning (HSOL) network (S. Lee, J.C. Pan, 1989), especially for handwritten numeral recognition, is presented. Experimental results based on a bidirectional HSOL network indicated that the method is robust in terms of variations, deformations, and corruption, achieving about 99% recognition rate for the test patterns.>

A novel optical proximity sensor system, capable of measuring the distance to the orientation and the discontinuity at a local area of an object surface, was designed and evaluated for robotic applications. The sensor system gets its name, Hexagonal Eye, (HexEYE) from its shape where seven identical hexagonal sensing units were configured hexagonally into a compact geometric structure. The seven sensing units were grouped into multiple combinations of three sensing units to measure the surface orientation as well as surface discontinuity. The distinctive features of HexEYE are an order of magnitude increased in distance sensitivity by optimizing the curvature of a conical mirror, the enhanced measurement accuracy based on multiple levels of sensor fusion, and the compactness in size due to a sensing mechanism based on the Gaussian lens law. A prototype of single sensing unit has been built and was evaluated experimentally.>

The author considers automatic assembly planning by developing a backward assembly planner which handles the case where an assembly sequence is not the same as the reverse of a disassembly sequence; improving planning efficiency with the reduction of search space by merging and grouping parts based on interconnection feasibility and special process precedence constraints; establishing assembly process planning by incorporating such special processes as testing, cleaning, etc., in assembly planning; and providing subassembly evaluation criteria with a direct connection to assembly cost. A method for the stability and directionality of an assembly is presented through which the required number of holding devices and reorientations during assembly are identified.>

The design goal of an advanced teleoperator control system is twofold: (1) to allow the operator's manual control to be robust to system nonlinearities such as time delays and operator's control errors, and (2) to support the high performance of teleoperation while reducing the operator's control burden by providing the master and slave arms with desirable dynamic properties and by allowing the slave arm to automatically perform such control tasks as compliance and force control in the form of task sharing. The authors present a novel teleoperator control system achieving the above design goal by taking the following into consideration: the human dynamics involved in generating control command based on visual and force feedback is modeled and incorporated into the controller design and evaluation: the dynamic characteristics of slave and master arms are actively modified in such a way as to implement the desirable dynamic characteristics; and the force feedback is redefined in terms of the combination of opposition and force discrepancies in order to establish the required man/machine dynamic coordination under shared control. The proposed control system with human dynamics in the control loop is simulated and compared with a number of conventional methods in the presence of human control errors and time delays.>

A framework for applying cognitive abstraction in the domain of qualitative physics to suppress irrelevant distinctions between the possible behaviors is presented. Cognitive abstraction is especially useful for inferring the behavior of a physical system based on incremental qualitative simulation. The framework of a prototype of cognitive abstraction, based on QSIM, is presented. The limitations of the approach are discussed.>

The radial basis bidirectional competitive and cooperative network (RCCN) is a bidirectional mapping network that accommodates and generates radial basis function units (RBFUs) with the help of efficient use of the accommodation boundaries. The analysis and simulation show that the automatic generation scheme provides the necessary and sufficient enhancement of the network, the hierarchical learning scheme ensures the desired accuracy in mapping, the mapping scheme processes the many-to-many relation for both directions with sufficient accuracy, and using ellipsoidal boundaries is more efficient and flexible compared to circles. RCCN may create an enormous number of RBFUs and degenerate in accuracy by learning with noisy samples. However, greater efficiency can be expected if RBFUs are allowed to have individual accommodation boundary sizes under the optimal learning scheme.>

This paper presents two different ap- proaches of deflning the manipulabilities and resistivi- ties of serial-parallel manipulator systems. In the flrst approach, the manipulability (resistivity) of the serial- parallel structure is dehed by imposing the kinematic constraint on the local manipulabilities (local resistiv- ities), each of which is obtained from the unit sphere in the joint velocity Ljoint torque) space of an individ- ual arm. In the second approach, the manipulability (resistivity) of the serial-parallel structure is defined by imposing the kinematic constraint directly on the unit sphere in the extended joint space formed by all the arms. The two approaches are shown to be com- plementary to each other: the fist approach is useful for the geometrical analysis of the interactions among local arms, while the latter is useful for the quanti- tative analysis of the performance of a serial-parallel structure as a whole.

In this paper, we propose the concept of as a means of coordinating be- tween operator and manipulator under the shared control and the use of human in the con- trol loop to achieve a valid design of a teleoper- ator control system. Especially, this paper shows that 1) In designing and controlling teleoperation system under time delay, it is necessary to incor- porate a dynamic behavior into the con- trol loop. 2) Under the compliance and force control implemented at the remote manipulator in the form of shared control, the conventional kinesthetic coupling between man and manipula- tors based on telepresence incurs an adverse ef- fect on performance. 3) Under the communication time delay, the proposed force feedback based on telemonitoring enhances system stability as well as performance significantly. Simulation results are given.

This paper presents a new neural net ap- proach to robot visuo-motor coordination. The ap- proach, refered to here as neurobotics , establishes neural net-based visual error servoing as a fundamen- tal mechanism of implementing visuo-motor coordi- nation. Visual error servoing is achieved by project- ing the robot task space on the 3D perception net (representing the robot internal 3D space) and gen- erating an incremental change of 3D space arm con- figuration in the 3D perception net based on poten- tial field-based reactive path planning. The 3D space arm configuration planned in the 3D perception net is then translated into the corresponding joint angles through the arm kinematic net. The arm kinematic net is formed based on hierarchically self-organizing competitive and cooperative network. Simulation re- sults are shown.

An approach to the representation of signatures in an offline environment that makes a tracing of a signature in a manner similar to that of a human normally does, and incorporates the dynamic information of the tracing sequence into the representation of the signature, is presented. Tracing involves hierarchical decision-making for stroke identification and ordering based on a set of heuristic rules. These heuristic rules work as operators that transform a two-dimensional (2-D) spatial pattern into a one-dimensional (1-D) temporal pattern, thus making it possible to extract the dynamic features of a signature. Following the stroke sequence of a signature identified by the tracing, a multiresolution critical-point segmentation method is proposed to extract local feature points, at varying degrees of scale and coarseness, for representation. To make the representation translation-, rotation-, and scaling-invariant, a critical-point normalization method is introduced. Experimental results are shown.>

An approach for representing signatures in an offline environment is presented. The approach first makes a tracing of a signature similar to the way a human normally does, using hierarchical decision-making for stroke identification and ordering based on a set of heuristic rules. The dynamic information from the tracing sequence is then incorporated into the representation of the signature. A multiresolution critical-point segmentation method is used to extract local feature points, at varying degrees of scale and coarseness, for subsequent representation of the signature. Experimental results are discussed.>

Accurately estimating the 3-D pose (position and orientation) of a moving object with reference to the world frame (or robot base frame) based on a sequence of stereo images taken by cameras mounted on the end-effector of a robot manipulator is considered. Emphasis is given to the 3-D pose estimation in the presence of measurement noise in 2-D images and camera position errors due to the random noise involved in the joint angles of a robot manipulator. To this end, a set of discrete Kalman filter equations is derived, based on the following: (1) the orientation error of the object frame due to the measurement noise in 2-D images is modeled with reference to the camera frame; (2) an extended Jacobian matrix is formulated by combining the result of (1) with the orientation error of the end-effector frame; and (3) the rotational motion of an object, which is nonlinear in nature, is linearized based on quaternions. Motion parameters are computed from the estimated quaternions based on the iterated least-squares method. Simulation results show a significant reduction in estimation errors.>

A method for achieving redundant arm kinematic control based on parameterization is presented. The method, referred to as the parameterization method, parameterizes properly chosen redundant joints, and obtains a close form of parameterized inverse kinematic solution to accomplish position-based kinematic control. The parameterization of redundant joints is based on: (1) selecting a point in the solution manifold of redundant joints that is closest to one of the reference points, where the reference points represent desirable arm configuration in the parameter space; and (2) adjusting the selected parameter values in the parameter space for the optimization of performance specified by a potential field over the joint space. The proposed method provides the computational efficiency and numerical stability, as well as the capability of trajectory planning in the joint space for redundant arm kinematic control. The case study demonstrates that the parameterization method can be successfully applied to the position-based kinematic control of the eight-degree-of-freedom redundant arm.>

An approach to a self-reconfigurable manipulator system which dynamically changes its topology according to given task requirements is presented. The proposed self-reconfiguration differs from that of conventional reconfigurable robots, in that it does not need to replace and reassemble modularized links to build a robot of desirable mechanical structure, but it customizes a multiple redundant manipulator system by reconfiguring its mechanical structure through serial, parallel, and bracing mechanisms of multiple arm cooperation. The authors present a methodology of generating serial, parallel, and bracing structures for a self-reconfigurable dual-arm system based on manipulabilities and resistivities. Such manipulabilities and resistivities quantify the efficiency and capability of a self-reconfigurable dual-arm system in generating the Cartesian motion and static force, thereby providing information essential to customizing a dual-arm topology suitable for a given task. An example is presented.>

In this panel session, the following topics are discussed: artificial intelligence in business; artificial intelligence in multimedia; neural networks as a tool for artificial intelligence: software engineering for knowledge-based systems: and artificial intelligence as a solution for software engineering.>

An assembly planning system is presented that operates based on a recursive decomposition of an assembly into subassemblies, and uses design for assembly (DFA) analysis to guide the generation of a preferred assembly plan. The planning in this system incorporates the special processes, such as cleaning, testing, labeling, etc., that must occur during the assembly. The system handles nonreversible as well as reversible assembly tasks through backward assembly planning.>

A method is presented for automatically generating a set of qualitative constraints as the model of a physical system. It emphasizes (1) the generation of qualitative constraints which provide minimum uncertainty for qualitative simulation, and (2) the generation of mathematical constraints for reducing the uncertainty during qualitative simulation. The first is accomplished by searching for a set of redundant constraints based on fundamental principles and the laws of physics. The second is accomplished by applying mathematical knowledge to generate mathematical constraints which are used as global filters during qualitative simulation.>

An optimal sensing strategy for an optical proximity sensor system engaged in the recognition and localization of 3D natural quadric objects is presented. The optimal sensing strategy consists of the selection of an optimal beam orientation and the determination of an optimal probing plane that compose an optimal data collection operation known as an optimal probing. The decision of an optimal probing is based on the measure of discrimination power of a cluster of surfaces on a multiple interpretation image, where the measure of discrimination power is defined in terms of a utility function computing the expected number of interpretations that can be pruned out by a probing. An object representation suitable for active sensing based on a surface description vector distribution graph and hierarchical tables is presented. Experimental results are shown.>

The authors present a Kalman filter approach for accurately estimating the 3D position and orientation of a moving object from a sequence of stereo images. One of the drawbacks of using a long sequence of images is that the noisy images taken from a longer distance result in larger errors in 3D reconstruction and, consequently, lead to a serious degradation in motion estimation. To overcome this drawback, the authors have derived a set of Kalman filter equations for motion estimation in the quaternion representation. The measurement equation is obtained by analyzing the effect of white Gaussian noise in 2D images on 3D positional errors, and incorporating the optimal 3D reconstruction under the consistency constraint. The state propagation equation is formulated by specifying the error between the true rotation and the nominal rotation in terms of the measurement noise in 2D images. Actual rotation parameters have been computed from the estimated quaternions by the iterated least-squares method. Simulation results indicate that the equations derived are accurate for 3D motion estimation.>

An algorithm is presented for the recognition and localization of 3-D polyhedral objects based on an optical proximity sensor system capable of measuring the depth and orientation of a local area of an object surface. Emphasis is given to the determination of an optimal sensor trajectory or an optimal probing, for efficient discrimination among all the possible interpretations. The determination of an optimal sensor trajectory for the next probing consists of the selection of optimal beam orientations based on the surface normal vector distribution of the multiple interpretation image (MII) and the selection of an optimal probing plane by projecting the MII onto the projection plane perpendicular to a selected beam orientation and deriving the optimal path on the projection plane. The selection of optimal beam orientation and probing plane is based on the measure of discrimination power of a cluster of surfaces of an MII. The measure of discrimination power is obtained by computing the utility of a cluster of surfaces, representing the expected number of interpretations that can be pruned. Simulation results are shown.>

A Kalman filter approach for accurately estimating the 3-D position and orientation of a moving object with respect to the robot base frame is proposed. This approach significantly differs from other approaches in that motion is estimated with the uncertainty of the position of a camera taken into account. Emphasis is also given to finding a solution to the following problem of motion estimation using a long sequence of images: the images taken from a longer distance suffer from a larger noise-to-signal ratio, which results in larger errors in 3-D reconstruction and, thereby, causes a series degradation in motion estimation. To solve this problem, the authors have derived a set of discrete Kalman filter equations for motion estimation. The measurement equation is obtained by analyzing the effect of white Gaussian noise in 2-D images on 3-D positional errors, and the system dynamic equation is formulated in terms of the measurement noise in 2-D images. Simulation results indicate that the Kalman filter equations derived present an accurate model for the estimation of 3-D position and orientation, thus providing significant error reduction in the presence of large measurement noise in a long sequence of images, as well as allowing a shorter transition period for convergence to the true values.>

The kinematic control of a redundant arm based on multiple tasks assigned to different locations of the manipulator is presented. This is equivalent to decomposing a redundant arm into two (or more) nonredundant local arms, the basearm and the forearm, at an intermediate arm location or task point called the elbow. A redundant arm is transformed into a serially cooperating dual-arm system, with the cooperation between the local arms being carried out at the elbow. Then, a manipulator end-effector motion specified by a given task is decomposed into motions of individual local arms as well as an end-effector free motion based on achieving maximum efficiency in task execution. To consider global optimization, elbow control is applied to reshaping and reorientation of a manipulability ellipsoid at the end effector during task execution. The resolved rate control of elbow in the end-effector null and free motion space is used to maximize arm homogeneity and to match the forearm motion space with the task space. Simulation results are shown.>

A method is presented for the automatic determination of assembly partial orders from a liaison graph representation of an assembly through the extraction of preferred subassemblies. In particular, the authors show how to select a set of tentative subassemblies by decomposing a liaison graph into a set of subgraphs based on feasibility and difficulty of disassembly, how to evaluate each of the tentative subassemblies in terms of assembly cost using the subassembly selection indices, and how to construct a hierarchical partial order graph as an assembly plan. The method provides an approach to assembly planning by identifying spatial parallelism in assembly as a means of constructing temporal relationships among assembly operations and solves the problem of finding a cost-effective assembly plan in flexible environment. A case study of the assembly planning of a mechanical assembly is presented.>

The authors present a novel method of accomplishing robot kinematic control based on a bidirectional mapping neural network (BMNN). The BMNN constructed is composed of a multilayer feedforward network with hidden units having sinusoidal activation functions and a feedback network forming a recurrent loop around the feedforward network. The feedforward network can be trained to accurately represent the forward kinematic equations of a robot arm. The feedback network iteratively generates joint-angle updates based on a Lyapunov function to modify the current joint angles in such a way that the output of the forward network converges to the desired Cartesian position and orientation. The proposed BMNN offers the following advantages over the conventional approaches: (1) the accurate computation of robot forward and inverse kinematic solutions with simple training; (2) the ability to handle one-too-many inverse mapping required for redundant arm kinematics solutions; and (3) the automatic generation of arm trajectories. Simulation results are shown

This paper presents a resource-level conflict detection and conflict resolution scheme which is combined with a state-level backward planning algorithm and provides efficient conflict detection and global conflict resolution for nonlinear planning. The scheme keeps track of the usage of individual resources during planning, and constructs a Resource-Usage Flow (RUF) structure (based on which conflict detection and resolution is accomplished). The RUF structure allows the system to perform minimal and nonredundant operations for conflict detection and resolution. Furthermore, resource-level conflict detection and resolution facilitates problem decomposition in terms of resources, thereby providing easy implementation in a parallel and distributed processing environment. Performance analysis indicates that the proposed architecture has a speed-up factor of the average depth of a plan network, D(Na), compared to the distributed NOAH, where Na (the total number of action nodes at the completion of planning) and D(Na) are considerably larger than the number of resources involved in planning as well as the number of initial goal states.

An approach to the representation of arbitrary line drawings is presented. With this approach the system makes a tracing of line drawings similar to the way a human normally does, and incorporates the result of the tracing sequence into the structural representation of the line drawing. Following the sequence of line drawings identified by the tracing, the critical point segmentation method is proposed to extract all the local feature points for representation. To make the representation size and rotation invariant, the authors introduce the critical point normalization method based on orthonormal transformation. Experimental results are shown.>

Concerns the learning of problem-solving methods by method generalization. The authors formalize the method generalization problem and present the computational approaches and techniques as well as the corresponding implementational issues for generalizing problem-solving methods. They investigate a restricted class of method generalization problems: given a concept hierarchy and a set of methods associated with individual concepts, the methods applicable to more general concepts are constructed from the methods associated with more specific concepts. A case study is presented which shows how methods for computing the properties of geometric entities in a hierarchy can be generalized to compute the properties of higher-level entities in the domain of Euclidean geometry.>

It is shown that the required motion and force trajectories of a given task can be abstracted by a series of desired manipulability ellipsoids and that task-oriented dual-arm manipulability (TODAMM) can be mathematically defined by quantifying how the manipulability of one arm affects the other and measuring the geometrical closeness between the desired and the actual manipulability ellipsoids. TODAMM can be used in the optimization of dual-arm joint configurations. The task-oriented manipulability measure (TOMM) developed can also be used in the joint configuration optimization of a single arm, providing efficient joint configurations in terms of joint motions and joint torques for the required Cartesian motions and static forces. The dual-arm joint configuration optimization based on TODAMM can be applied to a variety of asks which require dual-arm cooperation.>

This paper presents the conceptual design, analysis, synthesis and software organization of an advanced teleoperator control system with sensory feedback. The design requirements for the system are discussed in detail and an implementation strategy is presented. The resulting system features maximum autonomy of the local hand controller and remote manipulator subsystems, along with kinematic and dynamic coordination between these subsystems. The final design emphasizes cooperation and interaction between the human operator and the computers in control of the sensor-based manipulator system. The hardware and software modules being used to implement the system at JPL are described.