Unsupervised anomaly detection (UAD) is crucial in low-dose computed tomography (LDCT). Recent AI technologies, leveraging global features, have enabled effective UAD with minimal training data of normal patients. However, this approach, devoid of utilizing local features, exhibits vulnerability in detecting deep lesions within the lungs. In other words, while the conventional use of global features can achieve high specificity, it often comes with limited sensitivity. Developing a UAD AI model with high sensitivity is essential to prevent false negatives, especially in screening patients with diseases demonstrating high mortality rates. We have successfully pioneered a new LDCT UAD AI model that leverages local features, achieving a previously unattainable increase in sensitivity compared to global methods (17.5% improvement). Furthermore, by integrating this approach with conventional global-based techniques, we have successfully consolidated the advantages of each model - high sensitivity from the local model and high specificity from the global model - into a single, unified, trained model (17.6% and 33.5% improvement, respectively). Without the need for additional training, we anticipate achieving significant diagnostic efficacy in various LDCT applications, where both high sensitivity and specificity are essential, using our fixed model. Code is available at https://github.com/kskim-phd/Fusion-UADL.

Computer-aided diagnosis for low-dose computed tomography (CT) based on deep learning has recently attracted attention as a first-line automatic testing tool because of its high accuracy and low radiation exposure. We propose a method based on a deep neural network for computer-aided diagnosis called virtual multi-view projection and reconstruction for unsupervised anomaly detection (VMPR-UAD) in low-dose chest CT. Presumably, this is the novel method that only requires data from healthy patients for training to identify three-dimensional (3D) regions containing any anomalies. The method has three key components. Unlike existing computer-aided diagnosis tools that use conventional CT slices as the network input, our method (1) improves the recognition of 3D lung structures by virtually projecting an extracted 3D lung region to obtain two-dimensional (2D) images from diverse views to serve as network inputs, (2) accommodates the input diversity gain for accurate anomaly detection, and (3) achieves 3D anomaly/disease localization through a novel 3D map restoration method using multiple 2D anomaly maps. The proposed method based on unsupervised learning showed a high performance in pneumonia, tuberculosis, and both diseases with patient-level anomaly detection performance of 0.965 area under the curve (AUC) (95% confidence interval (CI); (0.955, 0.972)), 0.948 AUC (95% CI; (0.928, 0.966)), and 0.963 AUC (95% CI; (0.955, 0.970)), respectively. Additionally, our technology visualizes anomalous regions in the 3D perspective. This achieved 93% accuracy in visualizing the location of lung cancer lesions through external validation. These results highlight the potential of a new AI methodology without utilizing disease data learning; this can secure AI model prediction stability by reducing the false negative rate that occurs in various patterns of diseases.

Peripherally inserted central catheters (PICCs) have been widely used as one of the representative central venous lines (CVCs) due to their long-term intravascular access with low infectivity. However, PICCs have a fatal drawback of a high frequency of tip mispositions, increasing the risk of puncture, embolism, and complications such as cardiac arrhythmias. To automatically and precisely detect it, various attempts have been made by using the latest deep learning (DL) technologies. However, even with these approaches, it is still practically difficult to determine the tip location because the multiple fragments phenomenon (MFP) occurs in the process of predicting and extracting the PICC line required before predicting the tip. This study aimed to develop a system generally applied to existing models and to restore the PICC line more exactly by removing the MFs of the model output, thereby precisely localizing the actual tip position for detecting its misposition. To achieve this, we proposed a multi-stage DL-based framework post-processing the PICC line extraction result of the existing technology. Our method consists of the following three stages: 1. Existing PICC line segmentation network for a baseline, 2. Patch-based PICC line refinement network, 3. PICC line reconnection network. The proposed second and third-stage models address MFs caused by the sparseness of the PICC line and the line disconnection due to confusion with anatomical structures respectively, thereby enhancing tip detection. To verify the objective performance of the proposed MFCN, internal validation and external validation were conducted. For internal validation, learning (130 samples) and verification (150 samples) were performed with 280 data, including PICC among Chest X-ray (CXR) images taken at our institution. External validation was conducted using a public dataset called the Royal Australian and New Zealand College of Radiologists (RANZCR), and training (130 samples) and validation (150 samples) were performed with 280 data of CXR images, including PICC, which has the same number as that for internal validation. The performance was compared by root mean squared error (RMSE) and the ratio of single fragment images (RatioSFI) (i.e., the rate at which model predicts PICC as multiple sub-lines) according to whether or not MFCN is applied to seven conventional models (i.e., FCDN, UNET, AUNET, TUNET, FCDN-HT, UNET-ELL, and UNET-RPN). In internal validation, when MFCN was applied to the existing single model, MFP was improved by an average of 45 %. The RMSE improved over 63% from an average of 27.54 mm (17.16 to 35.80 mm) to 9.77 mm (9.11 to 10.98 mm). In external validation, when MFCN was applied, the MFP incidence rate decreased by an average of 32% and the RMSE decreased by an average of 65%. Therefore, by applying the proposed MFCN, we observed the consistent detection performance improvement of PICC tip location compared to the existing model. In this study, we applied the proposed technique to the existing technique and demonstrated that it provides high tip detection performance, proving its high versatility and superiority. Therefore, we believe, in countries and regions where radiologists are scarce, that the proposed DL approach will be able to effectively detect PICC misposition on behalf of radiologists.

This letter presents a robust and fast 6D camera motion estimation system that combines a FAST detector with a Multi-scale Descriptor (MSD). For a robust and fast camera motion estimation system, the descriptor is an especially important component, as it is the most time consuming step in motion estimation. Furthermore, the properties of the descriptor significantly affect the precision of the estimated motion. In this letter, we present a descriptor that requires a low computational burden and offers high precision for robust and fast motion estimation. The descriptor provides high precision through the use of multi-scale gradient information and computational efficiency by employing 4 integral images and an intensity gradient. We conclude this letter with an evaluation of the proposed motion estimation in comparison with different types of descriptors in practical situations. The results show that the proposed motion estimation provides good performance over previous methods in terms of processing time and precision of estimated camera motion.

A framework of robust online digital image stabilization based on point-feature trajectory is proposed. The conventional online digital image stabilization approaches, which have to use accumulative global motion with respect to the initial reference frame, might fail in the long run because error accumulation is inevitable with the elapse of time. Instead, the proposed scheme directly estimates the correction motion between unstable input and stabilized output images by exploiting original and smoothed point-feature trajectories without the accumulative global motion. As a result, ours is able to operate for a long time. The experimental results show that the proposed method is able to stabilize unstable video with real-time process for a long time and has good robustness to a variety of conditions.

This paper investigates the estimation of 3D head poses and its identity authentication with a partial ellipsoid model. To cope with large out-of-plane rotations and translation in-depth, we extend conventional head tracking with a single camera to a stereo-based framework. To achieve more robust motion estimation even under time-varying lighting conditions, we incorporate illumination correction into the aforementioned framework. We approximate the face image variations due to illumination changes as a linear combination of illumination bases. Also, by computing the illumination bases online from the registered face images, after estimating the 3D head poses, user-specific illumination bases can be obtained, and therefore illumination-robust tracking without a prior learning process can be possible. Furthermore, our unified stereo-based tracking is approximated as a linear least-squares problem; a closed-form solution is then provided. After recovering the full-motions of the head, we can register face images with pose variations into stabilized-view images, which are suitable for pose-robust face recognition. To verify the feasibility and applicability of our approach, we performed extensive experiments with three sets of challenging image sequences.

This paper presents a novel approach for multi-class pattern classification - face detection and facial expression recognition, which is based on discriminative multi-scale and multi-position Local Binary Pattern (MspLBP) features selected by a boosting technique called the AdaBoost+LDA (Ada-LDA) method. From a large pool of MspLBP features within a face image, the most discriminative MspLBP features trained by two alternative LDA methods depending on the singularity of the within-class scatter matrix, are selected under the framework of AdaBoost. To verify the feasibility of our approach, we performed two extensive experiments on the famous face databases in terms of face detection and facial expression recognition. First, face detection, a typical example of two-class pattern classification, was carried out on the MIT-CBCL and MIT+CMU face test sets. Second, facial expression recognition, a typical problem of multi-class pattern classification, was performed on the JAFFE face database. Given the same number of features, the proposed face detector shows over 25% higher detection rate than the well-known Viola's detector at a given false positive rate of 10%. It can also provide real-time operation with over 10 frames per second rate. For facial expression recognition, our approach also shows a better performance over at least 21% recognition rates than other linear subspace-based methods such as PCA, DCV, and PCA+LDA. Our proposed approach provides a considerable performance improvement with only a small number of discriminative MspLBP features in the multi-class pattern classification problem.

Attempts to put robots to practical use have been increasing, as robot becomes more human-friendly. In the human-robot interaction field, main issues are how variously the robot can express its emotion and how much the expression is socially acceptable. This paper proposes a robot's expression generation method based on the context information and combination of behavior database for emotional interactive service. Our goal is to generate synchronized multimodal expressions automatically using a few of pre-defined database and the context information inputs. Overall procedure is separated by offline task using a GUI-based editing toolkit and online task using an automatic generation method. In the offline task, behavior database is formulated corresponding to types of sentences and robot's emotional state. For automatic generation, we propose primitive database based on parameters and organizing behavior database. In the online task, using context information inputs, robot generates new expression automatically. Proposed generation method is to combine pre-defined behavior database using parameters of the primitive database and interpolation method. Finally, generated synchronized multimodal expressions are refined according to a robot's hardware.

This paper investigates the estimation of 3D head poses with a partial ellipsoid model. To cope with large out-of-plane rotations and translation in depth, we extend conventional head tracking with a single camera to a stereo-based framework. To achieve more robust motion estimation (even under time-varying lighting conditions), we incorporate illumination correction into the aforementioned framework. We approximate the face image variations due to illumination changes as a linear combination of illumination bases. Also, by computing the illumination bases online from the registered face images, after estimating the 3D head poses, user-specific illumination bases can be obtained, and therefore illumination-robust tracking without a prior learning process can be possible. Furthermore, our unified stereo-based tracking is approximated as a linear least-squares problem; a closed-form solution is then provided.

This paper investigates the estimation of 3D head poses and its identity authentication using a simple ellipsoid model. To achieve robust motion estimation even under time-varying lighting conditions, we incorporate illumination correction into the conventional 3D model-based tracking framework with a single camera. In addition, by computing the illumination bases online from the registered face images, after estimating the 3D head poses, user-specific illumination bases can be obtained, and therefore illumination-robust tracking without a prior learning process can be possible. Furthermore, our unified tracking is approximated as a linear least-squares problem; a closed-form solution is then provided. Therefore, it can be executed in real-time at 20 frames per second. After recovering full motion of the head, we can register face images with pose variations into stabilized (frontal) view images which are suitable for pose-robust face recognition. To verify the feasibility and applicability of our proposed 3D head-tracking framework, we performed extensive experiments with three sets of challenging image sequences.

In this paper, we propose a system which reconstructs the environment with both color and 3D information. We perform extrinsic calibration of a camera and a LRF (laser range finder) to fuse 3D information and color information of objects. We also formularize an equation to measure the result of the calibration. Moreover, we acquire 3D data by rotating 2D LRF with camera, and use ICP (iterative closest point) algorithm to combine data acquired in other places. We use the SIFT (scale invariant feature transform) matching for the initial estimation of ICP algorithm. It offers accurate and stable initial estimation robust to motion change compare to odometry. We also modify the ICP algorithm using color information. Computation time of ICP algorithm can be reduced by using color information.

A human face provides a variety of different communicative functions such as identification, the perception of emotional expression, and lip-reading. For these reasons, many applications in robotics require tracking and recognizing a human face. A novel face recognition system should be able to deal with various changes in face images, such as pose, illumination, and expression, among which pose variation is the most difficult one to deal with. Therefore, face registration (alignment) is the key of robust face recognition. If we can register face images into frontal views, the recognition task would be much easier. To align a face image into a canonical frontal view, we need to know the pose information of a human head. Therefore, in this paper, we propose a novel method for modeling a human head as a simple 3D ellipsoid. And also, we present 3D head tracking and pose estimation methods using the proposed ellipsoidal model. After recovering full motion of the head, we can register face images with pose variations into stabilized view images which are suitable for frontal face recognition. By doing so, simple and efficient frontal face recognition can be easily carried out in the stabilized texture map space instead of the original input image space. To evaluate the feasibility of the proposed approach using a simple ellipsoid model, 3D head tracking experiments are carried out on 45 image sequences with ground truth from Boston University, and several face recognition experiments are conducted on our laboratory database and the Yale Face Database B by using subspace-based face recognition methods such as PCA, PCA+LAD, and DCV.

A human face provides a variety of different communicative functions such as identification, the perception of emotional expression, and lip-reading. Many applications in robotics require recognizing a human face. A face recognition system should be able to deal with various changes in face images, such as pose, illumination, and expression, among which pose variation is the most difficult one to deal with. For this reason, face registration is the key of face recognition. If we can register face images into frontal views, the recognition task would be much easier. To align a face image into a canonical frontal view, we need to know the pose information of a human head. A human head can be approximately modeled as a 3D texture mapped ellipsoid. Then, any face image can be considered as a 2D image projection of a 3D ellipsoid at a certain pose. In this paper, both training and test face images are back projected to the surface of a 3D ellipsoid according to their estimated poses and registered into canonical frontal view images. Then, simple and efficient frontal face recognition can be carried out in the texture map domain instead of the original input image domain. To evaluate the feasibility of the proposed approach, several recognition experiments are conducted using subspace-based face recognition methods such as PCA, PCA+LAD, and DCV. By conducting experiments on our laboratory database and the Yale Face Database B, we show that the proposed algorithm provides good performance even when large out-of-plane rotations occur.

In the field of HRI, to take advantage of people’s innate ability of communication, researchers have thus far concentrated on facial expression in facilitating human robot communication. However, for the robot to express emotional intensity, other modalities such as gestures, movement, sound, and color are also needed. This paper suggests that the intensity of emotion can be expressed with color and blinking, which are applicable on robots via LED. Although color and emotion have certain relations, it is difficult to implement the findings of previous research in this area due to a lack of quantitative data. In this paper, we determined RGB values of color and blinking period to effectively express intensities of six basic emotions based on quantitative data. Color and blinking are implemented on an avatar and the intensities of emotions are evaluated through surveys with the avatar. It was found that color and blinking helped the avatar express emotional intensity of surprise, happiness, disgust, and anger. For fear, and sadness, the color and blinking did not play a significant role.

Recently, in the field of HRI, multimodal expression has been an issue. Synchronizing modalities and determining what modality to use are important aspect of multimodal expression. For example, when robots express emotional states, they may use only facial expressions or facial expressions with gestures, neck motions, sounds, etc. In this paper, emotional boundaries are proposed for multimodal expression in a three-dimensional affect space. The simultaneous expression of facial expression and gestures was demonstrated using proposed emotional boundaries on a simulator.

This paper introduces an emotion interaction system for a service robot. The purpose of emotion interaction systems in service robots is to make people feel that the robot is not a mere machine, but reliable living assistant in the home. The emotion interaction system is composed of the emotion recognition, generation, and expression systems. A user's emotion is recognized by multi-modality, such as voice, dialogue, and touch. The robot's emotion is generated according to a psychological theory about emotion: OCC (Ortony, Clore, and Collins) model, which focuses on the user's emotional state and the information about environment and the robot itself. The generated emotion is expressed by facial expression, gesture, and the musical sound of the robot. Because the proposed system is composed of all the three components that are necessary for a full emotional interaction cycle, it can be implemented in the real robot system and be tested. Even though the multi- modality in emotion recognition and expression is still in its rudimentary stages, the proposed system is shown to be extremely useful in service robot applications. Furthermore, the proposed framework can be a cornerstone for the design of emotion interaction and generation systems for robots.

It is expected that robots will be widely exposed to humans in the near future. Emotional communication is very important in human-robot interaction, as well as human- human interaction. Facial expression is an emotional expression method between humans, and it also enables human to recognize a robot's emotional state. Although there are lots of previous research regarding facial expressions, it is not easily applicable to different shapes of robot faces, if the number and types of the robot's control points are varied. In addition, the natural connection between emotions has not been considered or has been inefficiently implemented in previous research. In this paper, we propose a linear dynamic affect-expression model for continuous change of expression and diversifying characteristics of expressional changes. Therefore, the proposed model allows a robot's facial expression and expression changes to more closely resemble those of humans, and this model is applicable to various mascot-type robots irrespective of the number and types of robot's control points.

A robot's face is its symbolic feature, and its facial expressions are the best method for interacting with people with emotional information. Moreover, a robot's facial expressions play an important role in human-robot emotional interactions. This paper proposes a general rule for the design and realization of expressions when some mascot-type facial robots are developed. Mascot-type facial robots are developed to enable friendly human feelings. The number and type of control points for six basic expressions or emotions were determined through a questionnaire. A linear affect-expression space model is provided to realize continuous and various expressions effectively, and the effects of the proposed method are shown through experiments using a simulator and an actual robot system.

This paper deals with structure and motion estimation by using image sequences captured by a multi-camera system moving in a scene. Single camera systems with a relatively small field of view have limited accuracy because of the inherent confusion between translation and rotation. This is also the case for the stereo camera systems if the systems need the intersection of the fields of view for stereo correspondence. The cameras constituting the multi-camera system considered ultimately in this paper are arranged so that there is no, or a small, intersection of the fields of view. This configuration improves the accuracy of the structure and motion estimation because the ambiguity mentioned above decreases due to a large field of view. We propose a recursive algorithm for real time applications of the multi-camera system using long term image sequences, which have appearing and disappearing point features. The results of the experiments with long term real image sequences are presented to demonstrate the feasibility of the algorithm. 1

Image alignment is a widely used technique in computer vision, and it can be applied to many areas such as image registration and region tracking. Hager and Belhumeur proposed an efficient image alignment algorithm which can find out translation and affine deformations quickly. However, it is known that their algorithm cannot be applied to homographies. Image alignment with a homography is important because general motions of object in 3D space are observed as perspective motions which are another name of homographies. In this paper, we propose an expanded inverse additive algorithm for homographies, and apply the algorithm to region tracking. We conduct real-time tracking experiments to show the performance of the proposed algorithm

In this paper, a non-intrusive eye gaze estimation system is proposed. The proposed system consists of five light sources and two cameras, and the direction of the user's eye gaze can be computed by using a projective property without computing the geometrical relation among the eye, the cameras, and the monitor in 3D space. Our method is accurate under head movement, and is comparatively simple and fast. We introduce our method and show experimental results. The experimental results would verify the feasibility of the proposed system as a human-machine interface

Robotic systems like humanoid frequently move to undesirable pose while being visually controlled at the boundary of its workspace, because the working environment is quite huge as compared with the robot's workspace. The main objective of this paper is to deal with problems occurring at the boundary of the robot's workspace. In addition, current typical problems on IBVS and PBVS, e.g., control in 2D and 3D space, task singularity, robustness to camera and robot calibration error, and robot singularity will be considered. To achieve the objective, the proposed two-stage visual servo system consists of exploring and positioning stages. In the exploring stage, given camera parameters, the target pose of the camera is estimated from the obtained images without the knowledge of the object. In the next positioning stage, a new hybrid control law that moves the camera equipped to a robot end-effector to around the target pose, which is estimated in the first stage and then positions it to the target pose more accurately using IBVS is proposed. Although the target pose is located out of the robot's workspace, the proposed visual servo system shows the proper behavior at the boundary of its workspace. Finally simulation and experimental results shows the feasibility of the proposed visual servo system

In the last decade, face analysis, e.g. face recognition, face detection, face tracking and facial expression recognition, is a very lively and expanding research field. As computer animated agents and robots bring a social dimension to human computer interaction, interest in this research field is increasing rapidly. In this paper, we introduce an artificial facial expression mimic system which can recognize facial expressions of human and also imitate the recognized facial expressions. We propose a classifier that is based on weak classifiers obtained by using modified rectangular features to recognize human facial expression in real-time. Next, we introduce our robot that is manipulated by a distributed control algorithm and that can make artificial facial expressions. Finally, experimental results of facial expression recognition and facial expression generation are shown for the validity of our artificial facial expression imitator

This paper presents a vision based SLAM method by using stereo SFM technique. The proposed method is based on the stereo SFM presented in our former paper. The method do not need stereo correspondence but, nevertheless, can determine absolute scale factor, which is an important factor for long term navigation and SLAM. The method use only motion correspondence basically, which is easier to solve than stereo correspondence because the SFM algorithm can use a sequence of images taken at short time intervals. However, we infer the stereo correspondence inversely from the absolute estimates of structure and motion parameters and utilize this information to improve the performance of our method. Consequently, the method maintain the robustness to the stereo correspondence ambiguity and can avoid the degenerate configuration reported in the former paper. We also propose a simple initialization technique for the proposed method based on extended Kalman filter, which is critical issue for the methods using bearing-only measurements. The experimental results demonstrate the effectiveness of the algorithm.

For face tracking in a video sequence, various face tracking algorithms have been proposed. However, most of them have a difficulty in finding the initial position and size of a face automatically. In this paper, we present a fast and robust method for fully automatic multi-view face detection and tracking. Using a small number of critical rectangle features selected and trained by Adaboost learning algorithm, we can detect the initial position, size and view of a face correctly. Once a face is reliably detected, we can extract face and upper body color distribution from the detected facial regions and upper body regions for building a robust color modeling respectively. Simultaneously, each color modeling is performed by using k-means clustering and multiple Gaussian models. Then, fast and efficient multi-view face tracking is executed by using several critical features and a simple linear Kalman filter. Our proposed algorithm is robust to rotation, partial occlusions, and scale changes in front of dynamic, unstructured background. In addition, our proposed method is computationally efficient. Therefore, it can be executed in real-time.

In this paper, we propose a method of selecting new types of rectangle features that are suitable for facial expression recognition. The basic concept in this paper is similar to Violar's approach, which is used for face detection. Instead of previous Haar-like rectangle features, we choose rectangle features for facial expression recognition among all possible rectangle types in a 3/spl times/3 matrix form using the AdaBoost algorithm. Also, the facial expression recognition system constituted with the proposed rectangle features is compared to that with previous rectangle features with regard to its capacity. The results show that the proposed approach has better performance in facial expression recognition in terms of simulation and experimental results.

In recent decades, active vision systems have been studied widely because they can control the gaze direction of their mounted vision sensors and overcome many shortcomings of passive vision systems which have fixed vision sensors. Due to an increasing interest in humanoid robots and social robots, many researchers in the active vision area are focusing their research on developing active vision systems with socially interacting with humans. In this paper, we discuss an active vision system developed for interaction with humans, and we introduce a fast and simple system architecture which includes a high-speed serial communication bus (IEEE 1394a). In addition, we propose a distributed and jerkbounded control algorithm to ensure smooth system motion.

An eye gaze estimation system is proposed. The eye gaze estimation system estimates the eye-gaze direction of a user. The proposed system consisted of five IR LEDs and a CCD camera. The direction of the user's eye gaze can be computed without computing the geometrical relation among the eye, the camera, and the monitor in 3D space. Our method is comparatively simple and fast. The disabled can manipulate a computer using this system. We would introduce our method and show experimental results. The experimental results verify the feasibility of the proposed system as an interface for the disabled.

The stereo SFM combining stereo and motion is a useful vision cue to estimate both the motion of the cameras and the structure of a scene with an absolute scale factor. Most of the works related to the stereo SFM have used both motion correspondence and stereo correspondence. Because it is information redundancy to use these two correspondence simultaneously, they can give accurate results. However, the methods to solve the stereo correspondence problem are not yet reliable enough to have been used in practical applications, while the motion correspondence problem is easy to solve. Therefore, if an absolute scale factor can be acquired by only using motion correspondence without stereo correspondence, it is not necessary to use stereo correspondence for accuracy at the cost of reliability. This paper shows that the scale factor can be determined by using only motion correspondence. Moreover, it is shown that the degenerate cases reported in this paper are all the cases and can be easily avoided. This fact is verified from the two theorems and the simulation results. This paper also describes an implementation of the proposed algorithm. This implementation uses multiple images acquired continuously when a stereo camera moves in a scene and is formulated to be a recursive form for real-time applications. Some peculiar implementation techniques contribute to fast convergence rate and good stability in spite of large initial uncertainty. The simulation and experimental results demonstrate the effectiveness of the algorithm.

An active vision system can control the gaze direction of mounted vision sensors and can overcome many shortcomings of other passive vision systems, which have fixed vision sensors. Furthermore, most visual servoing approaches are concerned with a vision sensor or a passive stereo vision system which has fixed relative coordination between two cameras. In this paper, the problem of controlling a motion by visual servoing with an active stereo vision system is addressed. The main goal is to track the object with our active vision system. Firstly, we discuss how to stack image Jacobians in an active stereo vision system and how to control the active stereo cameras based on images. Next, we introduce our active vision system, which has a fast and simple system architecture that uses a high-speed serial communication bus (IEEE 1394a). It is manipulated by a distributed and jerk-bounded control algorithm to ensure smooth system motion. Finally, simulation and experimental results show both the validity of the proposed definition of the stacked Jacobian to control the system and the feasibility of the proposed the control method.

Vision sensors are attractive equipments for an autonomous mobile robot because they are information-rich and rarely have restrictions in range and various applications. However, there exists a difficult problem in using vision sensors to reconstruct geometrical information, which is a correspondence or matching problem. We present a simultaneous localization and map building (SLAM) algorithm for an autonomous mobile robot based on a vision sensor robust to the correspondence or matching problem with cooperation of structure from motion (SFM) and stereo. We use an omni-directional stereo vision sensor, which is suitable for SLAM. Due to its large field of view, we can acquire robust estimates and reduce the effect of motion drift from the fiducial coordinate frame. Results on real images illustrate the performance of the proposed method.

This paper presents a novel image space path planning method that takes mechanical constraints into consideration. The aim is to synthesize sub-goal images that can be used as reference inputs for image-based visual servoing systems. Synthesized images are constructed based on screw decompositions represented in a projective space. The projective representation allows image synthesis for three-dimensional objects to be made without any metric information or calibration processes. In the heart of the method, an orientation-generating operator is defined in the projective space to generate an image for an arbitrarily orientated object. The proposed scheme not only ensures field of view conditions by planning a straight path, but also provides physically realizable images of a properly oriented gripper to avoid mechanical constraints such as joint limits. Computer simulations are conducted to show the feasibility of the proposed scheme.

Although image-based visual servoing provides superior performance in many vision-based robotic applications, it reveals fatal limitations when initial pose discrepancy is large. The feature points may leave the camera's field of view, and also, the robot may not converge to the goal configuration. In this paper, a novel approach is proposed to resolve these limitations by planning trajectories in the image space using uncalibrated stereo cameras. Intermediate views of the robot gripper are synthesized to generate the image trajectories such that the gripper can track a straight path in the three-dimensional workspace. The proposed method utilizes screw motions of the gripper represented in the projective space with the help of the fundamental properties of projective geometry. Computational issues are also considered to enhance the estimation of the projective transformation from initial to goal configuration. The validity of this approach is demonstrated through computer simulations and limited experiments.

Recently, service area has been emerging field of robotic applications. Even though assistant robots play an important role for the disabled and the elderly, they still &er from operating the robots using conventional interface devices such as joysticks or keyboards. In this paper, we suggest a humanrobot interface using a new eye gaze estimation system. The eye gaze estimation system can estimate the direction of a user’s eye gaze. The proposed system consisted of five IR LEDs and a CCD camera. The direction of the user’s eye gaze can be computed without computing the geometrical relation among the eye, the camera, and the monitor in 30 space. Our method is comparatively simple and fast. The disabled, who is especially people with motor disabilities, can control a robot using this system. We will introduce the method and show experimental results. The experimental results will verify the feasibility of the proposed system as a human-robot interface for the disabled.

Describes a method to construct a stationary environmental map and estimate the ego-motion of a mobile robot from unknown planar motion by using an omni-directional vision sensor. Most environments where a mobile robot works are limited to two-dimensional space and the environmental map which is necessary for mobile robot navigation has also two dimensions. However conventional "structure from motion (SFM)" algorithms cannot be applied to two-dimensional space in perspective projection. We propose a SFM algorithm that can be applied to two-dimensional space. The proposed SFM algorithm exploits the azimuths of features which are obtained from an omni-directional vision sensor and gives robust results against the noise of image information by taking advantage of the large field of view. A relation between observed azimuths and motion parameters of a robot are constrained by a nonlinear equation and our method obtains all the motion parameters and an environmental map through a two-step procedure of solving the equation.

Deals with a matching problem of finding correspondences of features in two omnidirectional images. The proposed method combines the advantages of correlation-based matching and dynamic programming to yield reliable matching results. Our method works well even when some features in one image don't have corresponding features in the other. The search space of the dynamic programming can be reduced by the geometric constraints that the omnidirectional vision sensor provides. We also present a recursive scheme for position estimation of the matched features, so that on-line map generation is possible. Experimental results show that zero failure rate of matching in an indoor environment can be obtained and that a feature-drawing map can be successfully constructed by the proposed feature matching algorithm and map building method.

In spite of its strengths in many aspects, the image-based visual servoing scheme has fatal defects when a desired pose is far from the initial one; out of camera field of view, breaking the stability. One solution to these problems is to generate a desired trajectory in the image space such that the pose errors remain always small. We present a method for image-space trajectory generation that can be utilized in image-based visual servoing when initial errors are large. First, initial and desired points of a gripper are reconstructed up to an unknown projective transformation with an uncalibrated stereo rig. Second, intermediate configurations, with which a smooth and straight motion can be produced, are constructed in the projective space using a conjugate transformation and projective invariants. Finally, an image-space trajectory is obtained by re-projecting the coordinates of those points onto the initial image plane. A computer simulation and experiments are conducted to show the feasibility of the proposed scheme.

We present a new, method to select the optimal camera position for an active camera system from which camera pose (position and orientation) is modified according to the change of environment. We consider visibility and manipulability constraints to select the optimal position to monitor the target object and to modify the camera configurations during the task. The visibility constraint guarantees that a visual image of the target object can be obtained while the manipulability constraint guarantees that the manipulator can be controlled to locate the camera at the optimal position. From these considerations the optimal position and orientation of the eye-in-hand camera is selected and an eye-in-hand camera system automatically modifies its configuration during tasks.

Teleoperation is the extension of a person's sensing and manipulation capability to a remote location. Teleoperators can be generally modeled as a linear transfer function inherently including modeling uncertainties. Modeling uncertainties can make the system unstable and its performance poor. Thus we present a design framework for a bilateral controller of the teleoperator that achieves stability and performance in the presence of modeling uncertainties using /spl mu/-synthesis. Simulation and experiments are conducted to confirm the validity of the proposed method.

This paper presents a simple method for the calibration of a focal length and rotation parameters of a camera. We assume that the remaining calibration parameters of the camera change so little that they are considered fixed constants during a visual task. The employed vision system comprises a pan/tilt mechanism and a single camera, which is a typical setup for mobile robot navigation. The calibration process is performed by making use of vanishing points obtained from a rectangular pattern with an unknown size. Determination of vanishing points relies on a viewpoint invariant property in perspective transform-cross ratio. Focal length calibration is carried out by inner product of two vanishing point vectors. The rotation parameters are also estimated by projecting the image plane on the Gaussian sphere and extracting the surface normal vector of the calibration object. Possible errors in the calibrated parameters are studied by simulation and a preliminary result is presented.

Based on the similarity between prevalent fuzzy logic controllers (FLC) and the conventional robust controller, i.e., the variable structure controller, control theoretic analysis of a fuzzy control system is presented in the sense of Lyapunov. As well as the robustness of the fuzzy control system against uncertainties of a controlled process, this analysis gives an account of the relationship between control performance and the design parameters of the FLC, which has been obscure in the theory of fuzzy control.

This paper presents an iterative learning control scheme for high-geared industrial manipulators that perform repeated tasks. The input update law of the iterative learning control is given in the frequency domain, together with a sufficient condition for convergence of the iterative process. The proposed control scheme only utilizes locally measured signals and does not require the computation of complicated nonlinear manipulator dynamics. Moreover, on the assumption of repeating a task, unmodeled dynamics of a manipulator can be effectively canceled by the iterative learning control, without even a parametric model. Several important aspects of the iterative learning control scheme inherent in the frequency-domain design are discussed and the control performance is demonstrated through computer simulations.

Owing to load variation and unmodeled dynamics, a robot manipulator can be classified as a nonlinear dynamic system with structured and unstructured uncertainties. In this paper, the stability and robustness of a class of the fuzzy logic control (FLC) is investigated and a robust FLC is proposed for a robot manipulator with uncertainties. In order to show the performance of the proposed control algorithm, computer simulations are carried out on a simple two-link robot manipulator.

In this paper, a mobile robot is assumed to be equipped with a sonar and is capable of locating the acoustic reflectors. The acoustic landmarks are chosen from the acoustic reflectors whose positions are independent of the mobile robot position. By matching the two dimensional patterns of the acoustic landmarks and the acoustic reflectors that are newly recognized, the current mobile robot position is determined. A path is constructed by adding up the acoustic landmarks along which a mobile robot can navigate without error accumulation. Experiments were performed to verify the proposed method.

The authors point out that the proofs of Theorems 1 and 2 given in the above paper (Feng, 1995) are not correct and therefore the stability of the adaptive control laws proposed in the above paper is questionable at present.

Describes a method to extract multiple acoustic landmarks for the indoor navigation of a mobile robot. The environment is modeled by specular vertical walls. An ultrasonic sensor mounted on the mobile robot scans around the surrounding walls and detects multiple echo pulses. On the horizontal plane at the height of the ultrasonic sensor from the floor, the two-dimensional environment is described by the distance as a function of angle in the scan. Due to the specularity of the walls, the distance function has a manageable number of retroreflective parts that can be used as landmarks. By virtue of the constancy of the relative positions between the ultrasonic sensor and the walls in scanning, the echo pulses reflected from a certain reflection points (RP) have the same time-of-flight in the ultrasonic scan data. The authors determine the position of an RP from the collection of the echo pulses that have the same time-of-flight. The direction to an RP is estimated from the orientation of the ultrasonic sensor at which the maximum magnitude is obtained for the group of echo pulses that correspond to the RP. The mean time-of-flight of the group of echo pulses provides the distance to the RP. Some experimental results are provided to show the validity and the performance of the proposed method.

Proposes a novel local homing algorithm for indoor mobile robot navigation. In the algorithm, we divide the whole navigation task into simple local tasks in order to reduce the computational burden and the required memory size. We develop a new environment model based on the omnidirectional sensor data obtained from the Omnidirectional Range and Intensity Sensing System (ORISS), which consists of a set of ultrasonic sensors and a vision sensor. In order to enhance the reliability of the sensor information, we fuse the sensor data by means of the characteristics of the indoor environment structure and the sensor model. To verify the proposed algorithm, experiments with a mobile robot are carried out in a corridor.

This paper describes how to evaluate the performance of dexterity measures by investigating the properties of the inverse kinematic algorithms. These algorithms use redundancy to optimize the dexterity measure under the constraint of following a specified end-effector trajectory. To facilitate the investigation, a concept of the constraint locus is proposed. The constraint locus is the loci of configurations satisfying the necessary conditions for optimality of a dexterity measure in the configuration space. The topological properties of the inverse kinematic algorithm both on the configuration and operation spaces define the invertible workspace without singularities. Using the global properties of two dexterity measures, the authors provide a promising algorithm which provides a singular-free and conservative joint trajectory for tasks occupying almost the entire workspace.>

This authors point out that the proofs of the main results in the above paper by Su-Leung (ibid., vol.9, no.2, p.208-14 (1993)) are not correct. And rigorous proofs through more advanced stability results are presented.>

In the above paper by Su-Leung (IEEE Trans. Robot Automat., vol.9, no.2, p.208-14 (1993)), an adaptive sliding mode control scheme for the trajectory control of robot manipulators is developed. The proof of Theorem 1 in the paper is accomplished by resorting to the Lasalle theorem. It is well known that the Lasalle theorem can be applied to either autonomous systems or periodic nonautonomous systems, but it is not applicable to general nonautonomous systems. The Lasalle theorem is not applicable to the manipulator tracking control system given in the paper because the tracking control system considered in the paper is neither an autonomous system nor a periodic nonautonomous system but it can be classified into general nonautonomous systems. Therefore, the proof of Theorem 1 in the paper is not correct.>

A redundant manipulator can achieve additional tasks by utilizing the degree of redundancy, in addition to a basic motion task. While some additional tasks can be represented as an objective function to be optimized, other additional tasks can be represented as kinematic inequality constraints. In this paper, we reformulate a redundancy resolution problem with multiple criteria into a local constrained optimization problem, and propose a new method for solving it. The proposed method is especially efficient when the number of additional tasks is larger than the degree of redundancy. It also systematically assigns the priorities between the additional tasks. Besides of computational efficiency, the method has a cyclic property.>

A simple time delay method for avoiding collisions between two general robot arms is proposed. Links of the robots are approximated by polyhedra and the danger of collision between two robots is expressed by distance functions defined between the robots. The collision map scheme, which can describe collisions between two robots effectively, is adopted. The minimum delay time value needed for collision avoidance is obtained by a simple procedure of following the boundary contour of collision region on collision map. To demonstrate the effectiveness of the proposed time delay method, a computer simulation study is shown where a collision is likely to occur realistically.>

A numerical method for predetermined optimal redundancy resolution for a redundant manipulator is proposed. To resolve redundancy, a performance index is optimized globally in time. Instead of deriving the necessary condition for optimality and searching optimal boundary values, the authors predetermine the trajectories of redundant joints in terms of the Nth partial sum of Fourier series. Then the optimal coefficients of the Fourier series are determined using Powell's method. This results in an approximate optimal solution in a desirable homotopy class without topological liftings of paths. To show the validity of the method, the authors apply it to a three-link planar manipulator and analyze both optimal and extremal solutions using the fast Fourier transform (FFT).>

This paper proposes a measure construint locus to evaluate the properties of three dexterity meas- ures for singularity avoidance. The measure constraint locus is the set of configurations satisfying the necessary constraint for optimality of a dexterity measure in the configuration space. It gives a global look at the charac- teristics of dexterity measures in inverse kinematic algo- rithms imposing an optimality constraint function for a kinematically redundant manipulator, The invertible workspace without singularities and topological proper- ties both on the configuration and operational spaces are analyzed. We also discuss some limitations of the inverse kinematic algorithms and compare global pro- perties for each dexterity measure. And using the global properties for each dexterity measure, we will suggest a

A method for the motion control of a mobile robot in the presence of unknown moving obstacles as well as fixed obstacles is presented. The strategy consists of two planning stages: finding a collision-free path in known workspace and planning the velocity profile along the path to avoid a collision with moving obstacles measured through sensors on-line. In the path-planning stage, a neural optimization technique is proposed while considering distance, and safety between a mobile robot and fixed obstacles, and also considering the smoothness of the path of the robot. The optimization is accomplished with selected initial paths which are produced from a regional graph. For planning the velocity profile, the authors present a method of determining the acceleration of the robot by a neural optimization network approach. In this case, the acceleration of the robot is taken as variables of the neural network, and the time derivative equation is derived from the performance index which is the function of the velocity, acceleration, and distances between the obstacles and the robot.>

A manipulability constraint locus to characterize the performance of a subtask for a redundant manipulator is suggested. The manipulability constraint locus is the loci of configurations satisfying the necessary condition for optimality of the manipulability measure in the configuration space. Using the manipulability constraint locus, the topological property of the configuration space and the invertible workspace without singularities are analyzed. Through numerical example, some limitations of the inverse kinematic algorithms when one utilizes the redundancy are investigated. Finally, a promising algorithm which provides a conservative joint trajectory without singularities for almost entire workspace is proposed.>

The authors propose an optimal force distribution method, called weak point force minimization, which minimizes linear, one-norm, or infinite-norm objective functions. A weak point is a generalized one which can be any point in multiple robots as well as a joint and/or an end-effector. To solve the weak point force minimization, the weak point forces are converted to the contact forces of end-effectors so as to reduce the computational burden. Then, the contact forces are obtained by using linear programming. The proposed method is applicable to diverse areas such as a long object, weakly connected assembly parts long links and weak joints in cooperating multiple robots. The method is also applicable to the multilegged vehicle and the multifingered hand.>

In most cases, the joint trajectory for a point-to-point (PTP) task is generated from a given joint velocity profile. However, the given joint velocity profile may not generate a smooth joint trajectory and/or it may have undesirable frequency components for controlling a manipulator. The authors propose a PTP joint trajectory planning scheme in the joint space which generates a smooth joint trajectory profile for the given joint velocity profile. In order to exclude the undesirable frequency components, they approximate the given joint velocity profile in terms of the Nth partial sum of Fourier series by using the Fourier transform of the given joint velocity profile. The coefficients of the Fourier series are computed from the inverse kinematic solutions at each end-point of the given PTP tasks so that a smooth joint trajectory can be generated. The proposed method is applied to a PUMA 560 manipulator for the cases of the minimum energy and the minimum time. Also, the joint velocity profiles are analysed by the Fourier transform.>