“Grasping by caging” has been considered as a powerful tool to deal with uncertainty. In this paper, we continue to explore into “grasping by caging” and propose a new solution by using eigen-shapes and space mapping. For one thing, eigen-shapes fix dexterous hands into a series of finger formations and help to reduce dimensionality and computational complexity. For the other, space mapping builds a mapping between rasterized grids in 2-D Work space (W space) and rasterized voxels in 3-D Configuration space (C space) and helps to rapidly reconstruct C space so that we can efficiently measure the robustness of caging and find an optimal caging configuration for grasping. Our algorithm can work rapidly and squeezingly cage any 2-D shapes, including objects with either convex boundaries, concave boundaries, 1-order or high-order boundaries and even objects with inner holes. We implement the algorithm with MATLAB and carry out experiments with WEBOTS simulation to test its robustness to uncertainties. The results show that our algorithm can work well with various object shapes and can be robust to noisy control and noisy perception. It is promising in the power grasping tasks of dexterous hands.

In spite of IT innovation, people cannot get rid of non-creative tasks of searching daily-use objects at home. This paper presents a drawer-type storage system for supporting object search, “TansuBot”. By using this system and a smart device (e.g. smart phone), a user can review the photos of contents stored in the system. In addition, the system can present candidate drawers where the searching target object may be stored based on preliminary information such as usage histories. Concretely, LED blinking and pop-up actions (pushing drawers forward) are used for display. To realize these supports, a stacker crane type wall-moving robot is equipped at the backside of storage. The robot has a movable camera and mechanisms to push a drawer forward. For easy installation to a home, storage efficiency and cost reduction should be considered in the design of the instrument. Especially for cost reduction, this paper presents an approach to use wooden parts for main mechanisms. This approach also contributes to user-friendly presence and appearance of the instrument. This paper reports about the development of a prototype and an experiment to evaluate the functions for supporting object search. The results of the experiment prove the importance of the functions realized by the system; displaying contents' photos on a smart device and showing candidate drawers to investigate. The outcomes indicate that those functions have positive effects on reduction of searching time and mental burden.

Caging can offer robustness to uncertainties in grasping. If a robotic hand is designed based on the idea of caging, it would probably work well with noisy perception devices and low-quality control. This paper takes into account these merits and designs and implements a gripping hand based on the idea of caging. The gripping hand is concise and offers a low-cost alternative to co-operate with noisy data and low-quality control. According to previous work, we need four fingers to cage any 2D objects. That is to say, if each finger has one, two or three degree of freedoms, we will totally need four, eight or twelve actuators. The large number of actuators would be costly. This paper simplify the number of actuators into one by quantitatively analyzing finger formations with caging tests conducted on both random objects and objects from MPEG-7 shape database. It successfully lowers costs while maintains high performance. Following the simplified one-actuator design we implement a gripping hand by modifying a SCHUNK RH707 hand and carried out experiments with a manipulator built on the Neuronics Katana arm. The one-actuator gripping hand could work well with common depth cameras (Swiss Ranger) and pick up various objects. It bridges the gap between caging theories and applications and demonstrates the merits of caging.

Scores of health state for elderly people are regarded as important in nursing or medical fields. On the other hand, gaining the scores needs nurses to execute questionnaires. Owing to this, the execution rate for the health assessment is still low in ordinary homes. To solve this problem, we propose a method to predict the health score by using low-invasive sensors. We adopt regression as the prediction method and construct features to absorb the individual difference. As a part of feasibility study of social participation for elderly people, we execute the survey of health state using questionnaires by a nurse and install low-invasive sensors in real life simultaneously. Experimental result in the feasibility study shows a promise of the score prediction from sensor data. In addition, the result suggests that the extraction of features related to living behaviors improves the accuracy compared to using raw sensor data.

This paper presents a caging approach which deals with planar boundary clouds collected from a laser scanner. Given the boundary clouds of a target object and two fixed finger positions, our aim is to find potential third finger positions that can prevent target from escaping into infinity. The major challenge in working with boundary clouds lies in their uncertainty in geometric model fitting and the failure of critical orientations. In this paper, we track canonical motions according to the rotational intersection of Configuration space fingers and rasterize Work space with grids to compute the third caging positions. Our approach can generate the capture region with max(O(np),O(h2)) ≤ O(n2) cost where n denotes the resolution of grid rasterization, p denotes the resolution of canonical rasterization and h denotes the resolution of boundary rasterization or the number of boundary cloud points. Moreover, we propose a rough approximation which measures a subset of the possible positions by contracting rotations, indicating computational complexity of max(O(n),O(h2)). In the experimental part, our proposal is compared with state-of-the-art works and applied to many other objects. The approach makes caging fast and effective.

This paper presents a novel approach to deal with uncertainty in grasping. The basic idea is to initiate a caging manipulation state and then shrink fingers into immobilization to perform a practical grasping. Thanks to flexibility from caging, this procedure is intrinsically safe and gains tolerance towards uncertainty. Besides, we demonstrate that the minimum caging is immobilization and consequently propose using three or four fingers to manipulate planar convex objects in a grasping-by-caging way. Experimental results with physical simulation show the robustness and efficacy of our approach. We expect its leading benefits in saving finger number, conquering low-friction materials and especially, dealing with pose/shape uncertainty.

In this paper, we describe a novel sensor device which recognizes hand shapes using wrist contours. Although hand shapes can express various meanings with small gestures, utilization of hand shapes as an interface is rare in domestic use. That is because a concise recognition method has not been established. To recognize hand shapes anywhere with no stress on the user, we developed a wearable wrist contour sensor device and a recognition system. In the system, features, such as sum of gaps, were extracted from wrist contours. We conducted a classification test of eight hand shapes, and realized approximately 70% classification rate.

This research aims to develop a ceiling mobile robot that can sustain a large payload and can locomote freely under a ceiling space. In our approach, a perforated metal, one of the recent popular architectural materials, is utilized as a ceiling plate and the robot hooks and hangs the sequential holes of the ceiling plate by mechanical constraint. To realize the robot, the authors developed three key mechanisms, (1) ceiling hanging mechanism for the perforated metal, (2) horizontal locomotion mechanism like an inchworm, and (3) pantograph mechanism for smoothing horizontal locomotion speed and for load balancing. A unit testing of the ceiling hanging mechanism and the integration horizontal locomotion experiments confirmed the good performance of each mechanisms and the feasibility of mechanically constrained ceiling mobile robot. However, it was revealed that higher power motors and a stiffer body structure should be installed to sustain an extremely large payload like a human.

In this paper we propose a novel method for predicting resident's behaviors in a house from one's movement trajectories. The method consists of 1) segmentation of trajectory data into staying or moving and classification of the segments and 2) prediction by time-series association rules from transition events of each segment. The method predicts the start time of target behaviors for daily life support, such as eating, taking a bath etc. The time lag between the prediction and the target behavior can be set up manually, thus the method is adaptable to a variety of supporting systems. The experimental results using real residents' trajectory data of almost two years demonstrate that prediction of behaviors by the proposed method is feasible.

This paper describes a human shape reconstruction method from multiple cameras in daily living environment, which leads to robust markerless motion capture. Due to continual illumination changes in daily space, it had been difficult to get human shape by background subtraction methods. Recent statistical foreground segmentation techniques based on graph-cuts, which combine background subtraction information and image contrast, provide successful results; however, they fail to extract human shape when furniture such as tables and chairs are moved. In this paper, we focus on the results of face detectors that would be independent of such background changes and help to improve the robustness under movement of background objects. We propose a robust human shape reconstruction method with the following two characteristics. One is iterative image segmentation based on graph-cuts to integrate head position information into shape reconstruction. The other is high-precision head tracker to keep multi-view consistency. Experimental results show that proposed method has enhanced human pose estimation based on reconstructed human shape, and enables the system to deal with dynamic environment.

Vision based human articulated body pose tracking has been historically important. Because analyzing multiple human activities, especially interaction between human in cluttered scenes is essential in visual surveillance scenarios, multiple people tracking has enjoyed much attention in human robot interaction research in recent years. In this paper, we newly introduce a robust framework for multiple people pose tracking. The notable aspects of our approach are real-time ensuring speed (up to 30 fps), flexibility towards various complex motions and environments. Our work is inspired by the success of multiple view approach, especially voxel based techniques. The use of voxel data leads to viewpoint-free estimation, which benefits in that reconstruction of a training model is needless in different multi-camera arrangements. We add simple tracking-based volume segmentation algorithm to retain practical superiority of voxel based approach. Furthermore, our framework successfully obtains multiple body pose estimation in real-time even when people contacts with each other occurs in the scene, which is not addressed in the conventional approaches. We demonstrate the effectiveness of our approach with experiments on indoor cluttered scene sequences.

This paper describes an activity pattern mining method via inhomogeneous Poisson point processes (IPPPs) from time-series of count data generated in behavior detection by pyroelectric sensors. IPPP reflects the idea that typical human activity is rhythmic and periodic. We also focus on the idea that activity patterns are affected by exogenous phenomena, such as the day of the week, and weather condition. Because single IPPP could not tackle this idea, Dirichlet process mixtures (DPM) are leveraged in order to discriminate and discover different activity patterns caused by such factors. The use of DPM leads us to discover the appropriate number of the typical daily patterns automatically. Experimental result using long-term count data shows that our model successfully and efficiently discovers typical daily patterns.

This research aims to realize a container case hand-over task between robots. Although sophisticated cooperative control is essential to avoid destructive internal force in general cooperative transfer task, we propose a geometrical caging strategy which can simplify a hand-over task. To solve a problem of capture region mismatch during caging state transition, the proposed strategy does not utilize additional actuators and sensors but a compliant mechanism whose displacement range can be changed automatically by its configuration. By experiments, it was confirmed that the caging strategy can make the hand-over task drastically easy and robust, and a winch type quasi-compliant mechanism is effective to the problem of capture region mismatch.

A microwave Doppler sensor can monitor human motion without contact. It can sense wide range of motion from minute oscillation like respiration to large movement like walking because it measures the distance change between the target and the sensor as signal phase change. However, the proper method for the signal phase estimation is different between motion and respiration measurement. In this paper, we compare three methods for signal phase estimation and find the optimal method and parameters for each problem. For switching to the proper phase estimation method or monitoring human state, it is important to detect motion signal and respiration signal from the raw signal of sensor output. Three kinds of features, energy, frequency-domain entropy and histogram are extracted and are input into binary classifiers. We tested least squares, SVM and AdaBoost classifiers.

Laser based environment recognition technologies have been developed recently. Especially moving objects detection and classification by laser scanners mounted on a mobility is required for mobile robots and autonomous cars. In this paper, we propose a moving objects detection and classification method based on grid trajectories acquired from sequential laser scan data. Grid trajectories are obtained by voting sequential laser scan points on a grid map, and these trajectories not only work for a correct scan segmentation, but also represent the size and the speed of moving objects. We classify a moving object into either a person, a group of people, a bike, a car based on its grid trajectory. In our experiments, our mobility mounted laser scanners acquired scan data in the university campus, and the experimental results illustrate the effectiveness of the proposed method in outdoor environments.

This paper proposes an object movement detection method covering large areas of a room by using multiple cameras. When object movement detection for whole of a room is performed, there are several challenging difficulties: sizes of objects on the camera images are small, non-objects such as humans also exist on the images, objects are sometimes difficult to detect in specific viewpoints because of occlusion by humans or furniture or color similarity to near objects. In this work, to detect object movements robustly though the object sizes are small, we apply multiple view integration via features extracted from “stable changes” on each viewpoint. To discriminate between object and non-object, we focus on motion of changed regions. Our experiment in a room environment shows the multiple view integration method improves recall rate of object detection performance by about 0.2 when false positive rate is over 0.1.

In this paper, we propose a novel robust action recognition framework with the following capabilities: 1) online encoding motions to multi-label sequence where the output in each frame is a tuple of labels rather than a single label, 2) providing efficient automatic relevant motion selection framework, 3) learning systems so as to be optimal for online multi-label sequence classification. As for multi-label classification, our approach incorporates contextual information about action not only temporal information but hierarchical information of actions. Inference tends to be complex so as to achieve such complex recognition scheme, however, we propose an efficient Viterbi-like decoding algorithm which integrates forward algorithm and loopy message passing algorithm. As for the learning process, the algorithm optimizes the parameters so as to maximize log likelihood of the model. Boosting, ensemble approach of machine learning, is leveraged to provide efficient feature selection framework in the training process. The experimental results show that the proposed method successfully exploits the impact of contextual information then significantly outperforms the traditional approaches in dynamic gait motion classification.

In this paper, we present a novel framework to recover human body pose on multi camera systems. Our framework leverages 3D voxel data, which are reconstructed from multi-camera systems. The use of voxel data leads to viewpoint-free estimation, which benefits in that reconstruction of a training model is needless in different multi-camera arrangements. Other notable aspects of our approach are real-time ensuring speed (up to 30 fps), flexibility towards various complex motions and environments. We treat the pose estimation problem as estimating human pose label from the voxel features and tackle this by example based approach. To ensure the real-time speed and to improve precision of pose estimation, a newly fast and robust near-neighbor search metric is installed prior to the evaluation process, what we call CSI-PSH. We demonstrate the effectiveness of our approach with experiments on both synthetic and real image sequences.

Laser based tracking systems have been developed for mobile robotics and intelligent surveillance areas. Existing systems estimate only human positions. In this paper, we propose a method for human pose estimation represented by human head and waist position using only laser range finders. Two features of human cross-sectional contours are extracted from laser scanner data scanning on the height of waist. This method estimates human pose by using these features in the Bayesian filtering framework. Moreover, we develop a new particle filter framework with two transition models and two resampling steps. In this framework, position estimation and pose estimation are performed by many hypotheses. Our experimental results demonstrate the effectiveness of the method in pose estimation of multiple people by using only several laser scanners.

This paper presents three behavior labeling algorithms based on supervised learning using accumulated pyroelectric sensor data in the living space. We summarize features of each algorithm to use them in combination matched to usage of the livelihood support application. They are (1) labeling algorithms based on time attribution of ldquoon-offrdquo data, (2) one based on Hidden Markov Models, and (3) one based on switching model around a behavioral change-point. We show the behavior labeling results of three algorithms for one month data under the same conditions. Then we point out features on the basis of these results.

In this paper, a novel statistical indoor activity recognition algorithm is introduced. While conditional random fields (CRFs) have prominent properties to this task, no optimal performance is obtained due to the fact that the performance is optimized for offline estimation. Furthermore, no previous researches provide efficient training process to optimize classifiers in on-site recognition perspective. In this paper, we propose a novel sequence estimation model suitable for online activity recognition, what we call Just-in-Time random fields (JRFs). In JRFs, efficient training and feature selection process is provided via boosting. Empirical evaluation using synthetic and real indoor activity records shows that our model drastically outperforms the previous methods in view of the classification performance with respect to the training cost.

This paper describes development of home-use automated container storage/retrieval system, a sub-system of a logistical support robot system in living space. The s/r system has 3 features. (1) Elevator type structure enables not only auto stocking motion, but also human access to containers as if the auto rack is a general shelf. (2) A separated motion layout enables us to utilize upper ceiling space as a stock space, and decreases possibility of accidental stuck by reducing the volume of robotpsilas motion. (3) The collaboration of a container guide plate and sensing with RFID actualize to navigate people to place a container on a shelf with desired position and posture. By experiments, it is confirmed that the ACSRS can transfer a container and 5 [kg] load (total 8 [kg]). We summarize significant technical points to design a system of this type.

This paper describes an algorithm of behavior labeling and anomaly detection for elder people living alone. In order to grasp the personpsilas life pattern, we set some pyroelectric sensors in the house and measure the personpsilas movement data all the time. From those sequential data, we extract two kinds of information, time and duration, and calculate two-dimensional probabilistic density function of them. Using this function, we try to classify behavior labels and detect anomaly. Here, we assume two kinds of anomaly, ldquothe rare behaviorsrdquo and ldquothe changes of life patternrdquo. The algorithm is confirmed to work on real behavior data through the experiment on about 400 days data.

In this paper, we propose a robust recognition and segmentation method for daily actions with a novel multi-task sequence labeling algorithm called multi-task conditional random field (MT-CRF). Multi-Task sequence labeling is a task of assigning input sequence to sequence of multi-labels that consist of one or multiple symbols in single frame. Multi-Task sequence labeling is essential for action recognition, since motions can be often classified into multi-labels, e.g. he is folding arms while sitting. The MT-CRFs: extensions of conditional random fields (CRFs), incorporate jointly interaction between action labels as well as Markov property of actions, to improve the performance of the joint accuracy: the accuracy for whole labels at specific time. The MT-CRFs offer several advantages over the generative dynamic Bayesian networks (DBNs), which are often utilized as multi-task sequence labelers. First, the MT-CRFs allow relaxing the strong assumption of conditional independence of observed motion, which is used in DBNs. Second, the MT-CRFs exploit the power of non-Markovian discriminative classification frameworks instead of generative models in DBNs. With deep insight of the problem Multi-Task sequence labeling, the inference process of the classifier gains more efficiency than the previous Markov random fields that tackle multi-task sequence labeling. The experimental results show that classifiers with MT-CRFs have better performance than cascaded classifiers with a couple of CRFs.

The authors proposed a methodology of assisting the patients who were suffering from mobility impairment of finger and thumb joints, with exoskeletal devices. The types of exoskeletal assistive devices for such patients were categorized into several groups based on required functions which are derived from the patients' condition. One of the categories, "joint motion compensator" is a novel concept taking into consideration the difference between active-ROM (range of motion) and passive-ROM of joints, and with regard to one of subgroups of this category called "joint motion amplifier", a prototype device which amplifies the motion of index-finger PIP (proximal interphalangeal) joint was constructed and verification experiments for pinching objects were conducted. The results revealed that it was feasible to pick up and pinch objects which were not so small or thin enough to be picked without nails, and also was possible for the device to hold such objects as strongly as human hands do.

This paper reports development of an intelligent container, a unit of the logistical support robot system in living space. The container not only supports human to fix and arrange their commodities, but also supports robots to achieve their logistical tasks. In other words, the intelligent container is a mediator between human's request and robots' capabilities. The container has four major roles in our logistical support system. (1) Taking several commodities in and recognizing and acquiring contents' information. (2) Supporting human and robots to transfer the container. (3) Receiving users' command as a simple user-interface. (4) Communicating to a host computer and informs the contents of the container and users' command. To realize those roles, the container is equipped with a grasping navigation mechanism for robot handling, RFID reader for the recognition of contents, and LCD device to display supporting information to human and robot. Finally, a primary function is evaluated in experiment to confirm that an acceleration sensor enables the container to detect being transferred by human. In the future, the intelligent container will play a role of contact point for human and robots in our target system, that enables human-robot symbiotic life.

In this paper, we developed a tiny orientation device equipped with a wireless network function for a wearable motion capture. The wearable motion capture is defined that it not only measures the posture of the human body but also collects environmental information and the human’s internal state simultaneously and easily. Because the realized device automatically configures wireless networks and is small enough to attach it anywhere, it is easy to gather any sensor information. The feature of the orientation estimation method is that models are switched according to the environment to exclude the effect of motion disturbances. In experiments, by integrating of sole sensors and the orientation sensors, walking around, walking up and down, and jumping tasks were successfully measured. Because it is difficult to measure these motions with only the inertial sensors, it showed the importance of integrating various sensors for acquiring human motion.

In this paper, we propose wrapped boosting that is extension of boosting algorithm for robust online action recognition. Boosting algorithm is one of ensemble learning algorithm and is also known as a feature selector. In our previous work utilizing boosting, we achieved automatic feature selection and robust model-based action classifiers which had very small calculation cost based on posture information of human body joints. However, which joints we should allocate posture sensors to must be given by humans in advance. Our new learning framework of wrapped boosting provides not only automatic feature selection but also automatic sensor allocation to proper joints of humans for target actions. We evaluated our algorithm targeting gait motion based on motion data fetched by motion capturing system. In consequence, wrapped boosting was able to select proper joints to which limited sensors should be attached, and to construct more robust classifiers compared to constructing classifiers with all joints available. Classifiers constructed only with existing boosting algorithm were subject to over-fitting to training data.

In this paper, a novel approach is proposed to recover human body pose from 3D voxel data. The use of voxel data leads to viewpoint-free estimation, which benefits in that reconstruction of a training model is needless in different multi- camera arrangements. Other notable aspects of our approach are real-time ensuring speed (up to 30[FPS]), flexibility towards various complex motions, and robustness towards voxel noise. The main concept of our approach is based on an example based approach. Human posture candidates are constructed beforehand from a large motion capture database, and the most appropriate posture is estimated per frame by comparing the likelihoods between 3D voxel data and posture candidates. The evaluation is formulated by introducing a histogram- based feature vector that represents the 3D shape context of human body. In addition, a fast near-neighbor search metric is installed prior to the evaluation process, in order to reduce the computational cost and ensure real-time processing. Estimation stability is also improved by a graphical model of motion, which adds a smoothing effect to the motion sequence. We demonstrate the effectiveness of our approach with experiments on both synthetic and real image sequences.

This paper describes automatic generation and linking mechanism of sensor processing components, which are called sensor services, in network middleware for distributed sensors in a room. This mechanism reduces deployment process in the creation of applications for real sensorized environments. The mechanism consists of 1) dynamic program duplication triggered by matching generation condition with descriptions of components and 2) sensor data selection based on selection conditions and the descriptions. The description, which is called RDF sensor description, is written in resource description framework (RDF) for definition of sensor properties including the knowledge about sensor-attached object. The conditions for RDF sensor description are scripted flexibly with vocabulary of abstract sensor knowledge. Through the conditions, the components are generated dynamically according to change of the sensor composition. The generated components can select needed sensor data in multiple sensor environment. We create sensor services that detect certain behaviors from raw sensor data in our sensing room. The services show generation mechanism's independency on configurations of sensors in deployment of the components in the middleware

This paper presents a human like segmentation method for daily life actions, such as getting up, sitting down, walking. Unsupervised segmentation methods of many previous researches cannot always assure segmentation result that coincides with human's natural sense. While the proposed method utilizes human's teacher data of segmentation to conduct human like segmentation. We assume that latent dynamics changes at the segmentation points of action, and represent segmentation boundary by switching model of two linear dynamic systems. The problem is that human may segment actions according to wide variety of criteria depending on the attention point or other backgrounds. In this paper, those criteria are acquired by clustering segmentation boundaries extracted from teacher data made by human. Each of the cluster is characterized by body parts it pays attention to. Here, we focus on hierarchical aspect of human body that human body can be treated at various levels of abstraction (e.g. whole body, upper body, left arm), and represent it by tree structure. Experimental result shows that the proposed method can acquire human like segmentation criteria

In this paper, we propose a margin-based query learning algorithm for action recognition to reduce a laborious work on annotating action labels of time-series motion. The annotation is an inevitable task for designers of recognition systems with supervised learning techniques. Query learning is a kind of compensation approach for this, and can also be categorized into interactive learning. Our algorithm is a natural extension of maximum margin learning; a.k.a. support vector machines. Thanks to the theoretical analysis of the optimal condition of the maximum margin learning, the algorithm runs with a single and simple criterion. To prevent poor performance of the classifier learned with very few size of labeled motion data set, the algorithm exploits cluster information of massive unlabeled motion dataset. In contrast to the previous margin-based query learning methods, the algorithm has superiority in terms of stability. The empirical evaluation using real motion and synthetic dataset shows that our algorithm can achieve both drastic reduction of annotation cost and making robust classifiers

In this paper, we propose a fast and robust online action recognition method. The main features of the proposed method are: 1) to select a small number of critical motion features from a very large set of motion feature templates and to release humans from task of designing critical motion features, 2) to require very small calculation cost for recognition compared to conventional methods, 3) to exploit "combinational motion features" which we propose as a new conception so as to construct a robust action recognizer. We evaluated the proposed method to gait action recognition, such as walking and running, by utilizing motion capture data. In the result, the proposed method reduced parameters given by human to action recognizer and lessened human's task. In addition, the proposed method needed very small calculation cost for recognition, and can recognize robustly as much as conventional action recognition method based on support vector machine. Moreover, the introduction of combinational motion features enhanced recognition performance

This paper proposes an imitation learning system to generate trajectories by which a robot supports a human with close physical assistance adapting to human movements and daily life environments. The proposed system is composed of 1) division algorithms, 2) learning algorithms and 3) assistance algorithms. 1) In division algorithms, the system measures time series of human task execution data and divides them into multiple motion segments automatically. This division is based on standard deviations of motion errors between measured trajectories and an ideal trajectory where the ideal trajectory is mean of all measured human trajectories and is expected to achieve the purpose of human task successfully. Since an important motion parameter is paid attention to by the human and has small standard deviation of errors, series of measured data are divided into segment motions at the points where the importance of parameters changes suddenly. Thus this division is guaranteed to accord with human attention. 2) In learning algorithms, the system learns trajectories with dynamic neural network (DNN). Since the DNN has convergence, generated trajectories can converge to an ideal trajectory. The importance of each parameter, in other words how much attention human pays to the parameter, is evaluated as how small the standard deviation of errors is. The DNN learns trajectories reflecting the evaluated importance of parameters to accord with human feeling. 3) In assistance algorithms, the system judges when to start assistance by the assumption of multiplied errors of motion parameters by the respective importance. In assistance algorithms, the system also connects generated trajectories of motion segments smoothly. An experiment to support human drink task was performed successfully where the proposed system judged not only when to start assistance to the task but also execute assistance when a cup was about to incline too much not to spill water

In this paper, we propose a novel kernel computation algorithm between time-series human motion data for online action recognition. The proposed kernel is based on probabilistic models called switching linear dynamics (SLDs). SLD is one of the powerful tools for tracking, analyzing and classifying human complex time-series motion. The proposed kernel incorporates information about the latent variables in SLDs with simplified designing approach called marginalized kernels. The empirical evaluation using real motion data shows that a classifier using SVM with our proposed kernel has much better performance than the classifier with some conventional kernel techniques. Another experiment using walking around motion shows that a classifier with the proposed kernel can properly segment the start and the end of the target action.

In this paper, we constructed wireless ad hoc network for realizing various physical and informational support systems based on Lifelog that is the records of experiences in the daily life, and realized a prototype of electric appliances operational support system which is one of useful systems. Utilization of Lifelog reduces the burden for controlling a huge amount of convoluted electric appliances by constructing the probabilistic model of user's operational behavior based on Lifelog and predicting user's successive operations with this model. In this time, Lifelog accumulates user's operational behavior for electric appliances and environmental information through the wireless network. For the basis of collecting Lifelog including operational behavior, we made a portable Bluetooth-equipped device for wireless network which is necessary to communicate information in the ubiquitous computing environment because Bluetooth has good features such as the ad hoc networking function, sufficient data throughput, high resistance to noise and low power consumption. The realized device has abundant 110 connectors which are able to connect various sensors and actuators. By attaching these devices to electric appliances, they can easily participate in the wireless network and communicate information. Therefore the system can probabilistically model user's behavior, especially operations for electric appliances using Lifelog as training data, and predicts the user's next successive operations for surrounding appliances utilizing this model and the user's present state. The system gives prediction results to the user and executes operations via the wireless network after the user's confirmation. The various experimental results prove that the operational system in ubiquitous computing environment is useful and our realized device is sufficient performance in the daily life.

In this paper, we propose a human posture reconstruction method from the insufficient input posture data based on human posture probability density that is constructed by a long-term human motion capture data. Since the long continuous daily human motion data has high dimensions and becomes huge size, the human posture data should be effectively compressed. The long term posture data has nonlinear distribution on the posture space, since each specific posture such as standing and sitting has different property. The posture data is allocated into some subspaces and compressed for each subspace with mixtures of probabilistic principal component analyzer (MPPCA). MPPCA is improved by replacing conventional EM algorithm with deterministic annealing EM algorithm (DAEM) to avoid initial parameter sensitivity. The posture probability density is constructed over those subspaces. The adequate human posture can be reconstructed from the insufficient data by introducing the posture probability density into the sequential Monte Carlo framework. The experimental results show that the robust human posture estimation can be realized since this method does not estimate the unique posture but estimates the proper posterior posture density with using the posture prior knowledge.

In this paper, we propose a robust online action recognition algorithm with a segmentation scheme that detects start and end points of action occurrences. In other words, the algorithm estimates reliably what kind of actions occurring at present time. The algorithm has following characteristics: 1) The algorithm incorporates human knowledge about relation between action names in order to simplify and toughen the algorithm, thus our algorithm can label robustly multiple action names at the same time. 2) The algorithm uses time-series action probability that represents the likelihood of each action occurrence at every frame time. 3) The classification technique with hidden Markov models (HMMs) enables the algorithm to detect robustly and immediately the segmental points. The experimental results using real motion capture data show that our algorithm not only decreases effectively the latency for detecting the segmental points but also prevents the system from making unnecessary segments due to the error of time-series action probability.

This paper proposes a behavior prediction system for supporting our daily lives. The behaviors in daily-life are recorded in an environment with embedded sensors, and the prediction system learns the characteristic patterns that would be followed by the behaviors to be predicted. In this research, the authors applied a method of discovering time-series association rules, which discovers frequent combinations of events called episodes. The prediction system observes behaviors with the sensors and outputs the prediction of the future behaviors based on the rules.

In this paper, we develop an absolute orientation estimation device equipped with a wireless network. Accelerometers and magnetometers are used to measure the gravity and the geomagnetic field respectively. Gyroscope sensors are used to measure the local angular velocity. The geomagnetic field varies according to the environment. Therefore the device can obtain the information about the magnetic field through the wireless network. The orientation estimation task can be also requested to the other computers with the wireless network. By integrating the measured gravity and geomagnetic field with the local angular velocity using Sigma-Points Kalman Filters (SPKFs), the stability and the robustness of estimating the absolute orientation are improved over either sensor alone. We also propose an estimation method which excludes the effect of motion and magnetic disturbances for the accurate estimation.

This paper proposes a novel algorithm for extracting informative motion features in daily life action recognition based on support vector machine (SVM). The main advantage of the proposed method is not only to extract remarkable motion features, which fit into human intuition, but also to improve the performance of the recognition system. Concretely speaking, the main properties of the proposed method are 1) optimizing kernel parameters so as to minimize its generalization error, 2) extracting remarkable motion features in response to the sensitivity of the kernel function. Experimental result shows that the proposed algorithm improves the accuracy of the recognition system and enables human to identify informative motion features intuitively.

This paper presents a ceiling mobile robot platform that enables multiple robots to move with low possibility of interference with humans and obstacles in an intelligent environment. An advantage of this platform is the potential to reduce sharing space between humans and robots while maintaining the robots' access to humans when their supports are needed. To realize the platform, two key techniques are developed. The first one is permanent magnet inductive traction method for the robots hanging under the ceiling plate. The second one is multiple robots simultaneous position measurement method using matrix of two dimensional codes. Validity of permanent magnet inductive traction method is confirmed by an experiment using a mock-up ceiling plate. Repeatability of position estimation using matrix of two dimensional codes is demonstrated by evaluation experiment. Finally as a demonstration of human support by the platform, transportation task is successfully performed.

Many diverse methods have been developed in the field of biometric identification as a greater emphasis is placed on human friendliness in the area of intelligent systems. One emerging method is the use of footprint shape. However, in previous research, there were some limitations resulting from the spatial resolution of sensors. One possible method to overcome this limitation is through the use of additional and independent information such as gait information during walking. In this study, we suggest a new person-recognition scheme based on the center of pressure (COP) trajectory in the dynamic footprint. To make an efficient and automated footprint-based person recognition method using the COP trajectory, we use a hidden Markov model and a neural network. Finally, we demonstrate the usefulness of the suggested method, obtaining an approximately 80% recognition rate using only the COP trajectory in our experiment with 11 people. © 2004 Wiley Periodicals, Inc. Int J Int Syst 19: 1127–1141, 2004.

In this paper, a novel describing method of human locomotion trajectory record was proposed, and methods of its application to the path planning and control of human-symbiotic mobile robots were indicated. The novel describing method, named as Existence Record Probability Map (abbreviated ERPM hereafter), was generated through the following three steps: 1) measuring human locomotion path using pressure distribution sensor floor, 2) applying a probability potential model of human body to the measured path, and 3) integrating such probability potential maps for some time period. From the ERPM, an existence probability value, ridge lines of the potential, and the Q-value at a point on a ridge line, were elicited. As a usage of these information, methods of finding a position where a collision probability of a mobile robot and a human is high, finding major trajectories of human locomotion, and evaluation of path width to avoid collision with a human, were indicated.

This paper proposes robotic kitchen counter, a novel kitchen counter with sensors to monitor a user's action and with the capability of changing its top plate height according to his preference. The robotic kitchen counter was realized by installing various kinds of sensors to the commercial kitchen counter and by putting a jack mechanism underneath. The sensor data acquisition and the top plate height setting can be done through network. Cooking experiments with this robotic kitchen counter showed that 1) there's an optimal kitchen counter height according to an individual person and to his action during cooking behavior, and that 2) the dynamic reappearance of the kitchen counter height he manually adjusted in the stages during the previous cooking behavior was recognized handful, which is the typical example of the simplest action-adapted and personally-fit assistance. The outstanding point of this robotic kitchen counter is that it can measure the cooking action without disturbing the cooking behavior itself while providing action-adapted and personally fit assistance based on the data obtained. The authors believe that the hardware and software techniques obtained through this development can be the starting point to develop other action-adapted and personally fit assistance system.

This paper proposes a network middleware for flexible, easy and uniform utilization of sensors in a room. The middleware has three features; 1) concentrated network configuration, which consists of sensor services and a management server, achieves network transparency for flexible and uniform sensor utilization. 2) Abstraction of sensor information using XML description makes easy utilization and uniform access to multiple sensors. 3) Management of shapes as well as position information is introduced in the network enabling the search function based on the room information and management of spatial data. Not only the implementation for sensors and a management server but also an application software by the middleware show validation of the proposed network middleware.

This paper proposes an imitation algorithm for a robot to acquire typical tasks from multiple measured data of human tasks in the daily life. The algorithm consists of the following procedures: 1) Firstly, the system measures multiple human's object-transferring tasks on a table. Then it calculates qualitative description from measured raw data of positions of human hand and object, as well as the force applied to the table. This description is then converted to probabilistic description. 2) Secondly, the system finds typical human tasks with the maximum likelihood from the probabilistic description. 3) Thirdly, the trajectory, which enables the robot to imitate the typical human task, is extracted. 4) Finally, the imitation task within limited force to the environment is generated from the trajectory by simulation and adaptation. The experimental execution of the generated trajectory proves the validity of the algorithm.

This paper proposes a behavior log creation support system utilizing human pose/position sensors and behavior target sensors. The system is equipped with a human pose sensor and a staying room sensors as the pose/position sensors as a well as a voice sensor and a PC utilization history sensor to detect the target of the behavior. The human pose sensor classifies human behaviors into "standing", "sitting", and "walking". The staying room sensor records a name of the room where the user stays. The voice sensor detects the conversation which appears when the user is communicating with someone else. The PC utilization history sensor records not only whether a PC is used or not but also the names of the application software to serve as a detector of the behavior target during the computer work. The measured data from these sensors is displayed to the user to support creating the behavior log, i.e. to support the user to recall and to input contents and targets of his or her behaviors. The experiment of making use of the sensors and creating the behavior log proved that the recorded events of the behavior log per all events improve from 60% with no support to more than 90% with support of the system. The result quantitatively shows the capability of human behavior logging support of the system.

In this paper, we developed a bed-ridden human's body movement unrestraint estimation system by using a pressure sensors distribution bed. We classified body movements into gross and slight movements. In order to estimate both gross and slight movements, we realized the distinction methods between a human and an object and between sitting and lying status. We also realized the estimation methods of a posture, an articular movement, the respiration and the pulse. By integrating these methods to complement each other, a bed-ridden human's body movement from gross to slight movements can be estimated totally.

This paper proposes a human-like action recognition system which can output the result of human action recognition just like the case human does. The system targets actions associated with regular human activity such as walking or lying down, and uses three human recognition characteristics: using specific features of an action to recognize that action; recognition of simultaneous actions; and summarization of recognition results over a short time interval. Experimental results demonstrate the effectiveness of human-like recognition for identifying actions and the superior performance of the proposed system with respect to conventional action recognitions systems. Human-like recognition is expected to ensure smooth communication between humans and robots and enhances the support functionality.

This paper proposes a high resolution sensor floor which can detect both humans and robots simultaneously. Each sensor floor unit is 500mm square and is equipped with 4,096 pressure switches distributed in a 64 /spl times/ 64 array. A 2m by 2m sensor floor with 16 of these sensor floor units has been realized. Experiments with this sensor floor have determined successfully the positions of a human and a 4-wheeled cart and can distinguish between them. The distinction is easily achieved because of the high resolution of the floor. The modular structure of the sensor floor enables easy application to a real room of irregular shape, allowing unconstrained measurement of the locations of humans, robots and objects in the room. Consequently, sensor floor systems will be essential components of future human-robot symbiosis systems to assist humans in daily life, and will also play important roles in medical and welfare robot systems.

This paper describes a data accumulation system that records continuous sensor data related to human behavior. The system collects sensor data in a room and stores the data in a database via a network. The data is stored in a hierarchical table structure of raw sensor data and processed data in a time series according to a sensor module descriptor that contains information regarding the sensors and output in the sensor module. This allows any type or combination of sensors in a sensor module to be connected to the network dynamically. In addition, three network configurations are proposed for controlling network load and storage. The system is evaluated through experiments, and is shown to exhibit no more than 1% data loss and capture data that can be used effectively to reproduce or analyze human behavior. Such accumulated data will make it possible to model human behavior and customize human support systems based on long-term behavioral trends.

This paper proposes a plant-shaped robot object as an awareness driven display device of human daily life information. While a conventional monitor display (operation driven display) device transmits information to a person when it is operated intentionally (actively) by the person, the awareness driven display device transmits the information to the person when the device is made aware by chance (passively) by the person with a change of its appearance. The constructed display device is designed taking account of a houseplant. It consists of 12 leaf modules and a flower module. It displays the information by means of leaf poses and extent of flower opening. The leaves can also sense a touch action. A display experiment was conducted to evaluate the device. The activity of a remote target person is calculated from actual sensed daily life data and displayed to the subject person. Experimental results show the effectiveness of the display system. It reveals that a larger change of pose and flower opening evokes more effective awareness of the activity of the person. This display device in the future will be able to be utilized as a communication tool to generate a virtual feeling of cohabitation even though the remote target person lives apart from the subject person.

A lying person's motion tracking system by using a pressure distribution image and a full body model is proposed. The full body model consists of a skeleton and a surface model to cope with a variety of body shapes. BVH files are used as the skeleton model that describes a hierarchy of joints and links. Wavefront object files are used as the surface model that describes geometry of the surface. The bed has 210 pressure sensors that are under the mattress. It can measure a pressure distribution image of a lying person. The lying person's motion is tracked by considering potential energy, momentum and a difference between the measured pressure distribution image and a pressure distribution image that is calculated by the full body model. Experimental results reveal that the realized system can track not only horizontal motions such as opening and closing legs but also vertical motions such as raising the upper body.

Since human behavior patterns vary from person to person, computers have to understand someone's personal behavior in order to supply him with a robotic support system for his daily life. It is necessary to know his behavior over the long term. The ideal intelligent support system can measure the data about human actions in the daily life, accumulate them, understand the behavior and the patterns thereof, and adapt thereto. An accumulation and summarization system of human daily action data based on sensing in 1 room, "Robotic Room II", is designed and the accumulation system integrating with the measurement parts is implemented. It is confirmed that accumulation over the long term is possible by experiments for accumulation of human action data in ordinary life. Moreover, as the first step of an analysis of the accumulated data, a summarization algorithm based on the changes of the room states is proposed. It is confirmed that the summarized data enables us to catch the most of the changes of the room states. Finally, a possibility of application for life monitoring by means of summarization is also illustrated.

This paper proposes a human-like action recognition system which can recognize human actions like the human does. The recognition algorithm is constructed taking into account of the following characteristics of human action recognition: simultaneous recognition, priority among actions, fuzziness of judgment, multiple judge conditions for one action, and recognition ability from a partial view of the body. The experiment shows that the system can recognize human action like human does. The stated results ensure a natural understanding of human action by a system, which leads to smooth communication between computer systems and human.

Techniques of arranging micrometer-sized objects with high repeatability under a scanning electron microscope (SEM) are required to construct highly functional micro-devices. Since micro-objects tend to adhere to other objects mainly by electrostatic force, the adhesive force is often being considered in basic researches of micro-object manipulation. These techniques, however, still do not achieve the desired repeatability, since many of them are being used just for the empirical effectiveness. Some even include the process of trial-and-error. Thus, we analyze the micro-object operation theoretically by considering the adhesion and rolling-resistance factors in a dynamic system consisting of a sphere, needle-shaped tool, and substrate. Through this analysis, we show that it is possible to control the fracture of the contact interfaces of the system selectively by determining the tool-loading angle appropriately. Based on the analysis, we also propose a practical method for the pick and place operation of a micro-sphere under an SEM.

The authors developed a novel infant behavior recognition system based on a pressure distribution image. The system can recognize an infant's status (quiet, moving and crying), posture, body parts' positions and movement unrestrainedly. It can, in recognizing the behavior, cope with the infant's rapid growth and unique physique. The algorithm of the infant behavior recognition system is summarized as follows. 1) First, the system measures the pressure distribution image with 384 pressure sensors distributed in the bed. 2) The authors propose "activity score"; this is calculated by using the measured pressure distribution image and indicates kinetic energy of the infant's activity. Based on the activity score, the system decides the infant's status. 3) If the infant is quiet, the system estimates the infant's physique. 4) Based on the estimated physique, the system recognizes the infant's posture and body part movement. Experimental results reveal that the system successfully recognizes infants' status (quiet, moving and crying), posture, body parts position and movements.

This paper presents "Sensor Pillow System" to measure physiological parameters in sleep without restraint to a human. The system consists of an array of pressure sensors under the pillow, a one-chip microcomputer to digitize and transmit the pressure data to a desktop computer; and the computer to count respirations and turns in sleep. This paper also presents a simple motion model which explains the change of the head pressure distribution accompanied with respiration. Based on this model, respiration count algorithms is proposed. The effectiveness of this system is experimentally shown by comparing the number of respirations and turns counted by the sensor pillow system of a medical device and a video image.

A one-room-type sensing system named Robotic Room 11 for recognizing and accumulating human behavior in ordinary life is constructed. In order to recognize human behavior, we propose an idea that human behavior can be estimated from the human position, posture, the objects surrounding the human and time. Based on this idea, multiple sensors are distributed on such objects as floors, beds and desks. From these sensors, information of a human position and use conditions of household appliances are extracted and integrated. Experiments of sensor information description are successfully performed to show that the system can recognize and accumulate human behaviors.

We develop a body parts position and posture estimation system. This system consists of a pressure sensor distributed bed and body parts position and posture estimation software. The computer constructs many pressure distribution images based on the simple human models and accumulates these images to its memory, called the model-based pressure image templates. The pressure distribution image through is compared with model-based pressure image templates to find out the most matched pressure image template. The model-based pressure image templates contain body parts positions and joint angles information, so the body parts position contacting with the bed can easily be estimated by using the most matched model-based pressure image template. Finally, an estimated posture is displayed as a 3D computer graphics image. An experimental result reveals that the system not only can display the estimated lying human posture intuitively, but can also estimate the lying human's body parts position accurately where the body parts contacts with bed.

There is a great demand for the highly accurate and reliable arrangement of micro objects smaller than 100 /spl mu/m in order to construct micro devices. Since micro objects tend to adhere to other objects by electrostatic force, it is possible to pick them up easily by contact with a needle tip instead of grasping by tweezers. On the other hand, it is difficult to place them on a substrate. To solve this problem, we have proposed a handling method by controlling the contact face area, i.e. picking up the object by contact with the center of the tool-tip plane, and placing it by contact with the edge and also inclining the tool. However, it is difficult to execute this operation by manual control, because it requires delicate movement by the manipulator, in order not to break the object or flip it away. In this study, we automate this pick-and-place operation by visual and force control. Moreover, to arrange micro objects with high accuracy and reliability, all necessary functions such as calibration, object search, and positioning are integrated, and an automatic handling system is constructed. We successfully demonstrated a completely automatic arrangement of several micro objects of 30 /spl mu/m in diameter under SEM monitoring.

Visually based control, accumulation and assistance systems are presented as an effective environment for teleoperation. We developed the bilateral behavior media (BBM) paradigm and implemented systems which offer a status driven interface, collection of sampled control behavior, and operator assistance. The paradigm emphasizes a human-intuitive visual specification of task completion states without resorting to image understanding, modeling or extensive calibration. Visually based teleoperation has been effectively applied in a variety of tasks where an operator's past experience in the macroworld is not applicable to the physics and scenario experienced in the microworld. Experiments of manipulating individual biological cells have shown the success of the visually based BBM paradigm.

The basic strategy for reliable and precise pick-and-place operation is shown based on the actual measurement of adhesion under SEM. Dynamics for pick-and-place operation of a microsphere by a needle-shaped tool is analyzed. A new method of reliable pick and precise place is proposed on the basis of the basic strategy and the dynamics. An experimental system for executing the proposal operation is constructed based on the numerical-control of the position measurement of the target object. In this system, many numerically-controlled experiments with 2 /spl mu/m polystyrene spheres reveal that under the optimal conditions, the pick operation has never failed once out of over 20 trials and the place operation has 140 nm precision in position. Consequently, we conclude that the proposed method can realize pick operation with high reliability and place operation with high precision under SEM based on micro-physics.

Making and maintaining human-pet-bonds is the most essential task for pet robots. To realize this task, the authors propose a methodology of linking psychological interactions between a person and a pet robot. Production of an animal-likeness impression toward a person is one of the most important objectives for interactive pet robots. Animal-like impression can have power to effect human psychological conditions and also human behaviour. We propose 3 methods to produce animal-likeness in artifacts, namely morphological appearance method, body temperature method, and vital body movement stimulus method. In addition to producing impressions, the authors measure human behavioural uncertainty quantitatively in order to show the relationship between human behavioural attitudes towards a pet robot and what he/she feels about it. These methods for producing impressions and measuring human behaviour will be the key to founding a close and friendly bond between human beings and pet robots.

Proposes a visually based teleoperation with accumulation method and system where the visual and control information sequences are stored and available for syntactically indexed playback. Visually based teleoperation systems heretofore described in the literature unfortunately can only display an instantaneous representation of the control sequence. Past control experience should instead be accumulated and available to facilitate teleoperators. In this paper, the behavior sampling extraction theory and method, the behavior sampling data representation, and portions of the status on demand function-set are introduced. Behavior sampling utilizes semiotic analysis of the teleoperator control behavior to syntactically segment the control motions and the relevant visual objects into a syntactically indexable storage stream. A preliminary system used for manipulating single biological cells under an optical microscope (fitted with a video camera) is described. Repetitive manipulation experiments on Mato fluorescent granular perithelial (FGP) cells show the effectiveness of this enhancement to the visually based control approach.

This paper proposes a novel human symbiosis robot named Expressive Mobile Robot. Through its behavior and movement it successfully realize nonverbal communication to humans. Towards the future aging society human-robot symbiosis is keenly needed. To realize such human-robot symbiosis, familiarity of robot is crucial. The proposed Expressive Mobile Robot produces familiar impression to humans through its behavior, action and motion. It does no real task other than the expression. Based upon consideration of human gestures, a prototype of the Expressive Mobile Robot has been designed and implemented. It is puppy-sized and has 7 DOF with a head, two arms and a body. Utilizing the prototype robot, several psychological experiments are conducted. Experimental results make it clear that there are proper speed pattern, distance to human and poses for familiar impression. The results support the feasibility of the proposed idea and robot.

Human activity support is being established as a novel active area for robotics. It is expected to make clear the mechanism of understanding situation and intention of humans through regarding behavior as media and the mechanism of human support based on it. This paper shows a method to support handiwork by recognizing human action conforming to situation and shows a soldering support tool constructed based on the method as an example.

This paper proposes an idea that it is possible for a mobile robot to display behavioral expressions by its motion. Behavioral expressions are expressions of vividness, sense of distance and attention. To confirm the idea concretely, an expressive mobile robot has been designed and implemented to display the behavioral expressions. Utilizing the robot, psychological experiments have been conducted to evaluate impressions on three items: 1) velocity changing pattern, 2) distance between human and the robot, and 3) various poses. The experimental results indicate: firstly, there is a proper speed pattern for expression of vividness, the pattern being triangular along the time axis; secondly, there is a proper distance range between human and robot for expression of mental distance between them, its average value being about 2.5 m; thirdly, when the robot faces the human, the impression of attention is increased where the robot puts its head on one side or raises its hands. The implemented expressive mobile robot is puppy-sized and has 2 DOFs for motion, 2 DOFs for two swingable arms and 3 DOFs for pan, tilt and yaw of its head. The experimental results prove feasibility of the proposed idea of the behavioral expression by the robot.

To support human activities is being established as a novel activity area for robotics. The mechanism to understand situation and intention of human from her/his behavior is expected to be made clear. Action recognition method will be indispensable to realize the mechanism. In this paper, we describe a system which finds human in a scene and tracks the human using vision to recognize actions. The proposed system is network oriented which distributes a lot of fixed stereo pair cameras, and tracks human as a whole. The constructed vision system utilizes skin-color extraction and template matching based tracking. Human action recognition experiment using the system is shown.

This paper proposes a function for monitoring a patient's respiration and posture in sleep using a human symbiosis system in a sickroom. The novel features of the system function lie in noninvasive and unrestrained monitoring to ensure the symbiosis with the human. Noninvasive monitoring eliminates the need for monitoring needles or catheters to invade the patient's body; sensors thus do not impose a physiological burden such as pain on the patient. In unrestrained monitoring, sensors and their electrical cords do not limit degrees of freedom of the patient's movement. Unrestrained sensing therefore does not impose a psychological burden caused by the limitations on the patient. A "robotic bed", which is a system to realize the functions stated above, is a bed-shaped system with 221 pressure sensors for monitoring the patient's respiration and posture without preventing doctors and nurses from performing their tasks. The sensors are set 5 cm from each other on the bed and surround the patient. Experiments to monitor respiration and posture in sleep demonstrate that the proposed function is feasible for monitoring for over 6 hours.

This paper proposes a contact interaction robot (CIR) which utilizes contact behavior as the interaction means between a human and a robot. The CIR is a puppet robot designed so that the robot and the human touch each other. The psychological experiments are performed by utilizing a CIR equipped with pressure sensors on both sides of its neck and six servo motors in its neck, two arms and, two legs. The experimental results reveal that the CIR is able to moderate the painfulness perceived by the human as well as to bring a sense of relief.

This paper proposes a system design concept of a "concentrated visual field" for micro object handling. The proposed concept features: (1) multiple microscopes, (2) fields of view concentrated on manipulators' work space and (3) rotational DOF for tilting manipulators and a worktable. Micro Object Handling System II, which consists of two microscopes (a scanning electron microscope and an optical microscope), two manipulators (primary and secondary) and a worktable, is constructed based on the design concept. The primary manipulator is equipped with two rotational DOFs, and three highly precise translational DOFs. The secondary manipulator is utilized to fix objects on the worktable. The paper also proposes new micro object handling skills to control the adhesive forces which are crucial for micro manipulation. These skills features: (1) graspless handling of an object by a needlelike tool, (2) changing adhesive force between a tool and an object by changing the contact area between them, and (3) cooperating two tools whose adhesive-forces differ. Several micro object handling experiments were successfully performed by the system to prove the feasibility of the new handling skills and the system effectiveness.

Proposes a pick-and-place shape forming method: a unique fabrication method of 3D micro structures by heaping up fine particles typically ranging from 10 nm to 1 /spl mu/m in size with a micro manipulator under scanning electron microscope monitoring. Particles are picked up by van der Waals force. Pick-and-place control is performed by the optimization of material combination and the trajectory of tool movement. This fabrication method has the following advantages: excellent forming resolution as small as 10 nm, wide range of material selection, freedom of the 3D structure, and quick and flexible fabrication. A pyramidal 3D structure made of 35 polymer particles of 2 /spl mu/m in diameter is successfully fabricated to prove the effectiveness of the method. Since the structure of constructed device has almost the same size as that of light wavelength, it can perform as a quantum effect device in an optical field.

This paper proposes a novel human-robot symbiosis system, named a robotic office room (ROR). The understanding, recording and retrieving of telephone call behavior are successfully demonstrated as a typical example of an office work-supporting function of ROR. A unique object model is proposed, which accumulates information concerning motion of the object. This model enables the system to understand human behavior by monitoring the motion of the object-typically, a telephone or pen. Since the model requires the system to pay spatial and temporal attention only to the target object and related motions, it saves computing power. Repertoires of understandable behavior can easily be expanded by simply adding new object models. The ROR is equipped with networked machines. The concept of "ubiquitous link" consisting of unique network image links and remote control links is proposed. When a human starts to talk on the phone, the ROR not only starts to record the image of the human's behavior, utilizing a VCR, but also turns down the volume on an audio set. The ROR is experimentally proved to be a unique distributed robotic system to support human office work.

This paper proposes a system for computerized description of medical care, and also shows its possibility by experimenting visual understanding of medical care by a doctor. Proposed system supports forming the medical documents by sensing and inputting the doctor's behavior into a computer during medical care. In the system, the computer senses a doctors' behavior using visual, aural and force sensors, and uses these sensor outputs to understand what care is being performed. The visual understanding of medical care is a core function of the system. Taking application of medicine to the ear, nose and throat in ortholaryngology clinics as a typical example of medical care, visual understanding of medical care is successfully performed. The procedure of visual understanding in the experiment is as follows: Firstly, CCD cameras monitor the objects(tools, medicine and the patient's affected body parts) whose states change in accordance with the doctor's actions. Secondly, the doctor's action is identified with the change in the state of these objects. Finally, the doctor's behavior is understood as the sequence of these identified actions. In the procedure: visual behavior understanding is more easily realized by monitoring objects used for behavior than by conventional method of monitoring the persons. Medical care description in, this paper will contribute to improve the quality of medical service by reducing the burden of medical information recording on doctors.

This paper proposes a robot behavior notation named "Behavioral Score" and a methodology of generating "familiar" robot behavior. Behavioral Score, designed with analogies to a musical score, is an effective method of describing and controlling the expressive behavior of a robot. The effectiveness of Behavioral Score is demonstrated by a choreographed robot dance. The methodology described is based upon the hypothesis that the passive reactions of a robot produce a familiar, comfortable, imprecision in human beings. Utilizing the robot behavior for generating familiarity will facilitate human-robot symbiosis. The results of psychological experiments verifies the validity of the methodology.

This paper proposes a third generation teleoperation system based on a status-driven slave symbolically instructed by the master control panel. Furthering the first generation joint-angle control techniques and second generation coordinate transformation techniques, this system recognizes the target status specified by the operator by extracting the task status from visual sensors. The task environment is initially described via sensing points in the work environment and on the manipulated object. The relationships between the sensing points and the change of those relationships describes the task at hand. Operation can proceed in automatic mode where the system completely directs the slave or in shared mode where the system assists the operator by constraining the slave motion while the operator directs via a visual communication interface. Although status-driven techniques offer several improvements in teleoperated control, this work explores the robustness and reliability provided in dynamic workspace environments. One successful application of this technique is in micrometer scale part handling, the so-called "microworld assembly" where an operator's past experience in the macro world is not applicable to the physics experienced in the microworld.

Concerns nanomanipulation, defined as manipulation with precision 1mm to 10/spl mu/m. It uses scanning electron microscopy. It is at an early stage because we do not know much about the microphysics and the skills needed for micro/nano object handling. The differences between the micro and the macro worlds, and the difficulty of micro/nano manipulation originating from that gap are known, but no solution has been reported. We need a whole set of new robotics to cope with the difficulties of manipulations in micro and nano world, i.e. we need new manipulator mechanisms, control, and intelligence for micro and nano manipulation. We discuss the features of the micro/nano manipulation world and clarify the required functions for such a manipulation system. Then the paper proposes a micromanipulation system, a micro teleoperation system featuring manual task execution posture, an automatic manipulation system based on visual feedback control, and a pick-and-place forming method to realize optical quantum devices as a typical product of nano manipulation. The range of fields of application is discussed.

In consequence of the trend toward miniaturization, the importance of nano manipulation with precision ranging from 1 nm to 1 /spl mu/m, has become recognized. The paper describes the features of nano manipulation, emphasizes the necessity of the visual control to execute tasks in the nanometer world, and proposes an automatic nano manipulation system equipped with visual feedback function, the nano-hand-eye system. The system consists of nano manipulator, scanning electron microscope, real-time image processors and a host computer integrating hand and eye. Visually controlled nano manipulation under scanning electron microscope is first demonstrated, and the effectiveness of visual feedback is then proved by the engraving task of a thin line with uniform width of 3 /spl mu/m in desired direction.

This paper describes a new micro-teleoperation system that is inherently suitable for micro-object handling since it mitigates several drawbacks associated with human interface devices. The system uses an upward-facing CRT display monitor mounted beneath a table. An operator performs micro-tasks in a sitting posture at the table while looking down at the monitor with a touch sensor screen situated between the operator's arms. To execute these tasks, microscope-observed images of the target objects are utilized in conjunction with the touch sensor screen and a specially designed, pencil-shaped master manipulator having a force-feedback function. The system incorporates the following characteristic features: (1) Intuitive teleoperation is possible because coincidence of the operating and monitoring point is provided. (2) The pencil-shaped master manipulator enables sensitive bilateral micro teleoperation because of its light weight and small inertia for movement, and (3) With the side of the right hand fixed in a pen-holder grip, the operator uses the master manipulator to generate commands with a wide dynamic range of force feedback, i.e., the operator can simultaneously feel force feedback during operation from both pressure/slip sensory receptors on the fingertips and sensory receptors which detects stretching of the finger muscles. The stated features are proved experimentally by the constructed micro teleoperation prototype system.>

Understanding human intention is an essential function for a robot which can offer adequate support to human beings. It requires smooth communication between the human and the robot. Human behavior is an expressive media of communication. This paper proposes a new function of "active under standing of human intentions" by a robot through monitoring of human behavior. The unique feature of the proposed function lies in the fact that it utilizes multi-communication channels in parallel. i.e., human intentions are understood not only through conscious behavior but also through unconscious behavior. The paper also proposes a robot architecture to realize the function. The following points are the key features of the architecture: a robot possesses multi-sensors which surround the human; and information processing is carried out by dual loops-a loop for information exchange between the human and the robot and a loop for human intention understanding. As an example of a robot with human intention understanding functions, the authors constructed a micro-teleoperation robot. It can automatically understand an operator's intention through such unconscious behavior of touching a desk with an operator's hand. The understood result is utilized to change the control mode of a master-slave manipulation system from fine motion to rough motion and vice versa. The experimental results prove that the proposed function is effective in making the operation of the system easier. Consequently the system is friendly to the operator.>

Describes the construction of a tele-operated handling/machining system which operates in the micrometer to millimeter range. The system is equipped with a pair of three dimensional force sensors and can be controlled remotely using a three dimensional joystick. The paper presents a transformation method for converting multi-axis force information to auditory information. Such a transformation conveys a realistic sense of the state of the operation and allows high operability of the system. Experimental results are presented which demonstrate the effectiveness of the implemented system.

In recent years, demand for advanced sensing in robot tasks using multiple sensing modules has increased. A traditional approach for the construction of such a sensing system is to organize the modules using a single and centrally controlled system. However, there are various disadvantages in this approach with respect to flexibility, robustness, extendability, and efficiency in achieving tasks. To overcome these difficulties, decentralized and cooperative control of multirobots is a promising approach. The authors propose a decentralized and cooperative sensing system for robot vision tasks and present a prototype of such a visual sensing system implemented using a multirobot system. It performs visual inspection of water leakage on the sleeve of a valve. In this task, active and cooperative control of cameras and light sources play a very important role. The authors extend the system with 3-D model-based agents which utilize 3-D model information about the task environment for distributed sensor planning.

A manipulator system for micro-object handling is presented and the design concepts used for system construction are described. The manipulator consists of two robots (right- and left-hand robots) and a optical stereo microscope. The right-hand robot has a working tool and a three-axis force sensor and is equipped with two orientational DOF; the left-hand one has a work table and is equipped with two orientational and three translational DOF. This manipulator system has a wide range of translational and orientational motions. Since all axes of the orientational DOF intersect at the tip of the tool, and this tip coincides with the focus area, micro-object handling operations can subsequently be executed within the focus area. A micro air turbine (diameter = 502 /spl mu/m) comprised of three parts was successfully assembled by the manipulator system. A sucking tool composed of a drawn glass pipe and vacuum pump were used for its assembly.

Proposes a face recognition method which is characterized by structural simplicity, trainability and high speed. The method consists of two stages of feature extractions: first, higher order local autocorrelation features which are shift-invariant and additive are extracted from an input image; then those features are linearly combined on the basis of multivariate analysis methods so as to provide new effective features for face recognition in learning from examples.>

Presents two types of teleoperation system concepts proposed and demonstrated under the MITI advanced robot project. One is a tele-existence system which enables a human operator at the controls to perform remote manipulation tasks dexterously with the feeling that he or she actually exists in one of the remote anthromorphic robots in the remote environment. The other type is intelligent teleoperation systems. Two types of intelligent teleoperation systems have been developed under the advanced teleoperation concept: one is a model-based telerobot system and the other is a system with robot intelligence to interact with the human operator. Two integrated robots for nuclear power plant and subsea applications have been developed and demonstrated successfully.>

An automatic planning method of light source and camera placement for an active photometric stereo system is presented. Since photometric stereo systems normally use multiple light sources fixed to the environment, they cannot avoid shadows caused by surrounding objects. Using a movable light source and adapting its placement actively to the task environment eliminates shadows. Candidate positions for light source placement are obtained based on a three-dimensional model of the environment and image processing of a virtual sphere around the target objects. Possible combinations are evaluated using the combined criteria of reliability and detectability of the measurement. Experimental results obtained using a light source affixed to a manipulator demonstrate the usefulness of the method. To improve the detectability of the active photometric stereo system, an extension has been made by using a movable camera in place of a fixed camera.>

A qualitative reasoning planner for behavior improvement by selecting and determining robot control parameters to drive manipulation actions has been developed, integrated into a telerobot system, and demonstrated for a match striking task. The planner consists of a qualitative reasoner and a numerical execution history which interact to jointly direct and narrow the search for reliable numerical control parameter values. The planner algorithm, implementation, and an execution example are described.>

The language-aided robotic teleoperation system incorporates two sets of teieoperational languages with a master-slave manipulator. One offers flexible means, called software jigs, to specify motion constraints superimposed on operational motion, simplifying task motion. The other offers an easy method of teaching elementary tasks which frequently appear in teleoperation. Re-execution of the thus-taught program is effectively utilized.

This paper describes the three-dimensional underwater imaging method of "Synthetic Aperture Image Holography" which utlizes an acoustic lens and synthetic aperture sonar techniques. The imaging procedure is as follows : (1) a transmitter in the transmitting array projects pulsed ultrasound to the target, (2)the reflected ultrasound from the target is then focused by the acoustic lens, (3) the time series signals of the focused image are detected by the receiving array, (4) multiple images are obtained by using transmitters at different positions in the array and by repeating steps 1 though 3, and (5) the final three-dimensional image is calculated by synthesizing all of these images together. A method analysis reveals that synthetic aperture techniques increase the system's resolution over the normal resolution limit of the lens itself. It also shows that the acoustic lens reduces the amount of calculations needed to obtain the image because it does part of the calculations physically. A three-dimensional image is obtained by the experiments. Lateral and range resolutions obtained are nearly equal to theoretical ones.