This research aims to develop a ceiling mobile robot that can sustain a large payload and can locomote freely under a ceiling space. In our approach, a perforated metal, one of the recent popular architectural materials, is utilized as a ceiling plate and the robot hooks and hangs the sequential holes of the ceiling plate by mechanical constraint. To realize the robot, the authors developed three key mechanisms, (1) ceiling hanging mechanism for the perforated metal, (2) horizontal locomotion mechanism like an inchworm, and (3) pantograph mechanism for smoothing horizontal locomotion speed and for load balancing. A unit testing of the ceiling hanging mechanism and the integration horizontal locomotion experiments confirmed the good performance of each mechanisms and the feasibility of mechanically constrained ceiling mobile robot. However, it was revealed that higher power motors and a stiffer body structure should be installed to sustain an extremely large payload like a human.

In this paper we propose a novel method for predicting resident's behaviors in a house from one's movement trajectories. The method consists of 1) segmentation of trajectory data into staying or moving and classification of the segments and 2) prediction by time-series association rules from transition events of each segment. The method predicts the start time of target behaviors for daily life support, such as eating, taking a bath etc. The time lag between the prediction and the target behavior can be set up manually, thus the method is adaptable to a variety of supporting systems. The experimental results using real residents' trajectory data of almost two years demonstrate that prediction of behaviors by the proposed method is feasible.

This paper describes a human shape reconstruction method from multiple cameras in daily living environment, which leads to robust markerless motion capture. Due to continual illumination changes in daily space, it had been difficult to get human shape by background subtraction methods. Recent statistical foreground segmentation techniques based on graph-cuts, which combine background subtraction information and image contrast, provide successful results; however, they fail to extract human shape when furniture such as tables and chairs are moved. In this paper, we focus on the results of face detectors that would be independent of such background changes and help to improve the robustness under movement of background objects. We propose a robust human shape reconstruction method with the following two characteristics. One is iterative image segmentation based on graph-cuts to integrate head position information into shape reconstruction. The other is high-precision head tracker to keep multi-view consistency. Experimental results show that proposed method has enhanced human pose estimation based on reconstructed human shape, and enables the system to deal with dynamic environment.

Vision based human articulated body pose tracking has been historically important. Because analyzing multiple human activities, especially interaction between human in cluttered scenes is essential in visual surveillance scenarios, multiple people tracking has enjoyed much attention in human robot interaction research in recent years. In this paper, we newly introduce a robust framework for multiple people pose tracking. The notable aspects of our approach are real-time ensuring speed (up to 30 fps), flexibility towards various complex motions and environments. Our work is inspired by the success of multiple view approach, especially voxel based techniques. The use of voxel data leads to viewpoint-free estimation, which benefits in that reconstruction of a training model is needless in different multi-camera arrangements. We add simple tracking-based volume segmentation algorithm to retain practical superiority of voxel based approach. Furthermore, our framework successfully obtains multiple body pose estimation in real-time even when people contacts with each other occurs in the scene, which is not addressed in the conventional approaches. We demonstrate the effectiveness of our approach with experiments on indoor cluttered scene sequences.

This paper describes an activity pattern mining method via inhomogeneous Poisson point processes (IPPPs) from time-series of count data generated in behavior detection by pyroelectric sensors. IPPP reflects the idea that typical human activity is rhythmic and periodic. We also focus on the idea that activity patterns are affected by exogenous phenomena, such as the day of the week, and weather condition. Because single IPPP could not tackle this idea, Dirichlet process mixtures (DPM) are leveraged in order to discriminate and discover different activity patterns caused by such factors. The use of DPM leads us to discover the appropriate number of the typical daily patterns automatically. Experimental result using long-term count data shows that our model successfully and efficiently discovers typical daily patterns.

This research aims to realize a container case hand-over task between robots. Although sophisticated cooperative control is essential to avoid destructive internal force in general cooperative transfer task, we propose a geometrical caging strategy which can simplify a hand-over task. To solve a problem of capture region mismatch during caging state transition, the proposed strategy does not utilize additional actuators and sensors but a compliant mechanism whose displacement range can be changed automatically by its configuration. By experiments, it was confirmed that the caging strategy can make the hand-over task drastically easy and robust, and a winch type quasi-compliant mechanism is effective to the problem of capture region mismatch.

A microwave Doppler sensor can monitor human motion without contact. It can sense wide range of motion from minute oscillation like respiration to large movement like walking because it measures the distance change between the target and the sensor as signal phase change. However, the proper method for the signal phase estimation is different between motion and respiration measurement. In this paper, we compare three methods for signal phase estimation and find the optimal method and parameters for each problem. For switching to the proper phase estimation method or monitoring human state, it is important to detect motion signal and respiration signal from the raw signal of sensor output. Three kinds of features, energy, frequency-domain entropy and histogram are extracted and are input into binary classifiers. We tested least squares, SVM and AdaBoost classifiers.

Laser based environment recognition technologies have been developed recently. Especially moving objects detection and classification by laser scanners mounted on a mobility is required for mobile robots and autonomous cars. In this paper, we propose a moving objects detection and classification method based on grid trajectories acquired from sequential laser scan data. Grid trajectories are obtained by voting sequential laser scan points on a grid map, and these trajectories not only work for a correct scan segmentation, but also represent the size and the speed of moving objects. We classify a moving object into either a person, a group of people, a bike, a car based on its grid trajectory. In our experiments, our mobility mounted laser scanners acquired scan data in the university campus, and the experimental results illustrate the effectiveness of the proposed method in outdoor environments.

This paper proposes an object movement detection method covering large areas of a room by using multiple cameras. When object movement detection for whole of a room is performed, there are several challenging difficulties: sizes of objects on the camera images are small, non-objects such as humans also exist on the images, objects are sometimes difficult to detect in specific viewpoints because of occlusion by humans or furniture or color similarity to near objects. In this work, to detect object movements robustly though the object sizes are small, we apply multiple view integration via features extracted from “stable changes” on each viewpoint. To discriminate between object and non-object, we focus on motion of changed regions. Our experiment in a room environment shows the multiple view integration method improves recall rate of object detection performance by about 0.2 when false positive rate is over 0.1.

In this paper, we propose a novel robust action recognition framework with the following capabilities: 1) online encoding motions to multi-label sequence where the output in each frame is a tuple of labels rather than a single label, 2) providing efficient automatic relevant motion selection framework, 3) learning systems so as to be optimal for online multi-label sequence classification. As for multi-label classification, our approach incorporates contextual information about action not only temporal information but hierarchical information of actions. Inference tends to be complex so as to achieve such complex recognition scheme, however, we propose an efficient Viterbi-like decoding algorithm which integrates forward algorithm and loopy message passing algorithm. As for the learning process, the algorithm optimizes the parameters so as to maximize log likelihood of the model. Boosting, ensemble approach of machine learning, is leveraged to provide efficient feature selection framework in the training process. The experimental results show that the proposed method successfully exploits the impact of contextual information then significantly outperforms the traditional approaches in dynamic gait motion classification.

In this paper, we present a novel framework to recover human body pose on multi camera systems. Our framework leverages 3D voxel data, which are reconstructed from multi-camera systems. The use of voxel data leads to viewpoint-free estimation, which benefits in that reconstruction of a training model is needless in different multi-camera arrangements. Other notable aspects of our approach are real-time ensuring speed (up to 30 fps), flexibility towards various complex motions and environments. We treat the pose estimation problem as estimating human pose label from the voxel features and tackle this by example based approach. To ensure the real-time speed and to improve precision of pose estimation, a newly fast and robust near-neighbor search metric is installed prior to the evaluation process, what we call CSI-PSH. We demonstrate the effectiveness of our approach with experiments on both synthetic and real image sequences.

Laser based tracking systems have been developed for mobile robotics and intelligent surveillance areas. Existing systems estimate only human positions. In this paper, we propose a method for human pose estimation represented by human head and waist position using only laser range finders. Two features of human cross-sectional contours are extracted from laser scanner data scanning on the height of waist. This method estimates human pose by using these features in the Bayesian filtering framework. Moreover, we develop a new particle filter framework with two transition models and two resampling steps. In this framework, position estimation and pose estimation are performed by many hypotheses. Our experimental results demonstrate the effectiveness of the method in pose estimation of multiple people by using only several laser scanners.

This paper presents three behavior labeling algorithms based on supervised learning using accumulated pyroelectric sensor data in the living space. We summarize features of each algorithm to use them in combination matched to usage of the livelihood support application. They are (1) labeling algorithms based on time attribution of ldquoon-offrdquo data, (2) one based on Hidden Markov Models, and (3) one based on switching model around a behavioral change-point. We show the behavior labeling results of three algorithms for one month data under the same conditions. Then we point out features on the basis of these results.

This paper introduces a new foreground segmentation method. In contrast to most of the related works, our method uses only two image frames, a target frame to process, and a single reference frame. Our method first conducts simple thresholding like background subtraction, but then applies an iteration scheme we propose to estimate the pixel-wise likelihood of belonging to the foreground/background from the frame-to-frame difference. Finally, a further refinement considering edges is applied using graph-cut optimization. Experimental results show the effectiveness of our method, especially in that it keeps good performance over a wide range of the threshold value. That consistent performance will become an important step toward fully-automatic segmentation.

In this paper, a novel statistical indoor activity recognition algorithm is introduced. While conditional random fields (CRFs) have prominent properties to this task, no optimal performance is obtained due to the fact that the performance is optimized for offline estimation. Furthermore, no previous researches provide efficient training process to optimize classifiers in on-site recognition perspective. In this paper, we propose a novel sequence estimation model suitable for online activity recognition, what we call Just-in-Time random fields (JRFs). In JRFs, efficient training and feature selection process is provided via boosting. Empirical evaluation using synthetic and real indoor activity records shows that our model drastically outperforms the previous methods in view of the classification performance with respect to the training cost.

This paper describes development of home-use automated container storage/retrieval system, a sub-system of a logistical support robot system in living space. The s/r system has 3 features. (1) Elevator type structure enables not only auto stocking motion, but also human access to containers as if the auto rack is a general shelf. (2) A separated motion layout enables us to utilize upper ceiling space as a stock space, and decreases possibility of accidental stuck by reducing the volume of robotpsilas motion. (3) The collaboration of a container guide plate and sensing with RFID actualize to navigate people to place a container on a shelf with desired position and posture. By experiments, it is confirmed that the ACSRS can transfer a container and 5 [kg] load (total 8 [kg]). We summarize significant technical points to design a system of this type.

This paper describes an algorithm of behavior labeling and anomaly detection for elder people living alone. In order to grasp the personpsilas life pattern, we set some pyroelectric sensors in the house and measure the personpsilas movement data all the time. From those sequential data, we extract two kinds of information, time and duration, and calculate two-dimensional probabilistic density function of them. Using this function, we try to classify behavior labels and detect anomaly. Here, we assume two kinds of anomaly, ldquothe rare behaviorsrdquo and ldquothe changes of life patternrdquo. The algorithm is confirmed to work on real behavior data through the experiment on about 400 days data.

In this paper, we propose a robust recognition and segmentation method for daily actions with a novel multi-task sequence labeling algorithm called multi-task conditional random field (MT-CRF). Multi-Task sequence labeling is a task of assigning input sequence to sequence of multi-labels that consist of one or multiple symbols in single frame. Multi-Task sequence labeling is essential for action recognition, since motions can be often classified into multi-labels, e.g. he is folding arms while sitting. The MT-CRFs: extensions of conditional random fields (CRFs), incorporate jointly interaction between action labels as well as Markov property of actions, to improve the performance of the joint accuracy: the accuracy for whole labels at specific time. The MT-CRFs offer several advantages over the generative dynamic Bayesian networks (DBNs), which are often utilized as multi-task sequence labelers. First, the MT-CRFs allow relaxing the strong assumption of conditional independence of observed motion, which is used in DBNs. Second, the MT-CRFs exploit the power of non-Markovian discriminative classification frameworks instead of generative models in DBNs. With deep insight of the problem Multi-Task sequence labeling, the inference process of the classifier gains more efficiency than the previous Markov random fields that tackle multi-task sequence labeling. The experimental results show that classifiers with MT-CRFs have better performance than cascaded classifiers with a couple of CRFs.

The authors proposed a methodology of assisting the patients who were suffering from mobility impairment of finger and thumb joints, with exoskeletal devices. The types of exoskeletal assistive devices for such patients were categorized into several groups based on required functions which are derived from the patients' condition. One of the categories, "joint motion compensator" is a novel concept taking into consideration the difference between active-ROM (range of motion) and passive-ROM of joints, and with regard to one of subgroups of this category called "joint motion amplifier", a prototype device which amplifies the motion of index-finger PIP (proximal interphalangeal) joint was constructed and verification experiments for pinching objects were conducted. The results revealed that it was feasible to pick up and pinch objects which were not so small or thin enough to be picked without nails, and also was possible for the device to hold such objects as strongly as human hands do.

This paper reports development of an intelligent container, a unit of the logistical support robot system in living space. The container not only supports human to fix and arrange their commodities, but also supports robots to achieve their logistical tasks. In other words, the intelligent container is a mediator between human's request and robots' capabilities. The container has four major roles in our logistical support system. (1) Taking several commodities in and recognizing and acquiring contents' information. (2) Supporting human and robots to transfer the container. (3) Receiving users' command as a simple user-interface. (4) Communicating to a host computer and informs the contents of the container and users' command. To realize those roles, the container is equipped with a grasping navigation mechanism for robot handling, RFID reader for the recognition of contents, and LCD device to display supporting information to human and robot. Finally, a primary function is evaluated in experiment to confirm that an acceleration sensor enables the container to detect being transferred by human. In the future, the intelligent container will play a role of contact point for human and robots in our target system, that enables human-robot symbiotic life.

In this paper, we propose wrapped boosting that is extension of boosting algorithm for robust online action recognition. Boosting algorithm is one of ensemble learning algorithm and is also known as a feature selector. In our previous work utilizing boosting, we achieved automatic feature selection and robust model-based action classifiers which had very small calculation cost based on posture information of human body joints. However, which joints we should allocate posture sensors to must be given by humans in advance. Our new learning framework of wrapped boosting provides not only automatic feature selection but also automatic sensor allocation to proper joints of humans for target actions. We evaluated our algorithm targeting gait motion based on motion data fetched by motion capturing system. In consequence, wrapped boosting was able to select proper joints to which limited sensors should be attached, and to construct more robust classifiers compared to constructing classifiers with all joints available. Classifiers constructed only with existing boosting algorithm were subject to over-fitting to training data.

In this paper, a novel approach is proposed to recover human body pose from 3D voxel data. The use of voxel data leads to viewpoint-free estimation, which benefits in that reconstruction of a training model is needless in different multi- camera arrangements. Other notable aspects of our approach are real-time ensuring speed (up to 30[FPS]), flexibility towards various complex motions, and robustness towards voxel noise. The main concept of our approach is based on an example based approach. Human posture candidates are constructed beforehand from a large motion capture database, and the most appropriate posture is estimated per frame by comparing the likelihoods between 3D voxel data and posture candidates. The evaluation is formulated by introducing a histogram- based feature vector that represents the 3D shape context of human body. In addition, a fast near-neighbor search metric is installed prior to the evaluation process, in order to reduce the computational cost and ensure real-time processing. Estimation stability is also improved by a graphical model of motion, which adds a smoothing effect to the motion sequence. We demonstrate the effectiveness of our approach with experiments on both synthetic and real image sequences.

This paper describes automatic generation and linking mechanism of sensor processing components, which are called sensor services, in network middleware for distributed sensors in a room. This mechanism reduces deployment process in the creation of applications for real sensorized environments. The mechanism consists of 1) dynamic program duplication triggered by matching generation condition with descriptions of components and 2) sensor data selection based on selection conditions and the descriptions. The description, which is called RDF sensor description, is written in resource description framework (RDF) for definition of sensor properties including the knowledge about sensor-attached object. The conditions for RDF sensor description are scripted flexibly with vocabulary of abstract sensor knowledge. Through the conditions, the components are generated dynamically according to change of the sensor composition. The generated components can select needed sensor data in multiple sensor environment. We create sensor services that detect certain behaviors from raw sensor data in our sensing room. The services show generation mechanism's independency on configurations of sensors in deployment of the components in the middleware

This paper presents a human like segmentation method for daily life actions, such as getting up, sitting down, walking. Unsupervised segmentation methods of many previous researches cannot always assure segmentation result that coincides with human's natural sense. While the proposed method utilizes human's teacher data of segmentation to conduct human like segmentation. We assume that latent dynamics changes at the segmentation points of action, and represent segmentation boundary by switching model of two linear dynamic systems. The problem is that human may segment actions according to wide variety of criteria depending on the attention point or other backgrounds. In this paper, those criteria are acquired by clustering segmentation boundaries extracted from teacher data made by human. Each of the cluster is characterized by body parts it pays attention to. Here, we focus on hierarchical aspect of human body that human body can be treated at various levels of abstraction (e.g. whole body, upper body, left arm), and represent it by tree structure. Experimental result shows that the proposed method can acquire human like segmentation criteria

In this paper, we propose a margin-based query learning algorithm for action recognition to reduce a laborious work on annotating action labels of time-series motion. The annotation is an inevitable task for designers of recognition systems with supervised learning techniques. Query learning is a kind of compensation approach for this, and can also be categorized into interactive learning. Our algorithm is a natural extension of maximum margin learning; a.k.a. support vector machines. Thanks to the theoretical analysis of the optimal condition of the maximum margin learning, the algorithm runs with a single and simple criterion. To prevent poor performance of the classifier learned with very few size of labeled motion data set, the algorithm exploits cluster information of massive unlabeled motion dataset. In contrast to the previous margin-based query learning methods, the algorithm has superiority in terms of stability. The empirical evaluation using real motion and synthetic dataset shows that our algorithm can achieve both drastic reduction of annotation cost and making robust classifiers

In this paper, we propose a fast and robust online action recognition method. The main features of the proposed method are: 1) to select a small number of critical motion features from a very large set of motion feature templates and to release humans from task of designing critical motion features, 2) to require very small calculation cost for recognition compared to conventional methods, 3) to exploit "combinational motion features" which we propose as a new conception so as to construct a robust action recognizer. We evaluated the proposed method to gait action recognition, such as walking and running, by utilizing motion capture data. In the result, the proposed method reduced parameters given by human to action recognizer and lessened human's task. In addition, the proposed method needed very small calculation cost for recognition, and can recognize robustly as much as conventional action recognition method based on support vector machine. Moreover, the introduction of combinational motion features enhanced recognition performance

This paper proposes an imitation learning system to generate trajectories by which a robot supports a human with close physical assistance adapting to human movements and daily life environments. The proposed system is composed of 1) division algorithms, 2) learning algorithms and 3) assistance algorithms. 1) In division algorithms, the system measures time series of human task execution data and divides them into multiple motion segments automatically. This division is based on standard deviations of motion errors between measured trajectories and an ideal trajectory where the ideal trajectory is mean of all measured human trajectories and is expected to achieve the purpose of human task successfully. Since an important motion parameter is paid attention to by the human and has small standard deviation of errors, series of measured data are divided into segment motions at the points where the importance of parameters changes suddenly. Thus this division is guaranteed to accord with human attention. 2) In learning algorithms, the system learns trajectories with dynamic neural network (DNN). Since the DNN has convergence, generated trajectories can converge to an ideal trajectory. The importance of each parameter, in other words how much attention human pays to the parameter, is evaluated as how small the standard deviation of errors is. The DNN learns trajectories reflecting the evaluated importance of parameters to accord with human feeling. 3) In assistance algorithms, the system judges when to start assistance by the assumption of multiplied errors of motion parameters by the respective importance. In assistance algorithms, the system also connects generated trajectories of motion segments smoothly. An experiment to support human drink task was performed successfully where the proposed system judged not only when to start assistance to the task but also execute assistance when a cup was about to incline too much not to spill water

In this paper, we propose a novel kernel computation algorithm between time-series human motion data for online action recognition. The proposed kernel is based on probabilistic models called switching linear dynamics (SLDs). SLD is one of the powerful tools for tracking, analyzing and classifying human complex time-series motion. The proposed kernel incorporates information about the latent variables in SLDs with simplified designing approach called marginalized kernels. The empirical evaluation using real motion data shows that a classifier using SVM with our proposed kernel has much better performance than the classifier with some conventional kernel techniques. Another experiment using walking around motion shows that a classifier with the proposed kernel can properly segment the start and the end of the target action.

In this paper, we constructed wireless ad hoc network for realizing various physical and informational support systems based on Lifelog that is the records of experiences in the daily life, and realized a prototype of electric appliances operational support system which is one of useful systems. Utilization of Lifelog reduces the burden for controlling a huge amount of convoluted electric appliances by constructing the probabilistic model of user's operational behavior based on Lifelog and predicting user's successive operations with this model. In this time, Lifelog accumulates user's operational behavior for electric appliances and environmental information through the wireless network. For the basis of collecting Lifelog including operational behavior, we made a portable Bluetooth-equipped device for wireless network which is necessary to communicate information in the ubiquitous computing environment because Bluetooth has good features such as the ad hoc networking function, sufficient data throughput, high resistance to noise and low power consumption. The realized device has abundant 110 connectors which are able to connect various sensors and actuators. By attaching these devices to electric appliances, they can easily participate in the wireless network and communicate information. Therefore the system can probabilistically model user's behavior, especially operations for electric appliances using Lifelog as training data, and predicts the user's next successive operations for surrounding appliances utilizing this model and the user's present state. The system gives prediction results to the user and executes operations via the wireless network after the user's confirmation. The various experimental results prove that the operational system in ubiquitous computing environment is useful and our realized device is sufficient performance in the daily life.

In this paper, we propose a human posture reconstruction method from the insufficient input posture data based on human posture probability density that is constructed by a long-term human motion capture data. Since the long continuous daily human motion data has high dimensions and becomes huge size, the human posture data should be effectively compressed. The long term posture data has nonlinear distribution on the posture space, since each specific posture such as standing and sitting has different property. The posture data is allocated into some subspaces and compressed for each subspace with mixtures of probabilistic principal component analyzer (MPPCA). MPPCA is improved by replacing conventional EM algorithm with deterministic annealing EM algorithm (DAEM) to avoid initial parameter sensitivity. The posture probability density is constructed over those subspaces. The adequate human posture can be reconstructed from the insufficient data by introducing the posture probability density into the sequential Monte Carlo framework. The experimental results show that the robust human posture estimation can be realized since this method does not estimate the unique posture but estimates the proper posterior posture density with using the posture prior knowledge.

In this paper, we propose a robust online action recognition algorithm with a segmentation scheme that detects start and end points of action occurrences. In other words, the algorithm estimates reliably what kind of actions occurring at present time. The algorithm has following characteristics: 1) The algorithm incorporates human knowledge about relation between action names in order to simplify and toughen the algorithm, thus our algorithm can label robustly multiple action names at the same time. 2) The algorithm uses time-series action probability that represents the likelihood of each action occurrence at every frame time. 3) The classification technique with hidden Markov models (HMMs) enables the algorithm to detect robustly and immediately the segmental points. The experimental results using real motion capture data show that our algorithm not only decreases effectively the latency for detecting the segmental points but also prevents the system from making unnecessary segments due to the error of time-series action probability.

This paper proposes a behavior prediction system for supporting our daily lives. The behaviors in daily-life are recorded in an environment with embedded sensors, and the prediction system learns the characteristic patterns that would be followed by the behaviors to be predicted. In this research, the authors applied a method of discovering time-series association rules, which discovers frequent combinations of events called episodes. The prediction system observes behaviors with the sensors and outputs the prediction of the future behaviors based on the rules.

In this paper, we develop an absolute orientation estimation device equipped with a wireless network. Accelerometers and magnetometers are used to measure the gravity and the geomagnetic field respectively. Gyroscope sensors are used to measure the local angular velocity. The geomagnetic field varies according to the environment. Therefore the device can obtain the information about the magnetic field through the wireless network. The orientation estimation task can be also requested to the other computers with the wireless network. By integrating the measured gravity and geomagnetic field with the local angular velocity using Sigma-Points Kalman Filters (SPKFs), the stability and the robustness of estimating the absolute orientation are improved over either sensor alone. We also propose an estimation method which excludes the effect of motion and magnetic disturbances for the accurate estimation.

This paper proposes a novel algorithm for extracting informative motion features in daily life action recognition based on support vector machine (SVM). The main advantage of the proposed method is not only to extract remarkable motion features, which fit into human intuition, but also to improve the performance of the recognition system. Concretely speaking, the main properties of the proposed method are 1) optimizing kernel parameters so as to minimize its generalization error, 2) extracting remarkable motion features in response to the sensitivity of the kernel function. Experimental result shows that the proposed algorithm improves the accuracy of the recognition system and enables human to identify informative motion features intuitively.

This paper presents a ceiling mobile robot platform that enables multiple robots to move with low possibility of interference with humans and obstacles in an intelligent environment. An advantage of this platform is the potential to reduce sharing space between humans and robots while maintaining the robots' access to humans when their supports are needed. To realize the platform, two key techniques are developed. The first one is permanent magnet inductive traction method for the robots hanging under the ceiling plate. The second one is multiple robots simultaneous position measurement method using matrix of two dimensional codes. Validity of permanent magnet inductive traction method is confirmed by an experiment using a mock-up ceiling plate. Repeatability of position estimation using matrix of two dimensional codes is demonstrated by evaluation experiment. Finally as a demonstration of human support by the platform, transportation task is successfully performed.

This paper proposes robotic kitchen counter, a novel kitchen counter with sensors to monitor a user's action and with the capability of changing its top plate height according to his preference. The robotic kitchen counter was realized by installing various kinds of sensors to the commercial kitchen counter and by putting a jack mechanism underneath. The sensor data acquisition and the top plate height setting can be done through network. Cooking experiments with this robotic kitchen counter showed that 1) there's an optimal kitchen counter height according to an individual person and to his action during cooking behavior, and that 2) the dynamic reappearance of the kitchen counter height he manually adjusted in the stages during the previous cooking behavior was recognized handful, which is the typical example of the simplest action-adapted and personally-fit assistance. The outstanding point of this robotic kitchen counter is that it can measure the cooking action without disturbing the cooking behavior itself while providing action-adapted and personally fit assistance based on the data obtained. The authors believe that the hardware and software techniques obtained through this development can be the starting point to develop other action-adapted and personally fit assistance system.

This paper proposes a network middleware for flexible, easy and uniform utilization of sensors in a room. The middleware has three features; 1) concentrated network configuration, which consists of sensor services and a management server, achieves network transparency for flexible and uniform sensor utilization. 2) Abstraction of sensor information using XML description makes easy utilization and uniform access to multiple sensors. 3) Management of shapes as well as position information is introduced in the network enabling the search function based on the room information and management of spatial data. Not only the implementation for sensors and a management server but also an application software by the middleware show validation of the proposed network middleware.

This paper proposes an imitation algorithm for a robot to acquire typical tasks from multiple measured data of human tasks in the daily life. The algorithm consists of the following procedures: 1) Firstly, the system measures multiple human's object-transferring tasks on a table. Then it calculates qualitative description from measured raw data of positions of human hand and object, as well as the force applied to the table. This description is then converted to probabilistic description. 2) Secondly, the system finds typical human tasks with the maximum likelihood from the probabilistic description. 3) Thirdly, the trajectory, which enables the robot to imitate the typical human task, is extracted. 4) Finally, the imitation task within limited force to the environment is generated from the trajectory by simulation and adaptation. The experimental execution of the generated trajectory proves the validity of the algorithm.

This paper proposes a behavior log creation support system utilizing human pose/position sensors and behavior target sensors. The system is equipped with a human pose sensor and a staying room sensors as the pose/position sensors as a well as a voice sensor and a PC utilization history sensor to detect the target of the behavior. The human pose sensor classifies human behaviors into "standing", "sitting", and "walking". The staying room sensor records a name of the room where the user stays. The voice sensor detects the conversation which appears when the user is communicating with someone else. The PC utilization history sensor records not only whether a PC is used or not but also the names of the application software to serve as a detector of the behavior target during the computer work. The measured data from these sensors is displayed to the user to support creating the behavior log, i.e. to support the user to recall and to input contents and targets of his or her behaviors. The experiment of making use of the sensors and creating the behavior log proved that the recorded events of the behavior log per all events improve from 60% with no support to more than 90% with support of the system. The result quantitatively shows the capability of human behavior logging support of the system.

In this paper, we developed a bed-ridden human's body movement unrestraint estimation system by using a pressure sensors distribution bed. We classified body movements into gross and slight movements. In order to estimate both gross and slight movements, we realized the distinction methods between a human and an object and between sitting and lying status. We also realized the estimation methods of a posture, an articular movement, the respiration and the pulse. By integrating these methods to complement each other, a bed-ridden human's body movement from gross to slight movements can be estimated totally.

This article explains a modeling methodology of human physiology. An agent-based approach to a physiological digital human is proposed, and its design concept and architectural framework are discussed. The modeling is based on two types of agents: stationary agents which model human physiology, and mobile agents that model parameter interaction between the user and the human physiological model. The human thermoregulatory system is chosen as an example of model functions. A Java-based agent system is shown as an implementation of the model.

This paper proposes a human-like action recognition system which can output the result of human action recognition just like the case human does. The system targets actions associated with regular human activity such as walking or lying down, and uses three human recognition characteristics: using specific features of an action to recognize that action; recognition of simultaneous actions; and summarization of recognition results over a short time interval. Experimental results demonstrate the effectiveness of human-like recognition for identifying actions and the superior performance of the proposed system with respect to conventional action recognitions systems. Human-like recognition is expected to ensure smooth communication between humans and robots and enhances the support functionality.

This paper describes a data accumulation system that records continuous sensor data related to human behavior. The system collects sensor data in a room and stores the data in a database via a network. The data is stored in a hierarchical table structure of raw sensor data and processed data in a time series according to a sensor module descriptor that contains information regarding the sensors and output in the sensor module. This allows any type or combination of sensors in a sensor module to be connected to the network dynamically. In addition, three network configurations are proposed for controlling network load and storage. The system is evaluated through experiments, and is shown to exhibit no more than 1% data loss and capture data that can be used effectively to reproduce or analyze human behavior. Such accumulated data will make it possible to model human behavior and customize human support systems based on long-term behavioral trends.

This paper proposes a plant-shaped robot object as an awareness driven display device of human daily life information. While a conventional monitor display (operation driven display) device transmits information to a person when it is operated intentionally (actively) by the person, the awareness driven display device transmits the information to the person when the device is made aware by chance (passively) by the person with a change of its appearance. The constructed display device is designed taking account of a houseplant. It consists of 12 leaf modules and a flower module. It displays the information by means of leaf poses and extent of flower opening. The leaves can also sense a touch action. A display experiment was conducted to evaluate the device. The activity of a remote target person is calculated from actual sensed daily life data and displayed to the subject person. Experimental results show the effectiveness of the display system. It reveals that a larger change of pose and flower opening evokes more effective awareness of the activity of the person. This display device in the future will be able to be utilized as a communication tool to generate a virtual feeling of cohabitation even though the remote target person lives apart from the subject person.

A lying person's motion tracking system by using a pressure distribution image and a full body model is proposed. The full body model consists of a skeleton and a surface model to cope with a variety of body shapes. BVH files are used as the skeleton model that describes a hierarchy of joints and links. Wavefront object files are used as the surface model that describes geometry of the surface. The bed has 210 pressure sensors that are under the mattress. It can measure a pressure distribution image of a lying person. The lying person's motion is tracked by considering potential energy, momentum and a difference between the measured pressure distribution image and a pressure distribution image that is calculated by the full body model. Experimental results reveal that the realized system can track not only horizontal motions such as opening and closing legs but also vertical motions such as raising the upper body.

Since human behavior patterns vary from person to person, computers have to understand someone's personal behavior in order to supply him with a robotic support system for his daily life. It is necessary to know his behavior over the long term. The ideal intelligent support system can measure the data about human actions in the daily life, accumulate them, understand the behavior and the patterns thereof, and adapt thereto. An accumulation and summarization system of human daily action data based on sensing in 1 room, "Robotic Room II", is designed and the accumulation system integrating with the measurement parts is implemented. It is confirmed that accumulation over the long term is possible by experiments for accumulation of human action data in ordinary life. Moreover, as the first step of an analysis of the accumulated data, a summarization algorithm based on the changes of the room states is proposed. It is confirmed that the summarized data enables us to catch the most of the changes of the room states. Finally, a possibility of application for life monitoring by means of summarization is also illustrated.

This paper proposes a human-like action recognition system which can recognize human actions like the human does. The recognition algorithm is constructed taking into account of the following characteristics of human action recognition: simultaneous recognition, priority among actions, fuzziness of judgment, multiple judge conditions for one action, and recognition ability from a partial view of the body. The experiment shows that the system can recognize human action like human does. The stated results ensure a natural understanding of human action by a system, which leads to smooth communication between computer systems and human.

The authors developed a novel infant behavior recognition system based on a pressure distribution image. The system can recognize an infant's status (quiet, moving and crying), posture, body parts' positions and movement unrestrainedly. It can, in recognizing the behavior, cope with the infant's rapid growth and unique physique. The algorithm of the infant behavior recognition system is summarized as follows. 1) First, the system measures the pressure distribution image with 384 pressure sensors distributed in the bed. 2) The authors propose "activity score"; this is calculated by using the measured pressure distribution image and indicates kinetic energy of the infant's activity. Based on the activity score, the system decides the infant's status. 3) If the infant is quiet, the system estimates the infant's physique. 4) Based on the estimated physique, the system recognizes the infant's posture and body part movement. Experimental results reveal that the system successfully recognizes infants' status (quiet, moving and crying), posture, body parts position and movements.

This paper presents "Sensor Pillow System" to measure physiological parameters in sleep without restraint to a human. The system consists of an array of pressure sensors under the pillow, a one-chip microcomputer to digitize and transmit the pressure data to a desktop computer; and the computer to count respirations and turns in sleep. This paper also presents a simple motion model which explains the change of the head pressure distribution accompanied with respiration. Based on this model, respiration count algorithms is proposed. The effectiveness of this system is experimentally shown by comparing the number of respirations and turns counted by the sensor pillow system of a medical device and a video image.

A one-room-type sensing system named Robotic Room 11 for recognizing and accumulating human behavior in ordinary life is constructed. In order to recognize human behavior, we propose an idea that human behavior can be estimated from the human position, posture, the objects surrounding the human and time. Based on this idea, multiple sensors are distributed on such objects as floors, beds and desks. From these sensors, information of a human position and use conditions of household appliances are extracted and integrated. Experiments of sensor information description are successfully performed to show that the system can recognize and accumulate human behaviors.

We develop a body parts position and posture estimation system. This system consists of a pressure sensor distributed bed and body parts position and posture estimation software. The computer constructs many pressure distribution images based on the simple human models and accumulates these images to its memory, called the model-based pressure image templates. The pressure distribution image through is compared with model-based pressure image templates to find out the most matched pressure image template. The model-based pressure image templates contain body parts positions and joint angles information, so the body parts position contacting with the bed can easily be estimated by using the most matched model-based pressure image template. Finally, an estimated posture is displayed as a 3D computer graphics image. An experimental result reveals that the system not only can display the estimated lying human posture intuitively, but can also estimate the lying human's body parts position accurately where the body parts contacts with bed.

Visually based control, accumulation and assistance systems are presented as an effective environment for teleoperation. We developed the bilateral behavior media (BBM) paradigm and implemented systems which offer a status driven interface, collection of sampled control behavior, and operator assistance. The paradigm emphasizes a human-intuitive visual specification of task completion states without resorting to image understanding, modeling or extensive calibration. Visually based teleoperation has been effectively applied in a variety of tasks where an operator's past experience in the macroworld is not applicable to the physics and scenario experienced in the microworld. Experiments of manipulating individual biological cells have shown the success of the visually based BBM paradigm.

Making and maintaining human-pet-bonds is the most essential task for pet robots. To realize this task, the authors propose a methodology of linking psychological interactions between a person and a pet robot. Production of an animal-likeness impression toward a person is one of the most important objectives for interactive pet robots. Animal-like impression can have power to effect human psychological conditions and also human behaviour. We propose 3 methods to produce animal-likeness in artifacts, namely morphological appearance method, body temperature method, and vital body movement stimulus method. In addition to producing impressions, the authors measure human behavioural uncertainty quantitatively in order to show the relationship between human behavioural attitudes towards a pet robot and what he/she feels about it. These methods for producing impressions and measuring human behaviour will be the key to founding a close and friendly bond between human beings and pet robots.

Proposes a visually based teleoperation with accumulation method and system where the visual and control information sequences are stored and available for syntactically indexed playback. Visually based teleoperation systems heretofore described in the literature unfortunately can only display an instantaneous representation of the control sequence. Past control experience should instead be accumulated and available to facilitate teleoperators. In this paper, the behavior sampling extraction theory and method, the behavior sampling data representation, and portions of the status on demand function-set are introduced. Behavior sampling utilizes semiotic analysis of the teleoperator control behavior to syntactically segment the control motions and the relevant visual objects into a syntactically indexable storage stream. A preliminary system used for manipulating single biological cells under an optical microscope (fitted with a video camera) is described. Repetitive manipulation experiments on Mato fluorescent granular perithelial (FGP) cells show the effectiveness of this enhancement to the visually based control approach.

Human activity support is being established as a novel active area for robotics. It is expected to make clear the mechanism of understanding situation and intention of humans through regarding behavior as media and the mechanism of human support based on it. This paper shows a method to support handiwork by recognizing human action conforming to situation and shows a soldering support tool constructed based on the method as an example.

To support human activities is being established as a novel activity area for robotics. The mechanism to understand situation and intention of human from her/his behavior is expected to be made clear. Action recognition method will be indispensable to realize the mechanism. In this paper, we describe a system which finds human in a scene and tracks the human using vision to recognize actions. The proposed system is network oriented which distributes a lot of fixed stereo pair cameras, and tracks human as a whole. The constructed vision system utilizes skin-color extraction and template matching based tracking. Human action recognition experiment using the system is shown.

This paper proposes a function for monitoring a patient's respiration and posture in sleep using a human symbiosis system in a sickroom. The novel features of the system function lie in noninvasive and unrestrained monitoring to ensure the symbiosis with the human. Noninvasive monitoring eliminates the need for monitoring needles or catheters to invade the patient's body; sensors thus do not impose a physiological burden such as pain on the patient. In unrestrained monitoring, sensors and their electrical cords do not limit degrees of freedom of the patient's movement. Unrestrained sensing therefore does not impose a psychological burden caused by the limitations on the patient. A "robotic bed", which is a system to realize the functions stated above, is a bed-shaped system with 221 pressure sensors for monitoring the patient's respiration and posture without preventing doctors and nurses from performing their tasks. The sensors are set 5 cm from each other on the bed and surround the patient. Experiments to monitor respiration and posture in sleep demonstrate that the proposed function is feasible for monitoring for over 6 hours.

This paper proposes a contact interaction robot (CIR) which utilizes contact behavior as the interaction means between a human and a robot. The CIR is a puppet robot designed so that the robot and the human touch each other. The psychological experiments are performed by utilizing a CIR equipped with pressure sensors on both sides of its neck and six servo motors in its neck, two arms and, two legs. The experimental results reveal that the CIR is able to moderate the painfulness perceived by the human as well as to bring a sense of relief.

This paper describes a method for tracking moving objects with multiple cooperative visual attention regions. Each attention region is tracked using a sum of absolute difference (SAD) correlation technique. The method is based on an implementation of a transputer-based multi-processor vision system with a correlation processor. The vision system provides a large number of attention regions, each of which can be tracked using fast SAD correlation performed in an LSI MEP. We call these regions windows since the region resembles a clipped section of a whole view. On the top of the system, we built a software manager which facilitates program image processing, motion control of a window, and cooperation control of windows. The cooperation is achieved using constraints, which are tree-structured, between windows. Cooperation of multiple attention regions enables one to track route predictable objects, deformable objects, and structured objects in a natural scene. As examples, passing car tracking, human arm tracking, and finger action recognition are shown.

This paper proposes a system for computerized description of medical care, and also shows its possibility by experimenting visual understanding of medical care by a doctor. Proposed system supports forming the medical documents by sensing and inputting the doctor's behavior into a computer during medical care. In the system, the computer senses a doctors' behavior using visual, aural and force sensors, and uses these sensor outputs to understand what care is being performed. The visual understanding of medical care is a core function of the system. Taking application of medicine to the ear, nose and throat in ortholaryngology clinics as a typical example of medical care, visual understanding of medical care is successfully performed. The procedure of visual understanding in the experiment is as follows: Firstly, CCD cameras monitor the objects(tools, medicine and the patient's affected body parts) whose states change in accordance with the doctor's actions. Secondly, the doctor's action is identified with the change in the state of these objects. Finally, the doctor's behavior is understood as the sequence of these identified actions. In the procedure: visual behavior understanding is more easily realized by monitoring objects used for behavior than by conventional method of monitoring the persons. Medical care description in, this paper will contribute to improve the quality of medical service by reducing the burden of medical information recording on doctors.

This paper proposes a robot behavior notation named "Behavioral Score" and a methodology of generating "familiar" robot behavior. Behavioral Score, designed with analogies to a musical score, is an effective method of describing and controlling the expressive behavior of a robot. The effectiveness of Behavioral Score is demonstrated by a choreographed robot dance. The methodology described is based upon the hypothesis that the passive reactions of a robot produce a familiar, comfortable, imprecision in human beings. Utilizing the robot behavior for generating familiarity will facilitate human-robot symbiosis. The results of psychological experiments verifies the validity of the methodology.

This paper proposes a third generation teleoperation system based on a status-driven slave symbolically instructed by the master control panel. Furthering the first generation joint-angle control techniques and second generation coordinate transformation techniques, this system recognizes the target status specified by the operator by extracting the task status from visual sensors. The task environment is initially described via sensing points in the work environment and on the manipulated object. The relationships between the sensing points and the change of those relationships describes the task at hand. Operation can proceed in automatic mode where the system completely directs the slave or in shared mode where the system assists the operator by constraining the slave motion while the operator directs via a visual communication interface. Although status-driven techniques offer several improvements in teleoperated control, this work explores the robustness and reliability provided in dynamic workspace environments. One successful application of this technique is in micrometer scale part handling, the so-called "microworld assembly" where an operator's past experience in the macro world is not applicable to the physics experienced in the microworld.