Understanding how cities are perceived from on-site visitors' perspectives can provide valuable insights for urban planning and development applications. However, existing studies estimated people's perceptions by having them view photographed landscape images; the scores derived by these methods were thus merely quantified impressions of specific viewpoints that do not necessarily represent perceptions people would have were they at the site. To address this issue, we developed a framework, named Omni-CityMood, for quantifying people's on-site perceptions of urban atmospheres. Based on the idea that the viewpoint influences the perception of an urban landscape, the proposed framework identifies critical viewpoints of a location by using both visual-based features of landscape images and geographical characteristics of the site. In particular, Omni-CityMood enables the mood of a location to be evaluated from viewpoints over a range of 360 degrees by leveraging the techniques of neural recommendation systems. We evaluated Omni-CityMood on a dataset we built that includes perceived atmosphere experiences in various cities. Experiments and extensive analyses demonstrate the promising capability of modeling landscape viewpoints to quantify urban on-site atmospheres.

In recent years, wireless sensing techniques, such as Wi-Fi and Ultra-Wideband (UWB) signals, have gained attention for activity recognition due to their ability to address privacy concerns associated with traditional computer vision methods. While UWB Channel Impulse Response (CIR) is believed to be prominent approach as well as Wi-Fi Channel State Information (CSI), research on its application in activity recognition remains limited. Previous studies have not fully explored the brevity of single measurements or the potential for feature extraction from CIRs. This paper presents a novel approach to robust device-free activity recognition by exploiting periodic UWB CIR samples. Utilizing multi-level wavelet packet decomposition (WPD) and a customized attention mechanism, the proposed method effectively combines multi-resolution features, improving recognition accuracy and reducing the need for extensive fine-tuning. Experiments conducted in various scenarios validate the performance of the proposed approach, with ablation studies demonstrating the superiority of multi-resolution analysis over short-time Fourier transform (STFT) and highlighting the cost efficiency of the method.

When visiting universities, you might notice the distinctive atmospheres of each university, such as a calm and serious environment or a lively enthusiasm for sports. Capturing these atmospheres could help in promoting universities and fostering development in the communities around the universities. To explore the atmospheres of universities, we analyze the thoughts and interests of university community members, such as students and faculty members. Specifically, we use a large-scale dataset derived from search queries and GPS logs to quantify visitors’ interests. Additionally, to extract the meaningful atmospheres of universities, we apply topic modeling to the dataset.

Forecasting rail congestion is crucial for efficient mobility in transport systems. We present rail congestion forecasting using reports from passengers collected through a transit application. Although reports from passengers have received attention from researchers, ensuring a sufficient volume of reports is challenging due to passenger's reluctance. The limited number of reports results in the sparsity of the congestion label, which can be an issue in building a stable prediction model. To address this issue, we propose a semi-supervised method for congestion forecasting for trains, or SURCONFORT. Our key idea is twofold: firstly, we adopt semi-supervised learning to leverage sparsely labeled data and many unlabeled data. Secondly, in order to complement the unlabeled data from nearby stations, we design a railway network-oriented graph and apply the graph to semi-supervised graph regularization. Empirical experiments with actual reporting data show that SURCONFORT improved the forecasting performance by 14.9% over state-of-the-art methods under the label sparsity.

Forecasting the number of visitors at a public event, termed event crowd forecasting (ECF), has recently garnered attention due to its social significance. Although existing ECF methods have pioneered successful feature design by considering event contents with contexts (e.g., weather, type of day, time), their scalability across different event types is limited due to the necessity of costly feature engineering. To address this issue, we propose a novel ECF framework, named EventOutlook. Based on our observation of various events, online event announcements indicate the factors that induce crowded events. Thus, we incorporate event announcements into ECF methods. To handle such unstructured data, which have no unified format among events, we leverage large language models (LLM) to extract crowding factors and embed them into an LLM-driven crowding-indicator feature (LCIF). Empirical experiments with real-world event data show that EventOutlook significantly improved ECF performance compared to state-of-the-art methods.

Driving behavior modeling is crucial in autonomous driving systems for preventing traffic accidents. Inverse reinforcement learning (IRL) allows autonomous agents to learn complicated behaviors from expert demonstrations. Similar to how humans learn by trial and error, failed demonstrations can help an agent avoid failures. However, expert and failed demonstrations generally have some common behaviors, which could cause instability in an IRL model. To improve the stability, this work proposes a novel method that introduces time-series labeling for the optimization of IRL to help distinguish the behaviors in demonstrations. Experimental results in a simulated driving environment show that the proposed method converged faster than and outperformed other baseline methods. The results also show consistency for various data balances of the number of expert and failed demonstrations.

Prediction of the Received Signal Strength Indicator (RSSI) distribution is a very important task. However, most of the current research is on methods that complement the RSSI distribution for beacons that actually collect data. The most common method for fully online simulation without beacons is based on physical Ray-Tracing. However, the Ray-Tracing model requires the determination of the attenuation rate of the wall. This makes it difficult for ordinary people to perform the simulation. Also, without measuring the actual RSSI, it is impossible to know whether the simulation results are appropriate for the actual environment or not. To address these issues, we propose a data-driven RSSI simulation method. RSSI can be collected by various devices, and it is easy for the general public to obtain RSSI for each location. The simulation is based on the actual RSSI data, so that the simulation can be performed in a realistic environment. In this paper, we have realized a data-driven simulation that matches the environment by learning the attenuation of RSSI derived from the environment and actual data using Generative Adversarial Network(GAN). In order to conduct experiments in a real environment, the simulation model is trained in an office environment, and its accuracy is evaluated using actual RSSI values. As a result, the average absolute error of RSSI values was improved by 8% and the average positioning error of indoor localization was improved by 19% compared with the simulation using the radio propagation formula.

Indoor position information, which is difficult to obtain by GPS, can be used for various services and applications, thus indoor localization methods have been widely researched. Among them, device-free indoor localization does not require the target persons to possess a localization device, such as a smartphone, which can support the localization of all persons in the environment. Many methods using RSSI and Wi-Fi CSI have been proposed as device-free indoor localization using radio waves. However, RSSI is easily affected by environmental factors, such as multipath propagation. Wi-Fi CSI also has the disadvantage that there is no standard, so it relies on special hardware and software. Therefore, we propose device-free multi-person indoor localization using ToF information, which is less susceptible to noise. ToF information in indoor localization has mainly been used for highly accurate estimation of the receiver position, while this paper proposes a new application. In addition, distance measurements using ToF of Wi-Fi and UWB have been standardized and can be implemented with commercial equipment, which is highly practical. In this research, we considered the change in the distance measurement due to radio wave occlusion by a human body and built a device-free indoor localization system that can estimate multi-person position even if the model is trained on the data of only one person.

Advanced driver assistance systems have gained popularity as a safe technology that helps people avoid traffic accidents. To improve system reliability, a lot of research on driving behavior prediction has been extensively researched. Inverse reinforcement learning (IRL) is known as a prominent approach because it can directly learn complicated behaviors from expert demonstrations. Because driving data tend to have a couple of optimal behaviors from the drivers' preferences, i.e., sub-optimality issue, maximum entropy IRL has been getting attention with their capability of considering suboptimality. While accurate modeling and prediction can be expected, standard maximum entropy IRL needs to calculate the partition function, which requires large computational costs. Thus, it is not straightforward to apply this model to a high-dimensional space for detailed car modeling. In addition, existing research attempts to reduce these costs by approximating maximum entropy IRL; however, a combination of the efficient path planning and the proper parameter updating is required for an accurate approximation, and existing methods have not achieved them. In this study, we leverage a rapidly-exploring random tree (RRT) motion planner. With the RRT planner, we propose novel importance sampling for an accurate approximation from the generated trees. This ensures a stable and fast IRL model in a large high-dimensional space. Experimental results on artificial environments show that our approach improves stability and is faster than the existing IRL methods.

As intelligent sensing and smartphone technologies have progressed, a huge amount of highly heterogeneous data have come to be stored in smartphones and uploaded to servers for analysis on a daily basis. This has led to vast storage overheads for users and companies. Hence, data compression becomes the most efficient strategy for suppressing the increase in storage overhead. Compressed sensing (CS) technology is one approach to compressing data, but traditional CS-based algorithms are significantly time-consuming and have low reconstruction performance. In light of these drawbacks, this paper proposes a compressed sensing framework that instead takes advantage of the low time cost and adaptive learning capability of deep learning methods, wherein a convolutional neural network (CNN) is used for compressing and reconstructing acceleration data. Our experiments with actual smartphone acceleration data show that the proposed method dramatically improves the reconstruction performance with very little reconstruction time compared with traditional compressed sensing methods.

Early crowd dynamics forecasting, such as one week in advance, plays an important role in risk-aware decision-making in urban regions such as congestion mitigation or crowd control for public safety. Although previous approaches have addressed crowd dynamics prediction, they have failed to deal with the scarcity of anomalous events, which results in a large model bias and could not quantify the number of visitors in anomalous crowd gathering. To provide an elaborate early forecast, we focus on the successive properties of importance weighting (IW) to penalize the anomalous data in terms of model bias; however, leveraging the concept of IW is challenging because dividing dataset into normal and abnormal sets is difficult. Motivated by these challenges, we propose CityOutlook, a novel forecasting model based on unbiased regression with importance-based reweighting. To make IW applicable to our approach, we design an anomaly-aware data annotation scheme by utilizing the heterogeneous property of mobility data to determine the data anomaly. We evaluate CityOutlook using the datasets of large-scale mobility and transit search logs. The experimental results show that CityOutlook outperforms the state-of-the-art models on crowd anomaly forecast, providing the same level accuracy in forecasting normal dynamics.

With the pandemic of COVID-19, indoor crowd density monitoring has become one of the most critical responsibilities of public space managers. Beacon placement optimization has been tackled as fundamental research work as the performance of crowd density monitoring highly depends on how BLE beacons are allocated. In this research, we propose a novel beacon placement optimization approach to incrementally place the beacon on the updated detection status adaptively in favor of Bayesian optimization, which can help to provide the optimal beacon placement. Our proposed method can optimize the beacon placement effectively to improve the signal coverage quality in the given environment and minimize human workload.

Driving behavior modeling (DBM) is widely used in the intelligent vehicle field to prevent accidents, which predicts actions that vehicles should take to optimize safe driving behaviors. According to some statistics, accidents easily happen at un-signalized intersections. Modeling driving behavior at such places is of great importance. However, current inverse reinforcement learning-based DBM methods fail to predict proper behaviors at the un-signalized intersections in the aspects of smoothness and stopping behavior by just using a single Markov decision process (MDP). We propose a novel sequential MDPs approach to model the driving behavior at the un-signalized intersections to solve the problems. Our approach decomposes the target behavior through the un-signalized intersections into three parts and models each decomposition's driving behaviors with appropriate time durations by a stopping-time-interval distribution through dynamic programming. Experiments on real driving data show that the proposed method achieved a better result and successfully improved the smoothness and stopping awareness of the planned driving path compared to the baselines.

The analysis of crowd flow in urban regions (urban dynamics) from GPS traces has been actively explored over the last decade. However, the existing prediction models assume that the population density in the analysis area is almost uniform, making it difficult to analyze fine-grained urban dynamics on a nationwide scale, where urban and rural areas coexist. In this paper, we propose a predictive model, called mixed-order Poisson regression (MOIRE), to capture changes in active populations nationwide by combining lower-order patterns and higher-order interaction effects. The proposed method utilizes multiple pieces of contextual information that greatly affect crowd flows (e.g., time-of-day, day-of-the-week, weather situation, holiday calendar information). We evaluated MOIRE on two massive GPS datasets gathered in urban regions at different scales. The results show that it has better predictive performance than the state-of-the- art method. Moreover, we implemented an anomaly detection system in urban dynamics for the whole nation of Japan in accordance with MOIRE specifications. This application enabled us to confirm MOIRE's performance intuitively.

Forecasting anomalies in urban areas is of great importance for the safety of people. In this paper, we propose Supervised-CityProphet (SCP), an anomaly score matching-based method towards accurate prediction of anomalous crowds. We re-formulate CityProphet as a regression model via data source association with mobility logs and transit search logs to leverage user's schedules and the actual number of visitors. We evaluate Supervised-CityProphet using the datasets of real mobility and transit search logs. Experimental results show that Supervised-CityProphet can predict anomalous crowds 1 week in advance more accurately than baselines.

Research on advanced driver assistance systems for reducing risks to vulnerable road users (VRUs) has recently gained popularity because the traffic accident reduction rate for VRUs is still small. Dealing with unexpected VRU movements on residential roads requires proficient acceleration and deceleration. Although fine-grained prediction of driving behavior through inverse reinforcement learning (IRL) has been reported with promising results in recent years, learning of a precise model fails when driving strategies vary with contextual factors, i.e., weather, time of day, road width, and traffic direction. In this work, we propose a novel multi-task IRL approach with a multilinear reward function to incorporate contextual information into the model. This approach can provide precise long-term prediction of fine-grained driving behavior while adjusting to context. Experimental results using actual driving data over 141 km with various contexts and roads confirm the success of this approach in terms of predicting defensive driving strategy even in unknown situations.

People often turn off location logging when the batteries of their smartphones get low, to reduce the phone’s power consumption and prolong its operation. Here, we propose an innovative data sharing scheme called as the Parasitic Location Logging (PLL). PLL can acquire location of such users, what we call parasitic users, without invoking any location functionalities by the GPS and Bluetooth low energy (BLE) sensors of their smartphones. PLL estimates parasitic users’ location and trajectory by relying on other users who pass by the parasitic user, what we call host users, as evidence that they are located in close proximity. The results of field experiments showed that PLL dramatically decreases battery consumption of parasitic users’ smartphones and that the position of parasitic users can be identified accurately. Moreover, the battery consumption of PLL was rigorously evaluated in a laboratory setting to demonstrate its benefit. An agent simulation evaluating the proposed calculation algorithm under various conditions in realistic environments validated the robustness of PLL.

With the wide use of smartphones with Global Positioning System (GPS) sensors, the analysis of the population from GPS traces has been actively explored in the last decade. We propose herein a brand new population prediction model to capture the population trends in a fine-grained point of interest (POI) densely distributed over large areas and understand the relationship of each POI in terms of spatiality preservation. We propose a new framework, called Spatiality Preservable Factorized Regression (SPFR), to realize this model. The SPFR is inspired by the success of the recently proposed bilinear Poisson regression and the concept of multi-task learning with factorization approach and the graph proximity regularization. Given that the proposed model is written simply in terms of optimization, we achieve scalability using our model. The results of our empirical evaluation, which used a massive dataset of GPS logs in the Tokyo region over 32 M count logs, show that our model is comparable to the stateof-the-art methods in terms of capturing the population trend across meshes while retaining spatial preservation in finer mesh areas.

Wi-Fi fingerprint-based localization is known to be prominent for indoor positioning technology; however, it is still challenging on sustainability of its performance for long-term use due to distribution drifts of the signal strength across time. Therefore, the laborious continual surveys on fingerprint are inevitable. In this paper, we propose a new scheme for solving the large cost of maintaining common Wi-Fi fingerprint-based localization with machine-learning-based way by efficient incremental learning (retraining). Specifically, we propose a brand new retraining method, called GroupWi-Lo, that focuses on minimization of parameter variation with respect to the incremental surveys on fingerprint (i.e., calibration). Our method tries to keep the parameters of the previously trained model unchanged while minimizing the error on the dataset obtained in the last surveys. This formulation is helpful to keep robustness against overfitting from the limited size of the dataset per survey. The experimental results both in the lab and the uncontrolled environment show that GroupWi-Lo achieves competitive performance among the state-of-the-art methods, while its computational cost retains independent of the number of surveys compared with existing the semi-supervised approach and standard incremental training approach.

Predicting behaviors of a population from location-oriented log data from smartphones, i.e., urban population dynamics, has become more common in mobile and pervasive computing. A bilinear representation approach has been proposed to improve the prediction accuracy of urban population dynamics by adding contexts such as geographical information and day of the week. However, this approach has a strong limitation in that additional contexts can not be directly utilized in this representation with a unified manner. To resolve this issue, we propose a new predictive model for urban population dynamics based on multilinear Poisson regression so as to handle multiple contexts in a systematic manner. The model is parameterized using a tensor and can be optimized by using an efficient convex optimization with a sequence of matrix parameter optimizations. An empirical evaluation with large-scale smartphone location data showed that our model outperforms conventional approaches.

Maximum entropy inverse reinforcement learning (MaxEnt IRL) is an effective approach for learning the underlying rewards of demonstrated human behavior, while it is intractable in high-dimensional state space due to the exponential growth of calculation cost. In recent years, a few works on approximating MaxEnt IRL in large state spaces by graphs provide successful results, however, types of state space models are quite limited. In this work, we extend them to more generic large state space models with graphs where time interval consistency of Markov decision processes are guaranteed. We validate our proposed method in the context of driving behavior prediction. Experimental results using actual driving data confirm the superiority of our algorithm in both prediction performance and computational cost over other existing IRL frameworks.

Thanks to the recent popularity of GPS-enabled mobile phones, modeling people flow or population dynamics is attracting a great deal of attention. Advances in methods where regular population patterns with respect to factors such as holidays or weekdays are extracted have provided successful results in irregularity detection. With large-scale crowded events such as fireworks, it is crucial that there be enough time to take countermeasures against the irregular congestion, i.e., irregularity prediction. It remains a tough challenge to predict population from GPS trace logs with existing methods.

RSSI-based indoor localization is getting much attention. Thanks to a number of researchers, the localization accuracy has already reached a sufficient level. However, it is still not easy-to-use technology because of its heavy installation cost. When an indoor localization system is installed, it needs to collect RSSI data for training classifiers. Existing techniques need to collect enough data at each location. This is why the installation cost is very heavy. We propose a technique to gather data efficiently by using machine learning techniques. Our proposed algorithm is based on multi-task learning and Bayesian optimization. This algorithm can remove the need to collect data of all location labels and select location labels to acquire new data efficiently. We verify this algorithm by using a Wi-Fi RSSI dataset collected in a building. The empirical results suggest that the algorithm is superior to an existing algorithm applying single-task learning and Active Class Selection.

Understanding people flow in a city (urban dynamics) is of great importance in urban planning, emergency management, and commercial activity. With the spread of smart devices, many studies on urban dynamics modeling with mobility logs have been conducted. It is predictive analysis, not analysis of the past, that enables various applications contributing to a more prosperous society. To deal with the non-linear effects on urban dynamics from external factors, such as day of the week, national holiday, or weather, we propose a low-rank bilinear Poisson regression model, for a novel and flexible representation of urban dynamics predictive analysis. The results obtained from an experiment with one year's worth of mobility records suggest the high prediction accuracy of the proposed model. We also introduce the following applications: regional event detection via irregularities, visualization of urban dynamics corresponding to urban demographics, and extraction of urban demographics of unknown point of interests.

Predicting defensive driving is a promising technology for novel advanced driver assistance systems. In recent years, modeling driving behavior in residential roads through inverse reinforcement learning (IRL) has been attracting attention in intelligent vehicle community thanks to the superiority of this approach providing long-term prediction of fine-grained driving behavior. However, it suffers from poor performance in diverse environment due to the fact that the single reward function could not handle all the environment with large diversity. Towards this issue, a novel IRL framework with multiple reward functions to deal with environmental diversity is proposed in the paper. Specifically, the model employs Dirichlet process mixtures as a flexible and powerful Bayesian model to divide the environment into clusters and learns the parameters in each cluster simultaneously. Experimental result with expert driver behavior data shows that our model with multiple reward functions provides superior performance over the IRL model with single reward function. It also suggests that the clustering of environments based on the driving behavior of professional drivers could be useful on evaluating driving environments.

This paper describes a pedestrian population trend estimation method using location data of smartphone users. This technique is intended to be an alternative to traffic censuses using tally counters. Traffic censuses using tally counters are still commonly used to survey the number of pedestrians despite their cost and limitations in area and time.

Crowdsensing technologies are rapidly evolving and are expected to be utilized on commercial applications such as location-based services. Crowdsensing collects sensory data from daily activities of users without burdening users, and the data size is expected to grow into a population scale. However, quality of service is difficult to ensure for commercial use. Incentive design in crowdsensing with monetary rewards or gamifications is, therefore, attracting attention for motivating participants to collect data to increase data quantity. In contrast, we propose Steered Crowdsensing, which controls the incentives of users by using the game elements on location-based services for directly improving the quality of service rather than data size. For a feasibility study of steered crowdsensing, we deployed a crowdsensing system focusing on application scenarios of building processes on wireless indoor localization systems. In the results, steered crowdsensing realized deployments faster than non-steered crowdsensing while having half as many data.

“Grasping by caging” has been considered as a powerful tool to deal with uncertainty. In this paper, we continue to explore into “grasping by caging” and propose a new solution by using eigen-shapes and space mapping. For one thing, eigen-shapes fix dexterous hands into a series of finger formations and help to reduce dimensionality and computational complexity. For the other, space mapping builds a mapping between rasterized grids in 2-D Work space (W space) and rasterized voxels in 3-D Configuration space (C space) and helps to rapidly reconstruct C space so that we can efficiently measure the robustness of caging and find an optimal caging configuration for grasping. Our algorithm can work rapidly and squeezingly cage any 2-D shapes, including objects with either convex boundaries, concave boundaries, 1-order or high-order boundaries and even objects with inner holes. We implement the algorithm with MATLAB and carry out experiments with WEBOTS simulation to test its robustness to uncertainties. The results show that our algorithm can work well with various object shapes and can be robust to noisy control and noisy perception. It is promising in the power grasping tasks of dexterous hands.

In spite of IT innovation, people cannot get rid of non-creative tasks of searching daily-use objects at home. This paper presents a drawer-type storage system for supporting object search, “TansuBot”. By using this system and a smart device (e.g. smart phone), a user can review the photos of contents stored in the system. In addition, the system can present candidate drawers where the searching target object may be stored based on preliminary information such as usage histories. Concretely, LED blinking and pop-up actions (pushing drawers forward) are used for display. To realize these supports, a stacker crane type wall-moving robot is equipped at the backside of storage. The robot has a movable camera and mechanisms to push a drawer forward. For easy installation to a home, storage efficiency and cost reduction should be considered in the design of the instrument. Especially for cost reduction, this paper presents an approach to use wooden parts for main mechanisms. This approach also contributes to user-friendly presence and appearance of the instrument. This paper reports about the development of a prototype and an experiment to evaluate the functions for supporting object search. The results of the experiment prove the importance of the functions realized by the system; displaying contents' photos on a smart device and showing candidate drawers to investigate. The outcomes indicate that those functions have positive effects on reduction of searching time and mental burden.

Caging can offer robustness to uncertainties in grasping. If a robotic hand is designed based on the idea of caging, it would probably work well with noisy perception devices and low-quality control. This paper takes into account these merits and designs and implements a gripping hand based on the idea of caging. The gripping hand is concise and offers a low-cost alternative to co-operate with noisy data and low-quality control. According to previous work, we need four fingers to cage any 2D objects. That is to say, if each finger has one, two or three degree of freedoms, we will totally need four, eight or twelve actuators. The large number of actuators would be costly. This paper simplify the number of actuators into one by quantitatively analyzing finger formations with caging tests conducted on both random objects and objects from MPEG-7 shape database. It successfully lowers costs while maintains high performance. Following the simplified one-actuator design we implement a gripping hand by modifying a SCHUNK RH707 hand and carried out experiments with a manipulator built on the Neuronics Katana arm. The one-actuator gripping hand could work well with common depth cameras (Swiss Ranger) and pick up various objects. It bridges the gap between caging theories and applications and demonstrates the merits of caging.

Scores of health state for elderly people are regarded as important in nursing or medical fields. On the other hand, gaining the scores needs nurses to execute questionnaires. Owing to this, the execution rate for the health assessment is still low in ordinary homes. To solve this problem, we propose a method to predict the health score by using low-invasive sensors. We adopt regression as the prediction method and construct features to absorb the individual difference. As a part of feasibility study of social participation for elderly people, we execute the survey of health state using questionnaires by a nurse and install low-invasive sensors in real life simultaneously. Experimental result in the feasibility study shows a promise of the score prediction from sensor data. In addition, the result suggests that the extraction of features related to living behaviors improves the accuracy compared to using raw sensor data.

This paper presents a caging approach which deals with planar boundary clouds collected from a laser scanner. Given the boundary clouds of a target object and two fixed finger positions, our aim is to find potential third finger positions that can prevent target from escaping into infinity. The major challenge in working with boundary clouds lies in their uncertainty in geometric model fitting and the failure of critical orientations. In this paper, we track canonical motions according to the rotational intersection of Configuration space fingers and rasterize Work space with grids to compute the third caging positions. Our approach can generate the capture region with max(O(np),O(h2)) ≤ O(n2) cost where n denotes the resolution of grid rasterization, p denotes the resolution of canonical rasterization and h denotes the resolution of boundary rasterization or the number of boundary cloud points. Moreover, we propose a rough approximation which measures a subset of the possible positions by contracting rotations, indicating computational complexity of max(O(n),O(h2)). In the experimental part, our proposal is compared with state-of-the-art works and applied to many other objects. The approach makes caging fast and effective.

This paper presents a novel approach to deal with uncertainty in grasping. The basic idea is to initiate a caging manipulation state and then shrink fingers into immobilization to perform a practical grasping. Thanks to flexibility from caging, this procedure is intrinsically safe and gains tolerance towards uncertainty. Besides, we demonstrate that the minimum caging is immobilization and consequently propose using three or four fingers to manipulate planar convex objects in a grasping-by-caging way. Experimental results with physical simulation show the robustness and efficacy of our approach. We expect its leading benefits in saving finger number, conquering low-friction materials and especially, dealing with pose/shape uncertainty.

In this paper, we describe a novel sensor device which recognizes hand shapes using wrist contours. Although hand shapes can express various meanings with small gestures, utilization of hand shapes as an interface is rare in domestic use. That is because a concise recognition method has not been established. To recognize hand shapes anywhere with no stress on the user, we developed a wearable wrist contour sensor device and a recognition system. In the system, features, such as sum of gaps, were extracted from wrist contours. We conducted a classification test of eight hand shapes, and realized approximately 70% classification rate.

In this paper we propose a novel method for predicting resident's behaviors in a house from one's movement trajectories. The method consists of 1) segmentation of trajectory data into staying or moving and classification of the segments and 2) prediction by time-series association rules from transition events of each segment. The method predicts the start time of target behaviors for daily life support, such as eating, taking a bath etc. The time lag between the prediction and the target behavior can be set up manually, thus the method is adaptable to a variety of supporting systems. The experimental results using real residents' trajectory data of almost two years demonstrate that prediction of behaviors by the proposed method is feasible.

This paper describes a human shape reconstruction method from multiple cameras in daily living environment, which leads to robust markerless motion capture. Due to continual illumination changes in daily space, it had been difficult to get human shape by background subtraction methods. Recent statistical foreground segmentation techniques based on graph-cuts, which combine background subtraction information and image contrast, provide successful results; however, they fail to extract human shape when furniture such as tables and chairs are moved. In this paper, we focus on the results of face detectors that would be independent of such background changes and help to improve the robustness under movement of background objects. We propose a robust human shape reconstruction method with the following two characteristics. One is iterative image segmentation based on graph-cuts to integrate head position information into shape reconstruction. The other is high-precision head tracker to keep multi-view consistency. Experimental results show that proposed method has enhanced human pose estimation based on reconstructed human shape, and enables the system to deal with dynamic environment.

Vision based human articulated body pose tracking has been historically important. Because analyzing multiple human activities, especially interaction between human in cluttered scenes is essential in visual surveillance scenarios, multiple people tracking has enjoyed much attention in human robot interaction research in recent years. In this paper, we newly introduce a robust framework for multiple people pose tracking. The notable aspects of our approach are real-time ensuring speed (up to 30 fps), flexibility towards various complex motions and environments. Our work is inspired by the success of multiple view approach, especially voxel based techniques. The use of voxel data leads to viewpoint-free estimation, which benefits in that reconstruction of a training model is needless in different multi-camera arrangements. We add simple tracking-based volume segmentation algorithm to retain practical superiority of voxel based approach. Furthermore, our framework successfully obtains multiple body pose estimation in real-time even when people contacts with each other occurs in the scene, which is not addressed in the conventional approaches. We demonstrate the effectiveness of our approach with experiments on indoor cluttered scene sequences.

This paper describes an activity pattern mining method via inhomogeneous Poisson point processes (IPPPs) from time-series of count data generated in behavior detection by pyroelectric sensors. IPPP reflects the idea that typical human activity is rhythmic and periodic. We also focus on the idea that activity patterns are affected by exogenous phenomena, such as the day of the week, and weather condition. Because single IPPP could not tackle this idea, Dirichlet process mixtures (DPM) are leveraged in order to discriminate and discover different activity patterns caused by such factors. The use of DPM leads us to discover the appropriate number of the typical daily patterns automatically. Experimental result using long-term count data shows that our model successfully and efficiently discovers typical daily patterns.

Laser based environment recognition technologies have been developed recently. Especially moving objects detection and classification by laser scanners mounted on a mobility is required for mobile robots and autonomous cars. In this paper, we propose a moving objects detection and classification method based on grid trajectories acquired from sequential laser scan data. Grid trajectories are obtained by voting sequential laser scan points on a grid map, and these trajectories not only work for a correct scan segmentation, but also represent the size and the speed of moving objects. We classify a moving object into either a person, a group of people, a bike, a car based on its grid trajectory. In our experiments, our mobility mounted laser scanners acquired scan data in the university campus, and the experimental results illustrate the effectiveness of the proposed method in outdoor environments.

In this paper, we propose a novel robust action recognition framework with the following capabilities: 1) online encoding motions to multi-label sequence where the output in each frame is a tuple of labels rather than a single label, 2) providing efficient automatic relevant motion selection framework, 3) learning systems so as to be optimal for online multi-label sequence classification. As for multi-label classification, our approach incorporates contextual information about action not only temporal information but hierarchical information of actions. Inference tends to be complex so as to achieve such complex recognition scheme, however, we propose an efficient Viterbi-like decoding algorithm which integrates forward algorithm and loopy message passing algorithm. As for the learning process, the algorithm optimizes the parameters so as to maximize log likelihood of the model. Boosting, ensemble approach of machine learning, is leveraged to provide efficient feature selection framework in the training process. The experimental results show that the proposed method successfully exploits the impact of contextual information then significantly outperforms the traditional approaches in dynamic gait motion classification.

In this paper, we present a novel framework to recover human body pose on multi camera systems. Our framework leverages 3D voxel data, which are reconstructed from multi-camera systems. The use of voxel data leads to viewpoint-free estimation, which benefits in that reconstruction of a training model is needless in different multi-camera arrangements. Other notable aspects of our approach are real-time ensuring speed (up to 30 fps), flexibility towards various complex motions and environments. We treat the pose estimation problem as estimating human pose label from the voxel features and tackle this by example based approach. To ensure the real-time speed and to improve precision of pose estimation, a newly fast and robust near-neighbor search metric is installed prior to the evaluation process, what we call CSI-PSH. We demonstrate the effectiveness of our approach with experiments on both synthetic and real image sequences.

Laser based tracking systems have been developed for mobile robotics and intelligent surveillance areas. Existing systems estimate only human positions. In this paper, we propose a method for human pose estimation represented by human head and waist position using only laser range finders. Two features of human cross-sectional contours are extracted from laser scanner data scanning on the height of waist. This method estimates human pose by using these features in the Bayesian filtering framework. Moreover, we develop a new particle filter framework with two transition models and two resampling steps. In this framework, position estimation and pose estimation are performed by many hypotheses. Our experimental results demonstrate the effectiveness of the method in pose estimation of multiple people by using only several laser scanners.

This paper presents three behavior labeling algorithms based on supervised learning using accumulated pyroelectric sensor data in the living space. We summarize features of each algorithm to use them in combination matched to usage of the livelihood support application. They are (1) labeling algorithms based on time attribution of ldquoon-offrdquo data, (2) one based on Hidden Markov Models, and (3) one based on switching model around a behavioral change-point. We show the behavior labeling results of three algorithms for one month data under the same conditions. Then we point out features on the basis of these results.

In this paper, a novel statistical indoor activity recognition algorithm is introduced. While conditional random fields (CRFs) have prominent properties to this task, no optimal performance is obtained due to the fact that the performance is optimized for offline estimation. Furthermore, no previous researches provide efficient training process to optimize classifiers in on-site recognition perspective. In this paper, we propose a novel sequence estimation model suitable for online activity recognition, what we call Just-in-Time random fields (JRFs). In JRFs, efficient training and feature selection process is provided via boosting. Empirical evaluation using synthetic and real indoor activity records shows that our model drastically outperforms the previous methods in view of the classification performance with respect to the training cost.

This paper describes an algorithm of behavior labeling and anomaly detection for elder people living alone. In order to grasp the personpsilas life pattern, we set some pyroelectric sensors in the house and measure the personpsilas movement data all the time. From those sequential data, we extract two kinds of information, time and duration, and calculate two-dimensional probabilistic density function of them. Using this function, we try to classify behavior labels and detect anomaly. Here, we assume two kinds of anomaly, ldquothe rare behaviorsrdquo and ldquothe changes of life patternrdquo. The algorithm is confirmed to work on real behavior data through the experiment on about 400 days data.

In this paper, we propose a robust recognition and segmentation method for daily actions with a novel multi-task sequence labeling algorithm called multi-task conditional random field (MT-CRF). Multi-Task sequence labeling is a task of assigning input sequence to sequence of multi-labels that consist of one or multiple symbols in single frame. Multi-Task sequence labeling is essential for action recognition, since motions can be often classified into multi-labels, e.g. he is folding arms while sitting. The MT-CRFs: extensions of conditional random fields (CRFs), incorporate jointly interaction between action labels as well as Markov property of actions, to improve the performance of the joint accuracy: the accuracy for whole labels at specific time. The MT-CRFs offer several advantages over the generative dynamic Bayesian networks (DBNs), which are often utilized as multi-task sequence labelers. First, the MT-CRFs allow relaxing the strong assumption of conditional independence of observed motion, which is used in DBNs. Second, the MT-CRFs exploit the power of non-Markovian discriminative classification frameworks instead of generative models in DBNs. With deep insight of the problem Multi-Task sequence labeling, the inference process of the classifier gains more efficiency than the previous Markov random fields that tackle multi-task sequence labeling. The experimental results show that classifiers with MT-CRFs have better performance than cascaded classifiers with a couple of CRFs.

In this paper, we propose wrapped boosting that is extension of boosting algorithm for robust online action recognition. Boosting algorithm is one of ensemble learning algorithm and is also known as a feature selector. In our previous work utilizing boosting, we achieved automatic feature selection and robust model-based action classifiers which had very small calculation cost based on posture information of human body joints. However, which joints we should allocate posture sensors to must be given by humans in advance. Our new learning framework of wrapped boosting provides not only automatic feature selection but also automatic sensor allocation to proper joints of humans for target actions. We evaluated our algorithm targeting gait motion based on motion data fetched by motion capturing system. In consequence, wrapped boosting was able to select proper joints to which limited sensors should be attached, and to construct more robust classifiers compared to constructing classifiers with all joints available. Classifiers constructed only with existing boosting algorithm were subject to over-fitting to training data.

In this paper, a novel approach is proposed to recover human body pose from 3D voxel data. The use of voxel data leads to viewpoint-free estimation, which benefits in that reconstruction of a training model is needless in different multi- camera arrangements. Other notable aspects of our approach are real-time ensuring speed (up to 30[FPS]), flexibility towards various complex motions, and robustness towards voxel noise. The main concept of our approach is based on an example based approach. Human posture candidates are constructed beforehand from a large motion capture database, and the most appropriate posture is estimated per frame by comparing the likelihoods between 3D voxel data and posture candidates. The evaluation is formulated by introducing a histogram- based feature vector that represents the 3D shape context of human body. In addition, a fast near-neighbor search metric is installed prior to the evaluation process, in order to reduce the computational cost and ensure real-time processing. Estimation stability is also improved by a graphical model of motion, which adds a smoothing effect to the motion sequence. We demonstrate the effectiveness of our approach with experiments on both synthetic and real image sequences.

This paper presents a human like segmentation method for daily life actions, such as getting up, sitting down, walking. Unsupervised segmentation methods of many previous researches cannot always assure segmentation result that coincides with human's natural sense. While the proposed method utilizes human's teacher data of segmentation to conduct human like segmentation. We assume that latent dynamics changes at the segmentation points of action, and represent segmentation boundary by switching model of two linear dynamic systems. The problem is that human may segment actions according to wide variety of criteria depending on the attention point or other backgrounds. In this paper, those criteria are acquired by clustering segmentation boundaries extracted from teacher data made by human. Each of the cluster is characterized by body parts it pays attention to. Here, we focus on hierarchical aspect of human body that human body can be treated at various levels of abstraction (e.g. whole body, upper body, left arm), and represent it by tree structure. Experimental result shows that the proposed method can acquire human like segmentation criteria

In this paper, we propose a margin-based query learning algorithm for action recognition to reduce a laborious work on annotating action labels of time-series motion. The annotation is an inevitable task for designers of recognition systems with supervised learning techniques. Query learning is a kind of compensation approach for this, and can also be categorized into interactive learning. Our algorithm is a natural extension of maximum margin learning; a.k.a. support vector machines. Thanks to the theoretical analysis of the optimal condition of the maximum margin learning, the algorithm runs with a single and simple criterion. To prevent poor performance of the classifier learned with very few size of labeled motion data set, the algorithm exploits cluster information of massive unlabeled motion dataset. In contrast to the previous margin-based query learning methods, the algorithm has superiority in terms of stability. The empirical evaluation using real motion and synthetic dataset shows that our algorithm can achieve both drastic reduction of annotation cost and making robust classifiers

In this paper, we propose a fast and robust online action recognition method. The main features of the proposed method are: 1) to select a small number of critical motion features from a very large set of motion feature templates and to release humans from task of designing critical motion features, 2) to require very small calculation cost for recognition compared to conventional methods, 3) to exploit "combinational motion features" which we propose as a new conception so as to construct a robust action recognizer. We evaluated the proposed method to gait action recognition, such as walking and running, by utilizing motion capture data. In the result, the proposed method reduced parameters given by human to action recognizer and lessened human's task. In addition, the proposed method needed very small calculation cost for recognition, and can recognize robustly as much as conventional action recognition method based on support vector machine. Moreover, the introduction of combinational motion features enhanced recognition performance

In this paper, we propose a novel kernel computation algorithm between time-series human motion data for online action recognition. The proposed kernel is based on probabilistic models called switching linear dynamics (SLDs). SLD is one of the powerful tools for tracking, analyzing and classifying human complex time-series motion. The proposed kernel incorporates information about the latent variables in SLDs with simplified designing approach called marginalized kernels. The empirical evaluation using real motion data shows that a classifier using SVM with our proposed kernel has much better performance than the classifier with some conventional kernel techniques. Another experiment using walking around motion shows that a classifier with the proposed kernel can properly segment the start and the end of the target action.

In this paper, we propose a robust online action recognition algorithm with a segmentation scheme that detects start and end points of action occurrences. In other words, the algorithm estimates reliably what kind of actions occurring at present time. The algorithm has following characteristics: 1) The algorithm incorporates human knowledge about relation between action names in order to simplify and toughen the algorithm, thus our algorithm can label robustly multiple action names at the same time. 2) The algorithm uses time-series action probability that represents the likelihood of each action occurrence at every frame time. 3) The classification technique with hidden Markov models (HMMs) enables the algorithm to detect robustly and immediately the segmental points. The experimental results using real motion capture data show that our algorithm not only decreases effectively the latency for detecting the segmental points but also prevents the system from making unnecessary segments due to the error of time-series action probability.

This paper proposes a novel algorithm for extracting informative motion features in daily life action recognition based on support vector machine (SVM). The main advantage of the proposed method is not only to extract remarkable motion features, which fit into human intuition, but also to improve the performance of the recognition system. Concretely speaking, the main properties of the proposed method are 1) optimizing kernel parameters so as to minimize its generalization error, 2) extracting remarkable motion features in response to the sensitivity of the kernel function. Experimental result shows that the proposed algorithm improves the accuracy of the recognition system and enables human to identify informative motion features intuitively.

This paper proposes a human-like action recognition system which can output the result of human action recognition just like the case human does. The system targets actions associated with regular human activity such as walking or lying down, and uses three human recognition characteristics: using specific features of an action to recognize that action; recognition of simultaneous actions; and summarization of recognition results over a short time interval. Experimental results demonstrate the effectiveness of human-like recognition for identifying actions and the superior performance of the proposed system with respect to conventional action recognitions systems. Human-like recognition is expected to ensure smooth communication between humans and robots and enhances the support functionality.