Musculoskeletal diseases such as low back pain may cause physical disabilities and make it difficult to continue working. From this point of view, logging of workers' conditions in the working environment is an important issue. So far, limited information acquisition such as smart watches has been realized, but comprehensive information has not been realized with a single device. Therefore, in this study, we estimated the wearer's state using a device called THINKLET, which can acquire image information and inertial information at the same time. By acquiring the pose of the hand from the image information, we were able to estimate the work. More detailed information can be obtained by calculating the position at the same time. By integrating image information with inertial information, it becomes possible to measure the position and posture of the wearer in the working environment. By complementing each other's sensor information, it is possible to obtain robust and accurate self-location. In this research, we developed these basic technologies and evaluated their performance.

In this paper, we present a review of the competition track organized by us at IPIN 2017. Our competition track was named as “PDR challenge in warehouse picking”. Our pedestrian dead-reckoning (PDR) challenge focused on testing the PDR methods under a realistic scenario, which is tracking employees during the picking operation in a warehouse. Firstly, we introduce an overview of the PDR challenge including detailed information of all evaluation metrics. Secondly, we summarize the results and findings from the competition and list the challenges that should be tackled in the next competition. Thirdly, we introduce the design concept of an xDR challenge by reflecting the findings and feedback of design of the competition. The target of the PDR challenge was only the employees who move by foot during working in the warehouse. The main update in the xDR challenge is dealing with forklifts driven by employees as well as walking employees. We term dead-reckoning of a vehicle as vehicle dead-reckoning (VDR). The term xDR is derived from PDR plus VDR. Another update is providing reference data for the reconstructing situation of picking operation in the warehouse by using an IMU-based motion capture system.

This paper presents a novel method for detecting the floor level of a smartphone user in an indoor environment based on the atmospheric pressure measured with sensors in the user's smartphone. Atmospheric pressure is one of the most informative factors when estimating altitude, because its relative variation has a strong correlation with changes in altitude. However, it is difficult to directly estimate the altitude based on an observation of the absolute pressure via smartphones, owing to individual differences in the sensors and changes in climate. In this paper, we propose a robust method for estimating the floor level by decomposing the observed pressure into three components. The first component is the change in altitude, and this component is the primary measure. The second is the global variation, such as changes in climate. The third concerns device-dependent variation. We utilize localization infrastructure, such as beacons or Wi-Fi access points, sparsely located in the environment, because such localization mechanisms can provide floor-level information. Exploiting this mechanism, time-series references to the pressure on each floor level can be generated collectively by aggregating readings from the pressure sensors of multiple users. Using these as a reference, each output of the pressure can be decomposed into device-specific offsets, environmental trends, and the altitude-dependent component. By extracting the altitude-dependent component, the floor level of the user can be estimated robustly without relying on fixed observations of atmospheric pressure.

This paper presents a study of a method for estimating the position and orientation of a photo-shoot in indoor environments for augmented reality applications. Our proposed localization method is based on registered frame data of virtualized reality models, which are photos with known photo-shoot positions and orientations, and depth data. Because registered frame data are secondary product of modeling process, additional works are not necessary to create registered frame data especially for the localization. In the method, a photo taken by a mobile camera is compared to registered frame data for the localization. Since registered frame data are linked with photo-shoot position, orientation, and depth data, 3D coordinates of each pixel on the photo of registered frame data is available. We conducted experiments with employing five techniques of the estimation for comparative evaluations.

We have developed a GUI tool and a mobile MR system for reliving experiences at museums. The GUI tool was developed to provide services based on content made from activity logs and to enable effective analysis of visitors' behaviors using the logs. In addition, by running the GUI tool on a mobile MR system, users can browse visitors' activities with a sensation of realism on the spot. This paper describes the GUI tool and a pilot user study that was conducted to evaluate it.

In this paper, we propose a method of personal positioning for a wearable augmented reality (AR) system that allows a user to freely move around indoors and outdoors. The user is equipped with self-contained sensors, a wearable camera, an inertial head tracker and display. The method is based on sensor fusion of estimates for relative displacement caused by human walking locomotion and estimates for absolute position and orientation within a Kalman filtering framework. The former is based on intensive analysis of human walking behavior using self-contained sensors. The latter is based on image matching of video frames from a wearable camera with an image database that was prepared beforehand.

In this paper, we present a wearable augmented reality (AR) system with personal positioning based on walking locomotion analysis that allows a user to freely mover around indoors and outdoors. The user is equipped with self-contained sensors, a wearable camera, an inertial head tracker and display. The system is based on the sensor fusion of estimates for relative displacement caused by human walking locomotion and estimates for absolute position and orientation within a Kalman filtering framework. The former is based on intensive analysis of human walking behavior using self-contained sensors. The latter is based on image matching of video frames from a wearable camera with an image database that was prepared beforehand.

This paper describes a fast and robust approach to recovering structure and motion from video frames. It first describes a robust recursive factorization method for affine projection. Using the least median of squares (LMedS) criterion, the method estimates the dominant 3D affine motion and discards feature points regarded as outliers. The computational cost of the overall procedure is reduced by combining this robust statistics-based method with a recursive factorization method that can at each frame provide the updated 3D structure of an object at a fixed computational cost by using the principal component analysis. Experiments with synthetic data and with real image sequences demonstrate that the method can be used to estimate the dominant structure and the motion robustly and in real-time, on an off-the-shelf PC. Finally this paper describes preliminary online experiments on selecting feature points, which have the dominant 3D motion, from live video frames by using a PC-cluster system.

The paper describes a fast and robust image registration method that can be used to create a panoramic image/video from video sequences. To estimate alignment parameters for image registration, the method computes pseudo motion vectors that are rough estimates of optical flows at each selected pixel. Using the proposed method, we implemented a software system that can, with a low-cost PC, create and display panoramic images/videos in real time.