In this paper, we present a review of the competition track organized by us at IPIN 2017. Our competition track was named as “PDR challenge in warehouse picking”. Our pedestrian dead-reckoning (PDR) challenge focused on testing the PDR methods under a realistic scenario, which is tracking employees during the picking operation in a warehouse. Firstly, we introduce an overview of the PDR challenge including detailed information of all evaluation metrics. Secondly, we summarize the results and findings from the competition and list the challenges that should be tackled in the next competition. Thirdly, we introduce the design concept of an xDR challenge by reflecting the findings and feedback of design of the competition. The target of the PDR challenge was only the employees who move by foot during working in the warehouse. The main update in the xDR challenge is dealing with forklifts driven by employees as well as walking employees. We term dead-reckoning of a vehicle as vehicle dead-reckoning (VDR). The term xDR is derived from PDR plus VDR. Another update is providing reference data for the reconstructing situation of picking operation in the warehouse by using an IMU-based motion capture system.

The present paper addresses the problem of analyzing the work of forklift trucks. To solve the problem, a forklift truck working model based on a state transition model is constructed using human activity sensing data, where the key point is using forklift truck operating data to determine human intention. The novel contribution of the model is its use of data that express the intention of the forklift truck operator to categorize the work of the forklift truck. By using a graph model, the proposed method can be extended to analyzing the human characteristics of forklift truck work. With this technique, some features of the graph model can be used to distinguish the skill level of the forklift truck operator.

We have been developing a mixed reality system to support road maintenance using overlaid visual aids. Such a system requires a positioning method that can provide sub-meter accuracy and function even if the appearance of the road surface changes significantly caused by many factors such as construction phase, time and weather. Therefore, we are developing a real-time worker positioning method that can be applied to these situation by integrating laser range finder (LRF) and pedestrian dead-reckoning (PDR) data. In the field, multiple workers move around the workspace. Therefore, it is necessary to determine corresponding pairs of PDR-based and LRF-based trajectories by identifying similar trajectories. In this study, we propose a method to calculate the similarity between trajectories and a procedure to integrate corresponding pairs of trajectories to acquire the position and movement direction of a worker.

This paper presents a study of a method for estimating the position and orientation of a photo-shoot in indoor environments for augmented reality applications. Our proposed localization method is based on registered frame data of virtualized reality models, which are photos with known photo-shoot positions and orientations, and depth data. Because registered frame data are secondary product of modeling process, additional works are not necessary to create registered frame data especially for the localization. In the method, a photo taken by a mobile camera is compared to registered frame data for the localization. Since registered frame data are linked with photo-shoot position, orientation, and depth data, 3D coordinates of each pixel on the photo of registered frame data is available. We conducted experiments with employing five techniques of the estimation for comparative evaluations.

We have developed a GUI tool and a mobile MR system for reliving experiences at museums. The GUI tool was developed to provide services based on content made from activity logs and to enable effective analysis of visitors' behaviors using the logs. In addition, by running the GUI tool on a mobile MR system, users can browse visitors' activities with a sensation of realism on the spot. This paper describes the GUI tool and a pilot user study that was conducted to evaluate it.

We propose a novel fiducial-less 3-D object tracking method. Our method consists of three components: 1) bottom-up approach (BUA), 2) top-down approach (TDA), and 3) automatic database addition (ADA). An experimental result shows an accuracy and robustness of our method.

We have developed a method for recognizing and tracking known three-dimensional objects in scene images captured from a user's viewpoint. The method uses multiple color histogram matching and condensation tracking. We implemented a prototype 3-D online manual system based on our method.

This paper describes a method of estimating the position and orientation of a camera for constructing a mixed reality (MR) system. In an MR system 3D virtual objects should be merged into a 3D real environment at a right position in real time. To acquire the user's viewing position and orientation is the main technical problem of constructing an MR system. The user's viewpoint can be determined by estimating the position and orientation of a camera using images taken at the viewpoint. Our method estimates the camera pose using screen coordinates of captured color fiducial markers whose 3D positions are known. The method consists of three algorithms for perspective n-points problems and rises each algorithm selectively. The method also estimates the screen coordinates of untracked markers that are occluded or are out of the view. It has been found that an experimental MR system that is based on the proposed method can seamlessly merge 3D virtual objects into a 3D real environment at right position in real-time and allows users to look around an area in which numbers are placed.

In an augmented reality system, it is required to obtain the position and orientation of the user's viewpoint in order to display the composed image while maintaining a correct registration between the real and virtual worlds. All the procedures must be done in real time. This paper proposes a method for augmented reality with a stereo vision sensor and a video see-through head-mounted display (HMD). It can synchronize the display timing between the virtual and real worlds so that the alignment error is reduced. The method calculates camera parameters from three markers in image sequences captured by a pair of stereo cameras mounted on the HMD. In addition, it estimates the real-world depth from a pair of stereo images in order to generate a composed image maintaining consistent occlusions between real and virtual objects. The depth estimation region is efficiently limited by calculating the position of the virtual object by using the camera parameters. Finally, we have developed a video see-through augmented reality system which mainly consists of a pair of stereo cameras mounted on the HMD and a standard graphics workstation. The feasibility of the system has been successfully demonstrated with experiments.

Describes a prototype of an augmented reality system using vision-based registration. In order to build an augmented reality system with video see-through image composition, camera parameters for generating virtual objects must be obtained at video-rate. The system estimates camera parameters from four known markers in an image sequence captured by a small CCD camera mounted on a HMD (head mounted display). Virtual objects are overlaid upon a captured image sequence in real-time.