The integral three-dimensional (3D) display is an ideal visual 3D user interface. It is a display method that fulfills many of the physiological factors of human vision. However, in integral 3D displays for mobile applications that use direct-view flat panels to display elemental images, color moiré is a problem that occurs because of the sampling of subpixels by elemental lenses and the insufficient resolution and depth reproduction of the reconstructed 3D image. In the conventional moiré reduction method, the degree of defocus of elemental lenses has to be set to a large value, which is one of the factors that reduces the performance in terms of depth reproduction. In contrast, only one-step optics can be installed and the installation positions are limited in the conventional wobbling method. This is because, in the method, which uses a birefringent optical element, two-step optics are thicker than the focal length of the lens array. For this reason, it was difficult to achieve ideal moiré reduction and depth reproduction performance improvements. To solve these problems, we propose a method that utilizes multiple optical wobbling spatiotemporal multiplexing using polarization diffractive elements and liquid-crystal polarization controllers. Using the proposed method, the wobbling optics can be designed to be thin, allowing two-step optics to be installed between the display panel and lens array. When the moiré modulation degree without wobbling is normalized as 100%, it decreases to 25% with wobbling. The proposed method not only achieves effective color moiré reduction without deteriorating the 3D image quality, but also can double the resolution of the elemental images to improve the depth reproduction.

We aim to realize new image expressions—3D position analysis and computer graphics (CG) synthesis—by using multiple pan-tilt cameras for live broadcasts. For this purpose, a camera calibration technique is needed that is accurate over a wide space such as a stadium and has high operability. We thus propose a method to accurately calibrate multiple pan-tilt cameras that can be used in live broadcasts. This method enables cameras to be accurately calibrated over a wide space by using multiple calibration patterns and a system to be efficient prepared at a filming location. After the preparation, accurate camera parameters can be obtained in real time using the pan-tilt angles acquired from angle sensors in the camera platform. We showed the effectiveness of the proposed method through CG simulation and experiments using a real system. In addition, we developed a 3D ball tracking system that uses our calibration method.

This paper proposes a method of estimating depth from two kinds of stereo images: color stereo images and infrared stereo images. An infrared dot pattern is projected on a scene by a projector so that infrared cameras can capture the scene textured by the dots and the depth can be estimated even where the surface is not textured. The cost volumes are calculated for the infrared and color stereo images for each frame and are extended in the time direction to define a spatiotemporal cost volume (st-cost volume). We also extend the cost volume filter in the time direction by modifying the cross-based local multipoint filter (CLMF) and applying it to the st-cost volumes in order to restrain flicker on the time-varying depth maps. To get a reliable cost volume, the infrared and color st-cost volumes are integrated into a single cost volume by selecting the cost of either the infrared or the color st-cost volumes according to the size of the adaptive kernel used for the CLMF. Then, a graphcut is executed on the cost volume in order to estimate the disparity robustly even when the baselines of the stereo cameras are set wide enough to ensure spatially high resolution in the depth direction and the shapes of blocks are deformed by the affine transformation. A 2D graphcut is executed on each scan line to reduce the processing time and memory consumption. We experimented with the proposed method using infrared color stereo data sets of scenes in the real world and evaluated its effectiveness by comparing it with other recent stereo matching methods and depth cameras.

We have developed a real-time ball tracking system that can be used for volleyball games. Although a number of methods for visual object tracking have been proposed, tracking a fast-moving ball is still a challenging task because of the motion blur and the occlusion. We thus use a complementary tracking scheme in which tracking processes for multiple cameras help each other sharing the 3D position of the ball. The ball on each camera is accurately tracked by predicting its position at the next frame. The 3D ball positions measured by the system can be used for drawing the trajectory CG of a ball and for calculating statistical data related to ball movement. Evaluation results obtained using actual volleyball video sequences showed that the system would be effective for visualizing ball trajectories in live volleyball broadcasts.

This paper presents a method to estimate a depth map using an infrared projector and a pair of infrared color cameras that can capture infrared and color images simultaneously. The infrared projector projects a dot pattern so that the cameras capture infrared images of a scene textured by the dots with which depths to surfaces in the scene can be estimated regardless of whether they have visible textures. Cost volumes are calculated for each of the infrared and color stereo images and are processed with a cost volume filter that smoothes each cost map by a cross-based local multipoint filter. The filtered infrared and color cost volumes are then integrated into a single cost volume by selecting either the infrared or color cost for each pixel according to the size of the adaptive kernel used for the cross-based local multipoint filter. This improves the accuracy where the adaptive kernel is small. We propose a method that can find optimal local curved surfaces of adaptive kernels from the cost volume by using three dynamic programmings. We experimented this depth estimation method on real-world datasets that the infrared color stereo cameras captured. We also used it for color stereo matching and showed that it works with normal color stereo cameras as well.