We developed a system for visualizing stone trajectories in curling games for live broadcasts. Robustly tracking a moving stone from curling video sequences is difficult because the stone is frequently hidden by the brushes held by the players and the players' bodies during their sweeping actions. Although a number of methods for visual object tracking have been proposed, real-time tracking under heavy occlusion is still a challenging task. We thus propose an online machine learning method for tracking a curling stone to deal with changes in its appearance. The method creates a candidate-object image, which eliminates background noises, and is used as input to the kernelized correlation filter (KCF) tracker. Coordinate transformation is also applied to the system to improve its operability. Experimental results showed that our stone tracker is more accurate and faster than other conventional tracking methods. The developed system was used at All Japan Curling Championships 2017 to display stone trajectories during live broadcasts.

A handy system is presented for creating in real time a natural television image composed of a real scene and a computer graphics object. The light positions and colors (RGB values) in the real scene are estimated and then applied to the object by using simple equipment. Objective and subjective experiments demonstrated the effectiveness of this system. An enhanced algorithm is also presented that improves the accuracy of light estimation.

We have developed a robust method for estimating camera parameters for live TV production based on infrared markers whose feature-point extraction is simple and two-dimensional color histogram matching that considers the effects of specular reflection due to lighting. We first explain the principle of the proposed technique. We then present the results of a basic experiment for evaluating the accuracy of marker identification and the results of an image-compositing experiment using estimated camera parameters. We show that the proposed technique is effective based on those results.

When making a TV program in a studio, care must be taken that the camera does not shoot beyond the boundary of the studio set. In addition, limitations in cost and space for the set must be taken into account. In a virtual studio, on the other hand, we can solve this cost and space problem, but in turn, actors are requested to perform in front of a blue background screen, which is not always an easy task for them. To solve these problems associated with real studios and virtual studios, we have developed a new type of virtual studio called Real Space-based Virtual Studio in which a real space image and virtual space image are combined naturally with no boundary seam. There are two major advantages in using this new virtual-real hybrid system. One is that the actors can concentrate on their role in the real studio sets, and the other is that camera work can be done without worrying about off-screen areas of the set. In the present study, we constructed an omnidirectional image with ultra high-definition features and combined it as a virtual studio image with a real studio image. We have developed an integration system and from the experimentation we have shown that the omnidirectional images and the real studio images can combine smoothly and naturally.

We are studying an image-based virtual studio. This type of virtual studio makes it possible to reconstruct realistic synthesized video by extracting shape, texture, and other necessary information (called video components) from each object in the subject video for use on the set. When using video components made up of shape and texture information to reconstruct natural-looking video under any lighting conditions, lighting effects like highlights and shading that are present at the time of measurement must be removed from texture information. For this purpose, the authors have proposed a technique that uses polarized light on the lighting side and a polarized filter on the camera side, and have confirmed its effectiveness by experiments.