We developed a robot system which can play one-on-one volleyball games with a human player. It can not only hit a flying ball back to the player correctly but also have various autonomous functionalities to interact with humans necessary for playing games, i.e., it can pick up a ball of a requested color by recognizing voice instructions, can locate the players by recognizing their faces, and can shake hands. This robot performed the entire game sequence over 100 times at a company-sponsored exhibition, where the ball rally continued over 25 times with an arbitrary player. In this paper, we mainly describe the real-time visual feedback system of the robot which is essential for playing ball games.