We propose an automatic pan control system for broadcasting ball games by tracking face direction of audience. Presuming that the audience's face is directed to a notable play, we can shoot a broadcasting video by controlling the camera toward the audience's face direction. In our method, a court sensor which detects rough region where players exist is used in addition to the face sensor in order to obtain higher accuracy. Based on these sensors, the broadcasting video is generated by cylindrical mosaicing off-line. We conducted objective and subjective evaluation experiments and the result shows that our approach is better than previously proposed methods in respect of stable tracking and easy watching. Conclusively our method is as effective as the optimum camerawork.