An effective interaction in augmented reality (AR) requires utilization of different modalities. In this study, we investigated orienting the user in bimodal AR. Using auditory perception to support visual perception provides a useful approach for orienting the user to directions that are outside of the visual field-of-view (FOV). In particular, this is important in path-finding, where points-of-interest (POIs) can be all around the user. However, the ability to perceive the audio POIs is affected by the ventriloquism effect (VE), which means that audio POIs are captured by visual POIs. We measured the spatial limits for the VE in AR using a video see-through head-worn display. The results showed that the amount of the VE in AR was approx. 5°–15° higher than in a real environment. In AR, spatial disparity between an audio and visual POI should be at least 30° of azimuth angle, in order to perceive the audio and visual POIs as separate. The limit was affected by azimuth angle of visual POI and magnitude of head rotations. These results provide guidelines for designing bimodal AR systems.

Television (TV) programs are increasingly augmented with second-screen applications that can provide complementary information and social media interaction related to the program. However, there is a lack of understanding about the optimal design of such applications. In this paper, we study the effects of complementary information and tweets on the media experience indexed by a comprehensive self-report questionnaire. We built a custom second-screen tablet application and conducted an experiment to investigate the effects of complementary information and tweets on the media experience of TV programs. Apart from reducing concentration and immersion, neither the presence of complementary information nor the presence of tweets affected the media experience of the TV programs, whereas the media experience of second-screen content depended on which program it accompanied. Our results demonstrate that the media experience of complementary information and tweets presented on second screens differs between TV programs.