LAVA Challenge 2025 aims to improve the ability of large visual language models to accurately understand complex visual information such as data flow diagrams and Gantt charts contained in Japanese government and business documents. For this challenge, we adopted a two-stage approach consisting of retrieval and reading comprehension. Specifically, in the retrieval step, we select pages relevant to the question from multi-page PDF documents, and in the reading comprehension step, we perform question answering by referring to the top k images selected in the retrieval step. For the retrieval step, we employ ColQwen2, which performs visual information retrieval using the multilingual Qwen2-VL. For the reading comprehension step, we propose a method that performs question answering, including voting, using the multilingual visual language model Qwen2.5VL under different prompts, model sizes, and image qualities. In LAVA Challenge 2025, we clarify the importance of the two-step process of retrieval search and reading comprehension in visual question answering, and verify the effectiveness of a method for determining the multiple results of reading comprehension steps through voting.

This paper described our development dialog system on Kyoto tourist information assistance. Dialog part of our system helped user to make an appropriate query. Information analysis part would be assisted for user to select the retrieved information. Nowadays we can get most information through the Internet. However, we have a trouble to pick up expected information from the huge results with conventional search engines. Especially in mobile terminal, we are confronted with great difficulties for two factors. One is that most of users cannot make an appropriate query because their request is vague with theirselves. The other is that the retrieved information has huge variation and mobile terminal has small area for displaying them. Therefore, we aim to develop technologies for the users to input their requests by familiar way and clarify what they want to know with displaying the retrieved information with suitable method.

As the amount of receded TV content is increasing rapidly, people need active and interactive browsing methods. In this paper, we use both text information from closed captions and visual information from video frames to generate links to enable one to explore not only the original video content but also augmented information from the Web. This solution especially shows its superiority when the video content cannot be well represented only by closed captions. A prototype system was implemented and some experiments were carried out to prove the effectiveness and efficiency

A search engine that can handle TV programs and Web content in an integrated way is proposed. Conventional search engines have been able to handle Web content and/or data stored in a PC desktop as target information. In the future, however, the target information is expected to be stored in various places such as in hard-disk (HD)/DVD recorders, digital cameras, mobile devices, and even in real space as ubiquitous content, and a search engine that can search across such heterogeneous resources will become essential. Therefore, as a first step towards developing such next-generation search engine, a prototype search system for Web and TV programs is developed that performs integrated search of those content, and that allows chain search where related content can be accessed from each search result. The integrated search is achieved by generating integrated indices for Web and TV content based on vector space model and by computing similarity between the query and all the content described by the indices. The chain search of related content is done by computing similarity between the selected result and all other content based on the integrated indices. Also, the zoom-based display of the search results enables to control media transition and level of details of the contents to acquire information efficiently. In this paper, testing of a prototype of the integrated search engine validated the approach taken by the proposed method.

A method is proposed for viewing broadcast content that converts TV programs into Web content and integrates the results with complementary information retrieved using the Internet. Converting the programs into Web pages enables the programs to be skimmed over to get an overview and for particular scenes to be easily explored. Integrating complementary information enables the programs to be viewed efficiently with value-added content. An intuitive, user-friendly browsing interface enables the user to easily changing the level of detail displayed for the integrated information by zooming. Preliminary testing of a prototype system for next generation storage TV, “ WA-TV”, validated the approach taken by the proposed method.

We propose a method of generating views of TV programs based on viewer's perspectives expressed in live chats on the Web. Important scenes in a program and responses by particular viewers can be extracted efficiently by statistically computing and/or recognizing live chat data obtained in sync with the broadcast content. We show that by using the computed results, views can be generated that indicate the momentum of reactions by viewers and scenes of interest to particular viewers whose preferences are similar to those of the viewer, etc. This is a new way of viewing TV content from various perspectives.

This paper proposes the automatic annotation of sports video for content-based retrieval. Conventional methods using position information of objects such as locus, relative positions, their transitions, etc., as indices, have drawbacks that tracking errors of a certain object due to occlusions causes recognition failures, and that representation by position information essentially has a limited number of recognizable events in the retrieval. Our approach incorporates human behavior analysis and specific domain knowledge with conventional methods, to develop an integrated reasoning module for richer expressiveness of events and robust recognition. Based on the proposed method, we implemented a content-based retrieval system which can identify several actions on real tennis video. We select court and net lines, players' positions, ball positions, and players' actions, as indices. Court and net lines are extracted using a court model and Hough transforms. Players and ball positions are tracked by adaptive template matching and particular predictions against sudden changes of motion direction. Players' actions are analyzed by 2D appearance-based matching using the transition of players' silhouettes and a hidden Markov model. The results using two sets of tennis video is presented, demonstrating the performance and the validity of our approach.

A visual tracking system is presented in which a combination of techniques is used to obtain motion features of objects from a video sequence. Further processing of the motion features gives the spatio-temporal trajectories of the objects, that can be used as cues for annotation. The system solves problems found in tracking objects in unconstrained environments, such as in sports games, where there are multiple objects in motion, the camera performs pan, tilt and zoom movements, and there are objects other than the players in the background. Coarse segmentation is performed with multi-class statistical color models, constructed from samples of the representative colors of each team. Motion vectors are computed to find region correspondence between consecutive frames. Background elements are eliminated by using camera motion parameters, and other false matches are detected by analyzing motion pattern consistency. Finally, objects are registered by placing windows centered in each tracked region. An experimental realization was used to test the system for tracking players in a soccer game, but it could had also been used for generating annotation cues of videos from other sports as well.

The paper presents an approach for region based video mosaicing, treating moving objects separately from the background, and with improved ghost-like noise elimination. The mosaic images show the moving objects superimposed over a stationary background. Conventional technologies can reduce the ghost-like noise that occurs from moving objects by using temporal median filtering, but its efficiency depends on the ratio between the speeds of the camera and the moving object. Our technology eliminates these noises more efficiently by using segmented images of a spatio-temporal video sequence. Segmentation is performed using a novel technique that uses different configurations of quad-trees for the initial separation in the split-and-merge process. The segmented images are also used to display tracked moving objects on the panoramic image.

In this paper we present a method for representing and retrieving video sequences based on the domain-specific behavior of objects present in the scenes. The representation includes three parts: (a) Action description, representing the action performed by a single object. (b) Interaction description, which describes interactions between multiple objects and is mapped directly to the event semantics in the content domain. (c) Event Structure, which provides a set of spatial and temporal relationship functions, along with a syntax to define the necessary conditions that a particular interaction should meet. Retrieval is performed by processing Event Structures, interpreting object relationships and selecting the relevant combinations of Action descriptions which match the conditions defined in the Event Structures. We describe an implementation of this system for retrieving scenes of soccer plays from among several soccer video sequences.