Video-guided Machine Translation (VMT) seeks to enhance translation quality by incorporating contextual information derived from paired short video clips.However, many VMT samples are text-sufficient; even when visual information is needed, only minimal cues are required.Aiming to tackle these issues, we propose a novel framework DART (Disambiguation-Aware Reasoning for Videoguided Machine Translation).Reinforcement learning is used to incorporate multimodal large language models' multimodal reasoning into VMT.The model dynamically switches between text-only processing and multimodal integration, contingent on the necessity of visual disambiguation.Furthermore, we present TVRF (Translation-oriented Video Relevance Filtering), a systematic pipeline for constructing training data based on multimodal relevance to translation.This pipeline filters samples where video information is translationrelevant, mitigating training collapse caused by video-irrelevant data in conventional VMT.Experimental results show that our approach improves multimodal information utilization in VMT, yielding gains in both translation quality and computational efficiency.* Equal corresponding authors.Reasoning: Okay, I need to translate the input sentence ... So the answer is "哦！".(809 tokens in total) DA T Existing LMRMs … "It's hitting his yacht."Translation: 哦！(Oh!) Reasoning: Okay, so I need to translate the sentence ..

Current video-guided machine translation (VMT) approaches primarily use coarse-grained visual information, resulting in information redundancy, high computational overhead, and neglect of audio content. Our research demonstrates the significance of fine-grained visual and audio information in VMT from both data and methodological perspectives. From the data perspective, we have developed a large-scale dataset TriFine, the first vision-audio-subtitle tri-modal VMT dataset with annotated multimodal fine-grained tags. Each entry in this dataset not only includes the triples found in traditional VMT datasets but also encompasses seven fine-grained annotation tags derived from visual and audio modalities. From the methodological perspective, we propose a Fine-grained Information-enhanced Approach for Translation (FIAT). Experimental results have shown that, in comparison to traditional coarse-grained methods and text-only models, our fine-grained approach achieves superior performance with lower computational overhead. These findings underscore the pivotal role of fine-grained annotated information in advancing the field of VMT.

Video-guided Machine Translation (VMT) aims to improve translation quality by integrating contextual information from paired short video clips.Mainstream VMT approaches typically incorporate multimodal information by uniformly sampling frames from the input videos.However, this paradigm frequently incurs significant computational overhead and introduces redundant multimodal content, which degrades both efficiency and translation quality.To tackle these challenges, we propose SHIFT (Selected Helpful Informative Frame for Translation).It is a lightweight, plug-andplay framework designed for VMT with Multimodal Large Language Models (MLLMs).SHIFT adaptively selects a single informative key frame when visual context is necessary; otherwise, it relies solely on textual input.This process is guided by a dedicated clustering module and a selector module.Experimental results demonstrate that SHIFT enhances the performance of MLLMs on the VMT task while simultaneously reducing computational cost, without sacrificing generalization ability.