Table Understanding (TU) has achieved promising advancements, but it faces the challenges of the scarcity of manually labeled tables and the presence of complex table structures. To address these challenges, we propose HeGTa, a heterogeneous graph (HG)-enhanced large language model (LLM) designed for few-shot TU tasks. This framework aligns structural table semantics with the LLM's parametric knowledge through soft prompts and instruction tuning. It also addresses complex tables with a multi-task pre-training scheme, incorporating three novel multi-granularity self-supervised HG pre-text tasks. We empirically demonstrate the effectiveness of HeGTa, showing that it outperforms the SOTA for few-shot complex TU on several benchmarks.

Dehai Min, Zhiyang Xu, Guilin Qi, Lifu Huang, Chenyu You. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Text-video based multimodal event extraction refers to identifying event information from the given text-video pairs.Existing methods predominantly utilize video appearance features (VAF) and text sequence features (TSF) as input information.Some of them employ contrastive learning to align VAF with the event types extracted from TSF.However, they disregard the motion representations in videos and the optimization of contrastive objective could be misguided by the background noise from RGB frames.We observe that the same event triggers correspond to similar motion trajectories, which are hardly affected by the background noise.motivated by this, we propose a Three Stream Multimodal Event Extraction framework (TSEE) that simultaneously utilizes the features of text sequence and video appearance, as well as the motion representations to enhance the event extraction capacity.Firstly, we extract the optical flow features (OFF) as motion representations from videos to incorporate with VAF and TSF.Then we introduce a Multi-level Event Contrastive Learning module to align the embedding space between OFF and event triggers, as well as between event triggers and types.Finally, a Dual Querying Text module is proposed to enhance the interaction between modalities.Experimental results show that TSEE outperforms the state-of-theart methods, which demonstrates its superiority.