Hybrid-modal table understanding (HMTU), which targets leveraging multi-modal table evidence for multi-hop reasoning, has garnered widespread attention. Existing models primarily focus on effectively integrating multi-modal table evidence to enhance the table understanding capabilities of multi-modal large language models (MLLMs). However, these models ignore the fact that different types of table understanding questions lean toward different modalities of table evidence. Consequently, these models suffer from low utilization efficiency and poor interpretability. To address these issues, in this paper, we propose a modality preference alignment model, called ESTJ, which Enhances Structured Tendency Judgment in HMTU. Specifically, ESTJ first samples modality preference data from the responses generated by MLLMs. Then, it alleviates modality preference imbalance by adhering to the principle of least modality priority. Finally, ESTJ performs direct preference optimization (DPO) training based on structured tendency judgment to align modality preference effectively. Experimental results on TableQA and TableFV tasks demonstrate that our proposed model outperforms state-of-the-art baselines. Additionally, these results present fascinating phenomena and unveil profound insights into modality preference for table understanding.

Recently, large language models (LLMs) have made remarkable progress in table understanding, yet they remain vulnerable to the structural noise (SN) and the textual noise (TN). Existing methods usually employ biased denoising strategies such as structural matching and textual filtering, or overzealous denoising strategies such as introducing supplementary tasks like text-to-SQL and table-to-text to reduce these two types of noise. However, these methods either neglect one type of noise or introduce substantial external noise. Therefore, how to simultaneously mitigate the structural and textual noise without introducing extra noise and improve the performance of LLMs in table understanding is still an unresolved issue. In this paper, we rethink the bottlenecks in table understanding from the perspective of noise reduction and propose a novel dual-denoiser-reasoner model, called TabDDR, for balanced and effective denoising. Specially, our model consists of a structural-and-textual denoiser and a task-adaptive reasoner. The former removes two types of noise via triplet alignment and planning extraction to seek an interpretable balance between breaking structural barriers and preserving structural characteristics, eliminating textual noise and retaining maximal information; the latter ensures a simple but effective reasoning process which can adapt to various downstream tasks. To highlight the presence and impact of the structural and textual noise, we construct the WTQ-SN and WTQ-TN datasets based on the WikiTableQuestion (WTQ) dataset. Extensive experiments on these self-constructed datasets and two other public datasets demonstrate that our proposed method performs better than state-of-the-art baselines.