Medical Visual Question Answering (Med-Vqa) is a challenging task that requires a deep understanding of both medical images and textual questions. Although recent works leveraging Medical Vision-Language Pre-Training (Med-Vlp) have shown strong performance on the Med-Vqa task, there is still no unified solution for modality alignment, and the issue of hard negatives remains underexplored. Additionally, commonly used knowledge fusion techniques for Med-Vqa may introduce irrelevant information. In this work, we propose a framework to address these challenges through three key contributions: (1) a unified solution for heterogeneous modality alignments across multiple levels, modalities, views, and stages, leveraging methods like contrastive learning and optimal transport theory; (2) a hard negative mining method that employs soft labels for multi-modality alignments and enforces the hard negative pair discrimination; and (3) a Gated Cross-Attention Module for Med-Vqa that integrates the answer vocabulary as prior knowledge and selects relevant information from it. Our framework outperforms the previous state-of-the-art on widely used Med-Vqa datasets like RAD-VQA, SLAKE, PathVQA and VQA-2019. The code is available at https://github.com/AlexCo1d/AMiF

Ultrasound imaging is the most commonly used method for screening thyroid nodules due to its low cost and non-invasive nature. Thyroid nodule lesions have variable shapes, rich aspect ratios, unclear boundaries, calcified nodule-induced acoustic shadows, and noise interference, causing challenges in accurate segmentation. Recent methods ignore various scale features and details in different resolutions of images, leading to redundant or missing feature information and then affecting the segmentation performance. In this paper, we introduce a hybrid and full attention UNet model for ultrasound thyroid nodule segmentation. Self, spatial and channel attention are combined in a U-Net-like structure to extract global and local features simultaneously. A novel full attention multi-scale fusion stage is designed to enhance boundary features while suppressing noise features. At the same time, the model dynamically adjusts the number of skip connections corresponding to images of different resolutions to better utilize multi-scale features and detailed information. We evaluate our model on DDTI, TN3K and Stanford Cine-Clip datasets, including internal validation and cross-dataset testing. The results show that our proposed model for internal validation in the DDTI dataset increases the Dice score and mean intersection over union by 2.36 % and 1.04 % compared to the state-of-the-art model. In the TN3K dataset, they increase by 1.66 % and 3.05 %.