Report generation (RG) faces challenges in understanding complex medical images and establishing cross-modal semantic alignment in radiology image-report pairs. Previous methods often overlook fine-grained cross-modal interaction, leading to insufficient understanding of detailed information. Recently, various large multimodal models have been proposed for image-text tasks. However, such models still underperform on rare domain tasks like understanding complex medical images. To address these limitations, we develop a new framework of Knowledge-guided Implicit vision-language Alignment for radiology report generation, named KIA. To better understand medical reports and images and build alignment between them, multi-task implicit alignment is creatively introduced, forming comprehensive understanding of medical images and reports. Additionally, to further meet medical refinement requirements, we design novel masking strategies guided by medical knowledge to enhance pathological observation and anatomical landm

Multimodal large language models (MLLMs) have demonstrated strong language understanding and generation capabilities, excelling in visual tasks like referring and grounding. However, due to task type limitations and dataset scarcity, existing MLLMs only ground objects present in images and cannot reject non-existent objects effectively, resulting in unreliable predictions. In this paper, we introduce ROD-MLLM, a novel MLLM for Reliable Object Detection using free-form language. We propose a query-based localization mechanism to extract low-level object features. By aligning global and object-level visual information with text space, we leverage the large language model (LLM) for high-level comprehension and final localization decisions, overcoming the language understanding limitations of normal detectors. To enhance language-based object detection, we design an automated data annotation pipeline and construct the dataset ROD. This pipeline uses the referring capabilities of existing MLLMs and chain-of-thought techniques to generate diverse expressions corresponding to zero or multiple objects, addressing the shortage of training data. Experiments across various tasks, including referring, grounding, and language-based object detection, show that ROD-MLLM achieves state-of-the-art performance among MLLMs. Notably, in language-based object detection, our model achieves +13.7 AP improvement on D3benchmark over existing MLLMs and surpasses most specialized detection models, especially in scenarios requiring complex language understanding.

Sparse mobile crowdsensing (MCS) is a cost-effective data collection paradigm that aims to recruit users to collect data from a part of sensing subareas and infer the rest. In a more realistic scenario, users participate in real-time and collect data along the way. For missing data inference, the significance of data collected from different subareas often varies over time. However, since users’ trajectories are uncertain, recruiting users who can cover important spatio-temporal subareas presents a challenge. Additionally, how to segment the budget wisely during recruitment is another challenge. To tackle these challenges, we propose a dual reinforcement learning (RL)-based online user recruitment strategy with adaptive budget segmentation, called DualRL-U, which consists of two alternating decision steps, i.e., the user recruitment decision and the budget retention decision. Specifically, for the user recruitment decision, we use RL to connect the user with data inference accuracy to estimate their contributions. For the budget retention decision, we use RL to connect the budget with the number of times the user can sense to evaluate the cost effectiveness. In this way, a dual RL model is constructed to achieve effective recruitment by alternately executing user recruitment decisions and budget retention decisions. Extensive experiments on real-world sensing data sets show the effectiveness of DualRL-U.