The scarcity data of medical field brings the collaborative training in medical vision-language pre-training (VLP) cross different clients. Therefore, the collaborative training in medical VLP faces two challenges: First, the medical data requires privacy, thus can not directly shared across different clients. Second, medical data distribution across institutes is typically heterogeneous, hindering local model alignment and representation capabilities. To simultaneously overcome these two challenges, we propose the framework called personalized model selector with fused multimodal information (PMS-FM). The contribution of PMS-FM is two-fold: 1) PMS-FM uses embeddings to represent information in different formats, allowing for the fusion of multimodal data. 2) PMS-FM adapts to personalized data distributions by training multiple models. A model selector then identifies and selects the best-performing model for each individual client. Extensive experiments with multiple real-world medical datasets demonstrate the superb performance of PMS-FM over existing federated learning methods on different zero-shot classification tasks.

Despite their outstanding performance in numerous applications, large language models (LLMs) remain prone to hallucinations, generating content inconsistent with their pretraining corpora. Currently, almost all contrastive decoding approaches alleviate hallucinations by introducing a model susceptible to hallucinations and appropriately widening the contrastive logits gap between hallucinatory tokens and target tokens. However, although existing contrastive decoding methods mitigate hallucinations, they lack enough confidence in the factual accuracy of the generated content. In this work, we propose Multi-Model Contrastive Decoding (MCD), which integrates a pretrained language model with an evil model and a truthful model for contrastive decoding. Intuitively, a token is assigned a high probability only when deemed potentially hallucinatory by the evil model while being considered factual by the truthful model. This decoding strategy significantly enhances the model’s confidence in its generated responses and reduces potential hallucinations. Furthermore, we introduce a dynamic hallucination detection mechanism that facilitates token-by-token identification of hallucinations during generation and a tree-based revision mechanism to diminish hallucinations further. Extensive experimental evaluations demonstrate that our MCD strategy effectively reduces hallucinations in LLMs and outperforms state-of-the-art methods across various benchmarks.

Antimicrobial peptides (AMPs) are promising candidates for the development of new antibiotics due to their broad-spectrum activity against a range of pathogens. However, identifying AMPs through a huge bunch of candidates is challenging due to their complex structures and diverse sequences. In this study, we propose SenseXAMP, a cross-modal framework that leverages semantic embeddings of and protein descriptors (PDs) of input sequences to improve the identification performance of AMPs. SenseXAMP includes a multi-input alignment module and cross-representation fusion module to explore the hidden information between the two input features and better leverage the fusion feature. To better address the AMPs identification task, we accumulate the latest annotated AMPs data to form more generous benchmark datasets. Additionally, we expand the existing AMPs identification task settings by adding an AMPs regression task to meet more specific requirements like antimicrobial activity prediction. The experimental results indicated that SenseXAMP outperformed existing state-of-the-art models on multiple AMP-related datasets including commonly used AMPs classification datasets and our proposed benchmark datasets. Furthermore, we conducted a series of experiments to demonstrate the complementary nature of traditional PDs and protein pre-training models in AMPs tasks. Our experiments reveal that SenseXAMP can effectively combine the advantages of PDs to improve the performance of protein pre-training models in AMPs tasks.