One individual human’s immune repertoire consists of a huge set of adaptive immune receptors at a certain time point, representing the individual's adaptive immune state. Immune repertoire classification and associated receptor identification have the potential to make a transformative contribution to the development of novel vaccines and therapies. The vast number of instances and exceedingly low witness rate pose a great challenge to the immune repertoire classification, which can be formulated as a Massive Multiple Instance Learning (MMIL) problem. Traditional MIL methods, at both bag-level and instance-level, confront the issues of substantial computational burden or supervision ambiguity when handling massive instances. To address these issues, we propose a novel label disambiguation-based multimodal massive multiple instance learning approach (LaDM³IL) for immune repertoire classification. LaDM³IL adapts the instance-level MIL paradigm to deal with the issue of high computational cost and employs a specially-designed label disambiguation module for label correction, mitigating the impact of misleading supervision. To achieve a more comprehensive representation of each receptor, LaDM³IL leverages a multimodal fusion module with gating-based attention and tensor-fusion to integrate the information from gene segments and amino acid (AA) sequences of each immune receptor. Extensive experiments on the Cytomegalovirus (CMV) and Cancer datasets demonstrate the superior performance of the proposed LaDM³IL for both immune repertoire classification and associated receptor identification tasks. The code is publicly available at https://github.com/Josie-xufan/LaDM3IL.

The in silico prediction of antigen specificity in adaptive immune receptors (AIRs), such as T-cell receptors (TCRs), is essential for understanding immunological processes and developing targeted therapies. The V(D)J gene rearrangement is a critical biological process that generates diversity in amino acid (AA) sequences in antigen-binding regions, enabling AIRs to recognize a wide range of antigens from various pathogens and "altered self cells" observed in cancers. The huge diversity of AIRs presents a significant challenge to existing computational methods for antigen specificity prediction. To address these complexities, we introduce Multimodal-AIR-BERT, a novel multimodal pre-trained model aimed at enhancing the prediction of antigen-binding specificity in TCRs. It comprises a pre-trained sequence encoder, a gene encoder, and a multimodal fusion module with gating-based attention and tensor fusion to calibrate and integrate the V(D)J gene and AA sequence features of TCRs, thereby generating more informative representations. The integration of V(D)J gene information, which provides insights often unobtainable from sequences alone, benefits Multimodal-AIR-BERT in performance enhancement compared to its sequence-modality-only counterpart. Collectively, our work provides an advancement in the accurate prediction of antigen-binding specificity. As the precision of this specificity prediction improves, it can potentially pave the way for targeted immune therapies and deeper insights into the interactions within the immune system.