Protein language models (pLM) are beneficial to build antibody property prediction models. However, current pLMs lacks the ability to understand antibody properties because region and structure information is not effectively embedded. We propose the Region-Empowered Antibody Language Model (REALM), a pLM built by multi-task pretraining strategy of residue prediction and region prediction tasks in antibodies, to incorporate not only co-evolution but also region information of antibodies. We demonstrate that our REALM improves the understanding of antibody properties, including hydrophobicity and thermo-stability.

Monoclonal antibodies (mAbs) offer significant therapeutic benefits; however, their formulation requires careful optimization to prevent instability. Standard practices for determining optimal formulation conditions rely on time-consuming and costly wet lab experiments. We developed a machine learning-based approach to predict the optimal pH value for stabilizing mAbs using only their amino acid sequences by leveraging a protein language model. Due to the absence of directly relevant methods, we established a baseline by comparing various combinations of elements. We also conducted feature engineering to enhance the predictive performance by incorporating structural information and descriptors. Our approach achieved a high Pearson correlation coefficient of 0.88 on the test folds from 10-fold cross-validation, highlighting its potential to complement wet lab experiments and increase the efficiency of mAb formulation.

Radiology report generation systems have the potential to reduce the workload of radiologists by automatically describing the findings in medical images.To broaden the application of the report generation system, the system should generate reports that are not only factually accurate but also chronologically consistent, describing images that are presented in time order, that is, the correct order.We employ a planning-based radiology report generation system that generates the overall structure of reports as "plans" prior to generating reports that are accurate and consistent in order.Additionally, we propose a novel reinforcement learning and inference method, Coordinated Planning (CoPlan), that includes a content planner and a text generator to train and infer in a coordinated manner to alleviate the cascading of errors that are often inherent in planning-based models.We conducted experiments with single-phase diagnostic reports in which the factual accuracy is critical and multi-phase diagnostic reports in which the description order is critical.Our proposed CoPlan improves the content order score by 5.1 pt in time series critical scenarios and the clinical factual accuracy F-score by 9.1 pt in time series irrelevant scenarios, compared those of the baseline models without CoPlan.

Ryuji Kano, Takumi Takahashi, Toru Nishino, Motoki Taniguchi, Tomoki Taniguchi, Tomoko Ohkuma. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021.

Toru Nishino, Shotaro Misawa, Ryuji Kano, Tomoki Taniguchi, Yasuhide Miura, Tomoko Ohkuma. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.