While Large Language Models (LLMs) demonstrate remarkable performance on zero-shot annotation tasks, they often struggle with the specialized conventions of gold-standard benchmarks.We propose the systematic reuse and refinement of annotation guidelines as an alignment mechanism, introducing an iterative moderation framework that simulates the early phases of annotation projects.We evaluate three hypotheses: (1) the efficacy of guideline integration, (2) the advantage of reasoningoptimized models, and (3) the viability of moderation under minimal supervision.Testing across biomedical NER tasks (NCBI Disease, BC5CDR, BioRED) with three LLM families (GPT, Gemini, DeepSeek), our results empirically confirm all three hypotheses.While the iterative moderation framework shows good potential in effectively refining guidelines, our analysis also reveals substantial room for improvement.

This paper introduces a multi-lingual database containing translated texts of COVID-19 mythbusters. The database has translations into 115 languages as well as the original English texts, of which the original texts are published by World Health Organization (WHO). This paper then presents preliminary analyses on latin-alphabet-based texts to see the potential of the database as a resource for multilingual linguistic analyses. The analyses on latin-alphabet-based texts gave interesting insights into the resource. While the amount of translated texts in each language was small, character bi-grams with normalization (lowercasing and removal of diacritics) was turned out to be an effective proxy for measuring the similarity of the languages, and the affinity ranking of language pairs could be obtained. Additionally, the hierarchical clustering analysis is performed using the character bigram overlap ratio of every possible pair of languages. The result shows the cluster of Germanic languages, Romance languages, and Southern Bantu languages. In sum, the multilingual database not only offers fixed set of materials in numerous languages, but also serves as a preliminary tool to identify the language family using text-based similarity measure of bigram overlap ratio.

Bridging heterogeneous mutation data fills in the gap between various data categories and propels discovery of disease-related genes. It is known that genome-wide association study (GWAS) infers significant mutation associations that link genotype and phenotype. However, due to the differences of size and quality between GWAS studies, not all de facto vital variations are able to pass the multiple testing. In the meantime, mutation events widely reported in literature unveil typical functional biological process, including mutation types like gain of function and loss of function. To bring together the heterogeneous mutation data, we propose a 'Gene-Disease Association prediction by Mutation Data Bridging (GDAMDB)' pipeline with a statistic generative model. The model learns the distribution parameters of mutation associations and mutation types and recovers false-negative GWAS mutations that fail to pass significant test but represent supportive evidences of functional biological process in literature. Eventually, we applied GDAMDB in Alzheimer's disease (AD) and predicted 79 AD-associated genes. Besides, 12 of them from the original GWAS, 60 of them are supported to be AD-related by other GWAS or literature report, and rest of them are newly predicted genes. Our model is capable of enhancing the GWAS-based gene association discovery by well combining text mining results. The positive result indicates that bridging the heterogeneous mutation data is contributory for the novel disease-related gene discovery.

After mutation, a gene either gains or loses a function, which is considered to be a critical piece of information for understanding a related pathology and thus for drug discovery. By classifying association of genes and phenotypes using the information of gain-of-function or loss-of-function, the exploration for drug repurposing can be guided in a much more efficient way. We present the active gene annotation corpus (AGAC), which contains 500 manually annotated abstracts collected from PubMed. Five bio-concept labels and three regulatory concept labels were designed for concept level annotation. Furthermore, two relation types, “Theme” and “Cause”, were designed to interlink regulatory concepts with their thematic and causal elements. We evaluated AGAC from three aspects, the results of which indicate the high quality of the annotation. Eventually, a PubMed-wide case study was performed to show that by using AGAC the process of anti-epilepsy drug discovery can be enhanced. After text retrieval, filtering, text classification and matching with DrugBank entries, 281 gene-drug pairs and 112 drugs were obtained and 30 out of 112 were recorded in databases. Among 10 newly predicted multi-target drugs which were not recorded in databases, 6 of them were found to be related to epilepsy with literature support, i.e., Oxazepam, Temazepam, Halazepam, Prazepam, Zolpidem and Thiamylal. The result of the case study support the potential of AGAC for enhancing knowledge discovery for drug repurposing.

MOTIVATION: Most currently available text mining tools share two characteristics that make them less than optimal for use by biomedical researchers: they require extensive specialist skills in natural language processing and they were built on the assumption that they should optimize global performance metrics on representative datasets. This is a problem because most end-users are not natural language processing specialists and because biomedical researchers often care less about global metrics like F-measure or representative datasets than they do about more granular metrics such as precision and recall on their own specialized datasets. Thus, there are fundamental mismatches between the assumptions of much text mining work and the preferences of potential end-users. RESULTS: This article introduces the concept of Agile text mining, and presents the PubAnnotation ecosystem as an example implementation. The system approaches the problems from two perspectives: it allows the reformulation of text mining by biomedical researchers from the task of assembling a complete system to the task of retrieving warehoused annotations, and it makes it possible to do very targeted customization of the pre-existing system to address specific end-user requirements. Two use cases are presented: assisted curation of the GlycoEpitope database, and assessing coverage in the literature of pre-eclampsia-associated genes. AVAILABILITY AND IMPLEMENTATION: The three tools that make up the ecosystem, PubAnnotation, PubDictionaries and TextAE are publicly available as web services, and also as open source projects. The dictionaries and the annotation datasets associated with the use cases are all publicly available through PubDictionaries and PubAnnotation, respectively.

Over the past years, several challenges and calls for research projects have pointed out the dire need for pushing natural language interfaces. In this context, the importance of Semantic Web data as a premier knowledge source is rapidly increasing. But we are still far from having accurate natural language interfaces that allow handling complex information needs in a user-centric and highly performant manner. The development of such interfaces requires collaboration of a range of different fields, including natural language processing, information extraction, knowledge base construction and population, reasoning, and question answering. With the goal to join forces in the collaborative development of natural language QA systems, the second OKBQA workshop is organized within the 40th SIGIR conference.

The book begins with a declaration that “the intended audience of the book is natural language processing specialists who want to move into the biomedical domain.” It is indeed a great introductory textbook to the field of biomedical natural language processing (NLP), particularly for NLP specialists. Browsing the contents, many NLP specialists will find the titles of chapters familiar: “Named Entity Recognition,” “Relation Extraction,” and so on. Those familiar topics are reformulated in the context of biomedical informatics, with rich biomedical examples and a good amount of explanation.One of the most important characteristics of the book is that it is very easy to read. Because biomedical NLP is an interdisciplinary area, the quality of this kind of authoring counts on the depth of authors' insight into the relevant subject areas, which include NLP, bioinformatics, medical science, and linguistics. The level of clarity and conciseness throughout the book demonstrates that the authors have a clear and deep understanding of these areas. Complex technical details are successfully distilled into abstract ideas which are at a level that can be understood without much prior knowledge. It is an obviously important feature of an introductory textbook. Because of the ease of reading, I think the book is a good introductory resource even for non-NLP specialists (e.g., bioinformaticians who want to make use of NLP resources).Another notable characteristic of the book is that bio- and medical NLP are equally importantly described. Although biomedical NLP is often referred to as one subject area, the community has been quite divided into bio- and medical NLP, and it is in fact a rare chance to learn about both at the same time. The importance of understanding both is very well described in Chapter 2.3, “Clinical Text Mining,” in the context of translational research.In following sections, I add more detailed reviews on individual chapters. For convenience of review, I take the liberty of dividing the chapters into three parts.Simply speaking, for NLP specialists, entering into biomedical NLP means a change of the type of texts that are going to be dealt with. In this sense, Chapter 1, “Introduction to Natural Language Processing,” is an ideal starting point, presenting the characteristics of various types of biomedical texts and the language in them. Chapter 2, “Historical Background,” presents a concise history of the field. Seamlessly interweaving relevant parts of the history of biomedical informatics and biomedical NLP, it is a masterpiece of introduction to the field—if I were asked to recommend only one chapter of the book to read standing between bookshelves of a library, I would pick this chapter.In the book, explanations about biomedical information often go one step further than I would expect in similar writing. For example, in Chapter 2.4, “Types of Users of Biomedical NLP Systems,” model organisms are explained as major targets of database development. The explanation includes the reason why some organisms (e.g., C. elegans and zebrafish [D. rerio], have been chosen as model organisms. It may be a small addition, but I believe these additional explanations on bioinformatics will make a significant difference to the reader's understanding of biomedical NLP.As in NLP for other domains, named entity recognition, relation extraction, information retrieval, and concept normalization are fundamental tasks of biomedical NLP. Chapters 3–6 describe these four fundamental tasks, in which beginners of biomedical NLP will be immediately interested. Each chapter typically begins with a task definition in general NLP, which is followed by characteristic features specific to the biomedical context. As already mentioned in the previous section, rich explanation about biomedical terms is a strength of the book. For example, Chapter 3.3, “Why Gene Names Are the Way They Are,” will deepen the readers' understanding of gene names one step further.For each task, several software systems are described to exemplify different ideas of how problems of the task can be approached. Instead of briefly mentioning many systems, the authors have chosen to explain a number of representative systems in in-depth detail, which helps the readers to clearly understand the ideas of the approaches.The authors take a scientific approach to developing discussions. In many places, while describing the tasks, naturally occurring questions are captured, for example, how useful are gazetteers for named entity recognition?, how useful are co-occurrences for information extraction?, and so on. Then, discussions on how these questions have been addressed follow. This approach is effective in that it not only keeps the readers interested, but also enables the readers to develop a systematic view of the field, to be able to answer questions such as what are the important problems?, what problems are solved, and how?, and what problems are yet to be solved?Chapter 7, “Ontologies and Computational Lexical Semantics,” describes UMLS and GO as important lexical and ontological resources in biomedical NLP, and also describes contribution of NLP techniques to the development of ontologies. One characteristic of the book is that important resources are described repeatedly in different contexts. For example, UMLS, a compendium of many controlled vocabularies in the biomedical sciences, is referred to in five chapters (Chapters 3–7). This underscores the importance of the resource. The extensive explanation of UMLS, in various application cases, is one important contribution of the book. For biomedical ontologies, OBO (Smith et al. 2007) and BioPortal (Whetzel et al. 2011) are also important community resources. Readers are thus recommended to further read about them.Chapter 8, “Summarization,” and Chapter 9, “Question-Answering,” describe the tasks as the titles imply. Even for the readers who are not interested in these tasks themselves, the two chapters offer a good chance to learn about various user requirements and evaluation of the domain. For those who are interested in the question answering task, BioASQ (Tsatsaronis et al. 2015) is a recent important community effort, and further reading about it is recommended, too.Chapter 10, “Software Engineering,” is focused on software testing. One may think that the topic is a bit out of the focus for the book. However, considering that software testing is an important but often neglected issue, I think it is an appropriate choice for addressing the topic in an introductory textbook; otherwise, it may continue to be neglected. The chapter begins with an episode that showcases the importance of software testing in the field, which is very effective in motivating the readers to engage in software testing. From the perspective of software engineering, considering that the number of open source projects and interconnection of open Web services are increasing, software testing will become an increasingly important issue.The book ends with Chapter 11, “Corpus Construction and Annotation,” where the authors attempt to identify important features of successful corpora and annotation.I highly recommend the book not only to everyone new to biomedical NLP, but also to those who are already working in it. The level of understanding and vision of the authors is absolutely outstanding, and the book offers a good chance to acquire it. Through the book, I believe the readers will be able to get a sense of the big picture for biomedical NLP, and to understand motivations and requirements from the field of bioinformatics and medical science, as well as common problems and solutions that have to be addressed and developed.

BACKGROUND: The third edition of the BioNLP Shared Task was held with the grand theme "knowledge base construction (KB)". The Genia Event (GE) task was re-designed and implemented in light of this theme. For its final report, the participating systems were evaluated from a perspective of annotation. To further explore the grand theme, we extended the evaluation from a perspective of KB construction. Also, the Gene Regulation Ontology (GRO) task was newly introduced in the third edition. The final evaluation of the participating systems resulted in relatively low performance. The reason was attributed to the large size and complex semantic representation of the ontology. To investigate potential benefits of resource exchange between the presumably similar tasks, we measured the overlap between the datasets of the two tasks, and tested whether the dataset for one task can be used to enhance performance on the other. RESULTS: We report an extended evaluation on all the participating systems in the GE task, incoporating a KB perspective. For the evaluation, the final submission of each participant was converted to RDF statements, and evaluated using 8 queries that were formulated in SPARQL. The results suggest that the evaluation may be concluded differently between the two different perspectives, annotation vs. KB. We also provide a comparison of the GE and GRO tasks by converting their datasets into each other's format. More than 90% of the GE data could be converted into the GRO task format, while only half of the GRO data could be mapped to the GE task format. The imbalance in conversion indicates that the GRO is a comprehensive extension of the GE task ontology. We further used the converted GRO data as additional training data for the GE task, which helped improve GE task participant system performance. However, the converted GE data did not help GRO task participants, due to overfitting and the ontology gap.

BACKGROUND: The Genia task, when it was introduced in 2009, was the first community-wide effort to address a fine-grained, structural information extraction from biomedical literature. Arranged for the second time as one of the main tasks of BioNLP Shared Task 2011, it aimed to measure the progress of the community since 2009, and to evaluate generalization of the technology to full text papers. The Protein Coreference task was arranged as one of the supporting tasks, motivated from one of the lessons of the 2009 task that the abundance of coreference structures in natural language text hinders further improvement with the Genia task. RESULTS: The Genia task received final submissions from 15 teams. The results show that the community has made a significant progress, marking 74% of the best F-score in extracting bio-molecular events of simple structure, e.g., gene expressions, and 45% ~ 48% in extracting those of complex structure, e.g., regulations. The Protein Coreference task received 6 final submissions. The results show that the coreference resolution performance in biomedical domain is lagging behind that in newswire domain, cf. 50% vs. 66% in MUC score. Particularly, in terms of protein coreference resolution the best system achieved 34% in F-score. CONCLUSIONS: Detailed analysis performed on the results improves our insight into the problem and suggests the directions for further improvements.

BACKGROUND: Current research has shown that major difficulties in event extraction for the biomedical domain are traceable to coreference. Therefore, coreference resolution is believed to be useful for improving event extraction. To address coreference resolution in molecular biology literature, the Protein Coreference (COREF) task was arranged in the BioNLP Shared Task (BioNLP-ST, hereafter) 2011, as a supporting task. However, the shared task results indicated that transferring coreference resolution methods developed for other domains to the biological domain was not a straight-forward task, due to the domain differences in the coreference phenomena. RESULTS: We analyzed the contribution of domain-specific information, including the information that indicates the protein type, in a rule-based protein coreference resolution system. In particular, the domain-specific information is encoded into semantic classification modules for which the output is used in different components of the coreference resolution. We compared our system with the top four systems in the BioNLP-ST 2011; surprisingly, we found that the minimal configuration had outperformed the best system in the BioNLP-ST 2011. Analysis of the experimental results revealed that semantic classification, using protein information, has contributed to an increase in performance by 2.3% on the test data, and 4.0% on the development data, in F-score. CONCLUSIONS: The use of domain-specific information in semantic classification is important for effective coreference resolution. Since it is difficult to transfer domain-specific information across different domains, we need to continue seek for methods to utilize such information in coreference resolution.

Term clustering by measuring the string similarities between terms is known to be an effective method to improve the quality of texts and dictionaries. However, based on our observations, chemical names are difficult to cluster using string similarity measures such as the edit distance. To demonstrate this difficulty clearly, we compared the string similarities determined using the edit distance, the Monge-Elkan score, SoftTFIDF, and the bigram Dice coefficient for chemical names with those for other terms. The experimental results show that the discriminative application of string similarity methods to chemical and non-chemical names may be a simple but effective way to improve the performance of term clustering.

UNLABELLED: Often, the most informative genes have to be selected from different gene sets and several computer gene ranking algorithms have been developed to cope with the problem. To help researchers decide which algorithm to use, we developed the analysis of gene ranking algorithms (AGRA) system that offers a novel technique for comparing ranked lists of genes. The most important feature of AGRA is that no previous knowledge of gene ranking algorithms is needed for their comparison. Using the text mining system finding-associated concepts with text analysis. AGRA defines what we call biomedical concept space (BCS) for each gene list and offers a comparison of the gene lists in six different BCS categories. The uploaded gene lists can be compared using two different methods. In the first method, the overlap between each pair of two gene lists of BCSs is calculated. The second method offers a text field where a specific biomedical concept can be entered. AGRA searches for this concept in each gene lists' BCS, highlights the rank of the concept and offers a visual representation of concepts ranked above and below it. AVAILABILITY AND IMPLEMENTATION: Available at http://agra.fzv.uni-mb.si/, implemented in Java and running on the Glassfish server. CONTACT: [email protected].

BACKGROUND: Bio-molecular event extraction from literature is recognized as an important task of bio text mining and, as such, many relevant systems have been developed and made available during the last decade. While such systems provide useful services individually, there is a need for a meta-service to enable comparison and ensemble of such services, offering optimal solutions for various purposes. RESULTS: We have integrated nine event extraction systems in the U-Compare framework, making them intercompatible and interoperable with other U-Compare components. The U-Compare event meta-service provides various meta-level features for comparison and ensemble of multiple event extraction systems. Experimental results show that the performance improvements achieved by the ensemble are significant. CONCLUSIONS: While individual event extraction systems themselves provide useful features for bio text mining, the U-Compare meta-service is expected to improve the accessibility to the individual systems, and to enable meta-level uses over multiple event extraction systems such as comparison and ensemble.

This paper presents the preparation, results and analysis of the BioNLP’09 shared task on event extraction, organized to address the automatic extraction of fine-grained information from the scientific literature on molecular biology. A representation of this information was defined taking into account both the biological and computational requirements of the task, and corpus resources manually annotated by domain experts provided to task participants. To create a basis for further progress, emphasis was placed on providing fine-grained evaluation that isolates different subtasks and allows the analysis of different aspects of the results through various evaluation criteria. In introducing this new task to the community, we made an effort to reduce the cost of participation by making common natural language processing tools, data, and evaluation methods easily accessible. The task received community-wide participation, establishing the state-of-the-art performance at fine-grained event extraction as well as allowing the identification of remaining challenges and suggesting directions for future improvements. All the resources and results of the shared task are publicly available and an online evaluation on blind test data accessible at http://www-tsujii.is.s.u-tokyo.ac.jp/GENIA/SharedTask/.

We propose a method for classifying opinions which captures the role of linguistic modalities in the sentence. We use features than simple bag-of-words or opinion-holding predicates. The method is based on a machine learning and utilizes opinion-holding predicates and linguistic modalities as features. Two different detectors help to classify the opinions: the opinion-holding predicate detector and the modality detector. An opinion in the target is first parsed into a dependency structure, and then the opinion-holding predicates and modalities stick onto the leaf nodes of the dependency tree. The whole tree is regarded as input features of the opinion, and it becomes the input of tree kernel support vector machines. We have applied method to opinions in Japanese about television programs, and have confirmed the effectiveness of the method against conventional bag-of-words features, or against simple opinion-holding predicates features

BACKGROUND: The number of corpora, collections of structured texts, has been increasing, as a result of the growing interest in the application of natural language processing methods to biological texts. Many named entity recognition (NER) systems have been developed based on these corpora. However, in the biomedical community, there is yet no general consensus regarding named entity annotation; thus, the resources are largely incompatible, and it is difficult to compare the performance of systems developed on resources that were divergently annotated. On the other hand, from a practical application perspective, it is desirable to utilize as many existing annotated resources as possible, because annotation is costly. Thus, it becomes a task of interest to integrate the heterogeneous annotations in these resources. RESULTS: We explore the potential sources of incompatibility among gene and protein annotations that were made for three common corpora: GENIA, GENETAG and AIMed. To show the inconsistency in the corpora annotations, we first tackle the incompatibility problem caused by corpus integration, and we quantitatively measure the effect of this incompatibility on protein mention recognition. We find that the F-score performance declines tremendously when training with integrated data, instead of training with pure data; in some cases, the performance drops nearly 12%. This degradation may be caused by the newly added heterogeneous annotations, and cannot be fixed without an understanding of the heterogeneities that exist among the corpora. Motivated by the result of this preliminary experiment, we further qualitatively analyze a number of possible sources for these differences, and investigate the factors that would explain the inconsistencies, by performing a series of well-designed experiments. Our analyses indicate that incompatibilities in the gene/protein annotations exist mainly in the following four areas: the boundary annotation conventions, the scope of the entities of interest, the distribution of annotated entities, and the ratio of overlap between annotated entities. We further suggest that almost all of the incompatibilities can be prevented by properly considering the four aspects aforementioned. CONCLUSION: Our analysis covers the key similarities and dissimilarities that exist among the diverse gene/protein corpora. This paper serves to improve our understanding of the differences in the three studied corpora, which can then lead to a better understanding of the performance of protein recognizers that are based on the corpora.

BACKGROUND: Advanced Text Mining (TM) such as semantic enrichment of papers, event or relation extraction, and intelligent Question Answering have increasingly attracted attention in the bio-medical domain. For such attempts to succeed, text annotation from the biological point of view is indispensable. However, due to the complexity of the task, semantic annotation has never been tried on a large scale, apart from relatively simple term annotation. RESULTS: We have completed a new type of semantic annotation, event annotation, which is an addition to the existing annotations in the GENIA corpus. The corpus has already been annotated with POS (Parts of Speech), syntactic trees, terms, etc. The new annotation was made on half of the GENIA corpus, consisting of 1,000 Medline abstracts. It contains 9,372 sentences in which 36,114 events are identified. The major challenges during event annotation were (1) to design a scheme of annotation which meets specific requirements of text annotation, (2) to achieve biology-oriented annotation which reflect biologists' interpretation of text, and (3) to ensure the homogeneity of annotation quality across annotators. To meet these challenges, we introduced new concepts such as Single-facet Annotation and Semantic Typing, which have collectively contributed to successful completion of a large scale annotation. CONCLUSION: The resulting event-annotated corpus is the largest and one of the best in quality among similar annotation efforts. We expect it to become a valuable resource for NLP (Natural Language Processing)-based TM in the bio-medical domain.

BACKGROUND: Associating literature with pathways poses new challenges to the Text Mining (TM) community. There are three main challenges to this task: (1) the identification of the mapping position of a specific entity or reaction in a given pathway, (2) the recognition of the causal relationships among multiple reactions, and (3) the formulation and implementation of required inferences based on biological domain knowledge. RESULTS: To address these challenges, we constructed new resources to link the text with a model pathway; they are: the GENIA pathway corpus with event annotation and NF-kB pathway. Through their detailed analysis, we address the untapped resource, 'bio-inference,' as well as the differences between text and pathway representation. Here, we show the precise comparisons of their representations and the nine classes of 'bio-inference' schemes observed in the pathway corpus. CONCLUSIONS: We believe that the creation of such rich resources and their detailed analysis is the significant first step for accelerating the research of the automatic construction of pathway from text.

We present a practical HPSG parser for English, an intelligent search engine to retrieve MEDLINE abstracts that represent biomedical events and an efficient MED-LINE search tool helping users to find information about biomedical entities such as genes, proteins, and the interactions between them.

BACKGROUND: Automatic recognition of relations between a specific disease term and its relevant genes or protein terms is an important practice of bioinformatics. Considering the utility of the results of this approach, we identified prostate cancer and gene terms with the ID tags of public biomedical databases. Moreover, considering that genetics experts will use our results, we classified them based on six topics that can be used to analyze the type of prostate cancers, genes, and their relations. METHODS: We developed a maximum entropy-based named entity recognizer and a relation recognizer and applied them to a corpus-based approach. We collected prostate cancer-related abstracts from MEDLINE, and constructed an annotated corpus of gene and prostate cancer relations based on six topics by biologists. We used it to train the maximum entropy-based named entity recognizer and relation recognizer. RESULTS: Topic-classified relation recognition achieved 92.1% precision for the relation (an increase of 11.0% from that obtained in a baseline experiment). For all topics, the precision was between 67.6 and 88.1%. CONCLUSION: A series of experimental results revealed two important findings: a carefully designed relation recognition system using named entity recognition can improve the performance of relation recognition, and topic-classified relation recognition can be effectively addressed through a corpus-based approach using manual annotation and machine learning techniques.

This paper presents a method to develop a class of variable memory Markov models that have higher memory capacity than traditional (uniform memory) Markov models. The structure of the variable memory models is induced from a manually annotated corpus through a decision tree learning algorithm. A series of comparative experiments show the resulting models outperform uniform memory Markov models in a part-of-speech tagging task.