Information extraction and analysis in healthcare have benefited significantly from advances in the AI and NLP community in recent years, often directly impacting human lives or at least their well-being. However, processing and analyzing mostly unstructured and semi-structured health and biomedical information presents substantial challenges to researchers. The inherent complexity of the domain, limited data availability, data ambiguity, and the high cost of involving medical experts make supporting healthcare difficult. With the recent revolution in AI, particularly in Large Language Models (LLMs), the quality of analyses of these data has increased significantly - though it has not yet reached the level required by medical practitioners. In this work, we address the challenge of information reasoning and question answering, with a particular focus on healthcare. We conduct a detailed analysis of PubMedQA, one of the most challenging and prominent benchmarks. We design, implement, and evaluate Med-RCQ (Medical Reasoning by Concluding and Questioning), a novel LLM-based method to analyze health and biomedical information with the objective of supporting informed decisions. Our PubMedQA benchmark results show that MedRCQ outperforms most competing open approaches in terms of result quality and even provides similar results compared to large commercial models, while outperforming all other benchmark results in an ‘accuracy per number of model parameters’ metric.

The harvesting, management, and analysis of thematic document collections is a major challenge in a wide variety of applications. While the criteria for compiling such collections are individual, the entire process is largely standardized. Therefore, it is not efficient to build new systems over and over again to take over these tasks. In this work, we introduce Seer-Dock, a novel and easy-to deploy general-purpose dockerized framework to build a scholarly document harvesting and management system. It is based on CiteSeerX, the most widely used scholarly search engine. Seer-Dock uses docker containers for all components and thus enables its users to rapidly deploy a full-fledged document collection and management system on any operating system platform and tailor it to the specific needs of an application domain. Moreover, it is easy to scale, orchestrate, maintain, and recover. In this resource paper, we introduce the architecture of Seer-Dock and its components. Like its kernel CiteSeerX, Seer-Dock is available under an Apache 2 open source license.