2519

5930

The emergence of advanced reasoning capabilities in Large Language Models (LLMs) marks a transformative development in healthcare applications. Beyond merely expanding functional capabilities, these reasoning mechanisms enhance decision transparency and explainability-critical requirements in medical contexts. This survey examines the transformation of medical LLMs from basic information retrieval tools to sophisticated clinical reasoning systems capable of supporting complex healthcare decisions. We provide a thorough analysis of the enabling technological foundations, with a particular focus on specialized prompting techniques like Chain-of-Thought and recent breakthroughs in Reinforcement Learning exemplified by DeepSeek-R1. Our investigation evaluates purpose-built medical frameworks while also examining emerging paradigms such as multi-agent collaborative systems and innovative prompting architectures. The survey critically assesses current evaluation methodologies for medical validation and addresses persistent challenges in field interpretation limitations, bias mitigation strategies, patient safety frameworks, and integration of multimodal clinical data. Through this survey, we seek to establish a roadmap for developing reliable LLMs that can serve as effective partners in clinical practice and medical research.

The proliferation of textual data containing sensitive personal information across various domains requires robust anonymization techniques to protect privacy and comply with regulations, while preserving data usability for diverse and crucial downstream tasks. This survey provides a comprehen-sive overview of current trends and recent advances in text anonymization techniques. We begin by discussing foundational approaches, primarily centered on Named Entity Recognition, before examining the transformative impact of Large Language Models, detailing their dual role as sophisticated anonymizers and potent de-anonymization threats. The survey further ex-plores domain-specific challenges and tailored solutions in critical sectors such as healthcare, law, finance, and education. We investigate advanced methodologies incorporating formal privacy models and risk-aware frameworks, and address the specialized subfield of authorship anonymization. Additionally, we review evaluation frameworks, comprehensive metrics, benchmarks, and practical toolkits for real-world deployment of anonymization solutions. This review consolidates current knowledge, identifies emerging trends and persistent challenges, including the evolving privacy-utility trade-off, the need to address quasi-identifiers, and the implications of LLM capabilities, and aims to guide future research directions for both academics and practitioners in this field.

We propose STONK (Stock Optimization using News Knowledge), a multimodal framework integrating numerical market indicators with sentiment-enriched news embeddings to improve daily stock-movement prediction. By combining numerical & textual embeddings via feature concatenation and cross-modal attention, our unified pipeline addresses limitations of isolated analyses. Backtesting shows STONK outperforms numeric-only baselines. A comprehensive evaluation of fusion strategies and model configurations offers evidence-based guidance for scalable multimodal financial forecasting. Source code is available on GitHub11https://github.com/sarthak-12/thesis-dsaa/.

Large Language Models (LLMs) have demonstrated remarkable capabilities but their significant computational and memory demands hinder widespread deployment, especially on resource-constrained devices. Quantization, the process of reducing the numerical precision of model parameters, has emerged as a critical technique for compressing LLMs and accelerating inference. This paper provides an overview of LLM quantization, with a particular focus on the Post-Training Quantization (PTQ) methods implemented within the popular llama. cpp framework and its GGUF file format. We begin by covering quantization fundamentals, including the distinction between PTQ and Quantization-Aware Training (QAT). We then describe the specific PTQ schemes employed by llama. cpp, including legacy methods, advanced K-quants, and recent IQ-quants, along with their underlying mathematical principles. The paper also discusses the impact of these techniques on model fidelity, hardware requirements, inference speed, and traces the adoption of GGUF as a de facto standard in the open-source community. This work serves as a practical guide and comprehensive reference for researchers aiming to deploy LLMs on resource-constrained hardware. By systematically documenting and comparing the PTQ methods within llama. cpp, we provide the necessary insights to navigate the trade-offs between model fidelity, inference speed, and memory footprint. This enables informed decision-making for real-world applications, from local CPU-based inference to efficient edge deployment.

The auditing of financial documents, traditionally a labor-intensive task, is a promising field of application for Artificial Intelligence. Recommendation systems are capable of suggesting the most relevant passages from financial reports that meet accounting standards’ legal requirements. However, testing if the compliance requirements are satisfied is a non-trivial task. In this work, we tackle this problem from two directions. Our first approach leverages Large Language Models which we fine-tune specifically f or compliance checks. Our results show an improvement in performance over the generic baseline LLMs. A disadvantage of LLMs is that they result in high inference costs. For this reason, we explore a second approach in which we use smaller models that come with reduced running costs. Despite their smaller size, these models also show promising predictive performance.

This study explores efficient algorithms to enhance user matching in Unrare.me, a novel social networking platform designed to connect individuals affected by rare diseases. Our primary objective is to develop a recommender system that identifies and suggests users with similar medical conditions, facilitating meaningful connections within these unique communities. Utilizing textual user profile data, we train sentence embedder models to generate similar embeddings for users that have rated each other high. We investigate various fine-tuning strategies, as well as a hybrid approach between a dense embedder and sparse SPLADE embeddings. Furthermore, we investigate the efficacy of various clustering algorithms, such as TopicBERT for thematic analysis, K-Means for centroid-based grouping, and Latent Dirichlet Allocation (LDA) for probabilistic topic modeling, to reduce the matching complexity and enable better scalability of the platform.

We present RepLLaMA, a neural ranking model for optimizing patient matching in rare disease communities. Using data from Unrare.me consisting of over two thousand profiles and over ten thousand ratings, our bi-encoder architecture maps profiles to 4096-dimensional vectors, enabling efficient similarity computations. The system processes unstructured symptom descriptions and structured responses, incorporating expert-guided LLM enhancements. Results show Top-10 Recall of 49.36%$(\pm 2.03)$, surpassing baselines while maintaining generalization. The implementation provides a scalable solution for rare disease patient matching, addressing computational complexity challenges.

With over 7,000 known rare diseases and a prevalence of less than one in a thousand, rare diseases pose substantial challenges to advanced medical support networks. This study investigates the efficacy of Unrare.me, a novel social networking platform designed for individuals affected by rare diseases, including patients, their family members, and medical professionals, addressing data sparsity and combinatorial complexities in user matching. We demonstrate that simple matching heuristics already serve as a decent basis for collecting user feedback on match quality. Leveraging over 10,000 user matching feedback scores from more than 2,000 active users, we evaluate algorithms including collaborative filtering and user embedding similarity with state-of-the-art Large Language Models (LLMs). With a top-10 and top-5 hit-rate of 55% and 37%, respectively, we show that a combination of medical data augmentation and embeddings significantly enhances performance beyond the initial heuristic baseline.

Named Entity Recognition (NER) in the legal domain presents unique challenges due to specialized terminology and complex linguistic structures inherent in legal texts. While large language models (LLMs) like GPT-4, Llama-3, and others have significantly advanced natural language processing, their effectiveness in domain-specific tasks like legal Named Entity Recognition remains underexplored. This study conducts a comprehensive comparative analysis of eleven state-of-the-art LLMs on legal NER tasks across seven diverse datasets in five languages, namely English, Portuguese, German, Turkish, and Ukrainian. We evaluate the models’ performance using F1scores, focusing on their ability to accurately identify and classify legal entities. Our findings reveal significant variability in LLM performance across different languages and legal contexts, with proprietary models like GPT-4 achieving the highest overall scores. The results highlight the influence of model architecture, dataset characteristics, and prompt design on the effectiveness of legal NER tasks. This study provides valuable benchmarks for legal NER applications and offers insights into the strengths and limitations of current LLMs, guiding future research and development in legal natural language processing.

Risk and Quality (R&Q) assurance in highly regulated industries requires constant navigation of complex regulatory frameworks, with employees handling numerous daily queries demanding accurate policy interpretation. Traditional methods relying on specialized experts create operational bottlenecks and limit scalability. We present a novel Retrieval Augmented Generation (RAG) system leveraging Large Language Models (LLMs), hybrid search and relevance boosting to enhance R&Q query processing. Evaluated on 124 expert-annotated real-world queries, our actively deployed system demonstrates substantial improvements over traditional RAG approaches. Additionally, we perform an extensive hyperparameter analysis to compare and evaluate multiple configuration setups, delivering valuable insights to practitioners.

The auditing of financial documents, historically a labor-intensive process, stands on the precipice of transformation. AI-driven solutions have made inroads into streamlining this process by recommending pertinent text passages from financial reports to align with the legal requirements of accounting standards. However, a glaring limitation remains: these systems commonly fall short in verifying if the recommended excerpts indeed comply with the specific legal mandates. Hence, in this paper, we probe the efficiency of publicly available Large Language Models (LLMs) in the realm of regulatory compliance across different model configurations. We place particular emphasis on comparing cutting-edge open-source LLMs, such as Llama-2, with their proprietary counterparts like OpenAI’s GPT models. This comparative analysis leverages two custom datasets provided by our partner PricewaterhouseCoopers (PwC) Germany. We find that the open-source Llama-2 70 billion model demonstrates outstanding performance in detecting non-compliance or true negative occurrences, beating all their proprietary counterparts. Nevertheless, proprietary models such as GPT-4 perform the best in a broad variety of scenarios, particularly in non-English contexts.

Correct identification and correction of contradictions and inconsistencies within financial reports constitute a fundamental component of the audit process. To streamline and automate this critical task, we introduce a novel approach leveraging large language models and an embedding-based paragraph clustering methodology. This paper assesses our approach across three distinct datasets, including two annotated datasets and one unannotated dataset, all within a zero-shot framework. Our findings reveal highly promising results that significantly enhance the effectiveness and efficiency of the auditing process, ultimately reducing the time required for a thorough and reliable financial report audit.

Auditing financial documents is a very tedious and time-consuming process. As of today, it can already be simplified by employing AI-based solutions to recommend relevant text passages from a report for each legal requirement of rigorous accounting standards. However, these methods need to be fine-tuned regularly, and they require abundant annotated data, which is often lacking in industrial environments. Hence, we present ZeroShotALI, a novel recommender system that leverages a state-of-the-art large language model (LLM) in conjunction with a domain-specifically optimized transformer-based text-matching solution. We find that a two-step approach of first retrieving a number of best matching document sections per legal requirement with a custom BERT-based model and second filtering these selections using an LLM yields significant performance improvements over existing approaches.