Surgery continues to be perceived as an art, where proficiency is primarily achieved through years of experience. Artificial Intelligence research has yielded insight into the performance of expert surgeons and their associations with patient outcomes. Clinician expertise has led to the development of systematic assessments for fundamental skills (e.g., End-to-End Assessment of Suturing Expertise [EASE]) that contribute to positive outcomes. However, evaluating these skills requires manual expert review of endoscopic videos and is prone to inconsistencies between human raters. In this work, we present AutoEASE, the first end-to-end pipeline to automatically assess suturing performance from raw endoscopic video data using EASE rubrics. Our system utilizes a Mixture of Expert models (MoE) ; Multiscale vision transformers and 3D convolutional neural networks trained on Robot-assisted Radical Prostatectomy videos with over 13000 data points. For a given stitch clip, the MoE pipeline first determines each phase (needle handling, driving, withdrawal) of a continuous stitch and predicts a binary score (fail / ideal) for seven sub-skills based on rubrics defined in EASE. AutoEASE achieves 0.98 AUC while detecting each stitch phase. For EASE score prediction, the complete end-to-end pipeline attains ≥ 0.77 AUC in sub-skills associated with needle handling and driving. The promising performance of AutoEASE at the individual stitch level demonstrates the feasibility of developing more sophisticated assessment and reporting tools for complete surgical procedures objectively and at scale.

MOTIVATION: LLMs like GPT-4, despite their advancements, often produce hallucinations and struggle with integrating external knowledge effectively. While Retrieval-Augmented Generation (RAG) attempts to address this by incorporating external information, it faces significant challenges such as context length limitations and imprecise vector similarity search. ESCARGOT aims to overcome these issues by combining LLMs with a dynamic Graph of Thoughts and biomedical knowledge graphs, improving output reliability, and reducing hallucinations. RESULT: ESCARGOT significantly outperforms industry-standard RAG methods, particularly in open-ended questions that demand high precision. ESCARGOT also offers greater transparency in its reasoning process, allowing for the vetting of both code and knowledge requests, in contrast to the black-box nature of LLM-only or RAG-based approaches. AVAILABILITY AND IMPLEMENTATION: ESCARGOT is available as a pip package and on GitHub at: https://github.com/EpistasisLab/ESCARGOT.

MOTIVATION: Answering and solving complex problems using a large language model (LLM) given a certain domain such as biomedicine is a challenging task that requires both factual consistency and logic, and LLMs often suffer from some major limitations, such as hallucinating false or irrelevant information, or being influenced by noisy data. These issues can compromise the trustworthiness, accuracy, and compliance of LLM-generated text and insights. RESULTS: Knowledge Retrieval Augmented Generation ENgine (KRAGEN) is a new tool that combines knowledge graphs, Retrieval Augmented Generation (RAG), and advanced prompting techniques to solve complex problems with natural language. KRAGEN converts knowledge graphs into a vector database and uses RAG to retrieve relevant facts from it. KRAGEN uses advanced prompting techniques: namely graph-of-thoughts (GoT), to dynamically break down a complex problem into smaller subproblems, and proceeds to solve each subproblem by using the relevant knowledge through the RAG framework, which limits the hallucinations, and finally, consolidates the subproblems and provides a solution. KRAGEN's graph visualization allows the user to interact with and evaluate the quality of the solution's GoT structure and logic. AVAILABILITY AND IMPLEMENTATION: KRAGEN is deployed by running its custom Docker containers. KRAGEN is available as open-source from GitHub at: https://github.com/EpistasisLab/KRAGEN.

MOTIVATION: Biomedical and healthcare domains generate vast amounts of complex data that can be challenging to analyze using machine learning tools, especially for researchers without computer science training. RESULTS: Aliro is an open-source software package designed to automate machine learning analysis through a clean web interface. By infusing the power of large language models, the user can interact with their data by seamlessly retrieving and executing code pulled from the large language model, accelerating automated discovery of new insights from data. Aliro includes a pre-trained machine learning recommendation system that can assist the user to automate the selection of machine learning algorithms and its hyperparameters and provides visualization of the evaluated model and data. AVAILABILITY AND IMPLEMENTATION: Aliro is deployed by running its custom Docker containers. Aliro is available as open-source from GitHub at: https://github.com/EpistasisLab/Aliro.