A Whole Slide Image (WSI) is a high-resolution digital image created by scanning an entire glass slide containing a biological specimen, such as tissue sections or cell samples, at multiple magnifications. These images are digitally viewable, analyzable, and shareable, and are widely used for Artificial Intelligence (AI) algorithm development. WSIs play an important role in pathology for disease diagnosis and oncology for cancer research, but are also applied in neurology, veterinary medicine, hematology, microbiology, dermatology, pharmacology, toxicology, immunology, and forensic science. When assembling cohorts for AI training or validation, it is essential to know the content of a WSI. However, no standard currently exists for this metadata, and such a selection has largely relied on manual inspection, which is not suitable for large collections with millions of objects. We propose a general framework to generate 2D index maps (tissue maps) that describe the morphological content of WSIs using common syntax and semantics to achieve interoperability between catalogs. The tissue maps are structured in three layers: source, tissue type, and pathological alterations. Each layer assigns WSI segments to specific classes, providing AI-ready metadata. We demonstrate the advantages of this standard by applying AI-based metadata extraction from WSIs to generate tissue maps and integrating them into a WSI archive. This integration enhances search capabilities within WSI archives, thereby facilitating the accelerated assembly of high-quality, balanced, and more targeted datasets for AI training, validation, and cancer research.

This paper presents the development of an innovative communication interface between humans and robots, designed for human-in-the-loop interaction with a user interface through natural language. The novelty of the presented approach lies in integration of a Belief-Desire-Intention agent, communicating directly to a robot and ensuring safety properties and verifiable decision making, with large language models that excel in language understanding and generation. We establish a framework that leverages the strengths of both paradigms by allowing users to formulate commands in natural language, using the ability of large language models to interpret and ground them into actionable goals, with the proven ability of Belief-Desire Intention agents to perform verifiable reasoning and goal management. In addition, we utilize the ability of this agent to represent and store its mind state, including its belief base, plan library, and history of selected events and actions, to allow large language models based explanations of the system behavior. Our findings demonstrate that this architecture provides several benefits in terms of performance and safety, without impacting efficiency or requiring extensive integration effort. This research contributes a novel perspective on the combination of large language models with symbolic approaches for explainable Artificial Intelligence and aims at inspiring new developments in human-centered Artificial Intelligence systems.

Integrating large language models (LLMs) to retrieve targeted medical knowledge from electronic health records enables significant advancements in medical research. However, recognizing the challenges associated with using LLMs in healthcare is essential for successful implementation. One challenge is that medical records combine unstructured textual information with highly sensitive personal data. This, in turn, highlights the need for explainable Artificial Intelligence (XAI) methods to understand better how LLMs function in the medical domain. In this study, we propose a novel XAI tool to accelerate data-driven cancer research. We apply the Bidirectional Encoder Representations from Transformers (BERT) model to German language pathology reports examining the effects of domain-specific language adaptation and fine-tuning. We demonstrate our model on a real-world pathology dataset, analyzing the contextual representations of diagnostic reports. By illustrating decisions made by fine-tuned models, we provide decision values that can be applied in medical research. To address interpretability, we conduct a performance evaluation of the classifications generated by our fine-tuned model, as assessed by an expert pathologist. In domains such as medicine, inspection of the medical knowledge map in conjunction with expert evaluation reveals valuable information about how contextual representations of key disease features are categorized. This ultimately benefits data structuring and labeling and paves the way for even more advanced approaches to XAI, combining text with other input modalities, such as images which are then applicable to various engineering problems. • Engineering accurate datasets is a vital step in developing machine learning algorithms. • Electronic health records are the most important resource for clinical datasets. • Language models are adaptable to compute embeddings for pathology diagnostic reports. • Computational analysis shows embeddings correlate with medically relevant information. • An expert review by a pathologist confirms embeddings reveal key medical patterns.

ABSTRACT Interpreting the predictions of deep learning models on 3D point cloud data is an important challenge for safety‐critical domains such as autonomous driving, robotics and geospatial analysis. Existing counterfactual explainability methods often struggle with the sparsity and unordered nature of 3D point clouds. To address this, we introduce a generative framework for counterfactual explanations in 3D semantic segmentation models. Our approach leverages autoencoder‐based latent representations, combined with UMAP embeddings and Delaunay triangulation, to construct a graph that enables geodesic path search between semantic classes. Candidate counterfactuals are generated by interpolating latent vectors along these paths and decoding into plausible point clouds, while semantic plausibility is guided by the predictions of a 3D semantic segmentation model. We evaluate the framework on ShapeNet objects, demonstrating that semantically related classes yield realistic counterfactuals with minimal geometric change, whereas unrelated classes expose sharp decision boundaries and reduced plausibility. Quantitative results confirm that the method balances defined interpretability metrics, producing counterfactuals that are both interpretable and geometrically consistent. Overall, our work demonstrates that generative counterfactuals in latent space provide a promising alternative to input‐level perturbations.

Random Forests (RFs) are powerful ensemble learning algorithms that are widely used in various machine learning tasks. However, they tend to overfit noisy or irrelevant features, which can result in decreased generalization performance. Post-hoc regularization techniques aim to solve this problem by modifying the structure of the learned ensemble after training. We propose a novel post-hoc regularization via tree smoothing for classification tasks to leverage the reliable class distributions closer to the root node whilst reducing the impact of more specific and potentially noisy splits deeper in the tree. Our novel approach allows for a form of pruning that does not alter the general structure of the trees, adjusting the influence of nodes based on their proximity to the root node. We evaluated the performance of our method on various machine learning benchmark data sets and on cancer data from The Cancer Genome Atlas (TCGA). Our approach demonstrates competitive performance compared to the state-of-the-art and, in the majority of cases, and outperforms it in most cases in terms of prediction accuracy, generalization, and interpretability. • A novel post-regulation technique for Tree Ensembles called BBTS is introduced. • Interpretability is improved through posterior distributions in the leaf nodes. • This method allows the incorporation of domain knowledge through prior beliefs. • It was tested using ML benchmarks and real-world cancer data from the TCGA database. • BBTS has proven itself with the state-of-the-art and exceeds them on real-world cancer data.

Abstract Explainable AI (XAI) is an increasingly important area of machine learning research, which aims to make black-box models transparent and interpretable. In this paper, we propose a novel approach to XAI that uses the so-called counterfactual paths for model-agnostic global explanations. The algorithm measures feature importance by identifying sequential permutations of features that most influence changes in model predictions. It is particularly suitable for generating explanations based on counterfactual paths in knowledge graphs incorporating domain knowledge. Counterfactual paths introduce an additional graph dimension to current XAI methods in both explaining and visualizing black-box models. Experiments with synthetic and bio-medical data demonstrate the practical applicability of our approach.

Abstract Numerical association rule mining (NARM) is a popular method under the umbrella of data mining, focused on finding relationships between attributes in transaction databases. Numerical association rules for time series are a new paradigm that extends the applicability of NARM to the domain of time series. Association rule mining algorithms result in numerous rules, the interpretation of which is sometimes not easy for human experts. Therefore, various visualization methods have been developed to improve the explanation results of the rule mining process. This article is a novel contribution to the development of a new visualization method capable of presenting the association rules for time series developed according to the principles of explainable artificial intelligence. The experiments are conducted in the context of smart agriculture (i.e., agricultural time series data), and show the great potential of the proposed visualization method for the future.

Deep learning models like chatGPT exemplify AI success but necessitate a deeper understanding of trust in critical sectors. Trust can be achieved using counterfactual explanations, which is how humans become familiar with unknown processes; by understanding the hypothetical input circumstances under which the output changes. We argue that the generation of counterfactual explanations requires several aspects of the generated counterfactual instances, not just their counterfactual ability. We present a framework for generating counterfactual explanations that formulate its goal as a multiobjective optimization problem balancing three objectives: plausibility; the intensity of changes; and adversarial power. We use a generative adversarial network to model the distribution of the input, along with a multiobjective counterfactual discovery solver balancing these objectives. We demonstrate the usefulness of six classification tasks with image and 3D data confirming with evidence the existence of a trade-off between the objectives, the consistency of the produced counterfactual explanations with human knowledge, and the capability of the framework to unveil the existence of concept-based biases and misrepresented attributes in the input domain of the audited model. Our pioneering effort shall inspire further work on the generation of plausible counterfactual explanations in real-world scenarios where attribute-/concept-based annotations are available for the domain under analysis.

Artificial intelligence (AI) and especially reinforcement learning (RL) have the potential to enable agents to learn and perform tasks autonomously with superhuman performance. However, we consider RL as fundamentally a Human-in-the-Loop (HITL) paradigm, even when an agent eventually performs its task autonomously. In cases where the reward function is challenging or impossible to define, HITL approaches are considered particularly advantageous. The application of Reinforcement Learning from Human Feedback (RLHF) in systems such as ChatGPT demonstrates the effectiveness of optimizing for user experience and integrating their feedback into the training loop. In HITL RL, human input is integrated during the agent’s learning process, allowing iterative updates and fine-tuning based on human feedback, thus enhancing the agent’s performance. Since the human is an essential part of this process, we argue that human-centric approaches are the key to successful RL, a fact that has not been adequately considered in the existing literature. This paper aims to inform readers about current explainability methods in HITL RL. It also shows how the application of explainable AI (xAI) and specific improvements to existing explainability approaches can enable a better human-agent interaction in HITL RL for all types of users, whether for lay people, domain experts, or machine learning specialists. Accounting for the workflow in HITL RL and based on software and machine learning methodologies, this article identifies four phases for human involvement for creating HITL RL systems: (1) Agent Development, (2) Agent Learning, (3) Agent Evaluation, and (4) Agent Deployment. We highlight human involvement, explanation requirements, new challenges, and goals for each phase. We furthermore identify low-risk, high-return opportunities for explainability research in HITL RL and present long-term research goals to advance the field. Finally, we propose a vision of human-robot collaboration that allows both parties to reach their full potential and cooperate effectively.

BACKGROUND: Lack of trust in artificial intelligence (AI) models in medicine is still the key blockage for the use of AI in clinical decision support systems (CDSS). Although AI models are already performing excellently in systems medicine, their black-box nature entails that patient-specific decisions are incomprehensible for the physician. Explainable AI (XAI) algorithms aim to "explain" to a human domain expert, which input features influenced a specific recommendation. However, in the clinical domain, these explanations must lead to some degree of causal understanding by a clinician. RESULTS: We developed the CLARUS platform, aiming to promote human understanding of graph neural network (GNN) predictions. CLARUS enables the visualisation of patient-specific networks, as well as, relevance values for genes and interactions, computed by XAI methods, such as GNNExplainer. This enables domain experts to gain deeper insights into the network and more importantly, the expert can interactively alter the patient-specific network based on the acquired understanding and initiate re-prediction or retraining. This interactivity allows us to ask manual counterfactual questions and analyse the effects on the GNN prediction. CONCLUSION: We present the first interactive XAI platform prototype, CLARUS, that allows not only the evaluation of specific human counterfactual questions based on user-defined alterations of patient networks and a re-prediction of the clinical outcome but also a retraining of the entire GNN after changing the underlying graph structures. The platform is currently hosted by the GWDG on https://rshiny.gwdg.de/apps/clarus/.

Abstract Asymmetric Shapley values (ASVs) are an extension of Shapley values that allow a user to incorporate partial causal knowledge into the explanation process. Unfortunately, computing ASVs requires sampling permutations, which quickly becomes computationally expensive. We propose A-PDD-SHAP, an algorithm that employs a functional decomposition approach to approximate ASVs at a speed orders of magnitude faster compared to permutation sampling, which significantly reduces the amortized complexity of computing ASVs when many explanations are needed. Apart from this, once the A-PDD-SHAP model is trained, it can be used to compute both symmetric and asymmetric Shapley values without having to re-train or re-sample, allowing for very efficient comparisons between different types of explanations.

Abstract We explore the integration of domain knowledge graphs into Deep Learning for improved interpretability and explainability using Graph Neural Networks (GNNs). Specifically, a protein-protein interaction (PPI) network is masked over a deep neural network for classification, with patient-specific multi-modal genomic features enriched into the PPI graph’s nodes. Subnetworks that are relevant to the classification (referred to as “disease subnetworks”) are detected using explainable AI. Federated learning is enabled by dividing the knowledge graph into relevant subnetworks, constructing an ensemble classifier, and allowing domain experts to analyze and manipulate detected subnetworks using a developed user interface. Furthermore, the human-in-the-loop principle can be applied with the incorporation of experts, interacting through a sophisticated User Interface (UI) driven by Explainable Artificial Intelligence (xAI) methods, changing the datasets to create counterfactual explanations. The adapted datasets could influence the local model’s characteristics and thereby create a federated version that distils their diverse knowledge in a centralized scenario. This work demonstrates the feasibility of the presented strategies, which were originally envisaged in 2021 and most of it has now been materialized into actionable items. In this paper, we report on some lessons learned during this project.

Abstract The release of ChatGPT to the general public has sparked discussions about the dangers of artificial intelligence (AI) among the public. The European Commission’s draft of the AI Act has further fueled these discussions, particularly in relation to the definition of AI and the assignment of risk levels to different technologies. Security concerns in AI systems arise from the need to protect against potential adversaries and to safeguard individuals from AI decisions that may harm their well-being. However, ensuring secure and trustworthy AI systems is challenging, especially with deep learning models that lack explainability. This paper proposes the concept of Controllable AI as an alternative to Trustworthy AI and explores the major differences between the two. The aim is to initiate discussions on securing complex AI systems without sacrificing practical capabilities or transparency. The paper provides an overview of techniques that can be employed to achieve Controllable AI. It discusses the background definitions of explainability, Trustworthy AI, and the AI Act. The principles and techniques of Controllable AI are detailed, including detecting and managing control loss, implementing transparent AI decisions, and addressing intentional bias or backdoors. The paper concludes by discussing the potential applications of Controllable AI and its implications for real-world scenarios.

Abstract As machine learning (ML) has emerged as the predominant technological paradigm for artificial intelligence (AI), complex black box models such as GPT-4 have gained widespread adoption. Concurrently, explainable AI (XAI) has risen in significance as a counterbalancing force. But the rapid expansion of this research domain has led to a proliferation of terminology and an array of diverse definitions, making it increasingly challenging to maintain coherence. This confusion of languages also stems from the plethora of different perspectives on XAI, e.g. ethics, law, standardization and computer science. This situation threatens to create a “tower of Babel” effect, whereby a multitude of languages impedes the establishment of a common (scientific) ground. In response, this paper first maps different vocabularies, used in ethics, law and standardization. It shows that despite a quest for standardized, uniform XAI definitions, there is still a confusion of languages. Drawing lessons from these viewpoints, it subsequently proposes a methodology for identifying a unified lexicon from a scientific standpoint. This could aid the scientific community in presenting a more unified front to better influence ongoing definition efforts in law and standardization, often without enough scientific representation, which will shape the nature of AI and XAI in the future.

SUMMARY: Federated learning enables collaboration in medicine, where data is scattered across multiple centers without the need to aggregate the data in a central cloud. While, in general, machine learning models can be applied to a wide range of data types, graph neural networks (GNNs) are particularly developed for graphs, which are very common in the biomedical domain. For instance, a patient can be represented by a protein-protein interaction (PPI) network where the nodes contain the patient-specific omics features. Here, we present our Ensemble-GNN software package, which can be used to deploy federated, ensemble-based GNNs in Python. Ensemble-GNN allows to quickly build predictive models utilizing PPI networks consisting of various node features such as gene expression and/or DNA methylation. We exemplary show the results from a public dataset of 981 patients and 8469 genes from the Cancer Genome Atlas (TCGA). AVAILABILITY AND IMPLEMENTATION: The source code is available at https://github.com/pievos101/Ensemble-GNN, and the data at Zenodo (DOI: 10.5281/zenodo.8305122).

In this paper, we present a fundamental framework for defining different types of explanations of AI systems and the criteria for evaluating their quality. Starting from a structural view of how explanations can be constructed, i.e., in terms of an explanandum (what needs to be explained), multiple explanantia (explanations, clues, or parts of information that explain), and a relationship linking explanandum and explanantia, we propose an explanandum-based typology and point to other possible typologies based on how explanantia are presented and how they relate to explanandia. We also highlight two broad and complementary perspectives for defining possible quality criteria for assessing explainability: epistemological and psychological (cognitive). These definition attempts aim to support the three main functions that we believe should attract the interest and further research of XAI scholars: clear inventories, clear verification criteria, and clear validation methods.

Optimal performance is desired for decision-making in any field with binary classifiers and diagnostic tests, however common performance measures lack depth in information. The area under the receiver operating characteristic curve (AUC) and the area under the precision recall curve are too general because they evaluate all decision thresholds including unrealistic ones. Conversely, accuracy, sensitivity, specificity, positive predictive value and the F1 score are too specific-they are measured at a single threshold that is optimal for some instances, but not others, which is not equitable. In between both approaches, we propose deep ROC analysis to measure performance in multiple groups of predicted risk (like calibration), or groups of true positive rate or false positive rate. In each group, we measure the group AUC (properly), normalized group AUC, and averages of: sensitivity, specificity, positive and negative predictive value, and likelihood ratio positive and negative. The measurements can be compared between groups, to whole measures, to point measures and between models. We also provide a new interpretation of AUC in whole or part, as balanced average accuracy, relevant to individuals instead of pairs. We evaluate models in three case studies using our method and Python toolkit and confirm its utility.

The rapid increase of interest in, and use of, artificial intelligence (AI) in computer applications has raised a parallel concern about its ability (or lack thereof) to provide understandable, or explainable, output to users. This concern is especially legitimate in biomedical contexts, where patient safety is of paramount importance. This position paper brings together seven researchers working in the field with different roles and perspectives, to explore in depth the concept of explainable AI, or XAI, offering a functional definition and conceptual framework or model that can be used when considering XAI. This is followed by a series of desiderata for attaining explainability in AI, each of which touches upon a key domain in biomedicine.

MOTIVATION: Limited data access has hindered the field of precision medicine from exploring its full potential, e.g. concerning machine learning and privacy and data protection rules.Our study evaluates the efficacy of federated Random Forests (FRF) models, focusing particularly on the heterogeneity within and between datasets. We addressed three common challenges: (i) number of parties, (ii) sizes of datasets and (iii) imbalanced phenotypes, evaluated on five biomedical datasets. RESULTS: The FRF outperformed the average local models and performed comparably to the data-centralized models trained on the entire data. With an increasing number of models and decreasing dataset size, the performance of local models decreases drastically. The FRF, however, do not decrease significantly. When combining datasets of different sizes, the FRF vastly improve compared to the average local models. We demonstrate that the FRF remain more robust and outperform the local models by analyzing different class-imbalances.Our results support that FRF overcome boundaries of clinical research and enables collaborations across institutes without violating privacy or legal regulations. Clinicians benefit from a vast collection of unbiased data aggregated from different geographic locations, demographics and other varying factors. They can build more generalizable models to make better clinical decisions, which will have relevance, especially for patients in rural areas and rare or geographically uncommon diseases, enabling personalized treatment. In combination with secure multi-party computation, federated learning has the power to revolutionize clinical practice by increasing the accuracy and robustness of healthcare AI and thus paving the way for precision medicine. AVAILABILITY AND IMPLEMENTATION: The implementation of the federated random forests can be found at https://featurecloud.ai/. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

MOTIVATION: The tremendous success of graphical neural networks (GNNs) already had a major impact on systems biology research. For example, GNNs are currently being used for drug target recognition in protein-drug interaction networks, as well as for cancer gene discovery and more. Important aspects whose practical relevance is often underestimated are comprehensibility, interpretability and explainability. RESULTS: In this work, we present a novel graph-based deep learning framework for disease subnetwork detection via explainable GNNs. Each patient is represented by the topology of a protein-protein interaction (PPI) network, and the nodes are enriched with multi-omics features from gene expression and DNA methylation. In addition, we propose a modification of the GNNexplainer that provides model-wide explanations for improved disease subnetwork detection. AVAILABILITY AND IMPLEMENTATION: The proposed methods and tools are implemented in the GNN-SubNet Python package, which we have made available on our GitHub for the international research community (https://github.com/pievos101/GNN-SubNet). SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

The increasing prevalence of digitised workflows in diagnostic pathology opens the door to life-saving applications of artificial intelligence (AI). Explainability is identified as a critical component for the safety, approval and acceptance of AI systems for clinical use. Despite the cross-disciplinary challenge of building explainable AI (xAI), very few application- and user-centric studies in this domain have been carried out. We conducted the first mixed-methods study of user interaction with samples of state-of-the-art AI explainability techniques for digital pathology. This study reveals challenging dilemmas faced by developers of xAI solutions for medicine and proposes empirically-backed principles for their safer and more effective design.

Kandinsky Figures and Kandinsky Patterns are mathematically describable, simple, self-contained hence controllable synthetic test data sets for the development, validation and training of visual tasks and explainability in artificial intelligence (AI). Whilst Kandinsky Patterns have these computationally manageable properties, they are at the same time easily distinguishable by human observers. Consequently, controlled patterns can be described by both humans and computers. We define a Kandinsky Pattern as a set of Kandinsky Figures, where for each figure an “infallible authority” defines that the figure belongs to the Kandinsky Pattern. With this simple principle we build training and validation data sets for testing explainability, interpretability and context learning. In this paper we describe the basic idea and some underlying principles of Kandinsky Patterns. We provide a Github repository and invite the international AI research community to a challenge to experiment with our Kandinsky Patterns. The goal is to help expand and advance the field of AI, and in particular to contribute to the increasingly important field of explainable AI.

Data quality is of paramount importance for the smooth functioning of modern data-driven AI applications with machine learning as a core technology. This is also true for medical AI , where malfunctions due to "dirty data" can have particularly dramatic harmful implications. Consequently, data cleansing is an important part in improving the usability of (Big) Data for medical AI systems. However, it should not be overlooked that data cleansing can also have negative effects on data quality if not performed carefully. This paper takes an interdisciplinary look at some of the technical and legal challenges of data cleansing against the background of European medical device law, with the key message that technical and legal aspects must always be considered together in such a sensitive context.

Abstract In this paper, a neural network is trained to perform simple arithmetic using images of concatenated handwritten digit pairs. A convolutional neural network was trained with images consisting of two side-by-side handwritten digits, where the image’s label is the summation of the two digits contained in the combined image. Crucially, the network was tested on permutation pairs that were not present during training in an effort to see if the network could learn the task of addition, as opposed to simply mapping images to labels. A dataset was generated for all possible permutation pairs of length 2 for the digits 0–9 using MNIST as a basis for the images, with one thousand samples generated for each permutation pair. For testing the network, samples generated from previously unseen permutation pairs were fed into the trained network, and its predictions measured. Results were encouraging, with the network achieving an accuracy of over 90% on some permutation train/test splits. This suggests that the network learned at first digit recognition, and subsequently the further task of addition based on the two recognised digits. As far as the authors are aware, no previous work has concentrated on learning a mathematical operation in this way. This paper is an attempt to demonstrate that a network can learn more than a direct mapping from image to label, but is learning to analyse two separate regions of an image and combining what was recognised to produce the final output label.

Abstract Nowadays, a vast amount of clinical data scattered across different sites on the Internet hinders users from finding helpful information for their well-being improvement. Besides, the overload of medical information (e.g., on drugs, medical tests, and treatment suggestions) have brought many difficulties to medical professionals in making patient-oriented decisions. These issues raise the need to apply recommender systems in the healthcare domain to help both, end-users and medical professionals, make more efficient and accurate health-related decisions. In this article, we provide a systematic overview of existing research on healthcare recommender systems. Different from existing related overview papers, our article provides insights into recommendation scenarios and recommendation approaches. Examples thereof are food recommendation, drug recommendation, health status prediction, healthcare service recommendation, and healthcare professional recommendation. Additionally, we develop working examples to give a deep understanding of recommendation algorithms. Finally, we discuss challenges concerning the development of healthcare recommender systems in the future.

We propose a novel classification according to aggregation functions of mixed behaviour by variability in ordinal sums of conjunctive and disjunctive functions. Consequently, domain experts are empowered to assign only the most important observations regarding the considered attributes. This has the advantage that the variability of the functions provides opportunities for machine learning to learn the best possible option from the data. Moreover, such a solution is comprehensible, reproducible and explainable-per-design to domain experts. In this paper, we discuss the proposed approach with examples and outline the research steps in interactive machine learning with a human-in-the-loop over aggregation functions. Although human experts are not always able to explain anything either, they are sometimes able to bring in experience, contextual understanding and implicit knowledge, which is desirable in certain machine learning tasks and can contribute to the robustness of algorithms. The obtained theoretical results in ordinal sums are discussed and illustrated on examples.

Histopathology and cytopathology developed for the microscopic examination of tissue samples a specific terminology to describe type and shape of objects and patterns in the composition of nuclei, cells, tissue and anatomical elements. We map such a terminology to Bertin's visual variables and propose three methods to describe “shape grammar” (1) With a formal, mathematical language, (2) with graphs and (3) by natural language descriptions. Finally, we propose practical applications of shape properties and shape grammar for explainability of AI algorithms in computational pathology.

Extracting hierarchical properties from Whole Slide Images automatically and expanding the possibilities of visualising data from digital pathology will not only drastically improve speed and accuracy of the pathologists' work but also simplify the necessary pre-processing steps for machine learning tool-chains. The introduced pipeline identifies and converts areas of interest into binary masks and finds groups of areas that share similar locations using k-Means. This grouping is evaluated by the Silhouette Score which serves as a measure of confidence for the separability of clusters. Found objects are compared using structural similarities and HU-Moments. These results are then stored as measures of similarities creating virtual groups of the most similar objects. Finally, the information on similarities combined with further structural parameters are visualised.

Training in the medical field depends on the teaching of theoretical and practical skills. The acquisition of theoretical basics can be usually achieved by studying various documents, while the training of practical skills requires more intensive discussion and a special setup, such as an appropriate laboratory environment or a mentor-mentee connection. Aside from the conditions for transferring and gaining practical skills, teaching methods mostly concentrate on the sharing of explicit knowledge that can be designated and transferred easily.Therefore, the presented work focuses on the tracking of how to approach the diagnosis in pathological examinations and the visualization of any interaction with the specimen by recording microscopical exploration of an experienced pathologist. Those interactions include the full navigation path through a specimen, the panning and magnification of areas of interest, the observation duration as well as the spoken comments. Each maneuver and annotation that is relevant for approaching the diagnosis is documented, processed and visually prepared for trainees/residents to be able to comprehend argumentation via the look through the eyes of an expert.The collected data serve as a base for various medical applications like learning management systems to support education in the medical field, as well as machine learning and artificial intelligence applications in the area of digital pathology.

Advancements in Artificial Intelligence (AI) and Machine Learning (ML) are enabling new diagnostic capabilities. In this paper we argue that the very first step before introducing AI/ML into diagnostic workflows is a deep understanding of how pathologists work. To contribute to a deeper understanding of the diagnostic process in histopathology, we developed a visualization concept, including: (a) the sequence of the views observed by the pathologist (Observation Path), (b) the sequence of the spoken comments and statements of the pathologist (Dictation Path), (c) the underlying knowledge and experience of the pathologist (Knowledge Path), (d) information about the current phase of the diagnostic process and (e) the current magnification factor of the microscope chosen by the pathologist. We implemented the proofof-concept prototype as HTML5/CSS3/JavaScript application.

The combination of ontologies with machine learning (ML) approaches is a hot topic and not yet extensively investigated but having great future potential. This is due to the general fact that both, ontologies and ML, constitute two indispensable technologies for domain-specific knowledge extraction, actively used in knowledge-based systems. Whilst the primary goal of both these approaches are the same, knowledge discovery, little is yet known about how the two sources of knowledge can be successfully integrated. The main data source in digital pathology are whole slide images. For the effective generation of sufficiently large and high-quality training data we need to extract in addition information from medical reports, containing non-standardized text. Since full annotation on pixel level would be impracticably expensive, a practical solution is in weakly-supervised ML. In the project described in this paper we used ontology-guided natural language processing (NLP) for term extraction and a decision tree built with an expert-curated classification system. This demonstrates the practical value of our solution to analyze and structure training data sets for ML and as a tool for the generation of biobank catalogues.

State of the art genomic methods produce an abundance of data which ultimately increases the quantity within and of data repositories. Thereby, open data is of much importance to boost scientific studies for combating disease. More and more often, there are several data sources available, offering diverse sample data. Particularly, interpretation of genomic data remains a challenge due to data size and differing quality. We present an approach for visualizing uncertainty of heterogeneous data sources on mutation rates in genomic data for pediatric cancer analysis. Visualization as method for knowledge discovery will be of great importance in order to put inhomogeneity of sample data into perspective.

Since pathology is supported by information technology new opportunities and questions have arisen. The digital age enables analyzing histopathological data with artificial intelligence methods to reveal further information and correlations. In this paper existing approaches to visualization of medical decision processes are presented as well as the relevance of explainability in decision making. The first step for implementing decision-paths in systems is to retrace an experienced pathologist's diagnosis finding process. Recording a route through a landscape composed of human tissue in terms of a roadbook is one possible approach to collect information on how diagnoses are found. Choosing the roadbook metaphor provides a simple schema, that holds basic directions enriched with metadata regarding landmarks on a rally - in the context of pathology such landmarks provide information on the decision finding process.

Recent advances in automatic machine learning (aML) allow solving problems without any human intervention. However, sometimes a human-in-the-loop can be beneficial in solving computationally hard problems. In this paper we provide new experimental insights on how we can improve computational intelligence by complementing it with human intelligence in an interactive machine learning approach (iML). For this purpose, we used the Ant Colony Optimization (ACO) framework, because this fosters multi-agent approaches with human agents in the loop. We propose unification between the human intelligence and interaction skills and the computational power of an artificial system. The ACO framework is used on a case study solving the Traveling Salesman Problem, because of its many practical implications, e.g. in the medical domain. We used ACO due to the fact that it is one of the best algorithms used in many applied intelligence problems. For the evaluation we used gamification, i.e. we implemented a snake-like game called Traveling Snakesman with the MAX–MIN Ant System (MMAS) in the background. We extended the MMAS–Algorithm in a way, that the human can directly interact and influence the ants. This is done by “traveling” with the snake across the graph. Each time the human travels over an ant, the current pheromone value of the edge is multiplied by 5. This manipulation has an impact on the ant’s behavior (the probability that this edge is taken by the ant increases). The results show that the humans performing one tour through the graphs have a significant impact on the shortest path found by the MMAS. Consequently, our experiment demonstrates that in our case human intelligence can positively influence machine intelligence. To the best of our knowledge this is the first study of this kind.

MOTIVATION: Image augmentation is a frequently used technique in computer vision and has been seeing increased interest since the popularity of deep learning. Its usefulness is becoming more and more recognized due to deep neural networks requiring larger amounts of data to train, and because in certain fields, such as biomedical imaging, large amounts of labelled data are difficult to come by or expensive to produce. In biomedical imaging, features specific to this domain need to be addressed. RESULTS: Here we present the Augmentor software package for image augmentation. It provides a stochastic, pipeline-based approach to image augmentation with a number of features that are relevant to biomedical imaging, such as z-stack augmentation and randomized elastic distortions. The software has been designed to be highly extensible meaning an operation that might be specific to a highly specialized task can easily be added to the library, even at runtime. Although it has been designed as a general software library, it has features that are particularly relevant to biomedical imaging and the techniques required for this domain. AVAILABILITY AND IMPLEMENTATION: Augmentor is a Python package made available under the terms of the MIT licence. Source code can be found on GitHub under https://github.com/mdbloice/Augmentor and installation is via the pip package manager (A Julia version of the package, developed in parallel by Christof Stocker, is also available under https://github.com/Evizero/Augmentor.jl).

BACKGROUND: Understanding living systems is crucial for curing diseases. To achieve this task we have to understand biological networks based on protein-protein interactions. Bioinformatics has come up with a great amount of databases and tools that support analysts in exploring protein-protein interactions on an integrated level for knowledge discovery. They provide predictions and correlations, indicate possibilities for future experimental research and fill the gaps to complete the picture of biochemical processes. There are numerous and huge databases of protein-protein interactions used to gain insights into answering some of the many questions of systems biology. Many computational resources integrate interaction data with additional information on molecular background. However, the vast number of diverse Bioinformatics resources poses an obstacle to the goal of understanding. We present a survey of databases that enable the visual analysis of protein networks. RESULTS: We selected M=10 out of N=53 resources supporting visualization, and we tested against the following set of criteria: interoperability, data integration, quantity of possible interactions, data visualization quality and data coverage. The study reveals differences in usability, visualization features and quality as well as the quantity of interactions. StringDB is the recommended first choice. CPDB presents a comprehensive dataset and IntAct lets the user change the network layout. A comprehensive comparison table is available via web. The supplementary table can be accessed on http://tinyurl.com/PPI-DB-Comparison-2015. CONCLUSIONS: Only some web resources featuring graph visualization can be successfully applied to interactive visual analysis of protein-protein interaction. Study results underline the necessity for further enhancements of visualization integration in biochemical analysis tools. Identified challenges are data comprehensiveness, confidence, interactive feature and visualization maturing.

Physiological data in its different dimensions—bioelectrical, biomechanical, biochemical, or biophysical—and collected through existing sensors or specialized biomedical devices, image capture, or other sources is pushing the boundaries of physiological computing for human-computer interaction (HCI). Although physiological computing shows the potential to enhance the way in which people interact with digital content, systems remain challenging to design and build. The aim of this special issue is to present outstanding work related to use of physiological data in HCI, setting additional bases for next-generation computer interfaces and interaction experiences. Topics covered in this issue include methods and methodologies, human factors, the use of devices, and applications for supporting the development of emerging interfaces.

Cancer is a complex disease that has proven to be difficult to understand on the single-gene level. For this reason a functional elucidation needs to take interactions among genes on a systems-level into account. In this study, we infer a colon cancer network from a large-scale gene expression data set by using the method BC3Net. We provide a structural and a functional analysis of this network and also connect its molecular interaction structure with the chromosomal locations of the genes enabling the definition of cis- and trans-interactions. Furthermore, we investigate the interaction of genes that can be found in close neighborhoods on the chromosomes to gain insight into regulatory mechanisms. To our knowledge this is the first study analyzing the genome-scale colon cancer network.

Computers are incredibly fast, accurate, and stupid.Human beings are incredibly slow, inaccurate, and brilliant.Together they are powerful beyond imagination (Einstein never said that [1]).

BACKGROUND: Heart rate variability is the variation of the time interval between consecutive heartbeats. Entropy is a commonly used tool to describe the regularity of data sets. Entropy functions are defined using multiple parameters, the selection of which is controversial and depends on the intended purpose. This study describes the results of tests conducted to support parameter selection, towards the goal of enabling further biomarker discovery. METHODS: This study deals with approximate, sample, fuzzy, and fuzzy measure entropies. All data were obtained from PhysioNet, a free-access, on-line archive of physiological signals, and represent various medical conditions. Five tests were defined and conducted to examine the influence of: varying the threshold value r (as multiples of the sample standard deviation σ, or the entropy-maximizing rChon), the data length N, the weighting factors n for fuzzy and fuzzy measure entropies, and the thresholds rF and rL for fuzzy measure entropy. The results were tested for normality using Lilliefors' composite goodness-of-fit test. Consequently, the p-value was calculated with either a two sample t-test or a Wilcoxon rank sum test. RESULTS: The first test shows a cross-over of entropy values with regard to a change of r. Thus, a clear statement that a higher entropy corresponds to a high irregularity is not possible, but is rather an indicator of differences in regularity. N should be at least 200 data points for r = 0.2 σ and should even exceed a length of 1000 for r = rChon. The results for the weighting parameters n for the fuzzy membership function show different behavior when coupled with different r values, therefore the weighting parameters have been chosen independently for the different threshold values. The tests concerning rF and rL showed that there is no optimal choice, but r = rF = rL is reasonable with r = rChon or r = 0.2σ. CONCLUSIONS: Some of the tests showed a dependency of the test significance on the data at hand. Nevertheless, as the medical conditions are unknown beforehand, compromises had to be made. Optimal parameter combinations are suggested for the methods considered. Yet, due to the high number of potential parameter combinations, further investigations of entropy for heart rate variability data will be necessary.

BACKGROUND: This paper presents multilevel data glyphs optimized for the interactive knowledge discovery and visualization of large biomedical data sets. Data glyphs are three- dimensional objects defined by multiple levels of geometric descriptions (levels of detail) combined with a mapping of data attributes to graphical elements and methods, which specify their spatial position. METHODS: In the data mapping phase, which is done by a biomedical expert, meta information about the data attributes (scale, number of distinct values) are compared with the visual capabilities of the graphical elements in order to give a feedback to the user about the correctness of the variable mapping. The spatial arrangement of glyphs is done in a dimetric view, which leads to high data density, a simplified 3D navigation and avoids perspective distortion. RESULTS: We show the usage of data glyphs in the disease analyser a visual analytics application for personalized medicine and provide an outlook to a biomedical web visualization scenario. CONCLUSIONS: Data glyphs can be successfully applied in the disease analyser for the analysis of big medical data sets. Especially the automatic validation of the data mapping, selection of subgroups within histograms and the visual comparison of the value distributions were seen by experts as an important functionality.

BACKGROUND: Antibiotics are the widely prescribed drugs for children and most likely to be related with adverse reactions. Record on adverse reactions and allergies from antibiotics considerably affect the prescription choices. We consider this a biomedical decision-making problem and explore hidden knowledge in survey results on data extracted from a big data pool of health records of children, from the Health Center of Osijek, Eastern Croatia. RESULTS: We applied and evaluated a k-means algorithm to the dataset to generate some clusters which have similar features. Our results highlight that some type of antibiotics form different clusters, which insight is most helpful for the clinician to support better decision-making. CONCLUSIONS: Medical professionals can investigate the clusters which our study revealed, thus gaining useful knowledge and insight into this data for their clinical studies.

BACKGROUND: Professionals in the biomedical domain are confronted with an increasing mass of data. Developing methods to assist professional end users in the field of Knowledge Discovery to identify, extract, visualize and understand useful information from these huge amounts of data is a huge challenge. However, there are so many diverse methods and methodologies available, that for biomedical researchers who are inexperienced in the use of even relatively popular knowledge discovery methods, it can be very difficult to select the most appropriate method for their particular research problem. RESULTS: A web application, called KNODWAT (KNOwledge Discovery With Advanced Techniques) has been developed, using Java on Spring framework 3.1. and following a user-centered approach. The software runs on Java 1.6 and above and requires a web server such as Apache Tomcat and a database server such as the MySQL Server. For frontend functionality and styling, Twitter Bootstrap was used as well as jQuery for interactive user interface operations. CONCLUSIONS: The framework presented is user-centric, highly extensible and flexible. Since it enables methods for testing using existing data to assess suitability and performance, it is especially suitable for inexperienced biomedical researchers, new to the field of knowledge discovery and data mining. For testing purposes two algorithms, CART and C4.5 were implemented using the WEKA data mining framework.

In the so-called 'New Culture for Assessment' assessment has become a tool for Learning. Assessment is no more considered to be isolated from the learning process and provided as embedded assessment forms. Nevertheless, students have more responsibility in the learning process in general and in assessment activities in particular. They become more engaged in: developing assessment criteria, participating in self, peer-assessments, reflecting on their own learning, monitoring their performance, and utilizing feedback to adapt their knowledge, skills, and behavior. Consequently, assessment tools have emerged from being stand-alone represented by monolithic systems through modular assessment tools to more flexible and interoperable generation by adopting the service-oriented architecture and modern learning specifications and standards. The new generation holds great promise when it comes to having interoperable learning services and tools within more personalized and adaptive e-learning platforms. In this paper, integrated automated assessment forms provided through flexible and SOA-based tools are discussed. Moreover, it presents a show case of how these forms have been integrated with a Complex Learning Resource (CLR) and used for self-directed learning. The results of the study show, that the developed tool for self-directed learning supports students in their learning process.

The MEDLINE database (Medical Literature Analysis and Retrieval System Online) contains an enormously increasing volume of biomedical articles. There is urgent need for techniques which enable the discovery, the extraction, the integration and the use of hidden knowledge in those articles. Text mining aims at developing technologies to help cope with the interpretation of these large volumes of publications. Co-occurrence analysis is a technique applied in text mining and the methodologies and statistical models are used to evaluate the significance of the relationship between entities such as disease names, drug names, and keywords in titles, abstracts or even entire publications. In this paper we present a method and an evaluation on knowledge discovery of disease-disease relationships for rheumatic diseases. This has huge medical relevance, since rheumatic diseases affect hundreds of millions of people worldwide and lead to substantial loss of functioning and mobility. In this study, we interviewed medical experts and searched the ACR (American College of Rheumatology) web site in order to select the most observed rheumatic diseases to explore disease-disease relationships. We used a web based text-mining tool to find disease names and their co-occurrence frequencies in MEDLINE articles for each disease. After finding disease names and frequencies, we normalized the names by interviewing medical experts and by utilizing biomedical resources. Frequencies are normally a good indicator of the relevance of a concept but they tend to overestimate the importance of common concepts. We also used Pointwise Mutual Information (PMI) measure to discover the strength of a relationship. PMI provides an indication of how more often the query and concept co-occur than expected by change. After finding PMI values for each disease, we ranked these values and frequencies together. The results reveal hidden knowledge in articles regarding rheumatic diseases indexed by MEDLINE, thereby exposing relationships that can provide important additional information for medical experts and researchers for medical decision-making.

Abstract In this paper, we present three experimental proof‐of‐concepts: first, we demonstrate a ubiquitous computing framework (UCF), which is a network of interacting technologies that support humans ubiquitously. We then present practical work based on this UCF framework: TalkingPoints, which was originally developed for use at trading fairs in order to identify each participant and companyviatransponder and provide specific information during and after use. Finally, we propose GARFID, a concept for using advanced technologies for teaching young children. The main outcome of this research is that the concept of UCF raises a lot of possibilities, which can bring value and benefits for end‐users. When one follows the working‐is‐learning paradigm, it can be seen that the implementation of this type of technology can support life long learning (LLL), thereby providing evidence that technology can benefit everybody and make life easier. Copyright © 2008 John Wiley & Sons, Ltd.

The vision of ambient assisted living (AAL) is to provide technologies for supporting these people in their daily lives, allowing them to stay longer within their own home aiming at living independent and self-determined. We believe that intelligent user interfaces (IUI) can play an important role here. Consequently, this workshop addresses the questions related to the role, application, advantages (and also disadvantages) of IUI in the context of AAL. It aims at discussing on challenges, solutions and approaches related to these issues especially for elderly and impaired users.

The success of extreme programming (XP) is based, among other things, on an optimal communication in teams of 6-12 persons, simplicity, frequent releases and a reaction to changing demands. Most of all, the customer is integrated into the development process, with constant feedback. This is very similar to usability engineering (UE) which follows a spiral four phase procedure model (analysis, draft, development, test) and a three step (paper mock-up, prototype, final product) production model. In comparison, these phases are extremely shortened in XP; also the ideal team size in UE user-centered development is 4-6 people, including the end-user. The two development approaches have different goals but, at the same time, employ similar methods to achieve them. It seems obvious that there must be synergy in combining them. The authors present ideas in how to combine them in an even more powerful development method called extreme usability (XU). The most important issue of this paper is that the authors have embedded their ideas into software engineering education.

An enhanced version of metamorphic testing, namely n-iterative metamorphic testing, is proposed to systematically exploit more information out of metamorphic tests by applying metamorphic relations in a chain style. A contrastive case study, conducted within an integrated testing environment MTest, shows that n-iterative metamorphic testing exceeds metamorphic testing and special case testing in terms of their fault detection capabilities. Another advantage of n-iterative metamorphic testing is its high efficiency in test case generation.