Extracting hierarchical properties from Whole Slide Images automatically and expanding the possibilities of visualising data from digital pathology will not only drastically improve speed and accuracy of the pathologists' work but also simplify the necessary pre-processing steps for machine learning tool-chains. The introduced pipeline identifies and converts areas of interest into binary masks and finds groups of areas that share similar locations using k-Means. This grouping is evaluated by the Silhouette Score which serves as a measure of confidence for the separability of clusters. Found objects are compared using structural similarities and HU-Moments. These results are then stored as measures of similarities creating virtual groups of the most similar objects. Finally, the information on similarities combined with further structural parameters are visualised.

Training in the medical field depends on the teaching of theoretical and practical skills. The acquisition of theoretical basics can be usually achieved by studying various documents, while the training of practical skills requires more intensive discussion and a special setup, such as an appropriate laboratory environment or a mentor-mentee connection. Aside from the conditions for transferring and gaining practical skills, teaching methods mostly concentrate on the sharing of explicit knowledge that can be designated and transferred easily.Therefore, the presented work focuses on the tracking of how to approach the diagnosis in pathological examinations and the visualization of any interaction with the specimen by recording microscopical exploration of an experienced pathologist. Those interactions include the full navigation path through a specimen, the panning and magnification of areas of interest, the observation duration as well as the spoken comments. Each maneuver and annotation that is relevant for approaching the diagnosis is documented, processed and visually prepared for trainees/residents to be able to comprehend argumentation via the look through the eyes of an expert.The collected data serve as a base for various medical applications like learning management systems to support education in the medical field, as well as machine learning and artificial intelligence applications in the area of digital pathology.

Advancements in Artificial Intelligence (AI) and Machine Learning (ML) are enabling new diagnostic capabilities. In this paper we argue that the very first step before introducing AI/ML into diagnostic workflows is a deep understanding of how pathologists work. To contribute to a deeper understanding of the diagnostic process in histopathology, we developed a visualization concept, including: (a) the sequence of the views observed by the pathologist (Observation Path), (b) the sequence of the spoken comments and statements of the pathologist (Dictation Path), (c) the underlying knowledge and experience of the pathologist (Knowledge Path), (d) information about the current phase of the diagnostic process and (e) the current magnification factor of the microscope chosen by the pathologist. We implemented the proofof-concept prototype as HTML5/CSS3/JavaScript application.

The combination of ontologies with machine learning (ML) approaches is a hot topic and not yet extensively investigated but having great future potential. This is due to the general fact that both, ontologies and ML, constitute two indispensable technologies for domain-specific knowledge extraction, actively used in knowledge-based systems. Whilst the primary goal of both these approaches are the same, knowledge discovery, little is yet known about how the two sources of knowledge can be successfully integrated. The main data source in digital pathology are whole slide images. For the effective generation of sufficiently large and high-quality training data we need to extract in addition information from medical reports, containing non-standardized text. Since full annotation on pixel level would be impracticably expensive, a practical solution is in weakly-supervised ML. In the project described in this paper we used ontology-guided natural language processing (NLP) for term extraction and a decision tree built with an expert-curated classification system. This demonstrates the practical value of our solution to analyze and structure training data sets for ML and as a tool for the generation of biobank catalogues.

Since pathology is supported by information technology new opportunities and questions have arisen. The digital age enables analyzing histopathological data with artificial intelligence methods to reveal further information and correlations. In this paper existing approaches to visualization of medical decision processes are presented as well as the relevance of explainability in decision making. The first step for implementing decision-paths in systems is to retrace an experienced pathologist's diagnosis finding process. Recording a route through a landscape composed of human tissue in terms of a roadbook is one possible approach to collect information on how diagnoses are found. Choosing the roadbook metaphor provides a simple schema, that holds basic directions enriched with metadata regarding landmarks on a rally - in the context of pathology such landmarks provide information on the decision finding process.

Next-generation sequencing (NGS) has transformed genomic research by decreasing the cost of sequencing and increasing the throughput. Today the translation of NGS technology to (cancer) diagnostics and therapeutics is widely anticipated. In this paper we give an overview of design, user experience and usability (DUXU) requirements for NGS applications with a special focus on clinical and diagnostic settings and describe a data driven GUI and visualization framework for a NGS based clinical decision support system (DSS). In such a DSS it is hard to oversee the huge amount of information, especially if given only in textual form. To overcome this we developed a visual language for electronic health records (EHR) for navigation within a case and comparison of medical cases.

BACKGROUND: This paper presents multilevel data glyphs optimized for the interactive knowledge discovery and visualization of large biomedical data sets. Data glyphs are three- dimensional objects defined by multiple levels of geometric descriptions (levels of detail) combined with a mapping of data attributes to graphical elements and methods, which specify their spatial position. METHODS: In the data mapping phase, which is done by a biomedical expert, meta information about the data attributes (scale, number of distinct values) are compared with the visual capabilities of the graphical elements in order to give a feedback to the user about the correctness of the variable mapping. The spatial arrangement of glyphs is done in a dimetric view, which leads to high data density, a simplified 3D navigation and avoids perspective distortion. RESULTS: We show the usage of data glyphs in the disease analyser a visual analytics application for personalized medicine and provide an outlook to a biomedical web visualization scenario. CONCLUSIONS: Data glyphs can be successfully applied in the disease analyser for the analysis of big medical data sets. Especially the automatic validation of the data mapping, selection of subgroups within histograms and the visual comparison of the value distributions were seen by experts as an important functionality.

As a hub of information controlled by the patient, personal health records (PHR) collect information from the patient medical history including a wide variety of data sources as patient's observations, lab results, clinical findings and in the future maybe even personal genetic data and automatic recordings from monitoring devices. This development will on the one hand make health care more personalized and user controlled but on the other hand also overloads consumers with a huge amount of data. To address this issue we developed a framework for adaptive visual symbols (AVS). An AVS can adapt its appearance and level of detail during the communication process. Finally we demonstrate the AVS principle for the visualization of personal health records.

Researches in humanities and social sciences indicate that people using more visual information would be more creative and benefit from the power of the human mind to a higher degree. Therefore information should not only be recorded as static text. It is rather conceivable to construct new visual languages that are not based on letters but on icons. We developed a set of medical symbols that can be used in visual representations of medical records and that may also be used to build complex medical statements. Each of the basic symbols is available in 4 abstraction levels, which can be combined in a ‘visual sentence’. Complex visual representations are recognized more easily than simplified versions, while on the other hand the simple version can be perceived and memorized in a faster way than complex signs. To prove this hypothesis and to evaluate the perception of the symbols in a medical context we created a web-based evaluation tool covering two perception tests. With the help of the symbol library we developed a visual language for patient records and an interactive browser.

This paper presents a visual data mining approach using the combination of clinical data, pathways and gene-expression data. The visual exploration of medical data using pathways to navigate and filter the data allows a more systematic and efficient investigation of problems in modern life science. A multiplicity of hypothesis can be evaluated in the same period of time, enabling a much better exploitation of the data. We present a system for data preprocessing and automatic classification, a set of visualization views and finally the integration in the Caleydo visualization framework, which enables the ldquocouplingrdquo of genetic and a broad spectrum of clinical data. With the help of the Caleydo framework the medical expert can identify connections between genetic parameters, patient subgroups, and drug responses.

UNLABELLED: Understanding the relationships between pathways and the altered expression of their components in disease conditions can be addressed in a visual data analysis process. Caleydo uses novel visualization techniques to support life science experts in their analysis of gene expression data in the context of pathways and functions of individual genes. Pathways and gene expression visualizations are placed in a 3D scene where selected entities (i.e. genes) are visually connected. This allows Caleydo to seamlessly integrate interactive gene expression visualization with cross-database pathway exploration. AVAILABILITY: The Caleydo visualization framework is freely available on www.caleydo.org for non-commercial use. It runs on Windows and Linux and requires a 3D capable graphics card.

Biobanks are well-organized resources comprising biological samples and associated information that are accessible to scientific investigation. Across Europe, millions of samples with related data are held in different types of collections. While individual collections can be well organized and accessible, the resources are subject to fragmentation, insecurity of funding and incompleteness. To address these issues, a Biobanking and BioMolecular Resources Infrastructure (BBMRI) is to be developed across Europe, thereby implementing a European 'roadmap' for research infrastructures that was developed by a forum of EU member states and that has been received by the European Commission. In this review, we describe the work involved in preparing for the construction of BBMRI in a European and global context.

BACKGROUND: With the introduction of tissue microarrays (TMAs) researchers can investigate gene and protein expression in tissues on a high-throughput scale. TMAs generate a wealth of data calling for extended, high level data management. Enhanced data analysis and systematic data management are required for traceability and reproducibility of experiments and provision of results in a timely and reliable fashion. Robust and scalable applications have to be utilized, which allow secure data access, manipulation and evaluation for researchers from different laboratories. RESULTS: TAMEE (Tissue Array Management and Evaluation Environment) is a web-based database application for the management and analysis of data resulting from the production and application of TMAs. It facilitates storage of production and experimental parameters, of images generated throughout the TMA workflow, and of results from core evaluation. Database content consistency is achieved using structured classifications of parameters. This allows the extraction of high quality results for subsequent biologically-relevant data analyses. Tissue cores in the images of stained tissue sections are automatically located and extracted and can be evaluated using a set of predefined analysis algorithms. Additional evaluation algorithms can be easily integrated into the application via a plug-in interface. Downstream analysis of results is facilitated via a flexible query generator. CONCLUSION: We have developed an integrated system tailored to the specific needs of research projects using high density TMAs. It covers the complete workflow of TMA production, experimental use and subsequent analysis. The system is freely available for academic and non-profit institutions from http://genome.tugraz.at/Software/TAMEE.