The Chatbot Assessment Reporting Tool (CHART) is a reporting guideline developed to provide reporting recommendations for studies evaluating the performance of generative artificial intelligence (AI)-driven chatbots when summarizing clinical evidence and providing health advice, referred to as Chatbot Health Advice (CHA) studies. CHART was developed in several phases after performing a comprehensive systematic review to identify variation in the conduct, reporting and methodology in CHA studies. Findings from the review were used to develop a draft checklist that was revised through an international, multidisciplinary modified asynchronous Delphi consensus process of 531 stakeholders, three synchronous panel consensus meetings of 48 stakeholders, and subsequent pilot testing of the checklist. CHART includes 12 items and 39 subitems to promote transparent and comprehensive reporting of CHA studies. These include Title (subitem 1a), Abstract/Summary (subitem 1b), Background (subitems 2ab), Model Identifiers (subitem 3ab), Model Details (subitems 4abc), Prompt Engineering (subitems 5ab), Query Strategy (subitems 6abcd), Performance Evaluation (subitems 7ab), Sample Size (subitem 8), Data Analysis (subitem 9a), Results (subitems 10abc), Discussion (subitems 11abc), Disclosures (subitem 12a), Funding (subitem 12b), Ethics (subitem 12c), Protocol (subitem 12d), and Data Availability (subitem 12e). The CHART checklist and corresponding methodological diagram were designed to support key stakeholders including clinicians, researchers, editors, peer reviewers, and readers in reporting, understanding, and interpreting the findings of CHA studies.

Explanations in interactive machine-learning systems facilitate debugging and improving prediction models. However, the effectiveness of various global model-centric and data-centric explanations in aiding domain experts to detect and resolve potential data issues for model improvement remains unexplored. This research investigates the influence of data-centric and model-centric global explanations in systems that support healthcare experts in optimising models through automated and manual data configurations. We conducted quantitative (n=70) and qualitative (n=30) studies with healthcare experts to explore the impact of different explanations on trust, understandability and model improvement. Our results reveal the insufficiency of global model-centric explanations for guiding users during data configuration. Although data-centric explanations enhanced understanding of post-configuration system changes, a hybrid fusion of both explanation types demonstrated the highest effectiveness. Based on our study results, we also present design implications for effective explanation-driven interactive machine-learning systems.

Explainable artificial intelligence is increasingly used in machine learning (ML) based decision-making systems in healthcare. However, little research has compared the utility of different explanation methods in guiding healthcare experts for patient care. Moreover, it is unclear how useful, understandable, actionable and trustworthy these methods are for healthcare experts, as they often require technical ML knowledge. This paper presents an explanation dashboard that predicts the risk of diabetes onset and explains those predictions with data-centric, feature-importance, and example-based explanations. We designed an interactive dashboard to assist healthcare experts, such as nurses and physicians, in monitoring the risk of diabetes onset and recommending measures to minimize risk. We conducted a qualitative study with 11 healthcare experts and a mixed-methods study with 45 healthcare experts and 51 diabetic patients to compare the different explanation methods in our dashboard in terms of understandability, usefulness, actionability, and trust. Results indicate that our participants preferred our representation of data-centric explanations that provide local explanations with a global overview over other methods. Therefore, this paper highlights the importance of visually directive data-centric explanation method for assisting healthcare experts to gain actionable insights from patient health records. Furthermore, we share our design implications for tailoring the visual representation of different explanation methods for healthcare experts.

Built on the success of the past five years, KDD DSHealth 2023 will further catalyze the development of links between academic and industrial data science groups. The workshop aims to stimulate discussion on strategic areas for development and to facilitate future cross-disciplinary collaborations. In accordance with the multi-year goal to continue fostering this community via timely topics, this year the workshop will focus on the applications and new development of generative models in healthcare, including the new development and application of LLMs. The workshop invites full papers, as well as work-in-progress on the application of data science in healthcare. The workshop will feature two invited talks from eminent speakers, spanning academia, industry, clinical researchers, and governmental regulatory bodies. In addition, we will invite community members to submit their research works and bring them for discussion. The summary gives a brief description of the half-day workshop to be held on August 7th, 2023.

INTRODUCTION: Depression is a global concern, with a significant number of people affected worldwide, particularly in low- and middle-income countries. The rising prevalence of depression emphasizes the importance of early detection and understanding the origins of such conditions. OBJECTIVE: This paper proposes a framework for detecting depression using a hybrid visualization approach that combines local and global interpretation. This approach aims to assist in model adaptation, provide insights into patient characteristics, and evaluate prediction model suitability in a different environment. METHODS: This study utilizes R programming language with the Caret, ggplot2, Plotly, and Dalex libraries for model training, visualization, and interpretation. Data from the NHANES repository was used for secondary data analysis. The NHANES repository is a comprehensive source for examining health and nutrition of individuals in the United States, and covers demographic, dietary, medication use, lifestyle choices, reproductive and mental health data. Penalized logistic regression models were built using NHANES 2015-2018 data, while NHANES 2019-March 2020 data was used for evaluation at the global-specific and local level interpretation. RESULTS: The prediction model that supports this framework achieved an average AUC score of 0.748 (95% CI: 0.743-0.752), with minimal variability in sensitivity and specificity. CONCLUSION: The built-in prediction model highlights chest pain, the ratio of family income to poverty, and smoking status as crucial features for predicting depressive states in both the original and local environments.

KDD DSHealth 2022, aims to build on the success of the past four years to further catalyze the development of links between academic and commercial data science groups and the rapidly developing translational medicine informatics community. The workshop will stimulate discussion as to strategic areas for development and will lead to future cross-disciplinary collaborations. In accordance with the multi-year goal to continue fostering this community as a series of KDD workshops via timely topics, this year the workshop will focus on the transparency and human-centered AI in healthcare. The workshop invites full papers, as well as work-in-progress on the application of data science in healthcare. The workshop will feature four invited talks from eminent speakers, spanning academia, industry, clinical researchers, and governmental regulatory bodies. In addition, selected papers will be invited to publish in a special issue of Journal of Healthcare Informatics Research. The summary gives a brief description of the full-day workshop to be held on August 14th, 2022.

KDD Health Day/DSHealth 2021, aims to build on the success of the past 3 years to further catalyze the development of links between academic and commercial data science groups and the rapidly developing translational medicine informatics community. The workshop will stimulate discussion as to strategic areas for development and will lead to future cross-disciplinary collaborations. In accordance with the multi-year goal to continue fostering this community as a series of KDD workshops via timely topics, this year the workshop will focus on the state of explainability and trustworthiness in healthcare. The workshop invites full papers, as well as work-in-progress on the application of data science in healthcare. The workshop will feature 8 invited talks from eminent speakers across academia, industry, clinical researchers, and governmental regulatory bodies. In addition, selected papers will be invited to publish in a special issue of Artificial Intelligence in Medicine journal. The summary gives a brief description of the full-day workshop to be held on August, 2021 virtually.

As the `third age' of human life becomes noticeably longer, the opportunity for elderly to obtain new skills reduces the tendency to consider this period of life as being disadvantaged. Hence, the fundamental aim of the project PRIMER-ICT was to educate older people in four participating countries (Slovenia, Ireland, UK and Austria) in Information and Communication Technologies (ICT) skills/practice by using an inter-generational and multisectoral approach empowering elderly to use ICT on everyday basis to improve their quality of live and to re-engage them in the society. To achieve this aim, students, primarily from health and ICT related fields have been recruited to become `trainers/teachers'. These students in turn have and will utilise acquired skills and knowledge to help train multipliers (community nurses, nurses in elderly homes, family members, volunteers from different sectors/ages, elderly), who in turn help training elderly.

UNLABELLED: Often, the most informative genes have to be selected from different gene sets and several computer gene ranking algorithms have been developed to cope with the problem. To help researchers decide which algorithm to use, we developed the analysis of gene ranking algorithms (AGRA) system that offers a novel technique for comparing ranked lists of genes. The most important feature of AGRA is that no previous knowledge of gene ranking algorithms is needed for their comparison. Using the text mining system finding-associated concepts with text analysis. AGRA defines what we call biomedical concept space (BCS) for each gene list and offers a comparison of the gene lists in six different BCS categories. The uploaded gene lists can be compared using two different methods. In the first method, the overlap between each pair of two gene lists of BCSs is calculated. The second method offers a text field where a specific biomedical concept can be entered. AGRA searches for this concept in each gene lists' BCS, highlights the rank of the concept and offers a visual representation of concepts ranked above and below it. AVAILABILITY AND IMPLEMENTATION: Available at http://agra.fzv.uni-mb.si/, implemented in Java and running on the Glassfish server. CONTACT: [email protected].

Predicting protein solubility has gained lots of intention in the recent years and several descriptors have been defined to describe proteins in these works. Therefore, different feature selection methods have been used for selecting the most important attributes. An empirical study, that aims to explain the relationship between the number of samples and stability of seven different feature selection techniques for protein datasets, is presented.

BACKGROUND: Reproducibility of results can have a significant impact on the acceptance of new technologies in gene expression analysis. With the recent introduction of the so-called next-generation sequencing (NGS) technology and established microarrays, one is able to choose between two completely different platforms for gene expression measurements. This study introduces a novel methodology for gene-ranking stability analysis that is applied to the evaluation of gene-ranking reproducibility on NGS and microarray data. RESULTS: The same data used in a well-known MicroArray Quality Control (MAQC) study was also used in this study to compare ranked lists of genes from MAQC samples A and B, obtained from Affymetrix HG-U133 Plus 2.0 and Roche 454 Genome Sequencer FLX platforms. An initial evaluation, where the percentage of overlapping genes was observed, demonstrates higher reproducibility on microarray data in 10 out of 11 gene-ranking methods. A gene set enrichment analysis shows similar enrichment of top gene sets when NGS is compared with microarrays on a pathway level. Our novel approach demonstrates high accuracy of decision trees when used for knowledge extraction from multiple bootstrapped gene set enrichment analysis runs. A comparison of the two approaches in sample preparation for high-throughput sequencing shows that alternating decision trees represent the optimal knowledge representation method in comparison with classical decision trees. CONCLUSIONS: Usual reproducibility measurements are mostly based on statistical techniques that offer very limited biological insights into the studied gene expression data sets. This paper introduces the meta-learning-based gene set enrichment analysis that can be used to complement the analysis of gene-ranking stability estimation techniques such as percentage of overlapping genes or classic gene set enrichment analysis. It is useful and practical when reproducibility of gene ranking results or different gene selection techniques is observed. The proposed method reveals very accurate descriptive models that capture the co-enrichment of gene sets which are differently enriched in the compared data sets.

Data preprocessing is an important step in preparation of DNA microarray data for further analysis. There is a significant amount of genes that do not influence the final classification. One of the reasons to eliminate such genes is the increasing computational complexity of supervised machine learning methods, especially in modern microarray experiments with hundreds of samples. This empirical study aims to measure differences in classification performance when different numbers of gene expression measurements are removed in a preprocessing phase. Simple unsupervised gene selection based on variance level of genes across all samples was used to remove genes with extremely low level of variance. This study shows the importance of combining unsupervised and supervised feature selection techniques along with classification algorithm. It was shown that gene expression values removed using simple unsupervised gene selection method are not of significant importance to the final results of supervised gene selection followed by classification.

This paper presents an overview of past papers published at the CBMS symposiums from a content analysis point of view. A simple, yet effective word counting using Harvard Psycho-Social dictionary was used to estimate different aspects of sentiment that can be present even in scientific papers. Using simple statistics we uncover some of the very interesting trends in the last five CBMS symposiums. Additional to pure statistics we used some of the most advanced classification techniques to see if there are any significant differences in psycho-social texture of the accepted papers. It was also shown that building machine learning models on this kind of data can result in some very interesting generalizations of the underlying data.

A lot of research has been done in the field of assembling classifiers in ensembles and on the other hand selecting the most appropriate single classifiers for a given problem which was solved by meta-learning techniques. This paper presents application of recently proposed ensemble of classifiers called Rotation Forest to Grading meta-learning scheme, where it is used as one of the base classifiers and meta-level classifier at the same time. Our proposed Grading variation is compared to four widely used classifiers on 14 datasets from the domain of gene expression classification problems. Experimental evaluations show that using Rotation Forest at meta-level most significantly impacts the accuracy of Grading scheme and confirms that it can be used for estimation of classifiers regions of strong and weak classification.

Many software reliability studies attempt to develop a model for predicting the faults of a software module because the application of good prediction models provides important information on significant metrics that should be observed in the early stages of implementation during software development. In this article we propose a new method inspired by a multi-agent based system that was initially used for classification and attribute selection in microarray analysis. Best classifying gene subset selection is a common problem in the field of bioinformatics. If we regard the software metrics measurement values of a software module as a genome of that module, and the real world dynamic characteristic of that module as its phenotype (i.e. failures as disease symptoms) we can borrow the established bioinformatics methods in the manner first to predict the module behavior and second to data mine the relations between metrics and failures.

Many different classification models and techniques have been employed on gene expression data. These computational methods are in rapid and continuous evolution and there is no clear consensus on which methods are best to cope with the complex microarray data analysis. Currently ensembles of classifiers are regarded as one of the best classification techniques as they can achieve excellent classification accuracy in comparison to single classifiers methods. One of their main drawbacks is their incomprehensibility. This paper addresses the important issue of the tradeoff between accuracy and comprehensibility when building ensembles and proposes a novel visual technique for interactive interpretation of the knowledge from the small ensembles consisting of only a few decision trees. This way we can achieve better accuracy compared to single classifier, but still maintain a certain level of comprehensibility in small ensembles. The results show that our small ensembles outperform the single classifiers and still retain comprehensibility. Our study also points out that in order to take advantage of our proposed method we need more effective small ensemble building techniques

Current software quality estimation models often involve the use of supervised learning methods for building a software fault prediction models. In such models, dependent variable usually represents a software quality measurement indicating the quality of a module by risk-basked class membership, or the number of faults. Independent variables include various software metrics as McCabe, Error Count, Halstead, Line of Code, etc... In this paper we present the use of advanced tool for data mining called Multimethod on the case of building software fault prediction model. Multimethod combines different aspects of supervised learning methods in dynamical environment and therefore can improve accuracy of generated prediction model. We demonstrate the use Multimethod tool on the real data from the Metrics Data Project Data (MDP) Repository. Our preliminary empirical results show promising potentials of this approach in predicting software quality in a software measurement and quality dataset.

In usual modern hospital architectural design we try to separate the clean and dirty corridor that should facilitate traffic flow of clean and dirty items. However that is not the case in the hospitals with older architectural design, where clean and dirty pathways flow through a single corridor systems. The most common solution which enables separation of clean and dirty pathways in such systems is introduction of time allocation for dirty and clean items transportation. Currently such schedules are fixed and do not allow quick changes in case of specific events. We propose a solution to the problem using multi-agent based system able to quickly adapt the schedule of transports.

In this paper, medical decision support systems for TNM (tumor characteristics, lymph node involvement, and distant metastatic lesions) classification aiming to divide cancer patients to low and high risk patients are presented. In addition, the system also explained the decision in the form of IF-THEN rules and in this manner performed data mining and new knowledge discovery. The case studies show that the system is robust and not dependent on the database size and the noise. The accuracy was almost 80% which is comparable with the accuracy of physicians and much better then obtained with more conventional discriminant analysis (62% and 67%).