Automatic radiology report generation is pivotal in reducing the workload of radiologists, while simultaneously improving diagnostic accuracy and operational efficiency. Current methods face significant challenges, including the effective alignment of medical visual features with textual features and the mitigation of data bias. In this paper, we propose a method for radiology report generation that utilizes a Cross-modal Enhancement and Alignment Adapter (CmEAA) to connect a vision encoder with a frozen large language model. Specifically, we introduce two novel modules within CmEAA: Cross-modal Feature Enhancement (CFE) and Neural Mutual Information Aligner (NMIA). CFE extracts observation-related contextual features to enhance the visual features of lesions and abnormal regions in radiology images through a cross-modal enhancement transformer. NMIA maximizes neural mutual information between visual and textual representations within a low-dimensional alignment embedding space during training and provides potential global alignment visual representations during inference. Additionally, a weights generator is designed to enable the dynamic adaptation of cross-modal enhanced features and vanilla visual features. Experimental results on two prevailing datasets, namely, IU X-Ray and MIMIC-CXR, demonstrate that the proposed model outperforms previous state-of-the-art methods.

Salient and small lesions (e.g., microaneurysms on fundus) both play significant roles in real-world disease diagnosis under medical image examinations. Although deep neural networks (DNNs) have achieved promising medical image classification performance, they often have limitations in capturing both salient and small lesion information, restricting performance improvement in imbalanced medical image classification. Recently, with the advent of DNN-based style transfer in medical image generation, the roles of clinical styles have attracted great interest, as they are crucial indicators of lesions. Motivated by this observation, we propose a novel Adaptive Dual-Axis Style-based Recalibration (ADSR) module, leveraging the potential of clinical styles to guide DNNs in effectively learning salient and small lesion information from a dual-axis perspective. ADSR first emphasizes salient lesion information via global style-based adaptation, then captures small lesion information with pixel-wise style-based fusion. We construct an ADSR-Net for imbalanced medical image classification by stacking multiple ADSR modules. Additionally, DNNs typically adopt cross-entropy loss for parameter optimization, which ignores the impacts of class-wise predicted probability distributions. To address this, we introduce a new Class-wise Statistics Loss (CWS) combined with CE to further boost imbalanced medical image classification results. Extensive experiments on five imbalanced medical image datasets demonstrate not only the superiority of ADSR-Net and CWS over state-of-the-art (SOTA) methods but also their improved confidence calibration results. For example, ADSR-Net with the proposed loss significantly outperforms CABNet50 by 21.39% and 27.82% in F1 and B-ACC while reducing 3.31% and 4.57% in ECE and BS on ISIC2018.

BACKGROUND: Drug-drug interactions (DDIs) prediction is vital for pharmacology and clinical application to avoid adverse drug reactions on patients. It is challenging because DDIs are related to multiple factors, such as genes, drug molecular structure, diseases, biological processes, side effects, etc. It is a crucial technology for Knowledge graph to present multi-relation among entities. Recently some existing graph-based computation models have been proposed for DDIs prediction and get good performance. However, there are still some challenges in the knowledge graph representation, which can extract rich latent features from drug knowledge graph (KG). RESULTS: In this work, we propose a novel multi-view feature representation and fusion (MuFRF) architecture to realize DDIs prediction. It consists of two views of feature representation and a multi-level latent feature fusion. For the feature representation from the graph view and KG view, we use graph isomorphism network to map drug molecular structures and use RotatE to implement the vector representation on bio-medical knowledge graph, respectively. We design concatenate-level and scalar-level strategies in the multi-level latent feature fusion to capture latent features from drug molecular structure information and semantic features from bio-medical KG. And the multi-head attention mechanism achieves the optimization of features on binary and multi-class classification tasks. We evaluate our proposed method based on two open datasets in the experiments. Experiments indicate that MuFRF outperforms the classic and state-of-the-art models. CONCLUSIONS: Our proposed model can fully exploit and integrate the latent feature from the drug molecular structure graph (graph view) and rich bio-medical knowledge graph (KG view). We find that a multi-view feature representation and fusion model can accurately predict DDIs. It may contribute to providing with some guidance for research and validation for discovering novel DDIs.

Clinical multivariate time series can be used to predict clinical outcomes and in turn support clinical decision. Due to various reasons such as irregular sampling, clinical time series typically contain many missing values, and some missing values contain a large amount of information for clinical prediction. It is important to impute missing values as accurately as possible prior to clinical prediction. However, the missing rate of different variables in clinical time series varies greatly. In addition, clinical time data also has strong burst, which increases the difficulty of missing value imputation. In this paper, we propose a new method for missing value imputation in multivariate time series data, using a trainable weighted decay mechanism to improve the traditional Bi-LSTM model to capture temporal correlations, impute missing values based on bidirectional recurrent neural network and perform clinical outcomes predict. We take the missing rate, cumulative missing rate and burstiness of each variable as input to learn the features of different variables separately and improve the imputation effect of missing values. Experiments on two real-world clinical datasets show that our model achieves optimal performance in both imputation effect and prediction accuracy.

Clinically, physicians collect the benchmark medical data to establish archives for a stroke patient and then add the follow up data regularly. It has great significance on prognosis prediction for stroke patients. In this paper, we present an interpretable deep learning model to predict the one-year mortality risk on stroke. We design sub-modules to reconstruct features from original clinical data that highlight the dissimilarity and temporality of different variables. The model consists of Bidirectional Long Short-Term Memory (Bi-LSTM), in which a novel correlation attention module is proposed that takes the correlation of variables into consideration. In experiments, datasets are collected clinically from the department of neurology in a local AAA hospital. It consists of 2,275 stroke patients hospitalized in the department of neurology from 2014 to 2016. Our model achieves a precision of 0.9414, a recall of 0.9502 and an F1-score of 0.9415. In addition, we provide the analysis of the interpretability by visualizations with reference to clinical professional guidelines.

In recent years, thyroid nodule is being one of the most common nodular lesions. Ultrasonography is widely used in the clinical diagnosis of thyroid nodules. With the development of artificial intelligence, there emerge great progress in medical image diagnosis. Clinically, physician diagnose malignant or benign by many pathological features. In this work, we propose an ensemble architecture to resolve multi-label problem, which integrate three methods to extract features on thyroid nodule for ultrasound images. They are EfficientNet, feature engineering and feature pyramid network. We consider five kinds of pathological features quantified by Thyroid Imaging Report and Data System (TI-RADS). In the experiments, we use two datasets. One includes 587 original ultrasound images on thyroid nodules collected from the local health physical center of a 3A hospital. The other is a public dataset in the MICCAI 2020 competition. It contains 3644 ultrasound images of thyroid images. We use receiver operating characteristic curve (ROC) to evaluate the model, the area under curve (AUC) on every kind of features. The experimental results show that the proposed method can effectively assist doctors in diagnosing ultrasound images of thyroid nodules.

BACKGROUND: Multi-label classification of data remains to be a challenging problem. Because of the complexity of the data, it is sometimes difficult to infer information about classes that are not mutually exclusive. For medical data, patients could have symptoms of multiple different diseases at the same time and it is important to develop tools that help to identify problems early. Intelligent health risk prediction models built with deep learning architectures offer a powerful tool for physicians to identify patterns in patient data that indicate risks associated with certain types of chronic diseases. RESULTS: Physical examination records of 110,300 anonymous patients were used to predict diabetes, hypertension, fatty liver, a combination of these three chronic diseases, and the absence of disease (8 classes in total). The dataset was split into training (90%) and testing (10%) sub-datasets. Ten-fold cross validation was used to evaluate prediction accuracy with metrics such as precision, recall, and F-score. Deep Learning (DL) architectures were compared with standard and state-of-the-art multi-label classification methods. Preliminary results suggest that Deep Neural Networks (DNN), a DL architecture, when applied to multi-label classification of chronic diseases, produced accuracy that was comparable to that of common methods such as Support Vector Machines. We have implemented DNNs to handle both problem transformation and algorithm adaption type multi-label methods and compare both to see which is preferable. CONCLUSIONS: Deep Learning architectures have the potential of inferring more information about the patterns of physical examination data than common classification methods. The advanced techniques of Deep Learning can be used to identify the significance of different features from physical examination data as well as to learn the contributions of each feature that impact a patient's risk for chronic diseases. However, accurate prediction of chronic disease risks remains a challenging problem that warrants further studies.

A Multi-Label Problem Transformation Joint Classification (MLPTJC) method is developed to solve the multi-label classification problem for the health and disease risk prediction based on physical examination records. We adopt a multi-class classification problem transformation method to transform the multi-label classification problem to a multi-class classification problem. Then We propose a Joint Decomposition Subset Classifier method to reduce the infrequent label sets to deal with the imbalance learning problem. Based on MLPTJC, existing cost-sensitive multi-class classification algorithms can be used to train the prediction models. We conduct some experiments to evaluate the performance of the MLPTJC method. The Support Vector Machine (SVM) and Random Forest (RF) algorithms are used for multi-class classification learning. We use the 10-fold cross-validation and metrics such as Average Accuracy, Precision, Recall and F-measure to evaluate the performance. The real physical examination records were employed, which include 62 examination items and 110, 300 anonymous patients. 8 types of diseases were predicted. The experimental results show that the MLPTJC method has better performance in terms of accuracy.

This paper considers a hybrid hierarchical P2P overlay network structure that consists of both super and normal peers. The media streaming architecture is built upon a tree-structured network of super peers and the tree construction process has a significant impact on the overall system performance. We build network cost models and formulate a specific type of problem to maximize the minimum node throughput in Tree Construction (max-minTC), which aims at optimizing the system's stream rate by constructing an efficient spanning tree among super peers. We consider two scenarios: (i) When the overlay network has an arbitrary topology, we prove max-minTC to be NP-complete by reducing from the Degree Constrained Spanning Tree problem and propose an efficient heuristic algorithm. The performance superiority of the proposed algorithm is justified by experimental results collected by a live media streaming system deployed in real networks and is also illustrated by extensive simulations performed on a large set of simulated networks of various sizes from small to large scales in comparison with other methods, (ii) When the topology of the overlay network is complete, we rigorously prove that the same heuristic algorithm yields an optimal solution.

The client-server architecture widely adopted on the Internet is not adequate to meet the ever-increasing user loads and bandwidth demands in live streaming systems especially for multimedia content delivery. Peer-to-peer (P2P) overlay networks provide excellent system scalability and high resource utilization, which make it an attractive solution to this problem. We consider a hybrid hierarchical P2P overlay network that consists of both super and normal peers to support live streaming applications. This architecture is built upon a tree-structured network of super peers, which organize normal peers into clusters. The tree construction process has a significant impact on the overall system performance. We formulate a specific type of problem, max-minTC, to maximize the minimum node throughput in tree construction, where the system's stream rate is optimized by constructing an efficient spanning tree among super peers. We present a decentralized approach where super peers run the same algorithm in parallel to derive a tree from an identical database describing the topology of the streaming system. This approach is able to quickly converge to a new tree upon the detection of any topological changes in super peers. The performance superiority of the proposed solution is illustrated by extensive simulations on a large set of simulated networks of various sizes from small to large scales in comparison with other methods.

Peer-to-peer (P2P) overlay networks provide a highly effective and scalable solution to live media streaming systems that require the collective use of massively distributed network resources. A P2P media streaming architecture is typically built completely or partially upon a tree-structured network topology and the process of tree construction has a significant impact on the overall system performance. We build network cost models and formulate a specific type of topology construction problem, Maximum Average Bandwidth Spanning Tree (MABST), which aims at optimizing the system's average stream rate. We prove that MABST is NP-complete by reducing from Hamiltonian Path problem and propose an efficient heuristic algorithm. The performance superiority of the proposed algorithm is justified by experimental results using a live media streaming system deployed in real networks and is also illustrated by an extensive set of simulations on simulated networks of various sizes in comparison with other methods based on a degree constraint or a greedy strategy.