Adaptive treatment strategies (ATS) are sequential decision-making processes that enable personalized care by dynamically adjusting treatment decisions in response to evolving patient symptoms. While reinforcement learning (RL) offers a promising approach for optimizing ATS, its conventional online trial-and-error learning mechanism is not permissible in clinical settings due to risks of harm to patients. Offline RL tackles this limitation by learning policies exclusively from historical treatment data, but its performance is often constrained by data scarcity—a pervasive challenge in clinical domains. To overcome this, we propose Treatment Stitching (TreatStitch), a novel data augmentation framework that generates clinically valid treatment trajectories by intelligently stitching segments from existing treatment data. Specifically, TreatStitch identifies similar intermediate patient states across different trajectories and stitches their respective segments. Even when intermediate states are too dissimilar to stitch directly, TreatStitch leverages the Schrödinger bridge method to generate smooth and shortest possible bridging trajectories that connect dissimilar states. By augmenting these synthetic trajectories into the original dataset, offline RL can learn from a more diverse dataset, thereby improving its ability to optimize ATS. Extensive experiments across multiple treatment datasets demonstrate the effectiveness of TreatStitch in enhancing offline RL performance. Furthermore, we provide a theoretical justification showing that TreatStitch maintains clinical validity by avoiding out-of-distribution transitions.

The automatic generation of radiology reports has emerged as a promising solution to reduce a time-consuming task and accurately capture critical disease-relevant findings in X-ray images. Previous approaches for radiology report generation have shown impressive performance. However, there remains significant potential to improve accuracy by ensuring that retrieved reports contain disease-relevant findings similar to those in the X-ray images and by refining generated reports. In this study, we propose a Disease-aware image-text Alignment and self-correcting Re-alignment for Trustworthy radiology report generation (DART) framework. In the first stage, we generate initial reports based on image-to-text retrieval with disease-matching, embedding both images and texts in a shared embedding space through contrastive learning. This approach ensures the retrieval of reports with similar disease-relevant findings that closely align with the input X-ray images. In the second stage, we further enhance the initial reports by introducing a self-correction module that re-aligns them with the X-ray images. Our proposed framework achieves state-of-the-art results on two widely used benchmarks, surpassing previous approaches in both report generation and clinical efficacy metrics, thereby enhancing the trustworthiness of radiology reports.

While large language models (LLMs) have achieved remarkable performance across various natural language processing (NLP) tasks, LLMs exhibit a limited understanding of commonsense reasoning due to the necessity of implicit knowledge that is rarely expressed in text.Recently, retrieval-augmented language models (RALMs) have enhanced their commonsense reasoning ability by incorporating background knowledge from external corpora.However, previous RALMs overlook the implicit nature of commonsense knowledge, potentially leading to the retrieved documents not directly contain information needed to answer questions.In this paper, we propose Retrieval-augmented knowledge Connection, RECONNECT, which transforms indirectly relevant documents into a direct explanation to answer the given question.To this end, we extract relevant knowledge from various retrieved document subsets and aggregate them into a direct explanation.Experimental results show that RECONNECT outperforms state-of-the-art (SOTA) baselines, achieving improvements of +2.0% and +4.6% average accuracy on in-domain (ID) and outof-domain (OOD) benchmarks, respectively 1 .

Molecular discovery has attracted significant attention in scientific fields for its ability to generate novel molecules with desirable properties. Although numerous methods have been developed to tackle this problem, most rely on an online setting that requires repeated online evaluation of candidate molecules using the oracle. However, in real-world molecular discovery, the oracle is often represented by wet-lab experiments, making this online setting impractical due to the significant time and resource demands. To fill this gap, we propose the Molecular Stitching (MolStitch) framework, which utilizes a fixed offline dataset to explore and optimize molecules without the need for repeated oracle evaluations. Specifically, MolStitch leverages existing molecules from the offline dataset to generate novel `stitched molecules' that combine their desirable properties. These stitched molecules are then used as training samples to fine-tune the generative model using preference optimization techniques. Experimental results on various offline multi-objective molecular optimization problems validate the effectiveness of MolStitch. The source code is available online.

Glioblastoma (GBM) remains a brain tumor with extremely poor prognosis, necessitating precise survival prediction to guide personalized treatment planning. Each MRI modality highlights distinct biological features of GBM, while genetic information provides crucial context for understanding tumor and progression. This complementary information offers a more comprehensive understanding of GBM. However, existing survival prediction methods, using either statistical approaches or deep learning models, often fail to capture the intricate relationships between multimodal MRI data and genetic markers, causing significant challenges to effective integration. To address these challenges, we propose a novel framework that integrates multimodal MRI and genetic information through a tailored fusion approach reflecting the distinct biological characteristics of each modality. Our method integrates multimodal MRI data and genetic information through a modality-aware fusion approach, which preserves modality-specific features and adjusts feature representations to integrate genetic information. This design achieves superior performance by modeling modality-specific features and cross-modal interactions, while incorporating genetic information to refine the feature representation for survival prediction. As a result, this framework supports clinicians in making informed, personalized treatment decisions, ultimately enhancing patient outcomes.

Two-photon calcium imaging is a powerful technique for recording neuronal activities over extended periods. However, reliably capturing neuronal signals from the this data poses a significant challenge due to non-uniform neuropil distribution and densely packed neuronal populations. In this study, we leverage deep learning (DL) techniques to directly capture true neuronal signals from calcium imaging data, addressing these challenges effectively. Utilizing publicly available datasets, we demonstrate that our DL-based approach, which directly extracts neuronal signals from images, outperforms existing non-DL methods in accurately capturing neuronal signals. This highlights the significant potential of DL methods for unveiling neuronal activity hidden in calcium imaging data. Furthermore, we investigate the impact of calcium imaging data quality on the performance of DL models. While our approach demonstrates significant promise, ongoing improvements in the quality of calcium imaging data will further enhance DL techniques, leading to a deeper understanding of brain mechanisms.

Volumetric fluorescence microscopy is crucial for non-invasive three-dimension (3D) visualization of biological systems but faces challenges due to anisotropic blurring caused by the point spread function (PSF). Previous methods have struggled with adapting to the diverse PSFs and have not effectively addressed their overall impacts of PSF. We propose Isotropic Diffusion Posterior Sampling (IsotropicDPS), solving isotropic reconstruction as a blind inverse problem. Our method employs two specialized score-based diffusion models, each trained on high-resolution lateral images and a diverse set of blurring PSFs. This approach enables the joint estimation of both the clean axial images and the PSF through a conditional posterior sampling strategy with a parallel reverse diffusion process. Remarkably, IsotropicDPS achieves zero-shot reconstruction and PSF estimation without requiring axial images during training. We validated our method through experiments on synthetic and real data, demonstrating superior performance and adaptability to varying PSF scenarios compared to existing methods.

Transfer learning for motor imagery-based brain-computer interfaces (MI-BCIs) struggles with inter-subject variability, hindering its generalization to new users. This paper proposes an advanced implicit transfer learning framework, META-EEG, designed to overcome the challenge arising from inter-subject variability. By incorporating gradient-based meta-learning with an intermittent freezing strategy, META-EEG ensures efficient feature representation learning, providing a robust zero-calibration solution. A comparative analysis reveals that META-EEG significantly outperforms all the baseline methods and competing methods on three different public datasets. Moreover, we demonstrate the efficiency of the proposed model through a neurophysiological and feature-representational analysis. With its robustness and superior performance on challenging datasets, META-EEG provides an effective solution for calibration-free MI-EEG classification, facilitating broader usability.

The global prevalence of childhood and adolescent obesity is a major concern due to its association with chronic diseases and long-term health risks. Artificial intelligence technology has been identified as a potential solution to accurately predict obesity rates and provide personalized feedback to adolescents. This study highlights the importance of early identification and prevention of obesity-related health issues. To develop effective algorithms for the prediction of obesity rates and provide personalized feedback, factors such as height, weight, waist circumference, calorie intake, physical activity levels, and other relevant health information must be taken into account. Therefore, by collecting health datasets from 321 adolescents who participated in Would You Do It! application, we proposed an adolescent obesity prediction system that provides personalized predictions and assists individuals in making informed health decisions. Our proposed deep learning framework, DeepHealthNet, effectively trains the model using data augmentation techniques, even when daily health data are limited, resulting in improved prediction accuracy (acc: 0.8842). Additionally, the study revealed variations in the prediction of the obesity rate between boys (acc: 0.9320) and girls (acc: 0.9163), allowing the identification of disparities and the determination of the optimal time to provide feedback. Statistical analysis revealed that the performance of the proposed deep learning framework was more statistically significant (p 0.001) compared to the other general models. The proposed system has the potential to effectively address childhood and adolescent obesity.

Resting-state functional magnetic resonance imaging (rs-fMRI) has gained attention as a reliable technique for investigating the intrinsic function patterns of the brain. It facilitates the extraction of functional connectivity networks (FCNs) that capture synchronized activity patterns among regions of interest (ROIs). Analyzing FCNs enables the identification of distinctive connectivity patterns associated with mild cognitive impairment (MCI). For MCI diagnosis, various sparse representation techniques have been introduced, including statistical- and deep learning-based methods. However, these methods face limitations due to their reliance on supervised learning schemes, which restrict the exploration necessary for probing novel solutions. To overcome such limitation, prior work has incorporated reinforcement learning (RL) to dynamically select ROIs, but effective exploration remains challenging due to the vast search space during training. To tackle this issue, in this study, we propose an advanced RL-based framework that utilizes a divide-and-conquer approach to decompose the FCN construction task into smaller sub-problems in a subject-specific manner, enabling efficient exploration under each sub-problem condition. Additionally, we leverage the learned value function to determine the sparsity level of FCNs, considering individual characteristics of FCNs. We validate the effectiveness of our proposed framework by demonstrating its superior performance in MCI diagnosis on publicly available cohort datasets.

Major Depressive Disorder (MDD) imposes a substantial burden within the healthcare domain, impacting millions of individuals worldwide. Functional Magnetic Resonance Imaging (fMRI) has emerged as a promising tool for the objective diagnosis of MDD, enabling the investigation of functional connectivity patterns in the brain associated with this disorder. However, most existing methods focus on a single brain atlas, which limits their ability to capture the complex, multi-scale nature of functional brain networks. To address these limitations, we propose a novel multi-atlas fusion method that incorporates early and late fusion in a unified framework. Our method introduces the concept of the holistic Functional Connectivity Network (FCN), which captures both intra-atlas relationships within individual atlases and inter-regional relationships between atlases with different brain parcellation scales. This comprehensive representation enables the identification of potential disease-related patterns associated with MDD in the early stage of our framework. Moreover, by decoding the holistic FCN from various perspectives through multiple spectral Graph Convolutional Neural Networks and fusing their results with decision-level ensembles, we further improve the performance of MDD diagnosis. Our approach is easily implemented with minimal modifications to existing model structures and demonstrates a robust performance across different baseline models. Our method, evaluated on public resting-state fMRI datasets, surpasses the current multi-atlas fusion methods, enhancing the accuracy of MDD diagnosis. The proposed novel multi-atlas fusion framework provides a more reliable MDD diagnostic technique. Experimental results show our approach outperforms both single- and multi-atlas-based methods, demonstrating its effectiveness in advancing MDD diagnosis.

Resting-state functional magnetic resonance imaging (rs-fMRI) is a commonly used functional neuroimaging technique to investigate the functional brain networks. However, rs-fMRI data are often contaminated with noise and artifacts that adversely affect the results of rs-fMRI studies. Several machine/deep learning methods have achieved impressive performance to automatically regress the noise-related components decomposed from rs-fMRI data, which are expressed as the pairs of a spatial map and its associated time series. However, most of the previous methods individually analyze each modality of the noise-related components and simply aggregate the decision-level information (or knowledge) extracted from each modality to make a final decision. Moreover, these approaches consider only the limited modalities making it difficult to explore class-discriminative spectral information of noise-related components. To overcome these limitations, we propose a unified deep attentive spatio-spectral-temporal feature fusion framework. We first adopt a learnable wavelet transform module at the input-level of the framework to elaborately explore the spectral information in subsequent processes. We then construct a feature-level multi-modality fusion module to efficiently exchange the information from multi-modality inputs in the feature space. Finally, we design confidence-based voting strategies for decision-level fusion at the end of the framework to make a robust final decision. In our experiments, the proposed method achieved remarkable performance for noise-related component detection on various rs-fMRI datasets.

Major Depressive Disorder (MDD) is a pervasive disorder affecting millions of individuals, presenting a significant global health concern. Functional connectivity (FC) derived from resting-state functional Magnetic Resonance Imaging (rs-fMRI) serves as a crucial tool in revealing functional connectivity patterns associated with MDD, playing an essential role in precise diagnosis. However, the limited data availability of FC poses challenges for robust MDD diagnosis. To tackle this, some studies have employed Deep Neural Networks (DNN) architectures to construct Generative Adversarial Networks (GAN) for synthetic FC generation, but this tends to overlook the inherent topology characteristics of FC. To overcome this challenge, we propose a novel Graph Convolutional Networks (GCN)-based Conditional GAN with Class-Aware Discriminator (GC-GAN). GC-GAN utilizes GCN in both the generator and discriminator to capture intricate FC patterns among brain regions, and the class-aware discriminator ensures the diversity and quality of the generated synthetic FC. Additionally, we introduce a topology refinement technique to enhance MDD diagnosis performance by optimizing the topology using the augmented FC dataset. Our framework was evaluated on publicly available rs-fMRI datasets, and the results demonstrate that GC-GAN outperforms existing methods. This indicates the superior potential of GCN in capturing intricate topology characteristics and generating high-fidelity synthetic FC, thus contributing to a more robust MDD diagnosis.

Molecular property prediction has gained substantial attention due to its potential for various bio-chemical applications. Numerous attempts have been made to enhance the performance by combining multiple molecular representations (1D, 2D, and 3D). However, most prior works only merged a limited number of representations or tried to embed multiple representations through a single network without using representation-specific networks. Furthermore, the heterogeneous characteristics of each representation made the fusion more challenging. Addressing these challenges, we introduce the Fusion Transformer for Multiple Molecular Representations (FTMMR) framework. Our strategy employs three distinct representation-specific networks and integrates information from each network using a fusion transformer architecture to generate fused representations. Additionally, we use self-supervised learning methods to align heterogeneous representations and to effectively utilize the limited chemical data available. In particular, we adopt a combinatorial loss function to leverage the contrastive loss for all three representations. We evaluate the performance of FTMMR using seven benchmark datasets, demonstrating that our framework outperforms existing fusion and self-supervised methods.

In recent years, deep learning methods have shown promising capabilities for extracting informative and discriminative features from electroencephalography (EEG) data. However, several studies have reported that the feature selection process followed by feature extraction can be beneficial to achieve further performance improvement. Even though a recent work achieved promising results by using the single-agent reinforcement learning (RL)-based framework to select task-relevant features in the temporal domain, it still failed to consider other significant features in the spatial–spectral domain. To overcome such limitations, we propose a cooperative multiagent RL-based framework (MARS) that performs feature selection in both the spatial–spectral and temporal domains simultaneously for a motor imagery (MI)-EEG classification task. In this framework, we enable our RL agents to collaborate with each other as a team to solve a complex multiobjective feature selection problem. Furthermore, we adopt a counterfactual advantage function to overcome the free-rider problem, which is associated with the credit assignment issue in multiagent cases. To assess the MARS framework, we conduct extensive experiments with two public MI datasets under subject-dependent and subject-independent scenarios and we apply the MARS to different backbone networks. The experimental results demonstrate that our MARS outperforms other competing methods in terms of mean accuracy and achieves statistically significant improvements.

Brain-computer interfaces (BCIs) are a promising method for users to interact with machines using brain signals, primarily electroencephalography (EEG). Motor imagery (MI)-BCI, which decodes EEG signals induced by the user's imagination of moving body parts, has gained great attention due to its applicability in various fields such as robotics and rehabilitation. MI - EEG signals exhibit class-discriminative patterns such as event-related de/synchronization across spectral-spatio-temporal (SST) domains. Numerous studies adopted deep learning framework, especially convolutional neural network (CNN), for learning SST feature representations automatically in a data-driven manner. In particular, most of the CNN-based methods adopted temporal convolution to learn the spectral information, as it can act as a band-pass filter. In this paper, we propose SAT-Net, a SincNet-based attentive temporal convolutional network for motor imagery classification. The proposed method utilizes Sinc convolution from SincNet for explicit extraction of spectral information from the input EEG signals with high interpretability. Moreover, we adopt an attentive temporal convolutional network to effectively learn SST feature representations while making full use of temporal information. We evaluate our proposed SAT-Net on the public BCI Competition IV-2a dataset, comparing it not only to conventional CNN-based approaches but also to the state-of-the-art method. The experimental results, supported by statistical analysis, demonstrate that our approach outperforms the competing methods.

Currently, the expectations for autonomous vehicles (AVs) are increasing. However, it is expected to take at least a decade to develop a fully AV, where human intervention is completely unrequired. By then, human driving is required if necessary. Currently, when the AV hands over control to the driver, a safe driving environment can be created only if it is possible to determine whether the driver is in an abnormal state. Unfortunately, according to the sensory conflict theory, the risk of motion sickness (MS) is higher in AV than in ordinary vehicles. This is because neither passengers nor drivers can predict the movement path of the vehicle under AV, so there is more dissonance between vision and perception. Because the technology to remove MS when it occurs has not yet been developed, the best way to maintain the driver’s good condition is to quickly predict MS through the driver’s bio-signals and establish a system to prevent MS through advanced driver assistance systems. It is necessary to quickly predict early MS and provide feedback before it becomes severe. In this study, we collected dry electroencephalogram (EEG) data to predict MS in a real-world driving environment. For MS-based feature extraction, a normalized sample covariance matrix-based feature representation method was used, and they were classified using convolutional neural networks. As a result, we achieved 89.05% (±5.76) accuracy when averaging all four experimental sessions we conducted. We expect our proposed model to be a useful indicator for resolving MS issues in AV environments.

Detection of the pilots’ mental states is particularly critical because their abnormal mental states (AbSs) could cause catastrophic accidents. In this study, we presented the feasibility of classifying the various specific AbSs (namely, low fatigue, high fatigue, low workload, high workload, low distraction, and high distraction) by applying the deep learning method. To the best of our knowledge, this study is the first attempt to classify multiple AbSs of pilots. We proposed the hybrid deep neural networks with five convolutional blocks and two long short-term memory layers for decoding multiple AbSs. We designed the model to extract the informative features from electroencephalography signals. A total of ten pilots conducted the experiment in a simulated flight environment. Compared with five conventional models, our proposed model achieved the highest grand-average accuracy of 68.04$(\pm$5.26)% which is at least 6.55% higher than other conventional models for classifying seven mental states across all subjects. Our proposed model could distinguish and classify low and high levels for each status category and give appropriate feedback to the subjects. In addition, we found nine indicators that showed the statistically significant differences between two mental states (p$<$0.05). Hence, we believe that it will contribute significantly to autonomous driving or autopilot advances based on artificial intelligence technology in the future.

While convolutional neural network (CNN) has been demonstrating powerful ability to learn hierarchical spatial features from medical images, it is still difficult to apply it directly to resting-state functional MRI (rs-fMRI) and the derived brain functional networks (BFNs). We propose a novel CNN framework to simultaneously learn embedded features from BFNs for brain disease diagnosis. Since BFNs can be built by considering both static and dynamic functional connectivity (FC), we first decompose rs-fMRI into multiple static BFNs with modified independent component analysis. Then, the voxel-wise variability in dynamic FC is used to quantify BFN dynamics. A set of paired 3D images representing static/dynamic BFNs can be fed into 3D CNNs, from which we can hierarchically and simultaneously learn static/dynamic BFN features. As a result, the dynamic BFN features can complement static BFN features and, at the meantime, different BFNs can help each other toward a joint and better classification. We validate our method with a publicly accessible, large cohort of rs-fMRI dataset in early-stage mild cognitive impairment (eMCI) diagnosis, which is one of the most challenging problems to the clinicians. By comparing with a conventional method, our method shows significant diagnostic performance improvement by almost 10%. This result demonstrates the effectiveness of deep learning in preclinical Alzheimer's disease diagnosis, based on the complex and high-dimensional voxel-wise spatiotemporal patterns of the resting-state brain functional connectomics. The framework provides a new but intuitive way to fully exploit deeply embedded diagnostic features from rs-fMRI for a better-individualized diagnosis of various neurological diseases.