Whole slide image (WSI) analysis is gaining prominence within the medical imaging field. However, previous methods often fall short of efficiently processing entire WSIs due to their gigapixel size. Inspired by recent developments in state space models, this paper introduces a new Pathology Mamba (PAM) for more accurate and robust WSI analysis. PAM includes three carefully designed components to tackle the challenges of enormous image size, the utilization of local and hierarchical information, and the mismatch between the feature distributions of training and testing during WSI analysis. Specifically, we design a Bi-directional Mamba Encoder to process the extensive patches present in WSIs effectively and efficiently, which can handle large-scale pathological images while achieving high performance and accuracy. To further harness the local information and inherent hierarchical structure of WSI, we introduce a novel Local-aware Scanning module, which employs a local-aware mechanism alongside hierarchical scanning to adeptly capture both the local information and the overarching structure within WSIs. Moreover, to alleviate the patch feature distribution misalignment between training and testing, we propose a Test-time Importance Resampling module to conduct testing patch resampling to ensure consistency of feature distribution between the training and testing phases, and thus enhance model prediction. Extensive evaluation on nine WSI datasets with cancer subtyping and survival prediction tasks demonstrates that PAM outperforms current state-of-the-art methods and also its enhanced capability in modeling discriminative areas within WSIs. The source code is available at https://github.com/HKU-MedAI/PAM.

Traditional server failure prediction methods predominantly rely on single-modality data such as system logs or system status curves. This reliance may lead to an incomplete understanding of system health and impending issues, proving inadequate for the complex and dynamic landscape of contemporary cloud computing environments. The potential of multimodal data to provide comprehensive insights is widely acknowledged, yet the lack of a holistic dataset and the challenges inherent in integrating features from both structured and unstructured data have impeded the exploration of multimodal-based server failure prediction. Addressing these challenges, this paper presents an industrial-scale, comprehensive dataset for server failure prediction, comprising nearly 80 types of structured and unstructured data sourced from real-world industrial cloud systems 1. Building on this resource, we introduce MISP, a model that leverages multimodal fusion techniques for server failure prediction. MISP transforms multimodal data into multi-dimensional sequences, extracts and encodes features both within and across the modalities, and ultimately computes the failure probability from the synthesized features. Experiments demonstrate that MISP significantly outperforms existing methods, enhancing prediction accuracy by approximately 25% over previous state-of-the-art approaches.

Whole slide image (WSI) analysis is gaining prominence within the medical imaging field. Recent advances in pathology foundation models have shown the potential to extract powerful feature representations from WSIs for downstream tasks. However, these foundation models are usually designed for general-purpose pathology image analysis and may not be optimal for specific downstream tasks or cancer types. In this work, we present Concept Anchor-guided Task-specific Feature Enhancement (CATE), an adaptable paradigm that can boost the expressivity and discriminativeness of pathology foundation models for specific downstream tasks. Based on a set of task-specific concepts derived from the pathology vision-language model with expert-designed prompts, we introduce two interconnected modules to dynamically calibrate the generic image features extracted by foundation models for certain tasks or cancer types. Specifically, we design a Concept-guided Information Bottleneck module to enhance task-relevant characteristics by maximizing the mutual information between image features and concept anchors while suppressing superfluous information. Moreover, a Concept-Feature Interference module is proposed to utilize the similarity between calibrated features and concept anchors to further generate discriminative task-specific features. The extensive experiments on public WSI datasets demonstrate that CATE significantly enhances the performance and generalizability of MIL models. Additionally, heatmap and umap visualization results also reveal the effectiveness and interpretability of CATE.

Parkinson’s Disease (PD) is a prevalent and progressive neurodegenerative condition. Previous research has primarily identified the hippocampus as a key affected brain region in PD, with or without dementia, and even in cases without cognitive impairment. However, most earlier studies have treated the hippocampus as a singular global structure, focusing mainly on its overall volume or surface area alterations. There is a notable gap in detailed research on local morphological changes in the hippocampus during the early stages of PD. Our study involved T1-weighted MRI scans of forty-eight early-stage PD patients and forty-eight age-and sex-matched healthy controls. Initially, we assessed the global volume and surface area of the bilateral hippocam-pus. Subsequently, we delved into the detailed morphometric changes at the subfield scale of the bilateral hippocampus using a novel surface-based morphometric approach. Our findings indicate no significant global differences, except in the surface area ratio between the left and right hippocampus. However, significant deformations, primarily atrophic regions, were observed at the subfield level, particularly in the CA1 subfield, followed by the CA2-CA3 and Subiculum subfields. In summary, our study reveals that the CA1 sub-field is most vulnerable in early-stage PD, with atrophy likely progressing from CA1 to other subfields. This study also underscores that global metrics, such as volume or surface area, lack the sensitivity to accurately detect hippocampal deformations in early-stage PD.

Gait recognition is widely used in diversified practical applications. Currently, the most prevalent approach is to recognize human gait from RGB images, owing to the progress of computer vision technologies. Nevertheless, the perception capability of RGB cameras deteriorates in rough circumstances, and visual surveillance may cause privacy invasion. Due to the robustness and non-invasive feature of millimeter wave (mmWave) radar, radar-based gait recognition has attracted increasing attention in recent years. In this research, we propose a Hierarchical Dynamic Network (HDNet) for gait recognition using mmWave radar. In order to explore more dynamic information, we propose point flow as a novel point clouds descriptor. We also devise a dynamic frame sampling module to promote the efficiency of computation without deteriorating performance noticeably. To prove the superiority of our methods, we perform extensive experiments on two public mmWave radar-based gait recognition datasets, and the results demonstrate that our model is superior to existing state-of-the-art methods.

Transducer is one of the mainstream frameworks for streaming speech recognition. There is a performance gap between the streaming and non-streaming transducer models due to limited context. To reduce this gap, an effective way is to ensure that their hidden and output distributions are consistent, which can be achieved by hierarchical knowledge distillation. However, it is difficult to ensure the distribution consistency simultaneously because the learning of the output distribution depends on the hidden one. In this paper, we propose an adaptive two-stage knowledge distillation method consisting of hidden layer learning and output layer learning. In the former stage, we learn hidden representation with full context by applying mean square error loss function. In the latter stage, we design a power transformation based adaptive smoothness method to learn stable output distribution. It achieved 19% relative reduction in word error rate, and a faster response for the first token compared with the original streaming model in LibriSpeech corpus.

Whole slide image (WSI) analysis has become increasingly important in the medical imaging community, enabling automated and objective diagnosis, prognosis, and therapeutic-response prediction. However, in clinical practice, the ever-evolving environment hamper the utility of WSI analysis models. In this paper, we propose the FIRST continual learning framework for WSI analysis, named ConSlide, to tackle the challenges of enormous image size, utilization of hierarchical structure, and catastrophic forgetting by progressive model updating on multiple sequential datasets. Our framework contains three key components. The Hierarchical Interaction Transformer (HIT) is proposed to model and utilize the hierarchical structural knowledge of WSI. The Breakup-Reorganize (BuRo) rehearsal method is developed for WSI data replay with efficient region storing buffer and WSI reorganizing operation. The asynchronous updating mechanism is devised to encourage the network to learn generic and specific knowledge respectively during the replay stage, based on a nested cross-scale similarity learning (CSSL) module. We evaluated the proposed ConSlide on four public WSI datasets from TCGA projects. It performs best over other state-of-the-art methods with a fair WSI-based continual learning setting and achieves a better trade-off of the overall performance and forgetting on previous tasks.

Deep learning based approaches have achieved great success on the automatic cardiac image segmentation task. However, the achieved segmentation performance remains limited due to the significant difference across image domains, which is referred to as domain shift. Unsupervised domain adaptation (UDA), as a promising method to mitigate this effect, trains a model to reduce the domain discrepancy between the source (with labels) and the target (without labels) domains in a common latent feature space. In this work, we propose a novel framework, named Partial Unbalanced Feature Transport (PUFT), for cross-modality cardiac image segmentation. Our model facilities UDA leveraging two Continuous Normalizing Flow-based Variational Auto-Encoders (CNF-VAE) and a Partial Unbalanced Optimal Transport (PUOT) strategy. Instead of directly using VAE for UDA in previous works where the latent features from both domains are approximated by a parameterized variational form, we introduce continuous normalizing flows (CNF) into the extended VAE to estimate the probabilistic posterior and alleviate the inference bias. To remove the remaining domain shift, PUOT exploits the label information in the source domain to constrain the OT plan and extracts structural information of both domains, which are often neglected in classical OT for UDA. We evaluate our proposed model on two cardiac datasets and an abdominal dataset. The experimental results demonstrate that PUFT achieves superior performance compared with state-of-the-art segmentation methods for most structural segmentation.

Positron Emission Tomography (PET) has been widely used in the early diagnosis and treatment monitoring of Alzheimer’s Disease (AD). As two radiotracers of neurodegeneration, [18F]Fluorodeoxyglucose ([18F]FDG) and [18F]Florbetapir ([18F]AV45) PET have been used to measure cerebral glucose metabolism and $\beta$-amyloid $(A\beta)$ deposition, respectively. The combination of different modality PET images, such as FDG PET and AV45 PET, can provide complementary information for clinical diagnosis and evaluation. However, compared to the actual and available FDG PET data, AV45 PET data is always deficient due to the institution-specific tracers. In this paper, we propose a lightweight Generative Adversarial Network (GAN)-based model, which is termed “dual perceptual loss based generative adversarial network (DPGAN) for fast 2. 5D-based cross-modality generation of AV45 PET from FDG PET. This model provides a potential supplementary solution to those clinical situations that only the FDG PET image is acquired, but the AV45 PET is missing. Our experimental results showed that the DPGAN outperformed recent CycleGAN and pGAN, given its stronger ability in capturing the $A \beta$ deposition patterns on the whole-brain scale. All qualitative and quantitative metrics demonstrated the strong similarity between the generated AV45 PET images using DPGAN and the original AV45 PET images.

The novel 2019 Coronavirus (COVID-19) infection has spread worldwide and is currently a major healthcare challenge around the world. Chest computed tomography (CT) and X-ray images have been well recognized to be two effective techniques for clinical COVID-19 disease diagnoses. Due to faster imaging time and considerably lower cost than CT, detecting COVID-19 in chest X-ray (CXR) images is preferred for efficient diagnosis, assessment, and treatment. However, considering the similarity between COVID-19 and pneumonia, CXR samples with deep features distributed near category boundaries are easily misclassified by the hyperplanes learned from limited training data. Moreover, most existing approaches for COVID-19 detection focus on the accuracy of prediction and overlook uncertainty estimation, which is particularly important when dealing with noisy datasets. To alleviate these concerns, we propose a novel deep network named RCoNetksfor robust COVID-19 detection which employs Deformable Mutual Information Maximization (DeIM), Mixed High-order Moment Feature (MHMF), and Multiexpert Uncertainty-aware Learning (MUL). With DeIM, the mutual information (MI) between input data and the corresponding latent representations can be well estimated and maximized to capture compact and disentangled representational characteristics. Meanwhile, MHMF can fully explore the benefits of using high-order statistics and extract discriminative features of complex distributions in medical imaging. Finally, MUL creates multiple parallel dropout networks for each CXR image to evaluate uncertainty and thus prevent performance degradation caused by the noise in the data. The experimental results show that RCoNetksachieves the state-of-the-art performance on an open-source COVIDx dataset of 15 134 original CXR images across several metrics. Crucially, our method is shown to be more effective than existing methods with the presence of noise in the data.

Major depressive disorder (MDD) is accompanied by abnormal changes in functional connectivities (FC) among brain regions. However, most studies estimated the pairwise connectivity without the thorough consideration of the influence of other regions and assumed that the brain functional connectivity was static, which may be insufficient for the accurate identification of the pathological mechanisms underlying MDD. The purpose of this study was to explore the pathological mechanisms of MDD based on dynamic FC taking the influence of other regions into account. We performed time-varying connectivity analysis on resting-state functional magnetic resonance imaging (rs-fMRI) of 58 MDD patients and 63 matched healthy controls. The dynamic functional connectivity matrices were constructed using a novel Multivariate Vector Regression-based Connectivity (MVRC) method, which could regress time series of all regions while estimating the pairwise association between two regions. Then we analyzed two commonly used dynamic characteristics in brain network analysis, including dynamic FC (dFC) variability and node flexibility. Both dFC variability and node flexibility of MDD patients showed significant decreases in frontal, temporal, occipital, and parietal gyrus. Notably, we found the reduced dynamics of regions in frontal, temporal, and parietal gyrus was strongly negatively associated with depression severity. Our results demonstrated that the decreased dFC variability and node flexibility based on dynamic MVRC (dMVRC) were pathological manifestations of MDD.