Physiological signal extraction from video data is challenging in dynamic and occluded environments, requiring both accuracy and real-time performance. Existing methods struggle to balance accuracy with model efficiency, particularly under partial facial occlusion or redundant signals. We propose BRPDNet, a novel framework for efficient physiological signal extraction which includes a BioRegion Prompt module for adaptive convolution and a Hyper Distillation module to reduce signal redundancy, ensuring high accuracy and robustness, especially in dynamic and occluded environments. Additionally, the teacher-student network structure enhances the model's adaptability to occlusions and reduces computational complexity without relying on explicit segmentation. Experimental results show that BRPDNet outperforms state-of-the-art models in accuracy, robustness, and efficiency across multiple datasets. For instance, BRPDNet achieves an Mean Absolute Error (MAE) of 1.55 beats per minute (bpm) and a Pearson Correlation Coefficient (PCC) of 0.76 on PURE and UBFC-rPPG datasets with fewer parameters than existing models, ensuring efficient real-time performance.

Joint learning on diverse ultrasonic source scenarios presents a challenge in preserving stable gen-eralization due to the combination of heterogeneity of different sources and the inconsistency of joint learning features. Previous joint learning studies, which are not source-resilient frameworks, may not preserve stable generalization when trained on diverse source scenarios. Furthermore, the limited variations insingle-source data and the interference from ultrasound imaging, which are common in ultrasonic source scenarios, further decrease generalization. To address these problems, we pro posed a source-resilient joint learning framework consisting of three stages: 1) Source transforming, where our 1-to-N transformation unifies diverse source scenarios for source-resiliency. 2) Our feature enhancement modules model the source-resilient joint learning network, including a manifold-constraint normalization module (MCNM) for addressing heterogeneity by minimizing manifold-based loss, a task-consistent attention module (TCAM) shares the multi-scale features with self-attention to address inconsistency, and an adaptive feature-shifting module (AFSM) for feature-level augmentation to overcome single-source data.3) Our ultrasound-hybrid linear mapping (USmapping) cascades speckle randomization and mask-guiding Monge-Kantorovitch linear mapping to achieve ultrasonic style randomization for addressing the interference of ultrasonic data. Our framework was evaluated on eight ultrasound datasets from various scanners at multiple center sand surpassed previous comparable studies in both segmentation (DSCWAvgof 75.7%) and classification (AUROCWAvgof 68.8%) tasks. Our framework has the potential to serve as a general framework for enhancing the performance of joint learning under diverse ultrasonic source scenarios.

Accurate prediction of pathological subtypes on radiological images is one of the most important deep learning (DL) tasks for the appropriate selection of clinical treatment. It is challenging for conventional DL models to obtain sufficient pathological labels for training because of the heavy workload, invasive surgery, and knowledge requirements in pathological analysis. However, existing methods based on limited annotations, such as active learning (AL) and semi-supervised learning (SSL), have difficulty in capturing lesion's effective features because of the complicated semantic information of radiologic images. In this article, we introduce an efficient domain knowledge-guided semantic prediction framework that integrates domain knowledge-guided AL and SSL methods. This framework can effectively predict pathological subtypes on the basis of radiologic images with limited pathological annotations via three key modules: 1) the discriminative spatial-semantic feature extraction module captures the spatial-semantic features of lesions as semantic information that can better reflect the semantic relationship and effectively mitigate overfitting risk; 2) the explicit sign-guided anchor attention module measures the multimodal semantic distribution of samples under the guidance of clinical domain knowledge, thus selecting the most representative AL samples for pathological labeling; and 3) the implicit radiomics-guided dual-task entanglement module exploits the inherent constraint relationships between implicit radiomics features (IRFs) and pathological subtypes, facilitating the aggregation of unlabeled data. Experiments have been extensively conducted to evaluate our method in two clinical tasks: the pathological grading prediction in pancreatic neuroendocrine neoplasms (pNENs) and muscular invasiveness prediction in bladder cancer (BCa). The experimental results on both tasks demonstrate that the proposed method consistently outperforms the state-of-the-art approaches by a large margin.

Differential diagnosis of tumors is important for computer-aided diagnosis. In computer-aided diagnosis systems, expert knowledge of lesion segmentation masks is limited as it is only used during preprocessing or as supervision to guide feature extraction. To improve the utilization of lesion segmentation masks, this study proposes a simple and effective multitask learning network that improves medical image classification using self-predicted segmentation as guiding knowledge; we call this network RS$^{2}$-net. In RS$^{2}$-net, the predicted segmentation probability map obtained from the initial segmentation inference is added to the original image to form a new input, which is then reinput to the network for the final classification inference. We validated the proposed RS$^{2}$-net using three datasets: the pNENs-Grade dataset, which tested the prediction of pancreatic neuroendocrine neoplasm grading, and the HCC-MVI dataset, which tested the prediction of microvascular invasion of hepatocellular carcinoma, and ISIC 2017 public skin lesion dataset. The experimental results indicate that the proposed strategy of reusing self-predicted segmentation is effective, and RS$^{2}$-net outperforms other popular networks and existing state-of-the-art studies. Interpretive analytics based on feature visualization demonstrates that the improved classification performance of our reuse strategy is due to the semantic information that can be acquired in advance in a shallow network.

Diagnosis of tumors is an important direction of computer-aided diagnosis (CAD). The shape, size, and boundary of the tumor are widely-used diagnostic evidence, and the corresponding segmentation annotated by the radiologists is a vital expert knowledge, which can be used as supervision to guide feature extraction. Therefore, this study firstly introduces a multi-task learning (MTL) network integrating segmentation task for predicting grading of pancreatic neuroendocrine neoplasms (pNENs) and the microvascular invasion (MVI) of hepatocellular carcinoma (HCC). The proposed network combines a powerful split-attention-based encoder and a U-net decoder, and achieves the best performance in comparisons of other popular networks and previous studies. In addition, feature map visualization suggests that the reason for the improved classification performance may be that MTL makes the encoder pay more attention to lesions and extract more semantic information.

The identification of lesions is critical for the diagnostic evaluation of prostate cancer.68Ga-PSMA-11 PET/CT is a specific imaging for prostate cancer. However, this is extremely challenging considering the large number of lesions of varying size and uptake that may be distributed in various anatomical settings with different backgrounds throughout the body. In this paper, we propose a deep learning approach for automatic detection of whole-body prostate cancer lesions on PSMA imaging. We established and evaluated our model on the68Ga-PSMA-11 PET/CT image dataset of 107 patients with metastatic prostate cancer, and finally obtained Precision, Recall and F1-score of 82.9%, 100% and 90.6%, respectively, on the independent test set. Preliminary tests confirmed the potential of our method for disease detection on a systemic scale. Increasing the amount of training data can further improve the performance of the proposed deep learning method.

Approximately 13%–40% of patients with Crohn's disease (CD) show a primary loss of response to infliximab (IFX) therapy. Therefore, differentiating potential responders from primary nonresponders is clinically important. In this double-center study, we developed and validated a computed tomography enterography (CTE)-based radiomic signature (RS) for identification of CD patients at high risk of primary nonresponse (PNR) to IFX therapy, and demonstrated its incremental value to the clinical model. A total of 244 patients (training cohort, n = 119; test cohort 1, n = 51; test cohort 2, n = 74) were retrospectively recruited. Their clinical data and pretreatment CTE were retrieved and analyzed. All patients underwent IFX induction therapy. Reliability of clinical factors and radiomic-based features were assessed with the area under the receiver operating characteristic curve (AUC). In all, 1130 radiomic features were extracted from the whole inflamed gut in CTE images. In training cohort and test cohorts 1 and 2, the RS that discriminated PNR to IFX therapy yielded AUCs of 0.848, 0.789, and 0.789, respectively (all p < 0.05). By combining the clinical predictors (C-reactive protein, albumin, and body mass index) and RS, the radiomic-clinical model showed an increase in predicting performance (AUCs: 0.864, 0.794, and 0.791, respectively; all p < 0.05). Decision curve analysis and net reclassification improvement demonstrated the clinical usefulness of the radiomic-clinical model. In this study, the proposed RS showed potential as a clinical aid for the accurate identification of CD patients at high risk of PNR to IFX therapy before treatment. A combination of the RS and existing clinical factors might enable a step forward precise medicine.

Image-guided radiation therapy (IGRT) is the most effective treatment for head and neck cancer. The successful implementation of IGRT requires accurate delineation of organ-at-risk (OAR) in the computed tomography (CT) images. In routine clinical practice, OARs are manually segmented by oncologists, which is time-consuming, laborious, and subjective. To assist oncologists in OAR contouring, we proposed a three-dimensional (3D) lightweight framework for simultaneous OAR registration and segmentation. The registration network was designed to align a selected OAR template to a new image volume for OAR localization. A region of interest (ROI) selection layer then generated ROIs of OARs from the registration results, which were fed into a multiview segmentation network for accurate OAR segmentation. To improve the performance of registration and segmentation networks, a centre distance loss was designed for the registration network, an ROI classification branch was employed for the segmentation network, and further, context information was incorporated to iteratively promote both networks' performance. The segmentation results were further refined with shape information for final delineation. We evaluated registration and segmentation performances of the proposed framework using three datasets. On the internal dataset, the Dice similarity coefficient (DSC) of registration and segmentation was 69.7% and 79.6%, respectively. In addition, our framework was evaluated on two external datasets and gained satisfactory performance. These results showed that the 3D lightweight framework achieved fast, accurate and robust registration and segmentation of OARs in head and neck cancer. The proposed framework has the potential of assisting oncologists in OAR delineation.

Brain midline delineation plays an important role in guiding intracranial hemorrhage surgery, which still remains a challenging task since hemorrhage shifts the normal brain configuration. Most previous studies detected brain midline on 2D plane and did not handle hemorrhage cases well. We propose a novel and efficient hemisphere-segmentation framework (HSF) for 3D brain midline surface delineation. Specifically, we formulate the brain midline delineation as a 3D hemisphere segmentation task, and employ an edge detector and a smooth regularization loss to generate the midline surface. We also introduce a distance-weighted map to keep the attention on the midline. Furthermore, we adopt rectification learning to handle various head poses. Finally, considering the complex situation of ventricle break-in for hemorrhages in bilateral intraventricular (B-IVH) cases, we identify those cases via a classification model and design a midline correction strategy to locally adjust the midline. To our best knowledge, it is the first study focusing on delineating the brain midline surface on 3D CT images of hemorrhage patients and handling the situation of ventricle break-in. Extensive validation on our large in-house datasets (519 patients) and the public CQ500 dataset (491 patients), demonstrates that our method outperforms state-of-the-art methods on brain midline delineation.

Current clinical practice or radiomics studies of pancreatic neuroendocrine neoplasms (pNENs) require manual delineation of the lesions in computed tomography (CT) images, which is time-consuming and subjective. We used a semi-automatic deep learning (DL) method for segmentation of pNENs and verified its feasibility in radiomics analysis. This retrospective study included two datasets: Dataset 1, contrast-enhanced CT images (CECT) of 80 and 18 patients respectively collected from two centers; and Dataset 2, CECT of 56 and 16 patients respectively from two centers. A DL-based semi-automatic segmentation model was developed and validated with Dataset 1 and Dataset 2, and the segmentation results were used for radiomics analysis from which the performance was compared against that based on manual segmentation. The mean Dice similarity coefficient of the trained segmentation model was 81.8% and 74.8% for external validation with Dataset 1 and Dataset 2 respectively. Four classifiers frequently used in radiomics studies were trained and tested with leave-one-out cross-validation strategy. For pathological grading prediction with Dataset 1, the area under the receiver operating characteristic curve (AUC) with semi-automatic segmentation was up to 0.76 and 0.87 respectively for internal and external validation. For recurrence study with Dataset 2, the AUC with semi-automatic segmentation was up to 0.78. All these AUCs were not statistically significant from the corresponding results based on manual segmentation. Our study showed that DL-based semi-automatic segmentation is accurate and feasible for the radiomics analysis in pNENs.

Recently, an emerging trend in medical image classification is to combine radiomics framework with deep learning classification network in an integrated system. Although this combination is efficient in some tasks, the deep learning-based classification network is often difficult to capture an effective representation of lesion regions, and prone to face the challenge of overfitting, leading to unreliable features and inaccurate results, especially when the sizes of the lesions are small or the training dataset is small. In addition, these combinations mostly lack an effective feature selection mechanism, which makes it difficult to obtain the optimal feature selection. In this paper, we introduce a novel and effective deep semantic segmentation feature-based radiomics (DSFR) framework to overcome the above-mentioned challenges, which consists of two modules: the deep semantic feature extraction module and the feature selection module. Specifically, the extraction module is utilized to extract hierarchical semantic features of the lesions from a trained segmentation network. The feature selection module aims to select the most representative features by using a novel feature similarity adaptation algorithm. Experiments are extensively conducted to evaluate our method in two clinical tasks: the pathological grading prediction in pancreatic neuroendocrine neoplasms (pNENs), and the prediction of thrombolytic therapy efficacy in deep venous thrombosis (DVT). Experimental results on both tasks demonstrate that the proposed method consistently outperforms the state-of-the-art approaches by a large margin.