Traditional decision tree models, which rely exclusively on numerical variables, often face challenges in handling high-dimensional data and are limited in their ability to incorporate textual information effectively. To address these limitations, we propose the integrated multivariate segmentation tree (IMST), a comprehensive framework designed to improve credit evaluation for small- and medium-sized enterprises (SMEs) by integrating financial data with textual sources. This method comprises three core stages: (1) transforming textual data into numerical matrices through matrix factorization, (2) selecting salient financial features using Lasso regression, and (3) constructing a multivariate segmentation tree based on either the Gini index or entropy, with weakest-link pruning applied to control model complexity. Experimental results based on a dataset of 1,428 Chinese SMEs demonstrated that IMST achieved an accuracy rate of 88.9%, surpassing both baseline decision trees (87.4%) and conventional models such as support vector machines and neural networks. Furthermore, the proposed model demonstrated superior interpretability and computational efficiency, featuring a more streamlined architecture and improved risk detection capabilities

Cancer grading is crucial for patient care, but integrating diverse knowledge of cancer pathogenesis in deep-learning models remains challenging. Traditional graph neural networks (GNN) often overlook the need to connect and balance the information across different levels of granularity, which is essential in histopathology for accurately capturing cellular morphology, tissue architecture, and spatial relationships. To address these challenges and provide a more holistic view of the patient's condition, we propose an Adaptive Multi-Graph Fusion-based Attentive Graph Neural Network (AMGF-GNN), which includes (1) three distinct graphs constructed and processed based on community information, feature similarity, and a combination of both to learn cell-to-cell interactions from three different views, enabling to distinguish the influence of each node in every single graph, (2) Adaptive Attentive Multi-Graph Fusion module which adaptively fuses the embeddings from all three paths, ensuring that the most relevant features from each graph are prioritized, and learns to balance the contributions of the community and feature-based information, addressing the uncertainty of their relative importance for the final grading task, (3) a dual-level loss optimization incorporates intra-graph and inter-graph similarity measures, ensuring consistency and robustness in the learned embeddings. The proposed method was experimentally validated and compared with eight other state-of-the-art (SOTA) models using the glioma TCGA and invasive ductal carcinoma (IDC) datasets. It achieved a superior accuracy of 89.68% for the binary grading of the glioma TCGA dataset, outperforming other competing models and demonstrating the effectiveness of AMGF-GNN.

Automatic brain lesion segmentation enhances diagnostic efficiency by enabling detailed texture analysis and precise delineation of tumor subregions. Multimodal MRI has improved segmentation accuracy by combining complementary information from different modalities. Conventional methods either fuse all modalities uniformly, obscuring how individual modalities contribute to specific segmentation subtasks, or predefine modality-to-subregion mappings based on prior medical knowledge. The former limits interpretability on modality contribution during training, while the latter relies on parameter-heavy architectures like cascaded subnetworks, making models struggle to adapt to varying modalities. To address these challenges, this paper proposes SwitchNet, a novel model that integrates interpretability into the training process and optimizes parameter efficiency without relying on predefined modality selection. First, we propose Adaptive Encoder and Decoder Blocks employing dynamic switching mechanisms to efficiently allocate feature space and prioritize critical subtasks. These blocks enable the model to automatically identify and utilize the most informative modalities. By strategically allocating parameters to modalities, our model optimizes overall parameter complexity while maintaining strong performance. Second, we propose a Guide-Contribution Mechanism to provide interpretability during training by quantitatively revealing the contributions of individual modalities to the segmentation process. This mechanism offers valuable insights into how the model delineates tumor subregions. SwitchNet was validated on three benchmark datasets, including BraTS 2023, ISLES 2022, and UCSF-PDGM, achieving competitive segmentation performance while significantly enhancing interpretability and maintaining parameter efficiency without extra cost. These results highlight its potential for efficient tumor segmentation and clinical explainability.

Deep learning methods have been proposed for tooth segmentation on high-resolution intra-oral scans (IOS) that plays a crucial role in clinical dental practice. However, they generally segment teeth in a low-resolution data with a fixed receptive field and generate final segmentation by up-sampling interpolation, and neglect teeth's morphology priors: their similar dental arch structures and significantly different curvatures in different parts of each tooth. They thus lack adaptability to different parts of each tooth, and show less accurate segmentation of boundary points between teeth and gums due to the up-sampling computation. Further, cluttered poses of IOS limit their generalization and usability of teeth location and geometric information. To address these limitations, a morphology prior enhanced teeth segmentation framework is proposed in this paper. Firstly, a robust preprocessing is introduced to align poses of different IOS by computing their dental arch orientations, thereby improving segmentation generalization and usability of IOS geometric information. Secondly, a decomposition-merging strategy is designed to avoid the up-sampling limitation, which decomposes an IOS into multiple low-resolution data and merges their segmentation outcomes into a high-resolution result. Thirdly, an innovative module integrating semantic and geometric features is proposed to adaptively select deformable receptive fields. It geometrically samples within a variable probability space to construct receptive fields with varied graph relationships for different points, facilitating adaptive segmentation of different parts of each tooth. Experimental results on 6238 IOS from four centers demonstrate that our method significantly outperforms 11 state-of-the-art methods, achieving a 6.93% enhancement for cross-center testing.

While its performance has been widely proven, the large deep learning model poses a severe computational burden and a potential overfitting problem when lacking sufficient training data. The current lightweight models and network pruning provide solutions by reducing redundant parameters; however, fundamentally reducing the redundant features during the training process is yet to be investigated. This paper proposes an innovative lightweight convolution WaveConv based on the wavelet transform. By separating high- and low-frequency features and spatially non-overlapping encoding, the WaveConv decomposes features into low and multi-scale high frequencies to reduce similar features in channel and space dimensions. We have evaluated our method on the MNIST, CamVid, and RFMiD datasets. Experiments demonstrate that WaveConv used 8% of the parameters of vanilla convolution to achieve comparable results on the Cifar-10 dataset. When being adapted to SOTA lightweight models of EfficientNet and MobileNetV3, WaveConv achieved a further 55% parameter reduction while retaining performance on RFMid and CamVid datasets.

ABSTRACT Multiple sclerosis (MS) lesion segmentation from MR imaging is a prerequisite step in clinical diagnosis and treatment of brain diseases. However, automated segmentation of MS lesions remains a challenging task, owing to the variant morphology and uncertain distribution of lesions across subjects. Despite the achieved success by existing methods, two problems still persist in automated segmentation of MS lesions, namely the lack of an effective feature enhancement approach for capturing locality context and the lack of global coherence in prediction for pixels. Hence, we propose a correlation learning network for both local and global context in this work. Specifically, we propose a sparse spatial correlation module to learn the spatial correlations within neighbours for local context. Besides, we propose a global coherence module to encode long‐range dependencies for global context. The proposed method is evaluated on a public ISBI2015 datatset and a private in‐house dataset collected from hospital. Experimental results show the competitive performance of our method against state‐of‐the‐art methods.

Contrastive learning has been used to reduce heavy reliance on labeled data in supervised classification models by learning global high-level semantics. However, its deficit of learning essential local information poses prominent challenges particularly for processing multimodal medical images. To address these challenges, this paper proposes a novel 3D cross-modal spatial contrastive learning framework (CMSCL) for comprehensive medical representation learning. CMSCL retains global high-level semantics through intra-modality and cross-modality contrast; simultaneously, we introduce the spatial self-attention mechanism for learning the contrast of local attention features across different modalities. The contrastive triplet losses with regularization terms leveraging corresponding spatial attention maps enable the learned global semantic features to harness important local information from different modalities. Experiments on public datasets demonstrate improved classification accuracy when compared to the benchmark self-supervised models and validate that our model effectively utilizes multimodal unlabeled data to boost medical image classification.

Through truncating the weights and activations of a deep neural network, conventional binary quantization imposes limitations on the representation capability of the network parameters, which hence deteriorates the detection performance of the network. In this paper, a joint-guided distillation binary neural network via dynamic channel-wise diversity enhancement for object detection (JDBNet) is proposed to mitigate the gap of quantization errors. Our JDBNet includes a dynamic channel-wise diversity scheme and real-valued joint-guided teacher assistance to enhance the representation capability of the binary neural network in the object detection tasks. In the dynamic diversity scheme, the learning channel-wise bias (LCB) layer supports adjusting the magnitude of the parameters in which the sensitivity of the model parameters to the arbitrary quantization method is reduced, thereby improving the diversity expression ability of the feature parameters. In the joint-guided strategy, the single-precision implicit knowledge from the guiding teacher in the multilevel layer is utilized to supervise and penalize the quantitative model, enhancing the fitting performance of parameters in the binary quantized model. Extensive experiments on the PASCAL VOC, MS COCO, and VisDrone-DET datasets demonstrate that our JDBNet outperforms the state-of-the-art binary object detection networks in terms of mean Average Precision.

Delineating 3D blood vessels of various anatomical structures is essential for clinical diagnosis and treatment, however, is challenging due to complex structure variations and varied imaging conditions. Although recent supervised deep learning models have demonstrated their superior capacity in automatic 3D vessel segmentation, the reliance on expensive 3D manual annotations and limited capacity for annotation reuse among different vascular structures hinder their clinical applications. To avoid the repetitive and costly annotating process for each vascular structure and make full use of existing annotations, this paper proposes a novel 3D shape-guided local discrimination (3D-SLD) model for 3D vascular segmentation under limited guidance from public 2D vessel annotations. The primary hypothesis is that 3D vessels are composed of semantically similar voxels and often exhibit tree-shaped morphology. Accordingly, the 3D region discrimination loss is firstly proposed to learn the discriminative representation measuring voxel-wise similarities and cluster semantically consistent voxels to form the candidate 3D vascular segmentation in unlabeled images. Secondly, the shape distribution from existing 2D structure-agnostic vessel annotations is introduced to guide the 3D vessels with the tree-shaped morphology by the adversarial shape constraint loss. Thirdly, to enhance training stability and prediction credibility, the highlighting-reviewing-summarizing (HRS) mechanism is proposed. This mechanism involves summarizing historical models to maintain temporal consistency and identifying credible pseudo labels as reliable supervision signals. Only guided by public 2D coronary artery annotations, our method achieves results comparable to SOTA barely-supervised methods in 3D cerebrovascular segmentation, and the best DSC in 3D hepatic vessel segmentation, demonstrating the effectiveness of our method.

Autonomous vehicles and driver assistance systems achieve satisfactory perception of surroundings but lack cognitive capabilities to conduct knowledge-based risk assessment and key focus cognition. Therefore, we propose the initial research that simulates human cognition to decipher traffic scenarios, to direct attention allocation for driving safety. Firstly, our system imitates cognitive processes including sensing, perceiving, reasoning, and judging, as the pipeline to deliver cognitive understanding beyond sensory perception. Secondly, diverse information and knowledge embedded in our criteria, fuzzy systems, and neural networks vitalizes the cognitive processes and thus fosters interpretability and trustworthiness. It goes beyond the visual features and differs from the utilization of condition-action knowledge in trigger rules design. Thirdly, our system achieves a holistic understanding of the entire scene beyond foreground road users or obstacles. It also stresses cognitive-level ‘prominent hazards’ that impact driving safety and deserve priority attentions and provident strategies, beyond visual saliencies or collision-related risks. Experimental comparisons with state-of-the-art models on 22 metrics on two datasets revealed that our system exhibited the overall smallest deviation from ground truths, justifying the effectiveness of our proposed cognition imitation in understanding driving scenarios. Our better results than baseline annotations further verified the capability in assisting situation awareness. The robustness in diverse environmental conditions, alertness to multitype hazards, and the conformity to knowledge indicate the interpretability and trustworthiness in offering forewarnings and directing attentions.

Epilepsy is a neurological disorder related to frequent seizures. Automatic seizure prediction is crucial for the prevention and treatment of epilepsy. In this paper, we propose a novel model for seizure prediction that incorporates a convolutional neural network (CNN) with multi-head attention mechanism. In this model, the shallow CNN automatically captures the EEG features, and the multi-headed attention focuses on discriminating the effective information among these features for identifying pre-ictal EEG segments. Compared with current CNN models for seizure prediction, the embedded multi-headed attention empowers the shallow CNN to be more flexible, and enables improvement of the training efficiency. Hence, this compact model is more resistant to being trapped in overfitting. The proposed method was evaluated over the scalp EEG data from the two publicly available epileptic EEG databases, and achieved outperforming values of event-level sensitivity, false prediction rate (FPR), and epoch-level F1. Furthermore, our method achieved the stable length of seizure prediction time that was between 14 and 15 min. The experimental comparisons showed that our method outperformed other prediction methods in terms of prediction and generalization performance.

Contrastive learning, which aims to capture general representation from unlabeled images to initialize the medical analysis models, has been proven effective in alleviating the high demand for expensive annotations. Current methods mainly focus on instance-wise comparisons to learn the global discriminative features, however, pretermitting the local details to distinguish tiny anatomical structures, lesions, and tissues. To address this challenge, in this paper, we propose a general unsupervised representation learning framework, named local discrimination (LD), to learn local discriminative features for medical images by closely embedding semantically similar pixels and identifying regions of similar structures across different images. Specifically, this model is equipped with an embedding module for pixel-wise embedding and a clustering module for generating segmentation. And these two modules are unified by optimizing our novel region discrimination loss function in a mutually beneficial mechanism, which enables our model to reflect structure information as well as measure pixel-wise and region-wise similarity. Furthermore, based on LD, we propose a center-sensitive one-shot landmark localization algorithm and a shape-guided cross-modality segmentation model to foster the generalizability of our model. When transferred to downstream tasks, the learned representation by our method shows a better generalization, outperforming representation from 18 state-of-the-art (SOTA) methods and winning 9 out of all 12 downstream tasks. Especially for the challenging lesion segmentation tasks, the proposed method achieves significantly better performance.

Anomaly detection in fundus images remains challenging due to the fact that fundus images often contain diverse types of lesions with various properties in locations, sizes, shapes, and colors. Current methods achieve anomaly detection mainly through reconstructing or separating the fundus image background from a fundus image under the guidance of a set of normal fundus images. The reconstruction methods, however, ignore the constraint from lesions. The separation methods primarily model the diverse lesions with pixel-based independent and identical distributed (i.i.d.) properties, neglecting the individualized variations of different types of lesions and their structural properties. And hence, these methods may have difficulty to well distinguish lesions from fundus image backgrounds especially with the normal personalized variations (NPV). To address these challenges, we propose a patch-based non-i.i.d. mixture of Gaussian (MoG) to model diverse lesions for adapting to their statistical distribution variations in different fundus images and their patch-like structural properties. Further, we particularly introduce the weighted Schatten p-norm as the metric of low-rank decomposition for enhancing the accuracy of the learned fundus image backgrounds and reducing false-positives caused by NPV. With the individualized modeling of the diverse lesions and the background learning, fundus image backgrounds and NPV are finely learned and subsequently distinguished from diverse lesions, to ultimately improve the anomaly detection. The proposed method is evaluated on two real-world databases and one artificial database, outperforming the state-of-the-art methods.

Saliency detection by human refers to the ability to identify pertinent information using our perceptive and cognitive capabilities. While human perception is attracted by visual stimuli, our cognitive capability is derived from the inspiration of constructing concepts of reasoning. Saliency detection has gained intensive interest with the aim of resembling human 'perceptual' system. However, saliency related to human 'cognition', particularly the analysis of complex salient regions ('cogitating' process), is yet to be fully exploited. We propose to resemble human cognition, coupled with human perception, to improve saliency detection. We recognize saliency in three phases ('Seeing' - 'Perceiving' - 'Cogitating), mimicking human's perceptive and cognitive thinking of an image. In our method, 'Seeing' phase is related to human perception, and we formulate the 'Perceiving' and 'Cogitating' phases related to the human cognition systems via deep neural networks (DNNs) to construct a new module (Cognitive Gate) that enhances the DNN features for saliency detection. To the best of our knowledge, this is the first work that established DNNs to resemble human cognition for saliency detection. In our experiments, our approach outperformed 17 benchmarking DNN methods on six well-recognized datasets, demonstrating that resembling human cognition improves saliency detection.

The segmentation of multiple sclerosis (MS) lesions from MR imaging sequences remains a challenging task, due to the characteristics of variant shapes, scattered distributions and unknown numbers of lesions. However, the current automated MS segmentation methods with deep learning models face the challenges of (1) capturing the scattered lesions in multiple regions and (2) delineating the global contour of variant lesions. To address these challenges, in this paper, we propose a novel attention and graph-driven network (DAG-Net), which incorporates (1) the spatial correlations for embracing the lesions in distant regions and (2) the global context for better representing lesions of variant features in a unified architecture. Firstly, the novel local attention coherence mechanism is designed to construct dynamic and expansible graphs for the spatial correlations between pixels and their proximities. Secondly, the proposed spatial-channel attention module enhances features to optimize the global contour delineation, by aggregating relevant features. Moreover, with the dynamic graphs, the learning process of the DAG-Net is interpretable, which in turns support the reliability of segmentation results. Extensive experiments were conducted on a public ISBI2015 dataset and an in-house dataset in comparison to state-of-the-art methods, based on geometrical and clinical metrics. The experimental results validate the effectiveness of proposed DAG-Net on segmenting variant and scatted lesions in multiple regions.

Fusion of multimodal imaging data with nonimaging data is critically important for a more complete understanding of the disease characteristics and therefore essential to accurate computer-aided diagnosis. However, there are two major challenges. 1) Effective discovery of the discriminative multimodal information during the fusion process is hindered by the large dimension gap between raw medical images and clinical factors. 2) Interpreting the complex nonlinear cross-modal association, especially in deep-network-based fusion models, remains an unsolved challenge, which is essential for uncovering the disease mechanism. To address the two challenges, we propose an Interpretable Deep Multimodal Fusion (DMFusion) Framework based on Deep Canonical Correlation Analysis (CCA). Specifically, a novel DMFusion loss is proposed to optimize the discovery of discriminative multimodal representations in low-dimensional latent fusion space. It is achieved by jointly exploiting intermodal correlational association via CCA loss and intra-modal structural and discriminative information via reconstruction loss and cross-entropy loss. For interpreting the nonlinear cross-modal association in DMFusion network, we propose a cross-modal association (CA) score to quantify the importance of input features towards the correlated association, by harnessing integrated gradients in deep networks and canonical loading in CCA projection. The proposed fusion framework was validated on the differential diagnosis of demyelinating diseases in Central Nervous System (CNS) and outperformed six state-of-the-art methods on three fusion tasks.

In conventional unsupervised multi-view clustering (MVC), learning of representations from heterogeneous multiview data and its subsequent clustering are often separately optimized. The disparate optimization would lead to suboptimal performance because multi-view representation learning is not goal-directed. In this paper, we unify unsupervised multi-view learning and deep clustering in a novel discriminative Self-supervised Deep Correlational Multi-view Clustering (SDC-MVC) network. A new unified loss function is proposed to incorporate consensus information into discriminative representations, in which, the former is learnt by maximizing the canonical correlation among multi-view representations projected by neural networks, and the later is achieved through using confident clustering assignments as supervision. Further, multi-view representations are harnessed by our proposed Deep Serial Feature-level (DSF) Fusion layer. Experiments on three public datasets demonstrated that our method outperforms six state-of-the-art correlation-based MVC algorithms in terms of three evaluation metrics.

As a fundamental requirement to many computer vision systems, saliency detection has experienced substantial progress in recent years based on deep neural networks (DNNs). Most DNN-based methods rely on either sparse or dense labeling, and thus they are subject to the inherent limitations of the chosen labeling schemes. DNN dense labeling captures salient objects mainly from global features, which are often hampered by other visually distinctive regions. On the other hand, DNN sparse labeling is usually impeded by inaccurate presegmentation of the images that it depends on. To address these limitations, we propose a new framework consisting of two pathways and an Aggregator to progressively integrate the DNN sparse and DNN dense labeling schemes to derive the final saliency map. In our "zipper" type aggregation, we propose a multiscale kernels approach to extract optimal criteria for saliency detection where we suppress nonsalient regions in the sparse labeling while guiding the dense labeling to recognize more complete extent of the saliency. We demonstrate that our method outperforms in saliency detection compared to other 11 state-of-the-art methods across six well-recognized benchmarking datasets.

Data samples of complicated geometry and nonlinear separability are considered as common challenges to clustering algorithms. In this article, we first construct Mahalanobis distance in the kernel space and then propose a novel fuzzy clustering model with a kernelized Mahalanobis distance, namely KMD-FC. The key contributions of KMD-FC include: first, the construction of KMD matrix is innovatively transformed from the Euclidean distance kernel matrix, which is able to effectively avoid the problem of “curse of dimensionality” posed by explicitly calculating the sample covariance matrix in the kernel space; second, for the first time, the kernelized Gustafson–Kessel (GK) fuzzy C-means algorithm is achieved, which is critically important to extend the applications of the GK algorithm to the nonlinear classification tasks; finally, taking account of the overall distribution of samples in the kernel space after kernel mapping to improve the generalizability of the proposed KMD-FC clustering method. Comprehensive experiments conducted on a wide range of datasets, including synthetic datasets and machine learning repository (UCI) datasets, have validated that the proposed clustering algorithm outperformed the state-of-the-art methods in comparison.

Long noncoding RNAs (lncRNAs) have emerged as potential prognostic markers in various human cancers as they participate in many malignant behaviors. However, the value of lncRNAs as prognostic markers among diverse human cancers is still under investigation, and a systematic signature based on these transcripts that related to pan-cancer prognosis has yet to be reported. In this study, we proposed a framework to incorporate statistical power, biological rationale, and machine learning models for pan-cancer prognosis analysis. The framework identified a 5-lncRNA signature (ENSG00000206567, PCAT29, ENSG00000257989, LOC388282, and LINC00339) from TCGA training studies (n = 1,878). The identified lncRNAs are significantly associated (all P ≤ 1.48E-11) with overall survival (OS) of the TCGA cohort (n = 4,231). The signature stratified the cohort into low- and high-risk groups with significantly distinct survival outcomes (median OS of 9.84 years versus 4.37 years, log-rank P = 1.48E-38) and achieved a time-dependent ROC/AUC of 0.66 at 5 years. After routine clinical factors involved, the signature demonstrated better performance for long-term prognostic estimation (AUC of 0.72). Moreover, the signature was further evaluated on two independent external cohorts (TARGET, n = 1,122; CPTAC, n = 391; National Cancer Institute) which yielded similar prognostic values (AUC of 0.60 and 0.75; log-rank P = 8.6E-09 and P = 2.7E-06). An indexing system was developed to map the 5-lncRNA signature to prognoses of pan-cancer patients. In silico functional analysis indicated that the lncRNAs are associated with common biological processes driving human cancers. The five lncRNAs, especially ENSG00000206567, ENSG00000257989 and LOC388282 that never reported before, may serve as viable molecular targets common among diverse cancers.

Recognition of key morphological features in histological slides is crucial for pathological diagnosis and monitoring therapeutic progress. However, the typical routine microscopic workflow is conducted by hand which is time-consuming and has unavoidable intra- and inter-observer variability like all human work. Therefore, we propose a bifocal classification and fusion network for the automated recognition and cross-modality analysis of diagnostic features in whole-slide multimodal images (WSIs). In brief, paired image tiles cropped from digitized tissue sections were fed into a modified dual-path CNN which accepts asymmetric inputs for classification, and then the inference results were converted to feature distribution heatmaps, which permit qualitative as well as quantitative morphological analyses of entire histological sections, even in combination with adjacent sections that have been stained differently. The multimodal heatmaps were aligned using image registration and fused for cross-modality analysis. Our experiments showed that the network achieved high recognition performance (AUCs of 0.985 and 0.988, and accuracies of 94.7% and 96.1% on two WSI modalities, respectively, against expert markings) and outperformed state-of-the-art methods without training on a large cohort or utilizing domain transfer. In addition, the new method involves a self-contained inference and fusion process and thus harbors significant potential for speeding up microscopic analysis workflows.

Deep convolutional neural networks have been proven successful in multiple benchmark challenges in recent years. However, the performance improvements are heavily reliant on increasingly complex network architecture and a high number of parameters, which require ever increasing amounts of storage and memory capacity. Depthwise separable convolution (DSConv) can effectively reduce the number of required parameters through decoupling standard convolution into spatial and cross-channel convolution steps. However, the method causes a degradation of accuracy. To address this problem, we present depthwise multiception convolution, termed Multiception, which introduces layer-wise multiscale kernels to learn multiscale representations of all individual input channels simultaneously. We have carried out the experiment on four benchmark datasets, i.e. Cifar-10, Cifar-100, STL-10 and ImageNet32×32, using five popular CNN models, Multiception achieved accuracy promotion in all models and demonstrated higher accuracy performance compared to related works. Meanwhile, Multiception significantly reduces the number of parameters of standard convolution-based models by 32.48 % on average while still preserving accuracy.

Accurate segmentation of multiple sclerosis from MR images, faces the challenges imposed by the high variability in lesion appearance, and distant and disjoint lesion regions. Previous methods using multi-scale feature fusion or cascade networks, mostly rely on local feature representation learned from limited receptive field, which fail to leverage global context and model relations between multiple regions. To address these issues, we propose a hybrid feature network (HF-Net) driven by attention and graph convolution features, to improve the MS lesion segmentation from MR images. The attention features help to enhance discriminative feature representation. Specifically, the pyramid augmented attention module encodes spatial features into local features, while the channel augmented attention module models channel-wise interdependencies between features. Meanwhile, the graph feature module exploits the global relations between features over local receptive field. The proposed HF-Net was evaluated on the datasets from the MSSEG Challenge and the ongoing ISBI Challenge, which outperforms several state-of-the-art methods.

Feature engineering and deep learning have been widely used to characterize imaging features in medical applications. However, the importance of geometric structure and spatial relationship of multiple lesions for multifocal diseases are often neglected by these methods. In this paper, we propose a Multi-level Topological Analysis (MTA) framework based on persistent homology, by capturing global-level topological invariants underlying geometric structure and local-level spatial adjacency relationship among lesions and local structure. In particular, a novel Filtration-based Community Discovery algorithm is designed to efficiently partition lesions to local clusters. Experiments demonstrate that our MTA framework outperforms five state-of-the-art persistent homology methods and achieved AUC 824±0.132 on a task of differentiating two multifocal diseases, Multiple Sclerosis and Neuromyelitis Optica.

Automatic rib fracture recognition from chest X-ray images is clinically important yet challenging due to weak saliency of fractures. Weakly Supervised Learning (WSL) models recognize fractures by learning from large-scale image-level labels. In WSL, Class Activation Maps (CAMs) are considered to provide spatial interpretations on classification decisions. However, the high-responding regions, namely Supporting Regions of CAMs may erroneously lock to regions irrelevant to fractures, which thereby raises concerns on the reliability of WSL models for clinical applications. Currently available Mixed Supervised Learning (MSL) models utilize object-level labels to assist fitting WSL-derived CAMs. However, as a prerequisite of MSL, the large quantity of precisely delineated labels is rarely available for rib fracture tasks. To address these problems, this paper proposes a novel MSL framework. Firstly, by embedding the adversarial classification learning into WSL frameworks, the proposed Biased Correlation Decoupling and Instance Separation Enhancing strategies guide CAMs to true fractures indirectly. The CAM guidance is insensitive to shape and size variations of object descriptions, thereby enables robust learning from bounding boxes. Secondly, to further minimize annotation cost in MSL, a CAM-based Active Learning strategy is proposed to recognize and annotate samples whose Supporting Regions cannot be confidently localized. Consequently, the quantity demand of object-level labels can be reduced without compromising the performance. Over a chest X-ray rib-fracture dataset of 10966 images, the experimental results show that our method produces rational Supporting Regions to interpret its classification decisions and outperforms competing methods at an expense of annotating 20% of the positive samples with bounding boxes.

Imbalance data classification is a challenging task in automatic seizure detection from electroencephalogram (EEG) recordings when the durations of non-seizure periods are much longer than those of seizure activities. An imbalanced learning model is proposed in this paper to improve the identification of seizure events in long-term EEG signals. To better represent the underlying microstructure distributions of EEG signals while preserving the non-stationary nature, discrete wavelet transform (DWT) and uniform 1D-LBP feature extraction procedure are introduced. A learning framework is then designed by the ensemble of weakly trained support vector machines (SVMs). Under-sampling is employed to split the imbalanced seizure and non-seizure samples into multiple balanced subsets where each of them is utilized to train an individual SVM classifier. The weak SVMs are incorporated to build a strong classifier which emphasizes seizure samples and in the meantime analyzing the imbalanced class distribution of EEG data. Final seizure detection results are obtained in a multi-level decision fusion process by considering temporal and frequency factors. The model was validated over two long-term and one short-term public EEG databases. The model achieved a [Formula: see text]-mean of 97.14% with respect to epoch-level assessment, an event-level sensitivity of 96.67%, and a false detection rate of 0.86/h on the long-term intracranial database. An epoch-level [Formula: see text]-mean of 95.28% and event-level false detection rate of 0.81/h were yielded over the long-term scalp database. The comparisons with 14 published methods demonstrated the improved detection performance for imbalanced EEG signals and the generalizability of the proposed model.

As studies have found evidence of genetic associations to Parkinson's disease (PD), this paper investigates genetic associations on diagnosis of PD using conventional multimodality PD clinical tests instead of expensive genetic sequencing studies. An importance-driven approach is used to rank features from multimodality clinical tests to determine few key features out of the pool of features. Statistical significance tests are then conducted to validate ranked key features for differentiating genetic PD from other studied categories. The ranked key features are input into various machine learning models for the classification of different PD related categories (PD, GENPD, GENUN). The experiment results show that conventional PD clinical tests without expensive genetic sequencing studies can successfully identify genetic associations to PD from other studied categories.

Widely accessible biomedical data provides new opportunities to discover knowledge for quality healthcare. However, optimal selection of the most informative, representative and non-redundant features from the huge volume of datasets are to be better explored. To address this challenge, in this paper we propose an integrative clustering and supervised feature selection approach. In our framework, the unsupervised clustering contributes to reducing redundancy by exploring the correlation among features, while supervised learning selects informative and representative features by examining the relation between features and outputs. Our approach was evaluated on a high-dimensional biomedical image feature set and a clinical dataset for survival prediction. Experimental results demonstrated that the selected features had the higher discriminative ability and lower redundancy, and our feature selection outperformed two state-of-the-art methods in terms of feature significance and feature correlation. Furthermore, experiments revealed that redundancy reduction helped handle the overfitting problem.

Saliency detection on images has experienced substantial progress in recent years on the basis of deep neural network (DNN). However, there may exist secondary saliency in the background that distracts DNN learning and mistakes the secondary salient regions as saliency. To address this issue, we propose a dual-term energy to improve the inference of saliency on top of DNN estimation, where dense term smoothens salient regions in pixel scale and sparse term extracts prior knowledge to differentiate saliency and non-saliency superpixels. Our prior knowledge including extra- and intra-region priors, contributes to improving overall saliency detection. The extra-region prior knowledge estimates the saliency probabilities for different pre-partitioned regions to eliminate the secondary saliency. The intra-region prior knowledge helps to group the salient regions that otherwise could be ignored by DNN predictor, and thus to provide more complete saliency definition. We evaluated our model on 8,465 images from four well-recognized saliency detection benchmarking datasets, and compared our model to six state-of-the-art comparative methods. Experimental results demonstrated that our model outperformed the state-of-the-art counterpart with improvements of up to 2.51% in terms of F-measure.

This paper investigates End-User Development (EUD) for interactive data-analytic interfaces-building upon the ideas of making machine learning transparent. The research is carried out in a business operation environment (water pipe failure prediction in our case) motivated to integrate advanced analytics into decision-making processes of an urban Internet of Things (IoT) concept. We explore effects of revealing uncertainty and correlation on user confidence in a data-driven decision making scenario. It was found that user confidence varied significantly amongst various user groups when different machine learning models were displayed with/without supplementary information. Galvanic Skin Response (GSR) signals were analyzed and shown as reasonable indices for predicting user confidence levels. Supplementary data visualizations (of inherent uncertainty and correlation in data) contributed to explicability principles while GSR indexing added towards correctibility principles. We recommend transparent machine learning as the key to effective EUD for interactive data analytics.

Clustering is increasingly important for multiview data analytics and current algorithms are either based on the collaborative learning of local partitions or directly derived global clustering from multikernel learning. In this paper, we innovate a clustering model that unifies the local partitions and global clustering in a collaborative learning framework. We first construct a common multikernel space from a set of basis kernels to better reflect clustering information of each individual view. Then, considering that joint local partitions would conform to the global clustering, we fuse the local partitions and global clustering guidance as a single objective function in accordance with fuzzy clustering form. The collaborative learning strategy enables the mutual and interactive clustering from local partitions and global clustering. The validation was performed over two synthetic and four public databases and the clustering accuracy was measured by normalized mutual information and rand index. The experimental results demonstrated that the proposed algorithm outperformed the related state-of-the-art algorithms in comparison, which included multitask, multikernel, and multiview clustering approaches.

Safety-oriented visualization is one of significant approaches to gain insights from time-spatial data while neural net currently serves as a decent way to perform machine learning in data mining industry. This paper proposes a visual analytics pipeline for trajectory data enabling better understanding movements pattern of people using Neural Network as back-end and other visualization techniques as front-end for gaining information of preferences of attractions, similarities of groups, popularities of attractions and pattern of movement flow. Such understandings help to address the management issue by extracting the outstanding features to detect abnormal pattern such as detection of crime and predicting overall movements, and so on. Successfully dealing with those issues would have significant improvements of entire management of public facility such as parks and transportation.

Unsupervised segmentation and contour detection remains a challenging task. In graph-based unsupervised segmentation, the formulation of the affinity graph is pivotal to segmentation performance. Conventional graph-based approaches often only define pixels as graph nodes, and may overlook important regional information. In this paper, we propose a novel scheme for affinity graph construction, where the affinity weight matrix unifies the association across pixel-wise nodes and multilevel region-wise nodes of different scales. Integrating the multilevel regional information, which is formulated using superpixels, into the affinity graph contributes to better capture of image intensity and color cues. Experimental evaluation of our approach on the BSDS500 dataset showed that our proposed method achieved the second best performance compared to other nine unsupervised state-of-art methods commonly used for comparison.

In saliency object detection, inappropriate boundary-background priors is known to degrade performance in challenging image datasets, and even may lead to `inverse' results when saliency regions are attached to the image boundaries. This is an active field where many works have proposed various techniques to lessen such degradation by inappropriate boundary-background priors. Although the use of boundary-background priors has shown to be capable of improving the detection, inherently, these techniques confront serious challenges in background suppression. To overcome this limitation, we propose an adaptive background extractor to search background seeds without the need of boundary-background priors. With the adaptive background seeds, the saliency objects can be then extracted via our proposed hierarchical foreground estimation model. We evaluate our adaptive Background Search and Foreground Estimation (BSFE) algorithm in comparison with six state-of-the-art methods on four well-recognized public datasets. The experimental results demonstrate that our BSFE algorithm outperforms compared methods in majority of the datasets and in particular achieves double-winners in terms of F-measure and mean absolute error on two challenging datasets.

BACKGROUND: Direct volume rendering is one of flexible and effective approaches to inspect large volumetric data such as medical and biological images. In conventional volume rendering, it is often time consuming to set up a meaningful illumination environment. Moreover, conventional illumination approaches usually assign same values of variables of an illumination model to different structures manually and thus neglect the important illumination variations due to structure differences. RESULTS: We introduce a novel illumination design paradigm for volume rendering on the basis of topology to automate illumination parameter definitions meaningfully. The topological features are extracted from the contour tree of an input volumetric data. The automation of illumination design is achieved based on four aspects of attenuation, distance, saliency, and contrast perception. To better distinguish structures and maximize illuminance perception differences of structures, a two-phase topology-aware illuminance perception contrast model is proposed based on the psychological concept of Just-Noticeable-Difference. CONCLUSIONS: The proposed approach allows meaningful and efficient automatic generations of illumination in volume rendering. Our results showed that our approach is more effective in depth and shape depiction, as well as providing higher perceptual differences between structures.

Accurate liver segmentation from computed tomography (CT) images is problematic due to non-uniform density, weak boundaries and because there may be multiple liver tumors that have heterogeneous intensities in region(s) of interest (ROIs). So we propose a generalized energy framework that harnesses the statistical intensity approximation with image data on graphs. Our statistical energy term takes advantage of the mixture-of-mixtures Gaussian model to approximate the probability density distribution of the liver and background to better differentiate between the two. The probability density estimation can be combined with the spatial cohesion of the graph-based Dirichlet integral by using graph calculus. Matrix decomposition and differentiation are used to minimize our proposed energy functional. We tested our approach on 20 public high-contrast CT images with single and multiple liver tumors. Our method had an average dice similarity coefficient (DSC) of 93.75±1.29%, an average false positive (FP) rate of 9.43±3.52% and an average false negative (FN) rate of 3.48±1.48%. Our method outperformed the benchmark graph-based Random Walker algorithm (average DSC=81.97±4.09%, average FP rate 34.10±10.53%, and average FN rate 7.10±4.35%).

Lighting design plays critical roles in depicting structural details in volume rendering. Insufficient and excessive illumination can both affect effectiveness of presenting structural details in visualization. This paper introduces topological importance into the lighting design and proposes the importance-aware lighting. In the proposed approach, the lighting in volume rendering is enhanced based on topological importance. As a result, importance of structures can be depicted from the lighting perspective. The contour tree, one of topological data structures, is used to represent topology in this paper. Topological importance such as persistence derived from the contour tree is used to modulate lighting coefficients. The experimental results demonstrate that the importance-aware lighting not only helps to depict structural details more clearly but also reveal topological importance of structures in rendering. The importance-aware lighting is more meaningful to users but not a random selection without physical meanings based on preferences.

Intensity inhomogeneities and different types/levels of image noise are the two major obstacles to accurate image segmentation by region-based level set models. To provide a more general solution to these challenges, we propose a novel segmentation model that considers global and local image statistics to eliminate the influence of image noise and to compensate for intensity inhomogeneities. In our model, the global energy derived from a Gaussian model estimates the intensity distribution of the target object and background; the local energy derived from the mutual influences of neighboring pixels can eliminate the impact of image noise and intensity inhomogeneities. The robustness of our method is validated on segmenting synthetic images with/without intensity inhomogeneities, and with different types/levels of noise, including Gaussian noise, speckle noise, and salt and pepper noise, as well as images from different medical imaging modalities. Quantitative experimental comparisons demonstrate that our method is more robust and more accurate in segmenting the images with intensity inhomogeneities than the local binary fitting technique and its more recent systematic model. Our technique also outperformed the region-based Chan–Vese model when dealing with images without intensity inhomogeneities and produce better segmentation results than the graph-based algorithms including graph-cuts and random walker when segmenting noisy images.

Segmentation of the target object(s) from images that have multiple complicated regions, mixture intensity distributions or are corrupted by noise poses a challenge for the level set models. In addition, the conventional piecewise smooth level set models normally require prior knowledge about the number of image segments. To address these problems, we propose a novel segmentation energy function with two distribution descriptors to model the background and the target. The single background descriptor models the heterogeneous background with multiple regions. Then, the target descriptor takes into account the intensity distribution and incorporates local spatial constraint. Our descriptors, which have more complete distribution information, construct the unique energy function to differentiate the target from the background and are more tolerant of image noise. We compare our approach to three other level set models: 1) the Chan-Vese; 2) the multiphase level set; and 3) the geodesic level set. This comparison using 260 synthetic images with varying levels and types of image noise and medical images with more complicated backgrounds showed that our method outperforms these models for accuracy and immunity to noise. On an additional set of 300 synthetic images, our model is also less sensitive to the contour initialization as well as to different types and levels of noise.

Equation (16) in the above paper (ibid., vol. 22, no. 8, pp. 3296-3309, Aug. 2013) contained an error in the numerator. Equation (17) in the same paper contained errors in both the numerator and the denominator. The corrected versions of both equations are presented here.

We propose a novel joint probabilistic model that correlates a new probabilistic shape model with the corresponding global intensity distribution to segment multiple abdominal organs simultaneously. Our probabilistic shape model estimates the probability of an individual voxel belonging to the estimated shape of the object. The probability density of the estimated shape is derived from a combination of the shape variations of target class and the observed shape information. To better capture the shape variations, we used probabilistic principle component analysis optimized by expectation maximization to capture the shape variations and reduce computational complexity. The maximum a posteriori estimation was optimized by the iterated conditional mode-expectation maximization. We used 72 training datasets including low- and high-contrast CT images to construct the shape models for the liver, spleen and both kidneys. We evaluated our algorithm on 40 test datasets that were grouped into normal (34 normal cases) and pathologic (6 datasets) classes. The testing datasets were from different databases and manual segmentation was performed by different clinicians. We measured the volumetric overlap percentage error, relative volume difference, average square symmetric surface distance, false positive rate and false negative rate and our method achieved accurate and robust segmentation for multiple abdominal organs simultaneously.

Deformable registration has an important application in radiation therapy and evaluation of patient response to the treatment. However, the conventional deformable registration may introduce excessive deformation on tumor shape and volume, which thereby may lead to less accurate dose delivery on the targeted tumor and possible lesion on the surrounding healthy organs. In this paper, we proposed a deformable registration model to preserve accurate tumor information while retaining the global alignment of the temporal or multimodal biomedical images. Our experimental results on 12 clinical datasets with 15 tumors demonstrated that our deformable model was able to maintain the unique pathological features of the tumor and provide appropriate alignment and correspondence.

Euclidean distance transform is widely used in many applications of image analysis and processing. Traditional algorithms are time-consuming and difficult to realize. This paper proposes a novel fast distance transform algorithm. Firstly, mark each foreground's nearest background pixel's position in the row and column, and then use the marks scan the foreground area and figure out the first foreground pixel distance transform information, According to the first pixel' information, design four small regions for its 4-adjacent foreground pixel and also based on the marks search out each adjacent foreground pixel's nearest background pixel. As the region growing, iteratively process each adjacent pixel until all the foreground pixels been resolved. Our algorithm has high efficiency and is simple to implement. Experiments show that comparing to the existing boundary striping and contour tracking algorithm, our algorithm demonstrates a significant improvement in time and space consumption.

Combined PET-CT is now increasingly used for the clinical evaluation of cancer and is arguably the best tool to stage non-small cell lung cancer (NSCLC). We propose a framework to better delineate lung tumors which utilizes information from PET and CT images. The framework is based on a downhill region growing technique for PET and a Gaussian mixture model for CT images. We applied our framework in 20 PET-CT studies from patients with NSCLC. Experiments show that our method is able to delineate lung tumors in complex cases where the tumors are located near other organs with similar intensities in PET images or when the tumors extends into the chest wall or the mediastinum. We also compared 10 of the datasets with experts performing manual delineation, which produced a volumetric overlapped fraction of 0.78 ± 0.10.

Reliable automated or semiautomated lung tumor delineation methods in positron emission tomography should provide accurate tumor boundary definition and separation of the lung tumor from surrounding tissue or "hot spots" that have similar intensities to the lung tumor. We propose a tumor-customized downhill (TCD) method to achieve these objectives. Our approach includes: 1) automatic formulation of a tumor-customized criterion to improve tumor boundary definition, 2) a monotonic property of the standardized uptake value (SUV) of tumors to separate the tumor from adjacent regions of increased metabolism ("hot spot"), and 3) accounts for tumor heterogeneity. Three simulated lesions and 30 PET-CT studies, grouped into "simple" and "complex" groups, were used for evaluation. Our main findings are that TCD, when compared to the threshold based on 40% and 50% maximum SUV, adaptive threshold, Fuzzy c-means, and watershed techniques achieved the highest Dice's similarity coefficient average for simulation data (0.73) and "complex" group (0.71); the least volumetric error in the "simple" (1.76 mL) and the "complex" group (14.59 mL); and TCD solves the problem of leakage into adjacent tissues when many other techniques fail.

Automated liver segmentation is problematic due to variations in liver shape / size and because the liver has a similar density distribution to surrounding structures. We propose a method that: 1) utilizes iteratively constructed probabilistic liver and rib cage atlases, 2) conducts the Gaussian distribution analysis to avoid incorrectly classifying the irrelevant surrounding tissues as `liver region' in the conventional probabilistic atlas based method, and maps the intensity range of the input candidate liver region onto the liver atlas, 3) retrieves the `missing parts' of the liver by deformable registration. Our approach is automated and able to segment the liver from high-contrast and low-contrast CT volumes. Forty clinical CT studies were used for atlas construction and validation. Our method outperformed two other probabilistic atlas-based liver segmentation methods.

Registration of planning images with daily images is an important component for adaptive radiation therapy (ART). In this paper, a multiscale deformable registration framework is proposed by combining edge preserving scale space with the free form deformation (FFD) for registration of planning computed tomography (CT) images with daily cone beam CT (CBCT) images. The edge preserving scale space which is able to select edges and contours of an image according to their geometric size is derived from the total variation model with the L1 norm (TV-L1). At each scale, the selected edges and contours are sufficiently strong to drive the deformation using the FFD grid, then the deformation fields are gained by a coarse to fine manner. Furthermore, for automated registration we design an optimal estimation of the TV-L1 parameter by minimizing the defined offset. The experiments on CT and CBCT images show accuracy and robustness when compared to traditional methods.

Registration of abdominal images plays an important role in clinical practice, however, because of the complex deformations of organ structure and volume, abdominal image registration remains a challenge. In this paper, a hybrid registration approach is proposed, which consists of two procedures: intensity-based registration procedure and landmark-based registration procedure. In intensity-based registration, in order to speed up registration convergence and to improve registration computation efficiency, a wavelet-based hierarchical method is proposed, in which the global displacements are corrected using mutual information algorithm. In the landmark-based registration, firstly, the landmark points are selected automatically; and then, the local non-linear deformations are corrected using the elastically thin-plate splines. By combining the advantages of intensity-based registration method and that of landmark-based method, the proposed approach can register the images accurately and efficiently. The registration performance of the proposed algorithm is validated by experiments on clinical computed tomography (CT) abdominal images.