Magnetic Particle Imaging (MPI) is an innovative medical modality, providing nanomolar-scale in vivo sensitivity and radiation-free dynamic real-time detection for precision medicine. However, MPI faces a challenging problem in accurately visualizing nanoparticle distributions, where the reconstructed images with unidirectional scanning exhibit anisotropy. The anisotropy in spatial resolution leads to distortion and blurred image boundaries. Existing deep learning methods for anisotropy calibration are only limited to simulation data due to lacking of real-world MPI datasets. To address the aforementioned problems, we spent over three years designing and constructing a real-world MPI anisotropic image datasets (20,156 images) with diverse phantoms (sensitivity, resolution, vessel, shape) and animal scanning. Then, we introduce a novel Mamba-based method, MPI-Mamba, for anisotropic image calibration. Specifically, we propose a latent feature fusion state space model (LFF-SSM) block for feature fusion and leverage conditional latent diffusion model (CL-DM) branch for feature extraction. The CL-DM is performed to extract latent features in a highly compressed latent space for guiding the calibration and deblurring process. Next, we exploit the LFF-SSM to fully fuse the extracted multi-scale features to capture contextual information from the image structure, enabling the model to learn the overall distribution of signal concentration. We evaluate our method and competing methods on simulation dataset and our constructed diverse real-world MPI datasets. The results show that our proposed approach outperforms competing methods for anisotropic image calibration and deblurring.

Magnetic particle imaging (MPI) has demonstrated its advantages of high sensitivity and temporal resolution in various preclinical applications. However, during the imaging process, the signal is susceptible to different noises, resulting in severe stripe artifacts in reconstructed MPI images. This phenomenon will be further aggravated in scenarios with low-concentration particles, which is a standard practice in biological applications, thereby seriously hindering the identification of key information. To solve this problem, we propose a joint optimization approach called diffusion denoising model for MPI (DDMPI) that integrates diffusion model with Transformer to remove the artifacts directly from MPI images obtained in the low-concentration scenarios. In DDMPI, a latent encoder generates prior features containing the relevance mapping between the contents and the artifacts within MPI images, and a conditional latent diffusion model optimizes these prior features. A U-shape Transformer module incorporates the prior features by a hierarchical integration module and utilizes a strip-self-attention module to capture the spatial distribution of the artifacts. Ablation experiments demonstrate the effectiveness of these modules in DDMPI. Extensive experiments, including simulation, phantom and in vivo experiments, demonstrate that DDMPI effectively removes artifacts and recovers fine details. Additionally, DDMPI is independent of the primary image reconstruction methods of various scanning devices. Thus, DDMPI can be not only practically applied to in vivo imaging but also flexibly combined with various existing MPI devices to effectively improve the imaging quality and provide critical information about diseases.

Accurate segmentation of 3D vascular structures is essential for various medical imaging applications. The dispersed nature of vascular structures leads to inherent spatial uncertainty and necessitates location awareness, yet most current 3D medical segmentation models rely on the patch-wise training strategy that usually loses this spatial context. In this study, we introduce the Coordinate-aware Modulated Mamba Network (COMMA) and contribute a manually labeled dataset of 570 cases, the largest publicly available 3D cerebrovascular dataset to date. COMMA leverages both entire and cropped patch data through global and local branches, ensuring robust and efficient spatial location awareness. Specifically, COMMA employs a channel-compressed Mamba (ccMamba) block to efficiently encode full-resolution image data, capturing long-range dependencies while optimizing computational costs. Additionally, we propose a coordinate-aware modulated (CaM) block to enhance interactions between the global and local branches, allowing the local branch to better perceive spatial information. We evaluate COMMA on six datasets, covering two imaging modalities and five types of vascular tissues. The results demonstrate COMMA's superior performance compared to state-of-the-art methods with computational efficiency, especially in segmenting small vessels. Ablation studies further highlight the importance of our proposed modules and spatial information. The code will be available at COMMA.

As a critical statistical technique in oncology, survival prediction is used to estimate the probability of survival or time-to-event outcomes. Identifying survival-related factors from pathology and genomic data is a key approach for analyzing survival outcomes. However, current methods face several challenges, such as the suboptimal adaptation of pre-trained vision foundation models to specific tasks during feature extraction from whole slide images (WSIs), and the fact that many pathology-based models fail to integrate repetitive gene expression information during pre-training. In this study, we propose a plug-and-play multiple instance learning (MIL)-based foundation model tuning strategy to adapt vision foundation models for downstream tasks and incorporate knowledge from genomic data. Specifically, we introduce Task-specific Instance Selection, which utilizes zero-shot learning to efficiently select task-relevant WSI regions, improving tuning efficiency and reducing interference from irrelevant tissue areas. Additionally, we develop a multi-model prompt token for model fine-tuning, which integrates genetic information into the prompt-tuning process and transfers new modality information to pre-trained vision foundation models. To further enhance the model's ability to learn genetic information during fine-tuning, we introduce a Gene Distribution Aware Task as an auxiliary task to the traditional survival task. This auxiliary task helps the model better perceive multimodal information. Extensive experimental results on three public TCGA datasets demonstrate that our model outperforms all previous MIL-based methodologies and fine-tuning approaches in terms of performance.

Magnetic Particle Imaging (MPI) enables noninvasive temperature imaging without depth limitations. However, due to the lack of effective calibration strategies that can simultaneously address issues such as calibration infeasibility and environmental mismatch, its practical in vivo application remains challenging. In this work, we propose a novel in vivo temperature imaging method based on a dual-mode magnetic particle spectroscopy/magnetic particle imaging (MPS/MPI) system. First, MPS is employed to capture the differences in harmonic phase responses of magnetic nanoparticles (MNPs) under in vivo and in vitro conditions, thereby enabling the construction of calibration functions that are consistent with the in vivo environment. Second, an MLP based calibration strategy is proposed, which accounts for non-ideal deviations from the approximately linear temperature-phase relationship and integrates multi-parameter information into a unified network, thereby enabling accurate and stable temperature mapping. Comprehensive simulation, in vitro, and in vivo experiments demonstrate that, compared with conventional phantom-based temperature mapping methods, the proposed method reduces the in vivo temperature reconstruction error by approximately 17.24% and achieves an average absolute temperature error below $1.257~^{\circ } $ C. These results verify the feasibility of accurate in vivo temperature imaging using MPI and provide essential technical support for temperature-sensitive applications, including magnetic hyperthermia.

Aggregating features of tens of thousands of patches into Whole Slide Images (WSIs) representations via aggregators is a crucial step in computational pathology. However, existing aggregation strategies overlook the morphological variability of tissue regions in WSIs stemming from differences in clinical procedures and tumor characteristics, leading to two critical limitations: 1) attention collapse in long sequences caused by significant variation in patch numbers across WSIs (ranging from thousands to tens of thousands per WSI); 2) attention misallocation due to under-trained positional embeddings resulting from the non-uniform spatial coordinates introduced by irregular patch distributions. Consequently, current attention-based methods struggle to generalize across this morphological variability, resulting in inconsistent aggregation performance and compromised model reliability in clinical settings. To address these issues, we propose a Entropy-Stabilized Attention-based Multiple Instance Learning (StableMIL) framework, which incorporates an entropy-stabilized attention mechanism to ensure consistent aggregation across WSIs with varying patch numbers and a Randomly Projected 2D rotary position embedding to enhance spatial representation robustness across irregular patch distributions. Extensive theoretical and experimental analyses on nine WSI datasets spanning diverse cancer types, across both classification and survival prediction tasks, demonstrate that StableMIL effectively overcomes the challenges of handling long instance sequences and out-of-distribution spatial coordinates. Our framework consistently outperforms representative baselines, particularly in survival prediction, with stable improvements observed across all evaluated cancer types and morphological scenarios, highlighting its potential for real-world clinical applications. Our source code is available at https://github.com/theeeqi/stableMIL.

Endoscopes, both white light and fluorescence, face challenges like insufficient illumination during surgical procedures. Deep learning, especially diffusion models, demonstrates considerable potential for low-light enhancement in the medical field. However, challenges in enhancing endoscopic images persist, including high computational resource consumption, lengthy processing times, and potential result distortion. To address these issues, we propose a wavelet-based diffusion method for fast and efficient low-light surgical endoscopic image enhancement, dubbed WDiff. It utilizes wavelet transform to reduce computational resource and enhance inference speed while preserving key features. To avoid degradation during reverse process, we introduce a Degradation Corrector Unit (DCU) to ensure accurate sampling results. Moreover, we design a Detail Coefficients Restoration Block (DCRB) to reconstruct the local sparse information in detail coefficients across horizontal, vertical, and diagonal orientations. Extensive experiments on benchmark datasets demonstrate that our method outperforms other SOTA methods both visually and quantitatively, achieving an optimal balance between complexity and efficiency.

Nuclei instance segmentation and classification of hematoxylin and eosin (H&E) stained digital pathology images are essential for further downstream cancer diagnosis and prognosis tasks. Previous works mainly focused on bottom-up methods using a single-level feature map for segmenting nuclei instances, while multilevel feature maps seemed to be more suitable for nuclei instances with various sizes and types. In this paper, we develop an effective top-down nuclei instance segmentation and classification framework (NuHTC) based on a hybrid task cascade (HTC). The NuHTC has two new components: a watershed proposal network (WSPN) and a hybrid feature extractor (HFE). The WSPN can provide additional proposals for the region proposal network which leads the model to predict bounding boxes more precisely. The HFE at the region of interest (RoI) alignment stage can better utilize both the high-level global and the low-level semantic features. It can guide NuHTC to learn nuclei instance features with less intraclass variance. We conduct extensive experiments using our method in four public multiclass nuclei instance segmentation datasets. The quantitative results of NuHTC demonstrate its superiority in both instance segmentation and classification compared to other state-of-the-art methods.

Accurate segmentation of lesions plays a critical role in medical image analysis and diagnosis. Traditional segmentation approaches that rely solely on visual features often struggle with the inherent uncertainty in lesion distribution and size. To address these issues, we propose STPNet, a Scale-aware Text Prompt Network that leverages vision-language modeling to enhance medical image segmentation. Our approach utilizes multi-scale textual descriptions to guide lesion localization and employs retrieval-segmentation joint learning to bridge the semantic gap between visual and linguistic modalities. Crucially, STPNet retrieves relevant textual information from a specialized medical text repository during training, eliminating the need for text input during inference while retaining the benefits of cross-modal learning. We evaluate STPNet on three datasets: COVID-Xray, COVID-CT, and Kvasir-SEG. Experimental results show that our vision-language approach outperforms state-of-the-art segmentation methods, demonstrating the effectiveness of incorporating textual semantic knowledge into medical image analysis. The code has been made publicly on https://github.com/HUANGLIZI/STPNet.

Recently, Magnetic Particle Imaging, an emerging functional imaging modality, has exhibited outstanding spatial-temporal resolution and sensitivity. The general reconstruction pipeline of Magnetic Particle Imaging involves calibrating a System Matrix and then solving an ill-posed inverse problem combined with the measured particle signals. However, the introduction of noise during the System Matrix calibration procedure is inevitable, which degrades the detailed information in the reconstructed images. Therefore, frequency selection methods based on signal-to-noise ratio are commonly adopted. However, these methods lead to a decrease in the available high-frequency components, which damages the spatial resolution. To address this problem, we propose an unsupervised memory-guided denoising framework with unpaired noisy-clean System Matrix components, called U-N2C. Specifically, we design a Pattern Memory Block to memorize System Matrix patterns, directed by a position-aware frequency index embedding. Meanwhile, we devise a Noise Memory Block to implicitly approximate noise distributions. With the guidance of our dual memory blocks, we can disentangle the noise and content of the System Matrix in the latent space. Furthermore, benefiting from the ability to model complex noise, our method can generate pseudo but high-quality noisy-clean pairs and further enhance our denoising capability. Experiments on both synthetic and real noise demonstrate that our U-N2C achieves cutting-edge performance compared to other methods. Moreover, we conduct extensive qualitative and quantitative ablation studies to verify the effectiveness of our method. Our code has been available at U-N2C.

Prognostic assessment remains a critical challenge in medical research, often limited by the lack of well-labeled data. In this work, we introduce ContraSurv, a weakly-supervised learning framework based on contrastive learning, designed to enhance prognostic predictions in 3D medical images. ContraSurv utilizes both the self-supervised information inherent in unlabeled data and the weakly-supervised cues present in censored data, refining its capacity to extract prognostic representations. For this purpose, we establish a Vision Transformer architecture optimized for our medical image datasets and introduce novel methodologies for both self-supervised and supervised contrastive learning for prognostic assessment. Additionally, we propose a specialized supervised contrastive loss function and introduce SurvMix, a novel data augmentation technique for survival analysis. Evaluations were conducted across three cancer types and two imaging modalities on three real-world datasets. The results confirmed the enhanced performance of ContraSurv over competing methods, particularly in data with a high censoring rate.

Providing precise and comprehensive diagnostic information to clinicians is crucial for improving the treatment and prognosis of nasopharyngeal carcinoma. Multi-modal foundation models, which can integrate data from various sources, have the potential to significantly enhance clinical assistance. However, several challenges remain: (1) the lack of large-scale visual-language datasets for nasopharyngeal carcinoma; (2) the inability of existing pre-training and fine-tuning methods to capture the hierarchical features required for complex clinical tasks; (3) current foundation models having limited visual perception due to inadequate integration of multi-modal information. While curriculum learning can improve a model's ability to handle multiple tasks through systematic knowledge accumulation, it still lacks consideration for hierarchical features and their dependencies, affecting knowledge gains. To address these issues, we propose the Hierarchical Feature Fusion Curriculum Learning method, which consists of three stages: visual knowledge learning, coarse-grained alignment, and fine-grained fusion. First, we introduce the Hybrid Contrastive Masked Autoencoder to pre-train visual encoders on 755K multi-modal images of nasopharyngeal carcinoma CT, MRI, and endoscopy to fully extract deep visual information. Then, we construct a 65K visual instruction fine-tuning dataset based on open-source data and clinician diagnostic reports, achieving coarse-grained alignment with visual information in a large language model. Finally, we design a Mixture of Experts Cross Attention structure for deep fine-grained fusion of global multi-modal information. Our model outperforms previously developed specialized models in all key clinical tasks for nasopharyngeal carcinoma, including diagnosis, report generation, tumor segmentation, and prognosis.

The diagnosis of lymph node metastasis (LNM) is essential for colorectal cancer (CRC) treatment. The primary method of identifying LNM is to perform frozen sections and pathologic analysis, but this method is labor-intensive and time-consuming. Therefore, combining intraoperative fluorescence imaging with deep learning (DL) methods can improve efficiency. The majority of recent studies only analyze uni-modal fluorescence imaging, which provides less semantic information. In this work, we mainly established a multi-modal fluorescence imaging feature fusion prediction (MFI-FFP) model combining white light, fluorescence, and pseudo-color imaging of lymph nodes for LNM prediction. Firstly, based on the properties of various modal imaging, distinct feature extraction networks are chosen for feature extraction, which could significantly enhance the complementarity of various modal information. Secondly, the multi-modal feature fusion (MFF) module, which combines global and local information, is designed to fuse the extracted features. Furthermore, a novel loss function is formulated to tackle the issue of imbalanced samples, challenges in differentiating samples, and enhancing sample variety. Lastly, the experiments show that the model has a higher area under the receiver operating characteristic (ROC) curve (AUC), accuracy (ACC), and F1 score than the uni-modal and bi-modal models and has a better performance compared to other efficient image classification networks. Our study demonstrates that the MFI-FFP model has the potential to help doctors predict LNM and shows its promise in medical image analysis.

Directly predicting human epidermal growth factor receptor 2 (HER2) status from widely available hematoxylin and eosin (HE)-stained whole slide images (WSIs) can reduce technical costs and expedite treatment selection. Accurately predicting HER2 requires large collections of multi-site WSIs. Federated learning enables collaborative training of these WSIs without gigabyte-size WSIs transportation and data privacy concerns. However, federated learning encounters challenges in addressing label imbalance in multi-site WSIs from the real world. Moreover, existing WSI classification methods cannot simultaneously exploit local context information and long-range dependencies in the site-end feature representation of federated learning. To address these issues, we present a point transformer with federated learning for multi-site HER2 status prediction from HE-stained WSIs. Our approach incorporates two novel designs. We propose a dynamic label distribution strategy and an auxiliary classifier, which helps to establish a well-initialized model and mitigate label distribution variations across sites. Additionally, we propose a farthest cosine sampling based on cosine distance. It can sample the most distinctive features and capture the long-range dependencies. Extensive experiments and analysis show that our method achieves state-of-the-art performance at four sites with a total of 2687 WSIs. Furthermore, we demonstrate that our model can generalize to two unseen sites with 229 WSIs. Code is available at: https://github.com/boyden/PointTransformerFL

Imbalanced classification is a common and difficult task in many medical image analysis applications. However, most existing approaches focus on balancing feature distribution and classifier weights between classes, while ignoring the inner-class heterogeneity and the individuality of each sample. In this paper, we proposed a sample-specific fine-grained prototype learning (SFPL) method to learn the fine-grained representation of the majority class and learn a cosine classifier specifically for each sample such that the classification model is highly tuned to the individual's characteristic. SFPL first builds multiple prototypes to represent the majority class, and then updates the prototypes through a mixture weighting strategy. Moreover, we proposed a uniform loss based on set representations to make the fine-grained prototypes distribute uniformly. To establish associations between fine-grained prototypes and cosine classifier, we propose a selective attention aggregation module to select the effective fine-grained prototypes for final classification. Extensive experiments on three different tasks demonstrate that SFPL outperforms the state-of-the-art (SOTA) methods. Importantly, as the imbalance ratio increases from 10 to 100, the improvement of SFPL over SOTA methods increases from 2.2% to 2.4%; as the training data decreases from 800 to 100, the improvement of SFPL over SOTA methods increases from 2.2% to 3.8%.

Three-dimensional reconstruction from multiview images is considered as a longstanding problem in computer vision and graphics. In order to achieve high-fidelity geometry and appearance of 3-D scenes, this article proposes a novel geometric object learning method for multiview reconstruction withfuzzy set theory. We establish anew neural 3D reconstruction theoretical framecalled neural fuzzy geometric representation (NeuFG), which is a special type of implicit representation of geometric scene that only takes value in [0, 1]. NeuFG is essentially a volume image, and thus can be visualized directly with the conventional volume rendering technique. Extensive experiments on two public datasets, i.e., DTU and BlendedMVS, show that our method has the ability of accurately reconstructing complex shapes with vivid geometric details, without the requirement of mask supervision. Both qualitative and quantitative comparisons demonstrate that the proposed method has superior performance over the state-of-the-art neural scene representation methods. The code will be released on GitHub soon.

A core challenge in survival analysis is to model the distribution of time-to-event data, where the event of interest may be a death, failure, or occurrence of a specific event. Previous studies have showed that ranking and maximum likelihood estimation loss functions are widely-used learning approaches for survival analysis. However, ranking loss only focus on the ranking of survival time and does not consider potential effect of samples’ exact survival time values. Furthermore, the maximum likelihood estimation is unbounded and easily subject to outliers (e.g., censored data), which may cause poor performance of modeling. To handle the complexities of learning process and exploit valuable survival time values, we propose a time-adaptive coordinate loss function, TripleSurv, to achieve adaptive adjustments by introducing the differences in the survival time between sample pairs into the ranking, which can encourage the model to quantitatively rank relative risk of pairs, ultimately enhancing the accuracy of predictions. Most importantly, the TripleSurv is proficient in quantifying the relative risk between samples by ranking ordering of pairs, and consider the time interval as a trade-off to calibrate the robustness of model over sample distribution. Our TripleSurv is evaluated on three real-world survival datasets and a public synthetic dataset. The results show that our method outperforms the state-of-the-art methods and exhibits good model performance and robustness on modeling various sophisticated data distributions with different censor rates.

Open-sided magnetic particle imaging (OS-MPI) has garnered significant interest due to its potential for interventional applications. However, the system matrix calibration in OS-MPI using sequential scans is a time-consuming task and susceptible to the low signal-to-noise ratio (SNR) resulting from the small calibration sample size. These challenges have hindered the practical implementation of system matrix-based reconstruction for sequentially scanned OS-MPI. To address these issues, we propose a novel calibration method, named sequen- tial scan-based single-dimension multi-voxel calibration (SS-SDMVC), to efficiently obtain a high-SNR system matrix. This method was implemented in a cylindrical field of view (FOV), where a bar calibration sample parallel to the field-free line (FFL) was shifted along a fixed radial direction. A standard image reconstruction process was also introduced to verify the feasibility of SS-SDMVC. Through simulations, we analyzed the effects of noise levels and scanner imperfections on the SS-SDMVC-based reconstruction and demonstrated its robustness. In experiments, we compared the imaging performance of SS-SDMVC and the sequential scan-based traditional cubic-FOV SMC. The results showed that SS-SDMVC reduced the number of measurements by a factor of 210.94 and achieved higher reconstruction quality. Therefore, SS-SDMVC is expected to improve the reconstruction quality of human- scale or high-gradient FFL MPI scanners.

Magnetic particle imaging (MPI) uses nonlinear response signals to noninvasively detect magnetic nanoparticles in space, and its quantitative properties hold promise for future precise quantitative treatments. In reconstruction, the system matrix based method necessitates suitable regularization terms, such as Tikhonov or non-negative fused lasso (NFL) regularization, to stabilize the solution. While NFL regularization offers clearer edge information than Tikhonov regularization, it carries a biased estimate of thel1penalty, leading to an underestimation of the reconstructed concentration and adversely affecting the quantitative properties. In this paper, a new nonconvex regularization method including min-max concave (MC) and total variation (TV) regularization is proposed. This method utilized MC penalty to provide nearly unbiased sparse constraints and adds the TV penalty to provide a uniform intensity distribution of images. By combining the alternating direction multiplication method (ADMM) and the two-step parameter selection method, a more accurate quantitative MPI reconstruction was realized. The performance of the proposed method was verified on the simulation data, the Open-MPI dataset, and measured data from a homemade MPI scanner. The results indicate that the proposed method achieves better image quality while maintaining the quantitative properties, thus overcoming the drawback of intensity underestimation by the NFL method while providing edge information. In particular, for the measured data, the proposed method reduced the relative error in the intensity of the reconstruction results from 28% to 8%.

Detecting pneumonia, especially coronavirus disease 2019 (COVID-19), from chest X-ray (CXR) images is one of the most effective ways for disease diagnosis and patient triage. The application of deep neural networks (DNNs) for CXR image classification is limited due to the small sample size of the well-curated data. To tackle this problem, this article proposes a distance transformation-based deep forest framework with hybrid-feature fusion (DTDF-HFF) for accurate CXR image classification. In our proposed method, hybrid features of CXR images are extracted in two ways: hand-crafted feature extraction and multigrained scanning. Different types of features are fed into different classifiers in the same layer of the deep forest (DF), and the prediction vector obtained at each layer is transformed to form distance vector based on a self-adaptive scheme. The distance vectors obtained by different classifiers are fused and concatenated with the original features, then input into the corresponding classifier at the next layer. The cascade grows until DTDF-HFF can no longer gain benefits from the new layer. We compare the proposed method with other methods on the public CXR datasets, and the experimental results show that the proposed method can achieve state-of-the art (SOTA) performance. The code will be made publicly available at https://github.com/hongqq/DTDF-HFF.

Multiscale features are of great importance in modern convolutional neural networks, showing consistent performance gains on numerous vision tasks. Therefore, many plug-and-play blocks are introduced to upgrade existing convolutional neural networks for stronger multiscale representation ability. However, the design of plug-and-play blocks is getting more and more complex, and these manually designed blocks are not optimal. In this work, we propose PP-NAS to develop plug-and-play blocks based on neural architecture search (NAS). Specifically, we design a new search space PPConv and develop a search algorithm consisting of one-level optimization, zero-one loss, and connection existence loss. PP-NAS minimizes the optimization gap between super-net and subarchitectures and can achieve good performance even without retraining. Extensive experiments on image classification, object detection, and semantic segmentation verify the superiority of PP-NAS over state-of-the-art CNNs (e.g., ResNet, ResNeXt, and Res2Net). Our code is available at https://github.com/ainieli/PP-NAS.

Segmentation of COVID-19 lesions can assist physicians in better diagnosis and treatment of COVID-19. However, there are few relevant studies due to the lack of detailed information and high-quality annotation in the COVID-19 dataset. To solve the above problem, we propose C2FVL, a Coarse-to-Fine segmentation framework via Vision-Language alignment to merge text information containing the number of lesions and specific locations of image information. Introducing text information allows the network to achieve better prediction results on challenging datasets. We conduct extensive experiments on two COVID-19 datasets, including chest X-ray and CT, and the results demonstrate that our proposed method outperforms other state-of-the-art segmentation methods.

Automatic data augmentation is a technique to automatically search for strategies for image transformations, which can improve the performance of different vision tasks. RandAugment (RA), one of the most widely used automatic data augmentations, achieves great success in different scales of models and datasets. However, RA randomly selects transformations with equivalent probabilities and applies a single magnitude for all transformations, which is suboptimal for different models and datasets. In this paper, we develop Differentiable RandAugment (DRA) to learn selecting weights and magnitudes of transformations for RA. The magnitude of each transformation is modeled following a normal distribution with both learnable mean and standard deviation. We also introduce the gradient of transformations to reduce the bias in gradient estimation and KL divergence as part of the loss to reduce the optimization gap. Experiments on CIFAR-10/100 and ImageNet demonstrate the efficiency and effectiveness of DRA. Searching for only 0.95 GPU hours on ImageNet, DRA can reach a Top-1 accuracy of 78.19% with ResNet-50, which outperforms RA by 0.28% under the same settings. Transfer learning on object detection also demonstrates the power of DRA. The proposed DRA is one of the few that surpasses RA on ImageNet and has great potential to be integrated into modern training pipelines to achieve state-of-the-art performance. Our code will be made publicly available for out-of-the-box use.

The rapid development of computational pathology has brought new opportunities for prognosis prediction using histopathological images. However, the existing deep learning frameworks lack exploration of the relationship between images and other prognostic information, resulting in poor interpretability. Tumor mutation burden (TMB) is a promising biomarker for predicting the survival outcomes of cancer patients, but its measurement is costly. Its heterogeneity may be reflected in histopathological images. Here, we report a two-step framework for prognostic prediction using whole-slide images (WSIs). First, the framework adopts a deep residual network to encode the phenotype of WSIs and classifies patient-level TMB by the deep features after aggregation and dimensionality reduction. Then, the patients' prognosis is stratified by the TMB-related information obtained during the classification model development. Deep learning feature extraction and TMB classification model construction are performed on an in-house dataset of 295 Haematoxylin & Eosin stained WSIs of clear cell renal cell carcinoma (ccRCC). The development and evaluation of prognostic biomarkers are performed on The Cancer Genome Atlas-Kidney ccRCC (TCGA-KIRC) project with 304 WSIs. Our framework achieves good performance for TMB classification with an area under the receiver operating characteristic curve (AUC) of 0.813 on the validation set. Through survival analysis, our proposed prognostic biomarkers can achieve significant stratification of patients' overall survival (P $< $ 0.05) and outperform the original TMB signature in risk stratification of patients with advanced disease. The results indicate the feasibility of mining TMB-related information from WSI to achieve stepwise prognosis prediction.

Neoadjuvant chemotherapy (NAC) is the primary method to reduce the burden of tumor and metastasis; in the treatment of breast cancer, it may provide additional opportunities for breast-conserving surgery. Preoperative assessment of pathological complete response (PCR) to NAC is important for developing individualized treatment approaches and predicting patient prognosis. Compared to magnetic resonance imaging (MRI) and mammography, ultrasonography (US) has the advantages of simplicity, flexibility, and real-time imaging. Moreover, it does not require radiation and can provide multi-time acquisition of the tumor during NAC treatment. Recently, deep learning radiomics models based on multi-time-point US images for the prediction of NAC effectiveness have been proposed. To further improve the prediction performance, we carefully designed four supporting modules for our proposed dual-input transformer (DiT): isolated tokens-to-token patch embedding module, shared position embedding, time embedding, and weighted average pooling feature representation modules. The design of each module considers the characteristics of the US images at multiple time points. We validated our model on our retrospective US dataset composed of 484 cases from two centers whose consistency is not sufficiently high. Patients were allocated to training (n = 297), validation (n = 99), and external test (n = 88) sets. The results show that our model can achieve better performance than the Siamese CNN and the standard tokens-to-token vision transformer without using multi-time-point images. The ablation study also proved the effectiveness of each module designed for DiT.

Identifying squamous cell carcinoma and adenocarcinoma subtypes of metastatic cervical lymphadenopathy (CLA) is critical for localizing the primary lesion and initiating timely therapy. B-mode ultrasound (BUS), color Doppler flow imaging (CDFI), ultrasound elastography (UE) and dynamic contrast-enhanced ultrasound provide effective tools for identification but synthesis of modality information is a challenge for clinicians. Therefore, based on deep learning, rationally fusing these modalities with clinical information to personalize the classification of metastatic CLA requires new explorations. In this paper, we propose Multi-step Modality Fusion Network (MSMFN) for multi-modal ultrasound fusion to identify histological subtypes of metastatic CLA. MSMFN can mine the unique features of each modality and fuse them in a hierarchical three-step process. Specifically, first, under the guidance of high-level BUS semantic feature maps, information in CDFI and UE is extracted by modality interaction, and the static imaging feature vector is obtained. Then, a self-supervised feature orthogonalization loss is introduced to help learn modality heterogeneity features while maintaining maximal task-consistent category distinguishability of modalities. Finally, six encoded clinical information are utilized to avoid prediction bias and improve prediction ability further. Our three-fold cross-validation experiments demonstrate that our method surpasses clinicians and other multi-modal fusion methods with an accuracy of 80.06%, a true-positive rate of 81.81%, and a true-negative rate of 80.00%. Our network provides a multi-modal ultrasound fusion framework that considers prior clinical knowledge and modality-specific characteristics. Our code will be available at: https://github.com/RichardSunnyMeng/MSMFN.

Magnetic particle imaging (MPI) is an emerging technique for determining magnetic nanoparticle distributions in biological tissues. Although system-matrix (SM)-based image reconstruction offers higher image quality than the X-space-based approach, the SM calibration measurement is time-consuming. Additionally, the SM should be recalibrated if the tracer's characteristics or the magnetic field environment change, and repeated SM measurement further increase the required labor and time. Therefore, fast SM calibration is essential for MPI. Existing calibration methods commonly treat each row of the SM as independent of the others, but the rows are inherently related through the coil channel and frequency index. As these two elements can be regarded as additional multimodal information, we leverage the transformer architecture with a self-attention mechanism to encode them. Although the transformer has shown superiority in multimodal fusion learning across several fields, its high complexity may lead to overfitting when labeled data are scarce. Compared with labeled SM (i.e., full size), low-resolution SM data can be easily obtained, and fully using such data may alleviate overfitting. Accordingly, we propose a pseudo-label-based progressive pretraining strategy to leverage unlabeled data. Our method outperforms existing calibration methods on a public real-world OpenMPI dataset and simulation dataset. Moreover, our method improves the resolution of two in-house MPI scanners without requiring full-size SM measurements. Ablation studies confirm the contributions of modeling SM inter-row relations and the proposed pretraining strategy.

Automatic segmentation of lung nodules on computed tomography (CT) images is challenging owing to the variability of morphology, location, and intensity. In addition, few segmentation methods can capture intra-nodular heterogeneity to assist lung nodule diagnosis. In this study, we propose an end-to-end architecture to perform fully automated segmentation of multiple types of lung nodules and generate intra-nodular heterogeneity images for clinical use. To this end, a hybrid loss is considered by introducing a Faster R-CNN model based on generalized intersection over union loss in generative adversarial network. The Lung Image Database Consortium image collection dataset, comprising 2,635 lung nodules, was combined with 3,200 lung nodules from five hospitals for this study. Compared with manual segmentation by radiologists, the proposed model obtained an average dice coefficient (DC) of 82.05% on the test dataset. Compared with U-net, NoduleNet, nnU-net, and other three models, the proposed method achieved comparable performance on lung nodule segmentation and generated more vivid and valid intra-nodular heterogeneity images, which are beneficial in radiological diagnosis. In an external test of 91 patients from another hospital, the proposed model achieved an average DC of 81.61%. The proposed method effectively addresses the challenges of inevitable human interaction and additional pre-processing procedures in the existing solutions for lung nodule segmentation. In addition, the results show that the intra-nodular heterogeneity images generated by the proposed model are suitable to facilitate lung nodule diagnosis in radiology.

The traditional finite element method-based fluorescence molecular tomography (FMT)/ X-ray computed tomography (XCT) imaging reconstruction suffers from complicated mesh generation and dual-modality image data fusion, which limits the application of in vivo imaging. To solve this problem, a novel standardized imaging space reconstruction (SISR) method for the quantitative determination of fluorescent probe distributions inside small animals was developed. In conjunction with a standardized dual-modality image data fusion technology, and novel reconstruction strategy based on Laplace regularization and L1-fused Lasso method, the in vivo distribution can be calculated rapidly and accurately, which enables standardized and algorithm-driven data process. We demonstrated the method's feasibility through numerical simulations and quantitatively monitored in vivo programmed death ligand 1 (PD-L1) expression in mouse tumor xenografts, and the results demonstrate that our proposed SISR can increase data throughput and reproducibility, which helps to realize the dynamically and accurately in vivo imaging.

Magnetic particle imaging (MPI) is a radiation-free vessel- and target-imaging modality that can sensitively detect nanoparticles. A static magnetic gradient field, referred to as a selection field, is required in MPI to provide a field-free region (FFR) for spatial encoding. The image resolution of MPI is closely related to the size of the FFR, which is determined by the selection field gradient amplitude. Because of the limitations of existing gradient coil hardware, the image resolution of MPI cannot satisfy the clinical requirements of human in vivo imaging. Pulsed excitation has been confirmed to improve the image resolution of MPI by breaking down the 'relaxation wall.' This work proposes the use of a pulsed waveform magnetic gradient from magnetic resonance imaging to further improve the image resolution of MPI. Through alignment of the gradient direction along the field-free line (FFL), each location on the FFL is able to have a unique excitation field strength that generates a specific relaxation-induced decay signal. Through excitation of nanoparticles on the FFL with many gradient profiles, a high-resolution, one-dimensional (1D) image can be reconstructed on the FFL. For larger magnetic nanoparticles, simulation results revealed that a pulsed excitation field with a greater flat portion generates a 1D bar pattern phantom image with a higher correlation and spatial resolution. With parallel FFL and gradient coil movements, high-resolution, two-dimensional (2D) Shepp-Logan phantom and brain vessel maps were reconstructed through repetition of the spatially resolved measurement of magnetic nanoparticles on the FFL.

Glioma grading during surgery can help clinical treatment planning and prognosis, but intraoperative pathological examination of frozen sections is limited by the long processing time and complex procedures. Near-infrared fluorescence imaging provides chances for fast and accurate real-time diagnosis. Recently, deep learning techniques have been actively explored for medical image analysis and disease diagnosis. However, issues of near-infrared fluorescence images, including small-scale, noise, and low-resolution, increase the difficulty of training a satisfying network. Multi-modal imaging can provide complementary information to boost model performance, but simultaneously designing a proper network and utilizing the information of multi-modal data is challenging. In this work, we propose a novel neural architecture search method DLS-DARTS to automatically search for network architectures to handle these issues. DLS-DARTS has two learnable stems for multi-modal low-level feature fusion and uses a modified perturbation-based derivation strategy to improve the performance on the area under the curve and accuracy. White light imaging and fluorescence imaging in the first near-infrared window (650-900 nm) and the second near-infrared window (1,000-1,700 nm) are applied to provide multi-modal information on glioma tissues. In the experiments on 1,115 surgical glioma specimens, DLS-DARTS achieved an area under the curve of 0.843 and an accuracy of 0.634, which outperformed manually designed convolutional neural networks including ResNet, PyramidNet, and EfficientNet, and a state-of-the-art neural architecture search method for multi-modal medical image classification. Our study demonstrates that DLS-DARTS has the potential to help neurosurgeons during surgery, showing high prospects in medical image analysis.

Cancelable biometrics is a powerful remedy for information leakage caused by the extensive usage of unprotected biometric data. Current measures usually suffer from deteriorated accuracy, which is known as the security–performance tradeoff. Motivated by these concerns, in this article, a novel cancelable fingerprint approach, i.e., Indexing-Min–Max (IMM) hashing, is proposed to securely transform a fixed-length fingerprint feature vector to a discrete index hashed code. IMM hashing is essentially established upon the min–max hash and further strengthened by the integration of the partial Hadamard transform, which alleviates performance deterioration while maintaining a high security level. Extensive experiments on FVC2002 and FVC2004 fingerprint datasets coupled with comprehensive theoretical analyses demonstrate the favorable accuracy and strong anti-attack resilience of the proposed method. Besides, compared to the unprotected counterpart, the matching precision of the protected templates yields little accuracy loss or even improved performance, which means the security–performance tradeoff is well handled. Furthermore, IMM hashing also meets the unlinkability and revocability requisites of cancelable biometrics.

Cardiac medical image segmentation plays an important role in the diagnosis and clinical treatment of cardiovascular diseases. However, due to the variability of cardiac anatomy and the ambiguity between cardiac substructures, it is still difficult to quickly segment the entire heart from medical images. Most of the current researches utilize neural network structure to perform whole heart segmentation. Although good segmentation accuracy has been achieved, it usually requires a long training time. This paper aims to build a new whole heart segmentation model based on Deep Forest, called Multi-Resolution Deep Forest Framework(MRDFF), which performs segmentation through two stages. In the first stage, the heart region is located by rough binary classification, and similarity screening is used to reduce redundancy. The second stage subdivides the heart substructures based on the results of the first stage and uses multi-scale fusion to achieve high segmentation accuracy. The experimental results conducted on the public data set MM-WHS show that under the same training data and configuration, our model can be trained in only 4.5 hours, which is about 1/2 of the training time of neural network models, and can reach the accuracy not lower than neural network models, which shows the feasibility and efficiency of our model. The code will be made publicly available at https://github.com/xufeixf/MRDFF.

Automatic semantic segmentation in 2D echocardiography is vital in clinical practice for assessing various cardiac functions and improving the diagnosis of cardiac diseases. However, two distinct problems have persisted in automatic segmentation in 2D echocardiography, namely the lack of an effective feature enhancement approach for contextual feature capture and lack of label coherence in category prediction for individual pixels. Therefore, in this study, we propose a deep learning model, called deep pyramid local attention neural network (PLANet), to improve the segmentation performance of automatic methods in 2D echocardiography. Specifically, we propose a pyramid local attention module to enhance features by capturing supporting information within compact and sparse neighboring contexts. We also propose a label coherence learning mechanism to promote prediction consistency for pixels and their neighbors by guiding the learning with explicit supervision signals. The proposed PLANet was extensively evaluated on the dataset of cardiac acquisitions for multi-structure ultrasound segmentation (CAMUS) and sub-EchoNet-Dynamic, which are two large-scale and public 2D echocardiography datasets. The experimental results show that PLANet performs better than traditional and deep learning-based segmentation methods on geometrical and clinical metrics. Moreover, PLANet can complete the segmentation of heart structures in 2D echocardiography in real time, indicating a potential to assist cardiologists accurately and efficiently.

Objective: Radiomics, an emerging tool for medical image analysis, is potential towards precisely characterizing gastric cancer (GC). Whether using one-slice 2D annotation or whole-volume 3D annotation remains a long-time debate, especially for heterogeneous GC. We comprehensively compared 2D and 3D radiomic features' representation and discrimination capacity regarding GC, via three tasks (TLNM, lymph node metastasis' prediction; TLVI, lymphovascular invasion's prediction; TpT, pT4 or other pT stages' classification). Methods: Four-center 539 GC patients were retrospectively enrolled and divided into the training and validation cohorts. From 2D or 3D regions of interest (ROIs) annotated by radiologists, radiomic features were extracted respectively. Feature selection and model construction procedures were customed for each combination of two modalities (2D or 3D) and three tasks. Subsequently, six machine learning models (ModelLNM2D, ModelLNM3D; ModelLVI2D, ModelLVI3Ds ModelpT2D,s ModelpT3D) were derived and evaluated to reflect modalities' performances in characterizing GC. Furthermore, we performed an auxiliary experiment to assess modalities' performances when resampling spacing different. Results: Regarding three tasks, the yielded areas under the curve (AUCs) were: ModelLNM2D's 0.712 (95% confidence interval, 0.613-0.811), ModelLNM3D's 0.680 (0.584-0.775); ModelLVI2D's 0.677 (0.595-0.761), ModelLVI3D's 0.615 (0.528-0.703); ModelpT2D's 0.840 (0.779-0.901), ModelpT3D's 0.813 (0.747-0.879). Moreover, the auxiliary experiment indicated that Models2Dare statistically advantageous than Models3Dwith different resampling spacings. Conclusion: Models constructed with 2D radiomic features revealed comparable performances with those constructed with 3D features in characterizing GC. Significance: Our work indicated that time-saving 2D annotation would be the better choice in GC, and provided a related reference to further radiomics-based researches.

OBJECTIVE: Coronavirus disease 2019 (COVID-19) has caused considerable morbidity and mortality, especially in patients with underlying health conditions. A precise prognostic tool to identify poor outcomes among such cases is desperately needed. METHODS: Total 400 COVID-19 patients with underlying health conditions were retrospectively recruited from 4 centers, including 54 dead cases (labeled as poor outcomes) and 346 patients discharged or hospitalized for at least 7 days since initial CT scan. Patients were allocated to a training set (n = 271), a test set (n = 68), and an external test set (n = 61). We proposed an initial CT-derived hybrid model by combining a 3D-ResNet10 based deep learning model and a quantitative 3D radiomics model to predict the probability of COVID-19 patients reaching poor outcome. The model performance was assessed by area under the receiver operating characteristic curve (AUC), survival analysis, and subgroup analysis. RESULTS: The hybrid model achieved AUCs of 0.876 (95% confidence interval: 0.752-0.999) and 0.864 (0.766-0.962) in test and external test sets, outperforming other models. The survival analysis verified the hybrid model as a significant risk factor for mortality (hazard ratio, 2.049 [1.462-2.871], P < 0.001) that could well stratify patients into high-risk and low-risk of reaching poor outcomes (P < 0.001). CONCLUSION: The hybrid model that combined deep learning and radiomics could accurately identify poor outcomes in COVID-19 patients with underlying health conditions from initial CT scans. The great risk stratification ability could help alert risk of death and allow for timely surveillance plans.

Gastric cancer (GC) is the third leading cause of cancer-associated deaths globally. Accurate risk prediction of the overall survival (OS) for GC patients shows significant prognostic value, which helps identify and classify patients into different risk groups to benefit from personalized treatment. Many methods based on machine learning algorithms have been widely explored to predict the risk of OS. However, the accuracy of risk prediction has been limited and remains a challenge with existing methods. Few studies have proposed a framework and pay attention to the low-level and high-level features separately for the risk prediction of OS based on computed tomography images of GC patients. To achieve high accuracy, we propose a multi-focus fusion convolutional neural network. The network focuses on low-level and high-level features, where a subnet to focus on lower-level features and the other enhanced subnet with lateral connection to focus on higher-level semantic features. Three independent datasets of 640 GC patients are used to assess our method. Our proposed network is evaluated by metrics of the concordance index and hazard ratio. Our network outperforms state-of-the-art methods with the highest concordance index and hazard ratio in independent validation and test sets. Our results prove that our architecture can unify the separate low-level and high-level features into a single framework, and can be a powerful method for accurate risk prediction of OS.

Fluorescence molecular tomography (FMT) is a new type of medical imaging technology that can quantitatively reconstruct the three-dimensional distribution of fluorescent probes in vivo. Traditional Lp norm regularization techniques used in FMT reconstruction often face problems such as over-sparseness, over-smoothness, spatial discontinuity, and poor robustness. To address these problems, this paper proposes an adaptive parameter search elastic net (APSEN) method that is based on elastic net regularization, using weight parameters to combine the L1 and L2 norms. For the selection of elastic net weight parameters, this approach introduces the L0 norm of valid reconstruction results and the L2 norm of the residual vector, which are used to adjust the weight parameters adaptively. To verify the proposed method, a series of numerical simulation experiments were performed using digital mice with tumors as experimental subjects, and in vivo experiments of liver tumors were also conducted. The results showed that, compared with the state-of-the-art methods with different light source sizes or distances, Gaussian noise of 5%-25%, and the brute-force parameter search method, the APSEN method has better location accuracy, spatial resolution, fluorescence yield recovery ability, morphological characteristics, and robustness. Furthermore, the in vivo experiments demonstrated the applicability of APSEN for FMT.

Improving the quality of image-guided radiation therapy requires the tracking of respiratory motion in ultrasound sequences. However, the low signal-to-noise ratio and the artifacts in ultrasound images make it difficult to track targets accurately and robustly. In this study, we propose a novel deep learning model, called a Cascaded One-shot Deformable Convolutional Neural Network (COSD-CNN), to track landmarks in real time in long ultrasound sequences. Specifically, we design a cascaded Siamese network structure to improve the tracking performance of CNN-based methods. We propose a one-shot deformable convolution module to enhance the robustness of the COSD-CNN to appearance variation in a meta-learning manner. Moreover, we design a simple and efficient unsupervised strategy to facilitate the network's training with a limited number of medical images, in which many corner points are selected from raw ultrasound images to learn network features with high generalizability. The proposed COSD-CNN has been extensively evaluated on the public Challenge on Liver UltraSound Tracking (CLUST) 2D dataset and on our own ultrasound image dataset from the First Affiliated Hospital of Sun Yat-sen University (FSYSU). Experiment results show that the proposed model can track a target through an ultrasound sequence with high accuracy and robustness. Our method achieves new state-of-the-art performance on the CLUST 2D benchmark set, indicating its strong potential for application in clinical practice.

Fusion peptide (FP) is a pivotal domain for the entry of retrovirus into host cells to continue self-replication. The crucial role indicates that FP is a promising drug target for therapeutic intervention. A FP model proposed in our previous work is relatively not efficient to predict FP in retroviruses. Thus in this work, we come up with a new computational model to predict FP domains in all the retroviruses. It basically predicts FP domains through recognizing their start and end sites separately with SVM method combing the hydrophobicity knowledge of the subdomain around furin cleavage site. The classification accuracy rates are 91.91, 91.20 and 89.13 percent respectively corresponding to jack-knife, 10-fold cross-validation and 5-fold cross-validation test. Second, this model discovered 69,753 and 493 putative FPs after scanning amino acid sequences and HERV DNA sequences both without FP annotations. Subsequently, a statistical analysis was performed on the 69,753 putative FP sequences, which confirms that FP is a hydrophobic domain. Lastly, we depicted the distribution of the 493 putative FP sequences on each human chromosome and each HERV family, which shows that FP of HERV probably has chromosome and family preference.

The papers in this special son focus on This special issue invited contributions reporting on methodological breakthroughs in artificial intelligence in ophthalmology, and systems and insights that make use of large-scale datasets linking across multiple imaging modalities, image phenotyping, and imaging omics. These papers presented in this special issue introduce the latest advances in the field of ophthalmic image analysis and informatics, which enable and drive the research, development, and application of key technologies into ocular healthcare.

OBJECTIVE: The coronavirus disease 2019 (COVID-19) is rapidly spreading inside China and internationally. We aimed to construct a model integrating information from radiomics and deep learning (DL) features to discriminate critical cases from severe cases of COVID-19 using computed tomography (CT) images. METHODS: We retrospectively enrolled 217 patients from three centers in China, including 82 patients with severe disease and 135 with critical disease. Patients were randomly divided into a training cohort (n = 174) and a test cohort (n = 43). We extracted 102 3-dimensional radiomic features from automatically segmented lung volume and selected the significant features. We also developed a 3-dimensional DL network based on center-cropped slices. Using multivariable logistic regression, we then created a merged model based on significant radiomic features and DL scores. We employed the area under the receiver operating characteristic curve (AUC) to evaluate the model's performance. We then conducted cross validation, stratified analysis, survival analysis, and decision curve analysis to evaluate the robustness of our method. RESULTS: The merged model can distinguish critical patients with AUCs of 0.909 (95% confidence interval [CI]: 0.859-0.952) and 0.861 (95% CI: 0.753-0.968) in the training and test cohorts, respectively. Stratified analysis indicated that our model was not affected by sex, age, or chronic disease. Moreover, the results of the merged model showed a strong correlation with patient outcomes. SIGNIFICANCE: A model combining radiomic and DL features of the lung could help distinguish critical cases from severe cases of COVID-19.

Since its outbreak in December 2019, the persistent coronavirus disease (COVID-19) became a global health emergency. It is imperative to develop a prognostic tool to identify high-risk patients and assist in the formulation of treatment plans. We retrospectively collected 366 severe or critical COVID-19 patients from four centers, including 70 patients who died within 14 days (labeled as high-risk patients) since their initial CT scan and 296 who survived more than 14 days or were cured (labeled as low-risk patients). We developed a 3D densely connected convolutional neural network (termed De-COVID19-Net) to predict the probability of COVID-19 patients belonging to the high-risk or low-risk group, combining CT and clinical information. The area under the curve (AUC) and other evaluation techniques were used to assess our model. The De-COVID19-Net yielded an AUC of 0.952 (95% confidence interval, 0.928-0.977) on the training set and 0.943 (0.904-0.981) on the test set. The stratified analyses indicated that our model's performance is independent of age, sex, and with/without chronic diseases. The Kaplan-Meier analysis revealed that our model could significantly categorize patients into high-risk and low-risk groups (p < 0.001). In conclusion, De-COVID19-Net can non-invasively predict whether a patient will die shortly based on the patient's initial CT scan with an impressive performance, which indicated that it could be used as a potential prognosis tool to alert high-risk patients and intervene in advance.

Fluorescence molecular tomography (FMT), which can visualize the distribution of fluorescence biomarkers, has become a novel three-dimensional noninvasive imaging technique for in vivo studies such as tumor detection and lymph node location. However, it remains a challenging problem to achieve satisfactory reconstruction performance of conventional FMT in the first near-infrared window (NIR-I, 700-900nm) because of the severe scattering of NIR-I light. In this study, a promising FMT method for heterogeneous mice was proposed to improve the reconstruction accuracy using the second near-infrared window (NIR-II, 1000-1700nm), where the light scattering significantly reduced compared with NIR-I. The optical properties of NIR-II were analyzed to construct the forward model for NIR-II FMT. Furthermore, to raise the accuracy of solution of the inverse problem, we proposed a novel Gaussian weighted neighborhood fused Lasso (GWNFL) method. Numerical simulation was performed to demonstrate the outperformance of GWNFL compared with other algorithms. Besides, a novel NIR-II/NIR-I dual-modality FMT system was developed to contrast the in vivo reconstruction performance between NIR-II FMT and NIR-I FMT. To compare the reconstruction performance of NIR-II FMT with traditional NIR-I FMT, numerical simulations and in vivo experiments were conducted. Both the simulation and in vivo results showed that NIR-II FMT outperformed NIR-I FMT in terms of location accuracy and spatial overlap index. It is believed that this study could promote the development and biomedical application of NIR-II FMT in the future.

Cerenkov luminescence tomography (CLT) is a promising imaging tool for obtaining three-dimensional (3D) non-invasive visualization of the in vivo distribution of radiopharmaceuticals. However, the reconstruction performance remains unsatisfactory for biomedical applications because the inverse problem of CLT is severely ill-conditioned and intractable. In this study, therefore, a novel non-negative iterative convex refinement (NNICR) approach was utilized to improve the CLT reconstruction accuracy, robustness as well as the shape recovery capability. The spike and slab prior information was employed to capture the sparsity of Cerenkov source, which could be formalized as a non-convex optimization problem. The NNICR approach solved this non-convex problem by refining the solutions of the convex sub-problems. To evaluate the performance of the NNICR approach, numerical simulations and in vivo tumor-bearing mice models experiments were conducted. Conjugated gradient based Tikhonov regularization approach (CG-Tikhonov), fast iterative shrinkage-thresholding algorithm based Lasso approach (Fista-Lasso) and Elastic-Net regularization approach were used for the comparison of the reconstruction performance. The results of these experiments demonstrated that the NNICR approach obtained superior reconstruction performance in terms of location accuracy, shape recovery capability, robustness and in vivo practicability. It was believed that this study would facilitate the preclinical and clinical applications of CLT in the future.

Fluorescence molecular tomography (FMT) is a highly sensitive and noninvasive imaging modality for three-dimensional visualization of fluorescence probe distribution in small animals. However, the simplified photon propagation model and ill-posed inverse problem limit the improvement of FMT reconstruction. In this work, we proposed a novel K-nearest neighbor based locally connected (KNN-LC) network to improve the performance of morphological reconstruction in FMT. It directly builds the inverse process of photon transmission by learning the mapping relation between the surface photon intensity and the distribution of fluorescent source. KNN-LC network cascades a fully connected (FC) sub-network with a locally connected (LC) sub-network, where the FC part provides a coarse reconstruction result and LC part fine-tunes the morphological quality of reconstructed result. To assess the performance of our proposed network, we implemented both numerical simulation and in vivo studies. Furthermore, split Bregman-resolved total variation (SBRTV) regularization method and inverse problem simulation (IPS) method were utilized as baselines in all comparisons. The results demonstrated that KNN-LC network achieved accurate reconstruction in both source localization and morphology recovery in a short time. This promoted the in vivo application of FMT for visualizing the distribution of biomarkers inside biological tissue.

Fluorescence molecular tomography (FMT), as a powerful imaging technique in preclinical research, can offer the three-dimensional distribution of biomarkers by detecting the fluorescently labelled probe noninvasively. However, because of the light scattering effect and the ill-pose of inverse problem, it is challenging to develop an efficient reconstruction method, which can provide accurate location and morphology of the fluorescence distribution. In this research, we proposed a novel adaptive Gaussian weighted Laplace prior (AGWLP) regularization method, which assumed the variance of fluorescence intensity between any two voxels had a non-linear correlation with their Gaussian distance. It utilized an adaptive Gaussian kernel parameter strategy to achieve accurate morphological reconstructions in FMT. To evaluate the performance of the AGWLP method, we conducted numerical simulation and in vivo experiments. The results were compared with fast iterative shrinkage (FIS) thresholding method, split Bregman-resolved TV (SBRTV) regularization method, and Gaussian weighted Laplace prior (GWLP) regularization method. We validated in vivo imaging results against planar fluorescence images of frozen sections. The results demonstrated that the AGWLP method achieved superior performance in both location and shape recovery of fluorescence distribution. This enabled FMT more suitable and practical for in vivo visualization of biomarkers.

Recent research indicates the critical importance of preserving local geometric structure of data in unsupervised feature selection (UFS), and the well studied graph Laplacian is usually deployed to capture this property. By using a squared l 2 -norm, we observe that conventional graph Laplacian is sensitive to noisy data, leading to unsatisfying data processing performance. To address this issue, we propose a unified UFS framework via feature self-representation and robust graph regularization , with the aim at reducing the sensitivity to outliers from the following two aspects: i) an l 2, 1 -norm is used to characterize the feature representation residual matrix; and ii) an l 1 -norm based graph Laplacian regularization term is adopted to preserve the local geometric structure of data. By this way, the proposed framework is able to reduce the effect of noisy data on feature selection. Furthermore, the proposed l 1 -norm based graph Laplacian is readily extendible, which can be easily integrated into other UFS methods and machine learning tasks with local geometrical structure of data being preserved. As demonstrated on ten challenging benchmark data sets, our algorithm significantly and consistently outperforms state-of-the-art UFS methods in the literature, suggesting the effectiveness of the proposed UFS framework.

Increasing prevalence of dementia has posed several challenges for care-givers. Patients suffering from dementia often display wandering behavior due to boredom or memory loss. It is considered to be one of the challenging conditions to manage and understand. Traits of dementia patients can compromise their safety causing serious injuries. This paper presents investigation into the design and evaluation of wandering scenarios with patients suffering from dementia using an S-band sensing technique. This frequency band is the wireless channel commonly used to monitor and characterize different scenarios including random, lapping, and pacing movements in an indoor environment. Wandering patterns are characterized depending on the received amplitude and phase information of that measures the disturbance caused in the ideal radio signal. A secondary analysis using support vector machine is used to classify the three patterns. The results show that the proposed technique carries high classification accuracy up to 90% and has good potential for healthcare applications.

In [1], the affiliation for Y. Gao, K. Wang and J. Tian should have appeared as follows:.

In[1], the affiliation for Y. Gao, K. Wang and J. Tian should have appeared as follows:.

Fluorescence molecular tomography (FMT), as a promising imaging modality in preclinical research, can obtain the three-dimensional (3-D) position information of the stem cell in mice. However, because of the ill-posed nature and sensitivity to noise of the inverse problem, it is a challenge to develop a robust reconstruction method, which can accurately locate the stem cells and define the distribution. In this paper, we proposed a sparsity adaptive correntropy matching pursuit (SACMP) method. SACMP method is independent on the noise distribution of measurements and it assigns small weights on severely corrupted entries of data and large weights on clean ones adaptively. These properties make it more suitable for in vivo experiment. To analyze the performance in terms of robustness and practicability of SACMP, we conducted numerical simulation and in vivo mice experiments. The results demonstrated that the SACMP method obtained the highest robustness and accuracy in locating stem cells and depicting stem cell distribution compared with stagewise orthogonal matching pursuit and sparsity adaptive subspace pursuit reconstruction methods. To the best of our knowledge, this is the first study that acquired such accurate and robust FMT distribution reconstruction for stem cell tracking in mice brain. This promotes the application of FMT in locating stem cell and distribution reconstruction in practical mice brain injury models.

Accurate lung nodule segmentation from computed tomography (CT) images is of great importance for image-driven lung cancer analysis. However, the heterogeneity of lung nodules and the presence of similar visual characteristics between nodules and their surroundings make it difficult for robust nodule segmentation. In this study, we propose a data-driven model, termed the Central Focused Convolutional Neural Networks (CF-CNN), to segment lung nodules from heterogeneous CT images. Our approach combines two key insights: 1) the proposed model captures a diverse set of nodule-sensitive features from both 3-D and 2-D CT images simultaneously; 2) when classifying an image voxel, the effects of its neighbor voxels can vary according to their spatial locations. We describe this phenomenon by proposing a novel central pooling layer retaining much information on voxel patch center, followed by a multi-scale patch learning strategy. Moreover, we design a weighted sampling to facilitate the model training, where training samples are selected according to their degree of segmentation difficulty. The proposed method has been extensively evaluated on the public LIDC dataset including 893 nodules and an independent dataset with 74 nodules from Guangdong General Hospital (GDGH). We showed that CF-CNN achieved superior segmentation performance with average dice scores of 82.15% and 80.02% for the two datasets respectively. Moreover, we compared our results with the inter-radiologists consistency on LIDC dataset, showing a difference in average dice score of only 1.98%.

Fluorescence Molecular Tomography (FMT) is a powerful imaging modality for the research of cancer diagnosis, disease treatment and drug discovery. Via three-dimensional (3-D) imaging reconstruction, it can quantitatively and noninvasively obtain the distribution of fluorescent probes in biological tissues. Currently, photon propagation of FMT is conventionally described by the Finite Element Method (FEM), and it can obtain acceptable image quality. However, there are still some inherent inadequacies in FEM, such as time consuming, discretization error and inflexibility in mesh generation, which partly limit its imaging accuracy. To further improve the solving accuracy of photon propagation model (PPM), we propose a novel compactly supported radial basis functions (CSRBFs)-based meshless method (MM) to implement the PPM of FMT. We introduced a series of independent nodes and continuous CSRBFs to interpolate the PPM, which can avoid complicated mesh generation. To analyze the performance of the proposed MM, we carried out numerical heterogeneous mouse simulation to validate the simulated surface fluorescent measurement. Then we performed an in vivo experiment to observe the tomographic reconstruction. The experimental results confirmed that our proposed MM could obtain more similar surface fluorescence measurement with the golden standard (Monte-Carlo method), and more accurate reconstruction result was achieved via MM in in vivo application.

Bioluminescence tomography (BLT) is a powerful non-invasive molecular imaging tool for in vivo studies of glioma in mice. However, because of the light scattering and resulted ill-posed problems, it is challenging to develop a sufficient reconstruction method, which can accurately locate the tumor and define the tumor morphology in three-dimension. In this paper, we proposed a novel Gaussian weighted Laplace prior (GWLP) regularization method. It considered the variance of the bioluminescence energy between any two voxels inside an organ had a non-linear inverse relationship with their Gaussian distance to solve the over-smoothed tumor morphology in BLT reconstruction. We compared the GWLP with conventional Tikhonov and Laplace regularization methods through various numerical simulations and in vivo orthotopic glioma mouse model experiments. The in vivo magnetic resonance imaging and ex vivo green fluorescent protein images and hematoxylin-eosin stained images of whole head cryoslicing specimens were utilized as gold standards. The results demonstrated that GWLP achieved the highest accuracy in tumor localization and tumor morphology preservation. To the best of our knowledge, this is the first study that achieved such accurate BLT morphological reconstruction of orthotopic glioma without using any segmented tumor structure from any other structural imaging modalities as the prior for reconstruction guidance. This enabled BLT more suitable and practical for in vivo imaging of orthotopic glioma mouse models.

Cerenkov luminescence tomography (CLT) provides a novel technique for 3-D noninvasive detection of radiopharmaceuticals in living subjects. However, because of the severe scattering of Cerenkov light, the reconstruction accuracy and stability of CLT is still unsatisfied. In this paper, a modified weight multispectral CLT (wmCLT) reconstruction strategy was developed which split the Cerenkov radiation spectrum into several sub-spectral bands and weighted the sub-spectral results to obtain the final result. To better evaluate the property of the wmCLT reconstruction strategy in terms of accuracy, stability and practicability, several numerical simulation experiments and in vivo experiments were conducted and the results obtained were compared with the traditional multispectral CLT (mCLT) and hybrid-spectral CLT (hCLT) reconstruction strategies. The numerical simulation results indicated that wmCLT strategy significantly improved the accuracy of Cerenkov source localization and intensity quantitation and exhibited good stability in suppressing noise in numerical simulation experiments. And the comparison of the results achieved from different in vivo experiments further indicated significant improvement of the wmCLT strategy in terms of the shape recovery of the bladder and the spatial resolution of imaging xenograft tumors. Overall the strategy reported here will facilitate the development of nuclear and optical molecular tomography in theoretical study.

Orientation Field (OF) is one of the most significant characters to distinguish fingerprint images from non-fingerprint images. An effective definition of fingerprint OF pattern will not only benefit fingerprint enhancement, but also contribute to latent fingerprint detection and segmentation. The existing fingerprint OF models either require pre-knowledge of singular points, or cannot be generalized to all kinds of fingerprint OFs. In this paper, we propose to define the fingerprint OF patterns based on low rank decomposition and sparse coding. Then we apply this proposed method to fingerprint OF recognition and detection. Experimental results prove the effectiveness of our method.

The computation of model matrix in the iterative imaging reconstruction process is crucial for the quantitative photoacoustic tomography (PAT). However, it is challenging to establish an outstanding model matrix to improve the overall imaging quality in PAT due to the noisy signal acquisition and inevitable artifacts. In this work, we present a novel method, named as the curve-driven-based model-matrix inversion (CDMMI), to calculate the model matrix for tomographic reconstruction in photoacoustic imaging. It eliminated the use of interpolation techniques, and thus avoided all interpolation related errors. The conventional interpolated-matrix-model inversion (IMMI) method was applied to evaluate its performance in numerical simulation, tissue-mimicking phantom and in vivo small animal studies. Results demonstrated that CDMMI achieved better reconstruction accuracy until IMMI kept increasing discrete points to 10000. Furthermore, the proposed method can suppress the negative influence of noise and artifacts effectively, which benefited the overall imaging quality of photoacoustic tomography.

The accurate segmentation of lung lesions from computed tomography (CT) scans is important for lung cancer research and can offer valuable information for clinical diagnosis and treatment. However, it is challenging to achieve a fully automatic lesion detection and segmentation with acceptable accuracy due to the heterogeneity of lung lesions. Here, we propose a novel toboggan based growing automatic segmentation approach (TBGA) with a three-step framework, which are automatic initial seed point selection, multi-constraints 3D lesion extraction and the final lesion refinement. The new approach does not require any human interaction or training dataset for lesion detection, yet it can provide a high lesion detection sensitivity (96.35%) and a comparable segmentation accuracy with manual segmentation (P > 0.05), which was proved by a series assessments using the LIDC-IDRI dataset (850 lesions) and in-house clinical dataset (121 lesions). We also compared TBGA with commonly used level set and skeleton graph cut methods, respectively. The results indicated a significant improvement of segmentation accuracy . Furthermore, the average time consumption for one lesion segmentation was under 8 s using our new method. In conclusion, we believe that the novel TBGA can achieve robust, efficient and accurate lung lesion segmentation in CT images automatically.

Liver segmentation is still a challenging task in medical image processing area due to the complexity of the liver's anatomy, low contrast with adjacent organs, and presence of pathologies. This investigation was used to develop and validate an automated method to segment livers in CT images. The proposed framework consists of three steps: 1) preprocessing; 2) initialization; and 3) segmentation. In the first step, a statistical shape model is constructed based on the principal component analysis and the input image is smoothed using curvature anisotropic diffusion filtering. In the second step, the mean shape model is moved using thresholding and Euclidean distance transformation to obtain a coarse position in a test image, and then the initial mesh is locally and iteratively deformed to the coarse boundary, which is constrained to stay close to a subspace of shapes describing the anatomical variability. Finally, in order to accurately detect the liver surface, deformable graph cut was proposed, which effectively integrates the properties and inter-relationship of the input images and initialized surface. The proposed method was evaluated on 50 CT scan images, which are publicly available in two databases Sliver07 and 3Dircadb. The experimental results showed that the proposed method was effective and accurate for detection of the liver surface.

Overlapped fingerprints are commonly encountered in latent fingerprints lifted from crime scenes. Such overlapped fingerprints can hardly be processed by state-of-the-art fingerprint matchers. Several methods have been proposed to separate the overlapped fingerprints. However, these methods neither provide a robust separation results, nor could be generalized to most overlapped fingerprints. In this paper, we propose a novel overlapped fingerprint separation algorithm based on adaptive orientation model fitting. Different from existing methods, our algorithm estimates the initial orientation fields in a more accurate way and then separates the orientation fields for component fingerprints through an iterative correction process. Experimental results on latent overlapped fingerprints database demonstrate the advantage of our algorithm.

Overlapped fingerprints are commonly encountered in latent fingerprints lifted from crime scenes. Such overlapped fingerprints can hardly be processed by state-of-the-art fingerprint matchers. Several methods have been proposed to separate the overlapped fingerprints. However, these methods neither provide robust separation results, nor could be generalized for most overlapped fingerprints. In this paper, we propose a novel latent overlapped fingerprints separation algorithm based on adaptive orientation model fitting. Different from existing methods, our algorithm estimates the initial orientation fields in a more accurate way and then separates the orientation fields for component fingerprints through an iterative correction process. Global orientation field models are used to predict and correct the orientations in overlapped regions. Experimental results on the latent overlapped fingerprints database show that the proposed algorithm outperforms the state-of-the-art algorithm in terms of accuracy.

With the availability of live-scan palmprint technology, high resolution palmprint recognition has started to receive significant attention in forensics and law enforcement. In forensic applications, latent palmprints provide critical evidence as it is estimated that about 30 percent of the latents recovered at crime scenes are those of palms. Most of the available high-resolution palmprint matching algorithms essentially follow the minutiae-based fingerprint matching strategy. Considering the large number of minutiae (about 1,000 minutiae in a full palmprint compared to about 100 minutiae in a rolled fingerprint) and large area of foreground region in full palmprints, novel strategies need to be developed for efficient and robust latent palmprint matching. In this paper, a coarse to fine matching strategy based on minutiae clustering and minutiae match propagation is designed specifically for palmprint matching. To deal with the large number of minutiae, a local feature-based minutiae clustering algorithm is designed to cluster minutiae into several groups such that minutiae belonging to the same group have similar local characteristics. The coarse matching is then performed within each cluster to establish initial minutiae correspondences between two palmprints. Starting with each initial correspondence, a minutiae match propagation algorithm searches for mated minutiae in the full palmprint. The proposed palmprint matching algorithm has been evaluated on a latent-to-full palmprint database consisting of 446 latents and 12,489 background full prints. The matching results show a rank-1 identification accuracy of 79.4 percent, which is significantly higher than the 60.8 percent identification accuracy of a state-of-the-art latent palmprint matching algorithm on the same latent database. The average computation time of our algorithm for a single latent-to-full match is about 141 ms for genuine match and 50 ms for impostor match, on a Windows XP desktop system with 2.2-GHz CPU and 1.00-GB RAM. The computation time of our algorithm is an order of magnitude faster than a previously published state-of-the-art-algorithm.

Finding the center of rotation is an essential step for accurate three-dimensional reconstruction in optical projection tomography (OPT). Unfortunately current methods are not convenient since they require either prior scanning of a reference phantom, small structures of high intensity existing in the specimen, or active participation during the centering procedure. To solve these problems this paper proposes a fast and automatic center of rotation search method making use of parallel programming in graphics processing units (GPUs). Our method is based on a two step search approach making use only of those sections of the image with high signal to noise ratio. We have tested this method both in non-scattering ex vivo samples and in in vivo specimens with a considerable contribution of scattering such as Drosophila melanogaster pupae, recovering in all cases the center of rotation with a precision 1/4 pixel or less.

Vasculature geometry reconstruction from volumetric medical data is a crucial task in the development of computer guided minimally invasive vascular surgery systems. In this paper, a technique for reconstructing the geometry of vasculatures using bivariate implicit splines is developed. With the proposed technique, an implicit geometry representation of the vascular tree can be accurately constructed based on the voxels extracted directly from the surface of a certain vascular structure in a given volumetric medical dataset. Experimental results show that the geometric representation built using our method can faithfully represent the morphology and topology of vascular structures. In addition, both the qualitative and the quantitative validations have been performed to show that the reconstructed vessel geometry is of high accuracy and smoothness. An virtual angioscopy system has been implemented to indicate one of the strengths of our proposed method.

In the above-named article (ibid., vol. 31, no. 10, pp. 1849-1860, Oct. 2012), the author name Jian Tian should have been Jie Tian.

In in vivo optical projection tomography (OPT), object motion will significantly reduce the quality and resolution of the reconstructed image. Based on the well-known Helgason-Ludwig consistency condition (HLCC), we propose a novel method for motion correction in OPT under parallel beam illumination. The method estimates object motion from projection data directly and does not require any other additional information, which results in a straightforward implementation. We decompose object movement into translation and rotation, and discuss how to correct for both translation and general motion simultaneously. Since finding the center of rotation accurately is critical in OPT, we also point out that the system's geometrical offset can be considered as object translation and therefore also calibrated through the translation estimation method. In order to verify the algorithm effectiveness, both simulated and in vivo OPT experiments are performed. Our results demonstrate that the proposed approach is capable of decreasing movement artifacts significantly thus providing high quality reconstructed images in the presence of object motion.

In volume rendering, most optical models currently in use are based on the assumptions that a volumetric object is a collection of particles and that the macro behavior of particles, when they interact with light rays, can be predicted based on the behavior of each individual particle. However, such models are not capable of characterizing the collective optical effect of a collection of particles which dominates the appearance of the boundaries of dense objects. In this paper, we propose a generalized optical model that combines particle elements and surface elements together to characterize both the behavior of individual particles and the collective effect of particles. The framework based on a new model provides a more powerful and flexible tool for hybrid rendering of isosurfaces and transparent clouds of particles in a single scene. It also provides a more rational basis for shading, so the problem of normal-based shading in homogeneous regions encountered in conventional volume rendering can be easily avoided. The model can be seen as an extension to the classical model. It can be implemented easily, and most of the advanced numerical estimation methods previously developed specifically for the particle-based optical model, such as preintegration, can be applied to the new model to achieve high-quality rendering results.

Traditional minutiae matching algorithms assume that each minutia has the same discriminability. However, this assumption is challenged by at least two facts. One of them is that fingerprint minutiae tend to form clusters, and minutiae points that are spatially close tend to have similar directions with each other. When two different fingerprints have similar clusters, there may be many well matched minutiae. The other one is that false minutiae may be extracted due to low quality fingerprint images, which result in both high false acceptance rate and high false rejection rate. In this paper, we analyze the minutiae discriminability from the viewpoint of global spatial distribution and local quality. Firstly, we propose an effective approach to detect such cluster minutiae which of low discriminability, and reduce corresponding minutiae similarity. Secondly, we use minutiae and their neighbors to estimate minutia quality and incorporate it into minutiae similarity calculation. Experimental results over FVC2004 and FVC-onGoing demonstrate that the proposed approaches are effective to improve matching performance.

This letter develops a novel method for fingerprint singular point detection based on a new singularity representation of ridge-valley region called orientation entropy. The candidate singular point is obtained by the multiple-scale analysis of orientation entropy and some post processing steps are proposed to filter the spurious core and delta points. An iteration compensation scheme is proposed to search the precise location for core points against the offset further. Performance of the proposed method has been evaluated on the dataset of FVC2002 DB1. Experimental results show that the multiple-scale orientation entropy is correct and effective for singular detection and the location compensation scheme reduces the distance between the detection result and the truth singular point.

Retinal image registration is crucial for the diagnoses and treatments of various eye diseases. A great number of methods have been developed to solve this problem; however, fast and accurate registration of low-quality retinal images is still a challenging problem since the low content contrast, large intensity variance as well as deterioration of unhealthy retina caused by various pathologies. This paper provides a new retinal image registration method based on salient feature region (SFR). We first propose a well-defined region saliency measure that consists of both local adaptive variance and gradient field entropy to extract the SFRs in each image. Next, an innovative local feature descriptor that combines gradient field distribution with corresponding geometric information is then computed to match the SFRs accurately. After that, normalized cross-correlation-based local rigid registration is performed on those matched SFRs to refine the accuracy of local alignment. Finally, the two images are registered by adopting high-order global transformation model with locally well-aligned region centers as control points. Experimental results show that our method is quite effective for retinal image registration.

Volume rendering has long been used as a key technique for volume data visualization, which works by using a transfer function to map color and opacity to each voxel. Many volume rendering approaches proposed so far for voxels classification have been limited in a single global transfer function, which is in general unable to properly visualize interested structures. In this paper, we propose a localized volume data visualization approach which regards volume visualization as a combination of two mutually related processes: the segmentation of interested structures and the visualization using a locally designed transfer function for each individual structure of interest. As shown in our work, a new interactive segmentation algorithm is advanced via skeletons to properly categorize interested structures. In addition, a localized transfer function is subsequently presented to assign optical parameters via interested information such as intensity, thickness and distance. As can be seen from the experimental results, the proposed techniques allow to appropriately visualize interested structures in highly complex volume medical data sets.

Accurate estimation of fingerprint orientation field is an essential module in fingerprint recognition. This paper proposes a novel technique for improving fingerprint orientation field estimation by fingerprint orientation model based on weighted 2D fourier expansion(W-FOMFE). The motivation for the proposed method can be found by: 1)the original FOMFE is sensitive to abrupt changes in orientation field; 2) blocks of different quality should have different impacts on FOMFE. Thus, we take into account the information of the Harris-corner strength (HCS) for orientation field estimation. In our method, we first calculate the fingerprint’ HCS; then use the HCS to remove abrupt changes in orientation field; finally, incorporate the normalized HCS as weighted value into original FOMFE. We test our method on FVC2004DB1. Experimental results show that our method (W-FOMFE) has better orientation field estimation than FOMFE.

The biometric template, which is stored in the form of raw data, has become the greatest potential threat to the security of biometric authentication system. As the compromise of the biometric data is permanent, the protection of biometric data is particularly important. Consequently, biometric template protection technologies have aroused research highlights recently. One of the most popular template protection methods is biometric cryptosystem method. In this paper, we design a codebook named distance projection for biometric coding to generate secured biometric template, and propose a novel fingerprint biometric cryptosystem scheme based on the codebook. Experimental results on FVC2002 DB2 show that the proposed scheme can obtain positive results on both security and authentication accuracy.

Visualized freehand 3-D ultrasound reconstruction offers to image incremental reconstruction during acquisition and guide users to scan interactively for high-quality volumes. We originally used the graphics processing unit (GPU) to develop a visualized reconstruction algorithm that achieves real-time level. Each newly acquired image was transferred to the memory of the GPU and inserted into the reconstruction volume on the GPU. The partially reconstructed volume was then rendered using GPU-based incremental ray casting. After visualized reconstruction, hole-filling was performed on the GPU to fill remaining empty voxels in the reconstruction volume. We examine the real-time nature of the algorithm using in vitro and in vivo datasets. The algorithm can image incremental reconstruction at speed of 26-58 frames/s and complete 3-D imaging in the acquisition time for the conventional freehand 3-D ultrasound.

Katsevich reconstruction algorithm represents a breakthrough for helical cone-beam computed tomography (CT) reconstruction, because it is the first exact cone-beam reconstruction algorithm of filtered backprojection (FBP) type with 1-D shift-invariant filtering. Although FBP-type reconstruction algorithm is effective, 3-D CT reconstruction is time-consuming, and the accelerations of Katsevich algorithm on CPU or cluster have been widely studied. In this paper, Katsevich algorithm is accelerated by using graphics processing unit, including flat-detector and curved-detector geometry in the case of helical orbit. An overscan formula is derived, which helps to avoid unnecessary overscan in practical CT scanning. Based on the overscan formula, a volume-blocking method in device memory is proposed. One advantage of the blocking method is that it can reconstruct large volume with high speed.

2D splines are a powerful tool for shape modeling, either parametrically or implicitly. However, compared with regular grid-based tensor-product splines, most of the high-dimensional spline techniques based on nonregular 2D polygons, such as box spline and simplex spline, are generally very expensive to evaluate. Though they have many desirable mathematical properties and have been proved theoretically to be powerful in graphics modeling, they are not a convenient graphics modeling technique in terms of practical implementation. In shape modeling practice, we still lack a simple and practical procedure in creating a set of bivariate spline basis functions from an arbitrarily specified 2D polygonal mesh. Solving this problem is of particular importance in using 2D algebraic splines for implicit modeling, as in this situation underlying implicit equations need to be solved quickly and accurately. In this article, a new type of bivariate spline function is introduced. This newly proposed type of bivariate spline function can be created from any given set of 2D polygons that partitions the 2D plane with any required degree of smoothness. In addition, the spline basis functions created with the proposed procedure are piecewise polynomials and can be described explicitly in analytical form. As a result, they can be evaluated efficiently and accurately. Furthermore, they have all the good properties of conventional 2D tensor-product-based B-spline basis functions, such as non-negativity, partition of unit, and convex-hull property. Apart from their obvious use in designing freeform parametric geometric shapes, the proposed 2D splines have been shown a powerful tool for implicit shape modeling.

Information fusion offers a promising solution to the development of a high-performance classification system. In this paper, the problem of multiple gait features fusion is explored with the framework of the factorial hidden Markov model (FHMM). The FHMM has a multiple-layer structure and provides an alternative to combine several gait features without concatenating them into a single augmented feature. Besides, the feature concatenation is used to directly concatenate the features and the parallel HMM (PHMM) is introduced as a decision-level fusion scheme, which employs traditional fusion rules to combine the recognition results at decision level. To evaluate the recognition performances, McNemar's test is employed to compare the FHMM feature-level fusion scheme with the feature concatenation and the PHMM decision-level fusion scheme. Statistical numerical experiments are carried out on the Carnegie Mellon University motion of body and the Institute of Automation of the Chinese Academy of Sciences gait databases. The experimental results demonstrate that the FHMM feature-level fusion scheme and the PHMM decision-level fusion scheme outperform feature concatenation. The FHMM feature-level fusion scheme tends to perform better than the PHMM decision-level fusion scheme when only a few gait cycles are available for recognition.

Because of the randomness of biometric features, traditional methods cannot perform well in the encryption of fingerprint template. The Fuzzy Vault construct is a biometric cryptosystem which can bind the fingerprint features with the secret key, and give reliable protection to both of them. The performance of Fuzzy Vault is highly affected by the alignment of fingerprints in the encryption field. In this paper, we propose a novel helper data based on the topo-structure around the core. The topo-structure has the advantage that it does not reveal any information about the template minutiae, and can reduce the alignment calculation amount. Experimental results show that the novel helper data has a good performance in alignment between vaults with high security.

This paper proposes a novel minutiae indexing method to speed up fingerprint matching, which narrows down the searching space of minutiae to reduce the expense of computation. An orderly sequence of features are extracted to describe each minutia and the indexing score is defined to select minutiae candidates from the query fingerprint for each minutia in the input fingerprint. The proposed method can be applied in both minutiae structure-based verification and fingerprint identification. Experiments are performed on a large-distorted fingerprint database (FVC2004 DB1) to approve the validity of the proposed method.

The design of software platform for medical imaging application has been increasingly prioritized as the sophisticated application of medical imaging. With this demand, we have designed and implemented a novel software platform in traditional object-oriented fashion with some common design patterns. This platform integrates the mainstream algorithms for medical image processing and analyzing within a consistent framework, including reconstruction, segmentation, registration, visualization, etc., and provides a powerful tool for both scientists and engineers. The overall framework and certain key technologies are introduced in detail. Presented experiment examples, numerous downloads, extensive uses, and practical applications commendably demonstrate the validity and flexibility of the platform.

Current certificate-based information security technologies are facing a challenge of lacking the exact connection between cryptographic key and legitimate users and a problem of comprehensive management of certificates. In this paper we propose a novel networking identity authentication scheme based on fingerprint coding method and Identity-based encryption (IBE). Fingerprint code is a dual-factor authenticator which combines a token, e.g. USB key, with user's fingerprint feature by direct mixing of pseudo-random number generated by token and fingerprint feature. IBE allows for a sender to encrypt a message to a receiver without access to a public key certificate. This scheme can improve the security of user authentication and simplify comprehensive management of certificates, in the meantime, protect against fingerprint data fabrication and stolen. The security and efficiency of our proposed scheme meet the requirements of practical applications.

Most of secure email systems adopt PKI and IBE encryption schemes to meet security demands in communications via emails, however, both PKI and IBE encryption schemes have their own shortcomings and flaws and consequently bring security problems to email systems. This paper proposes a new secure email system based on a fingerprint authentication scheme which combines fingerprint authentication technology with IBE scheme. The system perfectly solves the existing problems encountered in email security protection implementations.

This paper introduces a robust fingerprint matching scheme based on the comprehensive minutia and the binary relation between minutiae. In the method, a fingerprint is represented as a graph, of which the comprehensive minutiae act as the vertex set and the local binary minutia relations provide the edge set. Then, the transformation-invariant and transformation-variant features are extracted from the binary relation. The transformation-invariant features are suitable to estimate the local matching probability, whereas the transformation-variant features are used to model the fingerprint rotation transformation with the adaptive Parzen window. Finally, the fingerprint matching is conducted with the variable bounded box method and iterative strategy. The experiments demonstrate that the proposed scheme is effective and robust in fingerprint alignment and matching.

This paper introduces a novel algorithm based on global comprehensive similarity with three steps. To describe the Euclidean space-based relative features among minutiae, we first build a minutia-simplex that contains a pair of minutiae as well as their associated textures, with its transformation-variant and invariant relative features employed for the comprehensive similarity measurement and parameter estimation, respectively. By the second step, we use the ridge-based nearest neighborhood among minutiae to represent the ridge-based relative features among minutiae. With these ridge-based relative features, minutiae are grouped according to their affinity with a ridge. The Euclidean space-based and ridge-based relative features among minutiae reinforce each other in the representation of a fingerprint. Finally, we model the relationship between transformation and the comprehensive similarity between two fingerprints in terms of histogram for initial parameter estimation. Through these steps, our experiment shows that the method mentioned above is both effective and suitable for limited memory AFIS owing to its less than 1k byte template size.

Coping with nonlinear distortions in fingerprint matching is a challenging task. This paper proposes a novel method, a fuzzy feature match (FFM) based on a local triangle feature set to match the deformed fingerprints. The fingerprint is represented by the fuzzy feature set: the local triangle feature set. The similarity between the fuzzy feature set is used to characterize the similarity between fingerprints. A fuzzy similarity measure for two triangles is introduced and extended to construct a similarity vector including the triangle-level similarities for all triangles in two fingerprints. Accordingly, a similarity vector pair is defined to illustrate the similarities between two fingerprints. The FFM method maps the similarity vector pair to a normalized value which quantifies the overall image to image similarity. The proposed algorithm has been evaluated with NIST 24 and FVC2004 fingerprint databases. Experimental results confirm that the proposed FFM based on the local triangle feature set is a reliable and effective algorithm for fingerprint matching with nonlinear distortions.

Coping with nonlinear distortions in fingerprint matching is a challenging task. This paper proposes a novel algorithm, normalized fuzzy similarity measure (NFSM), to deal with the nonlinear distortions. The proposed algorithm has two main steps. First, the template and input fingerprints were aligned. In this process, the local topological structure matching was introduced to improve the robustness of global alignment. Second, the method NFSM was introduced to compute the similarity between the template and input fingerprints. The proposed algorithm was evaluated on fingerprints databases of FVC2004. Experimental results confirm that NFSM is a reliable and effective algorithm for fingerprint matching with nonliner distortions. The algorithm gives considerably higher matching scores compared to conventional matching algorithms for the deformed fingerprints.

A biometric information processing toolkit (BITK) is designed and implemented to support the biometrics research, e.g. algorithm development and performance evaluation. BITK is an object-oriented C++ software development toolkit (SDK), and it provides a consistent, flexible and reusable framework to integrate algorithms, data structures, and visualization methods. In addition, an application platform based on BITK (BITKAPP) is developed. BITKAPP makes the best of BITK to serve the biometrics researchers with its friendly user interface and the plug-in architecture. The meaningful applications of the toolkit and platform confirm that they effectively support the biometrics research.

As medical imaging applications become more complex, the design of software platforms for medical imaging is getting greater priority. With this demand, we have designed and implemented a novel software platform for medical data visualization in traditional object-oriented fashion with some common design patterns. This platform integrates the mainstream medical data reconstruction and visualization algorithms and 3D human-computer interaction based on 3D widgets into a consistent framework. The design goals, the overall framework and the implementation of some key technologies are addressed in details, and some application examples are also given to demonstrate the abilities of this platform. The ultimate objective is to provide a flexible, reliable and extensible 3D medical data visualization platform for the medical imaging society.

Abstract This paper introduces a fingerprint identification algorithm by clustering similarity with the view to overcome the dilemmas encountered in fingerprint identification. To decrease multi-spectrum noises in a fingerprint, we first use a dyadic scale space (DSS) method for image enhancement. The second step describes the relative features among minutiae by building a minutia-simplex which contains a pair of minutiae and their local associated ridge information, with its transformation-variant and invariant relative features applied for comprehensive similarity measurement and for parameter estimation respectively. The clustering method is employed to estimate the transformation space. Finally, multi-resolution technique is used to find an optimal transformation model for getting the maximal mutual information between the input and the template features. The experimental results including the performance evaluation by the 2nd International Verification Competition in 2002 (FVC2002), over the four fingerprint databases of FVC2002 indicate that our method is promising in an automatic fingerprint identification system (AFIS).

A fast surface reconstruction algorithm is proposed in this paper for processing large scale industrial CT images with high resolution. Through the following main steps: surface tracking in the single layer, a data caching mechanism and triangle strip generation, the algorithm can extract and represent surfaces efficiently on current mainstream PCs. The experimental results tested in real industrial CT datasets are also reported.

Presents a method of fingerprint classification. The method integrates a recognition system with a feedback mechanism, based on a genetic algorithm. The system was tested on 2000 images in the NIST-14 database. For the five-class problem, classification error was 6.0% without any rejects, and classification error approximated 1% with a 20% rejection rate. For the four-class problem (with two similar classes combined into the same class), classification error can be reduced to 5.2%. The results are better than those of the fingerprint classification systems created by Karu and Jian (1996) and by Blue et al. (1994). Through comparison experiments, it was illustrated that the feedback mechanism can give the recognition system the capability of adaptation to various inputs, and effectively improve its accuracy.

In this paper we present an interactive image segmentation method based on DP (dynamic programming), and extend the method in two aspects. First, we append a training mechanism to enhance the robustness of our approach to local noise; Second, we integrate the region-based segmentation with DP to improve the accuracy of image segmentation. A number of experiments show that our approach performs well on a variety of medical images.