National audience

Segment Anything Model (SAM) has demonstrated state-of-the-art performance in most segmentation tasks. However, due to insufficient training in the medical domain, SAM’s ability to generalize to medical images is limited. Although preliminary efforts have fine-tuned SAM for the medical domain, the fine-tuned model still struggles with variability in medical tasks. Some recent studies have explored weakly supervised learning to mitigate SAM’s performance degradation in the medical domain. However, the effectiveness of weakly supervised learning is heavily dependent on the quality of weakly supervised information, with performance significantly dropping as the quality declines. Doctors’ attention is closely related to the target area during diagnosis. Integrating gaze information into SAM’s adaptation process for medical image segmentation enhances efficiency and significantly improves performance in medical tasks. In this paper, we first propose a Gaze-assisted medical segment Anything Model (GAM), which utilizes gaze information to enable the adaptation of SAM in medical images following doctor’s attention. It has two innovations: 1) Feature-level adaptation: Gaze Alignment (GA) learning makes the feature-level adaptation follow the doctor’s attention which mines the human guidance from gaze heatmaps and guides model to extract general features for downstream tasks. 2) Output-level adaptation: Gaze-Balance (GB) learning makes the output-level adaptation follow the doctor’s attention which utilizes gaze heatmaps to enhance the human-focused area and solve the problem of over/under segmentation from the output-level. Our promising results on 7 tasks with 12 targets have demonstrated the powerful adaptation ability of our GAM in the medical domain. Our GAM demonstrates significant potential for low-cost clinical assistance in medical diagnosis, enabling SAM to adapt to the medical image domain without disrupting clinical workflows. We have released the full source code on https://github.com/Ruiz1026/GAM.

Medical imaging has developed from an auxiliary means of clinical examination into a significant method and intuitive basis for clinical diagnosis of diseases, providing all-around and full-cycle health protection for the people. The Internet of Medical Things (IoMT) allows medical equipment, intelligent terminals, medical infrastructure, and other elements of medical production to be interconnected, eliminating information silos and data fragmentation. Medical images disseminated in IoMT contain a wide diversity of sensitive patient information, which means protecting the patient's personal information is vital. In this work, an Adversarial-improved reversible steganography network (Airs-Net) for computed tomography (CT) images in the IoMT is presented. Specifically, the Airs-Net adopting the prediction-embedding strategy mainly consists of an image restoration network, an embedded pixel location network, and a discriminator. The image restoration network is effective in restoring the pixel prediction error of the restoration set in integer and non-integer scaled images of arbitrary size when information is concealed. The embedded information location network can automatically select pixel locations for information embedding based on the interpolated image features of the degraded image. The restored image, embedding location map, and embedding information are fed into the embedder for information embedding, and the subsequent secret-carrying image is continuously optimized for the quality of the information-embedded image by the discriminator. Quantitative results show that Airs-Net outperforms state-of-the-art methods in both PSNR and SSIM. Further, the qualitative and quantitative results and analyses under specific clinical application scenarios and in coping with multiple types of medical image information hiding demonstrate the excellent generalization performance and practical application capability of the Airs-Net.

Deformable image registration (DIR) is critical for accurate clinical diagnosis and effective treatment planning. However, patient movement, significant intensity differences, and large breathing deformations hinder accurate anatomical alignment in multi-modal image registration. These factors exacerbate the entanglement of anatomical and modality-specific style information, thereby severely limiting the performance of multi-modal registration. To address this, we propose a novel LAtent Diffusion-based Domain-Adaptive feature disentangling (LADDA) framework for unsupervised multi-modal medical image registration, which explicitly addresses the representation disentanglement. First, LADDA extracts reliable anatomical priors from the Latent Diffusion Model (LDM), facilitating downstream content-style disentangled learning. A Domain-Adaptive Feature Disentangling (DAFD) module is proposed to promote anatomical structure alignment further. This module disentangles image features into content and style information, boosting the network to focus on cross-modal content information. Next, a Neighborhood-Preserving Hashing (NPH) is constructed to further perceive and integrate hierarchical content information through local neighbourhood encoding, thereby maintaining cross-modal structural consistency. Furthermore, a Unilateral-Query-Frozen Attention (UQFA) module is proposed to enhance the coupling between upstream prior and downstream content information. The feature interaction within intra-domain consistent structures improves the fine recovery of detailed textures. The proposed framework is extensively evaluated on large-scale multi-center datasets, demonstrating superior performance across diverse clinical scenarios and strong generalization on out-of-distribution (OOD) data.

In the field of medical image processing, vascular image segmentation plays a crucial role in clinical diagnosis, treatment planning, prognosis, and medical decision-making. Accurate and automated segmentation of vascular images can assist clinicians in understanding the vascular network structure, leading to more informed medical decisions. However, manual annotation of vascular images is time-consuming and challenging due to the fine and low-contrast vascular branches, especially in the medical imaging domain where annotation requires specialized knowledge and clinical expertise. Data-driven deep learning models struggle to achieve good performance when only a small number of annotated vascular images are available. To address this issue, this paper proposes a novel Conditional Virtual Imaging (CVI) framework for few-shot vascular image segmentation learning. The framework combines limited annotated data with extensive unlabeled data to generate high-quality images, effectively improving the accuracy and robustness of segmentation learning. Our approach primarily includes two innovations: First, aligned image-mask pair generation, which leverages the powerful image generation capabilities of large pre-trained models to produce high-quality vascular images with complex structures using only a few training images; Second, the Dual-Consistency Learning (DCL) strategy, which simultaneously trains the generator and segmentation model, allowing them to learn from each other and maximize the utilization of limited data. Experimental results demonstrate that our CVI framework can generate high-quality medical images and effectively enhance the performance of segmentation models in few-shot scenarios. Our code will be made publicly available online.

Ultrasound image segmentation is critical for tumor screening but is hindered by noise, artifacts, and high variability in lesion appearance. Challenges like blurred boundaries and morphological similarities further complicate accurate delineation. To address this, we propose the Dynamic Edge-aware Transformer with Cross-image Patch Dependency (DET-CPD). Our model integrates two key modules: a Dynamic Difference Convolution Module (DDCM) to enhance edge representation for varied lesions, and a Cross-Scale Semantic Enhancement Module (CSEM) that leverages cross-scale channel information to distinguish tumors from surrounding tissue. Crucially, we introduce a novel Cross-image Patch Dependency Loss (CPDLoss) that captures semantic dependencies across different images in a batch, improving robustness. Extensive experiments on four public datasets (BUSI, DatasetB, DDTI, and TN3K) demonstrate that DET-CPD achieves state-of-the-art segmentation performance.

Dual-energy CT exploits the different attenuation characteristics of substances under different energy X-rays and collects high- and low-energy data from the same area to differentiate and quantify specific substances, which is now widely used in clinical diagnosis, disease monitoring, and other fields. If the object scanned during dual-energy CT imaging contains metallic material, the reconstructed image will suffer from metal artifacts. Metal artifacts in dual-energy CT result in surrounding tissue structures presenting erroneous CT values, blurred image details, and unclear border demarcation lines, which seriously affects the quality of the images as well as the accuracy of clinical diagnosis. To reduce metal artifacts in dual-energy CT, we combine material decomposition techniques with metal artifact reduction to explore metal artifact reduction methods applied to dual-energy CT.

In some orthopedic surgeries, the use of three-dimensional (3D) computed tomography (CT) scanning technology is not feasible due to scene limitations, leaving doctors to rely on two-dimensional (2D) X-ray images for real-time diagnosis. However, X-ray images lack 3D information, making accurate diagnosis challenging. Developing an algorithm to convert 2D X-ray images into 3D CT images, while simultaneously combining high-quality 3D reconstruction with precise fracture segmentation, offers a promising solution to the problem. In this study, we propose a novel artificial intelligence (AI)-driven framework named 3D reconstruction and segment anything model (3DRecSAM). The reconstruction image enhancer (RIE) is designed to achieve high-precision 3D reconstruction and provide high-quality feature initialization for fracture segmentation. Meanwhile, the mamba segment anything model (MSAM), based on the segment anything model (SAM) architecture, is developed for accurate fracture segmentation. We introduce a Kolmogorov–Arnold network (KAN)-based attention fusion module (KAF), which facilitates the joint optimization of the RIE reconstruction network and the MSAM segmentation network. Furthermore, the selective scanning mamba with KAN (SKM) is incorporated to enhance feature extraction for both RIE and MSAM. Mamba efficiently captures long-range dependencies and sequential patterns, while KAN’s learnable activation functions facilitate adaptive feature fusion and non-linear representation. To train and evaluate 3DRecSAM, we introduce the real X-ray and CT paired dataset (XCPData), which is publicly available on GitHub: https://github.com/YuanGao1201/XCPData .

4D cone-beam computed tomography (CBCT) plays a critical role in adaptive radiation therapy for lung cancer. However, extremely sparse sampling projection data will cause severe streak artifacts in 4D CBCT images. Existing deep learning (DL) methods heavily rely on large labeled training datasets which are difficult to obtain in practical scenarios. Restricted by this dilemma, DL models often struggle with simultaneously retaining dynamic motions, removing streak degradations, and recovering fine details. To address the above challenging problem, we introduce a Deep Prior Image Constrained Motion Compensation framework (DPI-MoCo) that decouples the 4D CBCT reconstruction into two sub-tasks including coarse image restoration and structural detail fine-tuning. In the first stage, the proposed DPI-MoCo combines the prior image guidance, generative adversarial network, and contrastive learning to globally suppress the artifacts while maintaining the respiratory movements. After that, to further enhance the local anatomical structures, the motion estimation and compensation technique is adopted. Notably, our framework is performed without the need for paired datasets, ensuring practicality in clinical cases. In the Monte Carlo simulation dataset, the DPI-MoCo achieves competitive quantitative performance compared to the state-of-the-art (SOTA) methods. Furthermore, we test DPI-MoCo in clinical lung cancer datasets, and experiments validate that DPI-MoCo not only restores small anatomical structures and lesions but also preserves motion information.

Improving the information reliability of the audio system is critical to safeguarding the security of the audio system. Adversarial samples crafted by in-the-wild attackers by introducing perturbations to the audio become a severe threat to the trustworthiness of deep learning-based classifiers. To achieve dynamic defence against audio adversarial sample attacks, a low-resolution double deep audio waveform prior network (LowDDAWP-Net) for audio systems reliability defence is proposed. Specifically, LowDDAWP-Net consists of a noise audio prior extraction module ($\mathbf{DAWP}_{\mathbf{noise}}$), an speech prior extraction module ($\mathbf{DAWP}_{\mathbf{speech}}$), a low-resolution extraction module (LREM), and a voice activity detection module (VADM). The role of the VADM is to automatically detect voice activity signals and silent signals from the audio signal.$\mathbf{DAWP}_{\mathbf{speech}}$and$\mathbf{DAWP}_{\mathbf{noise}}$are encoder–decoders with the same architecture. The encoder extracts the superficial features of the input audio, and the decoder performs temporal fusion to form high-dimensional features and reconstructs them into waveform signals. A LREM is employed to extract low-resolution audio to facilitate the encoder–decoder to perform detail on low-resolution audio and to speed up the recovery of DAWP networks to high resolution. The adversarial samples generated by several diverse attack state-of-the-art on three different datasets and their corresponding benign samples form a novel private dataset. The qualitative and quantitative results of the novel private dataset demonstrate the effectiveness and superiority of LowDDAWP-Net.

Internet of Medical Things (IoMT) and telemedicine technologies utilize computers, communications, and medical devices to facilitate off-site exchanges between specialists and patients, specialists, and medical staff. If the information communicated in IoMT is illegally steganography, tampered or leaked during transmission and storage, it will directly impact patient privacy or the consultation results with possible serious medical incidents. Steganalysis is of great significance for the identification of medical images transmitted illegally in IoMT and telemedicine. In this article, we propose a Residual and Enhanced Discriminative Network (RED-Net) for image steganalysis in the internet of medical things and telemedicine. RED-Net consists of a steganographic information enhancement module, a deep residual network, and steganographic information discriminative mechanism. Specifically, a steganographic information enhancement module is adopted by the RED-Net to boost the illegal steganographic signal in texturally complex high-dimensional medical image features. A deep residual network is utilized for steganographic feature extraction and compression. A steganographic information discriminative mechanism is employed by the deep residual network to enable it to recalibrate the steganographic features and drop high-frequency features that are mistaken for steganographic information. Experiments conducted on public and private datasets with data hiding payloads ranging from 0.1bpp/bpnzac-0.5bpp/bpnzac in the spatial and JPEG domain led to RED-Net's steganalysis error$P_{\mathrm{E}}$in the range of 0.0732-0.0010 and 0.231-0.026, respectively. In general, qualitative and quantitative results on public and private datasets demonstrate that the RED-Net outperforms 8 state-of-art steganography detectors.

Quantitative infarct estimation is crucial for diagnosis, treatment and prognosis in acute ischemic stroke (AIS) patients. As the early changes of ischemic tissue are subtle and easily confounded by normal brain tissue, it remains a very challenging task. However, existing methods often ignore or confuse the contribution of different types of anatomical asymmetry caused by intrinsic and pathological changes to segmentation. Further, inefficient domain knowledge utilization leads to mis-segmentation for AIS infarcts. Inspired by this idea, we propose a pathological asymmetry-guided progressive learning (PAPL) method for AIS infarct segmentation. PAPL mimics the step-by-step learning patterns observed in humans, including three progressive stages: knowledge preparation stage, formal learning stage, and examination improvement stage. First, knowledge preparation stage accumulates the preparatory domain knowledge of the infarct segmentation task, helping to learn domain-specific knowledge representations to enhance the discriminative ability for pathological asymmetries by constructed contrastive learning task. Then, formal learning stage efficiently performs end-to-end training guided by learned knowledge representations, in which the designed feature compensation module (FCM) can leverage the anatomy similarity between adjacent slices from the volumetric medical image to help aggregate rich anatomical context information. Finally, examination improvement stage encourages improving the infarct prediction from the previous stage, where the proposed perception refinement strategy (RPRS) further exploits the bilateral difference comparison to correct the mis-segmentation infarct regions by adaptively regional shrink and expansion. Extensive experiments on public and in-house NCCT datasets demonstrated the superiority of the proposed PAPL, which is promising to help better stroke evaluation and treatment.

In this work, we address the task of few-shot medical image segmentation (MIS) with a novel proposed framework based on the learning registration to learn segmentation (LRLS) paradigm. To cope with the limitations of lack of authenticity, diversity, and robustness in the existing LRLS frameworks, we propose the better registration better segmentation (BRBS) framework with three main contributions that are experimentally shown to have substantial practical merit. First, we improve the authenticity in the registration-based generation program and propose the knowledge consistency constraint strategy that constrains the registration network to learn according to the domain knowledge. It brings the semantic-aligned and topology-preserved registration, thus allowing the generation program to output new data with great space and style authenticity. Second, we deeply studied the diversity of the generation process and propose the space-style sampling program, which introduces the modeling of the transformation path of style and space change between few atlases and numerous unlabeled images into the generation program. Therefore, the sampling on the transformation paths provides much more diverse space and style features to the generated data effectively improving the diversity. Third, we first highlight the robustness in the learning of segmentation in the LRLS paradigm and propose the mix misalignment regularization, which simulates the misalignment distortion and constrains the network to reduce the fitting degree of misaligned regions. Therefore, it builds regularization for these regions improving the robustness of segmentation learning. Without any bells and whistles, our approach achieves a new state-of-the-art performance in few-shot MIS on two challenging tasks that outperform the existing LRLS-based few-shot methods. We believe that this novel and effective framework will provide a powerful few-shot benchmark for the field of medical image and efficiently reduce the costs of medical image research. All of our code will be made publicly available online.

Multi-modal information plays a pivotal role in the segmentation of brain tumors. However, previous studies have largely overlooked the distinctive characteristics of individual modalities, which are correlated with the target tumor region due to distinct imaging principles. In this paper, we harness the distinctive traits of individual modalities and introduce a brain tumor segmentation model called specific modality guided brain tumor segmentation model (SMG-BTS). Our SMG-BTS adopts a three-branch encoder-decoder architecture. The main branch utilizes full modalities fused at input-level, while the two affiliated branches operate in parallel to provide guidance to the main branch in acquiring a robust representation. We propose a specific modality spatial guidance (SMSG) module to guide the process of feature extraction. Spatial information is obtained from selected modalities and utilized to enhance features extracted from the main branch. A shared-specific feature enhancement(SSFE) module is proposed to enhance the shared features across modalities and utilizes modality-specific features to further supplement specific information of modalities. Experimental results on the BraTS2021 benchmark dataset demonstrate the effectiveness of our proposed SMG-BTS over state-of-the-art brain tumor segmentation methods.

Learning inter-image similarity is crucial for 3D medical images self-supervised pre-training, due to their sharing of numerous same semantic regions. However, the lack of the semantic prior in metrics and the semantic-independent variation in 3D medical images make it challenging to get a reliable measurement for the inter-image similarity, hindering the learning of consistent representation for same semantics. We investigate the challenging problem of this task, i.e., learning a consistent representation between images for a clustering effect of same semantic features. We propose a novel visual similarity learning paradigm, Geometric Visual Similarity Learning, which embeds the prior of topological invariance into the measurement of the inter-image similarity for consistent representation of semantic regions. To drive this paradigm, we further construct a novel geometric matching head, the Z-matching head, to collaboratively learn the global and local similarity of semantic regions, guiding the efficient representation learning for different scale-level inter-image semantic features. Our experiments demonstrate that the pre-training with our learning of inter-image similarity yields more powerful inner-scene, inter-scene, and global-local transferring ability on four challenging 3D medical image tasks. Our codes and pre-trained models will be publicly available11https://github.com/YutingHe-list/GVSL.

Graph contrastive learning has gained popularity due to its success in self-supervised graph representation learning. Augmented views in contrastive learning greatly determine the quality of the learned representations. Handcrafted data augmentations in previous work require tedious trial-and- errors per dataset, which is time-consuming and resource-intensive. Here, we propose a Graph Contrastive learning framework with Learnable graph Augmentation called GraphCLA. Specifically, learnable graph augmentation trains augmented views by minimizing the mutual information (MI) between the input graphs and their augmented graphs. This paper designs a kernel contrastive loss function based on an end-to-end differentiable graph kernel to learn augmented views. In addition, this paper utilizes a min-max optimization strategy to learn challenging augmented graphs and to learn discriminative representations adversarially. Finally, we compared GraphCLA with state-of-the-art self-supervised learning baselines and experimentally validate the effectiveness of GraphCLA.

Pulmonary arterial hypertension (PAH) prognosis prediction on 3D non-contrast CT images is one of the most important tasks for PAH treatment. It will help clinicians stratify patients into different groups for early diagnosis and timely intervention via automatically extracting the potential biomarkers of PAH to predict mortality. However, it is still a task of great challenges due to the large volume and low-contrast regions of interest in 3D chest CT images. In this paper, we propose the first multi-task learning-based PAH prognosis prediction framework, P$^{2}$-Net, which effectively optimizes the model and powerfully represents task-dependent features via our Memory Drift (MD) and Prior Prompt Learning (PPL) strategies. 1) Our MD maintains a large memory bank to provide a dense sampling of the deep biomarkers' distribution. Therefore, although the batch size is very small caused by our large volume, a reliable (negative log partial) likelihood loss is still able to be calculated on a representative probability distribution for robust optimization. 2) Our PPL simultaneously learns an additional manual biomarkers prediction task to embed clinical prior knowledge into our deep prognosis prediction task in hidden and explicit ways. Therefore, it will prompt the prediction of deep biomarkers and improve the perception of task-dependent features in our low-contrast regions. Our P$^{2}$-Net achieves a high prognostic correlation of the prediction and great generalization with the highest 70.19% C-index and 2.14 HR. Extensive experiments with promising results on our PAH prognosis prediction reveal powerful prognosis performance and great clinical significance in PAH treatment. All of our code will be made publicly available online.

Low-dose computed tomography (LDCT) imaging faces great challenges. Although supervised learning has revealed great potential, it requires sufficient and high-quality references for network training. Therefore, existing deep learning methods have been sparingly applied in clinical practice. To this end, this paper presents a novel Unsharp Structure Guided Filtering (USGF) method, which can reconstruct high-quality CT images directly from low-dose projections without clean references. Specifically, we first employ low-pass filters to estimate the structure priors from the input LDCT images. Then, inspired by classical structure transfer techniques, deep convolutional networks are adopted to implement our imaging method which combines guided filtering and structure transfer. Finally, the structure priors serve as the guidance images to alleviate over-smoothing, as they can transfer specific structural characteristics to the generated images. Furthermore, we incorporate traditional FBP algorithms into self-supervised training to enable the transformation of projection domain data to the image domain. Extensive comparisons and analyses on three datasets demonstrate that the proposed USGF has achieved superior performance in terms of noise suppression and edge preservation, and could have a significant impact on LDCT imaging in the future.

The nature of thick-slice scanning causes severe inter-slice discontinuities of 3D medical images, and the vanilla 2D/3D convolutional neural networks (CNNs) fail to represent sparse inter-slice information and dense intra-slice information in a balanced way, leading to severe underfitting to inter-slice features (for vanilla 2D CNNs) and overfitting to noise from long-range slices (for vanilla 3D CNNs). In this work, a novel mesh network (MNet) is proposed to balance the spatial representation inter axes via learning. 1) Our MNet latently fuses plenty of representation processes by embedding multi-dimensional convolutions deeply into basic modules, making the selections of representation processes flexible, thus balancing representation for sparse inter-slice information and dense intra-slice information adaptively. 2) Our MNet latently fuses multi-dimensional features inside each basic module, simultaneously taking the advantages of 2D (high segmentation accuracy of the easily recognized regions in 2D view) and 3D (high smoothness of 3D organ contour) representations, thus obtaining more accurate modeling for target regions. Comprehensive experiments are performed on four public datasets (CT\&MR), the results consistently demonstrate the proposed MNet outperforms the other methods. The code and datasets are available at: https://github.com/zfdong-code/MNet

4D cone-beam computed tomography (CBCT) is an important imaging modality in image-guided radiation therapy to address the motion-induced artifacts caused by organ movements during the respiratory process. However, due to the extremely sparse projection data for each temporal phase, 4D CBCT reconstructions will suffer from severe streaking artifacts. Therefore, to tackle the streak artifacts and provide high-quality images, we proposed a framework termed Prior-Regularized Iterative Optimization Reconstruction (PRIOR) for 4D CBCT. The PRIOR framework combines the physics-based model and data-driven method simultaneously, with powerful feature extracting capacity, significantly promoting the image quality compared to single model-based or deep learning-based methods. Besides, we designed a specialized deep learning model named PRIOR-Net, which can effectively excavate the static information in the prior image reconstructed from the fully-sampled projections at the encoding stage to improve the reconstruction performance for individual phase-resolved images. Both the simulated and clinical 4D CBCT datasets were performed to evaluate the performance of the PRIOR-Net and the PRIOR framework. Compared with the advanced 4D CBCT reconstruction methods, the proposed methods achieve promising results quantitatively and qualitatively in streak artifact suppression, soft tissue restoration, and tiny detail preservation.

Accurate and robust cephalometric image analysis plays an essential role in orthodontic diagnosis, treatment assessment and surgical planning. This paper proposes a novel landmark localization method for cephalometric analysis using multiscale image patch-based graph convolutional networks. In detail, image patches with the same size are hierarchically sampled from the Gaussian pyramid to well preserve multiscale context information. We combine local appearance and shape information into spatialized features with an attention module to enrich node representations in graph. The spatial relationships of landmarks are built with the incorporation of three-layer graph convolutional networks, and multiple landmarks are simultaneously updated and moved toward the targets in a cascaded coarse-to-fine process. Quantitative results obtained on publicly available cephalometric X-ray images have exhibited superior performance compared with other state-of-the-art methods in terms of mean radial error and successful detection rate within various precision ranges. Our approach performs significantly better especially in the clinically accepted range of 2 mm and this makes it suitable in cephalometric analysis and orthognathic surgery.

The deep learning-based automatic recognition of the scanning or exposing region in medical imaging automation is a promising new technique, which can decrease the heavy workload of the radiographers, optimize imaging workflow and improve image quality. However, there is little related research and practice in X-ray imaging. In this paper, we focus on two key problems in X-ray imaging automation: automatic recognition of the exposure moment and the exposure region. Consequently, we propose an automatic video analysis framework based on the hybrid model, approaching real-time performance. The framework consists of three interdependent components: Body Structure Detection, Motion State Tracing, and Body Modeling. Body Structure Detection disassembles the patient to obtain the corresponding body keypoints and body Bboxes. Combining and analyzing the two different types of body structure representations is to obtain rich spatial location information about the patient body structure. Motion State Tracing focuses on the motion state analysis of the exposure region to recognize the appropriate exposure moment. The exposure region is calculated by Body Modeling when the exposure moment appears. A large-scale dataset for X-ray examination scene is built to validate the performance of the proposed method. Extensive experiments demonstrate the superiority of the proposed method in automatically recognizing the exposure moment and exposure region. This paradigm provides the first method that can enable automatically and accurately recognize the exposure region in X-ray imaging without the help of the radiographer.

Receptive field-based segmentation models represent features in receptive fields having weak perception for thin semantics in thin structures segmentation, due to the challenges in small local size and large global variation. High-frequency (HiFe) components have strong thin perception ability and is stable for global variation, but its weak adaptability limits its direct application. We propose a HiFe priori rule which enables the network to adaptively extract and fuse HiFe components, enhancing the thin semantics and making the network naturally prefer thin structures for their segmentation. We further propose High-Frequency Semantics Enhancement Network (HiFeNet) based on our HiFe priori rule, boosting the SOTA methods in thin structures segmentation: 1) Our Deep High Frequency (DHiFe) block learns to extract task-dependent HiFe components and adds them to feature maps, achieving great perception of thin structures. 2) Our Latent Residual Denoising (LRD) block progressively weakens task-independent features via hierarchical residuals and learns to fuse HiFe components back to feature maps, further enhancing the thin semantics and weakening the interference of global variation. Extensive experiments on the retinal vessel [1], [2], [3] and Massachusetts road [4] segmentation datasets show great superiority of our HiFeNet.

Accurate coronary lumen segmentation on coronary-computed tomography angiography (CCTA) images is crucial for quantification of coronary stenosis and the subsequent computation of fractional flow reserve. Many factors including difficulty in labeling coronary lumens, various morphologies in stenotic lesions, thin structures and small volume ratio with respect to the imaging field complicate the task. In this work, we fused the continuity topological information of centerlines which are easily accessible, and proposed a novel weakly supervised model, Examinee-Examiner Network (EE-Net), to overcome the challenges in automatic coronary lumen segmentation. First, the EE-Net was proposed to address the fracture in segmentation caused by stenoses by combining the semantic features of lumens and the geometric constraints of continuous topology obtained from the centerlines. Then, a Centerline Gaussian Mask Module was proposed to deal with the insensitiveness of the network to the centerlines. Subsequently, a weakly supervised learning strategy, Examinee-Examiner Learning, was proposed to handle the weakly supervised situation with few lumen labels by using our EE-Net to guide and constrain the segmentation with customized prior conditions. Finally, a general network layer, Drop Output Layer, was proposed to adapt to the class imbalance by dropping well-segmented regions and weights the classes dynamically. Extensive experiments on two different data sets demonstrated that our EE-Net has good continuity and generalization ability on coronary lumen segmentation task compared with several widely used CNNs such as 3D-UNet. The results revealed our EE-Net with great potential for achieving accurate coronary lumen segmentation in patients with coronary artery disease. Code at http://github.com/qiyaolei/Examinee-Examiner-Network.

Accurate segmentation of uterus, uterine fibroids, and spine from MR images is crucial for high intensity focused ultrasound (HIFU) therapy but remains still difficult to achieve because of 1) the large shape and size variations among individuals, 2) the low contrast between adjacent organs and tissues, and 3) the unknown number of uterine fibroids. To tackle this problem, in this paper, we propose a large kernel Encoder-Decoder Network based on a 2D segmentation model. The use of this large kernel can capture multi-scale contexts by enlarging the valid receptive field. In addition, a deep multiple atrous convolution block is also employed to enlarge the receptive field and extract denser feature maps. Our approach is compared to both conventional and other deep learning methods and the experimental results conducted on a large dataset show its effectiveness.

In this paper, an unsupervised cycle image registration convolutional neural network named CIRNet is developed for 3D medical image registration. Different from most deep learning based registration methods that require known spatial transforms, our proposed method is trained in an unsupervised way and predicts the dense displacement vector field. The CIRNet is composed by two image registration modules which have the same architecture and share the parameters. A cycle identical loss is designed in the CIRNet to provide additional constraints to ensure the accuracy of the predicted dense displacement vector field. The method is evaluated by the registration in 4D (3D+t) cardiac CT and MRI images respectively. Quantitative evaluation results demonstrate that our method performs better than the other two existing image registration algorithms. Especially, compared to the traditional image registration methods, our proposed network can finish 3D image registration in less than one second.

Accounting for nearly 2% of all adults, renal cell carcinomas are sensitive to laparoscopic partial nephrectomy (LPN) which needs an accurate diagnosis and localization before operation. Faced with various intensity distribution, erratic location, irregular shape, etc, the image classification and semantic segmentation on CT scans of renal tumor are challenges. This paper presents a multi-task network, segmentation and classification convolutional neural network (SCNet), for preoperative assessment of renal tumor. Via the combination of two tasks, semantic features are fed to the classification network and classification results give segmentation network feedbacks in return. Besides, a 2-step segmentation strategy is conducted to the segmentation module which improves the result by 2.8%. Our experimental results of classification and segmentation achieve 100% accuracy and 0.882 dice coefficient of tumor region respectively, which are better than the results of a single classification network and segmentation network.

The wide applications of X-ray computed tomography (CT) bring low-dose CT (LDCT) into a clinical prerequisite, but reducing the radiation exposure in CT often leads to significantly increased noise and artifacts, which might lower the judgment accuracy of radiologists. In this paper, we put forward a domain progressive 3D residual convolution network (DP-ResNet) for the LDCT imaging procedure that contains three stages: sinogram domain network (SD-net), filtered back projection (FBP), and image domain network (ID-net). Though both are based on the residual network structure, the SD-net and ID-net provide complementary effect on improving the final LDCT quality. The experimental results with both simulated and real projection data show that this domain progressive deep-learning network achieves significantly improved performance by combing the network processing in the two domains.

Renal cancer is one of ten most common cancers in human beings. The laparoscopic partial nephrectomy (LPN) becomes the main therapeutic approach in treating renal cancer. Accurate kidney and tumor segmentation in CT images is a prerequisite step in the surgery planning. However, automatic and accurate kidney and renal tumor segmentation in CT images remains a challenge. In this paper, we propose a new method to perform a precise segmentation of kidney and renal tumor in CT angiography images. This method relies on a three-dimensional (3D) fully convolutional network (FCN) which combines a pyramid pooling module (PPM). The proposed network is implemented as an end-to-end learning system directly on 3D volumetric images. It can make use of the 3D spatial contextual information to improve the segmentation of the kidney as well as the tumor lesion. The experiments conducted on 140 patients show that these target structures can be segmented with a high accuracy. The resulting average dice coefficients obtained for kidney and renal tumor are equal to 0.931 and 0.802 respectively. These values are higher than those obtained from the other two neural networks.

Noise detection accuracy is crucial in suppressing random-valued impulse noise. Both false and miss detections determine the final estimation performance. Deterministic detection methods, which distinctly classify pixels into noisy or uncorrupted pixels, tend to increase the estimation error because some uncorrupted edge points are hard to discriminate from the random-valued impulse noise points. This paper proposes an iterative structure-adaptive fuzzy estimation (SAFE) for random-valued impulse noise suppression. This SAFE method is developed in the framework of Gaussian maximum likelihood estimation. The structure-adaptive fuzziness is reflected by two structure-adaptive metrics based on pixel reliability and patch similarity, respectively. The reliability metric for each pixel (as noise free) is estimated via a novel-minimal-path-based structure propagation to give full consideration of the spatially varying image structures. A robust iteration stopping strategy is also proposed by evaluating the reestimation error of the uncorrupted intensity information. The comparative experimental results show that the proposed structure-adaptive fuzziness can lead to effective restoration. An efficient implementation of this SAFE method is also realized via graphics-processing-unit-based parallelization.

Minimal path techniques can efficiently extract geometrically curve-like structures by finding the path with minimal accumulated cost between two given endpoints. Though having found wide practical applications (e.g., line identification, crack detection, and vascular centerline extraction), minimal path techniques suffer from some notable problems. The first one is that they require setting two endpoints for each line to be extracted (endpoint problem). The second one is that the connection might fail when the geodesic distance between the two points is much shorter than the desirable minimal path (shortcut problem). In addition, when connecting two distant points, the minimal path connection might become inefficient as the accumulated cost increases over the propagation and results in leakage into some non-feature regions near the starting point (accumulation problem). To address these problems, this paper proposes an approach termed minimal path propagation with backtracking. We found that the information in the process of backtracking from reached points can be well utilized to overcome the above problems and improve the extraction performance. The whole algorithm is robust to parameter setting and allows a coarse setting of the starting point. Extensive experiments with both simulated and realistic data are performed to validate the performance of the proposed method.

A new divergence measure for rigid image registration is proposed that uses the properties of the Arimoto entropy. This Jensen-Arimoto divergence allows designing a novel registration method by minimizing a dissimilarity measure through the steepest gradient descent optimization method. Preliminary experiments on simulated magnetic resonance images with partial overlap and different degrees of noise have been carried out and a comparison has been conducted with other relevant information theoretic measures such as the normalized mutual information and the cross cumulative residual entropy. The results show that the proposed registration approach has better robustness to noise and can provide better registration accuracy, i.e. a sub pixel accuracy less than 0.1mm and 0.1 degree for translation and rotation. In addition, the calculation time for a 2D rigid registration is improved by approximately 10-20 % compared to the other two methods.

Low-dose computed tomography (LDCT) images are often severely degraded by amplified mottle noise and streak artifacts. These artifacts are often hard to suppress without introducing tissue blurring effects. In this paper, we propose to process LDCT images using a novel image-domain algorithm called "artifact suppressed dictionary learning (ASDL)." In this ASDL method, orientation and scale information on artifacts is exploited to train artifact atoms, which are then combined with tissue feature atoms to build three discriminative dictionaries. The streak artifacts are cancelled via a discriminative sparse representation operation based on these dictionaries. Then, a general dictionary learning processing is applied to further reduce the noise and residual artifacts. Qualitative and quantitative evaluations on a large set of abdominal and mediastinum CT images are carried out and the results show that the proposed method can be efficiently applied in most current CT systems.

The gyrator transform has been proposed in optics a few years ago. By using the theory of quaternion numbers, this paper presents the quaternion gyrator transform (QGT). It is shown that the QGT can be computed via the left-side type of quaternion Fourier transforms. The new transform is applied to color image encryption for validation, where the rotation angles are used as encryption keys making it more secure compared to a recent method using discrete quaternion Fourier transforms (DQFTs). Experimental results show that the proposed encryption algorithm for color image performs as well as the DQFTs method in terms of noise robustness, so that it could be a useful tool for color image encryption.

The derivation of moment invariants has been extensively investigated in the past decades. In this paper, we construct a set of invariants derived from Zernike moments which is simultaneously invariant to similarity transformation and to convolution with circularly symmetric point spread function (PSF). Two main contributions are provided: the theoretical framework for deriving the Zernike moments of a blurred image and the way to construct the combined geometric-blur invariants. The performance of the proposed descriptors is evaluated with various PSFs and similarity transformations. The comparison of the proposed method with the existing ones is also provided in terms of pattern recognition accuracy, template matching and robustness to noise. Experimental results show that the proposed descriptors perform on the overall better.

Prostate segmentation in 3-D transrectal ultrasound images is an important step in the definition of the intra-operative planning of high intensity focused ultrasound (HIFU) therapy. This paper presents two main approaches for the semi-automatic methods based on discrete dynamic contour and optimal surface detection. They operate in 3-D and require a minimal user interaction. They are considered both alone or sequentially combined, with and without postregularization, and applied on anisotropic and isotropic volumes. Their performance, using different metrics, has been evaluated on a set of 28 3-D images by comparison with two expert delineations. For the most efficient algorithm, the symmetric average surface distance was found to be 0.77 mm.

Processing blurred images is a key problem in many image applications. Existing methods to obtain blur invariants which are invariant with respect to centrally symmetric blur are based on geometric moments or complex moments. In this paper, we propose a new method to construct a set of blur invariants using the orthogonal Legendre moments. Some important properties of Legendre moments for the blurred image are presented and proved. The performance of the proposed descriptors is evaluated with various point-spread functions and different image noises. The comparison of the present approach with previous methods in terms of pattern recognition accuracy is also provided. The experimental results show that the proposed descriptors are more robust to noise and have better discriminative power than the methods based on geometric or complex moments.

Shape signatures are fundamental in medicine to characterize abnormal patterns and track their evolution over time. Beyond the diagnosis objective, they are critical to understand the relations between structures and functions as well as to design advanced image-guided therapy. One of the most critical steps in medical imaging remains segmentation, aimed at a fast, precise, robust and reproducible extraction of the organ boundaries. We present a 3D semi-automatic technique for segmenting the inner boundary of abdominal aortic aneurysms (AAA). The method that is proposed makes use of recent developments from graph theory, the graph-cut algorithms. It requires a limited interaction with the user (for learning the object and background statistical features). Its performance is highlighted on multi-detector CT angiography data. Some perspectives are then sketched in order to show how this result could be improved to feed fluid-structure simulations.

BACKGROUND: The SYMBIOmatics Specific Support Action (SSA) is "an information gathering and dissemination activity" that seeks "to identify synergies between the bioinformatics and the medical informatics" domain to improve collaborative progress between both domains (ref. to http://www.symbiomatics.org). As part of the project experts in both research fields will be identified and approached through a survey. To provide input to the survey, the scientific literature was analysed to extract topics relevant to both medical informatics and bioinformatics. RESULTS: This paper presents results of a systematic analysis of the scientific literature from medical informatics research and bioinformatics research. In the analysis pairs of words (bigrams) from the leading bioinformatics and medical informatics journals have been used as indication of existing and emerging technologies and topics over the period 2000-2005 ("recent") and 1990-1990 ("past"). We identified emerging topics that were equally important to bioinformatics and medical informatics in recent years such as microarray experiments, ontologies, open source, text mining and support vector machines. Emerging topics that evolved only in bioinformatics were system biology, protein interaction networks and statistical methods for microarray analyses, whereas emerging topics in medical informatics were grid technology and tissue microarrays. CONCLUSION: We conclude that although both fields have their own specific domains of interest, they share common technological developments that tend to be initiated by new developments in biotechnology and computer science.

This paper reports on an analysis of the bioinformatics and medical informatics literature with the objective to identify upcoming trends that are shared among both research fields to derive benefits from potential collaborative initiatives for their future. Our results present the main characteristics of the two fields and show that these domains are still relatively separated.

This paper describes an extended ray casting scheme for three-dimensional (3-D) navigation into the heart cavities for preoperative planning using multislice X-ray computed tomography data. The key benefit is that artifacts due to contrast inhomogeneities can be eliminated during volume traversal, thus improving the visual perception of the endocardial wall.

This paper reports on a method for left ventricle three-dimensional (3-D) reconstruction from two orthogonal ventriculograms. The proposed algorithm is voxel-based and takes into account the conical projection geometry associated with the biplane image acquisition equipment. The reconstruction process starts with an initial ellipsoidal approximation derived from the input ventriculograms. This model is subsequently deformed in such a way as to match the input projections. To this end, the object is modeled as a 3-D Markov-Gibbs random field, and an energy function is defined so that it includes one term that models the projections compatibility and another one that includes the space-time regularity constraints. The performance of this reconstruction method is evaluated by considering the reconstruction of mathematically synthesized phantoms and two 3-D binary databases from two orthogonal synthesized projections. The method is also tested using real biplane ventriculograms. In this case, the performance of the reconstruction is expressed in terms of the projection error, which attains values between 9.50% and 11.78 % for two biplane sequences including a total of 55 images.

Clinical interventional hemodynamic studies quantify the ventricular function from two-dimensional (2-D) X-ray projection images without having enough information of the actual three-dimensional (3-D) shape of this cardiac cavity. This paper reports a left ventricle 3-D reconstruction method from two orthogonal angiographic projections. This investigation is motivated by the lack of information about the actual 3-D shape of the cardiac cavity. The proposed algorithm works in 3-D space and considers the oblique projection geometry associated with the biplane image acquisition equipment. The reconstruction process starts by performing an approximate reconstruction based on the Cylindrical Closure Operation and the Dempster-Shafer theory. This approximate reconstruction is appropriately deformed in order to match the given projections. The deformation procedure is carried out by an iterative process that, by means of the Dempster-Shafer and the fuzzy integral theory, combines the information provided by the projection error and the connectivity between voxels. The performance of the proposed reconstruction method is evaluated by considering first the reconstruction of two 3-D binary databases from two orthogonal synthetized projections, obtaining errors as low as 6.48%. The method is then tested on real data, where two orthogonal preprocessed angiographic images are used for reconstruction. The performance of the technique, in this case, is assessed by means of the projection error, whose average for both views is 7.5%. The reconstruction method is also tested by performing the 3-D reconstruction of a ventriculographic sequence throughout an entire cardiac cycle.

In this paper, a model-based approach to medical image analysis is presented. It is aimed at understanding the influence of the physiological (related to tissue) and physical (related to image modality) processes underlying the image content. This methodology is exemplified by modeling first, the liver and its vascular network, and second, the standard computed tomography (CT) scan acquisition. After a brief survey on vascular modeling literature, a new method, aimed at the generation of growing three-dimensional vascular structures perfusing the tissue, is described. A solution is proposed in order to avoid intersections among vessels belonging to arterial and/or venous trees, which are physiologically connected. Then it is shown how the propagation of contrast material leads to simulate time-dependent sequences of enhanced liver CT slices.

This paper aims to define and describe features of the motion of coronary arteries in two and three dimensions, presented as geometrical parameters that identify motion patterns. The main left coronary artery centerlines, obtained from digital subtraction angiography (DSA) image sequences, are first reconstructed. Thereafter, global and local motion features are evaluated along the sequence. The global attributes are centerline and point trajectory lengths, displacement amplitude, and virtual reference point, while local attributes are displacement direction, perpendicular/radial components, rotation direction, and curvature and torsion. These kinetic features allow us to obtain a detailed quantitative description of the displacements of arteries' centerlines, as well as associated epicardium deformations. Our modeling of local attributes as quasi-homogeneous on a segment analysis, enables us to propose a novel numeric to symbolic image transformation, which provides the required facts for knowledge-based motion interpretation. Experimental results using real data are consistent with cardiac dynamic behavior.

Advancements in medicine and health care are being significantly influenced by the exploding information technology developments. The IEEE Transactions on Information Technology in Biomedicine will address the applications and the infrastructure innovations that would harness biomedical and health care programs in the 21st century.

Discusses and debates the role played by 3D visualization in medicine as a set of methods and techniques for displaying 3D spatial information related to the anatomy and the physiology of the human body.>

'IBIS' is a set of computer programs concerned with the processing of electron micrographs, with particular emphasis on the requirements for structural analyses of biological macromolecules. The software is written in FORTRAN 77 and runs on Unix workstations. A description of the various functions and the implementation mode is given. Some examples illustrate the user interface.

A new algorithm is presented for three-dimensional tomographic reconstruction from a reduced number of projections. It is based on the completion of data by interpolation of the measured data in the frequency domain. A filtered backprojection procedure, suited to conic geometry, performs the tomographic reconstruction from the augmented data set. A second reconstruction scheme is derived for high-contrast sparse objects. It improves the resolution by zeroing the positives values of the filtered projections before the backprojection. These methods have been tested on simulated data and data obtained on a 3-D X-ray acquisition system. They produce better quality 3-D images than analytical reconstruction working alone.>