Tumor segmentation is particularly important for ultrasound imaging-based diagnosis and therapy, such as breast cancer and gastrointestinal stromal tumor. However, the accurate ultrasound tumor segmentation remains challenging due to insufficient textures and edge features resulted from limited resolution, low signal-to-noise ratio (SNR), and artifacts in ultrasound images. To address the challenges, this paper proposes an novel tumor segmentation network that combines a multi-scale compression attention module (MCAM) and an edge detection module (EDM). MCAM fuses multi-scale features from a U-shaped backbone to capture global semantic features using a compression attention mechanism. EDM introduces multiple convolutional layers to extract the edge features of tumor. Furthermore, a semantic and texture fusion (STF) mechanism followed by an improved deep supervision is proposed to strengthen the network’s performance in resolving tumors. Experimental validations on a public dataset and a private dataset demonstrated the effectiveness of the proposed modules and the outperformance of our network over the current advanced segmentation networks in various metrics.

Spatiotemporal clutter filtering via robust principal component analysis (rPCA) has been widely used in ultrasound microvascular imaging. However, the performance of the rPCA clutter filtering highly relies on low-rank modeling for tissue signals and sparse modeling for blood flow signals. Moreover, current rPCA clutter filters are typically based on static processing and have to access a batch of beamformed frames for optimization. This prevents these filters from ultrafast realization. This paper adopts the iteratively reweighted least squares (IRLS) rPCA framework to model tissue and blood flow signals for improved clutter filtering. More importantly, the static IRLS-rPCA filter is upgraded to a spatiotemporal-constrained online method to instantaneously extract blood flow signals from the ongoing beamformed frame. Simulations and in-vivo experiments on a contrast-enhanced rat kidney and a contrast-free human liver demonstrated that the IRLS-rPCA clutter filter achieves higher sensitivity, contrast-to-noise ratio (CNR), and signal-to-noise ratio (SNR) than other rPCA methods. Particularly, the static IRLS-rPCA clutter filter obtains more than 2 dB improvements in CNR over the compared methods in the human liver dataset. The proposed online clutter filter achieves comparable image quality to the static version and processing time of $0.028~\pm ~0.004$ seconds per frame. The corresponding acceleration factor of the online clutter filter over all the tested methods is more than 20.

Automatic segmentation of pulmonary arteries and veins in CT has great clinical significance. Because the growth range of a single vessel is vast, and the arteries and veins have barely identical intensity values on CT and grow very close to or even interleaved, accurate segmentation of them requires intricate vascular texture information and long-distance vascular trunk information as the basis for artery and vein classification. In order to meet these two requirements simultaneously, we design a residual W-Unit, which concatenated two U-shaped structures. It allows the network to become deeper and improve the receptive field for global information while preserving the detailed features of the vessels. And we design a semantic embedding module using cross-attention, which enhances the expression of bronchial features and assists in further utilizing features. It explicitly leverages the anatomical knowledge of parallel growth between arteries and bronchi. Then we combine RWUs and SEMs to construct a concise network to extract and fuse the features with detailed information from different network depths and receptive fields. Finally, we use a post-processing scheme to reduce spatial inconsistency. We validated our networks on 40 training sets and 17 test sets, and the experimental results show that our networks outperform current segmentation methods.

Monocular endoscopic depth estimation is essential for surgical navigation. Current deeply learned estimation methods still suffer from lack of real data labels and porous, artifacts (e.g., bubbles), illumination variations (e.g., specular highlight), and weak texture in endoscopic video images. This paper proposes a new deep learning framework of cascade hole-aware transformers with geometric spatial consistency for accurate endoscopic depth estimation without using any image annotation. Specifically, this framework employs cascade hole-aware encoders to powerfully extract structural features of deep and shallow holes, while it further introduces multiscale filtering decoders to suppress non-hole region features, addressing the problems of specular highlights, weak textures or bubbles. Additionally, a geometric spatial consistency loss can strongly perceive geometric information and suppress the color difference between virtual and real images. We generated virtual endoscopic image data to train our network architecture and test it on both virtual and real endoscopic video images, with the experimental results showing that our method is robust to zero-shot evaluation of real data. Particularly, our method can attain lower root mean square error 1.551±1.147 mm and mean absolute error 1.004±0.632 mm than state-of-the-art deep learning approaches.

Pulmonary fissures are anatomical biomarkers used to evaluate the severity of chronic obstructive pulmonary disease. The completeness of the fissures is significantly associated with this disease. This work proposes a new fully automated fissure completeness assessment framework on the basis of deeply learned pulmonary fissure and lobe segmentation. This framework consists of automatic loop structure-aware learning for joint fissure-lobe segmentation and fissure integrity calculation. Specifically, the loop segmentation performs (1) attention-gated U-transformers for fissure segmentation, (2) 3-D U-Net to extract pulmonary lobes on the basis of the segmented fissures, and (3) attention-gated U-transformers to refine the segmented fissures using the extracted lobes. Based on accurately segmented fissures and interlobar boundaries, we develop a new fissure completeness assessment method. We evaluated our framework on 54 CT volumes, with the experimental results showing that our loop segmentation methods can extract fissures and lobar boundaries more accurately than state-of-the-art methods for the fissure completeness calculation. Particularly, our fissure completeness computing method provides chronic obstructive pulmonary disease with a promising assessment way.

Cervical vertebrae age estimation empowers clinicians to determine the development status of children and adolescents for precise orthodontic diagnosis and treatment. This work proposes a fully automated cervical vertebrae maturation staging framework that uses deeply learned CT image segmentation and classification. Specifically, such a two-step framework first employs convolutional neural networks (i.e., nnU-Net) to precisely segment the cervical vertebrae bones. Then, we propose parallel regression-classification networks for the bone staging using the segmented results. Specifically, the regression path introduces prior knowledge (i.e., bone anatomical parameters) to supervise the classification path. We evaluate our method on two clinical CT databases (60 balance volumes and 85 imbalance volumes), with the experimental results showing that our method attains higher classification accuracy than state-of-the-art methods. In particular, it can improve the accuracy from (0.5833, 0.5294) to (0.6667, 0.6471) on (balance, imbalance) data, respectively. Besides, our method also achieves an average accuracy of (0.65, 0.6353) for the 5-fold cross-validation.

Human fertility continues deteriorating globally in recent decades. Artificial assisted reproductive technology is an effective solution to infertility treatment. Using computer-aided semen analysis systems to visually analyze the motility of sperms and select high-quality targets is widely concerned. Selecting motile sperm requires accurate and robust tracking of the individual target in the microscopic videos. Unfortunately, existing methods may fail to track the sperms in real time, especially for some motile sperms that swim out of the focal plane for a few frames and then swim back, exhibiting temporary disappearance and subsequent reappearance. In this letter, we propose an adaptive color histogram-based marginalized particle filter to accurately and robustly track sperm in real time. The experimental results on both synthetic and clinical microscopic videos demonstrated that the proposed method achieves higher accuracy compared to the alternative methods. Particularly, our method can successfully track the motile sperms with complex movements, showing higher robustness than other methods.

Surface reconstruction is an essential way to expand surgical field of view during endoscopic surgery, but it certainly requires dense depth estimation of endoscopic video sequences. Unfortunately, such a dense depth recovery suffers from illumination variation, weak texture, and occlusion. To address these problems, this work proposes a new triple-supervision self-learning strategy that uses unannotated endoscopic video data to predict monocular endoscopic dense depth information. This strategy first employs an effective conventional method to estimate camera poses and sparse depth maps to establishing a sparse data self-supervision. Furthermore, our strategy still combines two consistency measures to supervise dense depth and photometric information. We evaluated our method on collected colonoscopic videos, with the experimental results showing that our triple-supervision learning framework works more effective and accurate than some current self-supervised and unsupervised learning methods.

Segmentation of hepatic and portal veins in abdominal magnetic resonance images plays an essential role in surgical planning of liver tumor ablation and resection. Accurately extracting these blood vessels is a challenging task due to the complex vessel structures with high noise and irregular vessel shapes caused by nearby tumors. This work presents a new deep learning method called local-global progressive U-Transformers for precise extraction of hepatic and portal veins. Specifically, our method embeds convolution into the Transformer frame to extract features progressively and uses window attention to achieve full fusion of local and global features, as well as it only requires a small number of training parameters close to lightweight networks. We evaluated the proposed method on 30 clinical abdominal scans, with the experimental results showing that our method works better than the other segmentation approaches, improving the dice similarity coefficient from 0.7885 to 0.8132 and significantly reducing the number of parameters from 93.19M to 7.57M. We also found that our method can address the problem of voxel intensity variations and irregular vessel structures.

Endoscopy 3D reconstruction can provide more intuitive perception of the lesions in minimally invasive surgery. The success of 3D reconstruction highly relies on high-quality feature matches between the monocular image pairs, which remains challenging in the textureless endoscopic scenario. In this paper, we propose an effective feature matching framework for monocular endoscopy 3D reconstruction. The framework contains a descriptor generation network (DGN) to generate high-quality feature descriptors in a local-to-global manner, and a local region expansion to fine tune the initial matches obtained from the DGN module. We evaluated our method on the public Hamlyn Centre Laparoscopic/Endoscopic Video Datasets. The experimental results demonstrated that our method can generate sufficient accurate feature matches. Particularly, our method performed better in sparse depth estimation of the endoscopic scenario when compared with the current conventional and deep-learning methods.

Pulmonary vessel CT segmentation is important to clinical diagnosis of lung diseases. But it is still a challenge due to limited CT quality and complicated vascular structures. This paper proposes new enhanced U-transformer networks that combine transformers, a contrast enhancement block with a reverse attention block to perform end-to-end vessel segmentation. Specifically, the contrast enhancement block directly augments edge or structural information while the reverse attention block conducts the network paying more attention to blurred boundaries and uncertain regions of vessels, leading to improving the accuracy and smoothness of pulmonary vessel segmentation. We validated our proposed method on 50 CT volumes selected from LIDC-IDRI, with the experimental results demonstrating that it works more effectively and stably than currently available approaches. Particularly, the average dice similarity coefficient and recall were improved from (85.23%, 85.37%) to (86.07%, 86.67%), respectively.

Colonoscopic polyp segmentation is essential and valuable to early diagnosis and treatment of colorectal cancer. It remains challenging to accurately extract these polyps due to their small sizes, irregular shapes, image artifacts, and illumination variations. This work proposes a new encoder-decoder architecture called pyramid transformer driven multibranch fusion to precisely segment different types of colorectal polyps during colonoscopy. Specifically, our architecture employs a simple, convolution-free pyramid transformer as its encoder that is a flexible and powerful feature extractor. Next, a multibranch fusion decoder is employed to reserve the detailed appearance information and fuse semantic global cues, which can deal with blurred polyp edges caused by nonuniform illumination and the shaky colonoscope. Additionally, a hybrid spatial-frequency loss function is introduced for accurate training. We evaluate our proposed architecture on colonoscopic polyp images with four types of polyps with different pathological features, with the experimental results showing that our architecture significantly outperforms other deep learning models. Particularly, our method improves the average dice similarity and intersection over union to 90.7% and 0.848, respectively.

Accurate segmentation of the cervical vertebrae in CT images can assist clinicians in analyzing the adolescent patient’s growth and development and establishing an effective orthodontic plan. This work develops an enhanced U2-Net architecture for fully automatic cervical vertebrae segmentation in CT images. Specifically, such an enhanced architecture first creates a deepwise separable residual U-shape module (DUM) in different levels and a convolutional attention module embedded into a U-structure for encoding and decoding and obtains a coarse segmentation. Then, it reuses a DUM to refine the segmentation. We evaluated our method on 60 CT scans, with the experimental results showing that our method attains much better segmentation performance than state-of-the-art network models. Particularly, it can improve the dices similarity coefficient, intersection over union, precision, and recall from (0.9586, 0.9207, 0.9584, 0.9591) to (0.9755, 0.9524, 0.9832, 0.9708), respectively, while it can reduce the model parameters from 44.0M to 16.5M.

Renal artery segmentation plays a fundamental role in nephrectomy, which can help surgeons get a better under-standing of vascular structures. However, the similar intensity between the renal arteries and cortex, the complex variations and tiny structures of arteries, bring challenges to accurate segmentation. To address these issues, we construct double U-PoolFormer networks (DUP-Net) to establish a coarse-to-fine framework for renal artery segmentation. Specifically, we use 2- D U - N et for the kidney extraction and then create 3-D DUP-Net for artery segmentation. DUP-Net is a serial network architecture that uses two U-PoolFormer modules to extract long-range spatial dependencies to create tree-like constraints while removing mis-segmentation of renal cortex through the serial structure. While DUP-Net improving the segmentation accuracy, it reduces memory cost during segmengtation. We evaluated our method on 70 cases of computed tomography urography data, with the experimental results showing that our proposed method certainly outperforms current 2-D and 3-D network models. Particularly, the average dice similarity coefficient of our method was improved from 81.51 % to 88.35%.

Volumetric (3D) ultrasound imaging using a 2D matrix array probe is increasingly developed for various clinical procedures. However, 3D ultrasound imaging suffers from motion artifacts due to tissue motions and a relatively low frame rate. Current Doppler-based motion compensation (MoCo) methods only allow 1D compensation in the in-range dimension. In this work, we propose a new 3D-MoCo framework that combines 3D velocity field estimation and a two-step compensation strategy for 3D diverging wave compounding imaging. Specifically, our framework explores two constraints of a round-trip scan sequence of 3D diverging waves, i.e., Doppler and pair-wise optical flow, to formulate the estimation of the 3D velocity fields as a global optimization problem, which is further regularized by the divergence-free and first-order smoothness. The two-step compensation strategy is to first compensate for the 1D displacements in the in-range dimension and then the 2D displacements in the two mutually orthogonal cross-range dimensions. Systematical in-silico experiments were conducted to validate the effectiveness of our proposed 3D-MoCo method. The results demonstrate that our 3D-MoCo method achieves higher image contrast, higher structural similarity, and better speckle patterns than the corresponding 1D-MoCo method. Particularly, the 2D cross-range compensation is effective for fully recovering image quality.

Unsupervised domain adaptation is to transfer knowledge from a well-annotated source domain and learn an accurate classifier for an unlabeled target domain, which is particularly useful in multimodal medical image processing. Currently available adaptation approaches strongly reduce the domain bias or inconsistency in the latent space, deteriorating inherent data structures. To appropriately leverage the reduction of the domain discrepancy and the maintenance of the intrinsic structure, this paper proposes a dual U-DenseTransformer generation domain adaptation framework to bridge the gap between source and target domains and achieve translation. Specifically, we create a DenseTransformer with multi-head attention embedded in U-shape network to establish a dual-generator strategy, which is further enhanced by a new hybrid loss function and an edge-aware mechanism that preserve inherent data structure consistent. We apply our proposed method to medical image segmentation, with the experimental results showing that it works more effective and stable than currently available approaches. Particularly, the dice similarity was improved from 79.3% to 82.8%, while the average symmetric surface distance was reduced from 2.5 to 1.9.

Unsupervised domain adaptation commonly uses cycle generative networks to produce synthesis data from source to target domains. Unfortunately, translated samples cannot effectively preserve semantic information from input sources, resulting in bad or low adaptability of the network to segment target data. This work proposes an advantageous domain translation mechanism to improve the perceptual ability of the network for accurate unlabeled target data segmentation. Our domain translation employs patchwise contrastive learning to improve the semantic content consistency between input and translated images. Our approach was applied to unsupervised domain adaptation based abdominal organ segmentation. The experimental results demonstrate the effectiveness of our framework that outperforms other methods.

Segmentation of abdominal organs such as the liver, pancreas, spleen, and kidneys plays an essential role in diagnosing and treating abdominal diseases. Although numerous deeply learned segmentation methods work well, they still suffer from partial volume effects, image noise, and data imbalance. This study aims to colorize CT images to boost or augment these segmentation approaches. We propose new residual U-structure nested generative adversarial nets that use residual U-blocks and spectral normalization for CT image colorization. Generated color CT images were introduced to train and validate V-Net and DenseV-Net for multiple abdominal organ segmentation. The experimental results demonstrate that colorized CT images can improve the dice similarity coefficient and reduce the Hausdorff distance from (0.32, 302.7) to (0.67, 78.2), significantly boosting the performance of V-Net and Dense V-Net for multiple abdominal organ segmentation.

Accurate pancreas segmentation is crucial for computer aided pancreas diagnosis and surgery. It still remains challenging to precisely extract the pancreas due to its small size, unclear boundary, and shape variations on CT images. This work proposes a new 3D end-to-end boundary-aware network architecture for automatic accurate pancreas segmentation from CT images. Specifically, this architecture introduces four hybrid blocks for feature extraction in accordance with 3D fully convolutional neural networks so that it can successfully extract and perceive spatial and contextual information from 3D CT data. Simultaneously, a reverse attention block and a boundary enhancement block are embedded into this architecture to enhance the ability to learn and extract feature maps with more context and boundary information. We evaluate our proposed method on publicly available pancreas data using 4-fold cross-validation, with the experimental results showing that our network model can obtain more accurate or comparable segmentation than other existing methods.

Two-way dynamic focusing in synthetic transmit aperture (STA) beamforming can benefit high-quality ultrasound imaging with higher lateral spatial resolution and contrast resolution. However, STA requires the complete dataset for beamforming in a relatively low frame rate and transmit power. This paper proposes a deep-learning architecture to achieve high frame rate STA imaging with two-way dynamic focusing. The network consists of an encoder and a joint decoder. The encoder trains a set of binary weights as the apodizations of the high-frame-rate plane wave transmissions. In this respect, we term our network ApodNet. The decoder can recover the complete dataset from the acquired channel data to achieve dynamic transmit focusing. We evaluate the proposed method by simulations at different levels of noise and in-vivo experiments on the human biceps brachii and common carotid artery. The experimental results demonstrate that ApodNet provides a promising strategy for high frame rate STA imaging, obtaining comparable lateral resolution and contrast resolution with four-times higher frame rate than conventional STA imaging in the in-vivo experiments. Particularly, ApodNet improves contrast resolution of the hypoechoic targets with much shorter computational time when compared with other high-frame-rate methods in both simulations and in-vivo experiments.