Three-dimensional (3D) medical image segmentation typically demands extensive labeled training samples, which is prohibitively time-consuming and requires significant expertise. Although this demand can be mitigated by special learning paradigms such as semi-supervised learning, the cost is still high due to the reader-unfriendly 3D data structure. In this paper, we seek a solution of robust 3D segmentation using extremely simplified annotation that delineates only a single slice per each volume for only a subset of the 3D samples. To this end, we propose two innovative modules: a correlation-enhanced 3D segmentation model (CE-Seg) and a hybrid 3D pseudo mask generator (Hy-Gen). CE-Seg aims to comprehensively understand the 3D targets under super-sparse single-slice supervision by maximizing its ability to mine correlations across slices, spaces and scales. Specifically, CE-Seg mimics the radiologist's interpretation by 'seeing' a dynamically scrolling 3D image to enrich the slice-correlated context. It also introduces a drop-then-restoration self-played task to enhance the spatial correlations of features, and uses a bidirectional cascaded attention to interactively fuse features across different scales. To train CS-Seg, Hy-Gen combines learning-based and learning-free strategies to generate reliable pseudo 3D masks as supervisions. Concretely, Hy-Gen first employs a level-set evolution to 'spread' the single annotation to its neighboring slices as initialization. It then builds a teacher-student framework to progressively refine the initialized 3D mask by dynamically merging the predictions of the CS-Seg's teacher-copy. Extensive experiments on three public and one in-house datasets indicate that our method exceeds eight state-of-the-art semi-supervised methods by at least 3$\%$ in dice, and is even on par with the full-supervised counterpart.

The vessel-wall-volume (VWV) measured based on three-dimensional (3D) carotid artery (CA) ultrasound (US) images can help to assess carotid atherosclerosis and manage patients at risk for stroke. Manual involvement for measurement work is subjective and requires well-trained operators, and fully automatic measurement tools are not yet available. Thereby, we proposed a fully automatic VWV measurement framework (Auto-VWV) using a CA prior-knowledge embedded U-Net (CAP-UNet) to measure the VWV from 3D CA US images without manual intervention. The Auto-VWV framework is designed to improve the repeated VWV measuring consistency, which resulted in the first fully automatic framework for VWV measurement. CAP-UNet is developed to improve segmentation accuracy on the whole CA, which composed of a U-Net type backbone and three additional prior-knowledge learning modules. Specifically, a continuity learning module is used to learn the spatial continuity of the arteries in a sequence of image slices. A voxel evolution learning module was designed to learn the evolution of the artery in adjacent slices, and a topology learning module was used to learn the unique topology of the carotid artery. In two 3D CA US datasets, CAP-UNet architecture achieved state-of-the-art performance compared to eight competing models. Furthermore, CAP-UNet-based Auto-VWV achieved better accuracy and consistency than Auto-VWV based on competing models in the simulated repeated measurement. Finally, using 10 pairs of real repeatedly scanned samples, Auto-VWV achieved better VWV measurement reproducibility than intra- and inter-operator manual measurements. The code is available at https://github.com/Yue9603/Auto-VWV.

Slice-to-volume registration that achieves high accuracy in real time is one of the enabling technologies for clinical imaging scenarios. Recently, deep learning methods have been explored to significantly improve the accuracy and efficiency of the registration. Nevertheless, such a 2D/3D registration problem is very challenging due to several considerable barriers including highly computational cost brought by dense sampling in six dimensional parameter space and significant modal appearance differences. We proposed a Differentiable resampling based Slice-to-Volume Registration network , which can achieve real-time and accurate registration in both mono- and multi-modal scenarios. The proposed network learns the out-of-plane transformation parameters in the spherical coordinates that decide the content and is invariant to the remaining in-plane parameters by feeding 2D rigid transformed data on-the-fly, thus reducing the training sample size remarkably. To further improve the performance, we introduce an auxiliary image similarity connected by differentiable resampling. For multi-modal scenarios, we optionally include the enhanced modality independent neighborhood descriptor to unify different modalities into common space. Training size is considerably reduced to 30k per dataset with half of all possible orientations and 80% of the volume space covered while the average registration error drops to 1.84 mm and 6.49 mm respectively for mono- and multi-modal experiments. Extensive experiments show the proposed methods’ superiority over existing deep learning methods for real-time 2D/3D rigid registration in both mono- and multi-modal settings.

This letter proposes a method of jointly designing orthogonal frequency division multiplexing (OFDM)-LFM waveforms and receive filter in spatial heterogeneous clutter. Different from the previous indirect method that first solves the optimal spectra and then optimizes the waveform parameters to approximate the spectra, this letter proposes a cyclic algorithm based on iterative sequence optimization (ISO), which can directly optimize multiple groups of subchirp durations of the OFDM-LFM waveforms, so as to improve the output SCNR. Finally, numerical results are provided to assess the proposed method. Results indicate that the waveforms optimized by the proposed method can simultaneously suppress clutter in both spatial azimuth and frequency domains. Moreover, compared with the indirect optimization method, the method that only optimizes the transmitting waveforms and the single-group subchirp durations optimization method, the proposed method has higher output SCNR.

Semi-supervised segmentation is highly significant in 3D medical image segmentation. The typical solutions adopt a teacher-student dual-model architecture, and they constrain the two models' decision consistency on the same segmentation task. However, the scarcity of medical samples can lower the diversity of tasks, reducing the effectiveness of consistency constraint. The issue can further worsen as the weights of the models gradually become synchronized. In this work, we have proposed to construct diverse joint-tasks using masked image modelling for enhancing the reliability of the consistency constraint, and develop a novel architecture consisting of a single teacher but multiple students to enjoy the additional knowledge decoupled from the synchronized weights. Specifically, the teacher and student models 'see' varied randomly-masked versions of an input, and are trained to segment the same targets but reconstruct different missing regions concurrently. Such joint-task of segmentation and reconstruction can have the two learners capture related but complementary features to derive instructive knowledge when constraining their consistency. Moreover, two extra students join the original one to perform an inter-student learning. The three students share the same encoding but different decoding designs, and learn decoupled knowledge by constraining their mutual consistencies, preventing themselves from suboptimally converging to the biased predictions of the dictatorial teacher. Experimental on four medical datasets show that our approach performs better than six mainstream semi-supervised methods. Particularly, our approach achieves at least 0.61% and 0.36% higher Dice and Jaccard values, respectively, than the most competitive approach on our in-house dataset. The code will be released at https://github.com/zxmboshi/DDL.

The interrupted sampling repeater jamming (ISRJ) can effectively degrade the image quality and affect the subsequent target recognition by creating deceptive multiple false targets on synthetic aperture radar (SAR) images. A time-domain filtering method based on pulse coding to counter ISRJ is proposed in this article. First, this coding method requires the radar to transmit a full pulse signal consisting of multiple subpulse signals several times in the original pulse repetition interval (PRI), and there is a difference in the time distribution of the subpulses transmitted at different moments. Then, use the observation matrix determined by the echo conditions contained in each receiving window to filter the echo in the time domain to obtain the echo corresponding to each subpulse. Finally, the subpulse echo of the jammer sampling section is discarded and the remaining uninterfered subpulse echoes are segmented for pulse compression and subsequent imaging processing to obtain SAR images with a low jamming-to-signal ratio (JSR). Several groups of simulations show that the proposed method is effective against different kinds of ISRJ, and this time-domain filtering method can improve the effect of radar anti-ISRJ and has a high freedom of waveform design.

In the field of disease diagnosis where only a small dataset of medical images may be accessible, the light-weight convolutional neural network (CNN) has become popular because it can help to avoid the over-fitting problem and improve computational efficiency. However, the feature extraction capability of the light-weight CNN is inferior to that of the heavy-weight counterpart. Although the attention mechanism provides a feasible solution to this problem, the existing attention modules, such as the squeeze and excitation module and the convolutional block attention module, have insufficient non-linearity, thereby influencing the ability of the light-weight CNN to discover the key features. To address this issue, we have proposed a spiking cortical model based global and local (SCM-GL) attention module. The SCM-GL module analyzes the input feature maps in parallel and decomposes each map into several components according to the relation between pixels and their neighbors. The components are weighted summed to obtain a local mask. Besides, a global mask is produced by discovering the correlation between the distant pixels in the feature map. The final attention mask is generated by combining the local and global masks, and it is multiplied by the original map so that the important components can be highlighted to facilitate accurate disease diagnosis. To appreciate the performance of the SCM-GL module, this module and some mainstream attention modules have been embedded into the popular light-weight CNN models for comparison. Experiments on the classification of brain MR, chest X-ray, and osteosarcoma image datasets demonstrate that the SCM-GL module can significantly improve the classification performance of the evaluated light-weight CNN models by enhancing the ability of discovering the suspected lesions and it is generally superior to state-of-the-art attention modules in terms of accuracy, recall, specificity and F1 score.

Multimodal medical image fusion (MMIF) is highly significant in such fields as disease diagnosis and treatment. The traditional MMIF methods are difficult to provide satisfactory fusion accuracy and robustness due to the influence of such possible human-crafted components as image transform and fusion strategies. Existing deep learning based fusion methods are generally difficult to ensure image fusion effect due to the adoption of a human-designed network structure and a relatively simple loss function and the ignorance of human visual characteristics during weight learning. To address these issues, we have presented the foveated differentiable architecture search (F-DARTS) based unsupervised MMIF method. In this method, the foveation operator is introduced into the weight learning process to fully explore human visual characteristics for the effective image fusion. Meanwhile, a distinctive unsupervised loss function is designed for network training by integrating mutual information, sum of the correlations of differences, structural similarity and edge preservation value. Based on the presented foveation operator and loss function, an end-to-end encoder-decoder network architecture will be searched using the F-DARTS to produce the fused image. Experimental results on three multimodal medical image datasets demonstrate that the F-DARTS performs better than several traditional and deep learning based fusion methods by providing visually superior fused results and better objective evaluation metrics.

Orthogonal frequency division multiplexing (OFDM) chirp waveform is considered a good choice in the waveform design for clutter suppression, which is due to its excellent characteristics such as spectral containment, phase diversity, great dynamic spectral allocation and high degree of freedom. Considering the purpose of clutter suppression, an OFDM chirp waveform design method based on multiple groups of subchirp durations optimization is proposed to improve the output signal-to-clutter-plus-noise ratio (SCNR) in this paper. The output SCNR is closely related to the waveform spectrum, so the waveforms’ energy spectral density functions are analyzed first to build the relation between the waveform parameters and spectra. Then, the multiple groups of subchirp durations optimization based on maximum SCNR is proposed and solved by an optimization method based on the sequential quadratic programming. Finally, the proposed method is verified and the optimized waveform is compared with the general waveform and the waveform with optimized single group of subchirp durations. The results show that the waveform optimized by the proposed method has higher output SCNR and the SCNR increment compared with the general waveform increases with the number of subcarriers. Besides, the high sidelobes of the general waveform are also greatly reduced.

2D/3D registration that achieves high accuracy and real-time computation is one of the enabling technologies for radiotherapy and image-guided surgeries. Recently, the Convolutional Neural Network (CNN) has been explored to significantly improve the accuracy and efficiency of 2D/3D registration. A pair of intraoperative 2-D x-ray images and synthetic data from pre-operative volume are often required to model the nonconvex mappings between registration parameters and image residual. However, a large clinical dataset collection with accurate poses for x-ray images can be very challenging or even impractical, while exclusive training on synthetic data can frequently cause performance degradation when tested on x-rays. Thus, we propose to train a model on source domain (i.e., synthetic data) to build appearance-pose relationship first and then use an unsupervised cross-modality domain adaptation network (UCMDAN) to adapt the model to target domain (i.e., X-rays) through adversarial learning. We propose to narrow the significant domain gap by alignment in both pixel and feature space. In particular, the image appearance transformation and domain-invariance feature learning by multiple aspects are conducted synergistically. Extensive experiments on CT and CBCT dataset show that the proposed UCMDAN outperforms the existing state-of-the-art domain adaptation approaches.

Measurement of total-plaque-area (TPA) is important for determining long term risk for stroke and monitoring carotid plaque progression. Since delineation of carotid plaques is required, a deep learning method can provide automatic plaque segmentations and TPA measurements; however, it requires large datasets and manual annotations for training with unknown performance on new datasets. A UNet++ ensemble algorithm was proposed to segment plaques from 2D carotid ultrasound images, trained on three small datasets (n = 33, 33, 34 subjects) and tested on 44 subjects from the SPARC dataset (n = 144, London, Canada). The ensemble was also trained on the entire SPARC dataset and tested with a different dataset (n = 497, Zhongnan Hospital, China). Algorithm and manual segmentations were compared using Dice-similarity-coefficient (DSC), and TPAs were compared using the difference (ΔTPA), Pearson correlation coefficient (r) and Bland-Altman analyses. Segmentation variability was determined using the intra-class correlation coefficient (ICC) and coefficient-of-variation (CoV). For 44 SPARC subjects, algorithm DSC was 83.3-85.7%, and algorithm TPAs were strongly correlated (r = 0.985-0.988; p <; 0.001) with manual results with marginal biases (0.73-6.75) mm$^2$ using the three training datasets. Algorithm ICC for TPAs (ICC = 0.996) was similar to intra- and inter-observer manual results (ICC = 0.977, 0.995). Algorithm CoV = 6.98% for plaque areas was smaller than the inter-observer manual CoV (7.54%). For the Zhongnan dataset, DSC was 88.6% algorithm and manual TPAs were strongly correlated (r = 0.972, p <; 0.001) with ΔTPA = -0.44±4.05 mm$^2$ and ICC = 0.985. The proposed algorithm trained on small datasets and segmented a different dataset without retraining with accuracy and precision that may be useful clinically and for research.

Vessel-wall-volume (VWV) is an important three-dimensional ultrasound (3DUS) metric used in the assessment of carotid plaque burden and monitoring changes in carotid atherosclerosis in response to medical treatment. To generate the VWV measurement, we proposed an approach that combined a voxel-based fully convolution network (Voxel-FCN) and a continuous max-flow module to automatically segment the carotid media-adventitia (MAB) and lumen-intima boundaries (LIB) from 3DUS images. Voxel-FCN includes an encoder consisting of a general 3D CNN and a 3D pyramid pooling module to extract spatial and contextual information, and a decoder using a concatenating module with an attention mechanism to fuse multi-level features extracted by the encoder. A continuous max-flow algorithm is used to improve the coarse segmentation provided by the Voxel-FCN. Using 1007 3DUS images, our approach yielded a Dice-similarity-coefficient (DSC) of 93.2±3.0% for the MAB in the common carotid artery (CCA), and 91.9±5.0% in the bifurcation by comparing algorithm and expert manual segmentations. We achieved a DSC of 89.5±6.7% and 89.3±6.8% for the LIB in the CCA and the bifurcation respectively. The mean errors between the algorithm-and manually-generated VWVs were 0.2±51.2 mm3for the CCA and -4.0±98.2 mm3for the bifurcation. The algorithm segmentation accuracy was comparable to intra-observer manual segmentation but our approach required less than 1s, which will not alter the clinical work-flow as 10s is required to image one side of the neck. Therefore, we believe that the proposed method could be used clinically for generating VWV to monitor progression and regression of carotid plaques.

Stochastic modeling in image analysis aims to represent the images features in a small number of parameters so as to recognize the source producing the images. In this paper we address the image segmentation problem in the case of significantly differ segments' sizes. A probabilistic model dealing the distribution of gray level in the observed image is based on the Gaussian Mixture Model identifying each component a segment. According to the general segmentation methodology for multi-modal gray levels images we presume that every region-of-interest attaches to a distinct substantial mode of the empirical distribution of gray levels. So, the number of the components is evaluated via a new resampling procedure involving the Expectation-Maximization algorithm used in order to estimate the significant histograms picks. Stable states of our model are associated within of the proposed method with the "true" segments quantities specified by the appropriate components' quantities. Numerical experiments demonstrate the high ability of the proposed method.

This paper presents a hybrid differential evolution (DE) with quantum-behaved particle swarm optimization (QPSO) for the unmanned aerial vehicle (UAV) route planning on the sea. The proposed method, denoted as DEQPSO, combines the DE algorithm with the QPSO algorithm in an attempt to further enhance the performance of both algorithms. The route planning for UAV on the sea is formulated as an optimization problem. A simple method of pretreatment to the terrain environment is proposed. A novel route planner for UAV is designed to generate a safe and flyable path in the presence of different threat environments based on the DEQPSO algorithm. To show the high performance of the proposed method, the DEQPSO algorithm is compared with the real-valued genetic algorithm, DE, standard particle swarm optimization (PSO), hybrid particle swarm with differential evolution operator, and QPSO in terms of the solution quality, robustness, and the convergence property. Experimental results demonstrate that the proposed method is capable of generating higher quality paths efficiently for UAV than any other tested optimization algorithms.

A new variant of particle swarm optimization (PSO), named phase angle-encoded and quantum-behaved particle swarm optimization (θ-QPSO), is proposed. Six versions of θ-QPSO using different mappings are presented and compared through their application to solve continuous function optimization problems. Several representative benchmark functions are selected as testing functions. The real-valued genetic algorithm (GA), differential evolution (DE), standard particle swarm optimization (PSO), phase angle-encoded particle swarm optimization ( θ-PSO), quantum-behaved particle swarm optimization (QPSO), and θ-QPSO are tested and compared with each other on the selected unimodal and multimodal functions. To corroborate the results obtained on the benchmark functions, a new route planner for unmanned aerial vehicle (UAV) is designed to generate a safe and flyable path in the presence of different threat environments based on the θ-QPSO algorithm. The PSO, θ-PSO, and QPSO are presented and compared with the θ-QPSO algorithm as well as GA and DE through the UAV path planning application. Each particle in swarm represents a potential path in search space. To prune the search space, constraints are incorporated into the pre-specified cost function, which is used to evaluate whether a particle is good or not. Experimental results demonstrated good performance of the θ-QPSO in planning a safe and flyable path for UAV when compared with the GA, DE, and three other PSO-based algorithms.

Based on evolutionary computation, a novel real-time route planner for unmanned air vehicles is presented. In the evolutionary route planner, the individual candidates are evaluated with respect to the workspace so that the computation of the configuration space is not required. The planner incorporates domain-specific knowledge, can handle unforeseeable changes of the environment, and take into account different kinds of mission constraints such as minimum route leg length and flying altitude, maximum turning angle, and fixed approach vector to goal position. Furthermore, the novel planner can be used to plan routes both for a single vehicle and for multiple ones. With Digital Terrain Elevation Data, the resultant routes can increase the surviving probability of the vehicles using the terrain masking effect.

Based on evolutionary computation, a new 3D route planner for unmanned air vehicles is presented. In our evolutionary route planner, the individual candidates are evaluated with respect to the workspace. Therefore a computation of the configuration space is avoided. With Digital Terrain Elevation Data, our approach can find a near-optimal route that can increase the surviving probability efficiently. By using a problem-specific representation of candidate solutions and genetic operators, the routes are generated in real-time and are able to take into account different kinds of mission constraints such as minimum route leg length and flying altitude, maximum turning angle, and fixed approach vector to goal position, etc.

This paper presents a novel curve based algorithm of stereo vision to reconstruct 3D line-like objects. B-spline approximations of 2D edge curves are selected as primitives for the reconstruction of their corresponding space curves so that, under the assumption of affine camera model, a 3D curve can be derived from reconstructing its control points according to the affine invariant property of B-Spline curves. The superiority of B-spline model in representing free-form curves gives good geometric properties of reconstruction results. Both theoretical analysis and experimental results demonstrate the validity of our approach.