Most attention-embedded networks fall short in effectively integrating spatial / channel-wise information across diverse scales, leading to suboptimal performance for coronary vessels segmentation in X-ray digital subtraction angiography images. To address this limitation, a multi-scale across attention incorporated network (named MS2A-Net) is introduced. MS2A-Net accepts original and enhanced images as inputs, leveraging complementary information provided by the different contrasts within the images. Furthermore, MS2A is designed to integrate features across multiple levels, scales and sources, for effectively extracting deep semantic information. After the incorporation of features with adaptive weightings, segmentation accuracy is refined. Qualitative and quantitative experiment results prove that MS2A-Net not only outperforms state-of-the-art tactics but also shows superior performance, e.g., higher intersection over union, Dice coefficient, broader areas under receiver operating characteristic curves.

Three-dimensional (3D) line segments contain richer geometric and structural information than 3D point clouds in man-made environments, which is beneficial for providing constraints to refine point-cloud-based mesh models or build accurate wireframes. However, the efficient reconstruction of 3D line segments with high scene coverage from multi-view images is still challenging. In this study, the depth maps obtained from the point cloud generation procedure are exploited to decrease the search range of two-dimensional (2D) line segment correspondences to improve the efficiency, precision, and recall rate of 2D line segment matching, thereby improving the construction efficiency and scene coverage of 3D line segments. For a line segment on the reference image (called reference line segment) of an image pair, a reliable virtual line segment is produced by projecting several sampled points of the reference line segment onto the search image based on the corresponding depth information. Then, a purely geometrical similarity measurement under the constraints of the virtual line segment is designed to obtain 2D line segment matches. Using the 2D line segment correspondences of all image pairs, a multi-view clustering operation is performed to construct 3D line segments from the redundant 2D matches. Finally, a simple 3D-line-segment-based mesh model refinement method is designed and the reconstructed 3D line segments are employed to improve the quality of the point-cloud-based mesh model. In our experiments, five open-source datasets are adopted to qualitatively and quantitatively evaluate the performance of the proposed 3D line segment reconstruction method and the potential of 3D line segments on mesh model refinement. The experimental results show that the proposed 3D line segment reconstruction method performs better than the state-of-the-art methods. Specifically, on five open-source datasets, our method exhibits an average improvement of 47.28% in the number of reconstructed 3D line segments over the best one among the compared methods. Additionally, the experimental results of the mesh model refinement show that the addition of 3D line segments is beneficial for improving the quality of the point-cloud-based mesh model.

Accurate assessment of the quality of concrete bridge decks and identification of corrosion induced delamination lead to economic management of bridge decks. It has been demonstrated that ground penetrating radar (GPR) can be successfully used for such purposes. The growing demand on GPR has brought into the challenge of developing automatic processes necessary to produce a final accurate interpretation. However, there have been few publications targeting at automatic detection of bridge deck delamination from GPR data. This paper proposes a novel method using partial differential equations to detect rebar (or steel-bar) mat signatures of concrete bridges from GPR data so that the delamination within the bridge deck can be effectively located. The proposed algorithm was tested on both synthetic and real GPR images and the experimental results have demonstrated its accuracy and reliability, even for diminished image contrast and low signal-to-noise ratio. Therefore, an accurate deterioration map of the bridge deck can be generated automatically.

Multi-exposure X-ray imaging can see through objects and separate different material into transparent layers. However, layer motion makes the separation task under-determined. Instead of aligning the non-rigid motion, we address the layer separation problem in gradient domain and propose an energy optimization framework to regularize it by explicitly enforcing independence constraint. It is shown that gradient domain allows more accurate and robust independence analysis between non-stationary signal using mutual information (MI) and hence achieves better separation. Furthermore, gradient fields contain sufficient information for full reconstruction of separated layers by solving the Poisson Equation. For efficient regularization of the gradient separation, energy terms based on the Taylor expansion of MI is further derived. Evaluation on both synthesized and real datasets proves the effectiveness of our algorithm and its robustness to complex tissue motion.

Principal curvatures and principal directions are fundamental local geometric properties. They are well defined on smooth surfaces. However, due to the nature as higher order differential quantities, they are known to be sensitive to noise. A recent work by Yang et al. combines principal component analysis with integral invariants and computes robust principal curvatures on multiple scales. Although the freedom of choosing the radius r gives results on different scales, in practice it is not an easy task to choose the most appropriate r for an arbitrary given model. Worse still, if the model contains features of different scales, a single r does not work well at all. In this work, we propose a scheme to automatically assign appropriate radii across the surface based on local surface characteristics. The radius r is not constant and adapts to the scale of local features. An efficient, iterative algorithm is used to approach the optimal assignment and the partition of unity is incorporated to smoothly combine the results with different radii. In this way, we can achieve a better balance between the robustness and the accuracy of feature locations. We demonstrate the effectiveness of our approach with robust principal direction field computation and feature extraction.

High-dynamic-range medical images have intensity values which cannot be visualized on current low-dynamic-range displays. In this paper, we introduce a fast range compression method which avoids common artifacts such as loss of contrast, haloes and gradient inversions. The proposed method first compresses the intensity range using a global transfer function. It then extracts and enhances weak structures using a multiscale decomposition and artifact correction. We show that artifact correction can be formulated as a linear-programming problem, for which we propose an efficient approximate solution. Experiments on real data demonstrate the effectiveness and speed of the proposed algorithm.

Dual energy X-ray imaging can separate materials of different properties into different layers for more detailed inspection. However, motion during the acquisition of dual energy images causes significant challenges to layer separation. We propose a novel separation algorithm based on the sparse features in gradient domain. Instead of decomposing the image intensity values, we separate sparse gradient features to different layers and then reconstruct the layers based on the decomposed gradient fields by solving the Poisson Equation. The structure tensor, based on directional quadrature filters, is used to classify and separate the gradient field. Evaluation on both synthesized data and real datasets proves its effectiveness and robustness to complex non-rigid motion.

Segmentation of arterial wall boundaries from intravascular images is an important problem for many applications in the study of plaque characteristics, mechanical properties of the arterial wall, its 3-D reconstruction, and its measurements such as lumen size, lumen radius, and wall radius. We present a shape-driven approach to segmentation of the arterial wall from intravascular ultrasound images in the rectangular domain. In a properly built shape space using training data, we constrain the lumen and media-adventitia contours to a smooth, closed geometry, which increases the segmentation quality without any tradeoff with a regularizer term. In addition to a shape prior, we utilize an intensity prior through a nonparametric probability-density-based image energy, with global image measurements rather than pointwise measurements used in previous methods. Furthermore, a detection step is included to address the challenges introduced to the segmentation process by side branches and calcifications. All these features greatly enhance our segmentation method. The tests of our algorithm on a large dataset demonstrate the effectiveness of our approach.

Traditional Bayesian restoration methods depend heavily on the accuracy of underlying generative models. For the challenging streak noise generated in the procedure of reconstruction from projections, Bayesian methods do not generalize well because accurate signal/noise models are not readily available. In this paper, we reformulate the reconstruction problem into a multi-image based restoration task and demonstrate that multiple images and mutual independence analysis can be utilized to significantly improve the generalization capability of traditional Bayesian frameworks in challenging scenarios. An efficient mutual independence analysis term is designed based on the properties of independent random variables to enforce the independent noise constraint between multiple images in an energy optimization framework, which can effectively detect and correct restoration error due to inaccurate generative models. Quantitative comparisons on phantom image and experiments on clinical scans both show significant improvements in accuracy and robustness of the proposed method.

Several studies dealt with medical ultrasound registration. Their similarity metrics relied on pixel-to-pixel intensity comparisons. Hence, they are not well suited to the case of speckled images. To better handle the speckle noise, our previous work proposed an information-theoretic feature detector-based registration approach. This work aims to extend it to the cases where the image speckle model is Rayleigh or normalized Fisher-Tippett distributed. Using speckle modeling based on these distributions, a speckle-speci.c informationtheoretic feature detector is constructed and applied to provide feature images. Those feature images are then registered using differential equations, the solution of which provides a transformation to bring the images into alignment. Compared to standard gradient-based techniques, the experimental results demonstrate the effectiveness of our method, particularly for low contrast ultrasound images.

Segmentation of ultrasound images is often a very challenging task due to speckle noise that contaminates the image. It is well known that speckle noise exhibits an asymmetric distribution as well as significant spatial correlation. Since these attributes can be difficult to model, many previous ultrasound segmentation methods oversimplify the problem by assuming that the noise is white and/or Gaussian, resulting in generic approaches that are actually more suitable to MR and X-ray segmentation than ultrasound. Unlike these methods, in this paper we present an ultrasound-specific segmentation approach that first decorrelates the image, and then performs segmentation on the whitened result using statistical region-based active contours. In particular, we design a gradient ascent flow that evolves the active contours to maximize a log likelihood functional based on the Fisher-Tippett distribution. We present experimental results that demonstrate the effectiveness of our method.

Image registration for X-ray dual energy imaging is challenging due to the overlaid transparent layers (i.e., the bone and soft tissue) and the different appearances between the dual images acquired with X-rays at different energy spectra. Moreover, subpixel accuracy is necessary for good reconstruction of the bone and soft-tissue layers. This paper addresses these problems with a novel coupled Bayesian framework, in which the registration and reconstruction can effectively reinforce each other. With the reconstruction results, we can design accurate matching criteria for aligning the dual images, instead of treating them as multi-modality registration. Furthermore, prior knowledge of the bone and soft tissue can be exploited to detect poor reconstruction due to inaccurate registration; and hence correct registration errors in the coupled framework. A multiscale freeform registration algorithm is implemented to achieve subpixel registration accuracy. Promising results are obtained in the experiments.

Traditional iterative tomographic reconstruction methods resort to gradient decent methods and require significant computation due to slow convergence. We divide the iterative reconstruction into two stages in the image and Radon spaces, respectively. First, we refine the reconstruction result by image space adaptive filtering. This finds a feasible update direction based on signal modeling. Second, we minimize the discrepancy between the sinograms along the update direction in Radon space and guarantee convergence. Reconstruction from clinical data using the proposed algorithm converges extremely fast and provides satisfactory reconstruction results in far fewer iterations than traditional methods.

We present a semi-automatic segmentation technique of the anatomical structures of the brain: cerebrum, cerebellum, and brain stem. The method uses graph cuts segmentation with an anatomic template for initialization. First, a skull stripping procedure is applied to remove non-brain tissues. Then, the segmentation is done hierarchically by first, extracting first the cerebrum from the brain, and then from the remaining volume the cerebellum and the brain stem are separated. This method is fast and can separate different anatomical structures of the brain in spite of weak boundaries. We describe our approach and present experimental results demonstrating its usefulness.

Accurate staging of nodal cancer still relies on surgical exploration because many primary malignancies spread via lymphatic dissemination. The purpose of this study was to utilize nanoparticle-enhanced lymphotropic magnetic resonance imaging (LN-MRI) to explore semi-automated noninvasive nodal cancer staging. We present a joint image segmentation and registration approach, which makes use of the problem specific information to increase the robustness of the algorithm to noise and weak contrast often observed in medical imaging applications. The effectiveness of the approach is demonstrated with a given lymph node segmentation problem in post-contrast pelvic MRI sequences.

Various methods have been proposed for image enhancement and restoration. The main difficulty is how to enhance the structures uniformly while suppressing the noise without artifacts. In this paper, we tackle this problem in the gradient domain instead of the traditional intensity domain. By enhancing the gradient field, we can enhance the structure uniformly without overshooting at the boundary. Because the gradient field is very sensitive to noise, we apply an orientation-isotropy adaptive filter to the gradient field, suppressing the gradients in the noise regions while enhancing along the object boundaries. Thus we obtain a modulated gradient field, which is usually not integrable. We reconstruct the enhanced image from the modulated gradient field with least square errors by solving a Poisson equation. This method can enhance the object contrast uniformly, suppress the noise with no artifacts, and avoid setting stopping time as in PDE methods. Experiments on noisy images show the efficacy of our method.

Image restoration has been extensively studied in the past. But multi-image based restoration/compounding is still surprisingly primitive. It usually starts with weighted averaging of the multiple images followed by single-image based restoration methods, which discards the abundant information hinted in the multiple images that can help the restoration process. In this paper, we utilize the fact that the images are corrupted by independent noise and design a new independence measurement based on the properties of independent random variables. The new independence measurement can be efficiently evaluated and imposed as an energy term into the traditional maximum a posteriori (MAP) framework, compensating to the generative models of signal and noise. It can effectively prevent the signal from being smoothed out as noise and hence dramatically improve the restoration quality and robustness, especially when accurate noise/signal models are difficult to obtain. Experiments on real medical images show very promising results.

Bayesian methods have been extensively used in various applications. However, there are two intrinsic issues rarely addressed, namely generalization and validity, in the context of multiple image restoration, we show that traditional Bayesian methods are sensitive to model errors and cannot guarantee valid results satisfying the underlying prior knowledge, e.g. independent noise property. To improve the Bayesian framework's generalization, we propose to explicitly enforce the validity of the result. Independent noise prior is very important but largely under-utilized in previous literature. In this paper, we use mutual information (MI) to explicitly enforce the independence. Efficient approximations based on Taylor expansion are proposed to adapt MI into standard energy forms to regularize the Bayesian methods. The new regularized Bayesian framework effectively utilizes the traditional generative signal/noise models but is much more robust to various model errors, as demonstrated in experiments on some demanding imaging applications.

Layered manufacturing is an additive manufacturing process where the parts are built layer by layer based on the geometry provided by a CAD system. There are several different technologies for layered manufacturing. We focus on fused deposition of ceramics. Here layers are formed by depositing material through a liquefier-head on adjacent roads in the shape of contours and/or rasters. As this technology is being pushed into new frontiers to build functional parts, the quality of the fabricated parts is becoming a major driving force for improving the technology. We present our preliminary results on an online monitoring system that can be used to detect defects. The ultimate objective is to use such a system to detect and remove the defects on any given layer prior to the next layer.

Direct layered manufacturing (LM) is an additive manufacturing process where part is built layer by layer based on its CAD file. Indirect LM is a process where mold for a part is built by direct LM process. In this work we demonstrate a machine vision setup for surface voids detection which can be used for surface quality inspection of parts built by indirect LM and certain direct LM techniques.

This paper presents the concept of process signature for the use of online defect detection in layered manufacturing. To achieve the high quality of parts built by the fused deposition of ceramics, an online process monitoring system is implemented to detect the processing defects. Using the process signature extracted from the original image captured by the monitoring system, an expected image is created which is then compared to the original image to detect and identify the defects.