In this paper, we study the adversarial robustness of deep neural networks (DNN) for classification against optimal classifiers. We look at the smallest magnitude of possible additive perturbations that can change a classifier's output. We provide a matrix-theoretic explanation of the adversarial fragility of DNNs for classification. In particular, our theoretical results show that the adversarial robustness of a neural network can degrade as the input dimension d increases. Analytically, we show that the adversarial robustness of neural networks can be only 1/√d of the best possible adversarial robustness of optimal classifiers. Our theories match remarkably well with empirical results. The matrix-theoretic explanation aligns with an earlier information-theoretic feature-compression-based explanation for the adversarial fragility of neural networks.

This paper considers the problem of recovering signals modeled by generative models from linear measurements contaminated with sparse outliers. We propose an outlier detection approach for reconstructing the ground-truth signals by solving an$\ell _{1}$norm minimization problem. We establish theoretical recovery guarantees for reconstruction of signals using generative models in the presence of outliers, giving lower bounds on the number of correctable outliers. Our results are applicable to both linear and nonlinear generator neural networks with an arbitrary number of layers. We propose an iterative and linearized alternating direction method of multipliers (ADMM) algorithm for solving the outlier detection problem via$\ell _{1}$norm minimization, and a gradient descent algorithm for solving the outlier detection problem via squared$\ell _{1}$norm minimization. We conduct extensive experiments using variational auto-encoder and deep convolutional generative adversarial networks, and the experimental results show that the signals can be successfully reconstructed under outliers using our approach. Our approach outperforms the traditional Lasso and$\ell _{2}$norm minimization approach.

This paper considers a novel application of deep AUC maximization (DAM) for multi-instance learning (MIL), in which a single class label is assigned to a bag of instances (e.g., multiple 2D slices of a CT scan for a patient). We address a neglected yet non-negligible computational challenge of MIL in the context of DAM, i.e., bag size is too large to be loaded into GPU memory for backpropagation, which is required by the standard pooling methods of MIL. To tackle this challenge, we propose variance-reduced stochastic pooling methods in the spirit of stochastic optimization by formulating the loss function over the pooled prediction as a multi-level compositional function. By synthesizing techniques from stochastic compositional optimization and non-convex min-max optimization, we propose a unified and provable muli-instance DAM (MIDAM) algorithm with stochastic smoothed-max pooling or stochastic attention-based pooling, which only samples a few instances for each bag to compute a stochastic gradient estimator and to update the model parameter. We establish a similar convergence rate of the proposed MIDAM algorithm as the state-of-the-art DAM algorithms. Our extensive experiments on conventional MIL datasets and medical datasets demonstrate the superiority of our MIDAM algorithm. The method is open-sourced at https://libauc.org/.

The rapid spread of SARS-CoV-2 has placed a significant burden on public health systems to provide swift and accurate diagnostic testing highlighting the critical need for innovative testing approaches for future pandemics. In this study, we present a novel sample pooling procedure based on compressed sensing theory to accurately identify virally infected patients at high prevalence rates utilizing an innovative viral RNA extraction process to minimize sample dilution. At prevalence rates ranging from 0-14.3%, the number of tests required to identify the infection status of all patients was reduced by 69.26% as compared to conventional testing in primary human SARS-CoV-2 nasopharyngeal swabs and a coronavirus model system. Our method provided quantification of individual sample viral load within a pool as well as a binary positive-negative result. Additionally, our modified pooling and RNA extraction process minimized sample dilution which remained constant as pool sizes increased. Compressed sensing can be adapted to a wide variety of diagnostic testing applications to increase throughput for routine laboratory testing as well as a means to increase testing capacity to combat future pandemics.

Scribbles in scribble-based interactive segmentation such as graph-cut are usually assumed to be perfectly accurate, i.e., foreground scribble pixels will never be segmented as background in the final segmentation. However, it can be hard to draw perfectly accurate scribbles, especially on fine structures of the image or on mobile touch-screen devices. In this paper, we propose a novel ratio energy function that tolerates errors in the user input while encouraging maximum use of the user input information. More specifically, the ratio energy aims to minimize the graph-cut energy while maximizing the user input respected in the segmentation. The ratio energy function can be exactly optimized using an efficient iterated graph cut algorithm. The robustness of the proposed method is validated on the GrabCut dataset using both synthetic scribbles and manual scribbles. The experimental results show that the proposed algorithm is robust to the errors in the user input and preserves the "anchoring" capability of the user input.

When segmenting intraretinal layers from multiple optical coherence tomography (OCT) images forming a mosaic or a set of repeated scans, it is attractive to exploit the additional information from the overlapping areas rather than discarding it as redundant, especially in low contrast and noisy images. However, it is currently not clear how to effectively combine the multiple information sources available in the areas of overlap. In this paper, we propose a novel graph-theoretic method for multi-surface multi-field co-segmentation of intraretinal layers, assuring consistent segmentation of the fields across the overlapped areas. After 2-D en-face alignment, all the fields are segmented simultaneously, imposing a priori soft interfield-intrasurface constraints for each pair of overlapping fields. The constraints penalize deviations from the expected surface height differences, taken to be the depth-axis shifts that produce the maximum cross-correlation of pairwise-overlapped areas. The method's accuracy and reproducibility are evaluated qualitatively and quantitatively on 212 OCT images (20 nine-field, 32 single-field acquisitions) from 26 patients with glaucoma. Qualitatively, the obtained thickness maps show no stitching artifacts, compared to pronounced stitches when the fields are segmented independently. Quantitatively, two ophthalmologists manually traced four intraretinal layers on 10 patients, and the average error ( 4.58 ±1.46 μm) was comparable to the average difference between the observers ( 5.86±1.72 μm). Furthermore, we show the benefit of the proposed approach in co-segmenting longitudinal scans. As opposed to segmenting layers in each of the fields independently, the proposed co-segmentation method obtains consistent segmentations across the overlapped areas, producing accurate, reproducible, and artifact-free results.

Identification of both the luminal and the wall areas of the bronchial tree structure from volumetric X-ray computed tomography (CT) data sets is of critical importance in distinguishing important phenotypes within numerous major lung diseases including chronic obstructive pulmonary diseases (COPD) and asthma. However, accurate assessment of the inner and outer airway wall surfaces of a complete 3-D tree structure is difficult due to their complex nature, particularly around the branch areas. In this paper, we extend a graph search based technique (LOGISMOS) to simultaneously identify multiple inter-related surfaces of branching airway trees. We first perform a presegmentation of the input 3-D image to obtain basic information about the tree topology. The presegmented image is resampled along judiciously determined paths to produce a set of vectors of voxels (called voxel columns). The resampling process utilizes medial axes to ensure that voxel columns of appropriate lengths and directions are used to capture the object surfaces without interference. A geometric graph is constructed whose edges connect voxels in the resampled voxel columns and enforce validity of the smoothness and separation constraints on the sought surfaces. Cost functions with directional information are employed to distinguish inner and outer walls. The assessment of wall thickness measurement on a CT-scanned double-wall physical phantom (patterned after an in vivo imaged human airway tree) achieved highly accurate results on the entire 3-D tree. The observed mean signed error of wall thickness ranged from -0.09 ±0.24 mm to 0.07 ±0.23 mm in bifurcating/nonbifurcating areas. The mean unsigned errors were 0.16±0.12 mm to 0.20±0.11 mm. When the airway wall surface was partitioned into meaningful subregions, the airway wall thickness accuracy was the same in most tested bifurcation/nonbifurcation and carina/noncarina regions (p=NS). Once validated on phantoms, our method was applied to human in vivo volumetric CT data to demonstrate relationships of airway wall thickness as a function of luminal dimension and airway tree generation. Wall thickness differences between the bifurcation/nonbifurcation regions were statistically significant (p < 0.05) for tree generations 6, 7, 8, and 9. In carina/noncarina regions, the wall thickness was statistically different in generations 1, 4, 5, 6, 7, and 8.

Segmentation of multiple surfaces in medical images is a challenging problem, further complicated by the frequent presence of weak boundary evidence, large object deformations, and mutual influence between adjacent objects. This paper reports a novel approach to multi-object segmentation that incorporates both shape and context prior knowledge in a 3-D graph-theoretic framework to help overcome the stated challenges. We employ an arc-based graph representation to incorporate a wide spectrum of prior information through pair-wise energy terms. In particular, a shape-prior term is used to penalize local shape changes and a context-prior term is used to penalize local surface-distance changes from a model of the expected shape and surface distances, respectively. The globally optimal solution for multiple surfaces is obtained by computing a maximum flow in a low-order polynomial time. The proposed method was validated on intraretinal layer segmentation of optical coherence tomography images and demonstrated statistically significant improvement of segmentation accuracy compared to our earlier graph-search method that was not utilizing shape and context priors. The mean unsigned surface positioning errors obtained by the conventional graph-search approach (6.30 ±1.58 μ m) was improved to 5.14±0.99 μ m when employing our new method with shape and context priors.

Positron emission tomography (PET)-computed tomography (CT) images have been widely used in clinical practice for radiotherapy treatment planning of the radiotherapy. Many existing segmentation approaches only work for a single imaging modality, which suffer from the low spatial resolution in PET or low contrast in CT. In this work, we propose a novel method for the co-segmentation of the tumor in both PET and CT images, which makes use of advantages from each modality: the functionality information from PET and the anatomical structure information from CT. The approach formulates the segmentation problem as a minimization problem of a Markov random field model, which encodes the information from both modalities. The optimization is solved using a graph-cut based method. Two sub-graphs are constructed for the segmentation of the PET and the CT images, respectively. To achieve consistent results in two modalities, an adaptive context cost is enforced by adding context arcs between the two sub-graphs. An optimal solution can be obtained by solving a single maximum flow problem, which leads to simultaneous segmentation of the tumor volumes in both modalities. The proposed algorithm was validated in robust delineation of lung tumors on 23 PET-CT datasets and two head-and-neck cancer subjects. Both qualitative and quantitative results show significant improvement compared to the graph cut methods solely using PET or CT.

Many computer vision applications can be formulated as labeling problems. However, multilabeling problems are usually very challenging to solve, especially when some ordering constraints are enforced. We solve in this paper a five-parts labeling problem proposed in [6, 7]. In this model, one wants to find an optimal labeling for an image with five possible parts: “left”, “right”, “top”, “bottom” and “center”. The geometric ordering constraints can be read naturally from the names. No previous method can solve the problem with globally optimal solutions in a linear space complexity. We propose an efficient dynamic programming based algorithm which guarantees the global optimal labeling for the five-parts model. The time complexity is O(N1.5) and the space complexity is O(N), with N being the number of pixels in the image. In practice, it runs faster than previous methods. Moreover, it works for both 4-neighborhood and 8-neighborhood settings, and can be easily parallelized for GPU.

In this paper, we propose a novel method to reduce the magnitude of 4D CT artifacts by stitching two images with a data-driven regularization constrain, which helps preserve the local anatomy structures. Our method first computes an interface seam for the stitching in the overlapping region of the first image, which passes through the "smoothest" region, to reduce the structure complexity along the stitching interface. Then, we compute the displacements of the seam by matching the corresponding interface seam in the second image. We use sparse 3D features as the structure cues to guide the seam matching, in which a regularization term is incorporated to keep the structure consistency. The energy function is minimized by solving a multiple-label problem in Markov Random Fields with an anatomical structure preserving regularization term. The displacements are propagated to the rest of second image and the two image are stitched along the interface seams based on the computed displacement field. The method was tested on both simulated data and clinical 4D CT images. The experiments on simulated data demonstrated that the proposed method was able to reduce the landmark distance error on average from 2.9 mm to 1.3 mm, outperforming the registration-based method by about 55%. For clinical 4D CT image data, the image quality was evaluated by three medical experts, and all identified much fewer artifacts from the resulting images by our method than from those by the compared method.

Efficient segmentation of globally optimal surfaces in volumetric images is a central problem in many medical image analysis applications. Intraclass variance has been successfully utilized for object segmentation, for instance, in the Chan-Vese model, especially for images without prominent edges. In this paper, we study the optimization problem of detecting a region (volume) between two coupled smooth surfaces by minimizing the intraclass variance using an efficient polynomial-time algorithm. Our algorithm is based on the shape probing technique in computational geometry and computes a sequence of minimum-cost closed sets in a derived parametric graph. The method has been validated on computer-synthetic volumetric images and in X-ray CT-scanned datasets of plexiglas tubes of known sizes. Its applicability to clinical data sets was also demonstrated. In all cases, the approach yielded highly accurate results. We believe that the developed technique is of interest on its own. We expect that it can shed some light on solving other important optimization problems arising in medical imaging. Furthermore, we report an approximation algorithm which runs much faster than the exact algorithm while yielding highly comparable segmentation accuracy.

Multiple surface searching with only image intensity information is a difficult job in the presence of high noise and weak edges. We present in this paper a novel method for globally optimal multi-surface searching with a shape prior represented by convex pairwise energies. A 3-D graph-theoretic framework is employed. An arc-weighted graph is constructed based on a shape model built from training datasets. A wide spectrum of constraints is then incorporated. The shape prior term penalizes the local topological change from the original shape model. The globally optimal solution for multiple surfaces can be obtained by computing a maximum flow in low-order polynomial time. Compared with other graph-based methods, our approach provides more local and flexible control of the shape. We also prove that our algorithm can handle the detection of multiple crossing surfaces with no shared voxels. Our method was applied to several application problems, including medical image segmentation, scenic image segmentation, and image resizing. Compared with results without using shape prior information, our improvement was quite impressive, demonstrating the promise of our method.

A novel method for simultaneous segmentation of multiple interacting surfaces belonging to multiple interacting objects, called LOGISMOS (layered optimal graph image segmentation of multiple objects and surfaces), is reported. The approach is based on the algorithmic incorporation of multiple spatial inter-relationships in a single n-dimensional graph, followed by graph optimization that yields a globally optimal solution. The LOGISMOS method's utility and performance are demonstrated on a bone and cartilage segmentation task in the human knee joint. Although trained on only a relatively small number of nine example images, this system achieved good performance. Judged by dice similarity coefficients (DSC) using a leave-one-out test, DSC values of 0.84 ± 0.04, 0.80 ± 0.04 and 0.80 ± 0.04 were obtained for the femoral, tibial, and patellar cartilage regions, respectively. These are excellent DSC values, considering the narrow-sheet character of the cartilage regions. Similarly, low signed mean cartilage thickness errors were obtained when compared to a manually-traced independent standard in 60 randomly selected 3-D MR image datasets from the Osteoarthritis Initiative database-0.11 ± 0.24, 0.05 ± 0.23, and 0.03 ± 0.17 mm for the femoral, tibial, and patellar cartilage thickness, respectively. The average signed surface positioning errors for the six detected surfaces ranged from 0.04 ± 0.12 mm to 0.16 ± 0.22 mm. The reported LOGISMOS framework provides robust and accurate segmentation of the knee joint bone and cartilage surfaces of the femur, tibia, and patella. As a general segmentation tool, the developed framework can be applied to a broad range of multiobject multisurface segmentation problems.

Content-aware video/image resizing is of increasing relevance to allow high-quality image and video resizing to be displayed on devices with different resolution. In this paper, we present a novel algorithm to find multiple 3-D surfaces simultaneously with globally optimal solution for video/image resizing. Our algorithm is based on graph theory and it first analyzes the video/image data to define the energy value for each voxel. Then, a 4-D graph is constructed and the costs are assigned according to the energy values. Finally, multiple 3-D surfaces are detected by a global optimization process which can be solved via s-t graph cuts. By removing or inserting these multiple 3-D surfaces, content-aware video/image resizing is achieved. We also have proved that our algorithm can find the globally optimal solution for crossing surfaces problem, in which several surfaces can cross each other. The proposed method is demonstrated on a variety of video/image data and compared to the state of the art in video/image resizing.

With the introduction of spectral-domain optical coherence tomography (OCT), much larger image datasets are routinely acquired compared to what was possible using the previous generation of time-domain OCT. Thus, the need for 3-D segmentation methods for processing such data is becoming increasingly important. We report a graph-theoretic segmentation method for the simultaneous segmentation of multiple 3-D surfaces that is guaranteed to be optimal with respect to the cost function and that is directly applicable to the segmentation of 3-D spectral OCT image data. We present two extensions to the general layered graph segmentation method: the ability to incorporate varying feasibility constraints and the ability to incorporate true regional information. Appropriate feasibility constraints and cost functions were learned from a training set of 13 spectral-domain OCT images from 13 subjects. After training, our approach was tested on a test set of 28 images from 14 subjects. An overall mean unsigned border positioning error of 5.69+/-2.41 microm was achieved when segmenting seven surfaces (six layers) and using the average of the manual tracings of two ophthalmologists as the reference standard. This result is very comparable to the measured interobserver variability of 5.71+/-1.98 microm.

Efficient segmentation of globally optimal surfaces in volumetric images is a central problem in many medical image analysis applications. Intra-class variance has been successfully utilized, for instance, in the Chan-Vese model especially for images without prominent edges. In this paper, we study the optimization problem of detecting a region (volume) between two coupled smooth surfaces by minimizing the intra-class variance using an efficient polynomial-time algorithm. Our algorithm is based on the shape probing technique in computational geometry and computes a sequence of minimum-cost closed sets in a derived parametric graph. The method has been validated on computer-synthetic volumetric images and in X-ray CT-scanned datasets of plexiglas tubes of known sizes. Its applicability to clinical data sets was demonstrated in human CT image data. The achieved results were highly accurate with mean signed surface positioning errors of the inner and outer walls of the tubes of +0.013mm and 0.012mm, respectively, given a voxel size of 0.39 × 0.39 × 0.6mm3. Comparing with the original Chan-Vese method [8], our algorithm expressed higher robustness. With its polynomialtime efficiency, our algorithm is ready to be extended to higher-dimensional image segmentation. In addition, the developed technique is of its own interest. We expect that it can shed some light on solving other important optimization problems arising in computer vision. To the best of our knowledge, the shape probing technique is for the first time introduced into the field of computer vision.

Current techniques for segmenting macular optical coherence tomography (OCT) images have been 2-D in nature. Furthermore, commercially available OCT systems have only focused on segmenting a single layer of the retina, even though each intraretinal layer may be affected differently by disease. We report an automated approach for segmenting (anisotropic) 3-D macular OCT scans into five layers. Each macular OCT dataset consisted of six linear radial scans centered at the fovea. The six surfaces defining the five layers were identified on each 3-D composite image by transforming the segmentation task into that of finding a minimum-cost closed set in a geometric graph constructed from edge/regional information and a priori determined surface smoothness and interaction constraints. The method was applied to the macular OCT scans of 12 patients (24 3-D composite image datasets) with unilateral anterior ischemic optic neuropathy (AION). Using the average of three experts' tracings as a reference standard resulted in an overall mean unsigned border positioning error of 6.1 +/- 2.9 microm, a result comparable to the interobserver variability (6.9 +/- 3.3 microm). Our quantitative analysis of the automated segmentation results from AION subject data revealed that the inner retinal layer thickness for the affected eye was 24.1 microm (21%) smaller on average than for the unaffected eye (p < 0.001), supporting the need for segmenting the layers separately.

Efficient segmentation of globally optimal surfaces representing object boundaries in volumetric data sets is important and challenging in many medical image analysis applications. We have developed an optimal surface detection method capable of simultaneously detecting multiple interacting surfaces, in which the optimality is controlled by the cost functions designed for individual surfaces and by several geometric constraints defining the surface smoothness and interrelations. The method solves the surface segmentation problem by transforming it into computing a minimum s-t cut in a derived arc-weighted directed graph. The proposed algorithm has a low-order polynomial time complexity and is computationally efficient. It has been extensively validated on more than 300 computer-synthetic volumetric images, 72 CT-scanned data sets of different-sized plexiglas tubes, and tens of medical images spanning various imaging modalities. In all cases, the approach yielded highly accurate results. Our approach can be readily extended to higher-dimensional image segmentation.

In this paper, we present a new geometric algorithm for the 3-D static leaf sequencing (SLS) problem arising in intensity-modulated radiation therapy (IMRT), a modern cancer treatment technique. The treatment time and machine delivery error are two crucial factors for measuring the quality of a solution (i.e., a treatment plan) for the SLS problem. In the current clinical practice, physicians prefer to use treatment plans with the lowest possible amount of delivery error, and are also very concerned about the treatment time. Previous SLS methods in both the literature and commercial treatment planning systems either cannot minimize the error or achieve that only by treatment plans which require a prolonged treatment time. In comparison, our new geometric algorithm is computationally efficient; more importantly, it guarantees that the output treatment plans have the lowest possible amount of delivery error, and the treatment time for the plans is significantly shorter. Our solution is based on a number of novel schemes and ideas (e.g., mountain reduction, block matching, profile-preserving cutting, etc) which may be of interest in their own right. Experimental results based on real medical data showed that our new algorithm runs fast and produces much better quality treatment plans than current commercial planning systems and well-known algorithms in medical literature.

The static leaf sequencing (SLS) problem arises in radiation therapy for cancer treatments, aiming to accomplish the delivery of a radiation prescription to a target tumor in the minimum amount of delivery time. Geometrically, the SLS problem can be formulated as a 3-D partition problem for which the 2-D problem of partitioning a polygonal domain (possibly with holes) into a minimum set of monotone polygons is a special case. In this paper, we present new geometric algorithms for a basic case of the 3-D SLS problem (which is also of clinical value) and for the general 3-D SLS problem. Our basic 3-D SLS algorithm, based on new geometric observations, produces guaranteed optimal quality solutions using Steiner points in polynomial time; the previously best known basic 3-D SLS algorithm gives optimal outputs only for the case without any Steiner points, and its time bound involves a multiplicative factor of a factorial function of the input. Our general 3-D SLS algorithm is based on our basic 3-D SLS algorithm and a polynomial time algorithm for partitioning a polygonal domain (possibly with holes) into a minimum set of x-monotone polygons, and has a fast running time. Experiments and comparisons using real medical data and on a real radiotherapy machine have shown that our 3-D SLS algorithms and software produce treatment plans that use significantly shorter delivery time and give better treatment quality than the current most popular commercial treatment planning system and the most well-known SLS algorithm. Some of our techniques and geometric procedures (e.g., for the problem of partitioning a polygonal domain into a minimum set of x-monotone polygons) are interesting in their own right.

We present efficient algorithms for solving the problem of computing an optimal penetration (a ray or a line segment) among weighted regions in 2-D and 3-D spaces.This problem finds applications in several areas, such as radiation therapy, geological exploration, and environmental engineering.Our algorithms are based on a combination of geometric techniques and optimization methods.Our geometric analysis shows that the optimal penetration problem in d-D (d = 2,3) can be reduced to solving O(n2td-l)) instances of certain special types of nonlinear optimization problems, where n is the total number of vertices of the regions.We also give implementation results of our 2-D algorithms. IntroductionIn this paper, we study the following geometric optimization problem (called optimal penetration problem): Given a subdivision R with a total of n vertices in 2-D or 3-D space, divided in m regions R..i, i = 1,2,. . ., m, find a ray L such that L