Multi-contrast MRI (MC-MRI) captures multiple complementary imaging modalities to aid in radiological decision-making. Given the need for lowering the time cost of multiple acquisitions, current deep accelerated MRI reconstruction networks focus on exploiting the redundancy between multiple contrasts. However, existing works are largely supervised with paired data and/or prohibitively expensive fully-sampled MRI sequences. Further, reconstruction networks typically rely on convolutional architectures which are limited in their capacity to model long-range interactions and may lead to suboptimal recovery of fine anatomical detail. To these ends, we present a dual-domain self-supervised transformer (DSFormer) for accelerated MC-MRI reconstruction. DSFormer develops a deep conditional cascade transformer (DCCT) consisting of cascaded Swin transformer reconstruction networks (SwinRN) trained under two deep conditioning strategies to enable MC-MRI information sharing. We further use a dual-domain (image and k-space) self-supervised learning strategy for DCCT to alleviate the costs of acquiring fully sampled training data. DSFormer generates high-fidelity reconstructions which outperform current fully-supervised baselines and approach the performance of full supervision.

Statistical learning theory relies on an assumption that the joint distributions of observations and labels are the same in training and testing data. However, this assumption is violated in many real world problems, such as training a detector of malicious network traffic that can change over time as a result of attacker's detection evasion efforts. We propose to address this problem by creating an optimized representation, which significantly increases the robustness of detectors or classifiers trained under this distributional shift. The representation is created from bags of samples (e.g. network traffic logs) and is designed to be invariant under shifting and scaling of the feature values extracted from the logs and under permutation and size changes of the bags. The invariance is achieved by combining feature histograms with feature self-similarity matrices computed for each bag and significantly reduces the difference between the training and testing data. The parameters of the representation, such as histogram bin boundaries, are learned jointly with the classifier. We show that the representation is effective for training a detector of malicious traffic, achieving 90% precision and 67% recall on samples of previously unseen malware variants.

Routine ultrasound exam in the second and third trimesters of pregnancy involves manually measuring fetal head and brain structures in 2-D scans. The procedure requires a sonographer to find the standardized visualization planes with a probe and manually place measurement calipers on the structures of interest. The process is tedious, time consuming, and introduces user variability into the measurements. This paper proposes an automatic fetal head and brain (AFHB) system for automatically measuring anatomical structures from 3-D ultrasound volumes. The system searches the 3-D volume in a hierarchy of resolutions and by focusing on regions that are likely to be the measured anatomy. The output is a standardized visualization of the plane with correct orientation and centering as well as the biometric measurement of the anatomy. The system is based on a novel framework for detecting multiple structures in 3-D volumes. Since a joint model is difficult to obtain in most practical situations, the structures are detected in a sequence, one-by-one. The detection relies on Sequential Estimation techniques, frequently applied to visual tracking. The interdependence of structure poses and strong prior information embedded in our domain yields faster and more accurate results than detecting the objects individually. The posterior distribution of the structure pose is approximated at each step by sequential Monte Carlo. The samples are propagated within the sequence across multiple structures and hierarchical levels. The probabilistic model helps solve many challenges present in the ultrasound images of the fetus such as speckle noise, signal drop-out, shadows caused by bones, and appearance variations caused by the differences in the fetus gestational age. This is possible by discriminative learning on an extensive database of scans comprising more than two thousand volumes and more than thirteen thousand annotations. The average difference between ground truth and automatic measurements is below 2 mm with a running time of 6.9 s (GPU) or 14.7 s (CPU). The accuracy of the AFHB system is within inter-user variability and the running time is fast, which meets the requirements for clinical use.

In this paper, we propose a novel framework for detecting multiple objects in 2D and 3D images. Since a joint multi-object model is difficult to obtain in most practical situations, we focus here on detecting the objects sequentially, one-by-one. The interdependence of object poses and strong prior information embedded in our domain of medical images results in better performance than detecting the objects individually. Our approach is based on Sequential Estimation techniques, frequently applied to visual tracking. Unlike in tracking, where the sequential order is naturally determined by the time sequence, the order of detection of multiple objects must be selected, leading to a Hierarchical Detection Network (HDN). We present an algorithm that optimally selects the order based on probability of states (object poses) within the ground truth region. The posterior distribution of the object pose is approximated at each step by sequential Monte Carlo. The samples are propagated within the sequence across multiple objects and hierarchical levels. We show on 2D ultrasound images of left atrium, that the automatically selected sequential order yields low mean detection error. We also quantitatively evaluate the hierarchical detection of fetal faces and three fetal brain structures in 3D ultrasound images.

In this paper, we present a method to automatically extract the vessel segments and construct the vascular tree with anatomical realism from a color retinal image. The significance of the work is to assist in clinical studies of diagnosis of cardio-vascular diseases, such as hypertension,which manifest abnormalities in either venous and/or arterial vascular systems. To maximize the completeness of vessel extraction, we introduce vessel connectiveness measure to improve on an existing algorithm which applies multiscale matched filtering and vessel likelihood measure.Vessel segments are grouped using extended Kalman filter to take into consideration continuities in curvature, width,and color changes at the bifurcation or crossover point. The algorithm is tested on five images from the DRIVE database,a mixture of normal and pathological images, and the results are compared with the ground truth images provided by a physician. The preliminary results show that our method reaches an average success rate of 92.1%.

The technique discussed in this article proposes to distinguish between unattended and stolen objects by combining shape and appearance similarity measures of foreground objects observed in consecutive frames of a video. Static objects are detected by examining trajectories and people are removed from consideration. Object shape boundary is first refined using active contours. Shape similarity is then defined by computing gradient magnitudes along the object boundaries and counting how many boundary pixels have values higher/lower than predefined thresholds. Appearance similarity is based on differences between histograms using the foreground mask on the current and background images. Probabilities are defined assuming the shape and appearance measures follow the Gaussian distribution (with trained parameters). Final measures of unattended/stolen objects are produced by averaging the probabilities and used to classify static-nonhuman objects. Experiments show that combining the three measures gives better results than using each of the measures alone.

In previous surveillance applications, algorithms for background modeling based on Gaussian mixture models (GMM) needed to specify two parameters: threshold T, which determines a proportion of the data that should be accounted for by the background, and a learning rate alpha specifying speed at which the distribution parameters change [Stauffer, CVPR 1999}. In the Basic Background Subtraction (BBS), foreground objects are found by subtracting a static foreground image. In the proposed algorithm, BBS is applied using background obtained from GMM. This way, threshold T is replaced by a foreground-background separation threshold S. The advantage is that S is less sensitive than T. To make the model respond faster to changes, recent observed value of the most dominant background component is used as a current value for a particular pixel, rather than the component mean value. Quantitative and qualitative results show the advantages of the proposed technique compared to GMM models.

The technique proposed in this paper combines three algorithms for shadow removal in indoor scenes. First algorithm takes an advantage of the assumption on the chromaticity consistency: the ambient light chromaticity is approximately the same as the chromaticity of the diffuse light. Magnitude of differences between chromaticity values of shadow and non-shadow regions for different hue values are different for RGB and HSV color spaces so both spaces are used in conjunction by setting thresholds on these differences. Second algorithm takes a potential shadow region and applies a threshold on the intensity reduction of each pixel and its neighboring pixels. It is ensured that the shadow regions are above minimum specified size. The third algorithm uses a threshold on the intensity reduction as a result of light being blocked by moving objects. This reduction depends on the background geometry, light source, moving object size, and position. The threshold reduction caused by a shadow is obtained by manually specifying shadow regions and recording these values which are later used for shadow removal in the same scene.

Retinal clinicians and researchers make extensive use of images, and the current emphasis is on digital imaging of the retinal fundus. The goal of this paper is to introduce a system, known as retinal image vessel extraction and registration system, which provides the community of retinal clinicians, researchers, and study directors an integrated suite of advanced digital retinal image analysis tools over the Internet. The capabilities include vasculature tracing and morphometry, joint (simultaneous) montaging of multiple retinal fields, cross-modality registration (color/red-free fundus photographs and fluorescein angiograms), and generation of flicker animations for visualization of changes from longitudinal image sequences. Each capability has been carefully validated in our previous research work. The integrated Internet-based system can enable significant advances in retina-related clinical diagnosis, visualization of the complete fundus at full resolution from multiple low-angle views, analysis of longitudinal changes, research on the retinal vasculature, and objective, quantitative computer-assisted scoring of clinical trials imagery. It could pave the way for future screening services from optometry facilities.

In this paper, we investigate the effect of substantial inter-image intensity changes and changes in modality on the performance of keypoint detection, description, and matching algorithms in the context of image registration. In doing so, we modify widely-used keypoint descriptors such as SIFT and shape contexts, attempting to capture the insight that some structural information is indeed preserved between images despite dramatic appearance changes. These extensions include (a) pairing opposite-direction gradients in the formation of orientation histograms and (b) focusing on edge structures only. We also compare the stability of MSER, Laplacian-of-Gaussian, and Harris corner keypoint location detection and the impact of detection errors on matching results. Our experiments on multimodal image pairs and on image pairs with significant intensity differences show that indexing based on our modified descriptors produces more correct matches on difficult pairs than current techniques at the cost of a small decrease in performance on easier pairs. This extends the applicability of image registration algorithms such as the Dual-Bootstrap which rely on correctly matching only a small number of keypoints.

This paper proposes a new registration algorithm, Co-variance Driven Correspondences (CDC), that depends fundamentally on the estimation of uncertainty in point correspondences. This uncertainty is derived from the covariance matrices of the individual point locations and from the covariance matrix of the estimated transformation parameters. Based on this uncertainty, CDC uses a robust objective function and an EM-like algorithm to simultaneously estimate the transformation parameters, their covariance matrix, and the likely correspondences. Unlike the Robust Point Matching (RPM) algorithm, CDC requires neither an annealing schedule nor an explicit outlier process. Experiments on synthetic and real images using a polynomial transformation models in 2D and in 3D show that CDC has a broader domain of convergence than the well-known Iterative Closest Point (ICP) algorithm and is more robust to missing or extraneous structures in the data than RPM.

Our goal is an automated 2D-image-pair registration algorithm capable of aligning images taken of a wide variety of natural and man-made scenes as well as many medical images. The algorithm should handle low overlap, substantial orientation and scale differences, large illumination variations, and physical changes in the scene. An important component of this is the ability to automatically reject pairs that have no overlap or have too many differences to be aligned well. We propose a complete algorithm, including techniques for initialization, for estimating transformation parameters, and for automatically deciding if an estimate is correct. Keypoints extracted and matched between images are used to generate initial similarity transform estimates, each accurate over a small region. These initial estimates are rank-ordered and tested individually in succession. Each estimate is refined using the Dual-Bootstrap ICP algorithm, driven by matching of multiscale features. A three-part decision criteria, combining measurements of alignment accuracy, stability in the estimate, and consistency in the constraints, determines whether the refined transformation estimate is accepted as correct. Experimental results on a data set of 22 challenging image pairs show that the algorithm effectively aligns 19 of the 22 pairs and rejects 99.8% of the misalignments that occur when all possible pairs are tried. The algorithm substantially out-performs algorithms based on keypoint matching alone.

In order to detect vessels at a variety of widths, we apply the matched filter at multiple scales (i.e., compute for multiple values and then combine the responses across scales). Unfortunately, the output amplitudes of spatial operators such as derivatives or matched filters generally decrease with increasing scale. To compensate for this effect, Lindeberg introduced gamma-normalized derivatives. We use this notion to define a gamma-normalized matched filter,

Our goal is a highly-reliable, fully-automated image registration technique that takes two images and correctly aligns them or decides that they can not be aligned. The technique should handle image pairs having low overlap, variations in scale, large illumination differences (e.g. day and night), substantial scene changes, and different modalities. Our approach is a combination of algorithms for initialization, estimation and refinement, and decision-making. It starts by extracting and matching keypoints. Rank-ordered matches are tested individually in succession. Each is used to generate a similarity transformation estimate in a small region of each image surrounding the matched keypoints. A generalization of the recently developed Dual-Bootstrap algorithm is then applied to generate an image-wide transformation estimate through a combination of matching and reestimation, model selection, and region growing, all driven by a new multiscale feature extraction technique. After convergence of the Dual-Bootstrap, the transformation is accepted if it passes a correctness test that combines measures of accuracy, stability and non-randomness; otherwise the process starts over with the next keypoint match. Experimental results on a suite of challenging image pairs shows the effectivenss of the complete system.

This paper presents a correspondence-based toolkit for image registration. Written in C++, the toolkit complements the capabilities of the insight toolkit (ITK). Major components include features, feature sets, match generators, error scale estimators, robust transformation estimators, and convergence testers, all combined and controlled by several different registration engines. Correspondence-based algorithms which can be implemented using the toolkit extend from ICP to hybrids of intensity-based and feature-based registration. The toolkit is being used both as an education tool and the foundation for developing new algorithms.

Motivated by the goals of improving detection of low-contrast and narrow vessels and eliminating false detections at nonvascular structures, a new technique is presented for extracting vessels in retinal images. The core of the technique is a new likelihood ratio test that combines matched-filter responses, confidence measures and vessel boundary measures. Matched filter responses are derived in scale-space to extract vessels of widely varying widths. A vessel confidence measure is defined as a projection of a vector formed from a normalized pixel neighborhood onto a normalized ideal vessel profile. Vessel boundary measures and associated confidences are computed at potential vessel boundaries. Combined, these responses form a six-dimensional measurement vector at each pixel. A training technique is used to develop a mapping of this vector to a likelihood ratio that measures the "vesselness" at each pixel. Results comparing this vesselness measure to matched filters alone and to measures based on the Hessian of intensities show substantial improvements, both qualitatively and quantitatively. The Hessian can be used in place of the matched filter to obtain similar but less-substantial improvements or to steer the matched filter by preselecting kernel orientations. Finally, the new vesselness likelihood ratio is embedded into a vessel tracing framework, resulting in an efficient and effective vessel centerline extraction algorithm.