In supervised machine learning, an agent is typically trained once and then deployed. While this works well for static settings, robots often operate in changing environments and must quickly learn new things from data streams. In this paradigm, known as streaming learning, a learner is trained online, in a single pass, from a data stream that cannot be assumed to be independent and identically distributed (iid). Streaming learning will cause conventional deep neural networks (DNNs) to fail for two reasons: 1) they need multiple passes through the entire dataset; and 2) non-iid data will cause catastrophic forgetting. An old fix to both of these issues is rehearsal. To learn a new example, rehearsal mixes it with previous examples, and then this mixture is used to update the DNN. Full rehearsal is slow and memory intensive because it stores all previously observed examples, and its effectiveness for preventing catastrophic forgetting has not been studied in modern DNNs. Here, we describe the ExStream algorithm for memory efficient rehearsal and compare it to alternatives. We find that full rehearsal can eliminate catastrophic forgetting in a variety of streaming learning settings, with ExStream performing well using far less memory and computation.

The Normalized Cut (NCut) objective function, widely used in data clustering and image segmentation, quantifies the cost of graph partitioning in a way that biases clusters or segments that are balanced towards having lower values than unbalanced partitionings. However, this bias is so strong that it avoids any singleton partitions, even when vertices are very weakly connected to the rest of the graph. Motivated by the Bühler-Hein family of balanced cut costs, we propose the family of Compassionately Conservative Balanced (CCB) Cut costs, which are indexed by a parameter that can be used to strike a compromise between the desire to avoid too many singleton partitions and the notion that all partitions should be balanced. We show that CCB-Cut minimization can be relaxed into an orthogonally constrained lt-minimization problem that coincides with the problem of computing Piecewise Flat Embeddings (PFE) for one particular index value, and we present an algorithm for solving the relaxed problem by iteratively minimizing a sequence of reweighted Rayleigh quotients (IRRQ). Using images from the BSDS500 database, we show that image segmentation based on CCB-Cut minimization provides better accuracy with respect to ground truth and greater variability in region size than NCut-based image segmentation.

LE-based methods have been shown to be effective for target detection in HSI. However, they can be slow due to the costly graph construction and eigenvector computation steps. In this paper, we proposed including a step of pre-segmenting an HSI into superpixels prior to dimensionality reduction. Carrying out experiments on an HIS from the SHARE 2012 data campaign, we show that incorporating superpixels in the BNC target detection method can yield much faster computation times without sacrificing accuracy. When incorporated in SE-based target detection, superpixels do cause a slight decrease in accuracy. Future work involves a more thorough validation on multiple datasets, and testing whether or not the inclusion of superpixels is useful for other target detection algorithms.

Since its introduction as a powerful graph-based method for image segmentation, the Normalized Cuts (NCuts) algorithm has been generalized to incorporate expert knowledge about how certain pixels or regions should be grouped, or how the resulting segmentation should be biased to be correlated with priors. Previous approaches incorporate hard must-link constraints on how certain pixels should be grouped as well as hard cannot-link constraints on how other pixels should be separated into different groups. In this paper, we reformulate NCuts to allow both sets of constraints to be handled in a soft manner, enabling the user to tune the degree to which the constraints are satisfied. An approximate spectral solution to the reformulated problem exists without requiring explicit construction of a large, dense matrix, hence, computation time is comparable to that of unconstrained NCuts. Using synthetic data and real imagery, we show that soft handling of constraints yields better results than unconstrained NCuts and enables more robust clustering and segmentation than is possible when the constraints are strictly enforced.

Dynamical systems analysis tools, like Recurrence Plotting (RP), allow for concise mathematical representations of complex systems with relatively simple descriptive metrics. These methods are invariant for phase-space trajectories of a time series from a dynamical system, allowing analyses on simplified data sets which preserve the system model's dynamics. In the past decade, recurrence methods have been applied to eye-tracking, but those analyses avoided Time-Delay Embedding (TDE). Without TDE, we lose the assumption that phase-space trajectories are being preserved in the recurrence plot. Thus, analysis has been typically limited to clustering fixation locations in the image space, instead of clustering data sequences in the phase space. We will show how classical recurrence analysis methods can be extended to allow for multi-modal data visualization and quantification, by presenting an open-source python implementation for analyzing eye movements.

Fractional partial differential equations can be used to model super-diffusive processes. Recent research into fractional PDEs has explored their impact on nonrigid image registration, which typically uses differential operators of integer order. In this paper, we propose a nonrigid registration algorithm that involves directly and rapidly solving a discretized fractional PDE at each time step that is subject to physically realistic image boundary conditions. The proposed algorithm is validated through registration experiments on breast MR imagery with simulated biomechanical deformations, indicating that the super-diffusive model yields lower average deformation errors than standard diffusion-based registration.

Spectral similarity measures can differentiate subtle differences between spectral footprints extracted from remotely sensed images. Prior research on urban scenes examined low-to-medium resolution images where many details of an urban scene are lost. For high spatial resolution imagers, a number of problems arise in the analysis of urban scenes since virtually all objects are resolved. Issues of spatially varying scene illumination and unavailability of surface reflectance data prevent classical spectral similarity measures from being extensively applied. Commonly used measures have limitations from a feature space perspective such as favoring either spectral direction or spectral magnitude. In this paper, a novel spectral similarity measure based on Mahalanobis distance is proposed to take into account the unique properties of high resolution urban scenes. A simplified radiometric transfer model is also incorporated. The results confirm the advantage of the new spectral similarity measure when applied to complex urban images.

Multi-modal image registration is a challenging problem in medical imaging. The goal is to align anatomically identical structures; however, their appearance in images acquired with different imaging devices, such as CT or MR, may be very different. Registration algorithms generally deform one image, the floating image, such that it matches with a second, the reference image, by maximizing some similarity score between the deformed and the reference image. Instead of using a universal, but a priori fixed similarity criterion such as mutual information, we propose learning a similarity measure in a discriminative manner such that the reference and correctly deformed floating images receive high similarity scores. To this end, we develop an algorithm derived from max-margin structured output learning, and employ the learned similarity measure within a standard rigid registration algorithm. Compared to other approaches, our method adapts to the specific registration problem at hand and exploits correlations between neighboring pixels in the reference and the floating image. Empirical evaluation on CT-MR/PET-MR rigid registration tasks demonstrates that our approach yields robust performance and outperforms the state of the art methods for multi-modal medical image registration.

Nonparametric image registration algorithms use deformation fields to define nonrigid transformations relating two images. Typically, these algorithms operate by successively solving linear systems of partial differential equations. These PDE systems arise by linearizing the Euler-Lagrange equations associated with the minimization of a functional defined to contain an image similarity term and a regularizer. Iterative linear system solvers can be used to solve the linear PDE systems, but they can be extremely slow. Some faster techniques based on Fourier methods, multigrid methods, and additive operator splitting, exist for solving the linear PDE systems for specific combinations of regularizers and boundary conditions. In this paper, we show that Fourier methods can be employed to quickly solve the linear PDE systems for every combination of standard regularizers (diffusion, curvature, elastic, and fluid) and boundary conditions (Dirichlet, Neumann, and periodic).

Conventional panoramic stereo camera heads have a horizontal displacement structure. When a camera head is used in generating a spatial panorama, it may encounter occlusion inconsistency in the overlap region of two adjacent viewing angles; that is, occlusion may occur from one viewing angle, but not from an adjacent angle. Also, computing spatial panorama may become unnecessarily complex. This paper reports a solution to the above problems by using a dual camera head with the two cameras being displaced vertically with parallel optical axes, and by letting the rotation axis pass through the focal points of the two cameras.