This article proposes a massively-parallel approach for solving the difficult problem matching high-altitude image pairs from different domains, e.g., Synthetic Aperture Radar (SAR) and visible light imagery (EO). The through-weather measurement capability of SAR allows this technology to yield vehicle position fixes in inclement weather and during either night or daytime for image-aided navigation. This work focuses on developing capabilities to match across a large range of variations in the unknown parameters of the homography that brings these image pairs into correspondence. This is a problem that is not well-solved by any existing approaches and is important in practice as cross-domain imagery from aerial platforms often exhibits large variations in scale, keystone, rotation and translation effects that can be different in the x and y axes. Our approach for cross-modal image matching uses a mutual information loss function and applies a massively-parallel search procedure in CUDA to detect and explore the loss function to find satisfactory homographies to match the image pairs. Experiments are performed using simulated image telemetry obtained by flying a fixed wing aircraft in a virtual environment with image data derived from Google Maps and RADARSAT Google Earth Engine image databases. Results show a comparison rate of 12.79 Gpixel/sec and has a search rate of 1.8M matches/sec allowing for exhaustive search solutions. Our approach is found to yield accurate homography values according to our normalized corner error metric for 68% of the image database pairs.

Synthetic Aperture Radar (SAR) systems sense electromagnetic backscatter from scenes generated from a sequence of excitation pulses of RF radiation emitted from the radar antenna varying spatial positions. Focusing the radar returns into coherent images requires highly accurate knowledge of the antenna position for the duration of the pulses. In this article a massively parallel approach is propose to solve the NP-hard problem of focusing radar data collected in the presence of large motion errors. Little research has been dedicated to the development of focusing algorithms capable of image formation when motion error magnitudes exceed the nominal wavelength of the radar excitation signal. This problem has been shown to be non-deterministic polynomial-time hard (NP-hard) to solve and computational challenges are exacerbated by the high computational cost of associated with the SAR focusing algorithms needed to conduct the search. The proposed approach seeks to address these challenges by restricting trajectories to smooth (low-order) curve trajectories and applying an optimized massively parallel GPU implementation of the SAR focusing algorithm to search over candidate trajectories for the trajectory yielding a focused SAR image.

Synthetic Aperture Radar (SAR) systems emit pulses of (Radio Frequency) RF energy from an antenna into the environment over short intervals in time. Reflected RF energy is received coherently by a receiving antenna and signal processing focusing algorithms process the received signals using one of many potential focusing algorithms to form 2D images of the scene. Formation of these images requires highly accurate knowledge of the antenna state, e.g., the position, orientation and their velocities, when transmitting and receiving RF signals. This information is typically obtained using a combination of a high-quality Global Positioning System (GPS) receiver and a highly-accurate Inertial Navigation System (INS). SAR dependence on GPS and INS data prohibit SAR image formation in GPS-denied contexts which limit the application of this technology. This work investigates approaches for focusing SAR images when GPS is unavailable.

This paper revisits the classical problem of detecting interest points, popularly known as “corners,” in 2D images by proposing a technique based on fitting algebraic shape models to contours in the edge image. Our method for corner detection is targeted for use on structural images, i.e., images that contain man-made structures for which corner detection algorithms are known to perform well. Further, our detector seeks to find image regions that contain two distinct linear contours that intersect. We define the intersection point as the corner, and, in contrast to previous approaches such as the Harris detector, we consider the spatial coherence of the edge points, i.e., the fact that the edge points must lie close to one of the two intersecting lines, an important aspect to stable corner detection. Comparisons between results for the proposed method and that for several popular feature detectors are provided using input images exhibiting a number of standard image variations, including blurring, affine transformation, scaling, rotation, and illumination variation. A modified version of the repeatability rate is proposed for evaluating the stability of the detector under these variations which requires a 1-to-1 mapping between matched features. Using this performance metric, our method is found to perform well in contrast to several current methods for corner detection. Discussion is provided that motivates our method of evaluation and provides an explanation for the observed performance of our algorithm in contrast to other algorithms. Our approach is distinct from other contour-based methods since we need only compute the edge image, from which we explicitly solve for the unknown linear contours and their intersections that provide image corner location estimates. The key benefits to this approach are: (1) performance (in space and time); since no image pyramid (space) and no edge-linking (time) is required and (2) compactness; the estimated model includes the corner location, and direction of the incoming contours in space, i.e., a complete model of the local corner geometry.

This article describes a multimedia system consisting of two sensors: (1) a laser range scanner (LIDAR) and (2) a conventional digital camera. Our work specifies a mathematical calibration model that allows for this data to be explicitly integrated. Data integration is accomplished by calibrating the system, i.e., estimating for each variable of the model for a specific LIDAR-and-camera pair. Our approach requires detection of feature points in both the LIDAR scan and the digital images. Using correspondences between feature points, we can then estimate the model variables that specify an explicit mathematical relationship between sensed (x, y, z) LIDAR points and (x, y) digital image positions. Our system is designed for 3D line scanners, i.e., scanners that detect positions that lie in a 3D plane which requires some special theoretical and experimental treatment. Results are provided for simulations of the system in a virtual environment and for a real LIDAR-and-camera system consisting of a SICK LMS200 and an inexpensive USB web-camera. Calibrated systems can integrate the data in real-time which is of particular use for autonomous vehicular and robotic navigation.

A prototype system for the automatic evolution of biomimetic structures using structural automata is described and its utility for generating digital sculpture is demonstrated. Sculptures are generated from a primordial shape which is represented in terms of a triangular mesh and sculpture is created by extending the original surface using tetrahedral structural elements. Recursively applicable rules or equivalently, automata, are defined which allow the sculptor to generate a volumetric scaffold from the original surface. This scaffold is generated using the stated rules for inserting and connecting together the tetrahedral elements. The software is operated as a generative process where sculptures are grown from an original triangular surface mesh as a sequence of layers. Each layer is created as a 2-step process. In step 1, we populate the surface with tetrahedral structures where the base of each tetrahedron coincides with a surface triangle. Step 2 re-triangulates the apexes of the tetrahedra from step 1 creating an offset and deformed version of the original surface mesh. The sculptor has artistic control of the process at all points and may assign or change rules to generate different biomimetic behaviors, i.e., structures which tend to replicate natural phenomena.

The Mid-Ocean Studio, Brown University's SHAPE lab, and Goldsmiths College are collaborating on a prototype system for the automatic evolution of biomimetic sculpture using structural automata. This collaboration is resulting in effective, computerized means to autogenerate large, increasingly complex works of art, and allowing for a long-anticipated development of the desire to create works that reflect and respond to the environment they are in. We propose to create an installation that allows visitors to a site at Goldsmiths College to experience and interact with the development of our structures.