Monitoring Earth activity using data collected from multiple satellite imaging platforms in a unified way is a significant challenge, especially with large variability in image resolution, spectral bands, and revisit rates. Further, the availability of sensor data varies across time as new platforms are launched. In this work, we introduce an adaptable framework and network architecture capable of predicting on subsets of the available platforms, bands, or temporal ranges it was trained on. Our system, called WATCH, is highly general and can be applied to a variety of geospatial tasks. In this work, we analyze the performance of WATCH using the recent IARPA SMART public dataset and metrics. We focus primarily on the problem of broad area search for heavy construction sites. Experiments validate the robustness of WATCH during inference to limited sensor availability, as well the the ability to alter inference-time spatial or temporal sampling. WATCH is open source and available for use on this or other remote sensing problems. Code and model weights are available at: https://gitlab.kitware.com/computer-vision/geowatch

Self-supervised learning aims to learn image feature representations without the usage of manually annotated lbabels. It is often used as a precursor step to obtain useful initial network weights which contribute to faster convergence and superior performance of downstream tasks. While self-supervision allows one to reduce the domain gap between supervised and unsupervised learning without the usage of labels, the self-supervised objective still requires a strong inductive bias to downstream tasks for effective transfer learning. In this work, we present our material and texture based self-supervision method named MATTER (MATerial and TExture Representation Learning), which is inspired by classical material and texture methods. Material and texture can effectively describe any surface, including its tactile properties, color, and specularity. By extension, effective representation of material and texture can describe other semantic classes strongly associated with said material and texture. MATTER leverages multitemporal, spatially aligned remote sensing imagery over unchanged regions to learn invariance to illumination and viewing angle as a mechanism to achieve consistency of material and texture representation. We show that our self-supervision pre-training method allows for up to 24.22% and 6.33% performance increase in unsupervised and finetuned setups, and up to 76% faster convergence on change detection, land cover classification, and semantic segmentation tasks. Code and dataset: https://github.com/periakiva/MATTER.

Polynomial radial lens distortion models are widely used in image processing and computer vision applications to compensate for when straight lines in the world appear curved in an image. While polynomial models are used pervasively in software ranging from PhotoShop to OpenCV to Blender, they have an often overlooked behavior: polynomial models can fold back onto themselves. This property often goes unnoticed when simply warping to undistort an image. However, in applications such as augmented reality where 3D scene geometry is projected and distorted to overlay an image, this folding can result in a surprising behavior. Points well outside the field of view can project into the middle of the image. The domain of a radial distortion model is only valid up to some (possibly infinite) maximum radius where this folding occurs. This paper derives the closed form expression for the maximum valid radius and demonstrates how this value can be used to filter invalid projections or validate the range of an estimated lens model. Experiments on the popular Lensfun database demonstrate that this folding problem exists on 30% of lens models used in the wild.

Structure-from-motion (SfM) is a well-studied problem in the computer vision field and is of particular interest for aerial imaging applications like mapping, terrain modeling, crop monitoring, etc. With the current rapid growth in the commercial UAV and small satellite markets, aerial SfM is becoming even more important. In recent years, free and open source software has enabled almost anyone to apply SfM to their data at no cost. Existing free packages are oriented toward processing unordered collections of photographs and are less efficient at processing ordered collections or video. They are also prone to failure with nearly planar scenes that often arise in aerial photography. While commercial solutions for aerial SfM exist, they are proprietary and expensive. This paper presents a new open source software toolkit named MAP-Tk that targets SfM for aerial video. It exploits temporal continuity and the aerial nature of the data to speed up and improve feature tracking, loop closure, and bundle adjustment. The system is highly configurable and modular. Dynamic plugins provide state-of-the art algorithms from other open source projects like OpenCV, VXL, and Ceres Solver. This paper presents the modular system design, user interface, novel algorithms exploiting aerial video, and results on various aerial data sets.

In automated surveillance, one is often interested in tracking road vehicles, measuring their shape in 3D world space, and determining vehicle classification. To address these tasks simultaneously, an effective approach is the constrained alignment of a prior model of 3D vehicle shape to images. Previous 3D vehicle models are either generic but overly simple or rigid and overly complex. Rigid models represent exactly one vehicle design, so a large collection is needed. A single generic model can deform to a wide variety of shapes, but those shapes have been far too primitive. This paper uses a generic 3D vehicle model that deforms to match a wide variety of passenger vehicles. It is adjustable in complexity between the two extremes. The model is aligned to images by predicting and matching image intensity edges. Novel algorithms are presented for fitting models to multiple still images and simultaneous tracking while estimating shape in video. Experiments compare the proposed model to simple generic models in accuracy and reliability of 3D shape recovery from images and tracking in video. Standard techniques for classification are also used to compare the models. The proposed model outperforms the existing simple models at each task.

In traffic surveillance applications a good prior model of vehicle shape and appearance is becoming increasingly more important for tracking, shape recovery, and recognition from video. The usefulness of 2-d vehicle models is limited to a fixed viewing direction; 3-d models are nearly always more suitable. Existing 3-d vehicle models are either generic but far too simple to utilize high resolution imagery, or far too complex and limited to specific vehicle instances. This paper presents a deformable vehicle model that spans these two extremes. The model is constructed with a multi-resolution approach to fit various image resolutions. At each resolution, a small number of parameters controls the deformation to accurately represent a wide variety of passenger vehicles. The parameters control both 3-d shape and appearance of parts that deform in the 2-d manifold of the vehicle surface. These parts are regions representing windows, headlights, taillights, etc. The combination of part boundaries and surface occluding contours account for the most consistent edges observed in images of vehicles. It is shown that the model parameters can be recovered by fitting the deformable model to real images of vehicles.

Abstract We present a planarity constraint and a novel three‐dimensional (3D) point reconstruction algorithm for a multiview laser range slit scanner. The constraint is based on the fact that all observed points on a projected laser line lie on the same plane of laser light in 3D. The parameters of the plane of laser light linearly parametrize a homography between a pair of images of the laser points. This homography can be recovered from point correspondences derived from epipolar geometry. The use of the planar constraint reduces outliers in the reconstruction and allows for the reconstruction of points seen in only one view. We derive an optimal reconstruction of points subject to the planar constraint and compare the accuracy to the suboptimal approach in prior work. We also construct a catadioptric stereo rig with high quality optical components to remove error due to camera synchronization and non‐uniform laser projection. The reconstruction results are compared to prior work that uses inexpensive optics and two cameras.

Retinal clinicians and researchers make extensive use of images, and the current emphasis is on digital imaging of the retinal fundus. The goal of this paper is to introduce a system, known as retinal image vessel extraction and registration system, which provides the community of retinal clinicians, researchers, and study directors an integrated suite of advanced digital retinal image analysis tools over the Internet. The capabilities include vasculature tracing and morphometry, joint (simultaneous) montaging of multiple retinal fields, cross-modality registration (color/red-free fundus photographs and fluorescein angiograms), and generation of flicker animations for visualization of changes from longitudinal image sequences. Each capability has been carefully validated in our previous research work. The integrated Internet-based system can enable significant advances in retina-related clinical diagnosis, visualization of the complete fundus at full resolution from multiple low-angle views, analysis of longitudinal changes, research on the retinal vasculature, and objective, quantitative computer-assisted scoring of clinical trials imagery. It could pave the way for future screening services from optometry facilities.

Vehicle tracking in video sequences is typically carried out by matching 2-d views (images or features) of the vehicle from one frame to the next. These views are adapted by gradual changes in 2-d image transforms between frames. This approach can work well for sequences where the vehicle projection is not highly perspective and 3-d vehicle orientation with respect to the camera varies slowly. In this paper, vehicle tracking is studied under conditions that violate these assumptions. Tracking is carried out on high-definition videos sequences of vehicles that pass near the camera, causing severe perspective distortion. Perspective effects are accounted for by tracking edge curves on the vehicle and reconstructing them in 3-d, using structure from motion on adjacent frames. The result is segmentation of the vehicle from the background and recovery of its geometry and motion in 3-d.

This paper presents a novel algorithm for simultaneous background appearance modeling and coarse-scale vehicle recognition in traffic surveillance applications. 3-d mesh models representing a small set of vehicle classes are used to the hypothesize image segmentations into background, shadow, and vehicle regions. The algorithm optimizes vehicle class and motion parameters to best agree with a Hidden Markov Model for the image appearance. The best hypothesis, combined with image data, is used to adapt the parameters of the appearance model. Experiments on real video show that an appearance model trained in this way performs almost as well as one trained using manually segmented images. 1