Rock detachment events along roadways pose public safety concerns but can be predicted and safely handled using geological measurements of discontinuities. With modern sensing technology, these measurements can be taken on 3-D point clouds and 2-D optical images that provide a high level of structural accuracy and visual detail. Doing so allows engineers to obtain the needed data with relative ease while eliminating the biases and hazards inherent in taking manual measurements. This letter presents an approach for fusing the 2-D and 3-D data in natural and unstructured scenes. This includes a novel method for visualizing imagery obtained with very different sensors to maximize their visual similarity making registration a more tangible task. To show the effectiveness of our registration methodology, we evaluate measurements taken manually and digitally on rock facet and cut discontinuity orientations in Rolla, MO. Our method is able to align the 2-D and 3-D data with an accuracy of under 2 cm. The median difference between measurements manually obtained by a geological engineer and those obtained with our proposed software is 3.65.

In the event of natural or man-made disasters, providing rapid situational awareness through video/image data collected at salient incident scenes is often critical to the first responders. However, computer vision techniques that can process the media-rich and data-intensive content obtained from civilian smartphones or surveillance cameras require large amounts of computational resources or ancillary data sources that may not be available at the geographical location of the incident. In this paper, we propose an incident-supporting visual cloud computing solution by defining a collection, computation, and consumption (3C) architecture supporting fog computing at the network edge close to the collection/consumption sites, which is coupled with cloud offloading to a core computation, utilizing software-defined networking (SDN). We evaluate our 3C architecture and algorithms using realistic virtual environment test beds. We also describe our insights in preparing the cloud provisioning and thin-client desktop fogs to handle the elasticity and user mobility demands in a theater-scale application. In addition, we demonstrate the use of SDN for on-demand compute offload with congestion-avoiding traffic steering to enhance remote user quality of experience in a regional-scale application. The optimization between fogs computing at the network edge with core cloud computing for managing visual analytics reduces latency, congestion, and increases throughput.

LIDAR range scans can be used to quickly create accurate 3D models for virtual reality and as a basis to visualize sets of photographs, videos, and virtual objects in a cohesive environment. The number of existing virtual reality programs that use LIDAR data as input has motivated our group to develop methods for fusing images with 3D scans and for augmenting the scans with both dynamic objects present in videos and virtual models. Bringing together as many data sources as possible increases users' abilities to present related information in one, intuitive venue. We demonstrate how to register a variety of 2D imagery with a range scan to construct photo-realistic models and to extract walking people captured in videos and model them in a 3D space. We also present a method for determining the sun position from a set of stitched photographs in order to apply correct lighting to virtual objects placed amongst real world data. Naturally lit objects can be inserted into original photographs using our 2D-3D registration information. These methods are all combined to display and study photos, videos, and virtual objects in a complete 3D environment.

Matching and aligning architectural imagery is an important step for many applications but can be a difficult task due to repetitive elements often present in buildings. Many keypoint descriptor and matching methods will fail to produce distinctive descriptors for each region of man-made structures, which causes ambiguity when attempting to match areas between images. In this paper, we outline a technique for reducing the search space for matching by taking a two-step approach, aligning pairs one dimension at a time and by abstracting images that originally contain many repetitive elements into a set of distinct, representative patches. We also present a simple, but very effective method for computing the intra-image saliency for a single image that allows us to directly identify unique areas in an image without machine learning. We use this information to find distinctive keypoint matches across image pairs. We show that our pipeline is able to overcome many of the pitfalls encountered when using traditional keypoint and regional matching techniques on commonly encountered images of urban scenes.

In the event of natural or man-made disasters, many videos may be collected by civilians and surveillance cameras that can be extremely useful for first responders trying to ascertain the extent of the damage. However, watching and analyzing numerous videos on separate screens can be a cumbersome task. Registering a set of 2D videos with a 3D model can provide an intuitive venue for viewing multiple videos simultaneously. In such a setup, it is likely that the user will want to work with the dynamic 3D environment from a remote location, requiring that videos be transferred over a network to be registered with a 3D model. In this paper, we propose combining the fields of computer vision, cloud computing, and high-speed networking to create a system that takes in HD videos, streams the data to a server where a dynamic 3D model is constructed, and provides a virtual scene navigation program for viewing the videos in a 3D scene from a mobile device. We test transferring the data of interest over different types of networks and processing the videos on various server configurations to determine the capabilities of such a system and the necessary requirements for it to provide a high-quality user experience.

We propose a contextual framework for 2D image matching and registration using an ensemble feature. Our system is beneficial for registering image pairs that have captured the same scene but have large visual discrepancies between them. It is common to encounter challenging visual variations in image sets with artistic rendering differences or in those collected over a period of time during which the lighting conditions and scene content may have changed. Differences between images may also be caused using a variety of cameras with different sensors, focal lengths, and exposure values. Local feature matching techniques cannot always handle these difficulties, so we have developed an approach that builds on traditional methods to consider linear and histogram of gradient information over a larger, more stable region. We also present a technique for using linear features to estimate corner keypoints, or pseudo corners, that can be used for matching. Our pipeline follows this unique matching stage with homography refinement methods using edge and gradient information. Our goal is to increase the size of accurate keypoint match sets and align photographs containing a combination of man-made and natural imagery. We show that incorporating contextual information can provide complimentary information for scale invariant feature transform and boost local keypoint matching performance, as well as be used to describe corner feature points.