PointNet has revolutionized how we think about representing point clouds. For classification and segmentation tasks, the approach and its subsequent variants/extensions are considered state-of-the-art. To date, the successful application of PointNet to point cloud registration has remained elusive. In this paper we argue that PointNet itself can be thought of as a learnable "imaging" function. As a consequence, classical vision algorithms for image alignment can be brought to bear on the problem -- namely the Lucas & Kanade (LK) algorithm. Our central innovations stem from: (i) how to modify the LK algorithm to accommodate the PointNet imaging function, and (ii) unrolling PointNet and the LK algorithm into a single trainable recurrent deep neural network. We describe the architecture, and compare its performance against state-of-the-art in several common registration scenarios. The architecture offers some remarkable properties including: generalization across shape categories and computational efficiency -- opening up new paths of exploration for the application of deep learning to point cloud registration. Code and videos are available at https://github.com/hmgoforth/PointNetLK.

We present a method for localization of Unmanned Aerial Vehicles (UAVs) which is meant to replace an onboard GPS system in the event of a noisy or unreliable GPS signal. Our method requires only a downward-facing monocular RGB camera on the UAV, and pre-existing satellite imagery of the flight location to which the UAV imagery is compared and aligned. To overcome differences in the image capturing conditions between the satellite and UAV, such as seasonal and perspective changes, we propose the use of Convolutional Neural Network (CNN) representations trained on readily available satellite data. To increase localization accuracy, we also develop an optimization which jointly minimizes the error between adjacent UAV frames as well as the satellite map. We demonstrate how our method improves on recent systems from literature by achieving greater performance in flight environments with very few landmarks. For a GPS-denied flight at 0.2km altitude, over a flight distance of 0.85km, we achieve average localization error of less than 8 meters. We make our source code and datasets available to encourage further work on this emerging topic.