The backprojection (BP) algorithm is a precise time-domain technique in synthetic aperture radar (SAR) imaging, known for its robustness in scenarios where frequency-domain algorithms struggle, such as nonlinear flight paths, high-resolution demands, or large imaging swaths. However, the unknown spectral structures of BP images pose significant challenges for frequency-domain autofocus algorithms, including phase gradient autofocus (PGA). This article introduces a novel interpretation of the BP algorithm, and then compares the image spectra between BP and the polar format algorithm (PFA). Our findings indicate that BP image’s spectrum suffers from severe range ambiguity and spectral misalignment, which is the key reason why BP images cannot be directly applied to PGA. To address these issues, we first improve the postprocessing method, and then propose a modified BP algorithm. Both the approaches reconstruct the signal spectrum, thereby facilitating the direct application of PGA. Compared with the postprocessing method, the proposed algorithm is more efficient. Simulation and real data experiments validate the feasibility and efficacy of the proposed method, demonstrating the proposed algorithm’s suitability for autofocusing in the frequency domain.

The Polar Format Algorithm (PFA) is a popular choice for general bistatic synthetic aperture radar (BiSAR) imaging due to its computational efficiency and adaptability to situations with complicated geometries or arbitrary flight trajectories. However, efficient and accurate compensation of two-dimensional (2-D) residual phase errors induced by the wavefront curvature remains challenging when obtaining high quality BiSAR PFA images. In this paper, an analytical expression for the phase errors in the wavenumber domain is derived. With it, the inherent structural characteristics of the phase errors are formulated, where the 2-D phase errors can be reduced to one-dimensional (1-D) phase errors for an optimal coordinate system. Moreover, the mapping relationship between distorted point targets in the optimal imaging coordinates and their actual point targets in the original imaging coordinates is investigated. By exploiting the structural characteristics and the mapping relationship, a highly efficient BiSAR PFA wavefront curvature compensation method is proposed. This allows a reduced-dimensional space-variant filter to be constructed to mitigate the 2-D defocusing effect without compromising the compensation accuracy, with the distortion correction being performed using interpolation. The proposed method significantly reduces the complexity required for residual 2-D phase error compensation while simplifying the distortion correction, resulting in a notable robustness. The effectiveness of the method is demonstrated using point target and scene target simulations.

Probabilistic graphical models (PGMs) have been widely used in computer vision, robotics, and statistics. Generative inference algorithms used to solve PGMs, such as belief propagation (BP), involve integration over high-dimensional variables, which becomes computationally infeasible. Drawing inspiration from particle filters, nonparametric belief propagation (NBP) combines the efficiency of Monte-Carlo sampling to capture the belief space of hidden variables while preserving accuracy and algorithm robustness. In this poster presentation, we describe a novel method to accelerate an NBP algorithm in hardware and evaluate our approach on the 6 degree-of-freedom (DoF) articulated object pose estimation problem. Our two major contributions include 1) identifying three independent computation flows in the algorithm to effectively overlap the two main steps of the algorithm: belief update and message update, and 2) creating deeply pipelined processing units that allow for fine-grained parallelism, which helps to balance the workloads on different computation streams. Results from this study demonstrate that our design achieves both improved runtime and energy efficiency. In particular, we achieved 26X energy saving compared to running the algorithm on a Titan Xp GPU, and 10X runtime speedup and 14X energy saving compared to running on the Jetson AGX . We believe that our FPGA implementation can greatly improve particle-based sampling methods for real time applications.

Perceiving the position and orientation of objects (i.e., pose estimation) is a crucial prerequisite for robots acting within their natural environment. We present a hardware acceleration approach to enable real-time and energy efficient articulated pose estimation for robots operating in unstructured environments. Our hardware accelerator implements Nonparametric Belief Propagation (NBP) to infer the belief distribution of articulated object poses. Our approach is on average, 26× more energy efficient than a high-end GPU and 11× faster than an embedded low-power GPU implementation. Moreover, we present a Monte-Carlo Perception Library generated from high-level synthesis to enable reconfigurable hardware designs on FPGA fabrics that are better tuned to user-specified scene, resource, and performance constraints.

The phase gratitude algorithm (PGA) is a robust autofocusing approach that can efficiently refocus defocused SAR imagery produced by frequency-domain algorithms. However, from a conventional viewpoint, PGA cannot be extended to refocus SAR imagery produced by time-domain algorithms, such as the Filtered Back Projection (FBP), as the spectrum of the FBP imagery is range ambiguous and azimuth space-variant. In this letter, a novel interpretation of FBP is presented, in which the spectrum structure of the FBP imagery is analyzed in detail. By incorporating the derived spectral information, an efficient spectrum preprocessing is proposed for spectrum restructuring. After this preprocessing, PGA is shown to be able to refocus defocused FBP imagery. The validity and feasibility of the proposed autofocusing approach are demonstrated using both simulated and experimental data.

The filtered backprojection (FBP) algorithm is viewed as a preferred candidate for general bistatic synthetic aperture radar (BiSAR) imaging since it does not pose any restrictions on SAR configurations or flight paths. However, high-efficient autofocus methods such as phase gradient autofocus (PGA) or Mapdrift (MD) cannot be effectively integrated with the FBP algorithm due to the unknown properties of the BiSAR FBP imagery spectrum. In this article, a novel Fourier-based interpretation of the BiSAR FBP algorithm is presented. Based on the new viewpoint, spectral characteristics of the BiSAR FBP imagery in the wavenumber domain, including range spectral ambiguity, space-variant spectral support, and the structural 2-D phase error, are derived in detail. Using these characteristics, a computationally efficient 2-D autofocus approach is proposed. First, a preprocessing is performed to eliminate the range spectral ambiguity and to align the skewed spectrum support, which facilitates the following phase error estimation and correction. Then, an estimation of the 1-D azimuth phase error (APE) is applied by combining multiple estimation results from different subband data. Finally, the 2-D phase error is computed directly from the estimated APE by exploiting the derived analytical structure of the 2-D phase error, which is then applied to restore the BiSAR FBP image. The simulation results are presented to show the effectiveness of the proposed approach.

Algorithms based on Monte-Carlo sampling have been widely adapted in robotics and other areas of engineering due to their performance robustness. However, these sampling-based approaches have high computational requirements, making them unsuitable for real-time applications with tight energy constraints. In this paper, we investigate 6 degree-of-freedom (6DoF) pose estimation for robot manipulation using this method, which uses rendering combined with sequential Monte-Carlo sampling. While potentially very accurate, the significant computational complexity of the algorithm makes it less attractive for mobile robots, where runtime and energy consumption are tightly constrained. To address these challenges, we develop a novel hardware implementation of Monte-Carlo sampling on an FPGA with lower computational complexity and memory usage, while achieving high parallelism and modularization. Our results show 12X-21X improvements in energy efficiency over low-power and high-end GPU implementations, respectively. Moreover, we achieve real time performance without compromising accuracy.

Hybrid machine learning algorithms that combine deep learning with probabilistic inference techniques provide highly accurate scene perception for robot manipulation. In particular, a 2-stage approach that combines object detection using convolutional neural networks with Monte-Carlo sampling for pose estimation has been shown to perform particularly well under adversarial scenarios. Unfortunately, this accuracy comes at the cost of high computational complexity, which affects runtime, resource utilization, and energy consumption. This paper describes various challenges in developing complexity-aware techniques for robust robot perception and presents a novel hardware accelerator that addresses these challenge. Experimental results show our design is at least 30% faster and consumes 97% less energy compared to an implementation on a high-end GPU. Compared to a low-power GPU implementation, our design is 95% faster while consuming 96% less energy, demonstrating that accurate, energy-efficient scene perception is possible in real time with targeted hardware acceleration.

Recent advancements have led to a proliferation of machine learning systems used to assist humans in a wide range of tasks. However, we are still far from accurate, reliable, and resource-efficient operations of these systems. For robot perception, convolutional neural networks (CNNs) for object detection and pose estimation are recently coming into widespread use. However, neural networks are known to suffer from overfitting during the training process and are less robust under unforeseen conditions (which makes them especially vulnerable to adversarial scenarios). In this work, we propose Generative Robust Inference and Perception (GRIP) as a two-stage object detection and pose estimation system that aims to combine the relative strengths of discriminative CNNs and generative inference methods to achieve robust estimation. Our results show that a second stage of sample-based generative inference is able to recover from false object detections by CNNs, and produce robust estimations in adversarial conditions. We demonstrate the efficacy of GRIP robustness through comparison with state-of-the-art learning-based pose estimators and pick-and-place manipulation in dark and cluttered environments.

Convolutional neural networks (CNNs) are of increasing widespread use in robotics, especially for object recognition. However, such CNNs still lack several critical properties necessary for robots to properly perceive and function autonomously in uncertain, and potentially adversarial, environments. In this paper, we investigate factors for accurate, reliable, and resource-efficient object and pose recognition suitable for robotic manipulation in adversarial clutter. Our exploration is in the context of a three-stage pipeline of discriminative CNN-based recognition, generative probabilistic estimation, and robot manipulation. This pipeline proposes using a SAmpling Network Density filter, or SAND filter, to recover from potentially erroneous decisions produced by a CNN through generative probabilistic inference. We present experimental results from SAND filter perception for robotic manipulation in tabletop scenes with both benign and adversarial clutter. These experiments vary CNN model complexity for object recognition and evaluate levels of inaccuracy that can be recovered by generative pose inference. This scenario is extended to consider adversarial environmental modifications with varied lighting, occlusions, and surface modifications.