Hyperspectral imaging is an excellent tool to remotely analyze the Earth from in-orbit devices. Satellites capture these images containing vast information about the ground pixels. To optimize storage and transmission speeds, compression is often performed onboard the satellite. To that end, algorithms such as the CCSDS 123.0-B-2 are implemented on FPGAs, enabling this process in an efficient and fast manner. Single-Event Upsets (SEU) are commonplace in this scenario, e.g. bit flips in the FPGA’s configuration memory which can catastrophically alter the algorithm’s output. In this paper, we propose a fault tolerance technique for this specific case. The compression core is checked periodically by running a golden model designed to excite the full internal datapath based on a synthetic image. A failure in this check will trigger a reconfiguration of the compression core. Results show better detection rates than Dual Modular Redundancy (DMR) at a fraction of the resource cost, proving this technique as a viable alternative. Furthermore, other algorithms with similar processing flows might benefit as well from this technique.

Since the advent of air and space-borne flight, remotely sensed images of the Earth’s surface have changed how the world is perceived: meteorology, navigation, surveillance, and environmental sciences are some of the areas, in which technology has fundamentally changed. Hyperspectral images extend the possibilities even further by capturing wavelengths outside the visible spectrum. This is particularly valuable for target detection, often performed using drones, satellites, or other unmanned aerial vehicles (UAVs). In many applications such as surveillance and security, it is critical to perform this detection in real-time. Response times can be optimized by performing these calculations on-board, which introduces additional power, radiation tolerance, and resource constraints. To that end, embedded devices such as field-programmable gate arrays (FPGAs) are ideal candidates, as their reprogrammable logic allows for a great degree of acceleration while having low power consumption. In this article, a real-time FPGA implementation of the linearly constrained minimum variance (LCMV) algorithm for target classification is presented. First, an analysis is performed to explore the most parallelizable and accurate numerical methods that fit within the resource constraints of FPGAs, and finally selecting LDL decomposition. Then, hardware modules for LCMV classification are designed in SystemVerilog, focusing on incremental correlation matrix computation and a solver for LDL matrix decomposition. The proposed implementation on a radiation-tolerant Xilinx XQRKU060 is compared to an embedded CUDA-accelerated graphics processing unit (GPU) and an embedded central processing unit (CPU), showing turnaround time improvements of$7.25\times $and$825\times $, respectively. It runs$15\times $faster than real-time, with sufficient margin to accommodate the next generation of hyperspectral sensors.

Hyperspectral image analysis represents an extremely complex procedure from a computational point of view, mainly due to the high dimensionality of the data. This computational cost represents a significant disadvantage in applications that require a real-time response, such as fire monitoring, prevention and monitoring of natural disasters, chemical spills, and other environmental pollutants. Dimensional reduction allows us to decrease the size of the image while preserving the important discriminating features, by eliminating redundant data or noise. Due to their reduced size, weight, and power consumption when compared to other high-performance computing systems, reconfigurable hardware solutions, such as field-programmable gate arrays, have been consolidated in recent years as one of the standard options for the quick processing of hyperspectral remotely sensed images. In this paper, we have implemented an optimized hardware version of the Fast Independent Component Analysis (FastICA) version for dimensional reduction of hyperspectral images using an FPGA device. Our implementation achieves a 34 × speedup compared with its equivalent software version, achieving the objective of real-time processing considering the capture time of the AVIRIS spectral sensor.

The decomposition of the observed pixel spectrum of hyperspectral data into its constituent spectral signatures (or endmembers) and a set of corresponding fractional abundances is known as hyperspectral unmixing. To carry out this analysis, it is required to use a computational technology with high-performance computing due to the high dimensionality of the data and the high complexity of the unmixing algorithms. Field Programmable Gate Arrays (FPGAs) are excellently suited to processing needs since they provide flexibility, low power consumption, and great performance. Faster alternative development methods are needed because the rate at which new algorithms for hyperspectral analysis are developed is substantially higher than the rate at which these algorithms are implemented on FPGAs. High-level synthesis (HLS) is a technology that assists with the transformation of a behavioral description of hardware into a register transfer level (RTL) model, thus reducing development times. In this work, the automatic target classification and detection algorithm (ATDCA) has been accelerated on FPGAs using Dataparalell C++ (DPC++) and Intel oneAPI. The analysis of the algorithm and the three accelerations carried out achieve a speedup above 13×.

In hyperspectral image analysis one of the most important tasks is target detection, requiring the execution of algorithms with high computational complexity. Recently, research efforts have focused on on-board real-time target detection to provide timely responses for swift decisions. Therefore, it is necessary to use a technology that provides the performance needed for real-time target detection, and at the same time meets the satellite payload requirements. Field-programmable gate arrays (FPGAs) have very interesting properties in terms of performance, size and power consumption, which have become the standard option for on-board processing. In this letter, we present a hardware optimized implementation for FPGAs of the automatic target detection and classification algorithm (ATDCA) using the Gram–Schmidt (GS) method for orthogonalization purposes. The ATDCA-GS algorithm is directly coded using VHDL and verified on a Virtex-7 XC7VX690T FPGA using real hyperspectral data (collected by Hyperspectral Digital Imagery Collection Experiment (HYDICE) sensor and by NASA’s Airborne Visible/Infrared Imaging Spectrometer (AVIRIS)) and a synthetic image. Experimental results demonstrate that our hardware version of the ATDCA-GS algorithm outperforms previous implementations (multicore processors, GPUs and accelerators) in both computation time (obtaining real-time performance) and power consumption, demonstrating the suitability of FPGAs for this purpose.

Hyperspectral images are a useful remote sensing tool that often reaches hundreds of megabytes in size. The CCSDS 123.0-B-2 is a recent algorithm that achieves lossless and near-lossless compression of hyperspectral images by introducing a configurable maximum error over its predecessor CCSDS 123.0-B-1. In this article, a field-programmable gate array (FPGA) implementation of the revised standard that works in real-time is presented. We have developed an extremely pipelined and fast core in VHDL, that is able to process a sample per cycle at over 250 MHz, working eight times faster than in real time for the Airborne Visible-Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG) sensor. New dependencies in the revised standard are avoided by using a novel sample ordering called frame interleaved by diagonal. The predictor stage has been designed to work in this order, and two reorder buffers encapsulate it to be band interleaved by pixel compliant. Predictor data are encoded using a novel FPGA implementation of the CCSDS 123.0-B-2 hybrid coder. The modules are tested and verified on a Virtex-7 VC709 board. For medium (256 bands$\times4096$frames$\times512$samples) and large ($512\times 4096\times 1024$) images, the core occupies, respectively, 14% and 50% of an XQRKU060 FPGA.

Segmented and pipelined execution has been a staple of computing for the past decades. Operations over different values can be carried out at the same time speeding up computations. Hyperspectral image compression sequentially processes samples, exploiting local redundancies to generate a predictable data stream that can be compressed. In this article, we take advantage of a low complexity predictive lossy compression algorithm which can be executed over an extremely long pipeline of hundreds of stages. We can avoid most stalls and maintain throughput close to the theoretical maximum. The different steps operate over integers with simple arithmetic operations, so they are especially well-suited for our FPGA implementation. Results on a Virtex-7 show a maximum frequency of over 300 MHz for a throughput of over 290 MB/s, with a space-qualified Virtex-5 reaching 258 MHz, being five times as fast as the previous FPGA designs. This shows that a modular pipelined approach is beneficial for these kinds of compression algorithms.

Remotely sensed hyperspectral imaging is a very active research area, with numerous contributions in the recent scientific literature. To carry out these investigations, it is necessary to collect large amounts of information that will be processed on satellite or airborne platforms using parallel processing techniques on multi core systems or Graphics Processing Units, trying to avoid excessive energy consumption. Due to the high dimensionality of the data, algorithms analyzing hyperspectral images have a high computational cost. This cost is a significant disadvantage in applications that require real-time response, such as fire tracing, prevention and monitoring of natural disasters, chemical spills and other environmental pollution, etc. To solve these problems, one of the solutions most used is the dimensional reduction, which removes noise and redundant information of images. Therefore, it is possible to reduce significantly the size of the images, and improve the complexity of the algorithms and data storage. Moreover, Field-Programmable Gate Arrays are specially recommended in remotely sensed applications that require real-time response due to their features such as reconfiguration, low consumption, compact size and high computing power on board. In this work, we propose the implementation in reconfigurable hardware of the principal component analysis (PCA) algorithm to carry out the dimensional reduction of hyperspectral images. Experimental results demonstrate that our hardware version of the PCA algorithm exhibits real-time performance.

Spectral unmixing is an important task for remotely sensed hyperspectral data exploitation. It amounts the identification of pure spectral signatures (endmembers) in the data, and the estimation of the abundance of each endmember in each (possibly mixed) pixel. A challenging problem in spectral unmixing is how to determine the number of endmembers in a given scene. For this purpose, many algorithms have been proposed in the recent literature, being the estimation of the Virtual Dimensionality (VD) of the hyperspectral image and the hyperspectral signal subspace estimator (HySime) two of the most popular choices. Unfortunately, the high dimensionality of the hyperspectral data provided by modern sensors as well as the inherent computational complexity clearly make the use of these algorithms prohibitive for applications under real-time or near real-time constraints. Hence, the utilization of high performance computing platforms in order to accelerate the process of unmixing a hyperspectral image becomes mandatory for such scenarios. Reconfigurable hardware solutions such as field programmable gate arrays (FPGAs) have consolidated during the last years as one of the preferred choices for the fast processing of hyperspectral remotely sensed images due to their advantages over other high performance computing systems, such as clusters of computers, multicore processors and/or graphical processing units (GPUs). This paper uncovers two FPGA-based architectures for accelerating the process of estimating the number of endmembers that constitute a hyperspectral image according to the VD and the HySime algorithms. The proposed methods have been implemented on a Virtex-7 XC7VX690T FPGA and tested using real hyperspectral data collected by NASAs Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Cuprite mining district in Nevada and the World Trade Center in New York. Experimental results demonstrate that the VD implementation exhibits real-time performance while the HySime implementation exhibits near real-time performance. Both implementations significantly outperform a software version, which makes our reconfigurable system appealing for onboard hyperspectral data processing.

In this article, we present a design of a specific processor for Blokus Duo game. This design has been submitted to the ICFPT'13 Design Competition and implemented on a low-cost Spartan-6 FPGA. Our player applies several techniques to identify which movements are potentially interesting, and applies a search-tree in order to evaluate the consequences of each of those options. To achieve an efficient implementation we have developed custom modules to manage the board and to identify whether a block can be placed or not in a given vertex. The results demonstrate that our design is competitive, even against advanced Blokus Duo players, such as the Pentobi software application considered as the best available software player.

Hyperspectral imaging is an important technique in remote sensing which is characterized by high spectral resolutions. With the advent of new hyperspectral remote sensing missions and their increased temporal resolutions, the availability and dimensionality of hyperspectral data is continuously increasing. This demands fast processing solutions that can be used to compress and/or interpret hyperspectral data onboard spacecraft imaging platforms in order to reduce downlink connection requirements and perform a more efficient exploitation of hyperspectral data sets in various applications. Over the last few years, reconfigurable hardware solutions such as field-programmable gate arrays (FPGAs) have been consolidated as the standard choice for onboard remote sensing processing due to their smaller size, weight, and power consumption when compared with other high-performance computing systems, as well as to the availability of more FPGAs with increased tolerance to ionizing radiation in space. Although there have been many literature sources on the use of FPGAs in remote sensing in general and in hyperspectral remote sensing in particular, there is no specific reference discussing the state-of-the-art and future trends of applying this flexible and dynamic technology to such missions. In this work, a necessary first step in this direction is taken by providing an extensive review and discussion of the (current and future) capabilities of reconfigurable hardware and FPGAs in the context of hyperspectral remote sensing missions. The review covers both technological aspects of FPGA hardware and implementation issues, providing two specific case studies in which FPGAs are successfully used to improve the compression and interpretation (through spectral unmixing concepts) of remotely sensed hyperspectral data. Based on the two considered case studies, we also highlight the major challenges to be addressed in the near future in this emerging and fast growing research area.

Hyperspectral remote sensing attempts to identify features in the surface of the Earth using sensors that generally provide large amounts of data. The data are usually collected by a satellite or an airborne instrument and sent to a ground station that processes it. The main bottleneck of this approach is the (often reduced) bandwidth connection between the satellite and the station, which drastically limits the information that can be sent and processed in real time. A possible way to overcome this problem is to include onboard computing resources able to preprocess the data, reducing its size by orders of magnitude. Reconfigurable field-programmable gate arrays (FPGAs) are a promising platform that allows hardware/software codesign and the potential to provide powerful onboard computing capability and flexibility at the same time. Since FPGAs can implement custom hardware solutions, they can reach very high performance levels. Moreover, using run-time reconfiguration, the functionality of the FPGA can be updated at run time as many times as needed to perform different computations. Hence, the FPGA can be reused for several applications reducing the number of computing resources needed. One of the most popular and widely used techniques for analyzing hyperspectral data is linear spectral unmixing, which relies on the identification of pure spectral signatures via a so-called endmember extraction algorithm. In this paper, we present the first FPGA design for N-FINDR, a widely used endmember extraction algorithm in the literature. Our system includes a direct memory access module and implements a prefetching technique to hide the latency of the input/output communications. The proposed method has been implemented on a Virtex-4 XC4VFX60 FPGA (a model that is similar to radiation-hardened FPGAs certified for space operation) and tested using real hyperspectral data collected by NASA's Earth Observing-1 Hyperion (a satellite instrument) and the Airborne Visible Infra-Red Imaging Spectrometer over the Cuprite mining district in Nevada and the Jasper Ridge Biological Preserve in California. Experimental results demonstrate that our hardware version of the N-FINDR algorithm can significantly outperform an equivalent software version and is able to provide accurate results in near real time, which makes our reconfigurable system appealing for onboard hyperspectral data processing.

New generation embedded systems demand high performance, efficiency, and flexibility. Reconfigurable hardware can provide all these features. However, the costly reconfiguration process and the lack of management support have prevented a broader use of these resources. To solve these issues we have developed a scheduler that deals with task-graphs at run-time, steering its execution in the reconfigurable resources while carrying out both prefetch and replacement techniques that cooperate to hide most of the reconfiguration delays. In our scheduling environment, task-graphs are analyzed at design-time to extract useful information. This information is used at run-time to obtain near-optimal schedules, escaping from local-optimum decisions, while only carrying out simple computations. Moreover, we have developed a hardware implementation of the scheduler that applies all the optimization techniques while introducing a delay of only a few clock cycles. In the experiments our scheduler clearly outperforms conventional run-time schedulers based on as-soon-as-possible techniques. In addition, our replacement policy, specially designed for reconfigurable systems, achieves almost optimal results both regarding reuse and performance.

In this article, we present a design of a Reversi player submitted to the FPT'10 Design Competition and implemented on a XC2VP30 Virtex-II Pro FPGA. Our player applies several techniques to explore the solution space attempting to look as many moves forward as possible for the given time, and uses several metrics to evaluate the quality of a given board. The most important metric is the mobility, basically our player attempts to maximise its available moves whereas minimising the opponent moves. With these techniques our player easily defeats the competition software opponent.

Satellites gather information from different sensors and send it using a temporal connection with very limited bandwidth. The bandwidth demand can be highly reduced if the satellite preprocesses the data and only sends relevant information. FPGAs offer the necessary flexibility and performance to carry out this preprocessing step. We have designed and implemented the well-known PPI algorithm for hyper spectral images providing better performance than previous HW implementations and reducing the penalties due to the I/O communications. We have also included a HW module for random number generation. The results show that our system can preprocess a hyper spectral image in a few seconds.

Due to the emergence of portable devices that must run complex dynamic applications there is a need for flexible platforms for embedded systems. Runtime reconfigurable hardware can provide this flexibility but the reconfiguration latency can significantly decrease the performance. When dealing with task graphs, runtime support that schedules the reconfigurations in advance can drastically reduce this overhead. However, executing complex scheduling heuristics at runtime may generate an excessive penalty. Hence, we have developed a hybrid design-time/runtime reconfiguration scheduling heuristic that generates its final schedule at runtime but carries out most computations at design-time. We have tested our approach in a PowerPC 405 processor embedded on a FPGA demonstrating that it generates a very small runtime penalty while providing almost as good schedules as a full runtime approach.