Deploying real-time intelligent systems at the edge poses significant challenges in dynamic, resource-constrained, and low-visibility environments. We present a bio-inspired event camera architecture for robust and adaptable continuous target detection and tracking under tough environmental conditions. Leveraging the high temporal resolution and dynamic range of event-based sensing, our approach addresses key limitations in small object detection and low-light imaging. Built on HARP (Hierarchical Attention-Oriented Region-Based Processing [16]), our hardware platform prioritizes salient regions and performs early-stage, sensor-level information extraction. This attentionguided strategy reduces redundant spatiotemporal processing and enables learning models to focus computation on critical regions. A hierarchical and parallel pipeline further maximizes throughput by exploiting high-bandwidth image access. We prototyped the system on FPGA and validated its performance in two key tasks: small object detection and low-light enhancement. Compared to prior FPGA-based designs, our system reduces logic resource usage by up to 88%, increases throughput by 47%, and lowers latency by over 30%. For enhancement, it achieves 14% lower MSE, 4.5% higher PSNR, and 7.1% higher SSIM. Our quantized detector maintains strong semantic accuracy with only 7.05% degradation from its float baseline, using just 1.1M MACs and 42K parameters-suitable for real-time deployment on embedded platforms.

In low-light environments, event cameras suffer from sparse and noisy outputs. We propose ASTEF, a near-sensor FPGA architecture using adaptive temporal filtering and dynamic thresholding to improve signal quality under poor illu-mination. Implemented on the Zynq-7000 SoC, ASTEF reduces Slice LUTs by 83.91% and 93.49% compared to HMAX and HARP, respectively, while lowering dynamic power by up to 83.88%. A Python-based simulator further verifies its robustness, outperformina ROI-based models in dark scenes.

FPGA-based cloud services face significant security challenges, especially in multi-tenant environments where protecting tenant Intellectual Property (IP) is crucial. Existing solutions have limitations, being vulnerable to attacks or requiring FPGA manufacturer involvement for each deployment. We propose CIVIC-FPGA (Confidential IP Validation In Cloud for FPGA), a secure protocol for validating FPGA designs in multi-tenant cloud environments, ensuring tenant design confidentiality.

Multimodal learning has gained much success in recent years. However, current multimodal fusion methods adopt the attention mechanism of Transformers to implicitly learn the underlying correlation of multimodal features. As a result, the multimodal model cannot capture the essential features of each modality, making it difficult to comprehend complex structures and correlations of multimodal inputs. This paper introduces a novel Multimodal Attention-based Normalizing Flow (MANGO) approach to developing explicit, interpretable, and tractable multimodal fusion learning. In particular, we propose a new Invertible Cross-Attention (ICA) layer to develop the Normalizing Flow-based Model for multimodal data. To efficiently capture the complex, underlying correlations in multimodal data in our proposed invertible cross-attention layer, we propose three new cross-attention mechanisms: Modality-to-Modality Cross-Attention (MMCA), Inter-Modality Cross-Attention (IMCA), and Learnable Inter-Modality Cross-Attention (LICA). Finally, we introduce a new Multimodal Attention-based Normalizing Flow to enable the scalability of our proposed method to high-dimensional multimodal data. Our experimental results on three different multimodal learning tasks, i.e., semantic segmentation, image-to-image translation, and movie genre classification, have illustrated the state-of-the-art (SoTA) performance of the proposed approach.

With the growing demand for enhanced performance and scalability in cloud applications and systems, data center architectures are evolving to incorporate heterogeneous computing fabrics that leverage CPUs, GPUs, and FPGAs. Unlike traditional processing platforms like CPUs and GPUs, FPGAs offer the unique ability for hardware reconfiguration at runtime, enabling improved and tailored performance, flexibility, and acceleration. FPGAs excel at executing large-scale search optimization, acceleration, and signal processing tasks while consuming low power and minimizing latency. Major public cloud providers, such as Amazon, Huawei, Microsoft, Alibaba, and others, have already begun integrating FPGA-based cloud acceleration services into their offerings. Although FPGAs in cloud applications facilitate customized hardware acceleration, they also introduce new security challenges that demand attention. Granting cloud users the capability to reconfigure hardware designs after deployment may create potential vulnerabilities for malicious users, thereby jeopardizing entire cloud platforms. In particular, multi-tenant FPGA services, where a single FPGA is divided spatially among multiple users, are highly vulnerable to such attacks. This article examines the security concerns associated with multi-tenant cloud FPGAs, provides a comprehensive overview of the related security, privacy and trust issues, and discusses forthcoming challenges in this evolving field of study.

Side-channel analysis has been proven effective at detecting hardware Trojans in integrated circuits (ICs). However, most detection techniques rely on large external probes and antennas for data collection and require a long measurement time to detect Trojans. Such limitations make these techniques impractical for run-time deployment and ineffective in detecting small Trojans with subtle side-channel signatures. To overcome these challenges, we propose a Programmable Sensor Array (PSA) for run-time hardware Trojan detection, localization, and identification. PSA is a tampering-resilient integrated on-chip magnetic field sensor array that can be re-programmed to change the sensors' shape, size, and location. Using PSA, EM side-channel measurement results collected from sensors at different locations on an IC can be analyzed to localize and identify the Trojan. The PSA has better performance than conventional external magnetic probes and state-of-the-art on-chip single-coil magnetic field sensors. We fabricated an AES-128 test chip with four AES Hardware Trojans. They were successfully detected, located, and identified with the proposed on-chip PSA within 10 milliseconds using our proposed cross-domain analysis.

Although FPGAs in cloud applications facilitate customized hardware acceleration, they also introduce new security challenges that demand attention. Granting cloud users, the capability to reconfigure hardware designs after deployment may create potential vulnerabilities for malicious users, thereby jeopardizing entire cloud platforms. Multi-tenant FPGA services, where a single FPGA is divided spatially among multiple users, are highly vulnerable to such attacks such as remote power side channel attacks, Denial of Service (DoS) attacks and Fault Injection attacks. Security solutions are limited by the architectural design of existing FPGAs. We propose a novel power distribution network for cloud FPGA security using physical CMOS-based galvanic isolation. In this architecture, each tenant is isolated spatially, providing protection against voltage spikes, ground loops, and electrical noise, the key premises of remote physical attacks. The isolation technique is carried out by using reconfigurable MoM (Metal-over-Metal) capacitors and switch banks, along with Power Management and Configuration Controller Unit. By implementing a Custom Configuration Memory (CCM), we aim to provide a dynamic and customizable solution that allows FPGA designers to selectively interconnect or isolate groups of Configurable Logic Blocks (CLBs). This approach involves the formation of distinct regions within the FPGA, each capable of sourcing power either from a dedicated CMOS isolation power supply or the standard FPGA voltage power supply. Our approach, leveraging physical isolation, can successfully prevent such attacks and can be established as the first line of defense for cloud FPGA security.

With the prevalence of networked and embedded devices adopting System-on-Chip architectures there is a need for accompanying security architectures to protect these devices. Most current SoCs do not contain security built into the hard- ware, relying solely on external security. This makes it possible for any software application running on the SoC to read the sensitive data of all hardware components, perform Denial- of-Service attacks, and more. Given this, on-chip security is a hot topic for SoCs. One security architecture that is of international interest is the Zero-Trust Architecture.

The ever-growing demand for intelligent devices capable of visual processing in real time at the edge requires a paradigm shift in computing architectures. Near-sensor computing offers a promising solution by bringing computation closer to the data source, enabling faster response times and reduced power consumption. However, traditional monolithic chip design struggles to meet the efficiency and flexibility demands of near- sensor visual intelligence tasks. This Ph.D. project investigates the transformative potential of chiplet interfaces in revolutionizing near-sensor computing for on-device visual intelligence. Chiplet technology offers a modular approach that enables the integration of heterogeneous cores and specialized hardware accelerators into a single package. Using this modularity, the project aims to achieve the following key objectives: (1) design and explore novel chiplet interface architectures, (2) hardware-software co-design for chiplet-based near-sensor processing targeting visual data, (3) power efficiency exploration and finally real-world application validation. The successful completion of this Ph.D. project is expected to yield significant contributions to the field of near- sensor computing. With expectations of proposing some novel chiplet interfaces for efficient visual on-device intelligence, the project has the potential to pave the way for a new generation of intelligent devices capable of real-time visual processing at the edge, with minimal reliance on cloud-based resources.

This abstract proposes a lightweight near-sensor image preprocessor for video data. The hardware system design targets low-power devices with constrained power and hardware resources. Our approach is a two-stage hierarchical architecture consisting of saliency-based generators that feed a saliency filter. Preliminary results demonstrated significant energy and hardware optimization compared to previous work in this field. Specifically, the proposed architecture uses 0.24%, 0.60%, and 9% of the total processing elements used by examined related work. Similarly, the total on-chip power usage of 0.686W, which is 34.5 % and 5.3% less for the same baseline. This motivates the integration of our near-sensor logic into a system where a high-level algorithm can benefit from data filtering.

Although FPGAs in cloud applications facilitate customized hardware acceleration, they also introduce new security challenges that demand attention. Granting cloud users, the capability to reconfigure hardware designs after deployment may create potential vulnerabilities for malicious users, thereby jeopardizing entire cloud platforms. Multi-tenant FPGA services, where a single FPGA is divided spatially among multiple users, are highly vulnerable to such attacks such as remote power side channel attacks, Denial of Service (DoS) attacks and Fault Injection attacks. We propose a novel power distribution network for cloud FPGA security using physical CMOS-based galvanic isolation. In this architecture, each tenant is isolated spatially, providing protection against voltage spikes, ground loops, and electrical noise, the key premises of remote power side-channel attacks. The isolation technique is carried out by using reconfigurable MoM (Metal-over-Metal) capacitors and switch banks, along with Power Management and Configuration Controller Unit. By implementing a Custom Configuration Memory (CCM), we aim to provide a dynamic and customizable solution that allows FPGA designers to selectively interconnect or isolate groups of Configurable Logic Blocks (CLBs). This approach involves the formation of distinct regions within the FPGA, each capable of sourcing power either from a dedicated CMOS isolation power supply or the standard FPGA voltage power supply. Our approach, leveraging physical isolation, can successfully prevent such attacks and can be established as the first line of defense for cloud FPGA security.

Robotic agents trained using reinforcement learning have the problem of taking unreliable actions in an out-of-distribution (OOD) state. Agents can easily become OOD in real-world environments because it is almost impossible for them to visit and learn the entire state space during training. Unfortunately, unreliable actions do not ensure that agents perform their original tasks successfully. Therefore, agents should be able to recognize whether they are in OOD states and learn how to return to the learned state distribution rather than continue to take unreliable actions. In this study, we propose a novel method for retraining agents to recover from OOD situations in a self-supervised manner when they fall into OOD states. Our in-depth experimental results demonstrate that our method substantially improves the agent’s ability to recover from OOD situations in terms of sample efficiency and restoration of the performance for the original tasks. Moreover, we show that our method can retrain the agent to recover from OOD situations even when in-distribution states are difficult to visit through exploration. Code and supplementary materials are available at https://github.com/SNUChanKim/SeRO.

The increasing adoption of Field-Programmable Gate Arrays (FPGA) into cloud and data center systems opens the way to the unprecedented acceleration of Machine Learning applications. Convolutional Neural Networks (CNN) have largely been adopted as algorithms for image classification and object detection. As we head towards FPGA multi-tenancy in the cloud, it becomes necessary to investigate architectures and mechanisms for the efficient deployment of CNN into multitenant FPGAs cloud Infrastructure. In this work, we propose an FPGA architecture and a design flow that support efficient integration of CNN applications into a cloud infrastructure that exposes multi-tenancy to cloud developers. We prototype the proposed approach on randomly allocated virtual regions to tenants. We study how space-sharing of a single device between multiple cloud tenants influence the design flow, the allocation of resources, and the performance in term of resource utilization and overall latency compared to single-tenant deployments. Prototyping results show a latency at most 8% lower than that of single-tenant deployment while achieving higher resource utilization. We also record a maximum frequency of up to 12% higher in multi-tenant implementations.

With the vast adoption of FPGAs in the cloud, it becomes necessary to investigate architectures and mechanisms for the efficient deployment of CNN into multi-FPGAs cloud Infrastructure. However, neural networks’ growing size and complexity, coupled with communication and off-chip memory bottlenecks, make it increasingly difficult for multi-FPGA designs to achieve high resource utilization. In this work, we introduce a scalable framework that supports the efficient integration of CNN applications into a cloud infrastructure that exposes multi-Die FPGAs to cloud developers. Our framework is equipped is with two mechanisms to facilitate the deployment of CNN inference on FPGA. First, we propose a model to find the parameters that maximize the parallelism within the resource budget while maintaining a balanced rate between the layers. Then, we propose an efficient Coarse-Grained graph partitioning algorithm for high-quality and scalable routability-drive placement of CNN’s components on the FPGAs. Prototyping results achieve an overall 37% higher frequency, with lower resource usage compared to a baseline implementation on the same number of FPGAs.

In this article, we survey existing academic and commercial efforts to provide Field-Programmable Gate Array (FPGA) acceleration in datacenters and the cloud. The goal is a critical review of existing systems and a discussion of their evolution from single workstations with PCI-attached FPGAs in the early days of reconfigurable computing to the integration of FPGA farms in large-scale computing infrastructures. From the lessons learned, we discuss the future of FPGAs in datacenters and the cloud and assess the challenges likely to be encountered along the way. The article explores current architectures and discusses scalability and abstractions supported by operating systems, middleware, and virtualization. Hardware and software security becomes critical when infrastructure is shared among tenants with disparate backgrounds. We review the vulnerabilities of current systems and possible attack scenarios and discuss mitigation strategies, some of which impact FPGA architecture and technology. The viability of these architectures for popular applications is reviewed, with a particular focus on deep learning and scientific computing. This work draws from workshop discussions, panel sessions including the participation of experts in the reconfigurable computing field, and private discussions among these experts. These interactions have harmonized the terminology, taxonomy, and the important topics covered in this manuscript.

Cloud deployments now increasingly exploit Field-Programmable Gate Array (FPGA) accelerators as part of virtual instances. While cloud FPGAs are still essentially single-tenant, the growing demand for efficient hardware acceleration paves the way to FPGA multi-tenancy. It then becomes necessary to explore architectures, design flows, and resource management features that aim at exposing multi-tenant FPGAs to the cloud users. In this article, we discuss a hardware/software architecture that supports provisioning space-shared FPGAs in Kernel-based Virtual Machine (KVM) clouds. The proposed hardware/software architecture introduces an FPGA organization that improves hardware consolidation and support hardware elasticity with minimal data movement overhead. It also relies on VirtIO to decrease communication latency between hardware and software domains. Prototyping the proposed architecture with a Virtex UltraScale+ FPGA demonstrated near specification maximum frequency for on-chip data movement and high throughput in virtual instance access to hardware accelerators. We demonstrate similar performance compared to single-tenant deployment while increasing FPGA utilization, which is one of the goals of virtualization. Overall, our FPGA design achieved about 2× higher maximum frequency than the state of the art and a bandwidth reaching up to 28 Gbps on 32-bit data width.

This paper presents an event-based split-CNN architecture (ESCA) for running time-critical vision applications with comparatively less memory footprint while consuming low power. ESCA has a dedicated hardware architecture and scheduling of on-chip memory buffering using a split-CNN that reduces memory requirements by splitting the feature maps into small patches and independently executes them. The model emulates the concepts of the biological vision system to obtain possible events from each patch. We save energy and time by processing only the patches with possible events. The data-level parallelism is employed with a deep pipeline strategy that accelerates the system. We implement the design in the Virtex UltraScale+ FPGA at 320 MHz. Simulation results show that the architecture obtains significant speedup while power-saving depends on each image patch's features.

Cloud and data center applications increasingly leverage FPGAs because of their performance/watt benefits and flexibility advantages over traditional processing cores such as CPUs and GPUs. As the rising demand for hardware acceleration gradually leads to FPGA multi-tenancy in the cloud, there are rising concerns about the security challenges posed by FPGA virtualization. Exposing space-shared FPGAs to multiple cloud tenants may compromise the confidentiality, integrity, and availability of FPGA-accelerated applications. In this work, we present a hardware/software architecture for domain isolation in FPGA-accelerated clouds and data centers with a focus on software-based attacks aiming at unauthorized access and information leakage. Our proposed architecture implements Mandatory Access Control security policies from software down to the hardware accelerators on FPGA. Our experiments demonstrate that the proposed architecture protects against such attacks with minimal area and communication overhead.

Cloud deployments now increasingly provision FPGA accelerators as part of virtual instances. While FPGAs are still essentially single-tenant, the growing demand for hardware acceleration will inevitably lead to the need for methods and architectures supporting FPGA multi-tenancy. In this paper, we propose an architecture supporting space-sharing of FPGA devices among multiple tenants in the cloud. The proposed architecture implements a network-on-chip (NoC) designed for fast data movement and low hardware footprint. Prototyping the proposed architecture on a Xilinx Virtex Ultrascale + demonstrated near specification maximum frequency for on-chip data movement and high throughput in virtual instance access to hardware accelerators. We demonstrate similar performance compared to single-tenant deployment while increasing FPGA utilization (we achieved $6 \times$ higher FPGA utilization with our case study), which is one of the major goals of virtualization. Overall, our NoC interconnect achieved about $2 \times$ higher maximum frequency than the state-of-the-art and a bandwidth of 25.6 Gbps.

In this paper, we propose an automated framework that takes as input a TensorFlow inference graph and generates high-performance accelerators on FPGA by assembling CNN pre-implemented components as a puzzle, based on the graph topology. Using pre-implemented components allows us the only use the minimum of resources necessary, predict the performance and a gain in productivity We adopt a unified representation based on systolic array to perform the computational-hungry operations of the model and provide novel analysis of design trade-offs for FPGA CNN accelerators. Experimental results show the great performance, low latency and flexibility provided by the proposed framework.

This work presents a network-on-chip based architecture enabling multi-tenant access to FPGAs in cloud infrastructures. Prototyping the proposed architecture on Xilinx Virtex Ultrascale + and Intel Stratix IV demonstrated performance similar to single tenant mode while enabling hardware consolidation.

This paper presents the MeXT-SE (Multiprocessor Exploration Tool with Security Extension), an FPGAbased MPSoC development tool capable of generating platformindependent MPSoCs with enforced hardware access control mechanism from a high-level abstraction.

Convolutional Neural Networks have been adopted in a wide range of vision-based application domains in recent times, due to their success in enabling ubiquitous machine vision and intelligent decisions. However, the overwhelming computation demand of the convolution operations has somewhat limited their use in resource-constrained embedded platforms. This paper presents a pixel processing architecture to facilitate CNN inference near the image sensor. The architecture exploits the high bandwidth available at the sensor interface and incorporates an array of pixel processors to perform inference directly on the sensor device. The proposed design addresses problems related to the mapping of computations onto an array of pixel processors and introduces a suitable network structure for communication. The pixel processors are highly optimized to provide low latency and power for CNN applications. While designing the pixel processors, we focused on reducing redundancies using the concepts of biological vision systems. We prototype the model in a Virtex UltraScale FPGA and implement it in ASIC using the TSMC 90nm technology library. The results suggest that the proposed architecture significantly reduces dynamic power consumption and achieves high-speed up surpassing the computational capabilities of existing embedded processors.

In this work, the authors propose a novel decentralised coordination scheme for autonomous ground vehicles to enable map building and path planning with a network of smart overhead cameras. Decentralised indoor smart camera mapping and hierarchical navigation supports the automatic generation of waypoint graphs for each camera in an environment and allows path planning through the environment across multiple camera fields of view, or subviews. The proposed solution utilises the growing neural gas algorithm to learn the topology of unoccupied working space in each subview for maintaining a dynamic waypoint graph on each camera. The authors’ pathing solution leverages a modified version of the A* algorithm to compute paths in a decentralised and hierarchical fashion. Waypoint generation was simulated and analysed on a generated environment to ensure it is both effective and efficient, while path planning was simulated on various randomised hierarchical graphs to effectively compare the proposed Decentralised‐A* (D‐A*) algorithm against standard greedy search. The proposed method efficiently handles the cases where other robot navigation methods are otherwise weak and ineffective, while still providing avenues for further optimisation of resource overhead for both the smart camera network as well as the robots themselves.

Hardware accelerators are increasingly employed in conjunction with general-purpose processors to meet stringent performance constraints. In these heterogeneous systems, security has become a prime concern. In this article, we present a design approach to generate platform-independent secure multiprocessor systems-on-chip (MPSoC) from a high-level abstraction. The aim of this article is to simplify the implementation of MPSoC, while ensuring security. The proposed design flow starts with a set of abstract and concrete specifications of a system, provided by the user, and ends up generating a generic description of the appropriate hardware design by setting up the communication structure of different components. The resulting abstract architecture is further processed using the vendor tool-chain to generate the target platform's configuration. To enforce security, we proposed a distributed isolation framework for multilevel security, resource assess control including access to and from hardware accelerators. From a user specification, the security layer is transparently generated and security rules and requirements transparently enforced at run-time. Experimentation results show that our proposed tool can generate MPSoC designs capable of providing secure hardware execution with negligible execution overhead.

This paper presents a reconfigurable hierarchical hardware architecture at the pixel and region level for smart image sensors to accelerate machine vision applications. This architecture maintains hierarchical processing that begins at the pixel level. It reduces the computational burden on the sequential processor and accelerates the overall processing. There are three hierarchical layers, and each layer passes only the relevant information to the next layer by removing redundant information. This method facilitates the final processor to respond if there is an event in the scene. Identifying relevant information is complex, and we adapted different bio-inspired algorithms in the hierarchical layers to meet this purpose. Besides, these processors are made runtime reconfigurable to different applications, and it adds flexibility after fabrication. This hierarchical processing breaks the traditional sequential image processing and introduces parallelism for the machine vision applications. We evaluate the design in FPGA and achieve the GDSII file in ASIC platform at 800MHz. Simulation results show that the area overhead and power penalty for adding reconfiguration feature stay in an acceptable range. Besides, removing redundant information 84.01% and 96.91% dynamic power can be saved at the pixel-level and region-level, respectively.

This paper presents a reconfigurable hardware architecture of smart image sensors to speed up low-level image processing applications at the pixel level. For each pixel in the sensor plane, the design includes an activation module and a processor. The processor has a basic structure which is common to all applications and reconfigurable segments for specific applications. Visual cortex inspired computing, like, Predictive Coding in time is implemented in the activation module to remove temporal redundancy. The ASIC implementation shows the design saves up to 84.01% dynamic power and achieves 9x speedup at 800 MHz by accurate prediction.

editorial Free Access Share on Introduction to the Special Section on Security in FPGA-accelerated Cloud and Datacenters Editors: Chistophe Bobda University of Florida Russell Tessier, University of Massachusetts Amherst University of Florida Russell Tessier, University of Massachusetts AmherstView Profile , Ken Eguro Microsoft Research Ryan Kastner, University of California, San Diego Microsoft Research Ryan Kastner, University of California, San DiegoView Profile Authors Info & Claims ACM Transactions on Reconfigurable Technology and SystemsVolume 12Issue 3September 2019 Article No.: 11epp 1–3https://doi.org/10.1145/3352060Published:13 September 2019Publication History 0citation190DownloadsMetricsTotal Citations0Total Downloads190Last 12 Months44Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF

Field-Programmable Gate Arrays (FPGAs) are becoming important components within commercially available cloud computing systems. However, the FPGAs are not yet sufficiently abstracted within existing software ecosystems. Contrary to how applications are transparently scheduled across general purpose processors, software processes need to explicitly provision and control communications with hardware circuits within the FPGAs. In this paper, we introduce a novel virtualization framework called FPGAVirt that leverages Virtio to implement an efficient communication scheme between virtual machines and the FPGAs. FPGAVirt avoids the overhead of context switches between virtual machine and host address spaces by using the in-kernel network stack for transferring packets to FPGAs. Experimental results show FPGAVirt can deliver an additional 2x to 35x performance increase compared to current state of the art virtualization approaches.

Domain isolation enforcement is one of the challenging issues in software environments. To address this problem, NSA, in conjunction with the Secure Computing Corporation and the University of Utah, developed the open-source Flux Advanced Security Kernel (Flask), the mandatory access control (MAC) security architecture underlying major Operating Systems/Hypervisors widely deployed in cloud/desktop environments. In this work, we extend this security architecture to FPGA-based heterogeneous systems. Specifically, we explore the design and implementation of a security framework for controlled sharing of FPGA hardware modules in MAC-based OS/Hypervisor environments. The proposed design guarantees that hardware modules execute in the same security context as of the processes calling them by propagating the latter security policies expressed at the software level, down to the hardware. We prototype the proposed framework with SELinux and demonstrate its utility by evaluating trade-offs between security performance and execution overhead incurred by example applications. The preliminary results show our proposed framework provides isolation with an average of 0.6% worst case performance overhead.

This paper presents a CPU-FPGA heterogeneous system that provides hardware support for computer vision libraries to attain acceleration over image processing applications. The architecture achieves this improvement by staying completely transparent to the developer while providing necessary acceleration on conventional software designs.

In this paper, we present a new security framework which allows controlled sharing and isolated execution of mutually distrusted FPGA-accelerators in heterogeneous cloud systems. The proposed framework enables the accelerators running in FPGAs in cloud computers to transparently inherit at run-time, software security policies of the virtual machines processes calling them. This capability allows system security policies enforcement mechanism to propagate access control privilege boundaries expressed at the hypervisor level, down to individual FPGA-accelerators. Furthermore, we present a software/hardware prototype implementation of the proposed security framework, showing that it can easily be transparently integrated within the virtual machine software stacks that run in today's cloud-based systems. Experimentation results show our proposed framework provides secure hardware execution with negligible execution overhead on guest VMs applications.

The Hybrid Memory Cube (HMC) is representative of emerging architectures that integrate FPGAs with multichannel interconnected 3-D stacked memory, offering great potential for high bandwidth streaming applications. However, creating new hardware components that tap the full potential of the concurrent communications channels requires the structural understanding of the memory layout and interconnect configurations. In this paper, we present a new development framework aimed at removing the need for software programmers to understand the underlying physical architecture. The proposed framework automates the creation of hardware/software co-designs for computer vision applications in a transparent way to the developer. The development system dynamically detects function calls in software kernels and replaces those calls by a hardware wrapper function that exploits the HMCs memory hierarchy and multichannel interconnect with the FPGA. Results show our flow can exploit the 3-D stacked memory and concurrent communications channels to achieve speed-up with no need to tune the original software application to the memory hierarchy.

One of the major obstacles to the adoption of FPGAs in high-performance computing is their programmability. It requires hardware design skills and long compilation times. Overlays have been proposed as a way to abstract FPGA resources. Unfortunately, most of the time, the topologies they use to connect computing cores impose restrictions on where tasks are placed and how they communicate. In this paper, we propose an overlay architecture designed for efficiency and flexibility. It features a novel Network-on-Chip (NoC) infrastructure making flexible, with no limitation, the placement of hardware tasks. The presented architecture allows tasks to communicate with a low latency and eases the reconfiguration of desired areas on the fabric at runtime. After prototyping the proposed architecture on an Altera Cyclone V FPGA, a maximum frequency of 282 MHz has been reached and a speedup ranging from 4x to 195x has been observed in some applications compared to the native execution.

In this paper, we introduce a novel framework for virtualizing FPGA resources in the cloud. The proposed framework targets hardware/software architectures that leverage the Virtio paradigm for efficient communication between virtual machines (VMs) and the FPGAs. Furthermore, we present an FPGA overlay that uses reconfigurable hardware tiles and a flexible network-on-chip (NoC) architecture for transparent and optimized allocation of FPGA resources to VMs. The proposed overlay makes it possible to merge several FPGA regions allocated to a VM into a larger area, thus allowing resizing of FPGA's resources on demand. Hardware sandboxes are then provided as a means to enforce domain separation between hardware tasks belonging to different VMs. The framework introduced prevents the overhead of context switches between the virtual machine and host address spaces by using the in-kernel network stack for transferring packets to FPGAs. Experimental results show a 2x to 35x performance increase compared to current state of the art virtualization approaches.

Pointcloud is a very precise digital format for recording objects in space. Pointclouds have received increasing attention lately, due to the higher amount of information it provides compared to images. In this paper, we propose a new deep learning architecture called R-CovNet, designed for 3D object recognition. Unlike previous architectures that usually sample or convert pointcloud into three-dimensional grids before processing, R-CovNet does not require any preprocessing. Our main goal is to provide a permutation invariant architecture specially designed for pointclouds data of any size. Experiments with well-known benchmarks show that R-CovNet can achieve an accuracy of 92.7%, thus outperforming all the volumetric methods.

We explore a reference monitor (RM) design which borrows from the Flask security architecture. Our RM design goal is to achieve complete mediation by checking and verifying the authority and authenticity of every access to every system object. Access decisions are administered by a security logic server implemented as an extension of the peripheral bus. Initial results show a minimal increase in resource overhead and no significant impact on the performance.

We propose a design flow for automatic generation of hardware sandboxes. Our tool, the Component Authentication Process for Sandboxed Layouts (CAPSL), generates sandboxes capable of detecting trojan activation and nullifying potential damage to a system at run-time. Our approach captures the behavioral properties of non-trusted IPs with formal models that are translated to checker automata and implemented within a untrusted partition of the system to isolate sandbox-system interactions upon deviation from the behavioral checkers.

This paper explores the use of hardware sand-boxes, conceptually similar to software sandboxes, for secure integration of non-trusted IPs in systems-on-chip (SoC) designs. The goal of the hardware sandbox is to only allow permissible interactions between the IP and the rest of the system. The hardware sandbox design achieves this by exposing the IP interface to isolated virtual resources and checking IP signals' "correctness" at run-time. We evaluated the hardware sandbox through a real world design implementation. Our sandbox can detect a majority of Trust-Hub.org Trojan benchmarks with a negligible increase in resource overhead.

The goal of this work is the design and implementation of a low-cost system-on-FPGA for handwritten digit recognition, based on a relatively deep and wide network of perceptrons. In order to increase the performance of the application on embedded processors whose performances are way below standard general purpose CPUs, a regularization method was used during the training phase of the neural network that allows for the drastic reduction of floating point operations. Our implementation can achieve a 3× speed-up toward a raw implementation without optimization, while keeping the accuracy in acceptable ranges. Our efforts reinforce the fact that FPGAs are suited for deploying complex artificial intelligence modules.

A synthesis approach based on Answer Set Programming (ASP) for heterogeneous system-on-chips to be used in distributed camera networks is presented. In such networks, the tight resource limitations represent a major challenge for application development. Starting with a high-level description of applications, the physical constraints of the target devices, and the specification of network configuration, our goal is to produce optimal computing infrastructures made of a combination of hardware and software components for each node of the network. Optimization aims at maximizing speed while minimizing chip area and power consumption. Additionally, by performing the architecture synthesis simultaneously for all cameras in the network, we are able to minimize the overall utilization of communication resources and consequently reduce power consumption. Because of its reconfiguration capabilities, a Field Programmable Gate Array (FPGA) has been chosen as the target device, which enhances the exploration of several design alternatives. We present several realistic network scenarios to evaluate and validate the proposed synthesis approach.

Security is becoming the primary concern in today's embedded systems. Network-on-Chip (NoC) based communication architectures have emerged as an alternative to shared bus mechanism in Multiprocessor System-on-Chip (MPSoC) devices, and the increasing number and functionality of processing cores has made such systems vulnerable to security attacks. In this paper an id and address verification (IAV) security module is presented, which is embedded in each router at the communication level. IAV verifies the identity and address range to be accessed by incoming and outgoing data packets in a NoC-based many-core shared memory architecture. Our IAV architecture is implemented on a FPGA device for functional verification and evaluated in terms of its area and power consumption overhead. For FPGA-based systems, the IAV module can be reconfigured at run-time through partial reconfiguration. In addition, a cycle-accurate simulation is carried out to analyse the performance overhead for different network configurations. The proposed IAV module has reduced area and power consumption overhead when compared with similar existing solutions.

Current high-level synthesis tools based on C/C++ offer only limited support for recursion and functions pointers. We present a novel approach for high-level synthesis that represents the program as a term rewriting system. Based on this concept, dynamic creation of threads, parallel recursive tasks and data-dependent branching can be supported in hardware. Complex examples are used to show the effectiveness of our method.

Incremental on-line scheduling of tasks on reconfigurable devices may lead to suboptimal placements of tasks, in particular when communicating tasks and components have to be placed far apart due to the current device occupation. This includes computing modules, memories and peripherals and leads to high latency and high network bandwidth in Dynamic Networks-on-Chip, thus reducing the performance of applications. Relocation of tasks at runtime provides a way to optimize the placement of tasks with the goal of reducing communication distance between communication partners, resulting in an increase of performance of the whole system. In this work, we introduce a communication-aware on-line scheduler, as well as relocation strategies to cope with the drawback of device fragmentation. By selecting a feasible relocation task set, reordering the selected tasks and replace them, we are able to show an improvement of up to 25% in communication costs, with a lower rejection rate of tasks.

Flexible Chip Multiprocessor (CMP) systems are implemented on field programmable devices to exploit both task-level parallelism and architecture customization for parallel programs. The idea is to simultaneously allocate processor resources, map and schedule tasks to them, and to allocate one or several intertask communication resources such that the throughput or execution time is optimized. The design space of such systems is huge, requiring means to automatically optimize design parameters so as to facilitate wide and disciplined explorations. The complexity resulting from corresponding system modeling necessitates the use of optimization heuristics to cope with excessively long runtime for large problem instances. This paper provides a formal proof for the existence of optimum linear time synthesis algorithms for one of two classes of problem instances, and proceeds to present three greedy-like heuristics which exploit the structure of the synthesis problem. A comparison of results for real-time and non-real-time parallel programs is given against integer linear programming, where a synthesis strategy is proposed to achieve good results.

In this paper, we present a new deadlock-free fault-tolerant adaptive routing algorithm for the 2D mesh NoC interconnections. The main contribution of this routing algorithm is that it allows both, routing of messages in the networks incorporating the regions not necessarily rectangular, and routing to all nodes which are not completely blocked by faulty nodes. The proposed routing algorithm is based on a modified turn model and well known XY algorithm. We detail the basic principle of this routing algorithm, prove its deadlock freeness, its feasibility and efficiency through the simulation results.

This paper presents a method for makespan minimization during automatic synthesis of multiprocessor systems from parallel programs. The method jointly conduct task mapping, resource allocation and real-time scheduling.

The design and implementing of a key point detector on embedded reconfigurable hardware is investigated. The major challenges are efficient hardware/software partitioning of the key point detector algorithm, data flow management as well as efficient use of memory, bus and processor. We present a modular and manual hardware/software co-design, with its implementation on a Xilinx XUP-Virtex II Pro board co-design to solve these issues.

The design and implementing of a key point detector on embedded reconfigurable hardware is investigated. The major challenges are efficient hardware/software partitioning of the key point detector algorithm, data flow management as well as efficient use of memory, bus and processor. We present a modular and manual hardware/software co-design, with its implementation on a Xilinx XUP-Virtex II Pro board to solve these issues.

In this article, we present CuNoC, a new paradigm for intercommunication between modules dynamically placed on a chip for FPGA-based reconfigurable devices. The CuNoC is based on scalable communication unit called CU which allows the simultaneous communication between several processing elements placed on the chip. We present the basic concept of this communication approach, its main advantages and drawbacks with regards to the other main NoC approaches already proposed.

We present a new concept as well as the implementation of an FPGA-based reconfigurable platform, the Erlangen Slot Machine (ESM). The main advantages of this platform are: first, the possibility for each module to access its peripheries independent from its location through a programmable crossbar, and distributed SRAMs among slices. This allows an unrestricted relocation of modules on the device. Second, the intermodule structure allows an unlimited communication among running modules.

A new paradigm to support the communication among modules dynamically placed on a reconfigurable device at run-time is presented. Based on the network on chip (NoC) infrastructure, we developed a dynamic communication infrastructure as well as routing methodologies capable to handle routing in a NoC with obstacles created by dynamically placed components. We prove the unrestricted reachability of components and pins, the deadlock-freeness and we finally show the feasibility of our approach by means on real life example applications.

The focus in This work is put onto extensions to typical FPGA hardware architectures in order to support some operating system functions. In particular, we examine the problem of preemptive multitasking and hardware defragmentation on a reconfigurable system based on FPGAs with a dynamically changing set of hardware tasks that can be replaced considering internal states.

Summary form only given. By increasing the amount of resources on reconfigurable platforms with the ability of partial reconfigurability, the issues of the management of these resources and their sharing among different tasks will become more of a concern. Online placement is one of these management issues that are investigated. We present a new approach for online placement of modules on reconfigurable devices, by managing the occupied space rather the free space on the device. Also an optimization of communication between running modules themselves and outside of the chip is proposed. The experimental results show a considerable decrease in communication and routing costs.

Summary form only given. Xilinx Virtex FPGAs have the possibility of dynamical partial run-time reconfiguration. If a system uses this feature with many different configuration bitstreams for substitution of parts in reconfiguration memory, the amount of necessary memory increases. The sum of memory amount which has to be provided for the configuration data is not negligible. This fact suggests the investigation of compressing data before they are stored in memory modules of a system. The compressed bitstream data has to be decompressed before transferring it to the FPGA. We show an approach of compressing configuration data at design time and decompressing them with a hardware module implemented on FPGA while run-time.

Summary form only given. This work explores various solutions to implement an application using runtime reconfigurable field programmable gate arrays (FPGA). The example is a mechatronic control system which has to adapt its behavior from time to time. Our model is a task-graph where every task is associated with an hardware module characterized by its required FPGA resource and its execution time. We propose various mappings of the tasks onto the FPGA. For the implementation of the tasks themselves the computation technique known as distributed arithmetic is used. We achieve numerous alternatives with different resource consumptions and execution times for every task. We estimate these characteristics and compare them to synthesis results. The received values are used to get the characteristics of the over-all system. The results show that the optimal mapping depends on the application timing constrains, on the complexity of the tasks as well as on the reconfiguration speed of the used FPGA.

Partial reconfiguration allows for mapping and executing several tasks on an FPGA during run time. One of the challenging problems in multitasking systems is high amount of communication costs. In this paper, we present two clustering methodologies that temporally cluster real-time tasks for a partially reconfigurable hardware and reduce communication overhead. The first algorithm aims at efficient use of resources by clustering close run-time tasks, and the second one makes the clustering with respect to a trade off between inter-task communication and resource utilization efficiency. The results show significant reduction communication costs.

We present a new approach for reconfigurable massively parallel computers. The approach uses FPGA as reconfigurable device to build parallel computers which can adapt their physical topology to match the virtual topology used to model the parallel computation paradigm of a given application. We use a case study in which a virtual ring topology is first simulated on a tree topology and then directly implemented in an FPGA configuration. Preliminary results show that we can increase the performance of the parallel computers which make use of message passing interface by a factor of up to 20% if a reconfigurable topology approach is used.