This paper provides an overview of the ongoing work to enable novel modalities of passive monocular neuromorphic vision in the NimbleAI sensing-processing architecture; namely, foveated and light-field event-driven vision with selective visual attention. The latter vision modality encodes 3D visual surroundings as sparse visual events in a 4D spatiotemporal domain, adding depth to current representation of visual information delivered by Dynamic Vision Sensors (DVS). The NimbleAI architecture implements hardware support for efficient execution of mainstream computer vision algorithms and AI models using these visual inputs. The architecture is designed to harness the latest advancements in 3D silicon integration, making it possible to squeeze sensing and spiking circuitry, memory, and processing engines into a miniature silicon volume.

The NimbleAI Horizon Europe project leverages key principles of energy-efficient visual sensing and processing in biological eyes and brains, and harnesses the latest advances in$\mathbf{33D}$stacked silicon integration, to create an integral sensing-processing neuromorphic architecture that efficiently and accurately runs computer vision algorithms in area-constrained endpoint chips. The rationale behind the NimbleAI architecture is: sense data only with high information value and discard data as soon as they are found not to be useful for the application (in a given context). The NimbleAI sensing-processing architecture is to be specialized after-deployment by tunning system-level trade-offs for each particular computer vision algorithm and deployment environment. The objectives of NimbleAI are: (1)$\mathbf{100x}$performance per mW gains compared to state-of-the-practice solutions (i.e., CPU/GPUs processing frame-based video); (2)$\mathbf{50x}$processing latency reduction compared to CPU/GPUs; (3) energy consumption in the order of tens of mWs; and (4) silicon area of approx. 50 mm2.

SiliconBurmuin is aimed at creating a multi-disciplinary neurocomputing community in the Basque Country, bringing together technology and scientific research centres and industry companies. This community will: (1) identify key biological structures and mechanisms that play a major role in vision across species, and (2) transform this knowledge into novel mathematical formalisms, neuromorphic designs and algorithms to solve industry challenges and enable new experiments of interest in neuroscience and clinical research. To achieve the latter objective in a time-effective manner, SiliconBurmuin will draw strong connections with the ongoing Horizon Europe NimbleAI project, with which it shares coordination. This is expected to allow reinforcement of ideas, knowledge and technology via a common prototyping platform where to implement IP from both projects. In addition to describing the research objectives and direction of SiliconBurmuin, this paper posits that co-coordination and co-funding of aligned projects at EU and regional levels might well be a catalyst for raising regional self-awareness of own potential and develop it to help fulfill global challenges, such as semiconductor sovereignty.

Standard wireless systems are rapidly evolving to support the strict requirements of industrial applications. In this research line, wireless time-sensitive networking (TSN) is gaining momentum, thanks to the interest of wired TSN among the industries. A critical aspect of the wireless TSN research is the integration of wired and wireless TSN to create large-scale hybrid TSN networks. This integration is currently being discussed for 5G, whereas the integration between wired TSN and IEEE 802.11 is a step behind. To address this gap in the state of the art, this article discusses the integration challenges of wired TSN and wireless local area network technologies and proposes a hybrid TSN device architecture. Based on the architecture, we have developed a hybrid TSN network proof-of-concept. The proof-of-concept demonstrates that the hybrid TSN architecture can maintain the essential TSN services, including providing guaranteed latency in the 100-microsecond level.

This paper discusses an implementation of a wireless-wired TSN network, named SHARP, which delivers hard real-time capabilities with bounded latencies of hundreds of microseconds and clock synchronization with tens of nanosecond accuracy along both wired and wireless segments. It has been demonstrated that the network is able to maintain the most important TSN services in various situations of critical real-time and non-critical traffic in the context of a representative Industry 4.0 application. The communication solution presented here is expected to pave the way for massive digitization of future flexible smart factories, especially when combined with other technologies to be commercially available in the short-medium term like 5G.

This paper presents a new phenomenon called error correlation prediction for lockstep processors. Lockstep processors run the same copy of a program, and their outputs are compared at every cycle to detect divergence, and have been popular in safety-critical systems. When the lockstep error checker detects an error, it alerts the safety-critical system by putting the lockstep processor in a safe state in order to prevent hazards. This is done by running the online diagnostics to identify the cause of the error because the lockstep processor has no knowledge of whether the error is caused by a transient or permanent fault. The online diagnostics can be avoided if the error is caused by a transient fault, and the lockstep processor can recover from it. If, however, it is caused by a permanent fault, having prior knowledge about error's likely location(s) within the CPU speeds up the diagnostics process. We discover that the error's type and likely location(s) inside CPUs from which the fault may have originated can be predicted by analyzing the output signals of the CPU(s) when the error is detected. We design a simple static predictor exploiting this phenomenon and show that system availability can be increased by 42-64% with an overhead of less than 2% in silicon area and power.

The Arm Triple Core Lock-Step (TCLS) architecture is the natural evolution of Arm Cortex-R Dual Core Lock-Step (DCLS) processors to increase dependability, predictability, and availability in safety-critical and ultra-reliable applications. TCLS is simple, scalable, and easy to deploy in applications where Arm DCLS processors are widely used (e.g., automotive), as well as in new sectors where the presence of Arm technology is incipient (e.g., enterprise) or almost non-existent (e.g., space). Specifically in space, COTS Arm processors provide optimal power-to-performance, extensibility, evolvability, software availability, and ease of use, especially in comparison with the decades old rad-hard computing solutions that are still in use. This article discusses the fundamentals of an Arm Cortex-R5 based TCLS processor, providing key functioning and implementation details. The article shows that the TCLS architecture keeps the use of rad-hard technology to a minimum, namely, using rad-hard by design standard cell libraries only to protect the critical parts that account for less than 4% of the entire TCLS solution. Moreover, when exposure to radiation is relatively low, such as in terrestrial applications or even satellites operating in Low Earth Orbits (LEO), the system could be implemented entirely using commercial cell libraries, relying on the radiation mitigation methods implemented on the TCLS to cope with sporadic soft errors in its critical parts. The TCLS solution allows thus to significantly reduce chip manufacturing costs and keep pace with advances in low power consumption and high density integration by leveraging commercial semiconductor processes, while matching the reliability levels and improving availability that can be achieved using extremely expensive rad-hard semiconductor processes. Finally, the article describes a TRL4 proof-of-concept TCLS-based System-on-Chip (SoC) that has been prototyped and tested to power the computer on-board an Airbus Defence and Space telecom satellite. When compared to the currently used processor solution by Airbus, the TCLS-based SoC results in a more than 5× performance increase and cuts power consumption by more than half.

This paper proposes a "high resilience" execution mode to increase the robustness of CPU pipelines to soft errors when executing critical software routines. The proposed execution mode reduces the error rate by approximately 11% in an ARM Cortex-R5 CPU, and requires only a few minor modifications to be made in its microarchitecture. These modifications do not impact the characteristic area, power consumption and performance features of the original CPU.

We present here one of the first research efforts conducted at Jet Propulsion Laboratory (JPL) to implement on a single chip (Xilinx Zynq) all the functionality necessary to control a NASA instrument, namely a Fourier Transform Spectrometer (FTS) that is proposed for deployment on future missions to Jupiter's moon Europa. The system requires custom logic to process the data delivered by the instrument, and software, to perform floating-point operations and to drive the interface with the main spacecraft computer. Three features are central in our SoC FTS implementation: (1) Efficiency, as the system achieves a high data processing throughput at relatively low power consumption, (2) Adaptivity, as the system can be configured from Earth based on the data observed while exploring a priori unknown space environments and (3) Fault-Tolerance, as the system needs to operate in the harsh radiation Jupiter magnetosphere where Europa orbits.

We present here an integrated SoC platform, called APEX-SoC, that is aimed at speeding-up the design of next-generation space flight instruments avionics by providing a convenient infrastructure for hardware and software based science data processing. We use a case-study drawn from the JPL Compositional Infrared Imaging Spectrometer (CIRIS) to illustrate the process of integrating instrument-dependent data acquisition and processing stages in this platform. In order to enable the use of APEX-SoC-based instruments in deep space missions, the platform implements Radiation Hardening By Design (RHBD) techniques and offers support for instantiating multiple processing stages that can be used at runtime to increase reliability or performance, based on the requirements of the mission at each particular stage. Finally, in the specific case of CIRIS, the data processing includes a stage to cope with radiation affecting the instrument photo-detector.

This article presents a new solution for easing the development of reconfigurable applications using Field-Programable Gate Arrays (FPGAs). Namely, our Reliable Reconfigurable Real-Time Operating System (R3TOS) provides OS-like support for partially reconfigurable FPGAs. Unlike related works, R3TOS is founded on the basis of resource reusability and computation ephemerality. It makes intensive use of reconfiguration at very fine FPGA granularity, keeping the logic resources used only while performing computation and releasing them as soon as it is completed. To achieve this goal, R3TOS goes beyond the traditional approach of using reconfigurable slots with fixed boundaries interconnected by means of a static communication infrastructure. Instead, R3TOS approaches a static route-free system where nearly everything is reconfigurable. The tasks are concatenated to form a computation chain through which partial results naturally flow, and data are exchanged among remotely located tasks using FPGA’s reconfiguration mechanism or by means of “removable” routing circuits. In this article, we describe the R3TOS microkernel architecture as well as its hardware abstraction services and programming interface. Notably, the article presents a set of novel circuits and mechanisms to overcome the limitations and exploit the opportunities of Xilinx reconfigurable technology in the scope of hardware multitasking and dependability.

This paper presents a new technique to be used in the context of reconfigurable computing to accelerate the online diagnosis of permanent damage on Xilinx FPGAs using Built-In Self Tests (BISTs). Detecting and locating permanently damaged resources with precision is central to keep the system implemented on the FPGA flawless at all times; i.e. upcoming hardware tasks are mapped to available functional resources, circumventing the use of the damaged ones. The proposed diagnostic technique exploits the Multiple Frame Write (MFW) feature available in Xilinx FPGAs to “clone” (i.e. replicate) a single basic BIST circuit along arbitrarily sized and shaped areas on the FPGA without incurring large time overheads. Hence, the proposed technique allows for creating at runtime on-demand tailored BIST circuits to satisfy any diagnosis requirements that may rise up. Moreover, the proposed solution allows for saving memory in the system as it only requires storing basic BIST circuits. Finally, the paper presents a diagnostic service for a Reliable Reconfigurable Real-Time Operating System (R3TOS) that is based on the BIST cloning technique and works in cooperation with the R3TOS fault-handling and recovery mechanisms.

Despite the clear potential of FPGAs to push the current power wall beyond what is possible with general-purpose processors, as well as to meet ever more exigent reliability requirements, the lack of standard tools and interfaces to develop reconfigurable applications limits FPGAs' user base and makes their programming not productive. R3TOS is our contribution to tackle this problem. It provides systematic OS support for FPGAs, allowing the exploitation of some of the most advanced capabilities of FPGA technology by inexperienced users. What makes R3TOS special is its nonconventional way of exploiting on-chip resources: These are used indistinguishably for carrying out either computation or communication tasks at different times. Indeed, R3TOS does not rely on any static infrastructure apart from its own core circuitry, which is constrained to a specific region within the FPGA where it is implemented. Thus, the rest of the device is kept free of obstacles, with the spare resources ready to be used as and whenever needed. At runtime, the hardware tasks are scheduled and allocated with the dual objective of improving computation density and circumventing damaged resources on the FPGA.

This paper presents a novel hardware mechanism to facilitate the design and implementation of soft processors on FPGAs using the Error-correcting code (ECC)-protected memory and Triple Modular Redundancy (TMR). Such techniques highly harden the fault tolerance of soft processors, especially their memories, which are the most radiation susceptible resources on FPGAs. This is demonstrated in the implementation of a fault-tolerant PicoBlaze processor on Xilinx FPGAs, in which we used an additional LookAhead technique to synchronize the processor with ECC-protected Block RAM (ECC BRAM). The resulting fault-tolerant PicoBlaze processor has the benefit of having a self-recoverable program memory in the presence of Single Error Upsets (SEUs), without halting the processor. Our techniques can be applied to other soft processors e.g. Xilinx MicroBlaze or Altera Nios.

This paper presents the implementation of several data coding functions used in Software Defined Radios, on R3TOS: a Reliable, Reconfigurable and Real-Time Operating System. The latter offers efficient high performance computing on FPGAs as well as protection against emerging faults, hence making it a perfect candidate for the implementation of SDRs. In particular, R3TOS' ICAP-based Inter-task Communication Infrastructure (I2CI) has been used for data feeding and collection from coding functions, while a task context saving and restoration procedure, and a fast function parameterization system have been developed in order to improve system performance. The design of the data coding functions has been carried out using Xilinx's rapid prototyping tool System Generator in order to ease their development.

This paper presents the details of a novel technique which allows for the implementation and execution of completely relocatable hardware tasks onto dynamically reconfigurable FPGAs. Our novel technique harnesses the internal configuration access port (ICAP) for inter-task communication and synchronization, leading to very little logic overheads. The advantages of this technique include fault-tolerance, as tasks could be relocated freely on the fabric to circumvent damaged resources, and high performance, due to better exploitation of the logic fabric. The work is part of a larger effort in our group which aims to build a fully operational dynamically reconfigurable computer which would satisfy the often conflicting requirements of high performance, fault-tolerance and high level programming.

This paper describes a novel scheduling algorithm for the execution of hardware tasks with real-time constraints onto partially and dynamically reconfigurable FPGAs. The Area-Time response Balancing scheduling algorithm (ATB) is inspired by the well-known Earliest Deadline First (EDF) algorithm, which is extended with a technique for reducing the fragmentation on FPGA's reconfigurable area. This technique promotes the reuse of the resources that are released when great area tasks finish their execution by smaller area tasks as long as the real-time constraints permit to do so. Providing an exclusively time-based algorithm, such as EDF, with support for dealing with area-related issues ensures the best results. Simulation results reported in this paper show that ATB misses 23% less deadlines than EDF. Moreover, since FPGA's damaged resources provoke unpredictable fragmentation on the device, ATB is currently the best scheduling option to be used in a Reliable Reconfigurable Real-Time Operating System (R3TOS).

This paper presents a new single event upset (SEU), multiple bit upset (MBU) and single hardware error (SHE) mitigation strategy to be used in Virtex-4 FPGAs. This strategy aims to increase not only the effectiveness of traditional triple module redundancy (TMR), but also the overall system availability. Frame readback with ECC detection and frame scrubbing are combined in a dynamically reconfigurable TMR architecture, designed under both spatial and implementation diversification premises. Moreover, since the strategy works on the device's bitstream domain, the basis for Virtex-4 FPGAs bitstream definition are also shown.