This paper provides an overview of the ongoing work to enable novel modalities of passive monocular neuromorphic vision in the NimbleAI sensing-processing architecture; namely, foveated and light-field event-driven vision with selective visual attention. The latter vision modality encodes 3D visual surroundings as sparse visual events in a 4D spatiotemporal domain, adding depth to current representation of visual information delivered by Dynamic Vision Sensors (DVS). The NimbleAI architecture implements hardware support for efficient execution of mainstream computer vision algorithms and AI models using these visual inputs. The architecture is designed to harness the latest advancements in 3D silicon integration, making it possible to squeeze sensing and spiking circuitry, memory, and processing engines into a miniature silicon volume.

The NimbleAI Horizon Europe project leverages key principles of energy-efficient visual sensing and processing in biological eyes and brains, and harnesses the latest advances in$\mathbf{33D}$stacked silicon integration, to create an integral sensing-processing neuromorphic architecture that efficiently and accurately runs computer vision algorithms in area-constrained endpoint chips. The rationale behind the NimbleAI architecture is: sense data only with high information value and discard data as soon as they are found not to be useful for the application (in a given context). The NimbleAI sensing-processing architecture is to be specialized after-deployment by tunning system-level trade-offs for each particular computer vision algorithm and deployment environment. The objectives of NimbleAI are: (1)$\mathbf{100x}$performance per mW gains compared to state-of-the-practice solutions (i.e., CPU/GPUs processing frame-based video); (2)$\mathbf{50x}$processing latency reduction compared to CPU/GPUs; (3) energy consumption in the order of tens of mWs; and (4) silicon area of approx. 50 mm2.

Many systems interact with their physical environment. Design of such systems need a modelling and simulation tool which can deal with both the continuous and discrete aspects. However, most current tools are not adequately able to do so, as they implement both continuous and discrete time signals as consisting of separate values at a single global simulation clock. The consequence is that simulation, of a time delay for example, either yields inaccurate results or becomes inefficient.

Until a decade ago, the concept of phased array beam forming was mainly implemented with mechanical or analog solutions. Today, digital hardware has become powerful enough to perform the massive number of operations required for real-time digital beam forming. While more and more applications are using beam forming to improve the communication channel utilization both in space and frequency, many dedicated digital architectures are proposed for the processing. By using a reconfigurable architecture, the same hardware platform can be reused for different applications with different processing needs. In this paper, we present a reconfigurable Multi-processor System-on-Chip based solution for phased array processing that supports advanced tracking mechanisms to continuously receive signals with a mobile receiver. An adaptive beam former for DVB-S satellite reception is presented, that uses a Constant Modulus Algorithm to track satellites. The processing of a receiver with 64 antennas and 3 beams is mapped on a reconfigurable processor named Montium TP. The total implementation of such a receiver requires about 570 clock cycles on a single Montium TP, but can also be partitioned over multiple Montium TPs to support larger phased arrays.

A system that uses adaptive beamforming techniques for mobile DVB-S reception is proposed in this paper. The purpose is to enable DVB-S reception in moving vehicles. Phased arrays are able to electronically track the desired signal during dynamic behaviour of the vehicle the array is mounted on. The proposed system uses blind beamforming to adapt the array steering vector to changing signal (conditions and) directions. Movement of the vehicle, the phased array is mounted on, leads to modulus and phase deviations at the beamformer output. An extended version of the CMA algorithm is used to adapt the steering vector weights to compensate for those deviations. For simulation of the proposed system a model of vehicle dynamics is used to generate realistic antenna data. Simulation of the proposed system based on this antenna data shows appropriate corrections for modulus and phase deviations.