The use of hyperspectral imaging (HSI) for autonomous navigation is a promising field of research that aims at improving the accuracy and robustness of detection, tracking, and scene understanding systems based on vision sensors. The combination of advanced computer algorithms, such as deep neural networks (DNNs), and small-size snapshot HSI cameras allows to strengthen the reliability of those vision systems. Using HSI, some intrinsic limitations of greyscale and RGB imaging in depicting physical properties of targets related to the spectral reflectance of materials (metamerism) are overcome. Despite the promising results of many published HSI-based computer vision developments, the strict requirements of safety-critical applications such as autonomous driving systems (ADS) regarding latency, resource consumption, and security are prompting the migration of machine learning (ML)-based solutions to edge platforms. This involves a thorough software/hardware co-design scheme to distribute and optimize the tasks efficiently among the limited resources of computing platforms. With respect to inference, the over-parameterized nature of DNNs poses significant computational challenges for real-time on-the-edge deployment. In addition, the intensive data preprocessing required by HSI, which is frequently overlooked, must be carefully managed in terms of memory arrangement and inter-task communication to enable an efficient integrated pipeline design on a system on chip (SoC). This work presents a set of optimization techniques for the practical co-design of a DNN-based HSI segmentation processor deployed on a field programmable gate array (FPGA)-based SoC targeted at ADS, including key optimizations such as functional software/hardware task distribution, hardware-aware preprocessing, ML model compression, and a complete pipelined deployment. Applied compression techniques significantly reduce the complexity of the designed DNN to 24.34% of the original operations and to 1.02% of the original number of parameters, achieving a 2.86× speed-up in the inference task without noticeable degradation of the segmentation accuracy.

As the deployment of artificial intelligence (AI) algorithms at edge devices becomes increasingly prevalent, enhancing the robustness and reliability of autonomous AI-based perception and decision systems is becoming as relevant as precision and performance, especially in applications areas considered safety-critical such as autonomous driving and aerospace. This paper delves into the robustness assessment in embedded Deep Neural Networks (DNNs), particularly focusing on the impact of parameter perturbations produced by single event upsets (SEUs) on convolutional neural networks (CNN) for image semantic segmentation. By scrutinizing the layer-by-layer and bit-by-bit sensitivity of various encoder–decoder models to soft errors, this study thoroughly investigates the vulnerability of segmentation DNNs to SEUs and evaluates the consequences of techniques like model pruning and parameter quantization on the robustness of compressed models aimed at embedded implementations. The findings offer valuable insights into the mechanisms underlying SEU-induced failures that allow for evaluating the robustness of DNNs once trained in advance. Moreover, based on the collected data, we propose a set of practical lightweight error mitigation techniques with no memory or computational cost suitable for resource-constrained deployments. The code used to perform the fault injection (FI) campaign is available at https://github.com/jonGuti13/TensorFI2, while the code to implement proposed techniques is available at https://github.com/jonGuti13/parameterProtection.

SiliconBurmuin is aimed at creating a multi-disciplinary neurocomputing community in the Basque Country, bringing together technology and scientific research centres and industry companies. This community will: (1) identify key biological structures and mechanisms that play a major role in vision across species, and (2) transform this knowledge into novel mathematical formalisms, neuromorphic designs and algorithms to solve industry challenges and enable new experiments of interest in neuroscience and clinical research. To achieve the latter objective in a time-effective manner, SiliconBurmuin will draw strong connections with the ongoing Horizon Europe NimbleAI project, with which it shares coordination. This is expected to allow reinforcement of ideas, knowledge and technology via a common prototyping platform where to implement IP from both projects. In addition to describing the research objectives and direction of SiliconBurmuin, this paper posits that co-coordination and co-funding of aligned projects at EU and regional levels might well be a catalyst for raising regional self-awareness of own potential and develop it to help fulfill global challenges, such as semiconductor sovereignty.

Most of current computer vision-based advanced driver assistance systems (ADAS) perform detection and tracking of objects quite successfully under regular conditions. However, under adverse weather and changing lighting conditions, and in complex situations with many overlapping objects, these systems are not completely reliable. The spectral reflectance of the different objects in a driving scene beyond the visible spectrum can offer additional information to increase the reliability of these systems, especially under challenging driving conditions. Furthermore, this information may be significant enough to develop vision systems that allow for a better understanding and interpretation of the whole driving scene. In this work we explore the use of snapshot, video-rate hyperspectral imaging (HSI) cameras in ADAS on the assumption that the near infrared (NIR) spectral reflectance of different materials can help to better segment the objects in real driving scenarios. To do this, we have used the HSI-Drive 1.1 dataset to perform various experiments on spectral classification algorithms. However, the information retrieval of hyperspectral recordings in natural outdoor scenarios is challenging, mainly because of deficient color constancy and other inherent shortcomings of current snapshot HSI technology, which poses some limitations to the development of pure spectral classifiers. In consequence, in this work we analyze to what extent the spatial features codified by standard, tiny fully convolutional network (FCN) models can improve the performance of HSI segmentation systems for ADAS applications. In order to be realistic from an engineering viewpoint, this research is focused on the development of a feasible HSI segmentation system for ADAS, which implies considering implementation constraints and latency specifications throughout the algorithmic development process. For this reason, it is of particular importance to include the study of the raw image preprocessing stage into the data processing pipeline. Accordingly, this paper describes the development and deployment of a complete machine learning-based HSI segmentation system for ADAS, including the characterization of its performance on different embedded computing platforms, including a single board computer, an embedded GPU SoC and a programmable system on chip (PSoC) with embedded FPGA. We verify the superiority of the FPGA-PSoC over the GPU-SoC in terms of energy consumption and, particularly, processing latency, and demonstrate that it is feasible to achieve segmentation speeds within the range of ADAS industry specifications using standard development tools.

Driver Assistance Systems (DAS) have been progressively incorporated into commercial vehicles in recent years. All these systems are paving the way for the forthcoming autonomous vehicle which will become a reality in the near future. Existing systems are based on numerous electronic systems with advanced skills, high performances, and high degrees of adaptability and intelligence. As is to be expected, these cutting-edge features require, in most cases, the use of powerful computing platforms. However, the deployment of such platforms is not an easy task, since they have to be integrated in the vehicle where there exist important restrictions regarding size, power consumption and cost. In this sense, every smart proposal aimed at reducing the complexity of these systems without degrading performance, is always a valuable contribution in the field. In this work, we propose a methodology to reduce the dimensionality of a driver distraction recognition system. The methodology is based on a multi-objective genetic algorithm that looks for the minimum set of useful features collected during the driving task and also for the simplest recognition system. The recognition algorithm is an Extreme Learning Machine (ELM) whose simplicity and fast learning procedure make it especially suitable to be used by a Genetic Algorithm which needs to evaluate thousands of candidate solutions. The proposed methodology has been tested with a real-world database collected from different drivers performing an itinerary with an instrumented car. The results obtained validate the proposal as a method to reduce the complexity of a driver distraction recognition system.

The use of reduced-precision formats is a valuable strategy to improve performance and reduce costs in embedded computing. In case of embedded model predictive control (MPC), utilizing reduced-precision numbers to speed-up underlying optimization algorithms can help to extend the application scope of MPC. In this paper we show how the improved spectral properties of linear systems inside interior point-proximal method of multipliers (IP-PMM) combined with the application of online regularization and instability correction mechanisms, can prevent embedded MPC controllers from failure when reduced-precision arithmetic units are used. Thus, the proposed approach can also contribute to designing efficient domain-specific processors for embedded MPC using custom floating-point formats. To our knowledge this is the first time an IP-PMM algorithm is applied to solve quadratic programming (QP) problems in MPC.

We present a structured dataset for the research and development of automated driving systems (ADS) supported by hyperspectral imaging (HSI). The dataset contains per-pixel manually annotated images selected from videos recorded in real driving conditions that have been organized according to four environment parameters: season, daytime, road type, and weather conditions. The aim is to provide high data diversity and facilitate the automatic generation of data subsets for the evaluation of machine learning (ML) techniques applied to the research of ADS in different driving scenarios and environmental conditions. The video sequences have been captured with a small-size 25-band VNIR (Visible-NearlnfraRed) snapshot hyperspectral camera mounted on a driving automobile. The current selection of classes for image annotation is aimed to provide reliable data for the spectral analysis of the items in the scenes; it is thus based on material surface reflectance patterns (spectral signatures). It is foreseen that future versions of the dataset will also incorporate alternative dense semantic labeling of the annotated images. The first version of the dataset, named HSI-Drive v1.0, is publicly available for download33http://ipaccess.ehu.eus/HSI-Drive.

This paper presents a system-level co-simulation and co-verification workflow to ease the transition from a software-only procedure, executed in a General Purpose processor, to the integration of a custom hardware accelerator developed in a Hardware Description Language (HDL).

The implementation of model predictive controllers on resource-constrained embedded platforms requires fast and computationally efficient processors to solve constrained convex optimization problems. We propose a set of design rules that make use of both, analytical guidelines and numerical simulations to achieve fixed-point implementations of an efficient interior-point (IP) algorithm for solving quadratic-programming (QP) problems. This is the first an necessary step in the development of a complete semiautomatic tool for the implemention of hardware accelerated predictive controllers for embedded and high throughput applications.

Autonomy, adaptability and reactivity are key capabilities of intelligent agents. Many applications of intelligent agents, such as control of ubiquitous computing environments or autonomous robotic systems, demand not only high performance and modeling capability but also the appropriate device or architecture (hardware and related software) for implementing the agent in a real environment. To deal with these challenges a new algorithm, termed PWM-FIS ELM, is proposed. It combines a particular type of adaptive fuzzy inference system with piecewise multi-linear behaviour (PWM-FIS), and an extreme learning machine (ELM) training scheme. The PWM-FIS is suitable for the development of efficient high-performance System-on-Chip (SoC), while ELM provides fast training, good generalization ability, and universal approximation capability. The proposed algorithm outperforms previous results obtained by the authors using the PWM-FIS endowed with a conventional two-pass training algorithm (i.e., least square estimator plus back propagation gradient descent method). Experimental results obtained in an inhabited intelligent environment are provided.

Hyperspectral remote sensing is becoming an active research field in the last decades thanks to the availability of efficient machine learning algorithms and also to the ever-increasing computation power. However, there exist application domains (e.g., embedded applications) in which the deployment of this kind of systems becomes unfeasible due to the high requirements related to the size, power consumption or processing speed. A way to overcome this trouble consists on using any method able to scale-down the dimensionality of the problem and/or to reduce the complexity of the machine learning models. In this paper, we propose the use of a multiobjective genetic algorithm to minimize both the dimension of the input space and the size of the machine learning model. In particular, we have developed a hyperspectral image classifier based on an Extreme Learning Machine (ELM) for which the number of system inputs (dimensionality) and the number of hidden neurons are minimized without decreasing its performance. The system is evaluated by using a known benchmark dataset.

The rapid increase in the use of robotic systems in industrial and domestic environments makes it necessary the development of more natural interaction procedures. This paper presents the development of a user-specific hand Gesture Recognition System (GRS) based on the information of a single tri-axial accelerometer to recognize 7 different dynamic gestures for natural Human Machine Interaction (HMI). The aim of this paper is to analyze and compare different computational methods for feature extraction, dimensionality reduction, and vector classification in order to select the most suitable combination of signal processing stages that meets the performance requirements for a single-chip, wearable GRS system. These requirements are lag-free response, low size, and low power consumption while keeping high recognition accuracy. Experimental results show that the overall achievable accuracy can be up to 98% for Artificial Neural Network (ANN) and Extreme Learning Machine (ELM) predictors, and 99% for Support Vector Machines (SVM).

Singular Value Decomposition (SVD) is a key linear algebraic operation in many scientific and engineering applications. In particular, many computational intelligence systems rely on machine learning methods involving high dimensionality datasets that have to be fast processed for real-time adaptability. In this paper we describe a practical FPGA (Field Programmable Gate Array) implementation of a SVD processor for accelerating the solution of large LSE problems. The design approach has been comprehensive, from the algorithmic refinement to the numerical analysis to the customization for an efficient hardware realization. The processing scheme rests on an adaptive vector rotation evaluator for error regularization that enhances convergence speed with no penalty on the solution accuracy. The proposed architecture, which follows a data transfer scheme, is scalable and based on the interconnection of simple rotations units, which allows for a trade-off between occupied area and processing acceleration in the final implementation. This permits the SVD processor to be implemented both on low-cost and highend FPGAs, according to the final application requirements.

Singular Value Decomposition (SVD) is a key linear algebraic operation in many scientific and engineering applications, many of them involving high dimensionality datasets and real-time response. In this paper we describe a scalable parallel processing architecture for accelerating the SVD of large m × n matrices. Based on a linear array of simple processing-units (PUs), the proposed architecture follows a double data-flow paradigm (FIFO memories and a shared-bus) for optimizing the time spent in data transferences. The PUs, which perform elemental column-pair evaluations and rotations, have been designed for an efficient utilization of available FPGA resources and to achieve maximum algorithm speed-ups. The architecture is fully scalable from a two-PU scheme to an arrangement with as many as n/2 PUs. This allows for a trade-off between occupied area and processing acceleration in the final implementation, and permits the SVD processor to be implemented both on low-cost and high-end FPGAs. The system has been prototyped on Spartan-6 and Kintex-7 devices for performance comparison.

This paper presents the development of an embedded intelligent agent able to perform real-time control of ambient-intelligence environments. The system has been implemented as a system-on-programmable chip (SoPC) on a field programmable gate array (FPGA). The scheme used for realizing the intelligent agent is an adaptive neuro-fuzzy system (NFS) enhanced with a principal component analysis (PCA) pre-processor. The PCA pre-processing stage allows a reduction of the input dimensions (features) with no meaningful loss of modeling capability. As a consequence, the computational complexity of the system is significantly reduced, allowing its implementation on a single electronic device. The NFS-PCA agent has been tested with data obtained in a real ubiquitous computing environment test bed. Results obtained show that the agent is able to perform real-time control of the environment in a proactive and non-intrusive way, and also to adapt to changes of user's preferences in a life-long mode.

In this paper we propose to apply the Dynamic Partial Reconfiguration (DPR) technology to embedded systems intended for Intelligent Environments. To reach this goal, we have developed a system based on a Field Programmable Gate Array (FPGA) in which high performance hardware modules can be reconfigured on-line according to the necessities of the system at each moment. Two different implementations have been carried out to measure the time required to reconfigure each module and also to measure the FPGA resources that can be saved if we keep configured only the modules that are required at each time. The Obtained results show how this technique offers advantages in cost, size and power when applied to embedded systems for intelligent environments.

This paper presents the development of a neuro-fuzzy agent for ambient-intelligence environments. The agent has been implemented as a system-on-chip (SoC) on a reconfigurable device, i.e., a field-programmable gate array. It is a hardware/software (HW/SW) architecture developed around a MicroBlaze processor (SW partition) and a set of parallel intellectual property cores for neuro-fuzzy modeling (HW partition). The SoC is an autonomous electronic device able to perform real-time control of the environment in a personalized and adaptive way, anticipating the desires and needs of its inhabitants. The scheme used to model the intelligent agent is a particular class of an adaptive neuro-fuzzy inference system with piecewise multilinear behavior. The main characteristics of our model are computational efficiency, scalability, and universal approximation capability. Several online experiments have been performed with data obtained in a real ubiquitous computing environment test bed. Results obtained show that the SoC is able to provide high-performance control and adaptation in a life-long mode while retaining the modeling capabilities of similar agent-based approaches implemented on larger computing machines.

An experimental study on the influence of the computation of basic nodal nonlinear functions on the performance of (NFSs) is described in this paper. Systems' architecture size, their approximation capability, and the smoothness of provided mappings are used as performance indexes for this comparative paper. Two widely used kernel functions, the sigmoid-logistic function and the Gaussian function, are analyzed by their computation through an accuracy-controllable approximation algorithm designed for hardware implementation. Two artificial neural network (ANN) paradigms are selected for the analysis: backpropagation neural networks (BPNNs) with one hidden layer and radial basis function (RBF) networks. Extensive simulation of simple benchmark approximation problems is used in order to achieve generalizable conclusions. For the performance analysis of fuzzy systems, a functional equivalence theorem is used to extend obtained results to fuzzy inference systems (FISs). Finally, the adaptive neurofuzzy inference system (ANFIS) paradigm is used to observe the behavior of neurofuzzy systems with learning capabilities.