Accurate human body keypoint detection is crucial in fields like medicine, entertainment, and VR. However, it often demands complex neural networks best suited for high-compute environments. This work instead presents a keypoint detection approach targeting embedded devices with very low computational resources, such as microcontrollers. The proposed end-to-end solution is based on the development and optimization of each component of a neural network specifically designed for highly constrained devices. Our methodology works top-down, from object to keypoint detection, unlike alternative bottom-up approaches relying instead on complex decoding algorithms or additional processing steps. The proposed network is optimized to ensure maximum compatibility with different embedded runtimes by making use of commonly used operators. We demonstrate the viability of our approach using an STM32H7 microcontroller with 2MB of Flash and 1MB of RAM. We achieve a maximum mAP of 57.9 without relying on external RAM, and good detection performance at latencies down to 133ms per frame.

The unprecedented development of deep learning approaches for video processing has caused growing privacy concerns. To ensure data analysis while maintaining privacy, it is essential to address how to protect individuals’ identities. One solution is to anonymize data at the source, avoiding the transmission or storage of information that could lead to identification. This study introduces XimSwap, a novel deep learning technique for real-time video anonymization, which can remove facial identification features directly on edge devices with minimal computational resources. Our approach offers a comprehensive solution that guarantees privacy by design. This novel method for implementing face-swapping ensures that the pose and expression of a target face remain unchanged and can be used on embedded devices with very limited computational resources. By incorporating style transfer layers into convolutional ones and optimizing the network’s operation, we achieved a reduction of over 98% in the required operations and parameters compared with state-of-the-art architectures. Our approach also significantly reduces RAM usage, making it possible to implement the anonymization process on tiny edge devices, including microcontrollers, such as the STM32H743.

Electroencephalography (EEG) provides unique insights into natural brain dynamics outside the laboratory setting. However, its usability is limited due to the presence of artifacts. Artifact Subspace Reconstruction (ASR) has been a popular method for enhancing the signal-to-noise ratio (SNR) in mobile EEG; nonetheless, its complexity restricts its applicability on lightweight, resource-constrained EEG devices. To address this challenge, we propose an innovative IMU-integrated approach for artifacts correction (IMU-ASR). Specifically, we replace ASR’s time-consuming calibration process with a simpler accelerometer-based method, significantly reducing computational time without compromising performance. We validate our approach on two publicly available datasets, one with low-density (8 channels) and the other with high-density (120 channels) EEG. Our findings demonstrate the potential of accelerometer-driven ASR for lightweight hardware-software EEG solutions, promising a more practical and efficient approach for artifact correction in mobile EEG applications.

The recent interest in the edge-to-cloud continuum paradigm has emphasized the need for simple and scalable architectures to deliver optimal performance on computationally constrained devices. However, resource-efficient neural networks usually optimize for parameter count and thus use operators such as depthwise convolutions, which do not maximally exploit the efficiency of resource-constrained devices. In this article, we propose XiNet, a novel convolutional neural architecture that targets edge devices. We derived the XiNet architecture from an extensive real-world efficiency analysis of various neural network operators (e.g., standard, depthwise, and pointwise convolutions). Compared to other mobile architectures, our approach substantially improves the performance-complexity trade-off by optimizing the number of operations, parameters, and working memory (RAM). Moreover, we show how XiNet can be easily adapted to different devices thanks to Hardware Aware Scaling (HAS), which enables disjoint optimization of RAM, FLASH, and operations count. We analyze the scaling properties of our architecture under different hardware constraints and validate it on the image classification task. Finally, we evaluate the performance of XiNet for object detection on the MS-COCO and VOC-2012 benchmarks and compare it with state-of-the-art mobile neural networks, achieving a 70% reduction in energy requirements with similar performance.

Sound Event Detection (SED) is a complex task simulating human ability to recognize what is happening in the surrounding from auditory signals only. This technology is a crucial asset in many applications such as smart cities. Here, urban sounds can be detected and processed by embedded devices in an Internet of Things (IoT) to identify meaningful events for municipalities or law enforcement. However, while current deep learning techniques for SED are effective, they are also resource- and power-hungry, thus not appropriate for pervasive battery-powered devices. In this paper, we propose novel neural architectures based on PhiNets for real-time acoustic event detection on microcontroller units. The proposed models are easily scalable to fit the hardware requirements and can operate both on spectrograms and waveforms. In particular, our architectures achieve state-of-the-art performance on UrbanSound8K in spectrogram classification (around 77%) with extreme compression factors (99.8%) with respect to current state-of-the-art architectures.

Keyword spotting (KWS) is a crucial function enabling the interaction with the many ubiquitous smart devices in our surroundings, either activating them through wake-word or directly as a human-computer interface. For many applications, KWS is the entry point for our interactions with the device and, thus, an always-on workload. Many smart devices are mobile and their battery lifetime is heavily impacted by continuously running services. KWS and similar always-on services are thus the focus when optimizing the overall power consumption. This work addresses KWS energy-efficiency on low-cost microcontroller units (MCUs). We combine analog binary feature extraction with binary neural networks. By replacing the digital preprocessing with the proposed analog front-end, we show that the energy required for data acquisition and preprocessing can be reduced by$29\times $, cutting its share from a dominating 85% to a mere 16% of the overall energy consumption for our reference KWS application. Experimental evaluations on the Speech Commands Dataset show that the proposed system outperforms state-of-the-art accuracy and energy efficiency, respectively, by 1% and$4.3\times $on a 10-class dataset while providing a compelling accuracy-energy trade-off including a 2% accuracy drop for a$71\times $energy reduction.

In the Internet of Things era, where we see many interconnected and heterogeneous mobile and fixed smart devices, distributing the intelligence from the cloud to the edge has become a necessity. Due to limited computational and communication capabilities, low memory and limited energy budget, bringing artificial intelligence algorithms to peripheral devices, such as end-nodes of a sensor network, is a challenging task and requires the design of innovative solutions. In this work, we present PhiNets , a new scalable backbone optimized for deep-learning-based image processing on resource-constrained platforms. PhiNets are based on inverted residual blocks specifically designed to decouple the computational cost, working memory, and parameter memory, thus exploiting all available resources for a given platform. With a YoloV2 detection head and Simple Online and Realtime Tracking (SORT), the proposed architecture achieves state-of-the-art results in (i) detection on the COCO and VOC2012 benchmarks, and (ii) tracking on the MOT15 benchmark. PhiNets obtain a reduction in parameter count of around 90% with respect to previous state-of-the-art models (EfficientNetv1, MobileNetv2) and achieve better performance with lower computational cost. Moreover, we demonstrate our approach on a prototype node based on an STM32H743 microcontroller (MCU) with 2 MB of internal Flash and 1MB of RAM and achieve power requirements in the order of 10 mW. The code for the PhiNets is publicly available on GitHub. 1

Artifacts preprocessing in EEG is remarkably significant to extract reliable neural responses in the downstream analysis. A recently emerging powerful preprocessing tool among the EEG community is Artifacts Subspace Reconstruction (ASR). ASR is an unsupervised machine learning algorithm to identify and correct the transient-like non-stationary noisy samples. ASR is fully automatic, therefore, suitable for online applications. However, the performance of ASR is strongly dependent on the user-defined hyperparameter k. A poor choice of k might lead to severe performance degradation.In this work, we benchmark the performance of ASR against its parameter k. Toward this goal, we used the Temple University Hospital EEG Artifact Corpus (TUAR), which consists of 310 EEG files recorded in clinical settings from epileptic patients. Remarkably, these files are annotated for artifacts by trained personnel with a high inter-rater agreement score (κ > 0.8). Considering these reliable labels as ground truth, ASR has shown the best performance in artifacts cleaning with k ranging between 20 and 40.

Sound event detection (SED) is a hot topic in consumer and smart city applications. Existing approaches based on deep neural networks (DNNs) are very effective, but highly demanding in terms of memory, power, and throughput when targeting ultra-low power always-on devices.

Technology increasingly offers parents more and more opportunities to monitor children, reshaping the way control and autonomy are negotiated within families. This paper investigates the views of parents and primary school children on mobile technology designed to support child independent mobility in the context of the local walking school buses. Based on a school-year long field study, we report findings on children's and parents' experience with proximity detection devices. The results provide insights into how the parents and children accepted and socially appropriated the technology into the walking school bus activity, shedding light on the way they understand and conceptualize a technology that collects data on children's proximity to the volunteers' smartphone. We discuss parents' needs and concerns toward monitoring technologies and the related challenges in terms of trust-control balance. These insights are elaborated to inform the future design of technology for child independent mobility.

Detection of human presence is a key feature in Human Computer Interaction. Solutions based on cameras are attractive, but require computer vision techniques to extract meaningful data, which can be expensive from a computational point of view. In this work, we present a new system that merges a low resolution thermal camera with advanced feature extraction techniques such as Convolutional Neural Networks. We demonstrate the possibility to adapt their execution to resource-constrained platform without significant loss of performance, by processing data on a 32-bit low power microcontroller, performing the classification on thermal video stream. It achieve 76.7% of accuracy in the microcontroller, requiring only 16.5 mW in continuous classification mode and using 6 kB of RAM.

In most classification tasks, wide and deep neural networks perform and generalize better than their smaller counterparts, in particular when they are exposed to large and heterogeneous training sets. However, in the emerging field of Internet of Things memory footprint and energy budget pose severe limits on the size and complexity of the neural models that can be implemented on embedded devices. The Student-Teacher approach is an attractive strategy to distill knowledge from a large network into smaller ones, that can fit on low-energy low-complexity embedded IoT platforms. In this paper, we consider the outdoor sound event detection task as a use case. Building upon the VGGish network, we investigate different distillation strategies to substantially reduce the classifier's size and computational cost with minimal performance losses. Experiments on the UrbanSound8K dataset show that extreme compression factors (up to 4.2 · 10−4 for parameters and 1.2 · 10−3 for operations with respect to VGGish) can be achieved, limiting the accuracy degradation from 75% to 70%. Finally, we compare different embedded platforms to analyze the trade-off between available resources and achievable accuracy.

This work introduces an ultra-low-power visual sensor node coupling event-based binary acquisition with Binarized Neural Networks (BNNs) to deal with the stringent power requirements of always-on vision systems for IoT applications. By exploiting in-sensor mixed-signal processing, an ultra-low-power imager generates a sparse visual signal of binary spatial-gradient features. The sensor output, packed as a stream of events corresponding to the asserted gradient binary values, is transferred to a 4-core processor when the amount of data detected after frame difference surpasses a given threshold. Then, a BNN trained with binary gradients as input runs on the parallel processor if a meaningful activity is detected in a pre-processing stage. During the BNN computation, the proposed Event-based Binarized Neural Network model achieves a system energy saving of 17.8% with respect to a baseline system including a low-power RGB imager and a Binarized Neural Network, while paying a classification performance drop of only 3% for a real-life 3-classes classification scenario. The energy reduction increases up to 8x when considering a long-term always-on monitoring scenario, thanks to the event-driven behavior of the processing sub-system.

After several decades of both market and scientific interest, indoor positioning is still a hot and not completely solved topic, fostered by the advancement of technology, pervasive market penetration of mobile devices and novel communication standards. In this work, we propose a two-step model-based indoor positioning algorithm based on Bluetooth Low-Energy, a pervasive and energy efficient standard protocol. In the first (i.e. ranging) step a Kalman Filter (KF) performs the fusion of both RSSI and Time-of-Flight measurement data. Thus, we demonstrate the benefit of not relying only on RSSI, comparing ranging performed with or without the help of ToF. In the second (i.e. positioning) step, the distance estimates from multiple anchors are combined into a quadratic cost function, which is minimized to determine the coordinates of the target node in a planar reference frame. The proposed solution is tailored to reduce the computational effort and target real-time execution on an embedded platform, demonstrating a limited loss of performance. The paper presents an experimental setup and discusses meaningful results, demonstrating a robust BLE-based indoor positioning solution for embedded systems.

Wearable technologies are changing the way we deal with health and fitness in our daily life. Nevertheless, while MEMS-enabled inertial sensors have conquered the consumer market, physiological monitoring has still to face barriers due to the complexity and costs of physical interfaces (e.g. electrodes), the degree of intuitiveness of the interaction and the processing required to reach satisfying performance. These limitations are mitigated by the embedded systems' growing integration of interfacing capabilities and efficient computing power. In this paper, we describe the main applications and the related technologies for the acquisition and processing of myoelectric (EMG) signals. Starting from well established active sensors and bench-top setups, we introduce a recent design based on the combination of an integrated Analog Front End (AFE) and embedded processing. This solution provides high quality signal acquisition and on-board digital processing capabilities with a contained power consumption. The system was tested within the prosthesis control application scenario, one of the most stringent EMG applications, achieving a 90% gesture recognition accuracy with real time on-board processing at a power consumption of 30 mW. Such promising results highlight the current trend in shifting EMG applications from dedicated analog solutions towards integrated digital devices, favouring the development of advanced, modular and low-power wearable solutions.

Wake-up radios (WuRs) are usually designed as a secondary near-zero power radio receiver used to trigger the main radio when a new communication is started over the air. Despite the tight relation with the main radio functions, it is usually implemented with separate hardware, firmware, and it is far from being integrated in the most diffused standard wireless protocols for sensors networks and for the Internet of Things (IoT). In this work, we want to check in-deep the coexistence between the Bluetooth Low Energy (BLE), which is one of the most used low-power short-range wireless standards for mobile and wearable communication, and a wake-up facility implemented in the same band. We focus on the analysis of the limits, advantages and drawbacks of adding this interesting feature in the BLE standard, and we discuss about the improvements in terms of power saving and latency of the communication. We target IoT applications and scenarios with a high density of BLE devices, analyzing the standard protocol and how WuRs can enhance its performance. Two common communication schemes will help us to show the limits and to define a design methodology for the application. Simulation results will show the trade-offs of the communication performance and the difference from a normal usage of the BLE. Finally, considerations about the usability of BLE with WuRs and guidelines will be provided as concluding remarks.

This paper presents a fully programmable Internet of Things visual sensing node that targets sub-mW power consumption in always-on monitoring scenarios. The system features a spatial-contrast 128 × 64 binary pixel imager with focal-plane processing. The sensor, when working at its lowest power mode (10 μW at 10 frames/s), provides as output the number of changed pixels. Based on this information, a dedicated camera interface, implemented on a low-power field-programmable gate array, wakes up an ultralow-power parallel processing unit to extract context-aware visual information. We evaluate the smart sensor on three always-on visual triggering application scenarios. Triggering accuracy comparable to RGB image sensors is achieved at nominal lighting conditions, while consuming an average power between 193 and 277 μW, depending on context activity. The digital subsystem is extremely flexible, thanks to a fully programmable digital signal processing engine, but still achieves 19× lower power consumption compared to MCU-based cameras with significantly lower on-board computing capabilities.

Wireless body area networks (WBANs) have the huge potential to enhance people's lives. They are already present in many application domains, for instance sport and fitness, but they are wide spreading in particular in health and rehabilitation. However, there are still challenging issues that limit their wide diffusion in real life: primarily, the limited lifetime due to the batteries that usually supply the devices. This limitation affects usability and force the data processing to be simple to match the power constraints. This work tries to address the energy limitation by enabling both efficient and complex signal-processing applications and extension of lifetime. We present a power management strategy combining an ultra-low power wake up radio with context awareness. The context aware power manager based on activity recognition decides which nodes must be activated exploiting a nano-power wake up radio and power management policies. Result shows that by using both approaches it is possible to extend battery life of sensor nodes from few hours to an entire week.

Body Area Networks (BANs) are widely used mainly for healthcare and fitness purposes. In both cases, the lifetime of sensor nodes included in the BAN is a key aspect that may affect the functionality of the whole system. Typical approaches to power management are based on a trade-off between the data rate and the monitoring time. Our work introduces a power management layer capable to opportunistically use data sampled by sensors to detect contextual information such as user activity and adapt the node operating point accordingly. The use of this layer has been demonstrated on a commercial sensor node, increasing its battery lifetime up to a factor of 5.

Mobile and wearable embedded devices connect the user with digital information in a continuous and pervasive way. A key benefit is given by the possibility to exploit multi-modal interaction capabilities that can dynamically act on different human senses and the cooperative capabilities of the small and pervasive devices.

Wireless Body Area Networks (WBANs) can take advantage of many wireless protocols. Among them, Bluetooth is a good candidate since its widespread adoption guarantees compatibility with a number of devices and significantly reduces development time. In most cases data collected from different sensors on different nodes need to be synchronized. We present a synchronization protocol that makes use of Bluetooth piconet internal clock to achieve near-millisecond accuracy with minimal radio communication overhead. Experimental results show that Bluetooth low power modes does not affect negatively accuracy, but improves it, obtaining less power consumption and higher synchronization accuracy.

A key factor in a successful sensor network deployment is finding a good balance between maximizing the number of measurements taken (to maintain a good sampling rate) and minimizing the overall energy consumption (to extend the network lifetime). In this work, we present a data-driven statistical model to optimize this tradeoff. Our approach takes advantage of the multivariate nature of the data collected by a heterogeneous sensor network to learn spatio-temporal patterns. These patterns enable us to employ an aggressive duty cycling policy on the individual sensor nodes, thereby reducing the overall energy consumption. Our experiments with the OMNeT++ network simulator using realistic wireless channel conditions, on data collected from two real-world sensor networks, show that we can sample just 20% of the data and can reconstruct the remaining 80% of the data with less than 9% mean error, outperforming similar techniques such is distributed compressive sampling. In addition, energy savings ranging up to 76%, depending on the sampling rate and the hardware configuration of the node.

Wearable gesture recognition enables context aware applications and unobtrusive HCI. It is realized by applying machine learning techniques to data from on-body sensor nodes. We present an gesture recognition system minimizing power while maintaining a run-time application defined performance target through dynamic sensor selection. Compared to the non managed approach optimized for recognition accuracy (95% accuracy), our technique can extend network lifetime by 4 times with accuracy >90% and by 9 times with accuracy >70%. We characterize the approach and outline its applicability to other scenarios.

The recent advances in microelectronics and related fields have made the dream of intelligent spaces and objects come true. Reduction of technology costs, power consumption and form factor is enabling to exploit increased processing capabilities of sensor nodes and to distribute the intelligence in the environment. In this paper, significant examples of on-going research on smart environments, sensorized tangible interfaces applied to smart spaces and assistive technologies are presented. Each of these examples outlines some of the AmI challenges, such as, in particular, space awareness, smart space flexibility to include new smart entities and natural interfaces.

Pyroelectric sensors are low-cost, low-power small components commonly used only to trigger alarm in presence of humans or moving objects. However, the use of an array of pyroelectric sensors can lead to extraction of more features such as direction of movements, speed, number of people and other characteristics. In this work a low-cost pyroelectric infrared sensor based wireless network is set up to be used for tracking people motion. A novel technique is proposed to distinguish the direction of movement and the number of people passing. The approach has low computational requirements, therefore it is well-suited to limited-resources devices such as wireless nodes. Tests performed gave promising results.

In this paper we describe TANGerINE, a tangible tabletop environment in which users can interact with digital contents manipulating tangible smart objects. Such objects provide continuous data about their status through the embedded wireless sensors, while an overhead computer vision module tracks their position and orientation. Merging sensing data, the system is able to detect a richer language of gestures and manipulations both on the tabletop and in its surroundings, enabling for a more expressive interaction language across different contexts.

Body Area Sensor Networks (BASN) are an emerging technology enabling the design of natural Human Computer Interfaces (HCI) in the context of Ambient Intelligence. This class of interactive applications poses new challenges on sensor network design that are hard to be faced using traditional solutions optimized for environmental monitoringlike applications. In this paper we present a novel solution for wireless and wearable posture recognition based on a custom-designed wireless body area sensor network, called WiMoCA. Nodes of the network, mounted on different parts of the human body, exploit tri-axial accelerometers to detect body postures. Afterwards we discuss results of interactive performance and power consumption optimizations required to match application constraints.

Motion capture is an emerging technology enabling the design of natural user interfaces for wearable devices based on gestural recognition. However, costs and energy requirements are critical factors to enable their diffusion to low-end wearable systems. Current commercial products do not match these requirements. For this reason, we developed a low-cost/low-power wearable motion tracking system, based on integrated accelerometers, called MOCA (motion capture with accelerometers). Our system is composed of sensing units connected to a control/acquisition board, responsible for reading and preprocessing data, and a mobile terminal running the recognition algorithm. Experiments performed to validate accuracy, power consumption and real-time performance demonstrate low-power and flexibility features of the proposed tracking system as well as its effectiveness as an input interface.