Event cameras, also known as neuromorphic cameras, are an emerging technology that offer advantages over traditional shutter and frame-based cameras, including high temporal resolution, low power consumption, and selective data acquisition. In this study we harnesses the capabilities of event-based cameras to capture subtle changes in the surface of the skin caused by the pulsatile flow of blood in the wrist region. We show how an event camera can be used for continuous non-invasive monitoring of heart rate (HR). Event camera video data from 25 participants with varying age groups and skin colours, was collected and analysed. Ground-truth HR measurements were used to evaluate of the accuracy of automatic detection of HR from event camera data. Our results demonstrate the feasibility of using event cameras for HR detection.

Distracted driving is a major cause of traffic accidents. Drivers undergoing cognitive and physical stress are not only factors that distract drivers from the task of driving but can also affect a person's long-term health. This work investigates the effect of distracted driving on body vital signs, specifically on breathing rate. Details of a recent experiment where subjects are asked to drive in a simulator while also being asked to perform various cognitively demanding or stressful tasks are provided. As part of a larger study on driver distraction, fatigue, and cognitive load, the primary focus of this work is on identifying a correlation between breathing rate variability during different stages of the experiment and examining how this change in breathing rate may differ between participants. Some of the tasks included in the data acquisition were participants being exposed to white noise and random news clips asked to answer questions or using their smartphones to look for directions. Other tasks included driving without distraction to provide a baseline cognitive level. Analysis of this data using visualization showed a clear correlation between breathing rate variability and cognitive demand of the experiment tasks. Thus, this study emphasizes the importance of non-contact driver cognitive load monitoring to help reduce accidents and increase road safety. This paper presents a method of analysis and is the first documentation of this data acquisition presented in the literature.

Event cameras also known as neuromorphic sensors are relatively a new technology with some privilege over the RGB cameras. The most important one is their difference in capturing the light changes in the environment, each pixel changes independently from the others when it captures a change in the environment light. To increase the user’s degree of freedom in controlling the output of these cameras, such as changing the sensitivity of the sensor to light changes, controlling the number of generated events and other similar operations, the camera manufacturers usually introduce some tools to make sensor level changes in camera settings. The contribution of this research is to examine and document the effects of changing the sensor settings on the sharpness as an indicator of quality of the generated stream of event data. To have a qualitative understanding this stream of event is converted to frames, then the average image gradient magnitude as an index of the number of edges and accordingly sharpness is calculated for these frames. Five different bias settings are explained and the effect of their change in the event output is surveyed and analyzed. In addition, the operation of the event camera sensing array is explained with an analogue circuit model and the functions of the bias foundations are linked with this model.

Driver monitoring systems (DMS) are a key component of vehicular safety and essential for the transition from semiautonomous to fully autonomous driving. A key task for DMS is to ascertain the cognitive state of a driver and to determine their level of tiredness. Neuromorphic vision systems, based on event camera technology, provide advanced sensing of facial characteristics, in particular the behavior of a driver’s eyes. This research explores the potential to extend neuromorphic sensing techniques to analyze the entire facial region, detecting yawning behaviors that give a complimentary indicator of tiredness. A neuromorphic dataset is constructed from 952 video clips (481 yawns, 471 not-yawns) captured with an RGB colour camera, with 37 subjects. A total of 95,200 neuromorphic image frames are generated from this video data using a video-to-event converter. From these data 21 subjects were selected to provide a training dataset, 8 subjects were used for validation data, and the remaining 8 subjects were reserved for an ‘unseen’ test dataset. An additional 12,300 frames were generated from event simulations of a public dataset to test against other methods. A convolutional neural network (CNN) with self-attention and a recurrent head was trained and tested with these data. Respective precision and recall scores of 95.9% and 94.7% were achieved on our test set, and 89.9% and 91% on the simulated public test set, demonstrating the feasibility to add yawn detection as a sensing component of a neuromorphic DMS.

Heart rate is one of the most vital health metrics which can be utilized to investigate and gain intuitions into various human physiological and psychological information.Estimating heart rate without the constraints of contact-based sensors thus presents itself as a very attractive field of research as it enables well-being monitoring in a wider variety of scenarios.Consequently, various techniques for camera-based heart rate estimation have been developed ranging from classical image processing to convoluted deep learning models and architectures.At the heart of such research efforts lies health and visual data acquisition, cleaning, transformation, and annotation.In this paper, we discuss how to prepare data for the task of developing or testing an algorithm or machine learning model for heart rate estimation from images of facial regions.The data prepared is to include camera frames as well as sensor readings from an electrocardiograph sensor.The proposed pipeline is divided into four main steps, namely removal of faulty data, frame and electrocardiograph timestamp dejittering, signal denoising and filtering, and frame annotation creation.Our main contributions are a novel technique of eliminating jitter from health sensor and camera timestamps and a method to accurately time align both visual frame and electrocardiogram sensor data which is also applicable to other sensor types.

Driver stress is a major cause of car accidents and death worldwide. Furthermore, persistent stress is a health problem, contributing to hypertension and other diseases of the cardiovascular system. Stress has a measurable impact on heart and breathing rates and stress levels can be inferred from such measurements. Galvanic skin response is a common test to measure the perspiration caused by both physiological and psychological stress, as well as extreme emotions. In this paper, galvanic skin response is used to estimate the ground truth stress levels. A feature selection technique based on the minimal redundancy-maximal relevance method is then applied to multiple heart rate variability and breathing rate metrics to identify a novel and optimal combination for use in detecting stress. The support vector machine algorithm with a radial basis function kernel was used along with these features to reliably predict stress. The proposed method has achieved a high level of accuracy on the target dataset.

Neuromorphic vision or event vision is an advanced vision technology, where in contrast to visible camera sensors that output pixels, the event vision generates neuromorphic events every time there’s a brightness change which exceeds a specific threshold in the field of view (FoV). This study focuses on leveraging neuromorphic event data for roadside object detection. This is a proof of concept towards building artificial intelligence (AI) based imaging pipelines which can be used for forward perception systems for advanced vehicular applications. The focus is on building efficient stateof- the-art object detection networks with better inference results for fast-moving forward perception using an event camera. In this article, the event simulated A2D2 dataset is manually annotated and trained on two different YOLOv5 networks (small and large variants). To further assess its robustness, single model testing and ensemble model testing are carried out.

Depth estimation from a single image frame is a fundamental challenge in computer vision, with many applications such as augmented reality, action recognition, image understanding, and autonomous driving. Large and diverse training sets are required for accurate depth estimation from a single image frame. Due to challenges in obtaining dense ground-truth depth, a new 3D pipeline of 100 synthetic virtual human models is presented to generate multiple 2D facial images and corresponding ground truth depth data, allowing complete control over image variations. To validate the synthetic facial depth data, we propose an evaluation of state-of-the-art depth estimation algorithms based on single image frames on the generated synthetic dataset. Furthermore, an improved encoder-decoder based neural network is presented. This network is computationally efficient and shows better performance than current state-of-the-art when tested and evaluated across 4 public datasets. Our training methodology relies on the use of synthetic data samples which provides a more reliable ground truth for depth estimation. Additionally, using a combination of appropriate loss functions leads to improved performance than the current state-of-the-art network performances. Our approach clearly outperforms competing methods across different test datasets, setting a new state-of-the-art for facial depth estimation from synthetic data.

Thermal infrared imaging holds promise for human-machine interaction in vehicles owing to superior performance in low-light and low-visibility conditions, and the potential for monitoring human psycho-physiological state. However, the shortage of large-scale 2D thermal image datasets and public benchmarks has hindered progress of deep-learning-based solutions. To tackle this problem, we develop a pipeline for creating a synthetic thermal image dataset. Firstly, 3D models of human heads are generated from uncalibrated TIR images (without additional visible or depth images) using photogrammetry techniques. A synthetic dataset of 100k images of 640×480 resolution are then generated by rendering each of the five 3D models for a range of head poses, camera positions and backgrounds using commercial animation software. The effectiveness of the approach is evaluated using a number of deep learning algorithms that may enable human-machine interaction such as head pose estimation and face detection. The neural networks are trained on the new synthetic thermal dataset, before fine tuning on real world data where possible.

We present the first algorithm for finding holes in high dimensional data that runs in polynomial time with respect to the number of dimensions. Previous algorithms are exponential. Finding large empty rectangles or boxes in a set of points in 2D and 3D space has been well studied. Efficient algorithms exist to identify the empty regions in these low-dimensional spaces. Unfortunately such efficiency is lacking in higher dimensions where the problem has been shown to be NP-complete when the dimensions are included in the input. Applications for algorithms that find large empty spaces include big data analysis, recommender systems, automated knowledge discovery, and query optimization. Our Monte Carlo-based algorithm discovers interesting maximal empty hyper-rectangles in cases where dimensionality and input size would otherwise make analysis impractical. The run-time is polynomial in the size of the input and the number of dimensions. We apply the algorithm on a 39-dimensional data set for protein structures and discover interesting properties that we think could not be inferred otherwise.

We address the problem of optimizing the distribution of partially replicated databases over a computer network. Replication is used to increase data availability in the presence of site or communication failures and to decrease retrieval costs by local access if possible. We present a new bio-inspired replication management approach which is adaptive, completely decentralized, and based on swarm intelligence. Each node has the autonomy to start at any time, depending on the internal state of its stored data objects, a redistribution process. "Redistribution" means replicate, create, delete, update, or move data objects to other nodes of the network. The redistribution process is a dynamic load-balancing scheme which runs with lower priority in the background. The system is event-driven, but the learning process is not synchronized with the events.