Abstract Efficient data ingestion and online data augmentation remain challenges in deep learning workflows, particularly when dealing with datasets containing non-standard formats or massive multidimensional arrays that natively optimised functions cannot fully manage. This work presents a parallel framework that integrates and global shared memory through a ring buffer architecture, enabling high-throughput data loading and flexible on-the-fly augmentation. The framework decouples data production from consumption, allowing multiple CPU workers to load and preprocess batches in parallel while completely bypassing the Python GIL and memory bottlenecks. Crucially, the framework supports both CPU-side and GPU-side augmentation strategies, adapting to whether complex conditional transformations or framework-native operations are required. The proposed approach was validated on two representative tasks: (i) sign language recognition from human pose CSV sequences, and (ii) hyperspectral image classification using massive arrays. Relative to standard sequential baselines, the proposed framework achieved up to $$27\times $$ 27 × acceleration in isolated data ingestion and up to $$28\times $$ 28 × in end-to-end training. Importantly, even against natively optimised parallel TensorFlow and PyTorch pipelines, it still delivered up to $$8\times $$ 8 × faster data loading and up to $$7\times $$ 7 × faster full training in memory-intensive scenarios. Overall, the proposed framework provides a scalable, multi-GPU compatible solution for deep learning pipelines, showing robust performance across both I/O-bound and memory-constrained scenarios in TensorFlow and PyTorch while alleviating memory fragmentation and allocation constraints.

Currently, Human Activity Recognition (HAR) applications need a large volume of data to be able to generalize to new users and environments. However, the availability of labeled data is usually limited and the process of recording new data is costly and time-consuming. Synthetically increasing datasets using Generative Adversarial Networks (GANs) has been proposed, outperforming cropping, time-warping, and jittering techniques on raw signals. Incorporating GAN-generated synthetic data into datasets has been demonstrated to improve the accuracy of trained models. Regardless, currently, there is no optimal GAN architecture to generate accelerometry signals, neither a proper evaluation methodology to assess signal quality or accuracy using synthetic data. This work is the first to propose conditional Wasserstein Generative Adversarial Networks (cWGANs) to generate synthetic HAR accelerometry signals. Furthermore, we calculate quality metrics from the literature and study the impact of synthetic data on a large HAR dataset involving 395 users. Results show that i) cWGAN outperforms original Conditional Generative Adversarial Networks (cGANs), being 1D convolutional layers appropriate for generating accelerometry signals, ii) the performance improvement incorporating synthetic data is more significant as the dataset size is smaller, and iii) the quantity of synthetic data required is inversely proportional to the quantity of real data.

Nowadays, smart environments (SEs) enable the monitoring of people with physical disabilities by incorporating activity recognition. Thermal cameras are being incorporated as they preserve privacy. Some deep learning (DL) solutions use the pose of the users because it removes external noise. Although there are robust DL solutions in the visible spectrum (VS), they fail in the thermal domain. Thus, we propose thermal human pose lite (THPoseLite), a convolutional neural network (CNN) based on MobileNetV2 that extracts pose from thermal images (TIs). In a novel way, an auto-labeling approach has been developed. It includes a background removal using an optical flow estimator. It also integrates Blazepose [a pose estimator for VS images (VSIs)] to obtain the poses in the preprocessed TIs. Results show that the preprocessing increases the percentage of detected poses by Blazepose from 19.55% to 76.85%. This allows the recording of human pose estimation (HPE) data sets in the VS without requiring VS cameras or manually annotating data sets. Furthermore, THPoseLite has been embedded in an Internet of Things (IoT) device incorporating an edge tensor processing unit (TPU) accelerator, which can process TIs recorded at 9 frames per second (FPS) in real time (12.28 FPS). It requires fewer than 6W of energy to run. It has been achieved using model quantization, decreasing the accuracy in estimating the poses by only 1%. The mean-squared error of MobileNetV2 in test images is 35.48, obtaining accurate poses in 21% of the images that Blazepose is not able to detect any pose.

One of the main objectives of smart homes is to facilitate daily life by increasing user comfort, with the potential to play a key role in revolutionizing healthcare for the elderly, the disabled and people with functional limitations. To achieve this end, smart homes will have to be able to distinguish the identity of users, their location and the activities they are performing, while also being implemented in a non-invasive way that protects the privacy of these users. Computer vision is one of the main technologies included in smart homes. However, there are drawbacks to traditional cameras, given their dependence on light and privacy-related concerns. Thermal cameras provide a solution, as they operate regardless of light conditions (e.g. at night) while respecting users’ privacy. In this work, image reconstruction and identification of inhabitants from facial images collected by low-resolution thermal sensors has been carried out by using Conditional Generative Adversarial Neural Networks (CGANs). The system has been implemented through an IoT device with raspberry Pi and dual-vision thermal and visible-spectrum sensors installed in a real smart home to automatically collect paired visible-spectrum and thermal images. Thus, different configurations of CGANs have been implemented and analyzed to achieve the following outcomes: (1) inhabitant identification (normal and masked face) with enhanced user privacy and (2) transfer from thermal to color images in the visible spectrum. Results show that the proposed CGAN achieves a recognition rate of 95% and 94% for uncovered and masked faces. This enables user identification without registering accurate facial expressions in the color image reconstruction, protecting user privacy. Furthermore, the developed system outperformed similar approaches using low-resolution datasets and has demonstrated that the accuracy of image reconstruction depends on the resolution of the input visible-spectrum images. In addition, a contribution to high-performance computing has been made by designing a CGAN that runs efficiently on multiple GPUs, achieving increased performance and response speed of the network, as well as applicability to larger problems.

Abstract Generative adversarial networks are gaining importance in problems such as image conversion, cross-domain translation and fast styling. However, the training of these networks remains unclear because it often results in unexpected behavior caused by non-convergence, model collapse or overly long training, causing the training task to have to be supervised by the user and vary with each dataset. To increase the speed of training in Pix2Pix (image-to-image translation) networks, this work incorporates multi-GPU training using mixed precision, along with optimizations in the GPU image input process. In addition, in order to make the training unsupervised and to terminate it when the best transformations are performed, an early stopping method using the peak signal noise ratio (PSNR) metric is proposed.