Benefiting from smartphones' powerful computing and sensing capabilities, biometric authentication is widely applied to them for conveniently verifying users' identities. However, most biometric features can be easily acquired or reproduced, making them vulnerable to replay and impersonation attacks. To address this issue, we propose DBreathLock, a non-contact deep breath-based authentication system that utilizes a smartphone emitting inaudible frequency-modulated continuous waves (FMCW)-based sonar signals and synchronously records breath sounds and sonar echoes of chest-abdominal-joint (C-A-joint) movements. Then, we implement a dual-protection barrier to defend against advanced replay attacks (ARAs). First, by analyzing the energy features of C-A-joint movements, we develop a Deep Breath Activity Detection method to detect deep breath fragments alongside the capability of resisting ARAs. Second, we take C-A-joint movements and smartphone vibrations caused by holding a smartphone as features and design a liveness detection mechanism to further fortify the resistance to ARAs. Furthermore, a multi-stream identity authentication model is designed to verify legitimate users by fusing biometric features from C-A-joint movements, deep breath sounds, and correlation sequences of both. Extensive real-world experiments with 40 users demonstrate DBreathLock's authentication accuracy of 95.97%. Additionally, it successfully defends against advanced replay, impersonation, simple hybrid, and advanced hybrid attacks, achieving the AUC of 0.9792, and FPRs of 2.17%, 2%, and 4.17%, respectively.

Frequency Modulated Continuous Wave (FMCW)-based mmWave radar has attracted widespread attention because of its non-contact and high spatial resolution for vital signs monitoring. Meanwhile, current studies focus mainly on how to improve the detection performance of steady multiple objects or unsteady single objects. In this work, we propose an innovative method for identity-based multi-object vital signs monitoring under unsteady scenarios. The method automatically distinguishes between steady and motion states, and conducts a best-effort vital signs monitoring during unsteady scenarios. To this end, we design a weight vector enhancement method combined with object spatial positioning for differentiating multiple objects, and identify each object according to the gait-based EfficientNet model. We also design a steady-state detector based on the MobileNet-V2 network to find the slots of object keeping steady for vital signs monitoring and then apply the variational mode decomposition (VMD) algorithm to extract the respiratory and heart rates of a single object. The experimental results showed that the mean absolute error of respiratory rate and heart rate decreased to 1.37 bpm and 2.56 bpm respectively in the case of multiple objects. In addition, the steady-state detector achieves close to 98.1% accuracy in recognizing motion types, and the average recognition rate of identity recognition based on gait features reaches about 93.26%.

Frequency Modulated Continuous Wave (FMCW)-based mmWave radar has attracted widespread attention due to its non-contact and high spatial resolution for multi-target vital signs monitoring. However, current works mostly focus on how to improve detection performance under the steady scenarios with a single target, while unsteady scenarios and physical mutual interference from multiple targets are rarely considered. In this work, we propose an innovative method for multi-target vital signs for unsteady scenarios, such as aerobic exercise. The method automatically distinguishes between steady state and motion state, and completes a best-effort vital signs detection during unsteady state. To differentiate multiple targets, we design a weight vector enhancement method combined with target space localization, and then, apply the variational mode decomposition (VMD) algorithm to extract the respiratory and heart rates of a single target. Moreover, for evaluating the efforts of exercise, we propose Multi-target Motion Recognition (MMR) based on MobileNet-V2 network to recognize the motion states of multiple targets. The experimental results showed that the mean absolute error of respiratory rate and heart rate decreased to 1.37 bpm and 2.56 bpm, respectively. Meanwhile, the MMR algorithm achieves close to 98.1% accuracy in recognizing motion states.

Gait recognition aims to identify users by their walking patterns. Compared with appearance-based methods, skeleton-based methods exhibit well robustness to cluttered backgrounds, carried items, and clothing variations. However, skeleton extraction faces the wrong human tracking and keypoints missing problems, especially under multiperson scenarios. To address above issues, this article proposes a novel gait recognition method using deep neural network specifically designed for multiperson scenarios. The method consists of individual gait separate module (IGSM) and fuzzy skeleton completion network (FU-SCN). To achieve effective human tracking, IGSM employs root–skeleton keypoints predictions and object keypoint similarity (OKS)-based skeleton calculation to separate individual gait sets when multiple persons exist. In addition, keypoints missing renders human poses estimation fuzzy. We propose FU-SCN, a deep neuro-fuzzy network, to enhances the interpretability of the fuzzy pose estimation via generating fine-grained gait representation. FU-SCN utilizes fuzzy bottleneck structure to extract features on low-dimension keypoints, and multiscale fusion to extract dissimilar relations of human body during walking on each scale. Extensive experiments are conducted on the CASIA-B dataset and our multigait dataset. The results show that our method is one of the SOTA methods and shows outperformance under complex scenarios. Compared with PTSN, PoseMapGait, JointsGait, GaitGraph2, and CycleGait, our method achieves an average accuracy improvement of 53.77%, 42.07%, 25.3%, 13.47%, and 9.5%, respectively, and it keeps low time cost with average 180 ms using edge devices.

Unmanned Aerial Vehicle (UAV)-enabled wirelessly powered Mobile Edge Computing (MEC) is emerging as a powerful technology for boosting computational capability and energy supplementation in Internet of Things (IoT). This work addresses the long-term task completion ratio maximization problem in UAV-enabled wirelessly powered MEC systems. Besides the large number of optimization parameters, the environment can only be partially observed as the UAVs cannot cover the whole network area. Then, it is very challenging to obtain good solutions due to the lack of global information. We introduce a novel distributed Multi-Agent Deep Reinforcement Learning (MADRL) framework for optimizing UAVs’ actions and resource allocation, considering the constraints of tasks that vary in size, arrival times, and required computation completion time. To decouple the complicated parameters, we divide the problem into two manageable subproblems—UAVs’ action decision and resource allocation under a given UAV’s action. We employ a distributed Deep Reinforcement Learning (DRL) scheme for the former subproblem to cope with the partially observable nature. By revealing some important properties of the later subproblem, we design an efficient two-stage optimal algorithm to minimize the total consumed energy of nodes while maximizing the task-completing number. Extensive simulations validate the effectiveness of the proposed framework, achieving over a 50% improvement in task completion ratio compared to baseline schemes in some scenarios.

Benefiting from smartphones’ powerful computing and sensing capabilities, biometric authentication is widely used on them to conveniently verify user’s identity. However, most biometric features can be easily acquired or reproduced, making them vulnerable to replay and impersonation attacks. To address these issues, we propose DBreathLock, a non-contact identity authentication system that leverages a smartphone to capture unique biometric features from users’ deep breaths. Specifically, the smartphone emits inaudible frequency-modulated continuous waves (FMCW)-based sonar signals to capture chest-abdominal-joint (C-A-joint) movements, increasing the difficulty of impersonating legitimate users. By analyzing the energy features of CA-joint movements, we develop a Deep Breath Activity Detection (DBAD) method to detect deep breath fragments alongside the capability of resisting advanced replay attacks. To further fortify the resistance to advanced replay attacks, we employ the morphological features of C-A-joint movements and SVC model to recognize these attacks. Following this, we construct mutual information (MI) sequences via calculating the correlation between C-A-joint movements and breath sounds to reduce identity authentication failure caused by physiological changes. Then, a multi-stream identity authentication model is designed to verify legitimate users by fusing the features from C-A-joint movements, deep breath sounds, and MI sequences. Extensive real-world experiments involving 20 users demonstrate that DBreathLock achieves an authentication accuracy of 98.33%. Additionally, it successfully defends against both advanced replay and impersonation attacks.

In product quality monitoring, surface defect detection (SDD) is an important part related to the appearance and performance of the product. Machine vision-based SDD usually needs to construct a product-specific detection model and manually label the dataset for training model by paid labeler. We try to transform the single labeler’s work into collaborative labeling works of multiple Internet users to reduce the labeling cost, and propose CrowdLab, a crowd-sourcing dataset labeling system for SDD. In CrowdLab, we tactfully segment the product surface image and employ verification image in web login to make multiple users collaboratively select the part of defect regions and finally recognize these regions. The experiment results illustrate that SDD model using CrowdLab dataset can achieve 96.7% accuracy, which is close to the personnel work.

With the rapid development of Internet of Vehicles technology, mobile communications are gradually integrated with various fields, and more and more Internet of Vehicles equipment are connected to the Internet. However, existing online information transmission methods mainly rely on the original network infrastructure. Once the communication infrastructure fails, it is likely that information transmission will fail or even be lost. In this paper, we utilize the optical modules that come with sensor nodes to implement a hybrid communication debugging system based on Visible Light Communication (VLC). To enhance uplink reliability, we’ve devised an optical camera-compatible frame synchronization method. Leveraging the Transformer algorithm, we predict frame header positions, thereby bolstering data collection reliability. Additionally, for efficient debugging information uploading, we logically group and organize the initial data, and use the Snappy compression algorithm to decrease empty time slot count to complete the data compression, saving time. Finally, confidentiality enhancement technology is introduced in the physical layer, and a new security enhancement optimization scheme based on Artificial Noise is proposed. The Artificial Noise (AN) sent by the sender enables the sender to counter eavesdropping interference, and the authorized recipient can cancel the Artificial Noise (AN). The results show that the proposed scheme is more secure.

The millimeter-wave(mm-Wave) radar based on frequency-modulated continuous wave (FMCW) owns the advantages of non-contact, privacy protection, high resolution, and anti-interference, and it became the hot point that applying the radar in monitoring vital signs for smart home. However, most of current studies focus on how to improve the detection performance under the steady scenarios and give little consideration on the unsteady scenarios with physical interference. In this paper, we propose a method to detect vital signs under unsteady scenarios by a best-effort way. This method automatically differentiates between the steady state and motion state (unsteady) by identifying the motion type, and extracts the vital sign under steady state without physical motion interference. For this end, we first figure out feature spectrograms with range-main velocity information from motion features. And then, employ a sliding windows sampling method to construct data set, and apply ResNet-18 network model in the motion type identification (including steady state). Based on the motion type, the phase signal during steady state and leverage the variational mode decomposition (VMD) algorithm to analyze respiration/heart rate. Experiment results show that using ResNet-18, the recognition accuracy of the motion state and motion type is close to 97%, and the recognition delay is less than 1.1s. Meanwhile, the mean absolute errors of the respiration rate and heart rate drop to 1.7bpm and 3.4bpm respectively.

As a vital metric of information timeliness, age of information (AoI) is important for real-time applications in Internet of Things (IoT), such as health monitoring. To satisfy these requirements, we study a wireless-powered IoT (WPIoT), where a static hybrid access point (HAP) coordinates the wireless energy transfer to mobile IoT nodes, and mobile IoT nodes transmit data to the HAP or static IoT nodes. We minimize the AoI of mobile IoT nodes by optimizing the selection of the HAP or static IoT node for transmission, the channel selection, the duration of data transmission, and the transmit power, and prove the AoI minimization problem as NP-hard. To tackle it, we propose a deep deterministic policy gradient (DDPG)-based distributed multi-node resource allocation (DDMRA) algorithm, which combines the advantages of distributed algorithms and centralized algorithms, and combines the selection of discrete actions in the DQN algorithm into the DDPG algorithm. In the DDMRA algorithm, mobile IoT nodes save the energy consumption of transmitting state information to the HAP. Numerical results validate the superior performance of the DDMRA algorithm compared with baseline algorithms.

Speech emotion recognition (SER) is an active research area in affective computing. Recognizing emotions from speech signals helps to assess human behaviour, which has promising applications in the area of human-computer interaction. The performance of deep learning-based SER methods relies heavily on feature learning. In this paper, we propose SCAR-NET, an improved convolutional neural network, to extract emotional features from speech signals and implement classification. This work includes two main parts: First, we extract spectral, temporal, and spectral-temporal correlation features through three parallel paths; and then split-convolve-aggregate residual blocks are designed for multi-branch deep feature learning. The features are refined by global average pooling (GAP) and pass through a softmax classifier to generate predictions for different emotions. We also conduct a series of experiments to evaluate the robustness and effectiveness of SCAR-NET which can achieve 96.45%, 83.13%, and 89.93% accuracy on the speech emotion datasets EMO-DB, SAVEE, and RAVDESS. These results show the outperformance of SCAR-NET.

As one widely applied wireless technique, WiFi has the potential to execute noncontact monitoring of vital signs based on channel state information (CSI). However, due to the dynamic of the surrounding environment, the bandwidth of the WiFi channel is not enough to identify the respiration-induced path from other movement-induced paths and this seriously limits the accuracy of respiration rate detection. In this article, we propose ExRadio, a system that can monitor respiration in high-dynamic environments via combining multiple WiFi channels. Specifically, the receiver synchronously switches the channels with the transmitter and samples CSI at multiple channels, and then the CSI data are combined and regarded as CSI data of one extended-bandwidth channel. However, the hardware-related noises from multiple channels are also accumulated. Eliminating these noises causes too heavy computation overhead to be afforded by the embedded devices and affects the real-time performance of respiration monitoring. To address this problem, we propose an effective approach that employs the ratio of CSI readings from the wireless channel and wire direct connection channel to shorten the time of eliminating the hardware-related noise. We deploy the ExRadio in commercial off-the-shelf embedded devices and conduct a series of experiments. The experimental results demonstrate that reducing the execution time is beneficial to respiration rate detection under high-dynamic environments, and the overall detection error of ExRadio is less than 0.5 bpm even when multiple persons who are 1.5-m away from the monitored person are fast walking.

Currently, powerful and ubiquitous mobile devices provide an opportunity to map physical conditions to cyberspace and realize Digital Twins enabled Healthcare (DTeH). Especially, the impact of the COVID-19 epidemic renders it necessary to keep an eye on the changing trend of respiration. Long-term respiration monitoring helps to assess personal health status and thus becomes an important issue in DTeH. However, previous mobile device-assistant methods mostly implement the monitoring via short-time detection in a best-effort way and with less consideration of identity recognition, the only mean to bind physical vital signs into personal profiles in digital twins space. Thus, it is necessary to introduce the identification to complete string multiple short-time detections and form long-term personal monitoring. To this end, we propose IDRes, an identity-based respiration monitoring system for DTeH. This system employs mobile devices to generate a high-frequency sonar signal to complete respiration detection and identity recognition. As well as it also estimates the respiration rate by tracking the phase change of the sonar signal and recognizes identity via the Doppler frequency shift of the signal to capture characteristics of chest movement. Moreover, via band-pass filtering to remove the low-frequency voice component of the received signals, the usage of the high-frequency sonar signal also enhances security at the physical level. At last, we conduct a series of experiments under different conditions. Experimental results illustrate that IDRes achieves the mean detection error of 0.49bpm with over 93.3% recognition accuracy, and manifest that IDRes can satisfy the requirements of mapping the accurate vital sign data to the personal profile of DTeH.

Theabnormal behavior detection is the vital for evaluation of daily-life health status of the patient with cognitive impairment. Previous studies about abnormal behavior detection indicate that convolution neural network (CNN)-based computer vision owns the high robustness and accuracy for detection. However, executing CNN model on the cloud possible incurs a privacy disclosure problem during data transmission, and the high computation overhead makes difficult to execute the model on edge-end IoT devices with a well real-time performance. In this paper, we realize a skeleton-based abnormal behavior detection, and propose a secure partitioned CNN model (SP-CNN) to extract human skeleton keypoints and achieve safely collaborative computing by deploying different CNN model layers on the cloud and the IoT device. Because, the data outputted from the IoT device are processed by the several CNN layers instead of transmitting the sensitive video data, objectively it reduces the risk of privacy disclosure. Moreover, we also design an encryption method based on channel state information (CSI) to guarantee the sensitive data security. At last, we apply SP-CNN in abnormal behavior detection to evaluate its effectiveness. The experiment results illustrate that the efficiency of the abnormal behavior detection based on SP-CNN is at least 33.2% higher than the state-of-the-art methods, and its detection accuracy arrives to 97.54%.

Segmenting reference bones from radiographs of the hand is important for bone age assessment. Due to the influence of the irregular shapes and the adjacent positions of the wrist reference bones, it is difficult for the expert to accurately estimate the mature indication of the wrist reference bones in the figures. How to precisely segment the reference bones automatically from the radiographs is a challenge. For this is problem, an improved U-Net, Attention Residual U-Net (ARU-Net) proposed in this paper. Firstly, we extract the reference bone region of interest (ROI) by faster region-based convolutional neural networks(R-CNN). Then, the pre-processed ROI is fed into ARU-Net for segmentation. On the basis of traditional U-Net, ARU-Net adds residual mapping and attention mechanism, which improves the utilization rate of features and the accuracy of reference bone segmentation. Finally, a post-processing method including the flood fill algorithm and the morphological operation is used to eliminate jagged edges and holes in the segmented result. The hamate is one of the most difficult reference bones to segment in the wrist. This paper takes it as an example to assess the performance of ARU-Net. Experiments show that compared with Fully Convolutional Neural Network (FCN), U-Net and ResUnet, the accuracy and F1 scores of ARU-Net are higher. Its accuracy rate is 96.41%, and F1 score is 0.9529. The post-processing method can further improve the result. Finally, the accuracy rate reaches 96.51%, and the F1 score reaches 0.9544. ARU-Net can precisely segment the reference bone, which facilitates the expert to assess its mature indication, so as to accurately evaluate the bone age.

During the period of over-the-air reprogramming, sensor nodes are easy to eavesdrop and even controlled by unauthorized person. That reminds us that security is key issue for over-the-air reprogramming. Most of previous studies discussed this problem from the aspect of data encryption, but give little consideration to the physical level. In this paper, we attempt to improve the security of reprogramming by changing the physical-level communication mode. We apply unidirectional Visible Light Communication (VLC) to the over-the-air reprogramming and use Commercial Off-The-Shelf device such as smartphone and sensor node to improve applicability. However, the unstable light source and low-cost light sensor make the procedure of reprogramming difficult. For this end, we put forward a novel reprogramming approach named ReVLC, which is twofold: firstly, we design a code block mechanism based on function similarity to reduce transmitting code. Secondly, we use compressing representation to optimize the Dual Header-Pulse Interval Modulation (DH-PIM) to save transmission time. The experiment results illustrate the effectiveness of ReVLC at the cost of extra 49.1% energy overhead compared with a traditional reprogramming approach.

Reprogramming is one of the key issues for long-term living wireless sensor networks. Most of current research focuses on optimizing the upper software in order to save energy or improve security. In this paper, we replace the electromagnetic communication with visible light communication applied in reprogramming, and then leverage general sensor node and smartphones for lowering the barrier of VLC applied in reprogramming. However, the unstable light source and low-cost light sensor make the reprogramming procedure difficult. For this, we present a novel incremental reprogramming approach named EasiVLC. This approach refers our previous research and optimizes visible light signal modulation based on VLC for reducing transmission time. The experiment results illustrate effectiveness of EasiVLC at the cost of increasing 52.5% energy overhead by comparison with EC-based reprogramming approach.

In fog computing, fog nodes running different tasks near the sources of data are required. Limited to on‐board resource, fog node finds it hard to execute multiple tasks and needs over‐air reprogramming to rearrange them. With respect to reprogramming, energy efficiency is one of the key issues for over‐air reprogramming. Most of traditional reprogramming approaches focus on the energy efficiency during data transmission within network. However, program rebuilding on fog node is, as another significant energy cost, caused by writing/reading local high‐power memory. We present a novel incremental reprogramming approach, RePage, in three stages. Firstly, we design a function paging mechanism that makes similar functions to one function page and caches them in low‐power volatile memory to save energy. Secondly, we design new cache replacement algorithm for function page considering both modification times and range on the page. At last, further reducing writing/reading operations, we also redesign function invocation manner by centralized managing function addresses. Experiment results show that RePage reduces the sum of reading/writing operations on volatile memory by 89.1% and 92.5% compared to EasiCache and Tiny Module‐link, and its hit rate is improved by 10.4% to Least Recently Used (LRU) algorithm.

Traditional wireless sensor networks (WSNs) can be integrated into Internet and be regarded as its sensing infrastructure, which supports development and running of multiple third-party applications simultaneously. Therefore, due to constrained resource of sensor nodes, it is necessary to establish a runtime framework to improve sensor sharing efficiency for concurrent third-party applications. This paper presents EasiCAE, a concurrent applications runtime framework, to enhance sensor sharing efficiency greatly by incorporating task allocation with redundancy elimination. In brief, EasiCAE decompose the applications into tasks and distributes tasks to the sensors which will bring the least energy to run them. EasiCAE has three salient features. Firstly, we define task-sensor correlation to indicate how many samplings of a sensor can be shared with the new task. Secondly, EasiCAE reduces energy consumption by assigning tasks to a sensor with higher task-sensor correlation. Finally, a light-weight merging algorithm is proposed to eliminate redundant samplings for the assigned sensors. Experimental results show that EasiCAE reduces energy consumption by 31% to 79% compared with existing methods, while introducing tolerable overheads. We also evaluate EasiCAE with various influencing parameters, showing that the performance of EasiCAE increases stably as the network scale and the number of concurrent applications increases.