Numerous deep learning-based works focusing on 3D semantic segmentation have been proposed and have achieved impressive performance. However, due to the catastrophic forgetting, existing methods will degrade dramatically in a real-world scenario where new 3D semantic categories are arriving continually. Straightforwardly applying typical class-incremental learning methods on 3D data even aggravates forgetting due to the irregular and noisy geometric structure. Aiming to address this realistic challenge, from the perspective of capturing local topological characteristics and mitigating global semantic shift, we propose a unified framework named Local topological Alignment and Global semantic Deconstruction (LAGD) to incrementally learn semantic knowledge of novel 3D categories while maintaining performance on previously learned knowledge. Specifically, we develop a novel Interaction Topological-aware Alignment (ITA) to maintain the learned knowledge efficiently by capturing the local geometric characteristics with interacted adjacent state-specific knowledge. Besides, to mitigate the forgetting caused by the global semantic shift, we deconstruct the logits into positive and negative parts which are distilled separately, achieving an elaborate distillation process in terms of Semantic-knowledge Deconstruction Distillation (SDD). With the cooperation of ITA and SDD, LAGD achieves a sota performance, especially in the long-term incremental learning scenario. Extensive experimental results illustrate the superiority of our proposed LAGD.

The widespread use of electronic devices has contributed to an increase in poor posture, particularly when it comes to the cervical spine, leading to various cervical vertebral pain disorders. In this article, we focus on accurately monitoring the motion status of the cervical spine using the accelerometers and gyroscope sensors embedded in earphones. Our aim is to gain a better understanding of cervical spine health. To address the individual differences among subjects, we introduce fuzzy rules to the Re-ISDA method, proposing a novel approach known as FuzRe-ISDA. Unlike traditional domain adaptation methods, the FuzRe-ISDA method offers flexibility in adjusting the contribution from different source domains. It takes into account the collective impact of multiple models on predicting new user behavior. Moreover, this method can quickly adapt to new users without requiring extensive datasets. Experimental results demonstrate that our FuzRe-ISDA approach outperforms popular domain adaptation methods in terms of accuracy when predicting cervical motion. This highlights the effectiveness of our approach in addressing individual differences and improving the reliability of cervical spine motion prediction.

Recognizing unauthorized Unmanned Aerial Vehicles (UAVs) within designated no-fly zones throughout the day and night is of paramount importance, where the unauthorized UAVs pose a substantial threat to both civil and military aviation safety. However, recognizing UAVs day and night with dual-vision cameras is nontrivial, since red–green–blue (RGB) images suffer from a low detection rate under an insufficient light condition, such as on cloudy or stormy days, while black-and-white infrared (IR) images struggle to capture UAVs that overlap with the background at night. In this paper, we propose a new optical flow-assisted graph-pooling residual network (OF-GPRN), which significantly enhances the UAV detection rate in day and night dual visions. The proposed OF-GPRN develops a new optical fusion to remove superfluous backgrounds, which improves RGB/IR imaging clarity. Furthermore, OF-GPRN extends optical fusion by incorporating a graph residual split attention network and a feature pyramid, which refines the perception of UAVs, leading to a higher success rate in UAV detection. A comprehensive performance evaluation is conducted using a benchmark UAV catch dataset. The results indicate that the proposed OF-GPRN elevates the UAV mean average precision (mAP) detection rate to 87.8%, marking a 17.9% advancement compared to the residual graph neural network (ResGCN)-based approach.

Phishing websites distribute unsolicited content and are frequently used to commit email and internet fraud; detecting them before any user information is submitted is critical. Several efforts have been made to detect these phishing websites in recent years. Most existing approaches use hand-crafted lexical and statistical features from a website’s textual content to train classification models to detect phishing web pages. However, these phishing detection approaches have a few challenges, including (1) the tediousness of extracting hand-crafted features, which require specialized domain knowledge to determine which features are useful for a particular platform; and (2) the difficulties encountered by models built on hand-crafted features to capture the semantic patterns in words and characters in URL and HTML content. To address these challenges, this paper proposes WebPhish, an end-to-end deep neural network trained using embedded raw URLs and HTML content to detect website phishing attacks. First, the proposed model automatically employs an embedding technique to extract the corresponding characters into homologous dense vectors. Then, the concatenation layer merges the URL and HTML embedding matrices. Following that, Convolutional layers are used to model its semantic dependencies. Extensive experiments were conducted with real-world phishing data, which yielded an accuracy of 98.1%, showing that WebPhish outperforms baseline detection approaches in identifying phishing pages.

Symmetric key generation methods are recently designed for wireless communication based on similar and unique observations of sensor measurements, such as wireless radio channels, inaudible sound channels, etc. Heartbeats, as unique biometrics, have been used for symmetric key generation. However, current solutions are designed for wearable devices with the integration of the same types of touchable heartbeat measurement equipment and fail because of the significant difference from different devices or the same devices with different deployment locations, which limits its large scale of deployment and application. To solve this problem, we propose a general heartbeat-based symmetric key generation solution by embracing observation variance from different devices, i.e., using an optical heart sensor on one finger and facing the camera of the second device to the user's face. We propose a novel data processing method to mitigate the significant difference and exploit key reconciliation to generate symmetric keys for paring devices and securing wireless communication. We have conducted extensive evaluations and shown our proposed method has good key matching rates up to 100% as well as good randomness. Security analysis has also been conducted to ensure the robustness of the proposed method.

The increasing proliferation of Internet-of-Things (IoT) devices in daily life has rendered secure Device-to-Device (D2D) communication increasingly crucial. Achieving secure D2D communication necessitates key agreement between various IoT devices without prior knowledge. Despite existing literature proposing numerous approaches, they exhibit limitations such as low key generation rates and short pairing distances. In this paper, we present InaudibleKey2.0, an inaudible acoustic signal based key generation protocol for mobile devices. Based on acoustic channel reciprocity, InaudibleKey2.0 exploits the acoustic channel frequency response of two legitimate devices as a shared secret for key generation. To significantly enhance performance, InaudibleKey2.0 incorporates novel technologies, including a deep learning-enabled channel prediction model for improved channel reciprocity, a quantization model for increased key generation rates, and a transformer-based reconciliation method for augmented key agreement rates. We conduct comprehensive experiments to evaluate InaudibleKey2.0 in diverse real-world environments. In comparison to state-of-the-art solutions, InaudibleKey2.0 achieves 1.3–9.1 times improvement in key generation rates, 3.2–44 times extension in pairing distances, and 1.2–16 times reduction in information reconciliation counts. Security analysis substantiates that InaudibleKey2.0 is resilient to numerous malicious attacks. Furthermore, we implement InaudibleKey2.0 on modern smartphones and resource-limited IoT devices. The results indicate that it is energy-efficient and can operate on both powerful and resource-limited IoT devices without causing excessive resource consumption.

Solar cells have been widely used for offering energy for Internet of Things (IoT) devices. Recently, solar cells have also been used as sensors for context awareness sensing due to their sensitivity to varying lighting conditions. In this article, we are the first to use solar cells for symmetric key generation. To generate symmetric keys, we take advantage of photovoltage measurements generated from solar cells equipped with a pair of IoT devices. Symmetric keys are essential for pairing IoT devices and further securing wireless communication. Despite the sensitivity to varying lighting conditions, challenges still remain for the use of solar cells for key generation, such as time unsynchronisation and noisy measurements. To solve these challenges, we design a novel key generation framework, SolarKey, which includes the starting point detection and a compressed sensing-based two-tier key reconciliation method. Extensive experiments have been conducted to evaluate the performance of our proposed key generation method in various environments, which shows the proposed method can improve the key matching rate by up to 25%. We also conduct security analysis and the randomness test, which shows that SolarKey is resilient to common attacks such as the eavesdropping attack and the imitating attack and sufficiently random.

Abstract Human activity recognition based on generated sensor data plays a major role in a large number of applications such as healthcare monitoring and surveillance system. Yet, accurately recognizing human activities is still challenging and active research due to people’s tendency to perform daily activities in a different and multitasking way. Existing approaches based on the recurrent setting for human activity recognition have some issues, such as the inability to process data parallelly, the requirement for more memory and high computational cost albeit they achieved reasonable results. Convolutional Neural Network processes data parallelly, but, it breaks the ordering of input data, which is significant to build an effective model for human activity recognition. To overcome these challenges, this study proposes causal convolution based on performers-attention and supervised contrastive learning to entirely forego recurrent architectures, efficiently maintain the ordering of human daily activities and focus more on important timesteps of the sensors’ data. Supervised contrastive learning is integrated to learn a discriminative representation of human activities and enhance predictive performance. The proposed network is extensively evaluated for human activities using multiple datasets including wearable sensor data and smart home environments data. The experiments on three wearable sensor datasets and five smart home public datasets of human activities reveal that our proposed network achieves better results and reduces the training time compared with the existing state-of-the-art methods and basic temporal models.

The capability of local smart sensing based on Internet of Things (IoT) devices is typically limited due to due to the inherent limitations of computational and storage capabilities. Recently, collaborative inference among multiple devices has been considered as an effective way to improve the sensing capabilities of individual IoT devices. However, the collaborative inference process still faces the challenges of data privacy leakage and inefficient collaboration. To alleviate the above issues, we design a blockchain-based collaborative inference system in this article, called CoBC, which allows each heterogeneous device node on the blockchain to customize a personalized local machine learning model according to its own hardware constraint and performance, thus improving the efficiency of resource utilization of the whole system. Meanwhile, each device node only needs to complete training locally, which significantly reduces the risk of privacy leakage due to the remote transmission of local data. CoBC improves the sensing capability of single device nodes by using collaborative inference that can obtain a more robust global inference. In addition, CoBC employs a Bayesian approximation training approach to evaluate the output uncertainty of each device node to further improve the efficiency of collaborative inference. To evaluate the performance, we deploy CoBC in a real environment and conduct a large number of simulations to evaluate the efficiency of CoBC. The simulation results demonstrate that CoBC exhibits good performance and good practicality in various criteria.

Recently we have witnessed the rise of Artificial Intelligence of Things (AIoT) and the shift of sensing paradigm from cloud-centric to the edge-centric, which effectively improves the sensing capability of intelligence transportation systems. To improve the real-time sensing performance, in this work we propose an ensemble sensing based scheme to solve the time-constraint synchronized inference problem and achieve robust inference with heterogeneous IoT devices in intelligence transportation systems. We design and implement Ensen, which incorporates various novel techniques such as customized DNN model design, KD-based model training, and dynamic deep ensemble management, etc., to achieve improved accuracy and maximize the computational resource usage of the whole sensing group. Extensive evaluations on different types of common IoT devices have shown that Ensen achieves a robust performance and can be easily extended to different types of convolutional neural networks.

Networked devices, such as wearable devices, laptops, smart home appliances, etc., are ubiquitous nowadays. To secure communication among those devices, symmetric keys are widely used because of their feasibility in resource-constrained networked devices. The ob-servations of sensors from independent devices have been adopted for symmetric key generation. The identical biometrics information or environment interference has been observed by sensors, and their corresponding patterns are used for key generation. Pop-ular signals from networked devices are inertial measurements, sound, wireless signals, etc. The existing sensor-based key gen-eration solutions use the same type of sensors for both devices. Different from the existing solutions, we are the first to propose a cross-sensor symmetric key generation system i2Key, where two devices collect inertial measurements from a motion sensor and inaudible sound from a microphone, respectively. A new coding framework is designed for general key generation. We also pro-pose an efficient and accurate time synchronisation method for key generation. Additionally, a multi-tier key reconciliation method is suggested to improve key generation performance. By using the proposed architecture, the key generation rate is improved by up to approximately 40% compared with the situation without using it. We also perform security analysis and randomness analysis over the proposed method.

In this paper, we propose a new ultra-wideband (UWB)-based simultaneous localization and wall-mapping (SLAM) system, which adopts two-way ranging optimization on UWB anchor and tag nodes to track the target's real-time movement in an unknown area. The proposed UWB-based SLAM system captures time difference of arrival (TDoA) of the anchor nodes' signals over a line-of-sight propagation path and reflected paths. The real-time location of the UWB tag is estimated according to the real-time TDoA measurements. To minimize the estimation error resulting from background noise in the two-way ranging, a Least Squares Method is implemented to minimize the estimation error for the localization of a static target, while Kalman Filter is applied for the localization of a mobile target. An experimental testbed is built based on off-the-shelf UWB hardware. Experiments validate that a reflector, e.g., a wall, and the UWB tag can be located according to the two-way ranging measurement. The localization accuracy of the proposed SLAM system is also evaluated, where the difference between the estimated location and the ground truth trajectory is less than 1 meter.

Smart space has emerged as a new paradigm that combines sensing, communication, and artificial intelligence technologies to offer various customized services. A fundamental requirement of these services is person identification. Although a variety of person-identification approaches has been proposed, they suffer from several limitations in practical applications, such as low energy efficiency, accuracy degradation, and privacy issue. This article proposes an energy-harvesting-based privacy-preserving gait recognition scheme for smart space, which is named PrivGait. In PrivGait, we extract discriminative features from 1-D gait signal and design an attention-based long short-term memory (LSTM) network to classify different people. Moreover, we leverage a novel Bloom filter-based privacy-preserving technique to address the privacy leakage problem. To demonstrate the feasibility of PrivGait, we design a proof-of-concept prototype using off-the-shelf energy-harvesting hardware. Extensive evaluation results show that the proposed scheme outperforms state of the art by 6%–10% and incurs low system cost while preserving user’s privacy.

Localisation is an important part of many applications. Our motivating scenarios are short-term construction work and emergency rescue. These scenarios also require rapid setup and robustness to environmental conditions additional to localisation accuracy. These requirements preclude the use of many traditional high-performance methods, e.g., vision-based, laser-based, Ultra-wide band (UWB) and Global Positioning System (GPS)-based localisation systems. To overcome these challenges, we introduceiMag+, an accurate and rapidly deployable inertial magneto-inductive (MI) mapping and localisation system, which only requires monitored workers to carry a single MI transmitter and an inertial measurement unit in order to localise themselves with minimal setup effort. However, one major challenge is to use distorted and ambiguous MI location estimates for localisation. To solve this challenge, we propose a novel method to use MI devices forsensing environmental distortionsfor accurate closing inertial loops. We also suggest a robust and efficient first quadrant estimator to sanitise the ambiguous MI estimates. By applying robust simultaneous localisation and mapping (SLAM), our proposed localisation method achieves excellent tracking accuracy and can improve performance significantly compared with only using a Magneto-inductive device or inertial measurement unit (IMU) for localisation.

Secure Device-to-Device (D2D) communication is becoming increasingly important with the ever-growing number of Internet-of-Things (IoT) devices in our daily life. To achieve secure D2D communication, the key agreement between different IoT devices without any prior knowledge is becoming desirable. Although various approaches have been proposed in the literature, they suffer from a number of limitations, such as low key generation rate and short pairing distance. In this paper, we present InaudibleKey, an inaudible acoustic signal based key generation protocol for mobile devices. Based on acoustic channel reciprocity, InaudibleKey exploits the acoustic channel frequency response of two legitimate devices as a common secret to generating keys. InaudibleKey employs several novel technologies to significantly improve its performance. We conduct extensive experiments to evaluate the proposed system in different real environments. Compared to state-of-the-art works, InaudibleKey improves key generation rate by 3 times, extends pairing distance by 3.2 times, and reduces information reconciliation counts by 2.5 times. Security analysis demonstrates that InaudibleKey is resilient to a number of malicious attacks. We also implement InaudibleKey on modern smartphones and resource-limited IoT devices. Results show that it is energy-efficient and can run on both powerful and resource-limited IoT devices without incurring excessive resource consumption.

Unmanned Aerial Vehicles (UAVs) can collaborate to collect and relay data for ground sensors in remote and hostile areas. In multi-UAV-assisted wireless sensor networks (MA-WSN), the UAVs' movements impact on channel condition and can fail data transmission, this situation along with newly arrived data give rise to buffer overflows at the ground sensors. Thus, scheduling data transmission is of utmost importance in MA-WSN to reduce data packet losses resulting from buffer overflows and channel fading. In this paper, we investigate the optimal ground sensor selection at the UAVs to minimize data packet losses. The optimization problem is formulated as a multi-agent Markov decision process, where network states consist of battery levels and data buffer lengths of the ground sensor, channel conditions, and waypoints of the UAV along the trajectory. In practice, an MA-WSN contains a large number of network states, while the up-to-date knowledge of the network states and other UAVs' sensor selection decisions is not available at each agent. We propose a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MUAIS) to minimize the data packet loss, where the UAVs learn the underlying patterns of the data and energy arrivals at all the ground sensors. Numerical results show that the proposed MUAIS achieves at least 46 % and 35% lower packet loss than an optimal solution with single-UAV and an existing non-learning greedy algorithm, respectively.

Research has shown the potential of device-free WiFi sensing for human identification. Each and every human has a unique gait and prior works suggest WiFi devices are able to capture the unique signature of a person's gait. In this article, we show for the first time that the monitored gait could be inconsistent and have mirror-like perturbations when individuals walk through WiFi devices in different directions, provided that the WiFi antenna array is horizontal to the walking path. Such inconsistent mirrored patterns are to negatively affect the uniqueness of gait and accuracy of human identification. Therefore, we propose a system called Gate-ID for accurately identifying individuals' identities irrespective of different walking directions. Gate-ID employs theoretical communication model and real measurements to demonstrate that antenna array orientations and walking directions contribute to the mirror-like patterns in WiFi signals. A novel heuristic algorithm is proposed to infer individual's walking directions. A set of methods are employed to extract and augment the representative spatial-temporal features of gait and enable the system performing irrespective of walking directions. We further propose a novel attention-based deep learning model that fuses various weighted features and ignores ineffective noises to uniquely identify individuals. We implement Gate-ID on commercial off-the-shelf devices. Extensive experiments demonstrate that our system can uniquely identify people with average accuracy of 90.7%-75.7% from a group of 6-20 people, respectively, and improve the accuracy by 12.5%-43.5% compared with baselines.

Recent research has devoted significant efforts on the utilization of WiFi signals to recognize various human activities. An individual's limb motions in the WiFi coverage area could interfere with wireless signal propagation, that manifested as unique patterns for activity recognition. Existing approaches though yielding reasonable performance in certain cases, are ignorant of two major challenges. The performed activities of the individual normally have inconsistent speed in different situations and time. Besides that the wireless signal reflected by human bodies normally carries substantial information that is specific to that subject. The activity recognition model trained on a certain individual may not work well when being applied to predict another individual's activities. Since only recording activities of limited subjects in a certain speed and scale, recent works commonly have a moderate amount of activity data for training the recognition model. The small-size data could often incur the overfitting issue that negative affect the traditional classification model. To address these challenges, we propose a WiFi-based human activity recognition system that synthesizes variant activities data through eight channel state information (CSI) transformation methods to mitigate the impact of activity inconsistency and subject-specific issues, and also design a novel deep-learning model that caters to the small-size WiFi activity data. We conduct extensive experiments and show synthetic data improve performance by up to 34.6% and our system achieves around 90% of accuracy with well robustness in adapting to small-size CSI data.

Abstract Systems of sensor human activity recognition are becoming increasingly popular in diverse fields such as healthcare and security. Yet, developing such systems poses inherent challenges due to the variations and complexity of human behaviors during the performance of physical activities. Recurrent neural networks, particularly long short-term memory have achieved promising results on numerous sequential learning problems, including sensor human activity recognition. However, parallelization is inhibited in recurrent networks due to sequential operation and computation that lead to slow training, occupying more memory and hard convergence. One-dimensional convolutional neural network processes input temporal sequential batches independently that lead to effectively executed operations in parallel. Despite that, a one-dimensional Convolutional Neural Network is not sensitive to the order of the time steps which is crucial for accurate and robust systems of sensor human activity recognition. To address this problem, we propose a network architecture based on dilated causal convolution and multi-head self-attention mechanisms that entirely dispense recurrent architectures to make efficient computation and maintain the ordering of the time steps. The proposed method is evaluated for human activities using smart home binary sensors data and wearable sensor data. Results of conducted extensive experiments on eight public and benchmark HAR data sets show that the proposed network outperforms the state-of-the-art models based on recurrent settings and temporal models.

Device-free context awareness is important to many applications. There are two broadly used approaches for device-free context awareness, i.e., video-based and radio-based. Video-based approaches can deliver good performance, but privacy is a serious concern. Radio-based context awareness applications have drawn researchers' attention instead, because it does not violate privacy and radio signal can penetrate obstacles. The existing works design explicit methods for each radio-based application. Furthermore, they use one additional step to extract features before conducting classification and exploit deep learning as a classification tool. Although this feature extraction step helps explore patterns of raw signals, it generates unnecessary noise and information loss. The use of raw CSI signal without initial data processing was, however, considered as no usable patterns. In this article, we are the first to propose an innovative deep learning–based general framework for both signal processing and classification. The key novelty of this article is that the framework can be generalised for all the radio-based context awareness applications with the use of raw CSI. We also eliminate the extra work to extract features from raw radio signals. We conduct extensive evaluations to show the superior performance of our proposed method and its generalisation.

This paper introduces an instance-aware hashing approach Region-DH for large-scale multi-label image retrieval. The accurate object bounds can significantly increase the hashing performance of instance features. We design a unified deep neural network that simultaneously localizes and recognizes objects while learning the hash functions for binary codes. Region-DH focuses on recognizing objects and building compact binary codes that represent more foreground patterns. Region-DH can flexibly be used with existing deep neural networks or more complex object detectors for image hashing. Extensive experiments are performed on benchmark datasets and show the efficacy and robustness of the proposed Region-DH model.

Recently, the development and implementation of phishing attacks require little technical skills and costs. This uprising has led to an ever-growing number of phishing attacks on the World Wide Web. Consequently, proactive techniques to fight phishing attacks have become extremely necessary. In this paper, we propose HTMLPhish, a deep learning based data-driven end-to-end automatic phishing web page classification approach. Specifically, HTMLPhish receives the content of the HTML document of a web page and employs Convolutional Neural Networks (CNNs) to learn the semantic dependencies in the textual contents of the HTML. The CNNs learn appropriate feature representations from the HTML document embeddings without extensive manual feature engineering. Furthermore, our proposed approach of the concatenation of the word and character embeddings allows our model to manage new features and ensure easy extrapolation to test data. We conduct comprehensive experiments on a dataset of more than 50,000 HTML documents that provides a distribution of phishing to benign web pages obtainable in the real-world that yields over 93% Accuracy and True Positive Rate. Also, HTMLPhish is a completely language-independent and client-side strategy which can, therefore, conduct web page phishing detection regardless of the textual language.

In this paper, we propose SolarSLAM, a batteryfree loop closure method for indoor localisation. Inertial Measurement Unit (IMU) based indoor localisation method has been widely used due to its ubiquity in mobile devices, such as mobile phones, smartwatches and wearable bands. However, it suffers from the unavoidable long term drift. To mitigate the localisation error, many loop closure solutions have been proposed using sophisticated sensors, such as cameras, laser, etc. Despite achieving high-precision localisation performance, these sensors consume a huge amount of energy. Different from those solutions, the proposed SolarSLAM takes advantage of an energy harvesting solar cell as a sensor and achieves effective battery-free loop closure method. The proposed method suggests the key-point dynamic time warping for detecting loops and uses robust simultaneous localisation and mapping (SLAM) as the optimiser to remove falsely recognised loop closures. Extensive evaluations in the real environments have been conducted to demonstrate the advantageous photocurrent characteristics for indoor localisation and good localisation accuracy of the proposed method.

Human activity recognition (HAR) is an important component in context awareness IoT applications such smart home, smart building etc. With the proliferation of WiFi-integrated devices, researchers exploit WiFi signals to recognize various human activities. In this work, we introduce a HAR as a Service (HARaaS) model for activity recognition services applied in IoT areas. HARaaS proposes a novel edge computing model in the concept of the Sensing as a Service (S2aaS) architecture to offer accurate and real-time activities recognition services with good energy efficiency. HARaaS distributes the resource-hungry computing workload i.e. training recognition model to edge terminals, and exploits the built-in intelligence of IoT devices. A WiFi-based activity recognition service is designed following the HARaaS architecture, and the lightweight machine learning and deep learning model are incorporated in the service for accurate activity recognition. Experiments are conducted and demonstrate the service achieves an activity recognition accuracy of 95% with extremely low latency and high energy efficiency.

Since ancient Greece, handshaking has been commonly practiced between two people as a friendly gesture to express trust and respect, or form a mutual agreement. In this article, we show that such physical contact can be used to bootstrap secure cyber contact between the smart devices worn by users. The key observation is that during handshaking, although belonged to two different users, the two hands involved in the shaking events are often rigidly connected, and therefore exhibit very similar motion patterns. We propose a novel key generation system, which harvests motion data during user handshaking from the wrist-worn smart devices such as smartwatches or fitness bands, and exploits the matching motion patterns to generate symmetric keys on both parties. The generated keys can be then used to establish a secure communication channel for exchanging data between devices. This provides a much more natural and user-friendly alternative for many applications, e.g., exchanging/sharing contact details, friending on social networks, or even making payments, since it doesn’t involve extra bespoke hardware, nor require the users to perform pre-defined gestures. We implement the proposed key generation system on off-the-shelf smartwatches, and extensive evaluation shows that it can reliably generate 128-bit symmetric keys just after around 1s of handshaking (with success rate >99%), and is resilient to different types of attacks including impersonate mimicking attacks, impersonate passive attacks, or eavesdropping attacks. Specifically, for real-time impersonate mimicking attacks, in our experiments, the Equal Error Rate (EER) is only 1.6% on average. We also show that the proposed key generation system can be extremely lightweight and is able to run in-situ on the resource-constrained smartwatches without incurring excessive resource consumption.

The activation function is introduced to determine the output of neural networks by mapping the resulting values of neurons into a specific range. The activation functions often suffer from ‘gradient vanishing’, ‘non zero-centred function outputs’, ‘exploding gradients’, and ‘dead neurons’, which may lead to deterioration in the classification performance. This paper proposes an activation function generation approach using the Takagi-Sugeno-Kang inference in an effort to address such challenges. In addition, the proposed method further optimises the coefficients in the activation function using the genetic algorithm such that the activation function can adapt to different applications. This approach has been applied to a digital forensics application of online grooming detection. The evaluations confirm the superiority of the proposed activation function for online grooming detection using an unbalanced data set.

Activity recognition is an important component of many pervasive computing applications. Radio-based activity recognition has the advantage that it does not have the privacy concern compared with camera-based solutions, and subjects do not have to carry a device on them. It has been shown channel state information (CSI) can be used for activity recognition in a device-free setting. With the proliferation of wireless devices, it is important to understand how radio frequency interference (RFI) can impact on pervasive computing applications. In this article, we investigate the impact of RFI on device-free CSI-based location-oriented activity recognition. We present data to show that RFI can have a significant impact on the CSI vectors. In the absence of RFI, different activities give rise to different CSI vectors that can be differentiated visually. However, in the presence of RFI, the CSI vectors become much noisier, and activity recognition also becomes harder. Our extensive experiments show that the performance may degrade significantly with RFI. We then propose a number of countermeasures to mitigate the impact of RFI and improve the performance. We are also the first to use complex-valued CSI along with the state-of-the-art Sparse Representation Classification method to enhance the performance in the environment with RFI.

Localisation is of importance for many applications. Our motivating scenarios are short-term construction work and emergency rescue. Not only is accuracy necessary, these scenarios also require rapid setup and robustness to environmental conditions. These requirements preclude the use of many traditional methods e.g. vision-based, laser-based, Ultra-wide band (UWB) and Global Positioning System (GPS)-based localisation systems. To solve these challenges, we introduce iMag, an accurate and rapidly deployable inertial magneto-inductive (MI) localisation system. It localises monitored workers using a single MI transmitter and inertial measurement units with minimal setup effort. However, MI location estimates can be distorted and ambiguous. To solve this problem, we suggest a novel method to use MI devices for sensing environmental distortions, and use these to correctly close inertial loops. By applying robust simultaneous localisation and mapping (SLAM), our proposed localisation method achieves excellent tracking accuracy, and can improve performance significantly compared with only using an inertial measurement unit (IMU) and MI device for localisation.

Face recognition is one of the important components of many smart devices apps, e.g., face unlocking, people tagging and games on smart phones, tablets, or smart glasses. Sparse Representation Classification (SRC) is a state-of-the-art face recognition algorithm, which has been shown to outperform many classical face recognition algorithms in OpenCV, e.g., Eigenface algorithm. The success of SRC is due to its use of 21 optimization, which makes SRC robust to noise and occlusions. Since 21 optimization is computationally intensive, SRC uses random projection matrices to reduce the dimension of the 21 problem. However, random projection matrices do not give consistent classification accuracy as they ignored the prior knowledge of the training set. In this paper, we propose to exploit the prior knowlege of the training set to improve the recognition accuracy. It first learns the optimized projection matrix from the training set to produce consistent recognition performance then applies 21-based classification based on the group sparsity structure of SRC to further improve the recognition accuracy. Our evaluations, based on publicly available databases and real experiment, show that face recognition using optimized projection matrix is 8-17 percent more accurate than its random counterpart and Eigenface algorithm, and the recognition accuracy can be further improved by up to 5 percent by exploiting group sparsity structure. Furthermore, the optimized projection matrix does not have to be re-calculated even if new faces are added to the training set. We implement the SRC with optimized projection matrix on Android smartphones and find that the computation of residuals in SRC is a severe bottleneck, taking up 85-90 percent of the computation time. To address this problem, we propose a method to compute the residuals approximately, which is 50 times faster with little sacrificing recognition accuracy. Lastly, we demonstrate the feasibility of our new algorithm by the implementation and evaluation of a new face unlocking app and show its robustness to variation of poses, facial expressions, lighting changes, and occlusions.

Prior research has shown the potential of device-free WiFi sensing for human activity recognition. In this paper, we show for the first time WiFi signals can also be used to uniquely identify people. There is strong evidence that suggests that all humans have a unique gait. An individual's gait will thus create unique perturbations in the WiFi spectrum. We propose a system called WiFi-ID that analyses the channel state information to extract unique features that are representative of the walking style of that individual and thus allow us to uniquely identify that person. We implement WiFi-ID on commercial off-the-shelf devices. We conduct extensive experiments to demonstrate that our system can uniquely identify people with average accuracy of 93% to 77% from a group of 2 to 6 people, respectively. We envisage that this technology can find many applications in small office or smart home settings.

Real-time target tracking is an important service provided by embedded camera networks. The first step in target tracking is to extract the moving targets from the video frames, which can be realised by using background subtraction. For a background subtraction method to be useful in embedded camera networks, it must be both accurate and computationally efficient because of the resource constraints on embedded platforms. This makes many traditional background subtraction algorithms unsuitable for embedded platforms because they use complex statistical models to handle subtle illumination changes. These models make them accurate but the computational requirement of these complex models is often too high for embedded platforms. In this paper, we propose a new background subtraction method which is both accurate and computationally efficient. We propose a baseline version which uses luminance only and then extend it to use colour information. The key idea is to use random projection matrics to reduce the dimensionality of the data while retaining most of the information. By using multiple datasets, we show that the accuracy of our proposed background subtraction method is comparable to that of the traditional background subtraction methods. Moreover, to show the computational efficiency of our methods is not platform specific, we implement it on various platforms. The real implementation shows that our proposed method is consistently better and is up to six times faster, and consume significantly less resources than the conventional approaches. Finally, we demonstrated the feasibility of the proposed method by the implementation and evaluation of an end-to-end real-time embedded camera network target tracking application.

Activity recognition is an important component of many pervasive computing applications. Device-free activity recognition has the advantage that it does not have the privacy concern of using cameras and the subjects do not have to carry a device on them. Recently, it has been shown that channel state information (CSI) can be used for activity recognition in a device-free setting. With the proliferation of wireless devices, it is important to understand how radio frequency interference (RFI) can impact on pervasive computing applications. In this paper, we investigate the impact of RFI on device-free CSI-based location-oriented activity recognition. We conduct experiments in environments without and with RFI. We present data to show that RFI can have a significant impact on the CSI vectors. In the absence of RFI, different activities give rise to different CSI vectors that can be differentiated visually. However, in the presence of RFI, the CSI vectors become much noisier and activity recognition also becomes harder. Our extensive experiments shows that the performance of state-of-the-art classification methods may degrade significantly with RFI. We then propose a number of counter measures to mitigate the impact of RFI and improve the location-oriented activity recognition performance. Our evaluation shows the proposed method can improve up to 10% true detection rate in the presence of RFI. We also study the impact of bandwidth on activity recognition performance. We show that with a channel bandwidth of 20 MHz (which is used by WiFi), it is possible to achieve a good activity recognition accuracy when RFI is present.

Radio tomographic imaging (RTI) enables device free localisation of people and objects in many challenging environments and situations. Its basic principle is to detect the changes in the statistics of radio signals due to the radio link obstruction by people or objects. However, the localisation accuracy of RTI suffers from complicated multipath propagation behaviours in radio links. We propose to use inexpensive and energy efficient electronically switched directional (ESD) antennas to improve the quality of radio link behaviour observations, and therefore, the localisation accuracy of RTI. We implement a directional RTI (dRTI) system to understand how directional antennas can be used to improve RTI localisation accuracy. We also study the impact of the choice of antenna directions on the localisation accuracy of dRTI and propose methods to effectively choose informative antenna directions to improve localisation accuracy while reducing overhead. Furthermore, we analyse radio link obstruction performance in both theory and simulation, as well as false positives and false negatives of the obstruction measurements to show the superiority of the directional communication for RTI. We evaluate the performance of dRTI in diverse indoor environments and show that dRTI significantly outperforms the existing RTI localisation methods based on omni-directional antennas.

The information-preserving sampling properties of compressive sensing have found a number of successful applications, such as sensor scheduling, localisation and tracking to deal with the resource constraints of the embedded systems. In this paper, we investigate an approach to improve the performance of compressive sensing applications through a novel strategy for optimising the projection matrix. We formulate the projection matrix optimisation problem and apply greedy algorithm to solve the optimisation problem efficiently. We evaluate the proposed approach by an emerging background subtraction method designed specifically for the embedded systems and show the proposed approach outperforms existing approaches significantly with little overhead.

Acoustic Sensor Networks (ASNs) have a wide range of applications in natural and urban environment monitoring, as well as indoor activity monitoring. In-network classification is critically important in ASNs because wireless transmission costs several orders of magnitude more energy than computation. The main challenges of in-network classification in ASNs include effective feature selection, intensive computation requirement and high noise levels. To address these challenges, we propose a sparse representation based feature-less, low computational cost, and noise resilient framework for in-network classification in ASNs. The key component of Sparse Approximation based Classification (SAC), ℓ1 minimization, is a convex optimization problem, and is known to be computationally expensive. Furthermore, SAC algorithms assumes that the test samples are a linear combination of a few training samples in the training sets. For acoustic applications, this results in a very large training dictionary, making the computation infeasible to be performed on resource constrained ASN platforms. Therefore, we propose several techniques to reduce the size of the problem, so as to fit SAC for in-network classification in ASNs. Our extensive evaluation using two real-life datasets (consisting of calls from 14 frog species and 20 cricket species respectively) shows that the proposed SAC framework outperforms conventional approaches such as Support Vector Machines (SVMs) and k-Nearest Neighbor (kNN) in terms of classification accuracy and robustness. Moreover, our SAC approach can deal with multi-label classification which is common in ASNs. Finally, we explore the system design spaces and demonstrate the real-time feasibility of the proposed framework by the implementation and evaluation of an acoustic classification application on an embedded ASN testbed.

Sparse approximation has now become a buzzword for classification in numerous research domains. We propose a distributed sparse approximation method based on l1 minimization for frog sound classification, which is tailored to the resource constrained wireless sensor networks. Our pilot study demonstrates that l1 minimization can run on wireless sensor nodes producing satisfactory classification accuracy.

Background subtraction is often the first step of many computer vision applications. For a background subtraction method to be useful in embedded camera networks, it must be both accurate and computationally efficient because of the resource constraints on embedded platforms. This makes many traditional background subtraction algorithms unsuitable for embedded platforms because they use complex statistical models to handle subtle illumination changes. These models make them accurate but the computational requirement of these complex models is often too high for embedded platforms. In this paper, we propose a new background subtraction method which is both accurate and computational efficient. The key idea is to use compressive sensing to reduce the dimensionality of the data while retaining most of the information. By using multiple datasets, we show that the accuracy of our proposed background subtraction method is comparable to that of the traditional background subtraction methods. Moreover, real implementation on an embedded camera platform shows that our proposed method is at least 5 times faster, and consumes significantly less energy and memory resources than the conventional approaches. Finally, we demonstrated the feasibility of the proposed method by the implementation and evaluation of an end-to-end real-time embedded camera network target tracking application.