Interpretability is crucial for ensuring RL systems align with human values. However, it remains challenging to achieve in complex decision making domains. Existing methods frequently attempt interpretability at the level of fundamental model units, such as neurons or decision nodes: an approach which scales poorly to large models. Here, we instead propose an approach to interpretability at the level of functional modularity. We show how encouraging sparsity and locality in network weights leads to the emergence of functional modules in RL policy networks. To detect these modules, we develop an extended Louvain algorithm which uses a novel `correlation alignment' metric to overcome the limitations of standard network analysis techniques when applied to neural network architectures. Applying these methods to 2D and 3D MiniGrid environments reveals the consistent emergence of distinct navigational modules for different axes, and we further demonstrate how these functions can be validated through direct interventions on network weights prior to inference.

The development of safe and reliable autonomous autonomous aerial vehicles (AAVs) relies on the ability of the system to recognize and adapt to changes in the local environment based on sensor inputs. State-of-the-art local tracking and trajectory planning are typically performed using camera sensor input to the flight control algorithm, but the extent to which environmental disturbances like rain affect the performance of these systems is largely unknown. In this article, we first describe the development of an open dataset comprising ~335k images to examine these effects for seven different classes of precipitation conditions and show that a worst case average tracking error of 1.5 m is possible for a state-of-the-art visual odometry system (VINS-Fusion). We then use the dataset to train a set of deep neural network models suited to mobile and constrained deployment scenarios to determine the extent to which it may be possible to efficiently and accurately classify these “rainy” conditions. The most lightweight of these models (MobileNetV3 small) can achieve an accuracy of 90% with a memory footprint of just 1.28 MB and a frame rate of 93 FPS, which is suitable for deployment in resource-constrained and latency-sensitive systems. We demonstrate a classification latency in the order of milliseconds using typical flight computer hardware. Accordingly, such a model can feed into the disturbance estimation component of an autonomous flight controller. In addition, data from AAVs with the ability to accurately determine environmental conditions in real time may contribute to developing more granular timely localized weather forecasting.

The orienteering problem (OP) is a well-studied routing problem that has been extended to incorporate uncertainties, reflecting stochastic or dynamic travel costs, prize-collection costs, and prizes. Existing approaches may, however, be inefficient in real-world applications due to insufficient modeling knowledge and initially unknowable parameters in online scenarios. Thus, we propose the uncertain and dynamic OP (UDOP), modeling travel costs as distributions with unknown and time-variant parameters. UDOP also associates uncertain travel costs with dynamic prizes and prize-collection costs for its objective and budget constraints. To address UDOP, we develop an Adaptive Approach for Probabilistic paths (ADAPT), iteratively performing “execution” and “online planning” based on an initial “offline” solution. The execution phase updates the system status and records online cost observations. The online planner employs a Bayesian approach to adaptively estimate power consumption and optimize path sequence based on safety beliefs. We evaluate ADAPT in a practical autonomous aerial vehicle charging scheduling problem (CSP) for wireless rechargeable sensor networks (WRSNs). The UAV must optimize its path to recharge sensor nodes efficiently while managing its energy under uncertain conditions. ADAPT maintains comparable solution quality and computation time while offering superior robustness. Extensive simulations show that ADAPT achieves a 100% mission success rate (MSR) across all tested scenarios, outperforming comparable heuristic-based and frequentist approaches that fail up to 70% (under challenging conditions) and averaging 67% MSR, respectively. This work advances the field of OP with uncertainties, offering a reliable and efficient approach for real-world applications in uncertain and dynamic environments.

Simultaneously accurate and reliable tracking control for quadrotors in complex dynamic environments is challenging. The chaotic nature of aerodynamics, derived from drag forces and moment variations, makes precise identification difficult. Consequently, many existing quadrotor tracking systems treat these aerodynamic effects as simple ‘disturbances’ in conventional control approaches. We propose a novel and interpretable trajectory tracker integrating a distributional Reinforcement Learning (RL) disturbance estimator for unknown aerodynamic effects with a Stochastic Model Predictive Controller (SMPC). Specifically, the proposed estimator ‘Constrained Distributional REinforced-Disturbance-estimator’ (ConsDRED) effectively identifies uncertainties between the true and estimated values of aerodynamic effects. Control parameterization employs simplified affine disturbance feedback to ensure convexity, which is seamlessly integrated with the SMPC. We theoretically guarantee that ConsDRED achieves an optimal global convergence rate, and sublinear rates if constraints are violated with certain error decreases as neural network dimensions increase. To demonstrate practicality, we show convergent training, in simulation and real-world experiments, and empirically verify that ConsDRED is less sensitive to hyperparameter settings compared with canonical constrained RL. Our system substantially improves accumulative tracking errors by at least 70%, compared with the recent art. Importantly, the proposed ConsDRED-SMPC framework balances the trade-off between pursuing high performance and obeying conservative constraints for practical implementations. Note to Practitioners—This work is motivated by challenges in training Reinforcement Learning (RL) for autonomous navigation in unmanned aerial vehicles, but its implications extend to other high-criticality applications in, for example, healthcare and financial services. The implementation of RL algorithm policies may exhibit various deficiencies, including (i) opaque or unstable training due to the blackbox nature of deep neural networks, (ii) difficulty in reproducing RL outcomes, and (iii) heightened costs associated with robustness investigation, e.g., hyperparameter tuning and generalization. We present a novel ‘Constrained Distributional REinforced-Disturbance-estimator’ (ConsDRED) to identify uncertainties arising from aerodynamic disturbances in agile flight. The proposed algorithm demonstrates at least an optimal convergence rate for both global optimization and constraint violations. Theoretical guarantees offered by this approach can make training convergence processes more transparent and provide a degree of confidence in the expected convergence outcomes in real-world applications, thereby addressing, to a considerable extent, the deficiencies related to (i) and (ii). Our robustness investigation, related to (iii), empirically demonstrates that ConsDRED is significantly less sensitive to hyperparameter settings compared to traditional constrained RL approaches. The experiments further showcase the generalization capability of ConsDRED by introducing new and previously unseen external forces in real-world scenarios. While ConsDRED strikes a reasonable balance between performance and interpretability, there remains a trade-off with conservative decision-making. Hence, we must strike a balance between the desired degree of interpretability and the acceptable level of scalability in RL algorithms. Our proposed flight control framework evidences such a balance between high performance and safety constraints. A fundamental limitation is the computational complexity, where we sample at 16 Hz for disturbance estimation owing to limited onboard computational resources. This may be improved in the future by implementing dedicated hardware.

Reinforcement learning can provide effective reasoning for sequential decision-making problems with variable dynamics. Such reasoning in practical implementation, however, poses a persistent challenge in interpreting the reward function and the corresponding optimal policy. Consequently, representing sequential decision-making problems as probabilistic inference can have considerable value, as, in principle, the inference offers diverse and powerful mathematical tools to infer the stochastic dynamics whilst suggesting a probabilistic interpretation of policy optimization. In this study, we propose a novel Adaptive Wasserstein Variational Optimization, namely AWaVO, to tackle these interpretability challenges. Our approach uses formal methods to achieve the interpretability: convergence guarantee, training transparency, and intrinsic decision-interpretation. To demonstrate its practicality, we showcase guaranteed interpretability including a global convergence rate $\Theta(1/\sqrt{T})$ not only in simulation but also in real-world quadrotor tasks. In comparison with state-of-the-art benchmarks, including TRPO-IPO, PCPO, and CRPO, we empirically verify that AWaVO offers a reasonable trade-off between high performance and sufficient interpretability.

Voice user interfaces and digital assistants are rapidly entering our lives and becoming singular touch points spanning our devices. These always-on services capture and transmit our audio data to powerful cloud services for further processing and subsequent actions. Our voices and raw audio signals collected through these devices contain a host of sensitive paralinguistic information that is transmitted to service providers regardless of deliberate or false triggers. As our emotional patterns and sensitive attributes like our identity, gender, and well-being are easily inferred using deep acoustic models, we encounter a new generation of privacy risks by using these services. One approach to mitigate the risk of paralinguistic-based privacy breaches is to exploit a combination of cloud-based processing with privacy-preserving, on-device paralinguistic information learning and filtering before transmitting voice data. In this article we introduce EDGY , a configurable, lightweight, disentangled representation learning framework that transforms and filters high-dimensional voice data to identify and contain sensitive attributes at the edge prior to offloading to the cloud. We evaluate EDGY’s on-device performance and explore optimization techniques, including model quantization and knowledge distillation, to enable private, accurate, and efficient representation learning on resource-constrained devices. Our results show that EDGY runs in tens of milliseconds with 0.2% relative improvement in “zero-shot” ABX score or minimal performance penalties of approximately 5.95% word error rate (WER) in learning linguistic representations from raw voice signals, using a CPU and a single-core ARM processor without specialized hardware.

Unmapped areas and aerodynamic disturbances render autonomous navigation with quadrotors extremely challenging. To fly safely and efficiently, trajectory planners and trackers must be able to navigate unknown environments with unpredictable aerodynamic effects in real-time. When encountering aerodynamic effects such as strong winds, most current approaches to quadrotor trajectory planning and tracking will not attempt to deviate from a determined plan, even if it is risky, in the hope that any aerodynamic disturbances can be resisted by a robust controller. This paper presents a novel systematic trajectory planning and tracking framework for autonomous quadrotors. We propose a Kinodynamic Jump Space Search (Kino-JSS) to generate a safe and efficient route in unknown environments with aerodynamic disturbances. A real-time Gaussian Process is employed to model the errors caused by aerodynamic disturbances, which we then integrate with a Model Predictive Controller to achieve efficient and accurate trajectory optimization and tracking. We demonstrate our system to improve the efficiency of trajectory generation in unknown environments by up to 75% in the cases tested, compared with recent state-of-the-art. We also show that our system improves the accuracy of tracking in selected environments with unpredictable aerodynamic effects. Our implementation is available in an open source package11https://github.com/Alex-yanranwang/Imperial-KinoJGM.

Sensor installation on water infrastructure is challenging due to requirements for service interruption, specialized personnel, regulations and reliability as well as the resultant high costs. Here, a minimally invasive installation method is introduced based on hot-tapping and immersion of a sensor probe. A modular architecture is developed that enables the use of interchangeable multisensor probes, nonspecialist installation and servicing, low-power operation and configurable sensing and connectivity. A prototype implementation with a temperature, pressure, conductivity and flow multisensor probe is presented and tested on an evaluation rig. This article demonstrates simple installation, reliable and accurate sensing capability as well as remote data acquisition. The demonstrated minimally invasive multisensor probes provide an opportunity for the deployment of water quality sensors that typically require immersion, such as pH and spectroscopic composition analysis. This design allows dynamic deployment on existing water infrastructure with expandable sensing capability and minimal interruption, which can be key to addressing important sensing parameters, such as optimal sensor network density and topology.

Fabricating 3D printed electronics using desktop printers has become more accessible with recent developments in conductive thermoplastic filaments. Because of their high resistance and difficulties in printing traces in vertical directions, most applications are restricted to capacitive sensing. In this paper, we introduce Thermoformed Circuit Board (TCB), a novel approach that employs the thermoformability of the 3D printed plastics to construct various double-sided, rigid and highly conductive freeform circuit boards that can withstand high current applications through copper electroplating. To illustrate the capability of the TCB, we showcase a range of examples with various shapes, electrical characteristics and interaction mechanisms. We also demonstrate a new design tool extension to an existing CAD environment that allows users to parametrically draw the substrate and conductive trace, and export 3D printable files. TCB is an inexpensive and highly accessible fabrication technique intended to broaden HCI researcher participation.

Voice assistive technologies have given rise to far-reaching privacy and security concerns. In this paper we investigate whether modular automatic speech recognition (ASR) can improve privacy in voice assistive systems by combining independently trained separation, recognition, and discretization modules to design configurable privacy-preserving ASR systems. We evaluate privacy concerns and the effects of applying various state-of-the-art techniques at each stage of the system, and report results using task-specific metrics (i.e. WER, ABX, and accuracy). We show that overlapping speech inputs to ASR systems present further privacy concerns, and how these may be mitigated using speech separation and optimization techniques. Our discretization module is shown to minimize paralinguistics privacy leakage from ASR acoustic models to levels commensurate with random guessing. We show that voice privacy can be configurable, and argue this presents new opportunities for privacy-preserving applications incorporating ASR.

Many industrial-Internet-of-Things applications require autonomous operation and incorporate devices in inaccessible locations. Recent advances in wireless power transfer (WPT) and autonomous vehicle technologies, in combination, have the potential to solve a number of residual problems concerning the maintenance of, and data collection from embedded devices. Equipping inexpensive unmanned aerial vehicles (UAV) and embedded devices with subsystems to facilitate WPT allows a UAV to become a viable mobile power delivery vehicle (PDV) and data collection agent. A key challenge is, therefore, to ensure that a PDV can optimally schedule power delivery across the network, such that it is as reliable and resource efficient as possible. To achieve this and out-perform naive on-demand recharging strategies, in this article, we propose a two-stage wireless power network (WPN) approach in which a large network of devices may be grouped into small clusters, where packets of energy inductively delivered to each cluster by the PDV are acoustically distributed to devices within the cluster. In this article, we describe a novel dynamic recharge scheduling algorithm that combines genetic weighted clustering with nearest neighbor search to jointly minimize PDV travel distance and WPT losses. The efficacy and performance of the algorithm are evaluated in simulation using experimentally derived traces, and the algorithm is shown to achieve ~ 90% throughput for large, dense networks.

LoRaWAN is one of the leading Low Power Wide Area Network (LPWAN) architectures. It was originally designed for systems consisting of static sensor or Internet of Things (IoT) devices and static gateways. It was recently updated to introduce new features such as nano-second timestamps which open up applications to enable LoRaWAN to be adopted for mobile device tracking and localisation. In such mobile scenarios, devices could temporarily lose communication with the gateways because of interference from obstacles or deep fading, causing throughput reduction and delays in data transmission. To overcome this problem, we propose a new data forwarding scheme. Instead of holding the data until the next contact with gateways, devices can forward their data to nearby devices that have a higher probability of being in contact with gateways. We propose a new network metric called Real-Time Contact-Aware Expected Transmission Count (RCA-ETX) to model this contact probability in real-time. Without making any assumption on mobility models, this metric exploits data transmission delays to model complex device mobility. We also extend RCA-ETX with a throughput-optimal stochastic backpressure routing scheme and propose Real-Time Opportunistic Backpressure Collection (ROBC), a protocol to counter the stochastic behaviours resulting from the dynamics associated with mobility. To apply our approaches seamlessly to LoRaWAN-enabled devices, we further propose two new LaRaWAN classes, namely Modified Class-C and Queue-based Class-A. Both of them are compatible with LoRaWAN Class-A devices. Our data-driven experiments, based on the London bus network, show that our approaches can reduce data transmission delays up to 25% and provide a 53% throughput improvement in data transfer performance.

The number of IoT devices continues to grow despite the alarming rate of identification of security and privacy issues. There is widespread concern that development of IoT devices is performed without sufficient attention paid to security and privacy issues. Consequently, networks have a higher probability of incorporating vulnerable IoT devices that may be easy to compromise to launch cyber attacks. Inclusion of IoT devices paves the way for a new category of anomalies to be introduced to networks. Traditional anomaly detection techniques (e.g., semi-supervised and signature-based methods), however, are likely inefficient in detecting IoT-based anomalies. This is because these techniques require static signatures of known attacks, specialized hardware, or full packet inspection. They are also expensive, and may be inaccurate or unscalable. Vulnerable IoT devices can be used to perform destructive attacks or invade privacy. The ability to find anomalies in IoT traffic has the potential to assist with early detection and deployment of countermeasures to thwart such attacks. Thus, new techniques for detecting infected IoT devices are needed to mitigate the associated security and privacy risks. In this research, we investigate the possibility to identify IoT traffic using a combination of behavioural profile, predefined blocklist and device fingerprint. Such a system may be able to detect anomalous and/or malicious devices and/or traffic reliably and quickly. Initial results show that for our implementation of such a system, IoT traffic can be identified using device behaviour profile, fingerprint, and contacted destinations. This work takes the first step towards designing and evaluating iDetector, a framework that can detect anomalous behaviour within IoT networks. In our experiments, iDetector was able to correctly identify 80--90% of all captured traffic traversing a home gateway.

Voice controlled devices and services are commonplace in consumer IoT. Cloud-based analysis services extract information from voice input using speech recognition techniques. Services providers can build detailed profiles of users' demographics, preferences and emotional states, etc., and may therefore significantly compromise privacy. To address this problem, a privacy-preserving intermediate layer between users and cloud services is proposed to sanitize voice input directly at edge devices by generating neutralized signals for forwarding. We show that a trained model, based on CycleGAN and deployed on a Raspberry Pi, enables identification and removal of sensitive emotional state information by ~91%, with minimal losses to speech recognition accuracy.

This paper describes the design, implementation and evaluation of a first of its kind cross-layer protocol for wireless communication between flying agents and terrestrial wireless sensors. The protocol is composed of three layers: a new application layer built upon a modified implementation of ContikiMAC over the IEEE 802.15.4 2.4 GHz physical layer. The experimental evaluation shows the protocol to have excellent energy efficiency, low latency and high reliability-approaching 100% for certain parameter settings and operational conditions. The effects of speed, altitude, and direction of approach are also experimentally evaluated, demonstrating that it is of critical importance to take these into account when planning mobile aerial data collection campaigns.

Sensing systems incorporating unmanned aerial vehicles have the potential to enable a host of hitherto impractical monitoring applications using wireless sensors in remote and extreme environments. Their use as data collection and power delivery agents can overcome challenges such as poor communications reliability in difficult RF environments and maintenance in areas dangerous for human operatives. Aerial interaction with wireless sensors presents some interesting new challenges, including selecting or designing appropriate communications protocols that must account for unique practicalities like the effects of velocity and altitude. This poster presents a practical evaluation of the effects of altitude when collecting sensor data using an unmanned aerial vehicle. We show that for an otherwise disconnected link over a long distance (70m), by increasing altitude (5m) the link is created and its signal strength continues to improve over tens of metres. This has interesting implications for protocol design and optimal aerial route planning.

Pervasive sensing-the capability to deploy large numbers of sensors, to link them to communication networks, and to analyze their collective data-is transforming many industries. In mining, networked sensors are already used for remote operation, automation, including driverless vehicles, health, and safety, and exploration. In this paper, the state-of-the-art sensing and monitoring technologies are assessed as solutions against the main challenges and opportunities in the mining industry. Localization, mapping, remote operation, maintenance, and health and safety are identified as the main beneficiaries from rapidly developing technologies, such as 3-D visualization, augmented reality, energy autonomous sensor nodes, distributed sensing, smart network protocols, and big data analytics. It is shown that the identification and management of ore grade, in particular, which transcends each stage of the mining process, may critically benefit from certain arising sensing technologies, where major efficiency improvements are possible in exploration, extraction, haulage, and processing activities.

A variety of wireless networks, including applications of Wireless Sensor Networks, Internet of Things, and Cyber-physical Systems, increasingly pervade our homes, retail, transportation systems, and manufacturing processes. Traditional approaches communicate data from all sensors to a central system, and users (humans or machines) query this central point for results, typically via the web. As the number of deployed sensors, and thus generated data streams, is increasing exponentially, this traditional approach may no longer be sustainable or desirable in some application contexts. Therefore, new approaches are required to allow users to directly interact with the network, for example, requesting data directly from sensor nodes. This is difficult, as it requires every node to be capable of point-to-point routing, in addition to identifying a subset of nodes that can fulfil a user's query. This paper presents Dragon, a platform that allows any node in the network to identify all nodes that satisfy user queries, i.e., request data from nodes, and relay the result to the user. The Dragon platform achieves this in a fully distributed way. No central orchestration is required, network overheads are low, and latency is improved over existing comparable methods. Dragon is evaluated on networks of various topologies and different network densities. It is compared with the state-of-the-art algorithms based on summary trees, like Innet and SENS-Join. Dragon is shown to outperform these approaches up to 88% in terms of network traffic required, also a proxy for energy efficiency, and 84% in terms of processing delay.

Nodes in wireless sensor networks (WSNs) typically have limited power supply and networks are often expected to be functional for extended periods. Therefore, the minimization of energy consumption and the maximization of network lifetime are key objectives in WSN. This paper proposes an overlay, energy optimized, sensor network to extend the functional lifetime of an energy-intensive sensor network application. The overlay network consists of additional nodes that exploit recent advances in energy harvesting and wake-up radio technologies, coupled with an application specific, complementary, ultra-low power sensor. The experimental results and simulations demonstrate that this approach can ensure survivability of energy-inefficient sensor networks. Simulating applications using energy-intensive video cameras and air quality sensors, combined with the proposed overlayed ultra-low power sensor network, demonstrates that this approach can increase functional lifetime toward perpetual operation and is suitable for WSN applications in which complementarity exists between the required energy-intensive sensors and low-cost sensors that can be used as triggers.

Body Sensor Network (BSN) technology is seeing a rapid emergence in application areas such as health, fitness and sports monitoring. Current BSN wireless sensors typically operate on a single frequency band (e.g. utilizing the IEEE 802.15.4 standard that operates at 2.45GHz) employing a single radio transceiver for wireless communications. This allows a simple wireless architecture to be realized with low cost and power consumption. However, network congestion/failure can create potential issues in terms of reliability of data transfer, quality-of-service (QOS) and data throughput for the sensor. These issues can be especially critical in healthcare monitoring applications where data availability and integrity is crucial. The addition of more than one radio has the potential to address some of the above issues. For example, multi-radio implementations can allow access to more than one network, providing increased coverage and data processing as well as improved interoperability between networks. A small number of multi-radio wireless sensor solutions exist at present but require the use of more than one radio transceiver devices to achieve multi-band operation. This paper presents the design of a novel prototype multi-radio hardware platform that uses a single radio transceiver. The proposed design allows multi-band operation in the 433/868MHz ISM bands and this, together with its low complexity and small form factor, make it suitable for a wide range of BSN applications.