Wireless Acoustic Sensor Networks (WASNs), or the Internet of Audio Things (IoAuT), enable intelligent acoustic sensing in IoT applications such as smart homes. A key challenge in such deployments is achieving accurate results under limited bandwidth and energy constraints. To efficiently process audio, sensor fusion techniques can be used. They can aggregate the raw audio signals, features, or local decisions of all sensors to make the final decision for tasks such as acoustic event classification. In contrast to a wired setting, connections within WASNs may be unstable due to interference. Additionally, transmitting large amounts of data reduces the sensors’ battery lifespan. A data/signal-level fusion preserves the full information by transmitting and fusing the raw audio, but it imposes an impractical bandwidth burden for WASNs. Conversely, decision-level fusion is communication-efficient and supports a variable number of sensors, it may compromise accuracy. In contrast, existing feature-level fusion methods can transmit richer information to the fusion center, but incur a higher communication overhead and often necessitate a fixed sensor topology, rendering them less suitable for wireless IoT settings. In this work, we propose a new feature-level fusion framework based on Graph Attention Networks (GATs) for acoustic event classification tasks using a WASN. Our approach supports dynamic WASN topologies and introduces a message condensation layer that reduces the volume of transmitted data, lowering the communication cost and bandwidth usage. Empirical results of a domestic acoustic event classification task show that our framework outperforms decision-level fusion techniques while maintaining a similar communication cost. Moreover, our framework outperforms prior feature-level and data-level fusion methods with a notably reduced communication cost by reducing the size of transmitted messages, and the additional flexibility of supporting dynamic WASN topologies, thus being robust to sensor failures, or the addition or removal of sensors.

Communication is crucial in multi-agent reinforcement learning when agents are not able to observe the full state of the environment. The most common approach to allow learned communication between agents is the use of a differentiable communication channel that allows gradients to flow between agents as a form of feedback. However, this is challenging when we want to use discrete messages to reduce the message size, since gradients cannot flow through a discrete communication channel. Previous work proposed methods to deal with this problem. However, these methods are tested in different communication learning architectures and environments, making it hard to compare them. In this paper, we compare several state-of-the-art discretization methods as well as a novel approach. We do this comparison in the context of communication learning using gradients from other agents and perform tests on several environments. In addition, we present COMA-DIAL, a communication learning approach based on DIAL and COMA extended with learning rate scaling and adapted exploration. COMA-DIAL uses COMA to learn the action policy while it uses the mechanism introduced in DIAL to learn the communication policy. Using COMA-DIAL allows us to perform experiments on more complex environments. Our results show that the novel ST-DRU method, proposed in this paper, achieves the best results out of all discretization methods across the different environments. It achieves the best or close to the best performance in each of the experiments and is the only method that does not fail on any of the tested environments.

Continual learning (CL) at the edge requires the model to learn from sequentially arriving small batches of data.A naive online learning strategy fails due to the catastrophic forgetting phenomenon.Previous literature introduced the 'latent replay' for CL at the edge, where the input is transformed into latent representations using a pre-trained feature extractor.These latent representations are used, in combination with the real inputs, to train the adaptive classification layers.This approach is prone to the stability gap problem, where the accuracies of learned classes drop when learning a new class, and they only recover during subsequent training iterations.We hypothesize that this is caused by the class imbalance between new class data from the new task, and the old class data in the replay memory.We validate this by applying two class balancing strategies in a latent replay-based CL method.Our empirical results demonstrate that class balancing strategies provide a notable accuracy improvement, and a reduction of the stability gap when using a latent replay-based CL method with a small replay memory size.

In Reinforcement Learning, Unsupervised Skill Discovery tackles the learning of several policies for downstream task transfer. Once these skills are learnt, the question of how best to use and combine them remains an open problem. The General Policy Improvement Theorem (GPI) creates a policy stronger than any individual skill by selecting the highest-valued policy, generally evaluated with Successor Features. However, the GPI policy is unable to mix and combine the skills at decision time to formulate stronger plans. In this paper, we propose to adopt a model-based setting in order to make such planning possible, and formally show that a forward search improves on the GPI policy and any shallower searches under some approximation term. We argue for decision-time planning, and design a family of algorithms, GPI-Tree Search Algorithms , to use Monte Carlo Tree Search (MCTS) with GPI. These algorithms foster the skills and Q-value priors of the GPI framework to guide and improve the search, which we back up with visual intuition for the different design choices. Our experiments show that the resulting policies are much stronger than the GPI policy alone, even under approximation; they can also improve beyond the linear constraint of Successor Features.

Learning to communicate in order to share state information is an active problem in the area of multi-agent reinforcement learning. The credit assignment problem, the non-stationarity of the communication environment and the problem of encouraging the agents to be influenced by incoming messages are major challenges within this research field which need to be overcome in order to learn a valid communication protocol. This paper introduces the novel multi-agent counterfactual communication learning (MACC) method which adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents. Next, the non-stationarity of the communication environment, while learning the communication Q -function, is overcome by creating the communication Q -function using the action policy of the other agents and the Q -function of the action environment. As the exact method to create the communication Q -function can be computationally intensive for a large number of agents, two approximation methods are proposed. Additionally, a social loss function is introduced in order to create influenceable agents, which is required to learn a valid communication protocol. Our experiments show that MACC is able to outperform the state-of-the-art baselines in four different scenarios in the particle environment. Finally, we demonstrate the scalability of MACC in a matrix environment.

Post-training quantization (PTQ) is an effective solution for deploying deep neural networks on edge devices with limited resources. PTQ is especially attractive because it does not require access to the entire original training dataset on the promise of being able to use a much smaller calibration dataset. However, many existing PTQ methods still require a sufficiently large calibration dataset (e.g., more than 1000 images) to achieve satisfactory model accuracy. In this paper, we present a novel post-training quantization method that estimates quantization parameters using a Bayesian Maximum A Posterior (MAP) estimator. By modeling the uncertainty of quantization operations, we formulate the neural network quantization as a Bayesian inference problem. In our method, we first employ probabilistic programming techniques to optimize quantization parameters by maximizing the posterior of quantization step sizes. In addition, we introduce a Minimum Description Length (MDL) prior that favors low quantization bit widths and a validation procedure, which enhances PTQ performance when learning from small calibration datasets. Comprehensive evaluations demonstrate that the proposed method can improve the PTQ performance using a minimal calibration dataset of just 64 images, and achieve nearly state-of-the-art PTQ performance. Furthermore, the proposed method shows strong generalization ability when calibrated on different data sources and tested across diverse data.

Modern networks, characterized by their complexity and heterogeneity, are transitioning from manual to automated and intelligent management, according to the vision of Autonomous Networks (ANs). Leveraging data-driven techniques, ANs aim to provide the "Zero-X" and "Self-X" experience, where intelligent and adaptable network operations are key cornerstones. Such is the case of intelligent resource scaling, where the goal is to optimize resource orchestration to maximize efficiency, reduce latency, and maintain high-quality service, even amid fluctuating network loads and changing service requirements. Unfortunately, current approaches for auto-scaling are computationally expensive to deploy in resource-constrained devices such as those found at the edge or beyond. This paper introduces an innovative four-phase approach to train a compact, data-driven Deep Reinforcement Learning (DRL) scaler that can be deployed on low-power devices. Our results demonstrate the scalability and efficiency of this model, achieving state-of-the-art scaling with up to 1003x fewer parameters, enhancing interpretability and computational efficiency, making it a robust solution for intelligent resource scaling in network environments. The resulting 1487x runtime speed improvement and 28.5x reduction in memory requirements allow the scaler to be deployed on low-power devices and still operate in real-time, which is essential for mission-critical and latency-sensitive applications.

Automated Guided Vehicles (AGV) are omnipresent, and are able to carry out various kind of preprogrammed tasks. Unfortunately, a lot of manual configuration is still required in order to make these systems operational, and configuration needs to be re-done when the environment or task is changed. As an alternative to current inflexible methods, we employ a learning based method in order to perform directed exploration of a previously unseen environment. Instead of relying on handcrafted heuristic representations, the agent learns its own environmental representation through its embodiment. Our method offers loose coupling between the Reinforcement Learning (RL) agent, which is trained in simulation, and a separate, on real-world images trained task module. The uncertainty of the task module is used to direct the exploration behavior. As an example, we use a warehouse inventory task, and we show how directed exploration can improve the task performance through active data collection. We also propose a novel environment representation to efficiently tackle the sim2real gap in both sensing and actuation. We empirically evaluate the approach both in simulated environments and a real-world warehouse.

To enable autonomous vessels to operate on inland waterways, they need to detect, track and localize objects at close range to safely navigate. We deployed current deep learning techniques to detect and track these objects. As there are no large labeled datasets of European inland waterways, we used transfer learning to overcome the lack of data. By using preexisting similar datasets, we were able to significantly decrease the required amount of labeled data from the target distribution. Furthermore, we improved the mean Average Precision from 0.461 to 0.814 by using a limited number of labeled target data samples. We estimated the relative distance of the objects based on the generated bounding boxes. The information from the camera is then combined with LiDar data to generate a top-view map of the environment which is used as input for an object-avoidance control agent. All these methods can run in real-time on the vessel with an fps of 1.83 on a 2.7GHz vCPU.

In recent years, interest in autonomous shipping in urban waterways has increased significantly due to the trend of keeping cars and trucks out of city centers. Classical approaches such as Frenet frame based planning and potential field navigation often require tuning of many configuration parameters and sometimes even require a different configuration depending on the situation. In this paper, we propose a novel path planning approach based on reinforcement learning called Model Predictive Reinforcement Learning (MPRL). MPRL calculates a series of waypoints for the vessel to follow. The environment is represented as an occupancy grid map, allowing us to deal with any shape of waterway and any number and shape of obstacles. We demonstrate our approach on two scenarios and compare the resulting path with path planning using a Frenet frame and path planning based on a proximal policy optimization (PPO) agent. Our results show that MPRL outperforms both baselines in both test scenarios. The PPO based approach was not able to reach the goal in either scenario while the Frenet frame approach failed in the scenario consisting of a corner with obstacles. MPRL was able to safely (collision free) navigate to the goal in both of the test scenarios.

Dynamic Spectrum Access allows using the spectrum opportunistically by identifying wireless technologies sharing the same medium. However, detecting a given technology is, most of the time, not enough to increase spectrum efficiency and mitigate coexistence problems due to radio interference. As a solution, recognizing traffic patterns may lead to select the best time to access the shared spectrum optimally. To this extent, we present a traffic recognition approach that, to the best of our knowledge, is the first non-intrusive method to detect traffic patterns directly from the radio spectrum, contrary to traditional packet-based analysis methods. In particular, we designed a Deep Learning (DL) architecture that differentiates between Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) traffic, burst traffic with different duty cycles, and traffic with varying rates of transmission. As input to these models, we explore the use of images representing the spectrum in time and time-frequency. Furthermore, we present a novel data randomization approach to generate realistic synthetic data that combines two state-of-the-art simulators. Finally, we show that after training and testing our models in the generated dataset, we achieve an accuracy of ≥ 96 % and outperform state-of-the-art methods based on IP-packets with DL.

Wireless networks are ubiquitous in today’s world and consist of an ever-expanding number of heterogeneous consumer devices and communication technologies. As modern devices support multiple communication technologies, efforts have been made to efficiently manage this plethora of technologies by supporting functionalities such as simultaneous usage or handovers.However, existing solutions are missing both the fine-grained control and intelligence to offer seamless inter-technology management and network optimizations. As a result, technologies operate in an isolated manner, network management is inefficient, and the requirements of users and modern applications are not being met. In contrast, we present intelligent and dynamic multi-technology network management that breaks this isolation, abstracting the connectivity decisions from the user and application level. Different contributions are made: first of all, we introduce a framework for inter-technology management that enables, among others, seamless handovers and packet-based load balancing. Next, we propose different algorithms that can be deployed on top of the novel, or existing, management solutions to increase the network-wide throughput by providing more intelligent network configurations, taking into account mobility and real-time requirements. Finally, as management approaches rely on an accurate overview of the network state, we also consider the monitoring aspect and investigate the detection of traffic patterns in the radio spectrum. Our contributions are evaluated through practical implementations in real-life prototypes.

Today's electronic devices have multiple communication technologies available at any time. Currently, the application layer or the user needs to manually switch between them, depending on the networks in range. No holistic and adaptive approach exists that can manage all technologies and devices at once. To this extent, we previously proposed the inter-technology management framework ORCHESTRA. The framework is the first of its kind in providing fine-grained packet-level control across different network technologies in a network-wide manner. In this paper, we present a real-life implementation of this framework and show that it can cope well with mobility requirements of users by offering seamless inter-technology handovers.

Modern connected devices are equipped with the ability to connect to the Internet using a variety of different wireless network technologies. Current network management solutions fail to provide a fine-grained, coordinated, and transparent answer to this heterogeneity, while the lower layers of the OSI stack simply ignore it by providing full separation of layers. To address this, we propose the ORCHESTRA framework to manage the different devices in heterogeneous wireless networks and introduce capabilities such as packet-level dynamic and intelligent handovers (both interand intra-technology), load balancing, replication, and scheduling. The framework is the first of its kind in providing a fine-grained packet-level control across different technologies by introducing a fully transparent virtual medium access control layer and an software-defined networking-like controller with global intelligence. Furthermore, we present a novel optimization problem formulation that can be solved to optimally configure the network. We provide a thorough evaluation through simulations and a prototype implementation. We show that our framework enables, in a real-life setting, transparent and realtime inter-technology handovers and that coordinated load balancing can double the network-wide throughput across different scenarios.

Today's local area networks consist of an ever-expanding number of heterogeneous consumer devices and communication technologies. Despite supporting multiple technologies, those devices tend to connect to the Internet using a single technology, based on predefined priorities. This static behavior does not allow the network to unlock its full potential, which becomes increasingly more important as the quality of service (QoS) requirements of services grow. Moreover, existing approaches make use of theoretical models that assume, unrealistically, full knowledge of the network. To this extent, we present a multi-technology flow-management load balancing framework that dynamically re-routes traffic through heterogeneous networks, in order to maximize the global throughput, based on changing network conditions and QoS demands. Along a problem formulation, we focus on the estimation of wireless and dynamic network characteristics and provide a thorough evaluation through simulations and a prototype implementation. We show that our framework is indeed capable of responding to dynamic network events in real-time and offers an increased overall throughput by optimally using the network's capacity. This results in a throughput increase of around 20 % on average.

Current mobile consumer devices are equipped with the ability to connect to the Internet using a variety of heterogeneous wireless network technologies (e.g., Wi-Fi and LTE). These devices generally opt to statically connect using a single technology, based on predefined priorities. This static behavior does not allow the network to unlock its full potential, which becomes increasingly more important as the requirements of services, in terms of for example throughput and reliability, grow. Multipath TCP (MPTCP) is a solution that allows the simultaneous use of multiple network interfaces. However, it does this uncoordinated for a single connection between two endpoints. Therefore, this paper proposes a Software-Defined Networking (SDN) architecture to enable coordinated multi-path routing across the several networks for mobile devices. Moreover, we propose a novel weighted MPTCP scheduler that allows the transmission of certain controllable percentages of data per network interface. The proposed idea is evaluated through a real-life prototype implementation with a smartphone.

Wi-Fi network roaming is the act of moving a wireless device from one Wi-Fi access point (AP) to another Wi-Fi AP. In urban environments, where APs are densely deployed, users would greatly benefit from roaming between these APs. Standards for Wi-Fi-network roaming have been developed (e.g. IEEE 802.11r), but are rarely implemented. The absence of a widely used standard leads to device-dependent roaming mechanisms, which brings numerous disadvantages. 5G-EmPOWER is an example of a framework that brings the Software-Defined Networking (SDN) paradigm to wireless networks. The framework solves the problem of network roaming by allowing users to connect to their own unique virtual AP and managing their connection to the WAN behind the scenes. This allows the 5G-EmPOWER controller to seamlessly handover users from one physical AP to another. Currently, a single physical controller manages the 5G-EmPOWER control plane. The use of a single system over a distributed system has known disadvantages (e.g. greater cost, single point of failure). In this paper, we present DiMob, which distributes the SDN control plane among multiple controllers. We show that DiMob maintains a seamless handover, while offering the advantages of a distributed system. We demonstrate, for example, that adding an additional node can save approximately 30 % in CPU usage for each controller.

Local area networks (LANs) are employed by a plethora of heterogeneous consumer devices, equipped with the ability to connect to the Internet using a variety of different wireless network technologies. Existing solutions and the lower layers of the OSI stack are unfit to cope with this heterogeneity. For instance, dynamical inter-technology switching is user-of application-based. We propose the ORCHESTRA framework to manage the different devices in heterogeneous wireless local area networks (WLANs) and introduce capabilities such as packet-level dynamic and intelligent handovers (both inter- and intratechnology), load balancing, replication, and scheduling. The framework consists of a controller that is capable of communicating with both existing Software-Defined Networking (SDN) and Network Function Virtualization (NFV) controllers and with devices containing a newly introduced virtual Medium Access Control (MAC) layer. We show that the virtual MAC enables transparent and real-time inter-technology handovers and that our solution scales up to two thousands of clients.

The current local area networks (LANs) are occupied by a large variety of heterogeneous consumer devices, equipped with the ability to connect to the Internet using a variety of different network technologies (e.g., Ethernet, 2.4 and 5GHz Wi-Fi). Nevertheless, devices generally opt to statically connect using a single technology, based on predefined priorities. This static behaviour does not allow the network to unlock its full potential, which becomes increasingly more important as the requirements of services, in terms of latency and throughput, grow. In this paper we present a real-life SDN-based implementation of our previously proposed algorithm that addresses this problem.

Today's local area networks (LANs) consist of a plethora of heterogeneous consumer devices, equipped with the ability to connect to the Internet using a variety of different network technologies (e.g., Ethernet, Power-Line, 2.4 and 5GHz Wi-Fi). Nevertheless, devices generally opt to statically connect using a single technology, based on predefined priorities. This static behaviour does not allow the network to unlock its full potential, which becomes increasingly more important as the requirements of services, in terms of latency and throughput, grow. To address this issue, we present a load balancing algorithm that dynamically selects a suitable interface and path through the network for each flow, based on service requirements, bandwidth availability and current link quality. The goal of the algorithm is to find an optimal path configuration for all the flows across the network that maximizes the global throughput. It dynamically adapts to changing network conditions, the arrival and departure of flows and link failures. The problem is formulated as a Mixed Integer Linear Program (MILP) and its solution, as well as that of a faster heuristic algorithm, is extensively tested in a series of ns-3 simulations with different network topologies and flow configurations. We differentiate from existing work by estimating flow rates and dynamic network conditions using real-time monitoring information, taking into account the shared medium of wireless networks and the specific fairness behaviour of TCP. Results show an increase in throughput up to 70% in heterogeneous LANs under dynamic network conditions.