Abstract Offline reinforcement learning (RL) has emerged as a promising paradigm for real-world applications since it aims to train policies directly from datasets of past interactions with the environment. The past few years, algorithms have been introduced to learn from high-dimensional observational states in offline settings. The general idea of these methods is to encode the environment into a latent space and train policies on top of this smaller representation. In this paper, we extend this general method to stochastic environments (i.e., where the reward function is stochastic) and consider a risk measure instead of the classical expected return. First, we show that, under some assumptions, it is equivalent to minimizing a risk measure in the latent space and in the natural space. Based on this result, we present Latent Offline Distributional Actor-Critic (LODAC), an algorithm which is able to train policies in high-dimensional stochastic and offline settings to minimize a given risk measure. Empirically, we show that using LODAC to minimize Conditional Value-at-Risk (CVaR) outperforms previous methods in terms of CVaR and return on stochastic environments.

We present a novel algorithm to match GPS trajectories onto maps offline (in batch mode) using techniques borrowed from the field of force-directed graph drawing. We consider a simulated physical system where each GPS trajectory is attracted or repelled by the underlying road network via electrical-like forces. We let the system evolve under the action of these physical forces such that individual trajectories are attracted towards candidate roads to obtain a map matching path. Our approach has several advantages compared to traditional, routing-based, algorithms for map matching, including the ability to account for noise and to avoid large detours due to outliers in the data whilst taking into account the underlying topological restrictions (such as one-way roads). Our empirical evaluation using real GPS traces shows that our method produces better map matching results compared to alternative offline map matching algorithms on average, especially for routes in dense, urban areas.

We present a model for predictive caching where a shared cache is used to improve performance across a grid. Unlike local caching mechanisms, shared, grid or cloud-based caches incur high costs or latency associated with the additional data transfer. Our proposed caching model, which is dynamically optimized and constantly updated over time, determines the optimal allocation of objects into the shared cache, in such a way that the total cost or latency is minimized. This is achieved by including in the caching algorithm design measures of grid latency, data retrieval costs and a predictive component based on the probability of cached objects being requested in the near future.

The design, development, and deployment of Ultra-Wideband (UWB) localization systems involves digital and Radio-Frequency (RF) hardware, embedded software, localization algorithms, security and reliability aspects, electromagnetics, and others. Design and integration decisions affect the performance of an UWB system, in particular the most important metrics: localization accuracy and position update rate. To facilitate further development of UWB localization systems and to analyze some of the major trade-offs we share our experience in deploying the EPFL UWB-Lite test bed (U-Lite). We describe an approach to numerical simulation modeling that can help in the design and evaluation of UWB localization systems. To validate our approach we show experimental results with one transmitter and one receiver. Our UWB test bed includes a mobile robot platform, so we can study and evaluate the UWB performance trade-offs in real-world conditions.

This paper deals with a communication infrastructure needed to allow a swarm of robots performing dust cleaning and garbage collection task in an urban area. It outlines the required communication links, analyses them regarding security, describes in details the implementation and figure out some performance test results.

The proposed IEEE 802.11e draft standard defines new MAC protocols for QoS in wireless networks, mainly HCF (hybrid coordination function) and EDCF (enhanced distributed coordination function). EDCF is a contention-based channel access scheme and is part of HCF for infrastructure networks and may be used as a separate coordination function for wireless ad-hoc networks. We propose to extend EDCF with a dynamic adaptation algorithm for the minimum contention window (CW/sub min/) that enables each station to tune the size of the CW/sub min/ used in its back-off algorithm at run time. The purpose of our scheme is to reduce delay and jitter and increase the efficiency of the transmission channel. Priorities between access categories are provisioned by tuning the size of the CW/sub min/ according to application requirements and channel conditions. The performances of the IEEE 802.11e EDCF, enhanced with our adaptation algorithm, are extensively investigated by simulation. Results obtained indicate that the CW/sub min/ adaptation scheme outperforms the 802.11e EDCF standard in terms of channel utilization, throughput, and packet delay. Indeed, the delay for the high priority access category decreases by up to 30%, total throughput increases by up to 18%, and channel capacity is at least 15% higher.

This work presents an adaptation of the IEEE 802.15.4 MAC layer to an ultra wide band (UWB) radiofrequency physical layer developed at the CSEM in the framework of a European project called URSafe (stands for "Universal Remote Signal Acquisition For Health"). A hardware demonstrator based on a low power technology has been realized for supporting different test applications, the final objective being to realize a body-worn wireless personal area networks (WPAN) for telemedicine applications.

We consider the multiplexing of several variable bit rate (VBR) connections over one variable bit rate connection where the multiplexing uses a multiplexing buffer of size B. The VBR trunk is itself a connection and has a multidimensional connection descriptor, reflecting peak and sustainable rates. Given a cost function for the VBR trunk and a connection admission control (CAC) method for the input connections, we focus on the problem of finding the VBR trunk connection descriptor that minimizes the cost function and is able to accept set given set of VBR input connections. First, we show that, under reasonable assumptions on the cost function, the optimization problem can be reduced to a simpler one. Then we consider the homogeneous, loss-free case, for which we give an explicit CAC method. In this case, we find that, for all reasonable cost functions, the optimal VBR trunk is either of the CBR type, or is truly VBR, with a burst duration equal to the burst duration of the input connections. We show that the optimal peak cell rate is fixed for a given B (thus for a CBR trunk), and a VBR choice can only be an improvement. Lastly, we take as an example of the cost function the equivalent capacity of the VBR trunk. These results are expected to form the basis for a general method for a connection manager at a multiplexing node in an integrated services packet network.