Popular safe Bayesian optimization (BO) algorithms learn control policies for safety-critical systems in unknown environments. However, most algorithms make a smoothness assumption, which is encoded by a known bounded norm in a reproducing kernel Hilbert space (RKHS). The RKHS is a potentially infinite-dimensional space, and it remains unclear how to reliably obtain the RKHS norm of an unknown function. In this work, we propose a safe BO algorithm capable of estimating the RKHS norm from data. We provide statistical guarantees on the RKHS norm estimation, integrate the estimated RKHS norm into existing confidence intervals and show that we retain theoretical guarantees, and prove safety of the resulting safe BO algorithm. We apply our algorithm to safely optimize reinforcement learning policies on physics simulators and on a real inverted pendulum, demonstrating improved performance, safety, and scalability compared to the state-of-the-art.

Multi Robot Systems are increasingly deployed in applications, such as intralogistics or autonomous delivery, where multiple robots collaborate to complete tasks efficiently. One of the key factors enabling their efficient cooperation is Multi-Robot Task Allocation (MRTA). Algorithms solving this problem optimize task distribution among robots to minimize the overall execution time. In shared environments, apart from the relative distance between the robots and the tasks, the execution time is also significantly impacted by the delay caused by navigating around moving people. However, most existing MRTA approaches are dynamics-agnostic, relying on static maps and neglecting human motion patterns, leading to inefficiencies and delays. In this paper, we introduce Human-Aware Task Allocation (HATA). This method leverages Maps of Dynamics (MoDs), spatio-temporal queryable models designed to capture historical human movement patterns, to estimate the impact of humans on the task execution time during deployment. HATA utilizes a stochastic cost function that includes MoDs Experimental results show that integrating MoDs enhances task allocation performance, resulting in reduced mission completion times by up to 26% compared to the dynamics-agnostic method and up to 19% compared to the baseline. This work underscores the importance of considering human dynamics in MRTA within shared environments and presents an efficient framework for deploying multi-robot systems in environments populated by humans.

How can robots learn and adapt to new tasks and situations with little data? Systematic exploration and simulation are crucial tools for efficient robot learning. We present a novel black-box policy search algorithm focused on data-efficient policy improvements. The algorithm learns directly on the robot and treats simulation as an additional information source to speed up the learning process. At the core of the algorithm, a probabilistic model learns the dependence between the policy parameters and the robot learning objective not only by performing experiments on the robot, but also by leveraging data from a simulator. This substantially reduces interaction time with the robot. Using the model, we can guarantee improvements with high probability for each policy update, thereby facilitating fast, goal-oriented learning. We evaluate our algorithm on simulated fine-tuning tasks and demonstrate the data-efficiency of the proposed dual-information source optimization algorithm. In a real robot learning experiment, we show fast and successful task learning on a robot manipulator with the aid of an imperfect simulator.

When deploying machine learning algorithms in the real world, guaranteeing safety is an essential asset. Existing safe learning approaches typically consider continuous variables, i.e., regression tasks. However, in practice, robotic systems are also subject to discrete, external environmental changes, e.g., having to carry objects of certain weights or operating on frozen, wet, or dry surfaces. Such influences can be modeled as discretecontextvariables. In the existing literature, such contexts are, if considered, mostly assumed to be known. In this work, we drop this assumption and show how we can perform safe learning when we cannot directly measure the context variables. To achieve this, we derive frequentist guarantees for multiclass classification, allowing us to estimate the current context from measurements. Furthermore, we propose an approach for identifying contexts through experiments. We discuss under which conditions we can retain theoretical guarantees and demonstrate the applicability of our algorithm on a Furuta pendulum with camera measurements of different weights that serve as contexts.

Learning optimal control policies directly on physical systems is challenging. Even a single failure can lead to costly hardware damage. Most existing model-free learning methods that guarantee safety, i.e., no failures, during exploration are limited to local optima. This work proposes GoSafeOpt as the first provably safe and optimal algorithm that can safely discover globally optimal policies for systems with high-dimensional state space. We demonstrate the superiority of GoSafeOpt over competing model-free safe learning methods in simulation and hardware experiments on a robot arm.

An important class of cyber-physical systems relies on multiple agents that jointly perform a task by coordinating their actions over a wireless network. Examples include self-driving cars in intelligent transportation and production robots in smart manufacturing. However, the scalability of existing control-over-wireless solutions is limited as they cannot resolve overload situations in which the communication demand exceeds the available bandwidth. This article presents a novel co-design of distributed control and wireless communication that overcomes this limitation by dynamically allocating the available bandwidth to agents with the greatest need to communicate. Experiments on a real cyber-physical testbed with 20 agents, each consisting of a low-power wireless embedded device and a cart-pole system, demonstrate that our solution achieves significantly better control performance under overload than the state of the art. We further prove that our co-design guarantees closed-loop stability for physical systems with stochastic linear time-invariant dynamics.

When learning policies for robotic systems from data, safety is a major concern, as violation of safety constraints may cause hardware damage. SafeOpt is an efficient Bayesian optimization (BO) algorithm that can learn policies while guaranteeing safety with high probability. However, its search space is limited to an initially given safe region. We extend this method by exploring outside the initial safe area while still guaranteeing safety with high probability. This is achieved by learning a set of initial conditions from which we can recover safely using a learned backup controller in case of a potential failure. We derive conditions for guaranteed convergence to the global optimum and validate GoSafe in hardware experiments.

Smart manufacturing aims to overcome the limitations of today's rigid assembly lines by making the material flow and manufacturing process more flexible, versatile, and scalable. The main economic drivers are higher resource and cost efficiency as the manufacturers can more quickly adapt to changing market needs and also increase the lifespan of their production sites. The ability to close feedback loops fast and reliably over long distances among mobile robots, remote sensors, and human operators is a key enabler for smart manufacturing. Thus, this article provides a perspective on control and coordination over wireless networks. Based on an analysis of real-world use cases, we identify the main technical challenges that need to be solved to close the large gap between the current state of the art in industry and the vision of smart manufacturing. We discuss to what extent existing control-over-wireless solutions in the literature address those challenges, including our own approach toward a tight integration of control and wireless communication. In addition to a theoretical analysis of closed-loop stability, practical experiments on a cyber-physical testbed demonstrate that our approach supports relevant smart manufacturing scenarios. This article concludes with a discussion of open challenges and future research directions.

Closing feedback loops fast and over long distances is key to emerging cyber-physical applications; for example, robot motion control and swarm coordination require update intervals of tens of milliseconds. Low-power wireless communication technology is preferred for its low cost, small form factor, and flexibility, especially if the devices support multi-hop communication. Thus far, however, feedback control over multi-hop low-power wireless networks has only been demonstrated for update intervals on the order of seconds. To fill this gap, this article presents a wireless embedded system that supports dynamic mode changes and tames imperfections impairing control performance (e.g., jitter and message loss), and a control design that exploits the essential properties of this system to provably guarantee closed-loop stability for physical processes with linear time-invariant dynamics in the presence of mode changes. Using experiments on a cyber-physical testbed with 20 wireless devices and multiple cart-pole systems, we are the first to demonstrate and evaluate feedback control and coordination with mode changes over multi-hop networks for update intervals of 20 to 50 milliseconds.

This abstract describes the first public demonstration of feedback control and coordination of multiple physical systems over a dynamic multi-hop low-power wireless network with update intervals of tens of milliseconds. Our running system can dynamically change between different sets of application tasks (e.g., sensing, actuation, control) executing on the spatially distributed embedded devices, while closed-loop stability is provably guaranteed even across those so-called mode changes. Moreover, any subset of the devices can move freely, which does not affect closed-loop stability and control performance as long as the wireless network remains connected.

The Internet of Things (IoT) interconnects multiple physical devices in large-scale networks. When the “things” coordinate decisions and act collectively on shared information, feedback is introduced between them. Multiple feedback loops are thus closed over a shared, general-purpose network. Traditional feedback control is unsuitable for design of IoT control because it relies on high-rate periodic communication and is ignorant of the shared network resource. Therefore, recent event-based estimation methods are applied herein for resource-aware IoT control allowing agents to decide online whether communication with other agents is needed, or not. While this can reduce network traffic significantly, a severe limitation of typical event-based approaches is the need for instantaneous triggering decisions that leave no time to reallocate freed resources (e.g., communication slots), which hence remain unused. To address this problem, novel predictive and self triggering protocols are proposed herein. From a unified Bayesian decision framework, two schemes are developed: 1) self triggers that predict, at the current triggering instant, the next one and 2) predictive triggers that check at every time step, whether communication will be needed at a given prediction horizon. The suitability of these triggers for feedback control is demonstrated in hardware experiments on a cart-pole, and scalability is discussed with a multivehicle simulation.

We sketch our ongoing work toward the first design, implementation, and evaluation of a low-power embedded system providing reliable wireless feedback control of several distributed processes over multiple hops with update rates of 10 Hz or higher.

In this two-part paper we present an innovative bio-inspired approach to control the lift of a limb of a humanoid robot called Myon from the stable rest state to the unstable upright position. The proposed paradigm outperforms the state-of-the art approach in terms of time- and energy-efficiency, while maintaining a good degree of adaptability to changes to the nominal operating conditions. This Part I paper introduces the theory behind the novel three-phase control strategy, while the companion Part II manuscript derives its circuit implementation, and analyses its performance to validate the theoretic findings. The critical steps in the proposed strategy are the determination of an estimate for the time duration of the first phase, the storage of the result of this computation over the time interval between two consecutive applications of the control action, and the adaptation of this calculation to changes to the nominal operating conditions. All these three tasks may be successfully accomplished by leveraging the computing, memory, and learning capabilities of a single non-volatile memristor.

The standard Go-Against-the-Force strategy to lift a limb of a humanoid robot from the stable rest state to the upright position and to maintain it there afterwards is slow and consumes a significant amount of energy due to the iterative cycle of sensing and driving operations its application consists of. In order to enhance the performance of the control action, the Part I paper introduced the theory behind an innovative three-phase strategy, called Kick-Fly-Catch paradigm, which is expected to lead to a faster limb motion under a lower energy cost as compared to the original approach. The combined ability of a non-volatile memristor to process data according to Ohm's law, store computation results at power off, and adapt its dynamic behaviour on the basis of its state equation is at the origin for the performance benefits of the proposed strategy over the standard approach. This Part II paper designs a circuit implementation for the overall dynamic system under the new control strategy, and validates the theoretic predictions of the Part I manuscript.