We develop a learning-based algorithm for the distributed formation control of networked multi-agent systems governed by unknown, nonlinear dynamics. Most existing algorithms either assume certain parametric forms for the unknown dynamic terms or resort to unnecessarily large control inputs in order to provide theoretical guarantees. The proposed algorithm avoids these drawbacks by integrating neural network-based learning with adaptive control in a two-step procedure. In the first step of the algorithm, each agent learns a controller, represented as a neural network, using training data that correspond to a collection of formation tasks and agent parameters. These parameters and tasks are derived by varying the nominal agent parameters and a user-defined formation task to be achieved, respectively. In the second step of the algorithm, each agent incorporates the trained neural network into an online and adaptive control policy in such a way that the behavior of the multi-agent closed-loop system satisfies the user-defined formation task. Both the learning phase and the adaptive control policy are distributed, in the sense that each agent computes its own actions using only local information from its neighboring agents. The proposed algorithm does not use any a priori information on the agents’ unknown dynamic terms or any approximation schemes. We provide formal theoretical guarantees on the achievement of the formation task. Note to Practitioners—This paper is motivated by control of multi-agent systems, such as teams of robots, smart grids, or wireless sensor networks, with uncertain dynamic models. Existing works develop controllers that rely on unrealistic or impractical assumptions on these models. We propose an algorithm that integrates offline learning with neural networks and real-time feedback control to accomplish a multi-agent task. The task consists of the formation of a pre-defined geometric pattern by the multi-agent team. The learning module of the proposed algorithm aims to learn stabilizing controllers that accomplish the task from data that are obtained from offline runs of the system. However, the learned controller might result in poor performance owing to potential data inaccuracies and the fact that learning algorithms can only approximate the stabilizing controllers. Therefore, we complement the learned controller with a real-time feedback-control module that adapts on the fly to such discrepancies. In practise, the data can be collected from pre-recorded trajectories of the multi-agent system, but these trajectories do need to accomplish the task at hand. The real-time feedback-control is a closed-form function of the states of each agent and its neighbours and the trained neural networks and can be straightforwardly implemented. The experimental results show that the proposed algorithm achieves greater performance than algorithms that use only the trained neural networks or only the real-time feedback-control policy. Our future research will address the sensitivity of the algorithm to the quality and quantity of the employed data as well as to the learning performance of the neural networks.

Task and motion planning (TAMP) can enhance intelligent multi-robot coordination. TAMP becomes signifi-cantly more complicated in obstacle-cluttered environments and in the presence of robot dynamic uncertainties. We propose a control framework that solves the motion-planning problem for multi-robot teams with uncertain dynamics, addressing a key component of the TAMP pipeline. The principal part of the proposed algorithm constitutes a decentralized feedback control policy for tracking of reference paths taken by the robots while avoiding collision and adapting in real time to the underlying dynamic uncertainties. The proposed framework further leverages sampling-based motion planners to free the robots from local-minimum configurations. Extensive experimental results in complex, realistic environments illustrate the superior efficiency of the proposed approach, in terms of planning time and number of encountered local minima, with respect to state-of-the-art baseline methods.

In this work, we develop a control algorithm for mobile manipulators manipulating an object within a leader-follower framework. Unlike existing literature, we avoid the knowledge of the object’s dynamics, and only the leader is aware of the tasks to be executed by the object. The followers are primarily tasked to lift the object and maintain a desired posture while the leader manipulates the object despite its unknown dynamic parameters. We employ a stiffness-based controller for the followers, allowing set-point stabilisation with permissible flexibility and a high-gain prescribed performance controller for the leader to facilitate manipulation from the object's equilibrium state. We present simulation results with two followers and one leader KUKA youbots to validate our proposed framework.

We study the problem of reinforcement learning for a task encoded by a reward machine. The task is defined over a set of properties in the environment, called atomic propositions, and represented by Boolean variables. One unrealistic assumption commonly used in the literature is that the truth values of these propositions are accurately known. In real situations, however, these truth values are uncertain since they come from sensors that suffer from imperfections. At the same time, reward machines can be difficult to model explicitly, especially when they encode complicated tasks. We develop a reinforcement-learning algorithm that infers a reward machine that encodes the underlying task while learning how to execute it, despite the uncertainties of the propositions' truth values. In order to address such uncertainties, the algorithm maintains a probabilistic estimate about the truth value of the atomic propositions; it updates this estimate according to new sensory measurements that arrive from exploration of the environment. Additionally, the algorithm maintains a hypothesis reward machine, which acts as an estimate of the reward machine that encodes the task to be learned. As the agent explores the environment, the algorithm updates the hypothesis reward machine according to the obtained rewards and the estimate of the atomic propositions' truth value. Finally, the algorithm uses a Q-learning procedure for the states of the hypothesis reward machine to determine an optimal policy that accomplishes the task. We prove that the algorithm successfully infers the reward machine and asymptotically learns a policy that accomplishes the respective task.

We present a distributed, event-triggered, and adaptive control algorithm for cooperative object manipulation with rolling contacts and unknown dynamic parameters. Whereas conventional cooperative manipulation methods require rigid contact points, our approach exploits rolling effects of passive end-effectors and does not require force/torque sensing. The removal of rigidity allows for more modular grasping, increased application to more object types, and online adjustment of the grasp. The proposed control algorithm exhibits the following properties: 1) it is distributed, in the sense that the robotic agents calculate their own control signal, under an event-triggered communication scheme. Such a scheme reduces the interagent communication requirements with respect to continuous communication schemes; 2) it uses an online adaptation mechanism to accommodate for unknown dynamic parameters of the object and the agents and 3) it adapts existing internal force controllers to guarantee no slip throughout the manipulation task despite the event-triggered nature of the communication scheme. Hardware implementation validates the effectiveness of the proposed approach.

We integrate sampling-based planning techniques with funnel-based feedback control to develop KDF, a new framework for solving the kinodynamic motion-planning problem via funnel control. The considered systems evolve subject to complex, nonlinear, and uncertain dynamics (also known as differential constraints). First, we use ageometricplanner to obtain a high-level safe path in a user-defined extended free space. Second, we develop a low-level funnel control algorithm that guarantees safe tracking of the path by the system. Neither the planner nor the control algorithm uses information on the underlying dynamics of the system, which makes the proposed scheme easily distributable to a large variety of different systems and scenarios. Intuitively, the funnel control module is able to implicitly accommodate the dynamics of the system, allowing hence the deployment of purely geometrical motion planners. Extensive computer simulations and hardware experiments with a 6-DOF robotic arm validate the proposed approach.

We study the design of autonomous agents that are capable of deceiving outside observers about their intentions while carrying out tasks in stochastic, complex environments. By modeling the agent's behavior as a Markov decision process, we consider a setting where the agent aims to reach one of multiple potential goals while deceiving outside observers about its true goal. We propose a novel approach to model observer predictions based on the principle of maximum entropy and to efficiently generate deceptive strategies via linear programming. The proposed approach enables the agent to exhibit a variety of tunable deceptive behaviors while ensuring the satisfaction of probabilistic constraints on the behavior. We evaluate the performance of the proposed approach via comparative user studies and present a case study on the streets of Manhattan, New York, using real travel time distributions.

We develop two consensus-based learning algorithms for multi-robot systems applied on complex tasks involving collision constraints and force interactions, such as the cooperative peg-in-hole placement. The proposed algorithms integrate multi-robot distributed consensus and normalizing-flow-based reinforcement learning. The algorithms guarantee the stability and the consensus of the multi-robot system's generalized variables in a transformed space. This transformed space is obtained via a diffeomorphic transformation parameterized by normalizing-flow models that the algorithms use to train the underlying task, learning hence skillful, dexterous trajectories required for the task accomplishment. We validate the proposed algorithms by parameterizing reinforcement learning policies, demonstrating efficient cooperative learning, and strong generalization of dual-arm assembly skills in a dynamics-engine simulator.

In this paper, we propose a novel safe, passive, and robust control law for mechanical systems. The proposed approach addresses safety from a physical human-robot interaction perspective, where a robot must not only stay inside a pre-defined region, but respect velocity constraints and ensure passivity with respect to external perturbations that may arise from a human or the environment. The proposed control is written in closed-form, behaves well even during singular configurations, and allows any nominal control law to be applied inside the operating region as long as the safety requirements (e.g., velocity) are adhered to. The proposed method is implemented on a 6-DOF robot to demonstrate its effectiveness during a physical human-robot interaction task.

This paper considers the optimal cooperative robotic manipulation problem in terms of energy resources. In particular, we consider rigid cooperative manipulation systems, i.e., with rigid grasping contacts, and study energy-optimal conditions in the sense of minimization of the arising internal forces, which are inter-agent forces that do not contribute to object motion. Firstly, we use recent results to derive a closed form expression for the internal forces. Secondly, by using a standard inverse dynamics control protocol, we provide novel conditions on the force distribution to the robotic agents for provable internal force minimization. Moreover, we derive novel results on the provable achievement of a desired non-zero inter-agent internal force vector. Extensive simulation results in a realistic environment verify the theoretical analysis.

Cooperative robotics is a trending topic nowadays as it makes possible a number of tasks that cannot be performed by individual robots, such as heavy payload transportation and agile manipulation. In this work, we address the problem of cooperative transportation by heterogeneous, manipulator- endowed robots. Specifically, we consider a generic number of robotic agents simultaneously grasping an object, which is to be transported to a prescribed set point while avoiding obstacles. The procedure is based on a decentralized leader-follower Model Predictive Control scheme, where a designated leader agent is responsible for generating a trajectory compatible with its dynamics, and the followers must compute a trajectory for their own manipulators that aims at minimizing the internal forces and torques that might be applied to the object by the different grippers. The Model Predictive Control approach appears to be well suited to solve such a problem, because it provides both a control law and a technique to generate trajectories, which can be shared among the agents. The proposed algorithm is implemented using a system comprised of a ground and an aerial robot, both in the robotic Gazebo simulator as well as in experiments with real robots, where the methodological approach is assessed and the controller design is shown to be effective for the cooperative transportation task.

This work proposes a novel method of incorporating calls to a motion planner inside a potential field control policy for safe multi-robot navigation with uncertain dynamics. The proposed framework can handle more general scenes than the control policy and has low computational costs. Our work is robust to uncertain dynamics and quickly finds high-quality paths in scenarios generated from real-world floor plans. In the proposed approach, we attempt to follow the control policy as much as possible, and use calls to the motion planner to escape local minima. Trajectories returned from the motion planner are followed using a path-following controller guaranteeing robustness. We demonstrate the utility of our approach with experiments based on floor plans gathered from real buildings.

This work addresses the problem of robot navigation under timed temporal specifications in workspaces cluttered with obstacles. We propose a hybrid control strategy that guarantees the accomplishment of a high-level specification expressed as a timed temporal logic formula, while preserving safety (i.e., obstacle avoidance) of the system. In particular, we utilize a motion controller that achieves safe navigation inside the workspace in predetermined time, thus allowing us to abstract the motion of the agent as a finite timed transition system among certain regions of interest. Next, we employ standard formal verification and convex optimization techniques to derive high-level timed plans that satisfy the agent's specifications. A simulation study illustrates and clarifies the proposed scheme.

This paper addresses the motion planning problem for a team of aerial agents under high level goals. We propose a hybrid control strategy that guarantees the accomplishment of each agent's local goal specification, which is given as a temporal logic formula, while guaranteeing inter-agent collision avoidance. In particular, by defining 3-D spheres that bound the agents' volume, we extend previous work on decentralized navigation functions and propose control laws that navigate the agents among predefined regions of interest of the workspace while avoiding collision with each other. This allows us to abstract the motion of the agents as finite transition systems and, by employing standard formal verification techniques, to derive a high-level control algorithm that satisfies the agents' specifications. Simulation and experimental results with quadrotors verify the validity of the proposed method.

This paper addresses the problem of cooperative object manipulation with the coordination relying solely on implicit communication. We consider a decentralized leader-follower architecture where the leading robot, that has exclusive knowledge of the object's desired trajectory, tries to achieve the desired tracking behavior via an impedance control law. On the other hand, the follower estimates the leader's desired motion via a novel prescribed performance estimation law, that drives the estimation error to an arbitrarily small residual set, and implements a similar impedance control law. Both control schemes adopt feedback linearization as well as load sharing among the robots according to their specific payload capabilities. The feedback relies exclusively on each robot's force/torque, position as well as velocity measurements and apart from a few commonly predetermined constant parameters, no explicit data is exchanged on-line among the robots, thus reducing the required communication bandwidth and increasing robustness. Finally, a comparative simulation study clarifies the proposed method and verifies its efficiency.

In this paper, we consider the two dimensional (2-D) predecessor-following control problem for a platoon of unicycle vehicles moving on a planar surface. More specifically, we design a decentralized kinematic control protocol, in the sense that each vehicle calculates its own control signal based solely on local information regarding its preceding vehicle, by its on-board camera, without incorporating any velocity measurements. Additionally, the transient and steady state response is a priori determined by certain designer-specified performance functions and is fully decoupled by the number of vehicles composing the platoon and the control gains selection. Moreover, collisions between successive vehicles as well as connectivity breaks, owing to the limited field of view of cameras, are provably avoided. Finally, an extensive simulation study is carried out in the WEBOTSTM realistic simulator, clarifying the proposed control scheme and verifying its effectiveness.