Compositionality is a critical aspect of scalable system design. Here, we focus on Boolean composition of learned tasks as opposed to functional or sequential composition. Existing Boolean composition for Reinforcement Learning focuses on reaching a satisfying absorbing state in environments with discrete action spaces, but does not support composable safety (i.e., avoidance) constraints. We provide three contributions: i) introduce two distinct notions of compositional safety semantics; ii) show how to enforce either safety semantics, prove correctness, and analyze the trade-offs between the two safety notions; and iii) extend Boolean composition from discrete action spaces to continuous action spaces. We demonstrate these techniques using modified versions of value iteration in a grid world, Deep Q-Network (DQN) in a grid world with image observations, and Twin Delayed DDPG (TD3) in a continuous-observation and continuous-action Bullet physics environment

Control barrier functions (CBFs) have become popular as a safety filter to guarantee the safety of nonlinear dynamical systems for arbitrary inputs. However, it is difficult to construct functions that satisfy the CBF constraints for high relative degree systems with input constraints. To address these challenges, recent work has explored learning CBFs using neural networks via neural CBFs (NCBFs). However, such methods face difficulties when scaling to higher dimensional systems under input constraints. In this work, we first identify challenges that NCBFs face during training. Next, to address these challenges, we propose policy neural CBFs (PNCBFs), a method of constructing CBFs by learning the value function of a nominal policy, and show that the value function of the maximum-over-time cost is a CBF. We demonstrate the effectiveness of our method in simulation on a variety of systems ranging from toy linear systems to a jet aircraft with a 16-dimensional state space. Finally, we validate our approach on a two-agent quadcopter system on hardware under tight input constraints.

Many existing approaches for coordinating heterogeneous teams of robots either consider small numbers of agents, are application-specific, or do not adequately address common real-world requirements, e.g., strict deadlines or intertask dependencies. We introduce scalable and robust algorithms for task-based coordination from high-level specifications (ScRATCHeS) to coordinate such teams. We define a specification language, capability temporal logic, to describe rich, temporal properties involving tasks requiring the participation of multiple agents with multiple capabilities, e.g., sensors or end effectors. Arbitrary missions and team dynamics are jointly encoded as constraints in a mixed integer linear program, and solved efficiently using commercial off-the-shelf solvers. ScRATCHeS optionally allows optimization for maximal robustness to agent attrition at the penalty of increased computation time. We include an online replanning algorithm that adjusts the plan after an agent has dropped out. The flexible specification language, fast solution time, and optional robustness of ScRATCHeS provide a first step toward a multipurpose on-the-fly planning tool for tasking large teams of agents with multiple capabilities enacting missions with multiple tasks. We present randomized computational experiments to characterize scalability and hardware demonstrations to illustrate the applicability of our methods.

This paper considers the combination of temporal logic (TL) specifications and local objective functions to create online, multiagent, motion plans. These plans are guaranteed to satisfy a persistent mission TL specification and locally optimize an objective function (e.g. in this paper, a cost based on information entropy). The presented approach decouples the two tasks by assigning sub-teams of agents to fulfill the TL specification, while unassigned agents optimize the objective function locally. This paper also presents a novel decoupling of the classic product automaton based approach while maintaining satisfaction guarantees. We also qualitatively show that optimality loss in the local greedy minimization due to the TL constraints can be approximated based on specification complexity. This approach is evaluated with a set of simulations and an experiment of 6 robots with real sensors.

Caterpillars are the soft bodied larvae of lepidopteran insects. They have evolved to occupy an extremely diverse range of natural environments and to locomote in complex three-dimensional structures without articulated joint or hydrostatic control. These animals make excellent bio-inspiration for the field of soft robotics because of their diversity and adaptability. In this paper, we present SquMA Bot, a caterpillar-inspired soft robot. The robot's body is primarily composed of a soft viscoelastic foam, and it is actuated using a motor-tendon system. SquMA Bot is able to mimic the inching gait of a caterpillar and can use its flexible body to adapt to a range of environments. This bio-inspired prototype demonstrates the effectiveness of a soft robot as a potential tool for exploring environments too dangerous for humans.