Existing multirobot task allocation (MRTA) research primarily assumes an environment with known geometric and semantic information. However, in most real-world scenarios, semantic information, such as the locations and classifications of landmarks, is often uncertain. This uncertainty is exacerbated by dynamic requirements, like the sudden addition or removal of tasks, making it challenging for robots to respond reactively. Moreover, tasks in MRTA typically involve multiple constraints, including temporal requirements, diverse capabilities, varying resource needs, and inter-task dependencies. To address these challenges, we propose a reactive task allocation framework for heterogeneous multirobot systems that accounts for temporal requirements and multiple task constraints described by $\mathrm {LTL^{\mathcal {R}}}$ . Our approach assumes an environment with known geometry but unknown semantic landmarks. To efficiently solve task allocations, we encode $\mathrm {LTL^{\mathcal {R}}}$ along with the system's states into a proposed planning decision tree for exploration. Upon detecting a relevant semantic landmark, the reactive multiconstraint planning decision tree (RMC-PDT) is triggered for re-planning. Extensive experiments validate three key features of our method: 1) efficient reactive planning; 2) multiconstraint task solving; and 3) scalability.

Formal languages effectively outline robots’ task specifications, yet current temporal logic struggles to balance semantic expression with solution speed. To address this challenge, we propose extended predicate-based temporal logic (E-pTL), augmenting conventional linear temporal logic with more expressive atomic predicates to reflect the time, space, and order attributes of task and represent complex tasks with dynamic propositions, time windows, and even relative time window. This approach, blending automata-based task abstraction and extended predicates, offers enhanced expressiveness and conciseness for intricate specifications. To cope with E-pTL, we introduce a novel planning framework, the Planning Decision Tree (PDT). PDT incrementally builds a tree through automata and system state searches, recording potential task plans. The proposed pruning method can reduce the exploration space. This method swiftly handles complex temporal tasks defined by E-pTL. Rigorous analysis confirms PDT-based planning’s feasibility (ensuring satisfactory planning aligned with task specifications) and completeness (guaranteeing a feasible solution if available). Moreover, PDT-based planning proves efficient, with solution times approximately linearly proportional to automaton states squared. Extensive simulations and experiments validate its effectiveness and efficiency. Note to Practitioners—Temporal logic, as a formal language, has been widely applied to describe the task of robotic systems. However, linear temporal logic (LTL) struggles with the explicit time constraints, while other temporal logics, e.g., signal temporal logic (STL), cannot compactly represent task progress via an automaton. Consequently, current solutions for complex task with temporal logic specification heavily lean on optimization-based approaches, which are time-consuming and impractical for real-time systems. This work presents a novel approach for fast motion planning in dynamic environments with extended predicate-based temporal logic (E-pTL). The proposed E-pTL not only enhances semantic expression but also can be checked by an automaton, simplifying progress checks. However, existing automata-based methods encounter limitations in dynamic tasks. Graph search methods require replanning at each time step and lack completeness guarantees, while sampling methods can’t efficiently handle the feasibility of each atomic proposition. In contrast, the developed planning decision tree (PDT)-based framework incrementally searches for automata and validates propositions without a transition system, significantly reducing the search space. The suggested pruning method further trims the search space without compromising feasibility. Overall, PDT enables rapid, reliable planning, outperforming optimization-based methods like ILP/MILP and proving well-suited for real-time applications.

Human-robot collaboration plays an important role in intelligent manufacturing. However, the main challenge is how the robot can make online reactive changes to the plan based on the observed human behavior to ensure the completion of user-defined tasks. Such a challenge is further exacerbated if eye-in-hand manipulation is considered since the local field of the camera view cannot capture global observations. Different from existing planning approaches that separate the perception and planning modules, and make strong assumptions about perception abilities, we develop a framework of real-time local reactive planning that enables the robot to quickly adapt its actions if necessary through its limited perception of surroundings using an eye-in-hand camera. Specifically, we develop a locally observable transition system (LOTS) and interpretably express the task using linear temporal logic (LTL). To improve the grasping performance using local visual perception, we propose a high-resolution grasp network (HRG-Net) that achieves state-of-the-art results on multiple datasets (99.50% in Cornell and 97.50% in Jacquard and 96% in Graspnet-1Billion) for the task. A physical experiment using a 7DoF Franka Emika Panda robot demonstrates the effectiveness of the reactive planning framework.Note to Practitioners—Intelligent manufacturing often requires the human operator to work collaboratively with the robot in a shared workspace. Due to possible (assistive or non-assistive) interference of human operators, it is highly desired that the robot can perceive human behaviors and react properly to ensure task accomplishment. Hence, this work is particularly motivated to develop a reactive planning framework that relies on real-time local visual perception (i.e., eye-in-hand camera) to quickly react to its dynamic surroundings and replan its motion when necessary. In future work, rather than using the observed human behavior, we will investigate how to predict human intentions to further improve human-robot collaboration.

Reactive planning enables the robots to deal with dynamic events in uncertain environments. However, existing methods heavily rely on the predefined hard-coded robot behaviors, e.g, a pre-coded temporal logic formula that specifies how robot should react. Little attention has been paid for autonomous generation of reactive tasks specifications during the runtime. As a first attempt towards this goal, this work develops a real-time decision-making and motion planning framework. It allows the robot to follow a global task planned offline while taking proactive decisions and generating temporal logic specifications for local reactive tasks when encountering dynamic events. Specifically, inspired by the causal knowledge graph, a proposition graph is developed, based on which the decision module encode the environment and the task as the Boolean logic and linear temporal logic (LTL), respectively. Based on the established proposition graph and perceived environment, the agent can autonomously generate an LTL formula to realize the local temporary task. A joint sampling algorithm is then developed, in which the automaton states of local and global task are jointly considered to generate a feasible planning that satisfies both global and local tasks. Experiments demonstrate the effectiveness of the proposed decision-making and motion planning.

This works exploits soft constraints in linear temporal logic task planning to enhance the agent’s capability in handling potentially conflicting or even infeasible tasks. Different from most existing works that focus on sticking to the original plan and trying to find a relaxed plan if the workspace does not permit, we augment the soft constraints to represent possible candidate sub-tasks that can be selected to fulfill the global task. Specifically, a hierarchical temporal logic specification is developed to represent LTL tasks with soft constraints and preferences. The hierarchical structure consists of an outer and inner layer, where the outer layer uses co-safe LTL to specify the task-level specifications and the inner layer specifies the low-level task-related atomic propositions via soft constraints. To cope with the hierarchical temporal logic specification, a hierarchical iterative search (HIS) algorithm is developed, which incrementally searches feasible atomic propositions and automaton states, and returns a task plan with minimum cost. Rigorous analysis shows that HIS based planning is feasible (i.e., the generated plan is applicable and satisfactory with respect to the task specification) and optimal (i.e, with minimum cost). Extensive simulation demonstrates the effectiveness of the proposed soft task planning approach.

Empowering legged robots with agile maneuvers is a great challenge. While existing works have proposed diverse control-based and learning-based methods, it remains an open problem to endow robots with animal-like perception and athleticism. Towards this goal, we develop an End-to-End Legged Perceptive Parkour Skill Learning (LEEPS) framework to train quadruped robots to master parkour skills in complex environments. In particular, LEEPS incorporates a vision-based perception module equipped with multi-layered scans, supplying robots with comprehensive, precise, and adaptable information about their surroundings. Leveraging such visual data, a position-based task formulation liberates the robot from velocity constraints and directs it toward the target using innovative reward mechanisms. The resulting controller empowers an affordable quadruped robot to successfully traverse previously challenging and unprecedented obstacles. We evaluate LEEPS on various challenging tasks, which demonstrate its effectiveness, robustness, and generalizability. Supplementary and videos are available at: https://sites.google.com/view/leeps

Fast motion planning is of great significance, espe-cially when a timely mission is desired. However, the complexity of motion planning can grow drastically with the increase of environment details and mission complexity. This challenge can be further exacerbated if the tasks are coupled with the desired locations in the environment. To address these issues, this work aims at fast motion planning problems with temporal logical specifications. In particular, we develop a hierarchical decoupling framework that consists of three layers: the high-level task planner, the decoupling layer, and the low-level motion planner. The decoupling layer is designed to bridge the high and low layers by providing necessary information exchange. Such a framework enables the decoupling of the task planner and path planner, so that they can run independently, which significantly reduces the search space and enables fast planing in continuous or high-dimension discrete workspaces. In addition, the implicit constraint during task-level planning is taken into account, so that the low-level path planning is guaranteed to satisfy the mission requirements. Numerical simulations demonstrate at least one order of magnitude speed up in terms of computational time over existing methods.