The primary objective of the dataset is to provide a better understanding of the coupling between human actions and gaze in a shared working environment with a cobot, with the aim of significantly enhancing the efficiency and safety of human-cobot interactions. More broadly, by linking gaze patterns with physical actions, the dataset offers valuable insights into cognitive processes and attention dynamics in the context of assembly tasks. The proposed dataset contains gaze and action data from approximately 80 participants, recorded during simulated industrial assembly tasks. The tasks were simulated using controlled scenarios in which participants manipulated educational building blocks. Gaze data was collected using two different eye-tracking setups -head-mounted and remote- while participants worked in two positions: sitting and standing.

This paper introduces SibylSatOpt, a novel approach to finding optimal plans for Totally-Ordered HTN (TOHTN) problems by leveraging greedy search techniques with MaxSAT. Unlike previous SAT-based HTN planners that employed a blind breadth-first search strategy, SibylSatOpt is guided by an admissible heuristic. This heuristic combines a relaxed MaxSAT encoding of the problem with the Task Decomposition Graph (TDG) heuristic. As we demonstrate, the admissibility of the heuristic guarantees that the found solution is optimal. Experimental results on IPC benchmarks show that SibylSatOpt significantly outperforms existing optimal TOHTN planners in both runtime and problem coverage.

This paper presents SibylSat, a novel SAT-based method designed to efficiently solve totally-ordered HTN problems (TOHTN). In contrast to prevailing SAT-based HTN planners that employ a breadth-first search strategy, SibylSat adopts a greedy search approach, enabling it to identify promising decompositions for expansion. The selection process is facilitated by a heuristic derived from solving a relaxed problem, which is also expressed as a SAT problem. Our experimental evaluations demonstrate that SibylSat outperforms existing SAT-based TOHTN approaches in terms of both runtime and plan quality on most of the IPC benchmarks, while also solving a larger number of problems.

This paper introduces TEP (Temporal Event Planning), a novel POCL (Partial Ordered Causal Link) HTN approach tailored for temporal Hierarchical Task Networks (HTNs). TEP stands out for its capability to address all three categories of temporal problems, as categorized by Cushing, distinguishing it from prevailing approaches that primarily tackle the initial two categories. TEP accomplishes this by translating temporal HTN problems into non-temporal counterparts and excels in performance by leveraging heuristics developed in non-temporal HTN planning to steer the search for temporal solutions. This innovative strategy initiates by refining tasks into instantaneous actions and relaxing duration constraints, ensuring alignment with conventional search heuristics while upholding the problem's innate expressiveness.

The Hierarchical Task Network (HTN) formalism is used to express a wide variety of planning problems and many techniques have been proposed to solve them. A particular technique is to encode hierarchical planning problems as classical STRIPS planning problems. One advantage of this technique is to benefit directly from the constant improvements made by STRIPS planners. However, there are still few effective and expressive encodings. In this paper, we present the first HTN to STRIPS encodings allowing to generate concurrent plans. We show experimentally that these encodings not only outperform previous approaches on hierarchical IPC benchmarks but also produce more expressive solution plans.

This paper presents a novel approach to learn PDDL domain called AMLSI (Action Model Learning with State machine Interaction) based on grammar induction. AMLSI learns with no prior knowledge from a training dataset made up of action sequences built by random walks and by observing state transitions. The domain learnt is accurate enough to be used without human proofreading in a planner even with very highly partial and noisy observations. Thus AMLSI takles a key issue for domain learning that is the ability to plan with the learned domains. It often happens that small learning errors lead to domains that are unusable for planning. AMLSI contribution is to learn domains from partial and noisy observations with sufficient accuracy to allow planners to solve new problems. Compared to other approaches, AMLSI uses smaller training datasets and exploits both feasible and infeasible generated action sequences.

Verbal and nonverbal communication skills are essential for human-robot interaction, in particular when the agents are involved in a shared task.We address the specific situation where the robot is the only agent knowing about both the plan and the goal of the task, and has to instruct the human partners.The case study is a brick assembly.We here describe a multilayered verbal depictor whose semantic, syntactic, and lexical settings have been collected and evaluated via crowdsourcing.One crowdsourced experiment involves a robot-instructed pick-and-place task.We show that implicitly referring to achieved subgoals (stairs, pillars, etc) increases the performance of human partners.

Hand-encoding PDDL domains is generally considered difficult, tedious and error-prone. Many different machine learning techniques have been proposed to deal with these issues. Recent novel approaches such as AMLSI (Action Model Learning with State Machine Interactions) achieve high level of accuracy, i.e. the learnt domains are accurate enough to solve planning problems without human proofreading. However, in most of the real world applications, training datasets are difficult and costly to acquire and become available gradually over time. To tackle this issue, we present IncrAMLSI, which is a version of AMLSI for incremental training datasets. We show that IncrAMLSI outperforms AMLSI to learn IPC benchmarks and converges with few learning iterations from partial and noisy data.

The research in hierarchical planning has made considerable progress in the last few years. Many recent systems do not rely on hand-tailored advice anymore to find solutions, but are supposed to be domain-independent systems that come with sophisticated solving techniques. In principle, this development would make the comparison between systems easier (because the domains are not tailored to a single system anymore) and – much more important – also the integration into other systems, because the modeling process is less tedious (due to the lack of advice) and there is no (or less) commitment to a certain planning system the model is created for. However, these advantages are destroyed by the lack of a common input language and feature set supported by the different systems. In this paper, we propose an extension to PDDL, the description language used in non-hierarchical planning, to the needs of hierarchical planning systems.

This paper presents a new algorithm based on grammar induction, called AMLSI (Action Model Learning with State machine Interactions), that retro-engineers planning domains described with Planning Problem Description Language (PDDL) by querying a state machine with action sequences and by observing the state transitions. AMLSI takes as input a training set of feasible and infeasible action sequences built from partial observations and returns a PDDL domain. A key issue for domain learning is the ability to plan with the learned domains. It often happens that a small learning error leads to a domain that is unusable for planning. Unlike other algorithms, we show that AMLSI is able to lift this lock by learning domains from partial observations with sufficient accuracy to allow planners to solve new problems.

How can we learn, transfer and extract handwriting styles using deep neural networks? This paper explores these questions using a deep conditioned autoencoder on the IRON-OFF handwriting data-set. We perform three experiments that systematically explore the quality of our style extraction procedure. First, We compare our model to handwriting benchmarks using multidimensional performance metrics. Second, we explore the quality of style transfer, i.e. how the model performs on new, unseen writers. In both experiments, we improve the metrics of state of the art methods by a large margin. Lastly, we analyze the latent space of our model, and we show that it separates consistently writing styles.

Hierarchical Task Networks (HTN) are one of the most expressive representations for automated planning problems. On the other hand, in recent years, the performance of SAT solvers has been drastically improved. To take advantage of these advances, we investigate how to encode HTN problems as SAT problems. In this paper, we propose two new encodings: GCT (Grammar-Constrained Tasks) and SMS (Stack Machine Simulation), which, contrary to previous encodings, address recursive task relationships in HTN problems. We evaluate both encodings on benchmark domains from the International Planning Competition (IPC), setting a new baseline in SAT planning on modern HTN domains.

Programming robots for general purpose applications is extremely challenging due to the great diversity of end-user tasks ranging from manufacturing environments to personal homes. Recent work has focused on enabling end-users to program robots using Programming by Demonstration. However, teaching robots new actions from scratch that can be reused for unseen tasks remains a difficult challenge and is generally left up to robotic experts. We propose iRoPro, an interactive Robot Programming framework that allows end-users to teach robots new actions from scratch and reuse them with a task planner. In this work we provide a system implementation on a two-armed Baxter robot that (i) allows simultaneous teaching of low-and high-level actions by demonstration, (ii) includes a user interface for action creation with condition inference and modification, and (iii) allows creating and solving previously unseen problems using a task planner for the robot to execute in real-time. We evaluate the generalisation power of the system on six benchmark tasks and show how taught actions can be easily reused for complex tasks. We further demonstrate its usability with a user study (N=21), where users completed eight tasks to teach the robot new actions that are reused with a task planner. The study demonstrates that users with any programming level and educational background can easily learn and use the system.

Arrangement of items on shelves in stores or warehouses is a tedious, repetitive task that can be feasible for robots to perform. The diversity of products that are available in stores and the different setups and preferences of each store makes pre-programming a robot for this task extremely challenging. Instead, our work argues for enabling end-users to customize the robot to their specific objects and setup at deployment time by programming it themselves. To that end, this paper contributes (i) a task representation for shelf arrangements based on a large dataset of grocery store shelf images, (ii) a method for inferring goal configurations from user inputs including demonstrations and direct parameter specifications, and (iii) a system implementation of the proposed approach that allows simultaneously learning task goals and actions. We evaluate our goal inference approach with ten different teaching strategies that combine alternative user inputs in different ways on the large dataset of grocery configurations, as well as with real human teachers through an online user study (N=32). We evaluate our full system implemented on a Fetch mobile manipulator on eight benchmark tasks that demonstrate end-to-end programming and execution of shelf arrangement tasks.

PDDL4J (Planning Domain Description Library for Java) is an open source toolkit for Java cross-platform developers meant (1) to provide state-of-the-art planners based on the Pddl language, and (2) to facilitate research works on new planners. In this article, we present an overview of the Automated Planning concepts and languages. We present some planning systems and their most significant applications. Then, we detail the Pddl4j toolkit with an emphasis on the available informative structures, heuristics and search algorithms.

Cobots (collaborative robots) are revolutionising industries by allowing robots to work in close collaboration with humans. But many companies hesitate their adoption, due to the lack of programming experts. In this work, we evaluate a robot programming framework for non-expert users, that requires users to teach action models expressed in a symbolic planning language (PDDL). These action models would allow the robot to leverage modern automated planners to achieve any user-defined goal. We conducted qualitative user experiments with a Baxter robot to evaluate the non-expert user's understanding of the symbolic planning language and the usability of the framework. We showed that users with little to no programming experience can adopt the symbolic planning language, and use the framework.

Many planning techniques have been developed to allow autonomous systems to act and make decisions based on their perceptions of the environment. Among these techniques, HTN (Hierarchical Task Network) planning is one of the most used in practice. Unlike classical approaches of planning. HTN operates by decomposing task into sub-tasks until each of these sub-tasks can be achieved an action. This hierarchical representation provide a richer representation of planning problems and allows to better guide the plan search and provides more knowledge to the underlying algorithms. In this paper, we propose a new approach of HTN planning in which, as in conventional planning, we instantiate all planning operators before starting the search process. This approach has proven its effectiveness in classical planning and is necessary for the development of effective heuristics and encoding planning problems in other formalism such as CSP or SAT. The instantiation is actually used by most modern planners but has never been applied in an HTN based planning framework. We present in this article a generic instantiation algorithm which implements many simplification techniques to reduce the process complexity inspired from those used in classical planning. Finally we present some results obtained from an experimentation on a range of problems used in the international planning competitions with a modified version of SHOP planner using fully instantiated problems.

Automated planning has achieved significant breakthroughs in recent years. Nonetheless, attempts to improve search algorithm efficiency remain the primary focus of most research. However, it is also possible to build on previous searches and learn from previously found solutions. Our approach consists in learning macro-actions and adding them into the planner's domain. A macro-action is an action sequence selected for application at search time and applied as a single indivisible action. Carefully chosen macros can drastically improve the planning performances by reducing the search space depth. However, macros also increase the branching factor. Therefore, the use of macros entails a utility problem: a trade-off has to be addressed between the benefit of adding macros to speed up the goal search and the overhead caused by increasing the branching factor in the search space. In this paper, we propose an online domain and planner-independent approach to learn 'useful' macros, i.e. macros that address the utility problem. These useful macros are obtained by statistical and heuristic filtering of a domain specific macro library. The library is created from the most frequent action sequences derived from an n-gram analysis on successful plans previously computed by the planner. The relevance of this approach is proven by experiments on International Planning Competition domains.

In the context of real-time planning, this paper investigates the contributions of two enhancements for selecting actions. First, the agenda-driven planning enhancement ranks relevant atomic goals and solves them incrementally in a best-first manner. Second, the committed actions enhancement commits a sequence of actions to be executed at the following time steps. To assess these two enhancements, we developed a real-time planning algorithm in which action selection can be driven by a goal-agenda, and committed actions can be done. Experimental results, performed on classical planning problems, show that agenda-planning and committed actions are clear advantages in the real-time context. Used simultaneously, they enable the planner to be several orders of magnitude faster and solution plans to be shorter.