Hierarchical goal networks (HGNs) provide a framework for goal-directed planning by decomposing high-level goals into ordered subgoals. While prior work has examined non-determinism for hierarchical planning (specifically, HTNs), scant work studies how HGNs can help in stochastic settings. We introduce a formalism for probabilistic HGN planning with action-insertion semantics, enabling probabilistic planners to incorporate domain knowledge from goal decomposition methods. We design and evaluate two UCT-based algorithms for solving probabilistic HGN planning problems: an asymptotically optimal approach and a compressed, shared-value approach that optimizes separately for each goal within the goal-subgoal hierarchy. We compare our two UCT-based HGN search algorithms experimentally on modified benchmark domains from the FOND HTN literature. Our results demonstrate that on larger problems, the compressed search converges more quickly and outperforms the asymptotically optimal search. This suggests that HGNs can be effective in probabilistic planning, and compression may yield better performance on large problems in anytime settings with stochastic action outcomes.

Landmarks—conditions that must be satisfied at some point in every solution plan—have contributed to major advancements in classical planning, but they have seldom been used in stochastic domains. We formalize probabilistic landmarks and adapt the UCT algorithm to leverage them as subgoals to decompose MDPs; core to the adaptation is balancing between greedy landmark achievement and final goal achievement. Our results in benchmark domains show that well-chosen landmarks can significantly improve the performance of UCT in online probabilistic planning, while the best balance of greedy versus long-term goal achievement is problem-dependent. The results suggest that landmarks can provide helpful guidance for anytime algorithms solving MDPs.

This paper provides theoretical and empirical comparisons of three recent hierarchical plan repair algorithms: SHOPFIXER, IPYHOPPER, and REWRITE. Our theoretical results show that the three algorithms correspond to three different definitions of the plan repair problem, leading to differences in the algorithms’ search spaces, the repair problems they can solve, and the kinds of repairs they can make. Understanding these distinctions is important when choosing a repair method for any given application. Building on the theoretical results, we evaluate the algorithms empirically in a series of benchmark planning problems. Our empirical results provide more detailed insight into the run- time repair performance of these systems and the coverage of the repair problems solved, based on algorithmic properties such as replanning, chronological backtracking, and back- jumping over plan trees.

We consider behavior of agents in a long-term multiagent coopetitive setting in which agents vary their cooperative and competitive stances over time. Using the game ofDiplomacyas a testbed, we study how successful agents vary their coopetitive behavior, developing a new “style of play” (SoP) characterization of player behavior. We assess five novel SoP hypotheses about successful behavior. We propose two algorithms to automatically compute an agent’s SoP vector and describe the important factors in this computation. As an agent’s SoP depends on the game state and its perception of threat, we develop a novel “means, motive, and opportunity” (MMO) model of threat and show that we can predict threats effectively using this model. We provide novel insights into how agents should behave to more successfully achieve their goals in long-term coopetitive settings.

Spatial evolutionary games are used to model large systems of interacting agents. In earlier work, a method was developed using Bayesian Networks to approximate the population dynamics in these games. One of the advantages of the Bayesian Network modeling approach is that it is possible to smoothly adjust the size of the network to get more accurate approximations. However, scaling the method up can be intractable if the number of strategies in the evolutionary game increases. In this paper, we propose a new method for computing more accurate approximations by using surrogate Bayesian Networks. Instead of computing inference on larger networks directly, we perform inference on a much smaller surrogate network extended with parameters that exploit the symmetry inherent to the domain. We learn the parameters on the surrogate network using KL-divergence as the loss function. We illustrate the value of this method empirically through a comparison on several evolutionary games.

Bayesian Networks are useful for analyzing the properties of systems with large populations of interacting agents (e.g., in social modeling applications and distributed service applications). These networks typically have large functions (CPTs), making exact inference intractable. However, often these models have additive symmetry. In this paper we show how summation-based CPTs, especially in the presence of symmetry, can be computed efficiently through the usage of the Fast Fourier Transform (FFT). In particular, we propose an efficient method using the FFT for reducing the size of Conditional Probability Tables (CPTs) in Bayesian Networks with summation-based causal independence (CI). We then show how to apply this reduction directly towards the acceleration of Bucket Elimination, and we subsequently provide experimental results demonstrating the computational speedup provided by our method.

We describe ACR-SDN, a system to monitor, diagnose, and quickly respond to attacks or failures that may occur in software-defined networks (SDNs). An integral part of ACR-SDN is its use of RAE+UPOM, an automated acting and planning engine that uses hierarchical refinement. To advise ACR-SDN on how to recover a target system from faults and attacks, RAE+UPOM uses attack recovery procedures written as hierarchical operational models. Our experimental results show that the use of refinement planning in ACR-SDN is successful in recovering SDNs from attacks with respect to five performance metrics: estimated time for recovery, efficiency, retry ratio, success ratio, and costEffectiveness.

The most common representation formalisms for planning are descriptive models. They abstractly describe what the actions do and are tailored for efficiently computing the next state(s) in a state transition system. But acting requires operational models that describe how to do things, with rich control structures for closed-loop online decision-making. Using descriptive representations for planning and operational representations for acting can lead to problems with developing and verifying consistency of the different models.We define and implement an integrated acting-and-planning system in which both planning and acting use the same operational models, which are written in a general-purpose hierarchical task-oriented language offering rich control structures. The acting component is inspired by the well-known PRS system, except that instead of being purely reactive, it can get advice from the planner. Our planning algorithm, RAEplan, plans by doing Monte Carlo rollout simulations of the actor’s operational models. Our experiments show significant benefits in the efficiency of the acting and planning system.

Abstract Adversarial search, or game‐tree search, is a technique for analyzing an adversarial game to determine what moves a player should make in order to win a game. Until recently, lookahead pathology (in which deeper game‐tree search results in worse play) has been thought to be quite rare. We provide an analysis that shows that every game should have some sections that are locally pathological, assuming that both players can potentially win the game. We also modify the minimax algorithm to recognize local pathologies in arbitrary games and cut off search accordingly (shallower search is more effective than deeper search when local pathologies occur). We show experimentally that our modified search procedure avoids local pathologies and consequently provides improved performance, in terms of decision accuracy, when compared with the minimax algorithm. In addition, we provide an experimental evaluation on the African game of Kalah, which shows the improved performances of our suggested error‐minimizing minimax algorithm when there is a large degree of pathology.

We present a principled way of extending a classical AI planning formalism with systems of state constraints, which relate - sometimes determine - the values of variables in each state traversed by the plan. This extension occupies an attractive middle ground between expressivity and complexity. It enables modelling a new range of problems, as well as formulating more efficient models of classical planning problems. An example of the former is planning-based control of networked physical systems - power networks, for example - in which a local, discrete control action can have global effects on continuous quantities, such as altering flows across the entire network. At the same time, our extension remains decidable as long as the satisfiability of sets of state constraints is decidable, including in the presence of numeric state variables, and we demonstrate that effective techniques for cost-optimal planning known in the classical setting - in particular, relaxation-based admissible heuristics - can be adapted to the extended formalism. In this paper, we apply our approach to constraints in the form of linear or non-linear equations over numeric state variables, but the approach is independent of the type of state constraints, as long as there exists a procedure that decides their consistency. The planner and the constraint solver interact through a well-defined, narrow interface, in which the solver requires no specialisation to the planning context.

Human social preferences have been shown to play an important role in many areas of decision-making. There is evidence from the social science literature that human preferences in interpersonal interactions depend partly on a measurable personality trait called, Social Value Orientation (SVO). Automated agents are often written by humans to serve as their delegates when interacting with other agents. Thus, one might expect an agent’s behaviour to be influenced by the SVO of its human designer. With that in mind, we present the following: first, we explore, discuss and provide a solution to the question of how SVO tests that were designed for humans can be used to evaluate agents’ social preferences. Second, we show that in our example domain there is a medium–high positive correlation between the social preferences of agents and their human designers. Third, we exemplify how the SVO information of the designer can be used to improve the performance of some other agents playing against those agents, and lastly, we develop and exemplify the behavioural signature SVO model which allows us to better predict performances when interactions are repeated and behaviour is adapted.

In a recent position paper in Artificial Intelligence, we argued that the automated planning research literature has underestimated the importance and difficulty of deliberative acting, which is more than just interleaving planning and execution. We called for more research on the AI problems that emerge when attempting to integrate acting with planning. To provide a basis for such research, it will be important to have a formalization of acting that can be useful in practice. This is needed in the same way that a formal account of planning was necessary for research on planning. We describe some first steps toward developing such a formalization, and invite readers to carry out research along this line.

There are many situations in which two or more agents (e.g., human or computer decision makers) interact with each other repeatedly in settings that can be modeled as repeated stochastic games. In such situations, each agent's performance may depend greatly on how well it can predict the other agents' preferences and behavior. For use in making such predictions, we adapt and extend the Social Value Orientation (SVO) model from social psychology, which provides a way to measure an agent's preferences for both its own payoffs and those of the other agents.

In this paper, we present a heuristic planning approach for guarding a valuable asset by a team of autonomous unmanned surface vehicles (USVs) operating in a continuous state-action space. The team's objective is to maximize the amount of time it takes an intruder boat to reach the asset. The team must cooperatively deal with uncertainty about which boats are actual intruders, employ active blocking to slow down intruders' movement towards the asset, and intelligently distribute themselves around the target to optimize future guarding opportunities. Our planner incorporates a market-based algorithm for allocating tasks to individual USVs by forward-simulating the mission and assigning estimated utilities to candidate task-allocation plans. The planner can be automatically adapted to a specific mission by optimizing the behaviors used to fulfil individual tasks. We present detailed simulation results that demonstrate the effectiveness of our approach.

No abstract available.

This work provides a game theoretic framework through which one can study the different trust and mitigation strategies a decision maker can employ when soliciting advice or input from a potentially self-interested third-party. The framework supports a single decision maker's interacting with an arbitrary number of either honest or malicious (and malicious in varying ways) advisors. We include some preliminary results on the analysis of this framework in some constrained instances and propose several avenues of future work.

For HTN planning, we formally characterize and classify four kinds of problem spaces in which each node represents a planning problem or subproblem. Two of the problem spaces are searched by current HTN planning algorithms; the other two problem spaces are new.This enables us to provide:Sufficient (and in one case, necessary) conditions for finiteness of each kind of problem space. The conditions can be evaluated up-front to see if an HTN planning problem is finite.Loop-detection tests that can be used in HTN planners to ensure termination when the problem space is finite.A way to compute the correct value for an upper-bound parameter in an HTN-to-PDDL translation algorithm published in IJCAI-2009.Planning algorithms that utilize the two new problem spaces to guarantee termination on broader classes of planning problems than previous HTN planning algorithms.

No abstract available.

No abstract available.

Researchers have invested much effort in constructing models of the state-dependent (sometimes risk-averse and sometimes risk-prone) nature of human decision making. An important open question is how state-dependent risk behavior can arise and remain prominent in populations. We believe that one part of the answer is the interplay between risk-taking and sequentiality of choices in populations subject to evolutionary population dynamics. To support this hypothesis, we provide simulation and analytical results for evolutionary lottery games, including results on evolutionary stability. We consider a parameterized class of imitation dynamics in which the parameter 0 ≤ α ≤ 1 yields the replicator dynamic with α = 1 and the imitate-the-better dynamic with α = 0. Our results demonstrate that for every population dynamic in this class except for the replicator dynamic, the interplay between risk-taking and sequentiality of choices allows state-dependent risk behavior to have an evolutionary advantage over expected-value maximization.

Despite the recent advances in planning for classical domains, the question of how to use domain knowledge in planning is yet to be completely and clearly answered. Some of the existing planners use domain-independent search heuristics, and some others depend on intensively-engineered domain-specific knowledge to guide the planning process. In this paper, we describe an approach to combine ideas from both of the above schools of thought. We present Duet, our planning system that incorporates the ability of using hierarchical domain knowledge in the form of Hierarchical Task Networks (HTNs) as in SHOP2 [14] and using domain-independent local search techniques as in LPG [8]. In our experiments, Duet was able to solve much larger problems than LPG could solve, with only minimal domain knowledge encoded in HTNs (much less domain knowledge than SHOP2 needed to solve those problems by itself).

To generate plans for collecting data for data mining, an important problem is information volatility during planning: the information needed by the planning system may change or expire during the planning process, as changes occur in the data being collected. In such situations, the planning system faces two challenges: how to generate plans despite these changes, and how to guarantee that a plan returned by the planner will remain valid for some period of time after the planning ends. The focus of our work is to address both of the above challenges. In particular, we provide: 1) A formalism for reactive query policies, a class of strategies for deciding when to reissue queries for information that has changed during the planning process. This class includes all query management strategies that have yet been developed. 2) A new reactive query policy called the presumptive strategy. In our experiments, the presumptive strategy ran exponentially faster than the lazy strategy, the best previously known query management strategy. In the hardest set of problems we tested, the presumptive strategy took 4.7% as much time and generated 6.9% as many queries as the lazy strategy

A significant challenge in developing planning systems for practical applications is the difficulty of acquiring the domain knowledge needed by such systems. One method for acquiring this knowledge is to learn it from plan traces, but this method typically requires a huge number of plan traces to converge. In this paper, we show that the problem with slow convergence can be circumvented by having the learner generate solution plans even before the planning domain is completely learned. Our empirical results show that these improvements reduce the size of the training set that is needed to find correct answers to a large percentage of planning problems in the test set. 1.

A great challenge in developing planning systems for practical applications is the difficulty of acquiring the domain information needed to guide such systems. This paper describes a way to learn some of that knowledge. More specifically, the following points are discussed. (1) We introduce a theoretical basis for formally defining algorithms that learn preconditions for Hierarchical Task Network (HTN) methods. (2) We describe Candidate Elimination Method Learner (CaMeL), a supervised, eager, and incremental learning process for preconditions of HTN methods. We state and prove theorems about CaMeL's soundness, completeness, and convergence properties. (3) We present empirical results about CaMeL's convergence under various conditions. Among other things, CaMeL converges the fastest on the preconditions of the HTN methods that are needed the most often. Thus CaMeL's output can be useful even before it has fully converged.

The SHOP2 planning system received one of the awards for distinguished performance in the 2002 International Planning Competition. This paper describes the features of SHOP2 which enabled it to excel in the competition, especially those aspects of SHOP2 that deal with temporal and metric planning domains.

Although game‐tree search works well in perfect‐information games, it is less suitable for imperfect‐information games such as contract bridge. The lack of knowledge about the opponents’ possible moves gives the game tree a very large branching factor, making it impossible to search a significant portion of this tree in a reasonable amount of time. This paper describes our approach for overcoming this problem. We represent information about bridge in a task network extended to represent multi‐agency and uncertainty. Our game‐playing procedure uses this task network to generate game trees in which the set of alternative choices is determined not by the set of possible actions, but by the set of available tactical and strategic schemes. We have tested this approach on declarer play in the game of bridge, in an implementation called Tignum 2. On 5000 randomly generated notrump deals, Tignum 2 beat the strongest commercially available program by 1394 to 1302, with 2304 ties. These results are statistically significant at the α= 0.05 level. Tignum 2 searched an average of only 8745.6 moves per deal in an average time of only 27.5 seconds per deal on a Sun SPARCstation 10. Further enhancements to Tignum 2 are currently underway.

Though the semantics of nonmonotonic logic programming has been studied extensively, relatively little work has been done on operational aspects of these semantics. In this paper, we develop techniques to compute the well-founded model of a logic program. We describe a prototype implementation and show, based on experimental results, that our technique is more efficient than the standard alternating fixpoint computation. Subsequently, we develop techniques to compute the set of all stable models of a deductive database. These techniques first compute the well-founded semantics and then use an intelligent branch and bound strategy to compute the stable models. We report on our implementation, as well as on experiments that we have conducted on the efficiency of our approach.>

Generating action sequences to achieve a set of goals is a computationally difficult task. When multiple goals are present, the problem is even worse. Although many solutions to this problem have been discussed in the literature, practical solutions focus on the use of restricted mechanisms for planning or the application of domain dependent heuristics for providing rapid solutions (i.e., domain‐dependent planning). One previously proposed technique for handling multiple goals efficiently is to design a planner or even a set of planners (usually domain‐dependent) that can be used to generate separate plans for each goal. The outputs are typically either restricted to be independent and then concatenated into a single global plan, or else they are merged together using complex heuristic techniques. In this paper we explore a set of limitations, less restrictive than the assumption of independence, that still allow for the efficient merging of separate plans using straightforward algorithmic techniques. In particular, we demonstrate that for cases where separate plans can be individually generated, we can define a set of limitations on the allowable interactions between goals that allow efficient plan merging to occur. We propose a set of restrictions that are satisfied across a significant class of planning domains. We present algorithms that are efficient for special cases of multiple plan merging, propose a heuristic search algorithm that performs well in a more general case (where alternative partially ordered plans have been generated for each goal), and describe an empirical study that demonstrates the efficiency of this search algorithm.

When performing a planning or design task in many domains it is often difficult to specify in advance what the precise goals are. It is therefore useful to have a system in which the planning process is performed interactively, with the solution approaching the users' intent incrementally through iterations of the planning process. A planning system intended to function in this way must be able to take goal specifications interactively rather than all at once at the beginning of the planning process. The planning process then becomes one of satisfying new goals as they are given by the user, modifying as little as possible the results of previous planning work. Incremental planning is an approach to interactive planning problems that allows a system to create a plan incrementally, modifying a previous plan to satisfy new or more precise goal specifications. In this paper we present an incremental planning system called the general constraint system (GCS) that is based on the conceptual programming environment (CP) developed at New Mexico State University and we show an example of the use of the system for a simple civil engineering design problem

The various approaches proposed to provide communication between CAD systems and process planning systems share the major problem that, due to geometric interactions among features, there may be several equally valid sets of manufacturable features describing the same part, and different sets of features may differ in their manufacturability. Thus, to produce a good process plan-or, in some cases, any plan at ll-it may be necessary to interpret the part as a different set of features than the one initially obtained from the CAD model. This is addressed using an algebra of features. Given a set of features describing a machinable part, other equally valid interpretations of the part can be produced by performing operations in the algebra. This will enable automated process planning systems to examine these interpretations in order to see which one is most appropriate for use in manufacturing. The feature algebra has been implemented for a restricted domain and integrated with the Protosolid solid modeling system and the EFHA process planning system.>

In most frame-based reasoning systems, the information being manipulated is represetned using frames, but the problem-solving knowledge that manipulates the frames is represented as production rules. One problem with this approach is that rules are not always a natrual way to represent knowledge; another is that systems containing lots of rules may suffer from problems with “exponetial blowup” in the amount of computation required. This paper describes a way to address these problems by organizing the problem-solving knowledge not as rules, but in a particular kind of frame hierarchy. the approach described in this paper has been implemented in a problem-solving system called SIPP (Semi-Intelligent Process Planner), which produces plans of action for the manufacture of metal parts. the paper gives an overview of SIPP, compares its knowledge representation and problem solving methods to approaches used in other knowledge-based systems, and describes goals for further research.

One important application of tree searching ~s "looking ahead" on a decision tree or game tree to try to predict the results of a decision.To guarantee correct results, substantial portions of the tree must be completely searched, which is physically impossible for very large trees.Aruficial intelligence researchers have obtained good results by searching the tree to some arbitrary depth and using a static evaluation function to estimate the values of the nodes at that depth.It is generally believed that when this is done, the quality of the decision improves as the search depth increases.This belief is based purely on empirical evidence.The author has developed a mathematical theory modeling the effects of search depth on the probability of making a correct decision.In this theory, the errors made by the evaluation function are modeled as independent, identically distributed random errors superimposed on the true values of the nodes evaluated.This research has produced the surprising result that there is an infinite class of game trees for which searching deeper does not increase the probability of making a correct decision, but instead causes the decision to become more and more random.The paper contains a mathematical proof of this statement, experimental verification of it, and a discussion of its significance.