We introduce an algorithm for model-based hierarchical reinforcement learning to acquire self-contained transition and reward models suitable for probabilistic planning at multiple levels of abstraction. We call this framework Planning with Abstract Learned Models (PALM). By representing subtasks symbolically using a new formal structure, the lifted abstract Markov decision process (L-AMDP), PALM learns models that are independent and modular. Through our experiments, we show how PALM integrates planning and execution, facilitating a rapid and efficient learning of abstract, hierarchical models. We also demonstrate the increased potential for learned models to be transferred to new and related tasks.

This Research to Practice Work-In-Progress offers an approach toward assessing self-reflections in e-portfolios written by undergraduate student-Scholars in a Grand Challenge Scholars Program within the College of Engineering at the University of Maryland, Baltimore County. The proposed approach applies two existing frameworks - Robert Grossman's levels of reflection and the Reflection and Self-Assessment criterion in AAC&U's Integrative Learning VALUE Rubric - to develop a methodology that could facilitate the assessment of self-reflections as instruments of students' learning. Preliminary analysis of portions of the e-portfolios submitted by Scholars in the first two cohorts to complete the program shows that the Scholars did not reach high levels of self-reflection when guided only by a generic prompt that asked them what they learned about themselves and how it changed or broadened their perspectives. Our analysis of portfolios is ongoing as additional Scholars graduate from the program, and we expect to see evidence of deeper levels of self-reflection and greater transformation in these newer portfolios as a result of the changes in expectations and prompts.

Effective options can make reinforcement learning easier by enhancing an agent's ability to both explore in a targeted manner and plan further into the future. However, learning an appropriate model of an option's dynamics in hard, requiring estimating a highly parameterized probability distribution. This paper introduces and motivates the Expected-Length Model (ELM) for options, an alternate model for transition dynamics. We prove ELM is a (biased) estimator of the traditional Multi-Time Model (MTM), but provide a non-vacuous bound on their deviation. We further prove that, in stochastic shortest path problems, ELM induces a value function that is sufficiently similar to the one induced by MTM, and is thus capable of supporting near-optimal behavior. We explore the practical utility of this option model experimentally, finding consistent support for the thesis that ELM is a suitable replacement for MTM. In some cases, we find ELM leads to more sample efficient learning, especially when options are arranged in a hierarchy.

High school computing education reform efforts have been ongoing across the United States, particularly in the past decade. Although national Computer Science (CS) for All initiatives are promising, states retain control over education policies. Recent computing education reform efforts in the state of Maryland (U.S.A.) focused on providing every public high school student with access to high-quality high school computing courses. Such access provides exposure to computing careers and better prepares a diverse pool of students for computing majors in college and the workforce. This comprehensive embedded multi-level case study examines the state’s computing education reform efforts from 2010 through 2016. The expansion of computing education indicates that while there was positive growth, the growth was not the same for all categories of public high school students. Top-down policies assist in providing leverage to elevate the need for CS; however, bottom-up efforts to support students and to enable teachers to retain autonomy and professionalism is also needed for CS expansion. Despite successes, barriers at the state, Local Education Agencies (LEA), school, and classroom levels persist and are discussed. The findings in this study can be applied to other states with similar governance structures and policies, and we provide specific recommendations.

No abstract available.

We present an initial proposal for modular object-oriented MDPs, an extension of OO-MDPs that abstracts complex domains that are partially observable and stochastic with multiple goals. Modes reduce the curse of dimensionality by reducing the number of attributes, objects, and actions into only the features relevant for each goal. These modes may also be used as an abstracted domain to be transferred to other modes or to another domain.

No abstract available.

Welcome to the Twenty-Seventh AAAI Conference on Artificial Intelligence, AAAI-13! As can be seen in these proceedings, AI’s scope and influence continue to grow. This year, we received 827 submissions across a variety of tracks, allowing us to put together a diverse and exciting technical program featuring the field’s top research.

The number of REU programs funded in Computer & Information Science and Engineering has increased from 6--12 per year prior to 2007 to a current level of 12--20 per year [www.nsf.gov]. Participation in research broadens students' experience base, increases their readiness for graduate school and the workforce, and develops their critical problem solving and communication skills. Research experiences are particularly effective at increasing the retention and success of women and minorities in computing fields [Cuny & Aspray, SIGCSE Bulletin 2002; Russell et al., Science 2007]. Despite the importance and prevalence of undergraduate research, many faculty are expected to involve undergraduates in research without having any direct experience or mentoring. Meanwhile, a growing interest in experiential learning at many institutions has led to a recent development of institutional environments in which undergraduate research is strongly supported. In many disciplines, such an environment naturally supports basic research programs that are inclusive of both under- and upperclassmen. However, computer science research at the undergraduate level presents a major challenge: most students who have the requisite knowledge to complete a novel research project are already close to graduation, which can make it difficult to involve undergraduates in longer-term research projects. In this bof, we hope to gather both experienced and novice research advisers to discuss strategies for running a continuous research effort across several graduating classes.

The Computation, Complexity, and Emergence honors seminar at UMBC is designed to introduce an interdisciplinary undergraduate audience to the principles of complex systems that permeate our world. The course goals are for students to understand how simple individual behaviors can lead to complex global behaviors and to be able to identify the sources and effects of complexity in natural and artificial systems. The course is designed to increase students' comfort and skill level in scientific writing, participating actively in written and oral discussions, and learning collaboratively in an interdisciplinary community. The paper describes the course and the innovative elements that lead to student engagement, then presents data from pre- and post-assessments about student attitudes and perceptions.

The goals of UMBC's CE21-Maryland project are to build community and to increase the accessibility, diversity, and quality of high school CS education in Maryland. The ultimate objective is for all Maryland students to have access to high-quality, college preparatory CS courses. We present findings from a survey of high school computing teachers regarding the status of CS education in Maryland. Some findings of interest are that urban and rural students have less access to computing courses than suburban students; female teachers are more likely to attract female students and to have larger AP CS classes; and neither teacher race nor gender is correlated with the number of minority students enrolled in CS classes. We describe community building successes through two Google CS4HS workshops, a Maryland CSTA chapter, and statewide summit meetings for educators and administrators. We also discuss how our methodology can be used as a model for other states who are working towards CS education reform at the high school level.

We present SmartRate, a trust and reputation-based decision framework for Smart Grid, based on the available ratings provided by other customers. This model considers multiple trust factors associated with the broker and the preferences of customers for each of these factors. We define a decision framework for broker selection based on multi-attribute utility function and show how learning customers' rating behaviors helps to increase a decision maker's utility. We evaluate this framework by simulating a market based on real-world data. Our results show that learning the characteristics of a rating population helps to interpret and personalize the ratings, which results in better decision making and an increase in customer satisfaction.

While intuitive time-series visualizations exist for common datasets, student course history data is difficult to represent using traditional visualization techniques due its concurrent nature. A visual composition process is developed and applied to reveal trends across various groupings. By working closely with educators, analytic strategies and techniques are developed to leverage the visualization composition to reveal unknown trends in the data. Furthermore, clustering algorithms are developed to group common course-grade histories for further analysis. Lastly, variations of the composition process are implemented to reveal subtle differences in the underlying data. These analytic tools and techniques enabled educators to confirm expected trends and to discover new ones.

The success of transfer learning on a target task is highly dependent on the selected source data. Instance transfer methods reuse data from the source tasks to augment the training data for the target task. If poorly chosen, this source data may inhibit learning, resulting in negative transfer. The current most widely used algorithm for instance transfer, TrAdaBoost, performs poorly when given irrelevant source data. We present a novel task-based boosting technique for instance transfer that selectively chooses the source knowledge to transfer to the target task. Our approach performs boosting at both the instance level and the task level, assigning higher weight to those source tasks that show positive transferability to the target task, and adjusting the weights of individual instances within each source task via AdaBoost. We show that this combination of task- and instance-level boosting significantly improves transfer performance over existing instance transfer algorithms when given a mix of relevant and irrelevant source data, especially for small amounts of data on the target task.

In 2010 a new annual symposium on Educational Advances in Artificial Intelligence (EAAI) was launched as part of the AAAI annual meeting. The event was held in cooperation with ACM SIGCSE and has many similar goals related to broadening and disseminating work in computer science education. EAAI has a particular focus, however, as the event is specific to educational work in Artificial Intelligence and collocated with a major research conference (AAAI) to promote more interaction between researchers and educators in that domain. This panel seeks to introduce participants to EAAI as a way of fostering more interaction between educational communities in computing. Specifically, the panel will discuss the goals of EAAI, provide an overview of the kinds of work presented at the symposium, and identify potential synergies between that EAAI and SIGCSE as a way of better linking the two communities going forward.

Many real-world applications, such as Supply Chain Management (SCM), can be modeled using multi-agent systems. One shortcoming of current SCM models is that their trust models are ad hoc and do not have a strong theoretical basis. We propose a trust model for SCM that is grounded in probabilistic game theory. In this model, trust can be gained through direct interactions, and/or by asking for information from other trustworthy agents. We will use this model to simulate and study supply chain market behavior.

Multi-view learning algorithms typically assume a complete bipartite mapping between the different views in order to exchange information during the learning process. However, many applications provide only a partial mapping between the views, creating a challenge for current methods. To address this problem, we propose a multi-view algorithm based on constrained clustering that can operate with an incomplete mapping. Given a set of pairwise constraints in each view, our approach propagates these constraints using a local similarity measure to those instances that can be mapped to the other views, allowing the propagated constraints to be transferred across views via the partial mapping. It uses co-EM to iteratively estimate the propagation within each view based on the current clustering model, transfer the constraints across views, and update the clustering model, thereby learning a unified model for all views. We show that this approach significantly improves clustering performance over several other methods for transferring constraints and allows multi-view clustering to be reliably applied when given a limited mapping between the views.

We present Confidence-based Feature Acquisition (CFA), a novel supervised learning method for acquiring missing feature values when there is missing data at both training and test time. Previous work has considered the cases of missing data at training time (e.g., Active Feature Acquisition, AFA [8]), or at test time (e.g., Cost-Sensitive Naive Bayes, CSNB [2]), but not both. At training time, CFA constructs a cascaded ensemble of classifiers, starting with the zero-cost features and adding a single feature for each successive model. For each model, CFA selects a subset of training instances for which the added feature should be acquired. At test time, the set of models is applied sequentially (as a cascade), stopping when a user-supplied confidence threshold is met. We compare CFA to AFA, CSNB, and several other baselines, and find that CFA's accuracy is at least as high as the other methods, while incurring significantly lower feature acquisition costs.

We describe the "Great Insights in Computer Science" courses that are taught at Rutgers and UMBC. These courses were designed independently, but have in common a broad, engaging introduction to computing for non-majors. Both courses include a programming component to help the students gain an intuition for computational concepts, but neither is primarily programming focused. We present data to show that these courses attract a diverse group of students; are rated positively; and increase students' understanding of, and attitudes towards, computing and computational issues.

Although there has been significant research on modelling and learning user preferences for various types of objects, there has been relatively little work on the problem of representing and learning preferences over sets of objects. We introduce a representation language, DD-PREF, that balances preferences for particular objects with preferences about the properties of the set. Specifically, we focus on the depth of objects (i.e. preferences for specific attribute values over others) and on the diversity of sets (i.e. preferences for broad vs. narrow distributions of attribute values). The DD-PREF framework is general and can incorporate additional object- and set-based preferences. We describe a greedy algorithm, DD-Select, for selecting satisfying sets from a collection of new objects, given a preference in this language. We show how preferences represented in DD-PREF can be learned from training data. Experimental results are given for three domains: a blocks world domain with several different task-based preferences, a real-world music playlist collection, and rover image data gathered in desert training exercises.

Previous algorithms for learning lexicographic preference models (LPMs) produce a "best guess" LPM that is consistent with the observations. Our approach is more democratic: we do not commit to a single LPM. Instead, we approximate the target using the votes of a collection of consistent LPMs. We present two variations of this method---variable voting and model voting---and empirically show that these democratic algorithms outperform the existing methods. We also introduce an intuitive yet powerful learning bias to prune some of the possible LPMs. We demonstrate how this learning bias can be used with variable and model voting and show that the learning bias improves the learning curve significantly, especially when the number of observations is small.

Previous studies of team formation in multi‐agent systems have typically assumed that the agent social network underlying the agent organization is either not explicitly described or the social network is assumed to take on some regular structure such as a fully connected network or a hierarchy. However, recent studies have shown that real‐world networks have a rich and purposeful structure, with common properties being observed in many different types of networks. As multi‐agent systems continue to grow in size and complexity, the network structure of such systems will become increasing important for designing efficient, effective agent communities. We present a simple agent‐based computational model of team formation, and analyze the theoretical performance of team formation in two simple classes of networks (ring and star topologies). We then give empirical results for team formation in more complex networks under a variety of conditions. From these experiments, we conclude that a key factor in effective team formation is the underlying agent interaction topology that determines the direct interconnections among agents. Specifically, we identify the property of diversity support as a key factor in the effectiveness of network structures for team formation. Scale‐free networks, which were developed as a way to model real‐world networks, exhibit short average path lengths and hub‐like structures. We show that these properties, in turn, result in higher diversity support; as a result, scale‐free networks yield higher organizational efficiency than the other classes of networks we have studied.

Context‐specific independence representations, such as tree‐structured conditional probability distributions, capture local independence relationships among the random variables in a Bayesian network (BN). Local independence relationships among the random variables can also be captured by using attribute‐value hierarchies to find an appropriate abstraction level for the values used to describe the conditional probability distributions. Capturing this local structure is important because it reduces the number of parameters required to represent the distribution. This can lead to more robust parameter estimation and structure selection, more efficient inference algorithms, and more interpretable models. In this paper, we introduce Tree‐Abstraction‐Based Search (TABS), an approach for learning a data distribution by inducing the graph structure and parameters of a BN from training data. TABS combines tree structure and attribute‐value hierarchies to compactly represent conditional probability tables. To construct the attribute‐value hierarchies, we investigate two data‐driven techniques: a global clustering method, which uses all of the training data to build the attribute‐value hierarchies, and can be performed as a preprocessing step; and a local clustering method, which uses only the local network structure to learn attribute‐value hierarchies. We present empirical results for three real‐world domains, finding that (1) combining tree structure and attribute‐value hierarchies improves the accuracy of generalization, while providing a significant reduction in the number of parameters in the learned networks, and (2) data‐derived hierarchies perform as well or better than expert‐provided hierarchies.

Interactive Visual Clustering (IVC) is a novel method that allows a user to explore relational data sets interactively, in order to produce a clustering that satisfies their objectives. IVC combines spring-embedded graph layout with user interaction and constrained clustering. Experimental results on several synthetic and real-world data sets show that IVC yields better clustering performance than alternative methods.

Many real-world domains resist analysis because they are best characterized by a variety of data types, including relational, spatial, and temporal components. Examples of such domains include disease outbreaks, criminal networks, and the World-Wide Web. We present two types of visualizations based on physical metaphors that facilitate fusion, analysis, and deep understanding of relational, spatio-temporal data. The first visualization is based on the metaphor of fluid flow through elastic pipes, and the second on wave propagation. We discuss both types of visualizations in the context of fusing information about the activities of scientists over time with the goal of constructing career histories

Most work on preference learning has focused on pairwise preferences or rankings over individual items. In this paper, we present a method for learning preferences over sets of items. Our learning method takes as input a collection of positive examples---that is, one or more sets that have been identified by a user as desirable. Kernel density estimation is used to estimate the value function for individual items, and the desired set diversity is estimated from the average set diversity observed in the collection. Since this is a new learning problem, we introduce a new evaluation methodology and evaluate the learning method on two data collections: synthetic blocks-world data and a new real-world music data collection that we have gathered.

BACKGROUND: Quantitative descriptions of amino acid similarity, expressed as probabilistic models of evolutionary interchangeability, are central to many mainstream bioinformatic procedures such as sequence alignment, homology searching, and protein structural prediction. Here we present a web-based, user-friendly analysis tool that allows any researcher to quickly and easily visualize relationships between these bioinformatic metrics and to explore their relationships to underlying indices of amino acid molecular descriptors. RESULTS: We demonstrate the three fundamental types of question that our software can address by taking as a specific example the connections between 49 measures of amino acid biophysical properties (e.g., size, charge and hydrophobicity), a generalized model of amino acid substitution (as represented by the PAM74-100 matrix), and the mutational distance that separates amino acids within the standard genetic code (i.e., the number of point mutations required for interconversion during protein evolution). We show that our software allows a user to recapture the insights from several key publications on these topics in just a few minutes. CONCLUSION: Our software facilitates rapid, interactive exploration of three interconnected topics: (i) the multidimensional molecular descriptors of the twenty proteinaceous amino acids, (ii) the correlation of these biophysical measurements with observed patterns of amino acid substitution, and (iii) the causal basis for differences between any two observed patterns of amino acid substitution. This software acts as an intuitive bioinformatic exploration tool that can guide more comprehensive statistical analyses relating to a diverse array of specific research questions.

In most graphics and visualization applications, the effects of the mapping parameters on the output domain are multidimensional, non-linear and discontinuous. The complexity of such mapping often makes it difficult for a user to manually explore and manipulate the design parameter space to produce the desired output. Computer assistance is therefore useful in setting the mapping parameter values to generate desired outputs. Existing systems rely on exploring the entire input parameter space, which can be time and resource-intensive, particularly if the number of input parameters is large. We introduce a new approach to handling a large number of mapping parameters more efficiently. The basis for our approach is the identification of a small and effective set of highlevel parameters that can be associated directly with the characteristics of the outputs. Users will have a better understanding of this small set of high-level parameters and can easily modify their values interactively to produce the desired outputs. We demonstrate this technique in manipulating mapping parameters for a non-photorealistic volume rendering application.

Using inductive learning techniques to construct classification models from large, high-dimensional data sets is a useful way to make predictions in complex domains. However, these models can be difficult for users to understand. We have developed a set of visualization methods that help users to understand and analyze the behavior of learned models, including techniques for high-dimensional data space projection, display of probabilistic predictions, variable/class correlation, and instance mapping. We show the results of applying these techniques to models constructed from a benchmark data set of census data, and draw conclusions about the utility of these methods for model understanding.

In order to build realistic AI planning systems, it is necessary to develop sophisticated tools for knowledge acquisition. This paper describes two knowledge acquisition tools for a crisis action planning system. The first is a graphical operator editor that enables users to develop new planning operators and revise existing operators. The second is an inductive learning system, based on the PAGODA learning model, that learns from simulator feedback and from choices made by the user during planning. This paper describes the work done so far, and proposed for the future, on these tools.>