Background and Context: Empirical researchers have a long-standing tradition of explicitly discussing the threats to and limitations of their research. In the past twenty years, these discussions emerged as a standard component of empirical research papers in computing education research (CER) as well.

Peer review is a mainstay of academic publication - indeed, it is the peer-review process that provides much of the publications' credibility. This working group is examining the ways peer review is used in various computing education venues and will use this examination to articulate community standards for peer review in this discipline.

This session announces the six best papers from the first five ICER conferences, as chosen by a committee of former and current ICER chairs. It explains how these papers were selected, and for each paper gives its abstract, and comments from the committee explaining why each paper is an exemplary ICER paper and contribution to the field of computing education research.

We present an analysis of the visual portrayals of data (including results) in the full papers and working group reports of ITiCSE from 2013 to 2018. We find that tables are the most common visual portrayal of data in these publications, but that a number of graphical forms are also widely used. We examine the quality of the data portrayals for tables, graphs, and images using visual quality indicators derived from the literature. Overall, our findings are not positive. We find that many papers present data in such a way that it cannot be readily interpreted. The most common problem is captions that do not adequately describe the table, figure, or image. In tables, the main issues affecting readability of numeric data are poor alignment of numbers within a column, unnecessary notations, and unwarranted precision. In graphs and images, the prevalent problem is text that is too small to be read. We conclude with guidelines for future authors to ITiCSE and other computing education venues, in the hope that we can contribute to an improvement in the quality of computing education publications.

Students' "copying" is often considered negatively. In this paper, we explore the ways in which copying and imitation are used positively by computing instructors in their teaching. We found that instructors expect their students to use these strategies in many different contexts and at many different levels.

In this paper, we investigate the degree to which ITiCSE Working Groups have served to connect new researchers to the larger computing education community. Previous analysis of computing education research collaborations has concluded that computing education researchers collaborate with fewer people and that newcomers have a harder time integrating into the community, compared with other computer-science fields.

Conferences such as ITiCSE have recently seen an increase in the number of papers presenting empirical research in computing education. While empirical research need not be quantitative, there has been a corresponding increase in the number of papers that present some level of statistical analysis to support their arguments. This poster introduces a project that is exploring how -- and how well -- statistics are used and reported in computing education research.

In order to help students learning to develop computer programs, several computing education researchers have analyzed the compiler error messages generated by novices' attempts to compile their programs. The goal is to help students diagnose the errors they make through the messages generated by the compiler. This paper builds on that previous work by applying a technique based on a heuristic well-known to programmers - fix the first error and ignore the rest - to the analysis of over 21 million compiler error messages from the Blackbox dataset. We find that the ranks and frequencies obtained by considering all error messages are generally consistent with previously published lists, but when we consider first messages only, these ranks and frequencies are different. These differences could have important implications for teaching, and can inform tool design and future research efforts.

In a survey of the computing education community, many respondents suggested "active learning" as a teaching approach that would increase the likelihood of student success. In light of these responses, we analyze the way in which active learning is described in the computing-education literature. We find a strong consensus that active learning is good, but a lack of precision in how the term is used, often without definition, to describe instructional techniques, rather than student learning. In addition, active learning techniques are often discussed as if they were all equally effective. We suggest that making clear distinctions, both between teaching techniques and active learning and among the teaching techniques, would be fruitful for both instructors and researchers. Finally, we propose some dimensions along which distinctions among techniques could usefully be made.

Drawing on the long debate about whether computer science (CS) and computational thinking skills are innate or learnable, this working group is based on the following hypothesis: The apparent innate ability of some CS learners who succeed in CS courses despite no prior exposure to computing is a manifestation of early childhood experiences and learning outside formal education.

In a survey of the CS-education community, we find a range of beliefs about the "geek gene" theory. We suggest an alternative term, the "geekiness quotient (GQ)". The GQ, grounded in Gardner's work on multiple intelligences, is a hypothetical measure of the student's current CS ability. The GQ supports a moderate view of the geek gene: that students arrive in our classrooms with a range of CS abilities, whether acquired through background or innate talent, and can improve their abilities through effort.

Peer Instruction (PI) is a student-centric pedagogy in which students move from the role of passive listeners to active participants in the classroom. Over the past five years, there have been a number of research articles regarding the value of PI in computer science. The present work adds to this body of knowledge by examining outcomes from seven introductory programming instructors: three novices to PI and four with a range of PI experience. Through common measurements of student perceptions, we provide evidence that introductory computing instructors can successfully implement PI in their classrooms. We find encouraging minimum (74%) and average (92%) levels of success as measured through student valuation of PI for their learning. This work also documents and hypothesizes reasons for comparatively poor survey results in one course, highlighting the importance of the choice of grading policy (participation vs. correctness) for new PI adopters.

In this article, we address the question of why computing students choose to learn computing topics on their own. A better understanding of why some students choose to learn on their own may help us to motivate other students to develop this important skill. In addition, it may help in curriculum design; if we need to leave some topics out of our expanding curriculum, a good choice might be those topics that students readily learn on their own. Based on a thematic analysis of 17 semistructured interviews, we found that computing students’ motivations for self-directed learning fall into four general themes: projects, social and peer interactions, joy of learning, and fear. Under these, we describe several more specific subthemes, illustrated in the words of the students. The project-related and social motivations are quite prominent. Although these motivations appear in the literature, they received greater emphasis from our interviewees. Perhaps most characteristic of computing is the motivation to learn to complete some project, both projects done for fun and projects required for school or work.

When students talk about their university experiences, they identify particular events as being significant in their development. We look at the experiences relating to coursework and career of two students who were interviewed throughout their undergraduate degree programs. We found that they shared some of the significant events but not all, and the reactions to the shared events were sometimes quite different.

This editorial marks the end of our tenure as founding co-editors-in-chief of the ACM Transactions on Computing Education (TOCE). We have three goals in this editorial. First, we provide a retrospective on how we positioned TOCE, both in terms of how it embodies our conception of Computing Education Research (CER) as a field, as well as the journal's role in the larger computing education community and the ACM. We focus on the process by which we determined what constitutes publishability for a manuscript submitted to TOCE, describing what is best understood as a living process negotiated among the authors, reviewers, associate editors, and editors-in-chief in interaction with manuscripts and one another. Second, having reviewed a bit of history, we look at some possible changes in TOCE's future, particularly in light of conversations that the ACM is engaged in at all levels regarding recent open publishing initiatives as well as the relationship between journal and conference publications. Finally, we thank the many people who have contributed to the success of TOCE for their significant effort in establishing TOCE as a publication venue for papers in computing education.

We expand upon previous research that looked at the question: "Can graduating students design software systems?" Specifically we want to examine students' understanding of the phenomenon "produce a design." What does this instruction mean to them? In order to investigate student understandings, we examined their designs using a phenomenographic approach.

Students in a maintenance-centric, introductory software engineering course were expected to understand, analyze and extend an open source software project of their choice, selected from a limited set of prepared applications. Students fell into two groups: those who chose a project based on its perceived and estimated difficulty, and those who chose a project based on the appeal of the subject matter. Students in both groups, however, cited value for themselves in terms of enhanced learning experience, and for users in terms of increased benefit, as reasons for their selection. These insights into students' thinking can guide future efforts in selecting projects that can simultaneously support the learning objectives as well as motivate the students, not only in software engineering but also in broader computing courses.

Aspiring software engineers must be able to comprehend and evolve legacy code, which is challenging because the code may be poorly documented, ill structured, and lacking in human support. These challenges of understanding and evolving existing code can be illustrated in academic settings by leveraging the rich and varied volume of Open Source Software (OSS) code. To teach SE with OSS, however, it is necessary to select uniform projects of appropriate size and complexity. This paper reports on our search for suitable OSS projects to teach an introductory SE course with a focus on maintenance and evolution. The search turned out to be quite labor intensive and cumbersome, contrary to our expectations that it would be quick and simple. The chosen projects successfully demonstrated the maintenance challenges, highlighting the promise of using OSS. The burden of selecting projects, however, may impede widespread integration of OSS into SE and other computing courses.

This special issue on computing education in (K-12) schools represents considerable effort by the editorial team, authors, and reviewers. It provides a series of country-specific case studies of computing education in schools that highlights the way in which curricula emerge from each country’s specific historical and cultural circumstances. As a result, not only is there much to learn from each of the case studies, but there are additional lessons in the commonalities and generalizations obtainable only by having a rich set of case studies such as these that can be viewed comparatively.

Software Engineering (SE) careers are overwhelmingly devoted to the maintenance and evolution of existing, large software systems, where the key challenge is code comprehension especially in the face of inadequate documentation and support. SE courses must thus prepare students to meet this challenge. Open Source Software (OSS) furnishes a valuable source of realistic, sizeable projects for inculcating the appreciation and skills involved in code comprehension and evolution. This paper describes experiences and lessons learnt in using OSS projects to teach an introductory, sophomore/junior-level SE course with an emphasis on comprehension, maintenance, and evolution. Students' reactions and undertakings, acquired through participant observation and homework assignments, suggest that OSS can meaningfully illustrate comprehension and evolution difficulties. Finally, it describes the characteristics of OSS projects that are conducive to highlighting maintenance challenges.

Theory of Computation provides students of the computing disciplines with understanding that some problems are not solvable, and that there is a range of complexities of problems, in terms of significant resources, including but not limited to time and memory. Theory of Computation acquaints students with reasoning processes by which they can know (for some cases) the complexity class of a given problem, and by which they can, for some problems they might find in the future, determine the complexity. People formulate these reasoning processes in symbols, and use logic with quantifiers; an example of this is the application of the pumping lemma for regular languages. We categorized the errors students made in applying the pumping lemma for regular languages, to develop an understanding of the possible misconceptions, in turn, to improve teaching.

The McCracken et al. working group paper is often cited for the proposition that students can't program. In that study, students from four different institutions were each assigned to implement one of three versions of a calculator. More than half of the students failed to produce a program that compiled and executed in the time assigned.

We developed a software engineering course that emphasizes code maintenance and evolution by having students reverse engineer and modify open-source projects. To evaluate whether this course had the desired effects on student learning, we analyze pre- and post-course survey data using qualitative methods. This analysis, in combination with other data, suggests that the students gained an appreciation and understanding of software maintenance, documentation, and tool use.

Threshold concepts can be used to both organize disciplinary knowledge and explain why students have difficulties at certain points in the curriculum. Threshold concepts transform a student's view of the discipline; before being learned, they can block a student's progress.

This study examines the extent to which novice computing students with minimal computer science coursework and no training in user interface (UI) evaluation consider UI concepts such as usability, user experience, and the context in which software will be used when evaluating an interface. In analyzing the responses of 149 novice computer science students who were asked to evaluate two interfaces for converting temperatures between Fahrenheit and Celsius, we observed that students generally considered usability and user experience factors, but were less likely to consider context. For educators, this exact task could be given to a class in order to initiate discussion of user-centered design; the study also provides a framework for structuring the discussion. More generally, the results of this study provide insight into some opportunities and challenges in teaching good interface design and evaluation skills.

Research has shown that most learning in the workplace takes place outside of formal training and, given the swiftly changing nature of the field, computer science graduates more than most workers, need to be able to learn computing topics outside of organized classes.

SIGCSE's special projects grant program provides small grants to SIGCSE members to carry out computing education projects whose results generalize to the whole computing education community. This session is a "showcase" for five recent grant projects: a MIPS simulator extended into FPGA hardware, a study of informal learning of computing, software engineering learning modules for the visually impaired, a study of student projects in computing, and curriculum resources for computer security.

We often learn of successful pedagogical experiments, but we seldom hear of the the ones that failed. For this special session we solicited submissions from the SIGCSE membership, selected the best from among these, and will have presentations at the session by the selected authors. Our contributions describe pedagogical approaches that seemed to be good ideas but turned out as failures. Contributors will describe their pedagogical experiment, the rationale for the experiment, evidence of failure, and lessons learned.

It is important for students to be able to select and apply the appropriate data structure for the problem to be solved. Testing this knowledge on exams can be difficult, however. We examined 59 data structures final exams and found only 36 that contained questions involving the application of data structures. To promote assessment of this knowledge in the data structures course, we present a framework for classifying apply exam questions, with illustrations from the exams collected. We then show how a number of questions can be developed by varying a single rich apply question along the dimensions of this framework

The goal of this editorial is to to provide entry points into the literature on making and warranting claims in the social and behavioral sciences that might be of use to computing educators. In addition, we provide some heuristic advice on getting started and continuing along this direction based on our experience as computing education researchers.

The goal of this editorial is to both encourage authors to continue to submit articles describing computational tools for computing education to TOCE, and to make explicit our expectations as the editors of TOCE for what makes a good “tools” article. We elaborate three key criteria, and provide examples from previous TOCE papers that satisfy these criteria: 1) establish that the tool is related essentially to computing education, 2) make explicit claims about the impact of the tool on learning, and 3) evidence these claims through actual use.

Is there consensus on what students should learn in CS2? Should they learn to use data structures, understand their specific implementation details, or both? Finally, has the computing education community's answer to the second question changed over time? In this paper, we begin to explore these questions based on an analysis of a key artifact instructors use to assess their students' performance: their final exams. Specifically, we look at two CS2 concepts as covered in those exams: stacks and hashtables.

No abstract available.

Scholarly communities in computing determine how to produce and validate knowledge within their domains of focus. Much of this knowledge can remain tacit because of shared ways of becoming a disciplinary scholar within any particular area of computing. But such tacitness presents challenges for computing education scholarship, since knowledge about how to create and validate claims about human thinking and learning is rarely a part of the training for computing educators. The editors-in-chief of TOCE have responded to these challenges by making explicit the epistemic practices associated with the publication of its manuscripts, by encouraging a range of borrowing from other disciplines, and by instituting feedback loops at all levels of review.

This paper investigates what students understand about algorithm efficiency before receiving any formal instruction on the topic. We gave students a challenging search problem and two solutions, then asked them to identify the more efficient solution and to justify their choice. Many students did not use the standard worst-case analysis of algorithms; rather they chose other metrics, including average-case, better for more cases, better in all cases, one algorithm being more correct, and better for real-world scenarios. Students were much more likely to choose the correct algorithm when they were asked to trace the algorithms on specific examples; this was true even if they traced the algorithms incorrectly.

We examine the changes in the ways computing students view their field as they learn, as reported by the students themselves in short written biographies. In many ways, these changes result in students thinking and acting more like computer scientists and identifying more with the computing community. Most of the changes are associated with programming and software engineering, rather than theoretical computer science, however.

We examine the transformations experienced by students during their study of computing. These transformations led to changes in the students' perceptions of computer science, in their sense of identity as computer scientists, their behavior and their confidence. The changes are caused by learning or using particular concepts, and often associated with writing computer programs, learning new programming languages, or interacting with peers.

No abstract available.

Most scientific articles address particular questions which are part of larger questions. The part-whole relationship varies.

The ACM Transactions on Computing Education (TOCE) represents a name change and a broadening of scope for the ACM Journal on Educational Resources on Computing (JERIC). The topics covered by this new journal will range across diverse aspects of computing education: traditional computer science, computer engineering, software engineering, information technology, and informatics; emerging aspects of computing; and applications of computing to other disciplines, such as computational biology. The common characteristics shared by these articles are a scholarly approach to teaching and learning, a broad appeal to educational practitioners, and a clear connection to student learning.

This paper examines transformational learning experiences of computing students as a way to better understand threshold concepts in computing. From empirical evidence we found that students often describe transformative experiences as learning situations in which they were led to use various kinds of abstraction, for example modularity, data abstraction, inheritance, polymorphism, reuse, design patterns, and complexity. Some students describe an abstract concept as coming first, and then needing to be made concrete though application; others describe transformations in which they learn the advantages of these abstract concepts from their experience of not using them.

Data sharing is common, and sometimes even required, in other disciplines. Creating a mechanism for data sharing in computer science education research will benefit both individual researchers and the community. While it is easy to say that data sharing is desirable, it is much more difficult to make it a practical reality.

We often learn of successful pedagogical experiments, but we seldom hear of the the ones that failed. For this special session we solicited submissions from the SIGCSE membership, selected the best from among these, and will have presentations at the session by the selected authors. Our contributions describe pedagogical approaches that seemed to be good ideas but turned out as failures. Contributors will describe their pedagogical experiment, the rationale for the experiment, evidence of failure, and lessons learned.

In this paper, we present the results of an experiment in which we sought to elicit students' understanding of object-oriented (OO) concepts using concept maps. Our analysis confirmed earlier research indicating that students do not have a firm grasp on the distinction between "class" and "instance." Unlike earlier research, we found that our students generally connect classes with both data and behavior. Students rarely included any mention of the hardware/software context of programs, their users, or their real-world domains. Students do mention inheritance, but not encapsulation or abstraction. And the picture they draw of OO is a static one: we found nothing that could be construed as referring to interaction among objects in a program. We then discuss the implications for teaching introductory OO programming.

Revising and extending conference articles for journal publication benefits both authors and readers. The new articles are more complete, and benefit from peer review, feedback from conference presentation, and greater editorial consistency. For those articles that are appropriate, we encourage authors to do this, and present two examples of such articles in this issue.

Some have proposed that realistic problem situations are better for learning. This issue contains two articles that examine the effects of “making it real” in computer architecture and human-computer interaction.

This special issue features a set of papers recently published in the 3rd International Workshop on Computing Education Research (ICER'07). The papers were selected because they closely meet the publication criteria for ACM/JERIC: stemming from computing education practice, grounded in relevant literature, containing analysis of primary empirical data collected to support specific claims, and providing implications for practice. In addition, the diverse set of research methods used demonstrates how empirical methods can be chosen to serve the specific questions of interest, whether for purposes of research or for improvement of teaching practice. This set of papers can therefore serve as examplars and resources for the educator wishing to develop a wider range of tools for probing issues of teaching and learning within the classroom.

We invite submissions to ACM/JERIC from teaching practitioners who are our readers. Grounding these submissions in the lived practice of teaching; using data that is already collected to assess student learning; and reporting the data collection, analysis, and context of use accurately and honestly are key aspects for taking a scholarly approach to teaching and learning. We hope to see more such scholarly inquiries by computing educators in the pages of future issues.

This paper is part of an ongoing series of projects in which we are investigating "threshold concepts": concepts that, among other things, transforms the way a student looks as the discipline and are often troublesome to learn. The word "threshold" might imply that students cross the threshold in a single "aha" moment, but often they seem to take longer. Meyer and Land introduce the term "liminal space" for the transitional period between beginning to learn a concept and fully mastering it.

In this paper we present a phenomenographic analysis of computer science instructors' perceptions of student success. The factors instructors believe influence student success fell into five categories which were related to: 1) the subject being taught, 2) intrinsic characteristics of the student, 3) student background, 4) student attitudes and behaviour and 5) instructor influence on student development. These categories provide insights not only into how instructors perceive students, but also how they perceive their own roles in the learning process. We found significant overlap between these qualitative results, obtained through analysis of semi-structured interviews, and the vast body of quantitative research on factors predicting student success. Studying faculty rather than students provides an alternative way to examine these questions, and using qualitative methods may provide a richer understanding of student success factors.

As the third in a series of projects investigating commonsense computing -- the relevant knowledge that students have before any formal study of computing -- we examine students' commonsense understanding of concurrency. Specifically, we replicated (with modifications) an experiment by Ben-David Kolikant. [2] Ben-David Kolikant's data were gathered from high-school seniors who had previously studied computing, at the beginning of an advanced class in concurrent and distributed programming. Modifying one of her questions to reflect our students' lack of background, we asked students at five different institutions, in the first week of CS1, to describe in English the problems that might arise when more than one person is selling seats to a concert.Almost all students (97%) identified the problem of interest -- that a race condition may occur between sellers. 73% of students identified at least one possible solution. We found that the categorizations developed by Ben-David Kolikant were also meaningful when applied to our data, that our beginning CS1 students are more likely to give centralized solutions (as opposed to decentralized ones) than Ben-David Kolikant's concurrency students, and that the granularity of solutions is finer among the more experienced students.

Students often get stuck when trying to learn new computing concepts and skills. In this paper, we present and categorize strategies that successful students found helpful in getting unstuck. We found that the students reported using a broad range of strategies, and that these strategies fall into a number of recognizably different categories.

Yes, and Yes.We are currently undertaking an pirical investigation of "Threshold Concepts" in Computer Science, with input from both instructors and students. We have found good pirical evidence that at least two concepts---Object-oriented programming and pointers--are Threshold Concepts, and that there are potentially many more others.In this paper, we present results gathered using various experimental techniques, and discuss how Threshold Concepts can affect the learning process.

No abstract available.

We examine students' commonsense understanding of computer science concepts before they receive any formal instruction in the field. For this study, we asked students on the first day of a CS1 class to describe in English how they would arrange a set of numbers in ascending, sorted order; we then repeated the experiment asking students to sort a list of dates (in mm/dd/yyyy format).We found that a majority of students described a coherent algorithm; some described versions of insertion or selection sort, but many gave unexpected algorithms. We also found significant differences between responses given for sorting numbers versus dates. Based on our analysis of the data we suggest that beginning-programming instructors more explicitly discuss data types, begin loop instruction with post-test loops, assist students in recognizing implicit conditional and iteration use in natural language solutions to probls, and recognize that novices and experts focus on different aspects of the probl in even basic probl solving tasks.

As the new Editors-in-Chief, we outline our vision for building JERIC to become the premier journal in computing education. Doing so will require authors who submit high-quality articles--their „best stuff”, reviewers who provide the feedback and guidance to ensure that each article embodies a scholarly approach to teaching and learning while being of practical use to the computing educator, and readers who read the articles, critically evaluate and discuss the findings with others, and adapt, adopt, and evaluate these findings in their own classrooms.

A goal of JERIC is to support educational practice by publishing articles that are useful to classroom instructors. One way of ensuring this utility is to require that claims are backed up by evidence. This evidence allows the reader to evaluate the potential effectiveness and transferability of any appraoches presented in these articles to their own courses and situations.

This issue is devoted to the curriculum guidelines from the Liberal Arts Computer Science Consortium. These guidelines provide a coherent and important model for computing education within a liberal arts context, giving primacy to critical reason, rigorous methods, and student engagement in the research process. In this regard, they are at the same time both conservative, by giving low priority to the latest technologies, and radical by resisting the managerial impulse that at its worst commidifies and devalues higher education.

We examine students' commonsense understanding of computer science concepts before they receive any formal instruction in the field. Specifically, we asked students on the first day of a CS1 class to describe in English how they would arrange a set of numbers in ascending, sorted order. We repeated the experiment with students in an introductory economics course, and again with a sub-population of the CS1 students after ten weeks of Java instruction.We found that a majority of beginning computing students could describe a coherent algorithm to correctly sort a list of numbers, while less than a third of general college students could do so. Many students gave versions of selection or insertion sort, but the most common algorithm treated numbers as strings and manipulated them digit by digit. Students who used iteration strongly preferred post-test loops. Finally, some aspects of student performance became worse after ten weeks of CS1 instruction.

We are studying what students know about computer science-related topics before they take formal coursework at the university level. Preliminary results suggest that entering students have a fairly sophisticated understanding of algorithms. We are exploring other central computing topics for similar shared commonsense understanding.

This paper describes Threshold Concepts, a theory of learning that distinguishes core concepts whose characteristics can make them troublesome in learning. With an eye to applying this theory in computer science, we consider this notion in the context of related topics in computer science education.

This paper examines software designs produced by students nearing completion of their Computer Science degrees. The results of this multi-national, multi institutional experiment present some interesting implications for educators.

Novice programmer knowledge contains a mixture of well-formed, in-transition and muddled conceptual structures. In this paper we describe an analysis of the in-transition and muddled items that are not fully integrated into the novices' cognitive structures. When participants were asked to perform card sorts of programming concepts into categories, 23% of their categories were "ragbags": categories with names such as "don't know," "not sure," or "not applicable"'' that indicate that the students have little or no knowledge of the concepts placed in those categories.In this study, we find that there are distinct differences in the uses of the ragbags. In particular, we find that terms considered more abstract tend to be placed into Don't Know and Not Sure ragbags more often than concrete terms; and students categorized as low performers tend to use Not Sure far more often than high performers but Don't Know and Not Applicable less often. We also find evidence that the meaningfulness of a concept is likely to be related to the vocabulary used in the classroom, suggesting that students may assimilate abstract concepts into their conceptual structures more quickly if one uses the terms more frequently.

We study how the relative values placed on a variety of software design criteria differs between beginning students, advanced students, and educators. We also consider how these values change depending on the specific design situation. Statistical analysis of data collectedfrom over 300 subjects reveals relatively small differences between the two student populations and significant differences between educators and students. In addition to often valuing different criteria, educators also value criteria more consistently across the design situations.

Knowledge about the characteristics of a discrete optimization problem can often be leveraged for improved algorithm performance. Having an understanding of the relationship between knowledge and performance can provide us useful hints and better prediction of algorithm behavior when we choose or design algorithms. Previously, we developed a directional tree (DT) model [X. Zhong et al., (2003)], which concurrently describes algorithm behavior and represents knowledge explicitly, and thus provided us with a tool to analyze the impact of knowledge on algorithm performance. By using DT, we analyzed the performance of an algorithm when that algorithm is used for solving problems from a problem space P. In our previous analysis, we only considered the situation where there is only one optimal solution in the solution space. Furthermore, our DT model assumes that knowledge about the problem is obtained prior to the execution of any probabilistic or deterministic algorithm. However, such prior knowledge may not be readily or immediately available. In this paper, we extend our DT results to cover multiple optimal solutions and then generalize our DT model to take into account acquisition of problem knowledge during execution. Our generalized directional tree (GDT) model provides a framework for incorporating knowledge obtained online into algorithms, and thus enables us to develop online optimization algorithms that dynamically updates their search decisions in response to obtained knowledge so far.

In this paper, we investigate the question of what assessment tools are being used in practice by United States computing programs and what the faculty doing the assessment think of the tools and of the assessment process in general. We report on the results of two surveys, distributed to the SIGCSE mailing list and to the chairs of all ABET-CAC accredited computer science programs. We found that both parts of these surveys--the multiple-choice sections and the comments--provided some interesting insights into program assessment from the perspective of those responsible for collecting and using the information.

Unlike most games which have fixed rules, the rules for war games can contain uncertainty. This uncertainty makes war games difficult to address with methods typically used for playing games by machine. The characteristics of war games match well with the domain for which genetic algorithms are effective. We explore the use of genetic algorithms for generating war game strategies.

In order to address the domain of more complex robots and problems, robot controllers and robot structures will need to be co-evolved. To achieve the goal of the automated design of complex robotics systems and their associated control programs, a method for managing the complexity of the solution during evolution is needed. One way to do this is through the use of a cost term in the evolutionary algorithm evaluation function. In this paper we explore the use of a cost term in an evolutionary robotics evaluation function. A general form of an evaluation function containing a cost term is developed. The cost term is dynamically scaled as a function of the amount of progress the robot makes towards its goal. It has the least influence during the initial development of a solution so as not to overly restrict exploration of the search space. As the robot is able to advance further towards its goal, the cost term becomes more prominent and is used to reduce unneeded sensors and increase the efficiency of the control program. A series of experiments using a simulated robot are described. The experiments increase in difficulty and show that the general form of the evaluation function is robust and can be successfully applied in several problem domains.

For any system that uses previous experience to solve problems in new situations, it is necessary to identify the features in the situation that should match features in the previous cases through some process of situation analysis. In this paper, we examine as issue spotting; in particular, we present how issues spotting is implemented in CHASER, a legal reasoning system that works in the domain of tort law.The approach presented here is a compromise between generality and efficiency, and is applicable to a range of problems and domains outside of legal reasoning. In particular, it presents a principled way to use multiple cases for a single problem by exploiting the inherent structure present in many domains.