For five academic years we have engaged in an on-going study of the effectiveness of online assessment of student programming abilities for introductory programming courses in Computer Science. Our results show that online evaluation can be implemented securely, efficiently, and can result in increased student motivation and programming efficacy; however, unless online components are integrated throughout the course evaluations, student competence will be underestimated. Our data reveals disadvantages of online evaluations, but also shows that both students and faculty benefit when online evaluations are implemented appropriately.

We present a foundational theory of software system reliability based on components. The theory describes how component developers can design and test their components to produce measurements that are later used by system designers to calculate composite system reliability, without implementation and test of the system being designed. The theory describes how to make component measurements that are independent of operational profiles, and how to incorporate the overall system-level operational profile into the system reliability calculations. In principle, the theory resolves the central problem of assessing a component, which is: a component developer cannot know how the component will be used and so cannot certify it for an arbitrary use; but if the component buyer must certify each component before using it, component based development loses much of its appeal. This dilemma is resolved if the component developer does the certification and provides the results in such a way that the component buyer can factor in the usage information later without repeating the certification. Our theory addresses the basic technical problems inherent in certifying components to be released for later use in an arbitrary system. Most component research has been directed at functional specification of software components; our theory addresses the other equally important side of the coin: component quality.

We have experimented with the use of weekly on-line quizzes to enhance student learning in our first-year computer science courses. In our experiments we compared the effectiveness of using quizzes to the alternative of using weekly marked laboratory assignments. The results of our experiments show that student learning and retention increase with on-line quizzes. Weekly quizzes would be impossible if they were administered and marked in the traditional fashion; thus, we developed and used a secure, online environment for administering, writing, and marking the quizzes, with most of the marking performed automatically via simple marking programs. In this paper we describe our experiment, present our observations about student learning, outline student opinion, relate problems we encountered and our solutions, and provide technical details of our closed-quiz and marking environment.

Online marking of assignments can lead to improved marking consistency and integrate well with on-line mark reporting. It can also be easier for the marker. Unfortunately, most such systems do not include good feedback mechanisms for the students. This paper describes an environment that provides online marking with convenient, structured and detailed feedback.

We have been using a combination of conventional and on-line programming examinations for the past 3 years in our first year programming courses. We plan to continue using on-line components because of the positive feedback from both students and faculty. We describe the operational and pedagogical lessons we have learned, illustrating problems encountered and our solutions, in the hope that others considering on-line examinations may also benefit from our experiences.

On-line programming tests and examinations were administered to approximately 120 first year computer science students in order to evaluate their practical skills. We describe our motivation for on-line testing, outline the technical details of our closed testing environment, and present our observations about student performance. We also compare the effectiveness of on-line tests versus conventional tests, report the problems we encountered and our solutions, relate student opinion regarding the on-line testing experiment, and present some insights gained by this experiment.

Copying garbage collectors are now standard for the memory-management subsystems of functional and object-oriented programming languages. Compacting garbage collection has correspondingly fallen out of favor. We revitalize the case for compaction by demonstrating that a simple compacting collector, extended with the generational garbage collection heuristic, exhibits performance as effectively as or better than a well-designed generational copying collector on real programs running in real environments. The observation that compaction preserves allocation order across collections leads to a new generalization of the generational heuristic that reduces the movement of long-lived objects. We measure the effect of substituting our compacting generational collector for a copying collector in Standard ML of New Jersey.