Automated Assessment Systems (AAS) are increasingly used in large computer science lectures to evaluate student solutions to programming assignments. The AAS normally carries out static and dynamic analysis of the program code. In addition, simple forms of learning analytics can often be generated quite easily. However, structural analyses and comparison of solutions for larger sets of student programs are, in many cases, complicated and time-consuming. In this article, we introduce a methodology with which thousands of programs can be analyzed in less than a second, for example, to search for the use of certain control structures or the application of recursion. For this purpose, we have developed a software that creates a structural representation for each programming solution in the form of a TGraph, which is inserted into a graph database using Neo4j. On this database, we can search for structural features by queries in the language Cypher. We have tested this methodology extensively for Java programs, measured its performance, and validated the results. Our software can also be applied to programs in other programming languages, such as Scratch. Additionally, we plan to make our software available to the community.

The workshop connects to the central theme “Educating for the Future” of CSEE&T 2020 and thus explores opportunities to improve Software Engineering education and training by using Essence. Essence, an OMG Standard, delivers essential, universal elements found in all Software Engineering (SE) endeavours and a language to describe and extend these elements and their use in concrete practices to tailor SE methods to teams' needs. Its Kernel separates the stable What and Why from the more adaptable How. That way, it provides an essential thinking framework facilitating the adoption and customization of practices and methods. As such, it supports multiple ways of educating SE students as well as training practitioners for the future. The first deployments of Essence in industry and academic courses around the globe show promising results. But since it is an entirely new way to think about SE practices, methods, and their building blocks—it requires some effort and thought to dive into the standard. This workshop facilitates the adoption by sharing experiences, best practices, and lessons learned.

To overcome the shortage of computer specialists, there is an increased need for correspondent study and training offers, in particular for learning programming. The automated assessment of solutions to programming tasks could relieve teachers of time-consuming corrections and provide individual feedback even in online courses without any personal teacher. The e-assessment system JACK has been successfully applied for more than 12 years up to now, e.g., in a CS1 lecture. However, there are only few solid research results on competencies and competence models for object-oriented programming (OOP), which could be used as a foundation for high-quality feedback.In a joint research project of research groups at two universities, we aim to empirically define competencies for OOP using a mixed-methods approach. In a first step, we performed a qualitative content analysis of source code (sample solutions and students’ solutions) and as a result identified a set of suitable competency components that forms the core of further investigations. Semi-structured interviews with learners will be used to identify difficulties and misconceptions of the learners and to adapt the set of competency components. Based on that we will use Item Response Theory (IRT) to develop an automatically evaluable test instrument for the implementation of abstract data types. We will further develop empirically founded and competency-based feedback that can be used in e-assessment systems and MOOCs.

At least since the first PISA surveys, the intentions of education are increasingly expressed in terms of competencies instead of learning goals. In consequence, learning outcomes should be measured in terms of these intended competencies as well. This holds for large-scale investigations like PISA as well as for the very small scale that is represented by the examinations that a single teaching person performs with his/her students. Yet, to assure validity, competency measurement requires proper models for competency structure and levels, which are based on empirical research. The goal of this project is to identify a set of relevant competencies in the domain of object-oriented programming (OOP), to develop a suitable competency model to represent their structure and to construct proper corresponding test instruments. This paper describes a study on the outcomes of several assignments that represents one step of this process with focus on the subdomain of data structures. The data were collected during an introductory lecture on programming in the winter term 2015/16. We analyzed the student solutions of three mini-projects that deal with the implementation of abstract data, applying a specific e-assessment system to score the students' solutions. To find suitable item sets that could be used for the definition and measurement of certain competencies, different methodologies were applied that are based on classical test theory as well as on contemporary Item Response Analysis. Up to now, several suitable item sets for the competencies were identified, which are required to implement and operate 1- and 2-dim arrays, linked lists, and binary trees.

While traditional project planning approaches focus on precise scheduling of tasks, the ESSENCE standard proposes a higher-level approach that focuses on monitoring. Hence, a new kind of process visualization that picks up ideas of Kanban boards and physical cards is sketched in the standard. This tool paper presents a dashboard application refining, extending, and implementing these ideas based on five use cases posed by two industry partners. It demonstrates that a high degree of support for project management can be achieved by using a relatively small set of visualization means.

Current approaches to automated assessment of UML diagram mainly focus on static analysis. As this may be insufficient for behavioural diagrams, this contribution presents an approach to dynamic checks of UML Activity Diagrams. The approach makes use of trace generation and sequence alignment and shows fair results in preliminary experiments.

Programming requires both to write code and to read code. In debugging tasks, students can use supporting tools that create additional artefacts like visualizations or traces, but these again need to be read and understood. This contribution presents a concept of code reading exercises that asks students to create the same artefacts for given code as they would use for debugging their own code.

The new software engineering initiative, Semat, is in the process of developing a kernel for software engineering that stands on a solid theoretical basis. So far, it has suggested a set of kernel elements for software engineering and basic language constructs for defining the elements and their usage. This paper describes a session during which Semat results and status will be presented. The presentation will be followed by a discussion panel.

Automated checks for software artefacts like UML diagrams used in automated assessment or tutoring systems do often rely on direct comparisons between a solution and a sample solution. This approach has drawbacks regarding flexibility in face of different possible solutions which are quite common in modeling tasks. This paper presents an alternative technique for checking UML class diagrams based on graph queries which promises to be more flexible.

Running test cases against a student's solution of a programming assignment is one of the easiest ways to generate feedback. If black-box tests are used, students may have difficulties to retrace the complete system behaviour and to find erroneous programming statements. This paper discusses the use of automated trace generation for assisting students in this task. Both manual and automated trace interpretation is discussed and evaluated by examples.