Machine learning (ML) is extensively used in production-ready applications, calling for mature engineering techniques to ensure robust development, deployment and maintenance. Given the potential negative impact machine learning (ML) can have on people, society or the environment, engineering techniques that can ensure robustness against technical errors and adversarial attacks are of considerable importance. In this work, we investigate how teams of experts develop, deploy and maintain software with ML components. Moreover, we link what teams do to the effects they aim to achieve and provide means for improvement. Towards this goal, we performed a mixed-methods study with a sequential exploratory strategy. First, we performed a systematic literature review through which we mined both academic and grey literature, and compiled a catalogue of engineering practices for ML. Second, we validated this catalogue using a large-scale survey, which measured the degree of adoption of the practices and their perceived effects. Third, we ran validation interviews with practitioners to add depth to the survey results. The catalogue covers a broad range of practices for engineering software systems with ML components and for ensuring non-functional properties that fall under the umbrella of trustworthy ML, such as fairness, security or accountability. Here, we present the results of our study, which indicate, for example, that larger and more experienced teams tend to adopt more practices, but that trustworthiness practices tend to be neglected. Moreover, we show that the effects measured in our survey, such as team agility or accountability, can be predicted quite accurately from groups of practices. This allowed us to contrast the importance of the practices for these effects as well as adoption rates, revealing, for example, that widely adopted practices are, in reality, less important with respect to some effects. For instance, writing reusable scripts for data cleaning and merging is highly adopted, but has a limited impact on reproducibility. Overall, our study provides a quantitative assessment of ML engineering practices and their impact on desirable properties of software with ML components, by which we open multiple avenues for improving the adoption of useful practices. Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Trustworthy and robust deployment of AI applications requires adherence to a range of AI engineering best practices. But, while professionals already have access to frameworks for deploying AI, case studies and developer surveys have found that many deployments do not follow best practices.We hypothesize that the adoption of AI deployment best practices can be improved by finding less complex framework designs that combine ease of use with built-in support for best practices. To investigate this hypothesis, we applied a design science approach to develop a new framework, called GreatAI, and evaluate its ease of use and best practice support.The initial design focusses on the domain of natural language processing (NLP), but with generalisation in mind. To assess applicability and generalisability, we conducted interviews with ten practitioners. We also assessed best practice coverage.We found that our framework helps implement 33 best practices through an accessible interface. These target the transition from prototype to production phase in the AI development lifecycle. Feedback from professional data scientists and software engineers showed that ease of use and functionality are equally important in deciding to adopt deployment technologies, and the proposed framework was rated positively in both dimensions.

Unique developmental and operational characteristics of machine learning (ML) components as well as their inherent uncertainty demand robust engineering principles are used to ensure their quality. We aim to determine how software systems can be (re-) architected to enable robust integration of ML components. Towards this goal, we conducted a mixed-methods empirical study consisting of (i) a systematic literature review to identify the challenges and their solutions in software architecture for ML, (ii) semi-structured interviews with practitioners to qualitatively complement the initial findings and (iii) a survey to quantitatively validate the challenges and their solutions. We compiled and validated twenty challenges and solutions for (re-) architecting systems with ML components. Our results indicate, for example, that traditional software architecture challenges (e.g., component coupling) also play an important role when using ML components; along with new ML specific challenges (e.g., the need for continuous retraining). Moreover, the results indicate that ML heightened decision drivers, such as privacy, play a marginal role compared to traditional decision drivers, such as scalability. Using the survey we were able to establish a link between architectural solutions and software quality attributes, which enabled us to provide twenty architectural tactics used to satisfy individual quality requirements of systems with ML components. Altogether, the results of the study can be interpreted as an empirical framework that supports the process of (re-) architecting software systems with ML components.

Abstract While many organizations embark on agile transformations, they can lack insight into the actual impact of these transformations across organizational layers. In this paper, we collect new and study existing evidence on the impact of agile transformations on organizational performance across teams, programs and portfolios. We conducted an international survey collecting the perceptions of agile coaches, transformation leads and other relevant roles, and we correlated levels of agile maturity to the perceptions on dimensions of organizational performance. Based on 134 responses from 29 countries across 16 industries, (1) we consolidated understanding of the benefits of agile transformations based on prior evidence and our data from a more diverse and larger sample, (2) we identified the dimensions impacted by agile transformations as being productivity, responsiveness, quality, workflow health and employee satisfaction & engagement and (3) we traced specific benefits on those dimensions to individual organizational layers of teams, programs and portfolios, showing the magnitude of impact of each dimension per layer. Overall, we can conclude that agile transformations have a variety of strong organizational benefits. This aggregated evidence allows reflection on transformation trends, but also enables organizations to optimize their agile transformation efforts.

Background. The increasing reliance on applications with machine learning (ML) components calls for mature engineering techniques that ensure these are built in a robust and future-proof manner.

Self driving cars have recently attracted academia and industry interest. As planning algorithms become responsible for critical decisions, many questions concerning traffic safety arise. An increased automation level demands proportional impact on safety requirements, currently governed by the ISO 26262 standard. However, ISO 26262 sees safety as a functional property of a system and fails to cover emergent concerns related to autonomous decisions. In order to fill this gap we propose the field of tactical safety, which extends safety analysis to planning and execution of driving maneuvers, response to traffic events or autonomous system failures. It is meant to complement, not to replace functional safety properties of a system and allows the analysis of autonomous agents from a safe behavior point of view. We draw the requirements for tactical safety from an automotive standard which defines functional elements for advanced driving automation systems.

Puppet is a declarative language for configuration management that has rapidly gained popularity in recent years. Numerous organizations now rely on Puppet code for deploying their software systems onto cloud infrastructures. In this paper we provide a definition of code quality for Puppet code and an automated technique for measuring and rating Puppet code quality. To this end, we first explore the notion of code quality as it applies to Puppet code by performing a survey among Puppet developers. Second, we develop a measurement model for the maintainability aspect of Puppet code quality. To arrive at this measurement model, we derive appropriate quality metrics from our survey results and from existing software quality models. We implemented the Puppet code quality model in a software analysis tool. We validate our definition of Puppet code quality and the measurement model by a structured interview with Puppet experts and by comparing the tool results with quality judgments of those experts. The validation shows that the measurement model and tool provide quality judgments of Puppet code that closely match the judgments of experts. Also, the experts deem the model appropriate and usable in practice. The Software Improvement Group (SIG) has started using the model in its consultancy practice.

The maintainability of software is an important cost factor for organizations across all industries, as maintenance makes up approximately 40% to 70% of the total development costs of a software system. Organizations are often stuck in the situation where software maintenance costs dominate IT budgets, leaving no room for enhancement and innovation. Building a benchmark for maintainability evolution is helpful in this context because it can help organizations decide on software improvement or replacement strategies. The prototype benchmark we study in this paper shows that software volume and maintainability levels are strong determinants of future maintainability evolution rates. We further describe the data collection and cleaning procedures that were applied to approximately 1,750 industrial software systems, and we provide an exploratory analysis of the resulting benchmark dataset.

Modern software systems often make use of third-party components to speed-up development and reduce maintenance costs. In return, developers need to update to new releases of these dependencies to avoid, for example, security and compatibility risks. In practice, prioritizing these updates is difficult because the use of outdated dependencies is often opaque. In this paper we aim to make this concept more transparent by introducing metrics to quantify the use of recent versions of dependencies, i.e. The system's "dependency freshness". We propose and investigate a system-level metric based on an industry benchmark. We validate the usefulness of the metric using interviews, analyze the variance of the metric through time, and investigate the relationship between outdated dependencies and security vulnerabilities. The results show that the measurements are considered useful, and that systems using outdated dependencies four times as likely to have security issues as opposed to systems that are up-to-date.

Known security vulnerabilities can be introduced in software systems as a result of being dependent upon third-party components. These documented software weaknesses are “hiding in plain sight” and represent low hanging fruit for attackers. In this paper we present the Vulnerability Alert Service (VAS), a tool-based process to track known vulnerabilities in software systems throughout their life cycle. We studied its usefulness in the context of external software product quality monitoring provided by the Software Improvement Group, a software advisory company based in Amsterdam, the Netherlands. Besides empirically assessing the usefulness of the VAS, we have also leveraged it to gain insight and report on the prevalence of third-party components with known security vulnerabilities in proprietary applications.

You can't control what you can't measure. And you can't decide if you are wandering around in the dark. Risk management in practice requires shedding light on the internals of the software product in order to make informed decisions. Thus, in practice, risk management has to be based on information about artifacts (documentation, code, and executables) in order to detect (potentially) critical issues. This tutorial presents experiences from industrial cases worldwide on qualitative and quantitative measurement of software products. We present our lessons learned as well as consolidated experiences from practice and provide a classification scheme of applicable measurement techniques. Participants of the tutorial receive an introduction to the techniques in theory and then apply them in practice in interactive exercises. This enables participants to learn how to shed light on the internals of their software and how to make risk management decisions efficiently and effectively.

\n Contains fulltext :\n 132652.pdf (Publisher’s version ) (Closed access)\n

For users of software libraries or public programming interfaces (APIs), backward compatibility is a desirable trait. Without compatibility, library users will face increased risk and cost when upgrading their dependencies. In this study, we investigate semantic versioning, a versioning scheme which provides strict rules on major versus minor and patch releases. We analyze seven years of library release history in Maven Central, and contrast version identifiers with actual incompatibilities. We find that around one third of all releases introduce at least one breaking change, and that this figure is the same for minor and major releases, indicating that version numbers do not provide developers with information in stability of interfaces. Additionally, we find that the adherence to semantic versioning principles has only marginally increased over time. We also investigate the use of deprecation tags and find out that methods get deleted without applying deprecated tags, and methods with deprecated tags are never deleted. We conclude the paper by arguing that the adherence to semantic versioning principles should increase because it provides users of an interface with a way to determine the amount of rework that is expected when upgrading to a new version.

Automated testing is a basic principle of agile development. Its benefits include early defect detection, defect causelocalization and removal of fear to apply changes to the code. Therefore, maintaining high quality test code is essential. This study introduces a model that assesses test code quality by combining source code metrics that reflect three main aspects of test codequality: completeness, effectiveness and maintainability. The model is inspired by the Software Quality Model of the SoftwareImprovement Group which aggregates source code metrics into quality ratings based on benchmarking. To validate the model we assess the relation between test code quality, as measured by the model, and issue handling performance. An experiment isconducted in which the test code quality model is applied to$18$open source systems. The test quality ratings are tested for correlation with issue handling indicators, which are obtained by mining issue repositories. In particular, we study the (1) defect resolution speed, (2) throughput and (3) productivity issue handling metrics. The results reveal a significant positive correlation between test code quality and two out of the three issue handling metrics (throughput and productivity), indicating that good test code quality positively influences issue handling performance.

Context: Research indicates that software quality, to a large extent, depends on cooperation within software teams [1] Since software development is a creative process that involves human interaction in the context of a team, it is important to understand the teamwork factors that influence performance. Objective: We present a study design in which we aim to examine the factors within software development teams that have significant influence on the performance of the team. We propose to consider factors such as communication, coordination of expertise, cohesion, trust, cooperation, and value diversity. The study investigates whether and to which extent these factors correlate with a performance of the team. In order to capture a variety of relevant teamwork factors, we created a new model extending the work of Hoegl and Gemuenden [2] and Liang et al. [3] Method: The study is based on quantitative research by means of an online questionnaire. We invited more than 20 software development teams in the Netherlands to participate in our team performance assessment, evaluating the teamwork and performance of the team. Based on an average team size of five people, one would therefore expect at least 100 participants in total. Also, product stakeholders will be asked to give their independent assessments of the performance of the team. Expected result: By analyzing the correlation between teamwork factors and team performance, we expect to gain a deeper understanding of how teamwork factors influence team performance. We also expect to validate the implemented extensions of teamwork model with respect to earlier work. Conclusion: Software teamwork factors are important to understand. In order to get a better understanding of the role of teamwork factors, this study should be conducted.

Using software metrics to keep track of the progress and quality of products and processes is a common practice in industry. Additionally, designing, validating and improving metrics is an important research area. Although using software metrics can help in reaching goals, the effects of using metrics incorrectly can be devastating. In this tutorial we leverage 10 years of metrics-based risk assessment experience to illustrate the benefits of software metrics, discuss different types of metrics and explain typical usage scenario's. Additionally, we explore various ways in which metrics can be interpreted using examples solicited from participants and practical assignments based on industry cases. During this process we will present the four common pitfalls of using software metrics. In particular, we explain why metrics should be placed in a context in order to maximize their benefits. A methodology based on benchmarking to provide such a context is discussed and illustrated by a model designed to quantify the technical quality of a software system. Examples of applying this model in industry are given and challenges involved in interpreting such a model are discussed. This tutorial provides an in-depth overview of the benefits and challenges involved in applying software metrics. At the end you will have all the information you need to use, develop and evaluate metrics constructively.

A wide range of software metrics targeting various abstraction levels and quality attributes have been proposed by the research community. For many of these metrics the evaluation consists of verifying the mathematical properties of the metric, investigating the behavior of the metric for a number of open-source systems or comparing the value of the metric against other metrics quantifying related quality attributes. Unfortunately, a structural analysis of the usefulness of metrics in a real-world evaluation setting is often missing. Such an evaluation is important to understand the situations in which a metric can be applied, to identify areas of possible improvements, to explore general problems detected by the metrics and to define generally applicable solution strategies. In this paper we execute such an analysis for two architecture level metrics, Component Balance and Dependency Profiles, by analyzing the challenges involved in applying these metrics in an industrial setting. In addition, we explore the usefulness of the metrics by conducting semi-structured interviews with experienced assessors. We document the lessons learned both for the application of these specific metrics, as well as for the method of evaluating metrics in practice.

We present the Maven Dependency Dataset (MDD), containing metrics, changes and dependencies of 148,253 jar files. Metrics and changes have been calculated at the level of individual methods, classes and packages of multiple library versions. A complete call graph is also presented which includes call, inheritance, containment and historical relationships between all units of the entire repository. In this paper, we describe our dataset and the methodology used to obtain it. We present different conceptual views of MDD and we also describe limitations and data quality issues that researchers using this data should be aware of.

Best practices in software development state that code that is likely to change should be encapsulated to localize possible modifications. In this paper, we investigate the application and effects of this design principle. We investigate the relationship between the stability, encapsulation and popularity of libraries on a dataset of 148,253 Java libraries. We find that bigger systems with more rework in existing methods have less stable interfaces and that bigger systems tend to encapsulate dependencies better. Additionally, there are a number of factors that are associated with change in library interfaces, such as rework in existing methods, system size, encapsulation of dependencies and the number of dependencies. We find that current encapsulation practices are not targeted at libraries that change the most. We also investigate the strength of ripple effects caused by instability of dependencies and we find that libraries cause ripple effects in systems using them and that these effects can be mitigated by encapsulation.

Backward compatibility is a major concern for any library developer. In this paper, we evaluate how stable a set of frequently used third-party libraries is in terms of method removals, implementation change, the ratio of change in old methods to change in new ones and the percentage of new methods in each snapshot. We provide a motivating example of a commercial company which demonstrates several issues associated with the usage of third-party libraries. To obtain dependencies from software systems we developed a framework which extracts dependencies from Maven build files and which analyzes system and library code. We propose four metrics which provide different insights in the implementation and interface stability of a library. The usage frequency of library methods is utilized as a weight in the final metric and is obtained from a dataset of more than 2300 snapshots of 140 industrial Java systems. We finally describe three scenarios and an example of the application of our metrics.

Software maintenance tasks are mainly related to fixing defects and implementing new features. Higher efficiency in performing such tasks is therefore going to reduce the costs of maintenance. A previous study involving open source systems has shown that higher software maintainability corresponds to faster speed in fixing defects [1]. In this paper we replicate the previous study by mining bug report data of three proprietary software projects. In one of the projects, a correlation between higher software maintainability and faster defect resolution is confirmed. The quality of issue handling process (e.g., issue registration accuracy and completeness, scope and complexity of issue workflow) should be considered in further research as it might explain the circumstances under which the correlation can be observed.

In this paper we introduce the concept of a “dependency profile”, a system level metric aimed at quantifying the level of encapsulation and independence within a system. We verify that these profiles are suitable to be used in an evaluation context by inspecting the dependency profiles for a repository of almost 100 systems. Furthermore we outline the steps we are taking to validate the usefulness and applicability of the proposed profiles.

Software metrics have been proposed as instruments, not only to guide individual developers in their coding tasks, but also to obtain high-level quality indicators for entire software systems. Such system-level indicators are intended to enable meaningful comparisons among systems or to serve as triggers for a deeper analysis.Common methods for aggregation range from simple mathematical operations (e.g. addition and central tendency) to more complex methodologies such as distribution fitting, wealth inequality metrics (e.g. Gini coefficient and Theil Index) and custom formulae.However, these methodologies provide little guidance for interpreting the aggregated results or to trace back to individual measurements.To resolve such limitations, a two-stage rating approach has been proposed where (i) measurement values are compared to thresholds to summarize them into risk profiles, and (ii) risk profiles are mapped to ratings.In this paper, we extend our approach for deriving metric thresholds from benchmark data into a methodology for benchmark-based calibration of two-stage aggregation of metrics into ratings.We explain the core algorithm of the methodology and we demonstrate its application to various metrics of the SIG quality model, using a benchmark of 100 software systems.We present an evaluation of the sensitivity of the algorithm to the underlying data.

The decomposition of a software system into components is a major decision in any software architecture, having a strong influence on many of its quality aspects. A system's analyzability, in particular, is influenced by its decomposition into components. But into how many components should a system be decomposed to achieve optimal analyzability? And how should the elements of the system be distributed over those components? In this paper, we set out to find answers to these questions with the support of a large repository of industrial and open source software systems. Based on our findings, we designed a metric which we call Component Balance. In a case study we show that the metric provides pertinent results in various evaluation scenarios. In addition, we report on an empirical study that demonstrates that the metric is strongly correlated with ratings for analyzability as given by experts.

A wide variety of software metrics have been proposed and a broad range of tools is available to measure them. However, the effective use of software metrics is hindered by the lack of meaningful thresholds. Thresholds have been proposed for a few metrics only, mostly based on expert opinion and a small number of observations. Previously proposed methodologies for systematically deriving metric thresholds have made unjustified assumptions about the statistical properties of source code metrics. As a result, the general applicability of the derived thresholds is jeopardized. We designed a method that determines metric thresholds empirically from measurement data. The measurement data for different software systems are pooled and aggregated after which thresholds are selected that (i) bring out the metric's variability between systems and (ii) help focus on a reasonable percentage of the source code volume. Our method respects the distributions and scales of source code metrics, and it is resilient against outliers in metric values or system size. We applied our method to a benchmark of 100 object-oriented software systems, both proprietary and open-source, to derive thresholds for metrics included in the SIG maintainability model.

This paper introduces a Software Architecture Complexity Model (SACM) based on theories from cognitive science and system attributes that have proven to be indicators of maintainability in practice. SACM can serve as a formal model to reason about why certain attributes influence the complexity of an implemented architecture. Also, SACM can be used as a starting point in existing architecture evaluation methods such as the ATAM. Alternatively, SACM can be used in a stand-alone fashion to reason about a software architecture's complexity.

We mined the issue database of GNOME to assess how issues are handled. How many issues are submitted and resolved? Does the backlog grow or decrease? How fast are issues resolved? Does issue resolution speed increase or decrease over time? In which subproject are issues handled most efficiently? To answer such questions, we apply several visualization and quantification instruments to the raw issue data. In particular, we aggregate issues into four risk categories, based on their resolution time. These categories are the basis both for visualizing and ranking, which are used in concert for issue database exploration.

Software architecture evaluation methods aim at identifying potential maintainability problems for a given architecture. Several of these methods exist, which typically prescribe the structure of the evaluation process. Often left implicit, however, are the concrete system attributes that need to be studied in order to assess the maintainability of implemented architectures. To determine this set of attributes, we have performed an empirical study on over 40 commercial architectural evaluations conducted during the past two years as part of a systematic ldquoSoftware Risk Assessmentrdquo. We present this study and we explain how the identified attributes can be projected on various architectural system properties, which provides an overview of criteria for the evaluation of the maintainability of implemented software architectures.

The ISO/IEC 9126 international standard for software product quality is a widely accepted reference for terminology regarding the multi-faceted concept of software product quality. Based on this standard, the Software Improvement Group has developed a pragmatic approach for measuring technical quality of software products. This quality model introduces another level below the hierarchy defined by ISO/IEC 9126, which consists of system properties such as volume, duplication, unit complexity and others. A mapping between system properties and ISO/IEC 9126 characteristics is defined in a binary fashion: a property either influences a characteristic or not. This mapping embodies consensus among three experts based, in an informal way, on their experience in software quality assessment. We have conducted a survey-based experiment to study the mapping between system properties and quality characteristics. We used the Analytic Hierarchy Process as a formally structured method to elicit the relative importance of system properties and quality characteristics from a group of 22 software quality experts. We analyzed the results of the experiment with two objectives: (i) to validate the original binary mapping and (ii) to refine the mapping using the elicited relative weights.

This paper presents techniques and tools to transform spreadsheets into relational databases and back. A set of data refinement rules is introduced to map a tabular datatype into a relational database schema. Having expressed the transformation of the two data models as data refinements, we obtain for free the functions that migrate the data. We use well-known relational database techniques to optimize and query the data. Because data refinements define bi-directional transformations we can map such database back to an optimized spreadsheet. We have implemented the data refinement rules and we constructed Haskell-based tools to manipulate, optimize and refactor Excel-like spreadsheets.

Test coverage is an important indicator for unit test quality. Tools such as Clover compute coverage by first instrumenting the code with logging functionality, and then logging which parts are executed during unit test runs. Since computation of test coverage is a dynamic analysis, it presupposes a working installation of the software. In the context of software quality assessment by an independent third party, a working installation is often not available. The evaluator may not have access to the required libraries or hardware platform. The installation procedure may not be automated or documented. In this paper, we propose a technique for estimating test coverage at method level through static analysis only. The technique uses slicing of static call graphs to estimate the dynamic test coverage. We explain the technique and its implementation. We validate the results of the static estimation by statistical comparison to values obtained through dynamic analysis using Clover. We found high correlation between static coverage estimation and real coverage at system level but closer analysis on package and class level reveals opportunities for further improvement.

Spreadsheets can be viewed as a highly flexible end-users programming environment which enjoys wide-spread adoption. But spreadsheets lack many of the structured programming concepts of regular programming paradigms. In particular, the lack of data structures in spreadsheets may lead spreadsheet users to cause redundancy, loss, or corruption of data during edit actions. In this paper, we demonstrate how implicit structural properties of spreadsheet data can be exploited to offer edit assistance to spreadsheet users. Our approach is based on the discovery of functional dependencies among data items which allow automatic reconstruction of a relational database schema. From this schema, new formulas and visual objects are embedded into the spreadsheet to offer features for auto-completion, guarded deletion, and controlled insertion. Schema discovery and spreadsheet enhancement are carried out automatically in the background and do not disturb normal user experience.

In the working session on Industrial Realities of Program Comprehension (IRPC 2008), experience and ideas are shared regarding the challenges and opportunities of industrial application of program comprehension techniques. Participants work together to formulate a set of useful guidelines for introducing new program comprehension techniques into industrial practice as well as consolidating and increasing their use.

Various programming languages allow the construction of structure-shy programs. Such programs are defined generically for many different datatypes and only specify specific behavior for a few relevant subtypes. Typical examples are XML query languages that allow selection of subdocuments without exhaustively specifying intermediate element tags. Other examples are languages and libraries for polytypic or strategic functional programming and for adaptive object-oriented programming.

Haskell's type system with multi-parameter constructor classes and functional dependencies allows static (compile-time) computations to be expressed by logic programming on the level of types. This emergent capability has been exploited for instance to model arbitrary-length tuples (heterogeneous lists), extensible records, functions with variable length argument lists, and (homogenous) lists of statically fixed length (vectors).We explain how type-level programming can be exploited to define a strongly-typed model of relational databases and operations on them. In particular, we present a strongly typed embedding of a significant subset of SQL in Haskell. In this model, meta-data is represented by type-level entities that guard the semantic correctness of database operations at compile time.Apart from the standard relational database operations, such as selection and join, we model functional dependencies (among table attributes), normal forms, and operations for database transformation. We show how functional dependency information can be represented at the type level, and can be transported through operations. This means that type inference statically computes functional dependencies on the result from those on the arguments.Our model shows that Haskell can be used to design and prototype typed languages for designing, programming, and transforming relational databases.

Abstract Program understanding tools manipulate program representations, such as abstract syntax trees, control‐flow graphs, or data‐flow graphs. This paper deals with the use of visitor combinators to conduct such manipulations. Visitor combinators are an extension of the well‐known visitor design pattern. They are small, reusable classes that carry out specific visiting steps. They can be composed in different constellations to build more complex visitors. We evaluate the expressiveness, reusability, ease of development, and applicability of visitor combinators to the construction of program understanding tools. To that end, we conduct a case study in the use of visitor combinators for control‐flow analysis and visualization as used in a commercial Cobol program understanding tool. Copyright © 2004 John Wiley & Sons, Ltd.

We want to use the advanced language processing technology available in the ASF+SDF in combination with general purpose programming languages. In particular, we want to combine the syntax definition formalism (SDF) and the associated components that support generalized LR parsing, with the object-oriented language Java. To this end, we implemented JJForester, a tool that generates class structures from SDF grammar definitions. The generated class structures implement a number of design patterns to facilitate construction and traversal of parse trees represented by object structures. In a detailed case study, we demonstrate how program analyses and transformations for languages of non-trivial size can be constructed with JJForester.

The Visitor design pattern allows the encapsulation of polymorphic behavior outside the class hierarchy on which it operates. A common application of Visitor is the encapsulation of tree traversals. A clean separation can be made between the generic parts of the combinator set and the parts that are specific to a particular class hierarchy. The generic parts form a reusable framework. The generic parts form a reusable framework. The specific parts can be generated from a (tree) grammar. Due to this separation, programming with visitor combinators becomes a form of generic programming with significant reuse of (visitor) code.