In the era of Big Code, when researchers seek to study an increasingly large number of repositories to support their findings, the data processing stage may require manipulating millions and more of records. In this work we focus on studies involving fine-grained AST level source code changes. We present how we extended the CodeDistillery source code mining framework with data manipulation capabilities, aimed to alleviate the processing of large datasets of fine grained source code changes. The capabilities we have introduced allow researchers to highly automate their repository mining process and streamline the data acquisition and processing phases. These capabilities have been successfully used to conduct a number of studies, in the course of which dozens of millions of fine-grained source code changes have been processed.

Lehman's Laws teach us that a software system will become progressively less satisfying to its users over time, unless it is continually adapted to meet new needs. A line of previous works sought to better understand software maintenance by studying how commits can be classified into three main software maintenance activities. Corrective: fault fixing; Perfective: system improvements; Adaptive: new feature introduction. In this work we suggest visualizations for exploring software maintenance activities in both project and individual developer scopes. We demonstrate our approach using a prototype we have built using the Shiny R framework. In addition, we have also published our prototype as an online demo. This demo allows users to explore the maintenance activities of a number of popular open source projects. We believe that the visualizations we provide can assist practitioners in monitoring and maintaining the health of software projects. In particular, they can be useful for identifying general imbalances, peaks, deeps and other anomalies in projects' and developers' maintenance activities.

Automatic testing is a widely adopted technique for improving software quality. Software developers add, remove and update test methods and test classes as part of the software development process as well as during the evolution phase, following the initial release. In this work we conduct a large scale study of 61 popular open source projects and report the relationships we have established between test maintenance, production code maintenance, and semantic changes (e.g, statement added, method removed, etc.). performed in developers' commits.We build predictive models, and show that the number of tests in a software project can be well predicted by employing code maintenance profiles (i.e., how many commits were performed in each of the maintenance activities: corrective, perfective, adaptive). Our findings also reveal that more often than not, developers perform code fixes without performing complementary test maintenance in the same commit (e.g., update an existing test or add a new one). When developers do perform test maintenance, it is likely to be affected by the semantic changes they perform as part of their commit.Our work is based on studying 61 popular open source projects, comprised of over 240,000 commits consisting of over 16,000,000 semantic change type instances, performed by over 4,000 software engineers.

Predictive models for software projects' characteristics have been traditionally based on project-level metrics, employing only little developer-level information, or none at all. In this work we suggest novel metrics that capture temporal and semantic developer-level information collected on a per developer basis. To address the scalability challenges involved in computing these metrics for each and every developer for a large number of source code repositories, we have built a designated repository mining platform. This platform was used to create a metrics dataset based on processing nearly 1000 highly popular open source GitHub repositories, consisting of 147 million LOC, and maintained by 30,000 developers. The computed metrics were then employed to predict the corrective, perfective, and adaptive maintenance activity profiles identified in previous works. Our results show both strong correlation and promising predictive power with R^2 values of 0.83, 0.64, and 0.75. We also show how these results may help project managers to detect anomalies in the development process and to build better development teams. In addition, the platform we built has the potential to yield further predictive models leveraging developer-level metrics at scale.

Socio-professional websites such as LinkedIn use various mechanisms such as network of colleagues, groups and discussions to assist their users in maintaining their professional network and keeping up with relevant discussions. Software professionals post hundreds of thousands of comments each week in these group discussions regarding technological and methodological aspects of their work. Analyzing these comments enables us, the software community at large, to better understand the state of the practice of many aspects of software development. In this paper we describe a case study in which we use such LinkedIn group discussion to learn about developers' perception of using code examples.

The evaluation of an advanced software engineering (SE) course is a challenging task that should be addressed as part of the course design. In this paper, we describe an implementation of the reflective practitioner perspective as part of an advanced SE course and introduce its use for feedback purposes. Reflective feedback goes beyond merely using feedback questionnaires. Furthermore, we show, using both qualitative and quantitative empirical data, that the course on which the perspective was applied meets its objectives and fosters students' awareness to the multidimensional nature of SE.

XML is the standard format of content representation and sharing on the Web. XML is a highly verbose language, especially regarding the duplication of meta-data in the form of elements and attributes. As XML content is becoming more widespread so is the demand to compress XML data volume. The paper presents the best XML compression ratios reported to date. Its advantage over other XML compression techniques is that it uses syntactic information to enhance compression. Therefore, it is a fully syntactic based XML compression. The syntactic information is parsed from XML documents by an innovative XML parser. We developed a new XML parser-generator for that purpose. Our parser-generator is based on a syntactic dictionary (DTD, XML-Schema, etc.) of the XML in order to create an efficient and compact XML parsers. This XML parser-generator is adopted to streaming technologies and can be used in a wide variety of XML applications such as validators, converters, gateways, routers, browsers editors etc. The parsers' symbols are encoded by a partial prediction matching (PPM) codec. We compare between the performance of our algorithm and other existing XML compression techniques. The proposed compression algorithm achieves better compression ratio in comparison to other XML compression techniques that do not utilize syntactic structure. The superiority of our compression technique is more evident when it is tested on XML data sets that contain only tags and not free text

Web services provide a standard means of interoperating between different software applications, running on a variety of platforms and/or frameworks. While the concepts of Web services are aimed at providing a standard means to support interoperable machine-to-machine interaction over a network, they do not solve the problem of trust between service requesters and providers. A trusted component is defined as a reusable software element possessing specified and guaranteed property qualities. The highly reusable nature of a Web service emphasizes the need for a "trust ensuring" mechanism between the requester and the provider of the service. The focus of this paper is to suggest a fortified Web services architecture introducing the concept of contracts to increase the level of trust between the requester and the provider of the requested service. In order to achieve this goal a new language is introduced into the fortified Web services architecture: WS-Contract. WS-Contract is a machine-processable specification of the Web service semantics, formally supporting the different levels of contract information. It defines the pre-and post conditions of the Web services interface, the synchronization policy of the service and the QoS parameters that should be maintained between the requester agent and the provider agent. WS-Contract relies, wherever possible, on existing Web services standards to build the necessary "trust ensuring" mechanism for Web services

UML provides several kinds of diagrams to model the behavior and structure of a system under development. A consistency problem may arise due to the fact that some aspects of the model may be described by more than one diagram. Hence, it is important that the consistency of the system description should be checked before implementing it. This paper describes an algorithmic approach to a consistency check between UML sequence and state diagrams. The algorithm we provide also handles complex state diagrams, e.g. diagrams that include forks, joins, and concurrent composite states. We describe the algorithm in detail, and elaborate on the design decisions made while developing the algorithm. We have implemented BVUML, a tool that assists in automating the validation process.

Despite vast interest in design patterns, the specification and application of patterns is generally assumed to rely on manual implementation. We describe a precise method of specifying how a design pattern is applied: by phrasing it as an algorithm in a meta-programming language. We present a prototype of a tool that supports the specification of design patterns and their realization in a given program. Our prototype allows automatic application of design patterns without obstructing the source code test from the programmer, who may edit it at will. We demonstrate pattern specification in meta-programming techniques and a sample outcome of its application.

Abstract We propose a methodology of reducing the space requirements of recursive procedures without destroying their fundamental recursive structure. This technique is quite general, and seems to apply to most recursive procedures, particularly those dealing with recursive data structures. A recursive procedure is transformed into three procedures — a shell which is called from the outside, and has the same interface as the original, a recursive procedure with a minimum number of parameters, and no local variables, and a procedure that performs the operations but has no recursive calls. Our methodology was successfully employed in an implementation of a decision support system. The paper also includes results of experiments, in which space was reduced by a factor of up to seven, and run time was also improved.

The authors explore the use of software transformations for software evolution. Meaning-preserving program transformations have been widely used for program development from a fixed initial specification. They consider a wider class of transformations to support development in which the specification evolves, rather than being fixed in advance. A new and general classification of transformations based on their effect on system interfaces, externally observable behavior, and abstraction level of a system description is presented. This classification is used to rearrange chronological derivation sequences containing meaning-changing transformations into lattices containing only meaning-preserving transformations. A process model for software evolution that utilizes prototyping techniques is described. Ways in which this class of transformations can be used to support such a process are considered. A set of examples are presented to illustrate the ideas. Software tool support and directions for future research are discussed.>

The OBSERV methodology for software development is based on rapid construction of an executable specification, or prototype, of a systems, which may be examined and modified repeatedly to achieve the desired functionality. The objectives of OBSERV also include facilitating a smooth transition to a target system, and providing means for reusing specification, design, and code of systems and subsystems. We are particularly interested in handling embedded systems, which are likely to have concurrency and have some real-time requirements. The OBSERV prototyping language combines several paradigms to express the behavior of a system. The object-oriented approach provides the basic mechanism for building a system from a collection of objects, with well-defined interfaces between them. We use finite-state machines to model the behavior of individual objects. At a lower level, activities that occur within objects, either upon entry to a state or in transition between thus allowing a nonprocedural description. The environment provided to a prototype builder is as important as the language. We have made an attempt to provide flexible tools for executing or simulating the prototype being built, as well as for browsing and static checking. The first implementation of the tools was window based but not graphic. A graphic front end, name CRUISE, was developed afterwards. A simulation sequence focuses on a single object, which can be as complex as necessary, possibly the entire system, and expects all the interactions between it and the outside world to be achieved by communication between the simulator and the user. The simulator allows the user to easily switch back and forth from one object to another, simulating each object in isolation. To enable testing the behavior of a prototype in a realistic environment, it is possible to construct objects that imitate the environment objects. We also allow simulation of systems with missing pieces, by calling upon the user to simulate any such missing piece by himself.

A system for user-interface specification is presented. The authors use OBSERV, an innovative methodology for prototyping reactive systems which combines three paradigms: the object-oriented approach, state transition diagrams, and logic programming. This combination is very useful for specifying user interfaces, which can be viewed as reactive systems. The usability of OBSERV is enhanced by a user-interface, CRUISE, that provides graphic editing capabilities.>

An iteration theorem for simple precedence languages ts presented.The theorem is then used to prove very easily that certain languages are not simple precedence.In addttion, a strong characterizauon of simple precedence languages is established.This is obtained by analyzing a nontrivlal famdy of languages that includes typical languages which are not stmple precedence languages and some similar languages which are m fact simple precedence.The iterauon theorem is then generahzed to deal with certain families of extended precedence languages Using this theorem, a rs shown that none of the languages proved to be nonstmple precedence is umquely invertible (1, k) precedence for any k _> 1.

We present a linear time algorithm for eliminating null rules in context free grammars. Until recently all algorithms given in the literature for this problem required exponential time.