In recent years, the development of systems for processing and analyzing large amounts of data (so-called Big Data) has become an important sub-discipline of software engineering. However, to date there exits no comprehensive summary of the specific idiosyncrasies and challenges that the development of Big Data systems imposes on software engineers. With this paper, we aim to provide a first step towards filling this gap based on our collective experience from industry and academic projects as well as from consulting and initial literature reviews. The main contribution of our work is a concise summary of 26 challenges in engineering Big Data systems, collected and consolidated by means of a systematic identification process. The aim is to make practitioners more aware of common challenges and to offer researchers a solid baseline for identifying novel software engineering research directions.

This paper describes a large, unabridged data-set of Java source code gathered and shared as part of the Merobase Component Finder project of the Software-Engineering Group at the University of Mannheim. It consists of the complete index used to drive the search engine, www.merobase.com, the vast majority1of the source code modules accessible through it, and a tool that enables researchers to efficiently browse the collected data. We describe the techniques used to collect, format and store the data set, as well as the core capabilities of the Merobase search engine such as classic keyword-based, interface-based and test-driven search. This data-set, which represents one of the largest searchable collections of source and binary modules available online, has been recently made available for download and use in further research projects. All files are available at http://merobase.informatik.uni-mannheim.de/sources/.

The prime obstacle to automated defect testing has always been the generation of "correct" results against which to judge the behavior of the system under test - the "oracle problem". So called "back-to-back" testing techniques that exploit the availability of multiple versions of a system to solve the oracle problem have mainly been restricted to very special, safety critical domains such as military and space applications since it is so expensive to manually develop the additional versions. However, a new generation of software search engines that can find multiple copies of software components at virtually zero cost promise to change this situation. They make it economically feasible to use the knowledge locked in reusable software components to dramatically improve the efficiency of the software testing process. In this paper we outline the basic ingredients of such an approach.

Today's testing technologies and tools are all essentially based on the assumption that testers have full, unlimited access to systems under test along with the actual and expected results of test executions. This reflects the fact that, historically, software systems were usually developed by a single organization using components completely under its own control, and were tested in isolation under carefully controlled, "offline" conditions. However, with the rise of service-oriented architectures and the drive for more software reuse, these conditions are changing and the assumption that testers must be given full, trusted access to systems under test is no longer universally valid. In this paper we make the case for a new form of testing -- penurious testing -- in which testers do not have full, trusted access to the entities they test. By means of small examples we motivate its use and describe a language and framework that supports this new form of testing.

One of the fundamental pillars of the Web service vision is a brokerage system that enables services to be published to a searchable repository and later retrieved by potential users. This is the basic motivation for the UDDI standard, one of the three standards underpinning current Web service technology. However, this aspect of the technology has been the least successful, and the few Web sites that today attempt to provide a Web service brokerage facility do so using a simple cataloguing approach rather than UDDI. In this paper we analyze why the brokerage aspect of the Web service vision has proven so difficult to realize in practice and outline the technical difficulties involved in setting up and maintaining useful repositories of Web services. We then describe a pragmatic approach to web service brokerage based on automated indexing and discuss the required technological foundations. We also suggest some ideas for improving the existing standards to better support this approach and Web service searching in general.

Reliability testing is of even greater importance in component-based development than it is in traditional development methods, since it is the only practical way of measuring how well assembled components match each others' expectations. However, support for reliability testing has improved little over the last few years and it remains one of the most costly elements of software engineering. In this paper we present an approach that tackles the most fundamental obstacle to more efficient reliability testing - namely, the problem of automating the generation of "correct" reference results against which to evaluate component outputs. The approach employs the same basic technologies that can be used to discover components for reuse, but instead of building them into the final system it uses them, in combination with others, as an automated pseudo-oracle for measuring the reliability of self-built components. This significantly enhances the utility of component discovery technology, since it allows value to be derived from components which do not pass the threshold needed for full reuse but are sufficiently reliable to form a functioning component of a test oracle.