Few-Shot Learning has emerged as a topic that maximises DNN performance based on very few samples. In Generalised Few-Shot Learning, a model has to learn new few-shot classes while recalling earlier large-scale training classes. Learning the new classes leads to a drop in performance on the base ones. In this work, we identify and explore the parallels between Generalised Few-Shot Object Detection (G-FSOD) and Continual Learning (CL), focusing on two areas in particular: gradient manipulation methods and sampling strategies. Through extensive experimentation we demonstrate that gradient manipulation methods appear to be no better than existing techniques and do not improve performance, but actually harm performance unless the gradients are averaged. Our investigations into sampling strategies consider a number of aspects: the impact of removing the base limit and the effectiveness of different distance measures (with respect to a class prototype) for sample selection. Our experiments into these aspects reveal illuminating insights into their impact on Average Precision on the COCO and VOC datasets. Consequently, we suggest that G-FSOD research focus on the replay aspect and investigate other sampling strategies.

Delirium, affecting up to 52% of cardiac surgery patients, can have serious long-term effects on patients by damaging cognitive ability and causing subsequent functional decline. This study reports on the development and evaluation of predictive models aimed at identifying the likely onset of delirium on an hourly basis in intensive care unit following cardiac surgery. Most models achieved a mean AUC > 0.900 across all lead times. A support vector machine achieved the highest performance across all lead times of AUC = 0.941 and Sensitivity = 0.907, and BARTm, where missing values were replaced with missForest imputation, achieved the highest Specificity of 0.892. Being able to predict delirium hours in advance gives clinicians the ability to intervene and optimize treatments for patients who are at risk and avert potentially serious and life-threatening consequences.

While prior object-oriented software maintainability literature acknowledges the role of machine learning techniques as valuable predictors of potential change, the most suitable technique that achieves consistently high accuracy remains undetermined. With the objective of obtaining more consistent results, an ensemble technique is investigated to advance the performance of the individual models and increase their accuracy in predicting software maintainability of the object-oriented system. This paper describes the research plan for predicting object-oriented software maintainability using ensemble techniques. First, we present a brief overview of the main research background and its different components. Second, we explain the research methodology. Third, we provide expected results. Finally, we conclude summary of the current status.

Design of text entry on small screen devices, e.g. smartwatches, faces two related challenges: trading off a reasonably sized keyboard area against space to display the entered text and the concern over "fat fingers". This paper investigates tap accuracy and revisits layered interfaces to explore a novel layered text entry method. A two part user study identifies preferred typing and reading tilt angles and then investigates variants of a tilting layered keyboard against a standard layout. We show good typing speed (29 wpm) and very high accuracy on the standard layout - contradicting fears of fat-fingers limiting watch text-entry. User feedback is positive towards tilting interaction and we identify ∼14° tilt as a comfortable typing angle. However, layering resulted in slightly slower and more erroneous entry. The paper contributes new data on tilt angles and key offsets for smartwatch text entry and supporting evidence for the suitability of QWERTY on smartwatches.

Developments in the automation of test data generation have greatly improved efficiency of the software testing process, but the so-called oracle problem (deciding the pass or fail outcome of a test execution) is still primarily an expensive and error-prone manual activity. We present an approach to automatically detect passing and failing executions using cluster-based anomaly detection on dynamic execution data based on firstly, just a system’s input/output pairs and secondly, amalgamations of input/output pairs and execution traces. The key hypothesis is that failures will group into small clusters, whereas passing executions will group into larger ones. Evaluation on three systems with a range of faults demonstrates this hypothesis to be valid—in many cases small clusters were composed of at least 60 % failures (and often more). Concentrating the failures in these small clusters substantially reduces the numbers of outputs that a developer would need to manually examine following a test run and illustrates that the approach has the potential to improve the effectiveness and efficiency of the testing process.

A key component of software testing is deciding whether a test case has passed or failed: an expensive and error-prone manual activity. We present an approach to automatically classify passing and failing executions using semi-supervised learning on dynamic execution data (test inputs/outputs and execution traces). A small proportion of the test data is labelled as passing or failing and used in conjunction with the unlabelled data to build a classifier which labels the remaining outputs (classify them as passing or failing tests). A range of learning algorithms are investigated using several faulty versions of three systems along with varying types of data (inputs/outputs alone, or in combination with execution traces) and different labelling strategies (both failing and passing tests, and passing tests alone). The results show that in many cases labelling just a small proportion of the test cases - as low as 10% - is sufficient to build a classifier that is able to correctly categorise the large majority of the remaining test cases. This has important practical potential: when checking the test results from a system a developer need only examine a small proportion of these and use this information to train a learning algorithm to automatically classify the remainder.

Context: Bug reports are the primary means by which users of a system are able to communicate a problem to the developers, and their contents are important - not only to support developers in maintaining the system, but also as the basis of automated tools to assist in the challenging tasks of finding and fixing bugs.

SUMMARY Identifying bug origins – the point where erroneous code was introduced – is crucial for many software engineering activities, from identifying process weaknesses to gathering data to support bug detection tools. Unfortunately, this information is not usually recorded when fixing bugs, and recovering it later is challenging. Recently, the text approach and the dependence approach have been developed to tackle this problem. Respectively, they examine textual and dependence‐related changes that occurred prior to a bug fix. However, only limited evaluation has been carried out, partially because of a lack of available implementations and of datasets linking bugs to origins. To address this, origins of 174 bugs in three projects were manually identified and compared to a simulation of the approaches. Both approaches were partially successful across a variety of bugs – achieving 29–79% precision and 40–70% recall. Results suggested the precise definition of program dependence could affect performance, as could whether the approaches identified a single or multiple origins. Some potential improvements are explored in detail and identify pragmatic strategies for combining techniques along with simple modifications. Even after adopting these improvements, there remain many challenges: large commits, unrelated changes and long periods between origins and fixes all reduce effectiveness. Copyright © 2013 John Wiley & Sons, Ltd.

Testing is an essential component of the software development process, but also one which is exceptionally difficult to manage and control. For example, it is well understood that testing techniques are not guaranteed to detect all faults, but more frustrating is that after the application of a testing technique the tester has little or no knowledge of how many faults might still be left undiscovered. This paper investigates the performance of a range of capture-recapture models to determine the accuracy with which they predict the number of defects remaining after testing. The models are evaluated with data from two empirical testing-related studies and from one larger publicly available project and the factors affecting the accuracy of the models are analysed. The paper also considers how additional information (such as structural coverage data) may be used to improve the accuracy of the estimates. The results demonstrate that diverse sets of faults resulting from different testers using different techniques tend to produce the most accurate results, and also illustrate the sensitivity of the estimators to the patterns of fault data.

Abstract Software development projects are inherently cooperative, requiring many developers to coordinate their efforts to produce complex systems. An integral part of this process is the development and maintenance of a shared understanding concerning the state of software project tasks, artefacts, and the activities and expertise of developers working on the project. In co‐located settings, this vitalawareness informationthat concerns developers, either directly or tangentially, is typically acquired using formal, regular meetings, informal personal interruptions and electronic communication. Developing and maintaining such an awareness are far more difficult in distributed teams where developers are separated by time and space. The goal of this review is to help bridge the gap between the theoretical benefits of distributed collaborative software development and the practical impediments to successful implementation, and more specifically, achieving similar levels of awareness in distributed settings to those available in co‐located settings. The review starts with a detailed discussion of awareness and awareness types with particular relevance to collaboration. This is followed by a discussion of how awareness information is disseminated during collaborative processes. Awareness information needs and dissemination mechanisms in software development teams are then presented. The main contribution of the review, derived from a comparison of awareness needs in distributed collaborative software engineering and current support from tools and techniques, is that more support is required for the type of awareness known ascontext awareness. Context awareness crosscuts other (more basic) forms of awareness and arises because, over time, individual collaborators work on different sets of tasks and artefacts and form different perceptions of their colleagues; their colleagues' work, roles, and relationships; and, the workspace. This awareness is highlycontextualand therefore cannot be generalized across individuals, entities or activities that are found in a collaboration space. Copyright © 2010 John Wiley & Sons, Ltd.

Recent research has found that many novice programmers often hold non-viable mental models of basic programming concepts which can limit their potential to develop appropriate programming skills. Previous work by the authors suggests that a teaching model that integrates cognitive conflict and program visualisation can help novices formulate appropriate mental models. This paper first outlines a 'concepts roadmap' that provides an ordered approach to learning programming concepts allowing students to build on fundamental base knowledge. It then reports the results of a series of studies investigating the use of the Jeliot visualisation tool as the visualisation component of the proposed learning model when applied to these concepts. The findings include: the ease with which Jeliot can be tailored to visualise a range of concepts using a variety of examples; the Jeliot visualisation of object reference was too complex for CS1 students; further evidence that CS1 students struggle to develop appropriate understanding of a range of key programming concepts; and, further evidence that an integrated cognitive conflict/visualisation strategy can help students develop an appropriate understanding of key programming concepts.

Requirements traceability is a challenge for modern software projects where task dependencies and technical expertise are spread across system developers, abstract model representations such as use cases, and a myriad of code artefacts. This paper presents an approach that monitors the navigation trails left by developers when building code artefacts to realise project use cases. These trails are analysed to generate a relevance ranking of entities that constitute a traceability link between uses cases and code artefacts and the developers responsible for them. Investigation in a software development scenario shows that a range of use case traceability questions can be answered through visualisations which present ordered relevance lists of the entities associated with use cases and by the use of trace graphs where the size of nodes show the importance, or 'information centrality', of system entities.

Software development is a global activity unconstrained by the bounds of time and space. A major effect of this increasing scale and distribution is that the shared understanding that developers previously acquired by formal and informal face-to-face meetings is difficult to obtain. This paper proposes a shared awareness model that uses information gathered automatically from developer IDE interactions to make explicit orderings of tasks, artefacts and developers that are relevant to particular work contexts in collaborative, and potentially distributed, software development projects. The research findings suggest that such a model can be used to: identify entities (developers, tasks, artefacts) most associated with a particular work context in a software development project; identify relevance relationships amongst tasks, developers and artefacts e.g. which developers and artefacts are currently most relevant to a task or which developers have contributed to a task over time; and, can be used to identify potential bottlenecks in a project through a ‘social graph’ view. Furthermore, this awareness information is captured and provided as developers work in different locations and at different times.

Recent research has found that many novice programmers often hold non-viable mental models of basic programming concepts such as assignment and object reference, which can limit their potential to develop programming skills. This paper proposes a constructivist-based teaching model that integrates cognitive conflict and program visualisation with the aim of supporting novice programmers in the formulation of appropriate mental models. The results of an initial empirical study produced three findings of note. Firstly, a teaching model based on either visualisation alone or cognitive conflict integrated with visualisation can help students develop viable models of value assignment. Secondly, there was evidence to suggest that cognitive conflict integrated with visualisation outperformed visualisation alone in helping students develop viable models of the more challenging concept of object reference assignment. And thirdly, there was evidence of an improvement in students' understanding of value and object reference assignment using the teaching model based on visualisation and cognitive conflict.

It is not unusual for a software development organization to expend 40% of total project effort on testing, which call be a very laborious and time-consuming process. Therefore, there is a big necessity for test automation. This paper describes an approach to automatically generate test-data for 00 software exploiting a Genetic Algorithm (GA) to achieve high levels of data-flow (d-u) coverage. A proof-of-concept tool is presented. The experimental results from testing six Java classes helped us identify three categories of problematic test targets, and suggest that in the future full d-u coverage with a reasonable computational cost may be possible if we overcome these obstacles.

Today's large software projects are often characterised by distributed environments with numerous developers separated in space and/or time. This separation means that the common understanding and tacit knowledge that is a feature of closely co-located project teams is very hard to come by. As a consequence, relatively simple tasks such as identifying functionally related modules or finding individuals who are experts in aspects of the system become more challenging and time-consuming. This paper presents a continuum of relevance index (CRI) model that uses information gathered from developer IDE interactions to generate orderings of relevant tasks, project artefacts and developers. A case study is used to demonstrate how the model can be used to attain a shared knowledge and common understanding of the extent to which tasks, artefacts and developers are relevant in a group development work context.

Identifying and isolating the source code associated with a particular feature is a problem that frequently arises in many maintenance tasks. The delocalised nature of object-oriented systems, where the code associated with a feature is distributed across many interrelated objects, makes this problem particularly challenging. This paper presents an approach that combines 'landmark' methods that have a key role in the execution of a particular feature with slicing to create a call graph of related code. The size of this call graph is constrained by the identification of 'barrier' methods which exclude parts of the graph that are not of interest. The approach is supported by a tool, and the evaluation on three open-source systems yields encouraging results and demonstrates the practical applicability of the technique.

This paper describes an investigation into the viability of mental models used by novice programmers at the end of a first year Java programming course. The qualitative findings identify the range of mental models of value and reference assignment held by the participants. The quantitative analysis reveals that approximately one third of students held non-viable mental models of value assignment and only 17% of students held a viable mental model of reference assignment. Further, in terms of a comparison between the participants' mental models and their performance in in-course assessments and final examination, it was found that students with viable mental models performed significantly better than those with non-viable models. These findings are used to propose a more "constructivist" approach to teaching programming based on the integration of "cognitive conflict" and program visualisation.

A testability transformation is a source-to-source transformation that aims to improve the ability of a given test generation method to generate test data for the original program. We introduce testability transformation, demonstrating that it differs from traditional transformation, both theoretically and practically, while still allowing many traditional transformation rules to be applied. We illustrate the theory of testability transformation with an example application to evolutionary testing. An algorithm for flag removal is defined and results are presented from an empirical study which show how the algorithm improves both the performance of evolutionary test data generation and the adequacy level of the test data so-generated.

The First International Workshop on Verification and Validation of Enterprise Information Systems (VVEIS 2003

We describe the development and evaluation of a rigorous approach aimed at the effective and efficient inspection of object-oriented (OO) code. Since the time that inspections were developed they have been shown to be powerful defect detection strategies. However, little research has been done to investigate their application to OO systems, which have very different structural and execution models compared to procedural systems. This suggests that inspection techniques may not be currently being deployed to their best effect in the context of large-scale OO systems. Work to date has revealed three significant issues that need to be addressed - the identification of chunks of code to be inspected, the order in which the code is read, and the resolution of frequent nonlocal references. Three techniques are developed with the aim of addressing these issues: one based on a checklist, one focused on constructing abstract specifications, and the last centered on the route that a use case takes through a system. The three approaches are evaluated empirically and, in this instance, it is suggested that the checklist is the most effective approach, but that the other techniques also have potential strengths. For the best results in a practical situation, a combination of techniques is recommended, one of which should focus specifically on the characteristics of OO.

Frameworks are an attractive form of reuse due to their paradigmatic simplicity. Unfortunately their size and complexity makes understanding how to use them difficult. In addition documentation to support framework reuse often lacks experimental validation and there is little understanding of what artefacts must be documented to increase the effectiveness of documentation techniques. This report describes an empirical investigation into framework reuse. Its aim is to identify the major problems of reuse and the impact of current documentation techniques on these problems. A qualitative approach is employed and developer experiences are captured from three separate reuse studies using multiple forms of data capture. Four major reuse problems are identified by this report. Understanding the functionality of components; the interactions between components; the mapping from the problem domain to the framework implementation and understanding the architectural assumptions in the framework design. The effectiveness of current forms of documentation is evaluated using these problem categories and, as a result, a number of suggestions for improving developer support during reuse are made.

This paper describes the development and experimental evaluation of a rigorous approach for effective object-oriented (OO) code inspection. Since their development, inspections have been shown to be powerful defect detection strategies but little research has been done to investigate their application to OO systems, which have very different structural and execution models compared to procedural systems. Previous investigations have demonstrated that the delocalised nature of OO software - the resolution of frequent non-local references, and the incongruous relationship between its static and dynamic representations, are primary inhibitors to its effective inspection. The experiment investigates a set of three complementary code reading techniques devised specifically to address these problems: one based on a checklist adapted to address the identified problems of OO inspections, one focused on the systematic construction of abstract specifications, and the last centered on the dynamic slice that a use-case takes through a system. The analysis shows that there is a significant difference in the number of defects found between the three reading techniques. The checklist-based technique emerges as the most effective approach but the other techniques also have noticeable strengths and so for the best results in a practical situation a combination of techniques is recommended.

Software inspection is now widely accepted as an effective technique for defect detection. This acceptance is largely based on studies using procedural program code. This paper presents empirical evidence that raises significant questions about the application of inspection to object-oriented code.

The benefits of the object-oriented paradigm are widely cited. At the same time, inspection is deemed to be the most cost-effective means of detecting defects in software products. Why then, is there no published experience, let alone quantitative data, on the application of inspection to object-oriented systems? This paper describes the facilities of the object-oriented paradigm and the issues that these raise when inspecting object-oriented code. Several problems are caused by the disparity between the static code structure and its dynamic runtime behaviour. The large number of small methods in object-oriented systems can also cause problems. This is followed by a description of three areas which may help mitigate problems found. Firstly, the use of various programming techniques may assist in making object-oriented code easier to inspect. Secondly, improved program documentation can help the inspector understand the code under inspection. Finally, tool support can help the inspector analyse the dynamic behaviour of the code. In conclusion, while both the object-oriented paradigm and inspection provide excellent benefits on their own, combining the two may be a difficult exercise, requiring extensive support if it is to be successful.

The object-oriented paradigm is becoming increasingly popular as a result of expert opinion and anecdotal evidence and not on the basis of sound empirical data. The questionnaire survey was undertaken as part of a programme of research to validate unsupported claims about the paradigm. The questionnaire follows structured interviews of experienced object-oriented developers with the intention of confirming the findings on a wider practitioner group. It was posted to relevant electronic newsgroups and to members of an object-oriented (postal) mailing list. The survey received 167 responses to the electronic questionnaire and 119 responses (30% response rate) to the postal version. Results show that respondents are of the view that: (i) the object-oriented paradigm has advantages over other paradigms in terms of ease of analysis and design, programmer productivity, software reuse, and ease of maintenance; (ii) inheritance can introduce difficulties when trying to understand object-oriented software; (iii) missing design documentation and poor or inappropriate design are prevalent problems; (iv) maintenance causes degradation of object-oriented software, but less frequently than conventional software; (v) C++ has many deficiencies in comparison to other purer object-oriented languages.

The empirical study was undertaken as part of a programme of research to explore unsupported claims about the object-oriented paradigm: a series of experiments tested the effect of inheritance on the maintainability of object-oriented software. Subjects were asked to modify object-oriented software with a hierarchy of 3 levels of inheritance depth and equivalent object-based software with no inheritance. The collected timing data showed that subjects maintaining object-oriented software using inheritance performed the modification tasks, on average, approximately 20% quicker than those maintaining equivalent object-based software with no inheritance. An initial inductive analysis revealed that 2 out of 3 subjects performed faster when maintaining the object-oriented software with inheritance. The findings are sufficiently important that attempts to verify the results should be made by independent researchers. Subsequent studies should seek to scale up the findings to the maintenance of more complex software by professional programmers.

This paper investigates the application of object-oriented construction to fault tolerant systems. The resulting system provides traditional fault tolerance within objects, but also a new form of fault tolerance between objects: object diversity. Object diversity extends current practice by integrating diversity in two directions: data and algorithm. This resulting form will allow increased diversity to be incorporated within fault tolerant systems. Further benefits are derived from the use of the inheritance hierarchy as a natural source of redundant components. As class libraries (both general and application specific) grow, more and more "free" redundant components will become available, yielding increasing savings on production costs.>

Empirical studies carried out to help understand the problems of software maintenance are widely held to be of value. A view perhaps less widely recognised within the software engineering domain is that experiments should be replicated both internally and externally to validate the results and build up a cohesive body of knowledge. This paper presents the external replication findings of an experiment which tested the benefits to maintenance of using modular code against nonmodular (monolithic) code. The results of our replication were strikingly different from those of the original which showed that a modular program could be maintained significantly faster than an equivalent monolithic version. An inductive analysis, undertaken to investigate the reasons for this, uncovered evidence of an ability effect and the suggestion that the experiment may have been too artificial.>

Abstract This paper presents a way of generating acceptance and systems test data from documents produced as part of the traditional structured approaches to systems analysis and design. Entity‐relationship diagrams are analysed in a fashion similar to boundary‐value analysis to generate combinations of entities that should be supported by the system. Data flow diagrams are analysed in a way that generates tests to exercise the passage of each data item through the system. This information is augmented with details from the process specifications to ensure, firstly, that the specifications themselves are tested to branch coverage level and secondly, that no infeasible tests are generated. Finally, the ‘entity life histories’ are examined and tests generated to evaluate the full range of possible states experienced by the entities within the system. The result is a common pool of test data requirements and expected results that may be used to test the corresponding development stage (typically, system or acceptance testing) or may be fed into the next development stage as walkthrough or review data. The paper concludes by presenting the results of an initial evaluation of the methods.

Abstract A collection of 69 references drawn from books, papers and conference proceedings on the subject of software testing is presented. Each reference is accompanied by a paragraph describing its contents. The aim when selecting the references was to minimize the number and maximize the coverage of the subject. For this reason, the bibliography is primarily intended for the researcher or practitioner who is relatively new to the subject. The bibliography is organized according to the following sections: general introductions; background; testing methods; experimental studies; and tools.

Abstract A new structural testing method (STM) is presented for use with JSP (Jackson structured programming) designed programs. The method involves the creation of a path expression from the program structure and subsequent expansion of the expression into a set of paths using an algebra described in the paper. The treatment of each JSP construct is dealt with in detail. A worked example demonstrates the use of the method to generate test cases. The method is compared in terms of coverage to other well‐known structural testing methods. The advantages and disadvantages of the method are discussed and further work currently being undertaken by the authors is outlined.