APIs matter in software development, but determining how to use them can be challenging. Developers often refer to a small set of API usage examples, analyzing the information there to understand and adapt them to their own context. Generalization over many examples may aid in understanding commonalities and differences, reducing information overload while including greater variety. We propose ASGard, a novel approach that generates API usage templates from examples. Approximating the formal problem of E-generalization, ASGard generalizes all syntactic and some semantic information within the examples to arrive at pseudocode representations that retain the commonality of the usage examples but abstract the varying aspects. We evaluate the templates from our approach and the patterns generated from PAM and MUDetect (two existing tools for API data mining), using a total of 1,954 API usage examples across 59 different APIs. We measure the quality of the resulting templates: ASGard’s templates have superior completeness and compression. We perform a user study on ASGard with 12 participants to compare the use of these templates in solving programming tasks, compared to MUDetect. We find that participants solved the programming tasks in significantly less time with ASGard. Participants expressed a general preference for using ASGard templates.

We present a process utilizing an evolutionary learning method to explore combinations of security mechanisms with regard to performance problems they might create for a particular user profile. For each combination, the process uses an evolutionary search to identify sequences of interactions with a computer (in form of a virtual machine) that stress the system to a much larger degree with the combination installed than without it. The process then compares the mechanism combinations using the “best sequences” for each combination to suggest the combination that overall has the least impact on performance. The process also explores interaction sequences that caused system failure, or were not able to finish within the given time limit, to identify incompatibilities between security mechanisms. For evaluation, the process was applied to create a tool for finding the best set of multiple anti-virus software systems for Windows XP. In the primary evaluation, the tool identified a set of five mechanisms that did not degrade performance too far, while providing the intended security coverage. At the same time, the tool found a clear incompatibility between two mechanisms as demonstrated by a zip operation failure after only a few interactions.

From the point of view of a user, a security mechanism for a computer should protect it from the particular kind of attacks it is designed for, while influencing the performance of the computer for the user's applications as little as possible. In this paper, we present an evolutionary learning approach for exploratory testing of the performance consequences of a security mechanism on the user's usage profile, i.e. the applications the user is using.By learning application interaction sequences with performances that are the most negatively influenced by the installation of a security mechanism, a user can evaluate if the performance losses are acceptable and by applying our approach to several mechanisms with comparable protection, a user can make an informed decision which mechanism is better for him/her. As proof-of-concept, we used our approach to explore anti-virus security mechanisms operating in a Windows XP environment. Our experiments show that different usage profiles are indeed better served by different security mechanism.

In this paper, we present the use of a so-called Game Management AI to classify players not just by passively observing them, but by actively manipulating the game to get the players to provide data currently missing to achieve the classification. We call this "Active Probing". The Game Management AI uses two sets of rules, one set that contains rules that are intended to represent the knowledge allowing a classification and one set that contains rules that indicate which game events can contribute to triggering conditions used in the first rule set. When a rule of the first set comes near to being triggered, the event suggested by an appropriate rule in the second set is then offered to the player in the game. We instantiated this use of a Game Management AI to identify players with a very high interest level for the role playing game "Realm of Dreams", a game that we created for this purpose. Our experimental evaluation showed that using the active probing by the Game Management AI allows us to identify players in our targeted class in a quarter of the time that was needed to classify such players without active probing.

We present the idea of using variants of simple games as an easy additional application area to establish generality of AI concepts. We substantiate this idea by using multi-hammer Whack-a-mole as application area for the efficiency improvement advisor and extended efficiency improvement advisor concepts for self-organizing multi-agent systems. Both concepts have previously been applied to pickup-and-delivery problems and were claimed to be general concepts for improving solving dynamic task fulfillment problems. Our experiments with multi-hammer Whack-a-mole show similar improvements to the other area for both concepts, giving credit to the generality claim.

Early detection of lung nodules in thoracic Computed Tomography (CT) scans is of great importance for the successful diagnosis and treatment of lung cancer. Due to improvements in screening technologies, and an increased demand for their use, radiologists are required to analyze an ever increasing amount of image data, which can affect the quality of their diagnoses. Computer-Aided Detection (CADe) systems are designed to assist radiologists in this endeavor. Here, we present a CADe system for the detection of lung nodules in thoracic CT images. Our system is based on (1) the publicly available Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) database, which contains 1018 thoracic CT scans with nodules of different shape and size, and (2) a deep Convolutional Neural Network (CNN), which is trained, using the back-propagation algorithm, to extract valuable volumetric features from the input data and detect lung nodules in sub-volumes of CT images. Considering only those test nodules that have been annotated by four radiologists, our CADe system achieves a sensitivity (true positive rate) of 78.9% with 20 false positives (FPs) per scan, or a sensitivity of 71.2% with 10 FPs per scan. This is achieved without using any segmentation or additional FP reduction procedures, both of which are commonly used in other CADe systems. Furthermore, our CADe system is validated on a larger number of lung nodules compared to other studies, which increases the variation in their appearance, and therefore, makes their detection by a CADe system more challenging.

Modern surveillance systems for practical applications with diverse and mobile sensors are large, complex, and expensive. It is known that unexpected behaviors can emerge from such systems, and when these behaviors correspond to weaknesses in a surveillance system, we call them emergent vulnerabilities. Given their cost and importance to security, it is essential to test these systems for such vulnerabilities prior to deployment. To that end, we automate the testing process with multiagent systems and machine learning. However, the conventional—and most intuitive–approach is to focus the machine learning on the subject system, which leads to a high‐dimensional problem that is intractable. Instead, we demonstrate in this paper that learning attacks on the system is tractable and provides a viable testing method. We demonstrate this with a series of studies in simulation and with a small‐scale model system featuring elements typically found in real physical surveillance systems. Our machine learning method finds successful attacks in simulation, which we can duplicate with the physical system. The method is scalable, with the implication that it could be used to test larger, real surveillance installations.

Incremental Adaptive Corrective Learning is a method for testing ad-hoc wireless networks for vulnerabilities that adversaries can exploit. It is based on an evolutionary search for tests that define behaviors for adversary-controlled network nodes. The search incrementally increases the number of such nodes and first adapts each new node to the behaviors of the already existing attackers before improving the behavior of all attackers. Tests are evaluated in simulations and behaviors are corrected to fulfill all protocol induced obligations that are not explicitly targeted for an exploit. In this paper, we substantiate the claim that this is a general method by instantiating it for different vulnerability goals and by presenting an application for cooperative collision avoidance using VANETs. In all those instantiations, the method is able to produce concrete tests that demonstrate vulnerabilities.

Understanding how organisms develop from a single cell into a functioning multicellular organism is one of the key questions in developmental biology. Research in this area goes back decades ago, but only recently have improvements in technology allowed biologists to achieve experimental results that are more quantitative and precise. Here, we show how large biological datasets can be used to learn a model for predicting the patterns of gene expression in Drosophila melanogaster (fruit fly) throughout embryogenesis. We also explore the possibility of considering spatial information in order to achieve unique patterns of gene expression in different regions along the anterior-posterior (head-tail) axis of the egg. We then demonstrate how the resulting model can be used to (1) classify these regions into the various segments of the fly, and (2) to conduct a virtual gene knockout experiment. Our learning algorithm is based on a model that has biological meaning, which indicates that its structure and parameters have their correspondence in biology.

Dynamic particle swarm optimization (PSO) problems are generally characterized by the exhaustively examined issues of the changing location of optima, the changing fitness of optima, and measurement noise/errors. However, the challenging issue of continuously changing problem dimensionality has not been similarly examined. Given that in anytime dynamic resource allocation it is necessary to maintain a high quality solution, we argue that, rather than restarting the PSO algorithm, a more appropriate approach is to design an algorithm that robustly handles changing problem dimensionality. Specifically, we propose an indirect particle encoding scheme specifically designed for a dynamic multi-dimensional PSO algorithm for proportional fair constrained resource allocation. This PSO algorithm is implemented for the proportional fair allocation of power and users to channels within a simulation of an Orthogonal Frequency-Division Multiple Access (OFDMA) wireless network with mobile users switching cells as they traverse the simulation environment. The proposed PSO algorithm is evaluated using simulations, which demonstrate the ability of the proposed indirect encoding scheme to maximize the overall proportional fair optimization goal, without unfairly penalizing the individual components of the solution related to newly introduced problem dimensions.

We present an agent architecture and a hybrid behavior learning method for it that allows the use of communicated intentions of other agents to create agents that are able to cooperate with various configurations of other agents in fulfilling a task.

We present incremental adaptive corrective learning as a method to test ad-hoc wireless network protocols and applications. This learning method allows for the evolution of complex, variable-length, cooperative behaviour patterns for adversarial agents acting in such networks. We used the method to test precision agriculture sensor networks for vulnerabilities which could be exploited by attackers to significantly increase power consumption within the network. Our technique was able to find behaviours which increased power consumption by at least a factor of 3.6 for a node in each of the tested scenarios.

Large, complex systems can exhibit unforeseen behaviours. In the case of surveillance and security systems, these behaviours can be weaknesses that should be discovered by automated testing and ameliorated. Previous work has shown that such automated testing can be done using particle swarm optimization to learn behaviours that allow a set of attackers to defeat the system. However, for the optimization to succeed, it must have some knowledge about what constitutes a successful attack in order to guide the swarm. This knowledge is encapsulated in a goal ordering structure. In this paper, we examine the goal ordering structure and its role in the learning of system weakness. We specifically look at applications in harbour surveillance and security, and show how knowledge of the likely properties of a successful attack can be added to the goal ordering structure. Our experimental results show that adding knowledge to the goal ordering structure improves the search, when that knowledge is correctly inserted into the structure.

When an interaction mechanism such as argumentation is considered for use in open multi agent domains, such as E-Commerce or other business applications, it is necessary to consider the possibility of agents performing malicious actions. Common themes when studying malicious actions in communication protocols are that of withholding information and misrepresenting information. In argumentation, however, the use of a complex underlying formal logic allows for the possibility of another type of malicious action: the introduction of superfluous complexity into information, designed to overwhelm the reasoning capacity of another agent. We examine a malicious strategy in open multi agent systems based on exploiting the complexity of the formal logic underlying argumentation in order to manipulate the outcome of argument acceptability evaluation. Further, we briefly discuss the general problem of defensive strategies against this type of malicious argumentation, and the inherent difficulty in detecting occurrences of it.

We study the effects of an imitation mechanism on a population of animats capable of individual ontogenetic learning. An urge to imitate others augments a network-based reinforcement learning strategy used in the control system of the animats. We test populations of animats with imitation against populations without for their ability to find, and maintain over generations, successful foraging behavior in an environment containing three necessary resources: food, water, and shelter. We conclude that even simple imitation mechanisms are effective at increasing the frequency of success when measured over time and over populations of animats.

We present a general scheme for testing multiagent systems, respectively policies used by them, for unwanted emergent behavior using learning of cooperative behavior via particle swarm systems. By using particle swarm systems in this setting, we are able to create agents interacting/attacking the tested agents that can use parameterised high-level actions. We also can evaluate the quality of an attack using several measures that can be prioritised and used in a multi-objective manner in the search. This solves some general problems of other testing approaches using learning. We instantiate this general scheme to test harbour patrol and interception policies for two Canadian harbours, showing that our approach is able to find problems in these policies.

Source code examples are valuable to developers needing to use an unfamiliar application programming interface (API). Numerous approaches exist to help developers locate source code examples; while some of these help the developer to select the most promising examples, none help the developer to reuse the example itself. Without explicit tool support for the complete end-to-end task, the developer can waste time and energy on examples that ultimately fail to be appropriate; as a result, the overhead required to reuse an example can restrict a developer's willingness to investigate multiple examples to find the "best" one for their situation. This paper outlines four case studies involving the end-to-end use of source code examples: we investigate the overhead and pitfalls involved in combining a few state-of-the-art techniques to support the end-to-end use of source code examples.

We present an extension to the definition of a concept in an ontology that allows an agent to simultaneously communicate with a group of agents that might have different understandings of some concepts. We also provide a way to learn such non-unanimo

Knowledge discovery systems extract knowledge from data that can be used for making prediction about incomplete data items. Utility is a measure of the usefulness of the discovered knowledge and satisfaction of the user with that knowledge. We motivate and address the question of usefulness of sanitized data using the notion of utility in data mining systems. For this we measure the success of patterns and rules discovered from the original data to make predictions about the sanitized data using a previously developed framework. Using experimental results on a set of medical data we demonstrate that it is possible to make useful predictions about the sanitized medical data when rules discovered from the original unsanitized medical data are used. We explain our results and compare it with the case where no sanitization is involved.

Developers perform small-scale reuse tasks to save time and to increase the quality of their code, but due to their small scale, the costs of such tasks can quickly outweigh their benefits. Existing approaches focus on locating source code for reuse but do not support the integration of the located code within the developer's system, thereby leaving the developer with the burden of performing integration manually. This paper presents an approach that uses the developer's context to help integrate the reused source code into the developer's own source code. The approach approximates a theoretical framework (higher-order anti-unification modulo theories), known to be undecidable in general, to determine candidate correspondences between the source code to be reused and the developer's current (incomplete) system. This approach has been implemented in a prototype tool, called Jigsaw, that identifies and evaluates candidate correspondences greedily with respect to the highest similarity. Situations involving multiple candidate correspondences with similarities above a defined threshold are presented to the developer for resolution. Two empirical evaluations were conducted: an experiment comparing the quality of Jigsaw's results against suspected cases of small-scale reuse in an industrial system; and case studies with two industrial developers to consider its practical usefulness and usability issues.

In this paper, we describe BuildBot, a robotic interface developed to assist with the continuous integration process utilized by co-located agile software development teams. BuildBot's physical nature allows us to engage the agile software development team members through vision, hearing and touch. In this way, BuildBot becomes an active part of the development process by bringing together human-robot interaction, human group dynamics and software engineering concepts through a number of interaction modalities. In this paper we describe the design and implementation of the BuildBot prototype, a robotic interface that can sense virtual stimuli, in this case the state of a software build, and react accordingly in a physical way via vision, sound and touch. We present an early evaluation comparing BuildBot to two other tools used by an agile team to monitor the continuous integration process. We also show preliminary results indicating that BuildBot may be more noticeable to the developers and contribute to a fun and lighthearted atmosphere. We argue that by increasing awareness of the state of the software build, BuildBot can assist in the self-supervision of agile software engineering teams and can help the team achieve its goals in a more engaging and sociable manner.

Generalization tasks are important for continual improvement to the design of an evolving code base, eliminating redundancy where it has accumulated. An important step in generalization is identifying the detailed structural correspondence between two pieces of code being considered for generalization. Unfortunately, tool support for this step is insufficient, leaving the developer to resort to tedious and error-prone manual determination of correspondence. This paper presents an approach for automatically determining correspondences as an early step in a generalization task. The approach is implemented in a proof-of-concept plug-in to the Eclipse integrated development environment. Two small empirical evaluations of the tool have been conducted: a comparison between human attempts to determine detailed correspondences and those of the tool; and, a comparison of the use of the tool to the use of diff/CCFinder in performing generalization tasks.

We present CoLe, a cooperative data mining approach for discovering hybrid knowledge. It employs multiple different data mining algorithms, and combines results from them to enhance the mined knowledge. For our medical application area, we analyse several focusing strategies that allowed us to gain medically significant results.

We present an approach to use evolutionary learning of behavior to improve testing of commercial computer games. After identifying unwanted results or behavior of the game, we propose to develop measures on how near a sequence of game states comes to the unwanted behavior and to use these measures within the fitness function of a GA working on action sequences. This allows to find action sequences that produce the unwanted behavior, if they exist. Our experimental evaluation of the method with the FIFA-99 game and scoring a goal as unwanted behavior shows that the method is able to find such action sequences, allowing for an easy reproduction of critical situations and improvements to the tested game.

We present an improvement to distributed GAs based on migration of individuals between several concurrently evolving populations. The idea behind our improvement is to not only use the fitness of an individual as criterion for selecting the individuals that migrate, but also to consider the diversity of individuals versus the currently best individual. We experimentally show that a distributed GA using a weighted sum of fitness and a diversity measure for selecting migrating individuals finds the known optimal solutions to benchmark problems from literature (that offer a lot of local optima) on average substantially faster than the distributed GA using only fitness for selection. In addition, the run times of several runs of the distributed GA to the same problem instance vary much less with our improvement than in the base case, thus resulting in a more stable behavior of a distributed GA of this type.

We present a multi-agent based approach for achieving cooperation between search systems employing different search paradigms. The search agents periodically interrupt their search, select interesting information from their states that is transmitted to the other agents, filter the information sent to them with respect to their own demands, integrate the remaining information into their search, and then continue the search. There are different kinds of information to be exchanged and the selection is both success- and demand-driven. We demonstrate the usefulness of this approach by coupling a search system based on a genetic algorithm and a branch-and-bound based system for job-shop-scheduling. Our experiments show that the cooperation results in finding better solutions within a given time limit and in finding solutions comparable to those generated by the best system working alone in less time. The speed-up factors for some examples even exceed the number of agents (computers) used.