Abstract Testing Android apps effectively requires a systematic exploration of the app’s possible states by simulating user interactions and system events. While existing approaches have proposed several fuzzing techniques to generate various text inputs and trigger user and system events for GUI state exploration, achieving high code coverage remains a significant challenge in Android app testing. The main challenges are (1) reasoning about the complex and dynamic layout of GUI screens; (2) generating required inputs/events to deal with certain widgets like pop-ups; and (3) coordination between current test inputs and previous inputs to avoid getting stuck in the same GUI screen without improving test coverage. To address these problems, we propose VLM-Fuzz , a novel automated approach for Android GUI testing. At its foundation, VLM-Fuzz utilizes a heuristic-based, recursive depth-first search (DFS) strategy that is intelligently guided by a Vision Language Model (VLM) to effectively explore the app’s complex GUI states. The core innovation of VLM-Fuzz is not simply the use of a VLM, but its strategic, on-demand integration within a hybrid exploration framework. Our approach combines a fast, heuristic-based DFS for standard GUI interactions with targeted, VLM-assisted analysis for visually complex screens. We use static analysis to analyze the Android Manifest file and the runtime GUI hierarchy XML to extract the list of components, intent-filters and interactive GUI widgets. VLM is used to reason about complex GUI layout and widgets on an on-demand basis. Based on the inputs from static analysis, VLM, and the current GUI state, we use some heuristics to deal with the above-mentioned challenges. We evaluated VLM-Fuzz based on a benchmark containing 59 apps obtained from a recent work and compared it against two state-of-the-art approaches: APE and DeepGUI . VLM-Fuzz outperforms the best baseline by 9.0% , 3.7% , and 2.1% in terms of class coverage, method coverage, and line coverage, respectively. We also ran VLM-Fuzz on 80 recent Google Play apps (i.e., updated in 2024). VLM-Fuzz detected 52 unique crashes in 12 apps, which have been reported to respective developers.

The increasing integration of IoT devices into critical infrastructure has made them prime targets for cyberattacks. Many of these devices rely on outdated or legacy software, which introduces inherent vulnerabilities and complicates firmware updates, making identifying and testing these weaknesses essential. Traditional methods typically employ black-box approaches, mutating network requests generated during device operation to craft potential attack vectors. However, these methods face limitations when dealing with encrypted or proprietary protocols. Recent tools, such as Diane and IoTFuzzer, interact with IoT devices through their mobile companion apps and use fuzzing techniques to modify request content, causing crashes in IoT device software. Although these approaches can effectively trigger software crashes, they do not generate actual exploits, as they do not precisely target or exploit specific vulnerabilities. To address these limitations, we introduce MITHRAS, the first approach that uses mobile companion apps to deliver maliciously mutated requests directly to IoT devices, explicitly targeting Remote Code Execution (RCE) vulnerabilities. MITHRAS uses Deep Reinforcement Learning to efficiently navigate the communication code within companion apps, dynamically mutating request payloads before transmission. Adapting to previous attack outcomes, MITHRAS refines its strategy, mimicking human decision-making to improve the effectiveness of exploit generation.

As the software engineering research community continues to propose novel approaches to automated test case generation for REST APIs, researchers face the labor-intensive task of empirically validating their methodologies and comparing them with the state-of-the-art. This process requires assembling a benchmark of case studies (notoriously difficult to find in the context of REST API testing), building and running each API, gathering competitor tools, conducting experimental testing sessions to collect effectiveness and efficiency metrics, and processing the results. These extensive engineering efforts consume time that could be otherwise spent on more research-oriented tasks. This paper introduces RESTgym, a flexible empirical infrastructure designed to assess the performance of REST API testing tools and facilitate comparative analysis with state-of-the-art approaches. By providing a standardized environment for comparison (currently consisting of 11 benchmark APIs and 6 state-of-the-art tools packed into containers, but easily extensible to add new APIs and tools) and an orchestration engine, RESTgym significantly reduces the time and effort required for researchers to evaluate REST API testing methodologies. The paper details the architecture and components of RESTgym and demonstrates its utility through a practical example, highlighting its potential to speed up research and development in automatedREST API testing. Video:http://tiny.cc/restgym-video

This issue of the Journal of Software:Evolution and Process focuses on the foundation of software engineering-the source code itself.While much of the software engineering community properly emphasizes aspects like specification, design, and requirements engineering, the source code provides the only precise description of a system's behavior.Therefore, the analysis and manipulation of source code remain critical concerns.This issue contains, among others, the extended version of the best papers presented at the 22nd

In the contemporary security landscape, the incorporation of photonics has emerged as a transformative force, unlocking a spectrum of possibilities to enhance the resilience and effectiveness of security primitives. This integration represents more than a mere technological augmentation; it signifies a paradigm shift towards innovative approaches capable of delivering security primitives with key properties for low-power systems. This not only augments the robustness of security frameworks, but also paves the way for novel strategies that adapt to the evolving challenges of the digital age. This paper discusses the security layers and related services that will be developed, modeled, and evaluated within the Horizon Europe NEUROPULS project. These layers will exploit novel implementations for security primitives based on physical un-clonable functions (PUFs) using integrated photonics technology. Their objective is to provide a series of services to support the secure operation of a neuromorphic photonic accelerator for edge comnuting applications.

Hyperproperties are used to define correctness requirements that involve relations between multiple program executions. This allows, for instance, to model security and concurrency requirements, which cannot be expressed by means of trace properties.

Automatically crafting test scenarios for REST APIs helps deliver more reliable and trustworthy web-oriented systems. However, current black-box testing approaches rely heavily on the information available in the API's formal documentation, i.e., the Open API Specification (OAS for short). While useful, the OAS mostly covers syntactic aspects of the API (e.g., producer-consumer relations between operations, input value properties, and additional constraints in natural language), and it lacks a deeper understanding of the API business logic. Missing semantics include implicit ordering (logic dependency) between operations and implicit input-value constraints. These limitations hinder the ability of black-box testing tools to generate truly effective test cases automatically.

Java bytecode is a quite high-level language and, as such, it is fairly easy to analyze and decompile with malicious intents, e.g., to tamper with code and skip license checks. Code obfuscation was a first attempt to mitigate malicious reverse-engineering based on static analysis. However, obfuscated code can still be dynamically analyzed with standard debuggers to perform step-wise execution and to inspect (or change) memory content at important execution points, e.g., to alter the verdict of license validity checks. Although some approaches have been proposed to mitigate debugger-based attacks, they are only applicable to binary compiled code and none address the challenge of protecting Java bytecode. In this article, we propose a novel approach to protect Java bytecode from malicious debugging. Our approach is based on automated program transformation to manipulate Java bytecode and split it into two binary processes that debug each other (i.e., a self-debugging solution). In fact, when the debugging interface is already engaged, an additional malicious debugger cannot attach. To be resilient against typical attacks, our approach adopts a series of technical solutions, e.g., an encoded channel is shared by the two processes to avoid leaking information, an authentication protocol is established to avoid Man-in-the-middle attacks, and the computation is spread between the two processes to prevent the attacker to replace or terminate either of them. We test our solution on 18 real-world Java applications, showing that our approach can effectively block the most common debugging tasks (either with the Java debugger or the GNU debugger) while preserving the functional correctness of the protected programs. While the final decision on when to activate this protection is still up to the developers, the observed performance overhead was acceptable for common desktop application domains.

This special session paper introduces the Horizon Europe NEUROPULS project, which targets the development of secure and energy-efficient RISC-V interfaced neuromorphic accelerators using augmented silicon photonics technology. Our approach aims to develop an augmented silicon photonics platform, an FPGA-powered RISC-V-connected computing platform, and a complete simulation platform to demonstrate the neuromorphic accelerator capabilities. In particular, their main advantages and limitations will be addressed concerning the underpinning technology for each platform. Then, we will discuss three targeted use cases for edge-computing applications: Global National Satellite System (GNSS) anti-jamming, autonomous driving, and anomaly detection in edge devices. Finally, we will address the reliability and security aspects of the stand-alone accelerator implementation and the project use cases.

Mass assignment is one of the most prominent vulnerabilities in RESTful APIs that originates from a misconfiguration in common web frameworks. This allows attackers to exploit naming convention and automatic binding to craft malicious requests that (massively) override data supposed to be read-only. In this paper, we adopt a black-box testing perspective to automatically detect mass assignment vulnerabilities in RESTful APIs. Indeed, execution scenarios are generated purely based on the OpenAPI specification, that lists the available operations and their message format. Clustering is used to group similar operations and reveal read-only fields, the latter are candidates for mass assignment. Then, test interaction sequences are automatically generated by instantiating abstract testing templates, with the aim of trying to use the found read-only fields to carry out a mass assignment attack. Test interactions are run, and their execution is assessed by a specific oracle, in order to reveal whether the vulnerability could be successfully exploited. The proposed novel approach has been implemented and evaluated on a set of case studies written in different programming languages. The evaluation highlights that the approach is quite effective in detecting seeded vulnerabilities, with a remarkably high accuracy.

RESTful services are commonly documented using OpenAPI specifications. Although numerous automated testing techniques have been proposed that leverage the machine-readable part of these specifications to guide test generation, their human-readable part has been mostly neglected. This is a missed opportunity, as natural language descriptions in the specifications often contain relevant information, including example values and inter-parameter dependencies, that can be used to improve test generation. In this spirit, we propose NLPtoREST, an automated approach that applies natural language processing techniques to assist REST API testing. Given an API and its specification, NLPtoREST extracts additional OpenAPI rules from the human-readable part of the specification. It then enhances the original specification by adding these rules to it. Testing tools can transparently use the enhanced specification to perform better test case generation. Because rule extraction can be inaccurate, due to either the intrinsic ambiguity of natural language or mismatches between documentation and implementation, NLPtoREST also incorporates a validation step aimed at eliminating spurious rules. We performed studies to assess the effectiveness of our rule extraction and validation approach, and the impact of enhanced specifications on the performance of eight state-of-the-art REST API testing tools. Our results are encouraging and show that NLPtoREST can extract many relevant rules with high accuracy, which can in turn significantly improve testing tools’ performance.

The immutable nature of Ethereum transactions, and consequently Ethereum smart-contracts, has stimulated the proliferation of many approaches aiming at detecting defects and security issues before the deployment of smart-contracts on the blockchain. Indeed, the actions performed by smart-contracts instantiated on the blockchain, possibly involving substantial financial value, cannot be undone. Unfortunately, smart-contracts source code is not always available, hence approaches based on static analysis have very often to face the problem of inspecting the compiled Ethereum Virtual Machine (EVM) bytecode, retrieved directly from the blockchain. However, due to the intrinsic complexity of EVM bytecode (especially in jumps address resolution), the state-of-the-art static analysis-based solutions have poor accuracy in the automated detection of Ethereum smart-contracts programming defects and vulnerabilities. This paper presents a novel approach based on symbolic execution of the EVM operands stack that allows to resolve jumps address in the EVM bytecode and to construct a precise Control-Flow Graph (CFG) of compiled smart-contracts. Many static analysis techniques are based on a CFG-based representation of the smart-contract to validate, and would therefore benefit from our approach. We have implemented the CFG reconstruction algorithm in a tool called EtherSolve . Then, we have validated the tool on a large dataset of real-world Ethereum smart-contracts, showing that EtherSolve extracts more precise CFGs, w.r.t. state-of-the-art available approaches. Finally, we have extended EtherSolve with two detectors for two of the most prominent Ethereum smart-contracts vulnerabilities (Reentrancy and Tx.origin). Experimental results show that exploiting the proposed CFG reconstruction static analysis, leads to more accurate vulnerabilities detection, w.r.t. state-of-the-art security tools. Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Over the past few years, several novel black-box testing approaches targeting RESTful APIs have been proposed. In order to assess their effectiveness, such testing strategies had to be implemented as a prototype tool and validated on empirical data. However, developing a testing tool is a time-consuming task, and reimplementing from scratch the same common basic features represents a waste of resources that causes a remarkable overhead in the "time to market" of research results.In this paper, we present RestTestGen, an extensible framework for implementing new automated black-box testing strategies for RESTful APIs. The framework provides a collection of commonly used components, such as a robust OpenAPI specification parser, dictionaries, input value generators, mutation operators, oracles, and others. Many of the provided components are customizable and extensible, enabling researchers and practitioners to quickly prototype, deploy, and evaluate their novel ideas. Additionally, the framework facilitates the development of novel black-box testing strategies by guiding researchers, by means of abstract components that explicitly identify those parts of the framework requiring a concrete implementation.As an adoption example, we show how we can implement nominal and error black-box testing strategies for RESTful APIs, by reusing primitives and features provided by the framework, and by concretely extending very few abstract components.RestTestGen is open-source, actively maintained, and publicly available on GitHub at https://github.com/SeUniVr/RestTestGen

Software is constantly changing as developers add new features or make changes. This directly impacts the effectiveness of the test suite associated with that software, especially when the new modifications are in an area where no test case exists. This article addresses the issue of developing a high-quality test suite to repeatedly cover a given point in a program, with the ultimate goal of exposing faults affecting the given program point. Our approach, IFRIT, uses Deep Reinforcement Learning to generate diverse inputs while keeping a high level of reachability of the desired program point. IFRIT achieves better results than state-of-the-art and baseline tools, improving reachability, diversity and fault detection.

Products and materials traceability is essential in modern manufacturing, where the production must meet certain standards that range from Quality Control (QC) to the quality of the used materials. In this environment, blockchain applications allow certifying data provenience and subsequent modification, offering trust and security along the entire supply chain. Nonetheless, the design and the development of such applications are usually performed manually and, thus, subject to errors.In this paper, we propose a methodology allowing to automatically generate smart contracts starting from a SysML model. This approach allows easing the integration of blockchain applications in a production system: by abstracting the implementations with models, it is possible to generate smart contracts for different blockchains, connecting to multiple production environments.We applied the proposed methodology on a real manufacturing system, assessing the quality of a case-study production.

On behalf of the SCAM 2022 Conference and Program Committee, we would like to welcome you to the beautiful Limassol, Cyprus for the 22nd IEEE International Working Conference on Source Code Analysis and Manipulation, co-located with the 38th IEEE International Conference on Software Maintenance and Evolution (ICSME 2022). We are grateful for organizing the first in-person conference edition after COVID-19.

Abstract RESTful APIs (or REST APIs for short) represent a mainstream approach to design and develop web APIs using the REpresentational State Transfer architectural style. Black‐box testing, which assumes only the access to the system under test with a specific interface, is the only viable option when white‐box testing is impracticable. This is the case for REST APIs: their source code is usually not (or just partially) available, or a white‐box analysis across many dynamically allocated distributed components (typical of a micro‐services architecture) is computationally challenging. This paper presentsRestTestGen, a novel black‐box approach to automatically generate test cases for REST APIs, based on their interface definition (an OpenAPI specification). Input values and requests are generated for each operation of the API under test with the twofold objective of testing nominal execution scenarios and error scenarios. Two distinct oracles are deployed to detect when test cases reveal implementation defects. While this approach is mainly targeting the research community, it is also of interest to developers because, as a black‐box approach, it is universally applicable across different programming languages, or in the case external (compiled only) libraries are used in a REST API. The validation of our approach has been performed on more than 100 of real‐world REST APIs, highlighting the effectiveness of the approach in revealing actual faults in already deployed services.

The state space of Android apps is huge, and its thorough exploration during testing remains a significant challenge. The best exploration strategy is highly dependent on the features of the app under test. Reinforcement Learning (RL) is a machine learning technique that learns the optimal strategy to solve a task by trial and error, guided by positive or negative reward, rather than explicit supervision. Deep RL is a recent extension of RL that takes advantage of the learning capabilities of neural networks. Such capabilities make Deep RL suitable for complex exploration spaces such as one of Android apps. However, state-of-the-art, publicly available tools only support basic, Tabular RL. We have developed ARES, a Deep RL approach for black-box testing of Android apps. Experimental results show that it achieves higher coverage and fault revelation than the baselines, including state-of-the-art tools, such as TimeMachine and Q-Testing. We also investigated the reasons behind such performance qualitatively, and we have identified the key features of Android apps that make Deep RL particularly effective on them to be the presence of chained and blocking activities. Moreover, we have developed FATE to fine-tune the hyperparameters of Deep RL algorithms on simulated apps, since it is computationally expensive to carry it out on real apps.

Several automated tools have been proposed to detect vulnerabilities. These tools are mainly evaluated in terms of their accuracy in detecting vulnerabilities, but the evaluation of their usability is a commonly neglected topic. Usability of automated security tools is particularly crucial when dealing with problems of cryptographic protocols for which even small—apparently insignificant—changes in their configuration can result in vulnerabilities that, if exploited, pave the way to attacks with dramatic consequences for the confidentiality and integrity of exchanged messages. This becomes even more acute when considering such ubiquitous protocols as the one for Transport Layer Security (TLS for short). In this paper, we present the design and the lessons learned of a user study, meant to compare two different approaches when reporting misconfigurations. Results reveal that including contextualized actionable mitigations in security reports significantly impact the accuracy and the time needed to patch TLS vulnerabilities. Along with the lessons learned, we share the experimental material that can be used during cybersecurity labs to let students configure and patch TLS first-hand.

Test coverage is a standard measure used to evaluate the completeness of a test suite. Coverage is typically computed on source code, by assessing the extent of source code entities (e.g., statements, data dependencies, control dependencies) that are exercised when running test cases. When considering REST APIs, an alternative perspective to assess test suite completeness is with respect to the service definition. This paper presents Restats, a test coverage tool for REST APIs that supports eight state-of-the-art test coverage metrics with a black-box perspective, i.e., only relying on the OpenAPI interface specification of the REST API under test. In fact, metrics are computed by only observing the HTTP requests and responses occurring at testing time, and no access to source/compiled code of the REST API is required. These coverage metrics come in handy for: (i) developers and test engineers working at development and maintenance tasks; (ii) stakeholders and customers who want to evaluate the completeness of acceptance tests; (iii) researches interested in comparing different automated test case generation strategies. Restats GitHub repository: https://github.com/SeUniVr/restats Restats demo video: https://smarturl.it/restats-demo

The degree of code coverage reached by a test suite is an important indicator of the thoroughness of testing. Most coverage tools for Android apps work at the bytecode level and provide no information to developers about which source code lines have not yet been exercised by any test case. In this paper, we present COSMO, the first fully automated Android app instrumenter publicly available that operates at the source code level in a completely transparent way, making it fully compatible with existing system level testing technologies and Android test generators. The experiments that we have conducted on a large benchmark of Android apps show that COSMO can successfully instrument most apps without altering their execution traces, introducing a small, acceptable runtime overhead.

The peculiar novelty of smart contracts is a computational model where irreversible transactions are stored in a distributed persistent data storage, namely the blockchain. The technical nature of this new type of software opens to new kinds of faults, which require specific test capabilities to be revealed. In this paper we present SOCRATES, an extensible and modular framework to automatically test smart contracts. The distinctive features of SOCRATES are: (1) a collection of composable behaviours that exercise smart contracts in the blockchain; (2) it deploys a society of bots, with the purpose of detecting defects arising from multi-user interactions, which are impossible to reveal when deploying a single bot. Our empirical investigation demonstrates that SOCRATES is able expose both known and previously unknown faults in smart contracts that are actively run in the official Ethereum blockchain. Moreover, we show that a society of multiple bots is more efficient in fault exposure than a single bot alone.

Motivated by the immutable nature of Ethereum smart contracts and of their transactions, quite many approaches have been proposed to detect defects and security problems before smart contracts become persistent in the blockchain and they are granted control on substantial financial value. Because smart contracts source code might not be available, static analysis approaches mostly face the challenge of analysing compiled Ethereum bytecode, that is available directly from the official blockchain. However, due to the intrinsic complexity of Ethereum bytecode (especially in jump resolution), static analysis encounters significant obstacles that reduce the accuracy of exiting automated tools. This paper presents a novel static analysis algorithm based on the symbolic execution of the Ethereum operand stack that allows us to resolve jumps in Ethereum bytecode and to construct an accurate control-flow graph (CFG) of the compiled smart contracts. EtherSolve is a prototype implementation of our approach. Experimental results on a significant set of real world Ethereum smart contracts show that EtherSolve improves the accuracy of the execrated CFGs with respect to the state of the art available approaches. Many static analysis techniques are based on the CFG representation of the code and would therefore benefit from the accurate extraction of the CFG. For example, we implemented a simple extension of EtherSolve that allows to detect instances of the re-entrancy vulnerability.

In literature, we can find research tools to automatically generate test cases for RESTful APIs, addressing the specificity of this particular programming domain. However, no direct comparison of these tools is available to guide developers in deciding which tool best fits their REST API project.In this paper, we present the results of an empirical comparison of automated black-box test case generation approaches for REST APIs. We surveyed the available black-box testing tools that have been proposed in recent literature, finding four usable prototypes: RestTestGen, RESTler, bBOXRT and RESTest. We used these tools to generate test cases for 14 real-world REST services. Then, testing results have been analyzed and compared in terms of robustness (i.e., success rate) and test coverage.Among the considered tools, RESTler appears to be the most solid, able to successfully test all case studies (the other tools experienced crashes). Conversely, test cases generated by RestTestGen scored the highest coverage, suggesting that its testing strategy is the most effective in testing REST APIs.

The ecosystem in which mobile applications run is highly heterogeneous and configurable. All layers upon which mobile apps are built offer wide possibilities of variations, from the device and the hardware, to the operating system and middleware, up to the user preferences and settings. Testing all possible configurations exhaustively, before releasing the app, is unaffordable. As a consequence, the app may exhibit different, including faulty, behaviours when executed in the field, under specific configurations.In this paper, we describe a framework that can be instantiated to support in-vivo testing of a mobile app. The framework monitors the configuration in the field and triggers in-vivo testing when an untested configuration is recognized. Experimental results show that the overhead introduced by monitoring is unnoticeable to negligible (i.e., 0-6%) depending on the device being used (high- vs. low-end). In-vivo test execution required on average 3s: if performed upon screen lock activation, it introduces just a slight delay before locking the device.

Automated web testing is an appealing option, especially when continuous testing practices are adopted. However, web test cases are known to be fragile and to break easily when a web application evolves. The Page Object (PO) design pattern addresses such problem by providing a layer of indirection that decouples test cases from the internals of the web page, where web page elements are located and triggered by the web tests. However, PO development could potentially introduce an additional burden to the already strictly constrained testing activities. This paper reports an empirical investigation of costs and benefits due to the introduction of the PO pattern in web test suite development. In particular, we conducted a family of controlled experiments in which test cases were developed with and without the PO pattern. While the benefits of POs did not compensate for the extra development effort they require in the limited experimental setting of our study, results indicate that when the test suite to be developed is at least 10× larger, test development becomes more efficient with than without POs.

RESTful APIs (or REST APIs for short) represent a mainstream approach to design and develop Web APIs using the REpresentational State Transfer architectural style. When their source code is not (or just partially) available or the analysis across many dynamically allocated distributed components (typical of a micro-services architecture) poses obstacles to white-box testing, black-box testing becomes a viable option. Black-box testing, in fact, only assumes access to the system under test with a specific interface. This paper presents RESTTESTGEN, a novel approach to automatically generate test cases for REST APIs, based on their interface definition (in Swagger). Input values and requests are generated for each operation of the API under test, with the twofold objective of testing nominal execution scenarios and of testing error scenarios. Two distinct oracles are deployed to detect when test cases reveal implementation defects. Our empirical investigation shows that this approach is effective in revealing actual faults on 87 real-world REST APIs.

Abstract The Android platform facilitates reuse of app functionalities by allowing an app to request an action from another app through inter-process communication mechanism. This feature is one of the reasons for the popularity of Android, but it also poses security risks to the end users because malicious, unprivileged apps could exploit this feature to make privileged apps perform privileged actions on behalf of them. In this paper, we investigate the hybrid use of program analysis, genetic algorithm based test generation, natural language processing, machine learning techniques for precise detection of permission re-delegation vulnerabilities in Android apps. Our approach first groups a large set of benign and non-vulnerable apps into different clusters, based on their similarities in terms of functional descriptions. It then generates permission re-delegation model for each cluster, which characterizes common permission re-delegation behaviors of the apps in the cluster. Given an app under test, our approach checks whether it has permission re-delegation behaviors that deviate from the model of the cluster it belongs to. If that is the case, it generates test cases to detect the vulnerabilities. We evaluated the vulnerability detection capability of our approach based on 1,258 official apps and 20 mutated apps. Our approach achieved 81.8% recall and 100% precision. We also compared our approach with two static analysis-based approaches — Covert and IccTA — based on 595 open source apps. Our approach detected 30 vulnerable apps whereas Covert detected one of them and IccTA did not detect any. Executable proof-of-concept attacks generated by our approach were reported to the corresponding app developers.

Malicious reverse engineering is a problem when a program is delivered to the end users. In fact, an end user might try to understand the internals of the program, in order to elaborate an attack, tamper with the software and alter its behaviour. Code obfuscation represents a mitigation to these kind of malicious reverse engineering and tampering attacks, making programs harder to analyze (by a tool) and understand (by a human). In this paper, we present Oblive, a tool meant to support developers in applying code obfuscation to their programs. A developer is required to specify security requirements as single-line code annotations only. Oblive, then, reads annotations and applies state-of-the-art data and code obfuscation, namely xormask with opaque mask and java-to-native code, while the program is being compiled. Oblive is successfully applied both to plain Java programs and Android apps. Showcase videos are available for the code obfuscation part https://youtu.be/Bml-BkKP3CU and for the data obfuscation part https://youtu.be/zbizYVK42ps.

Code obfuscation is a popular approach to turn program comprehension and analysis harder, with the aim of mitigating threats related to malicious reverse engineering and code tampering. However, programming languages that compile to high level bytecode (e.g., Java) can be obfuscated only to a limited extent. In fact, high level bytecode still contains high level relevant information that an attacker might exploit. In order to enable more resilient obfuscations, part of these programs might be implemented with programming languages (e.g., C) that compile to low level machine-dependent code. In fact, machine code contains and leaks less high level information and it enables more resilient obfuscations. In this paper, we present an approach to automatically translate critical sections of high level Java bytecode to C code, so that more effective obfuscations can be resorted to. Moreover, a developer can still work with a single programming language, i.e., Java.

Code protections aim at blocking (or at least delaying) reverse engineering and tampering attacks to critical assets within programs. Knowing the way hackers understand protected code and perform attacks is important to achieve a stronger protection of the software assets, based on realistic assumptions about the hackers' behaviour. However, building such knowledge is difficult because hackers can hardly be involved in controlled experiments and empirical studies. The FP7 European project Aspire has given the authors of this paper the unique opportunity to have access to the professional penetration testers employed by the three industrial partners. In particular, we have been able to perform a qualitative analysis of three reports of professional penetration test performed on protected industrial code. Our qualitative analysis of the reports consists of open coding, carried out by 7 annotators and resulting in 459 annotations, followed by concept extraction and model inference. We identified the main activities: understanding, building attack, choosing and customizing tools, and working around or defeating protections. We built a model of how such activities take place. We used such models to identify a set of research directions for the creation of stronger code protections.

Data obfuscations are program transformations used to complicate program understanding and conceal actual values of program variables. The possibility to hide constant values is a basic building block of several obfuscation techniques. For example, in XOR Masking a constant mask is used to encode data, but this mask must be hidden too, in order to keep the obfuscation resilient to attacks. In this paper, we present a novel technique based on the k-clique problem, which is known to be NP-complete, to generate opaque constants, i.e. values that are difficult to guess by static analysis. In our experimental assessment we show that our opaque constants are computationally cheap to generate, both at obfuscation time and at runtime. Moreover, due to the NP-completeness of the k-clique problem, our opaque constants can be proven to be hard to attack with state-of-the-art static analysis tools.

Static analysis and penetration testing are common techniques used to discover security bugs in implementation code. Penetration testing is often performed in black-box way by probing the attack surface of a running system and discovering its security holes. Static analysis techniques operate in a white-box way by analyzing the source code of a system and identifying security weaknesses. Because of their different nature, the two techniques report their findings in two different ways. This paper presents an exploratory study meant to determine whether a vulnerability report generated by a security tool based on static analysis is more or less useful than a report generated by a security tool based on penetration testing. The usefulness is judged from the perspective of the developers that have to devise a vulnerability-fixing patch. The initial results show an advantage when using penetration testing in one of the two cases we investigated.

Security testing is a pivotal activity in engineering secure software. It consists of two phases: generating attack inputs to test the system, and assessing whether test executions expose any vulnerabilities. The latter phase is known as the security oracle problem.

Obfuscation techniques are a general category of software protections widely adopted to prevent malicious tampering of the code by making applications more difficult to understand and thus harder to modify. Obfuscation techniques are divided in code and data obfuscation, depending on the protected asset. While preliminary empirical studies have been conducted to determine the impact of code obfuscation, our work aims at assessing the effectiveness and efficiency in preventing attacks of a specific data obfuscation technique - VarMerge. We conducted an experiment with student participants performing two attack tasks on clear and obfuscated versions of two applications written in C. The experiment showed a significant effect of data obfuscation on both the time required to complete and the successful attack efficiency. An application with VarMerge reduces by six times the number of successful attacks per unit of time. This outcome provides a practical clue that can be used when applying software protections based on data obfuscation.

This paper describes the ASPIRE reference architecture designed to tackle one major problem in this domain: the lack of a clear process and an open software architecture for the composition and deployment of multiple software protections on software applications.

Several techniques and tools have been proposed for the automatic generation of test cases. Usually, these tools are evaluated in terms of fault-revealing or coverage capability, but their impact on the manual debugging activity is not considered. The question is whether automatically generated test cases are equally effective in supporting debugging as manually written tests. We conducted a family of three experiments (five replications) with humans (in total, 55 subjects) to assess whether the features of automatically generated test cases, which make them less readable and understandable (e.g., unclear test scenarios, meaningless identifiers), have an impact on the effectiveness and efficiency of debugging. The first two experiments compare different test case generation tools (Randoop vs. EvoSuite). The third experiment investigates the role of code identifiers in test cases (obfuscated vs. original identifiers), since a major difference between manual and automatically generated test cases is that the latter contain meaningless (obfuscated) identifiers. We show that automatically generated test cases are as useful for debugging as manual test cases. Furthermore, we find that, for less experienced developers, automatic tests are more useful on average due to their lower static and dynamic complexity.

Programs often run under strict usage conditions (e.g., license restrictions) that could be broken in case of code tampering. Possible attacks include malicious reverse engineering, tampering using static, dynamic and hybrid techniques, on standard devices as well as in labs with additional special purpose hardware equipment. ASPIRE (http://www.aspire-fp7.eu) is a European FP7 research project devoted to the elaboration of novel techniques to mitigate and prevent attacks to code integrity, to code/data confidentiality and to code lifting. This paper presents the ongoing activity to define a set of metrics aimed at quantifying the effect on code of the ASPIRE protections. The metrics have been conceived based on a measurement framework, which prescribes the identification of the relevant code features to consider and of their relationships with attacks and protections.

Attackers can tamper with programs to break usage conditions. Different software protection techniques have been proposed to limit the possibility of tampering. Some of them just limit the possibility to understand the (binary) code, others react more actively when a change attempt is detected. However, the validation of the software protection techniques has been always conducted without taking into consideration a unified process adopted by attackers to tamper with programs. In this paper we present an extension of the mini-cycle of change, initially proposed to model the process of changing program for maintenance, to describe the process faced by an attacker to defeat software protections. This paper also shows how this new model should support a developer when considering what are the most appropriate protections to deploy.

Automatically generated test cases are usually evaluated in terms of their fault revealing or coverage capability. Beside these two aspects, test cases are also the major source of information for fault localization and fixing. The impact of automatically generated test cases on the debugging activity, compared to the use of manually written test cases, has never been studied before. In this paper we report the results obtained from two controlled experiments with human subjects performing debugging tasks using automatically generated or manually written test cases. We investigate whether the features of the former type of test cases, which make them less readable and understandable (e.g., unclear test scenarios, meaningless identifiers), have an impact on accuracy and efficiency of debugging. The empirical study is aimed at investigating whether, despite the lack of readability in automatically generated test cases, subjects can still take advantage of them during debugging.

More and more web applications suffer the presence of cross-site scripting vulnerabilities that could be exploited by attackers to access sensitive information (such as credentials or credit card numbers). Hence proper tests are required to assess the security of web applications. In this paper, we resort to a search based approach for security testing web applications. We take advantage of static analysis to detect candidate cross-site scripting vulnerabilities. Input values that expose these vulnerabilities are searched by a genetic algorithm and, to help the genetic algorithm escape local optima, symbolic constraints are collected at run-time and passed to a solver. Search results represent test cases to be used by software developers to understand and fix security problems. We implemented this approach in a prototype and evaluated it on real world PHP code.

Often, legacy data management systems provide no native support to transactions. Programmers protect data from concurrent access by adopting commonly agreed patterns, relying on low level concurrency primitives, such as semaphores. In such cases, consistent data access is granted only if all code components are compliant with the adopted mutual exclusion patterns. When migrating legacy systems to modern data management systems, the ad hoc mechanisms for data protection must be replaced with modern constructs for transaction management. In such cases, a literal translation may expose problems and bugs, which were originally masked by the specific implementation and patterns in use. In this paper, we propose a static flow analysis that determines the existence of potentially incompatible locks within the same thread, which require specific code re-engineering before migrating to a modern data management system. We report the results obtained on a concrete instance of this problem.

Abstract Legacy information systems, such as banking systems, are usually organized around their data model. Hence, when these systems are migrated to modern environments, translation of the data model involves the most critical decisions, having strong implications on the rest of the translation. In this paper, we report our experience and describe the approaches adopted in migrating a large banking system (ten million lines of code) to Java, starting from a proprietary data model which gives programmers explicit control of the variable overlay in memory. After presenting the basic translation scheme, we discuss the exceptions that may occur in practice. Then, we consider two heuristic approaches useful to reduce the number of cases where a behavior equivalent to that of unions must be reproduced in Java. Finally, we comment on the experimental results obtained so far. Copyright © 2009 John Wiley & Sons, Ltd.

In recent years, several design notations have been proposed to model domain-specific applications or reference architectures. In particular, Conallen has proposed the UML Web Application Extension (WAE): a UML extension to model Web applications. The aim of our empirical investigation is to test whether the usage of the Conallen notation supports comprehension and maintenance activities with significant benefits, and whether such benefits depend on developers ability and experience. This paper reports and discusses the results of a series of four experiments performed in different locations and with subjects possessing different experience-namely, undergraduate students, graduate students, and research associates-and different ability levels. The experiments aim at comparing performances of subjects in comprehension tasks where they have the source code complemented either by standard UML diagrams or by diagrams stereotyped using the Conallen notation. Results indicate that, although, in general, it is not possible to observe any significant benefit associated with the usage of stereotyped diagrams, the availability of stereotypes reduces the gap between subjects with low skill or experience and highly skilled or experienced subjects. Results suggest that organizations employing developers with low experience can achieve a significant performance improvement by adopting stereotyped UML diagrams for Web applications.

Source code obfuscation is a protection mechanism widely used to limit the possibility of malicious reverse engineering or attack activities on a software system. Although several code obfuscation techniques and tools are available, little knowledge is available about the capability of obfuscation to reduce attackers' efficiency, and the contexts in which such an efficiencymay vary.

Test-driven software development tackles the problem of operationally defining the features to be implemented by means of test cases. This approach was recently ported to the early development phase, when requirements are gathered and clarified. Among the existing proposals, Fit (Framework for Integrated Testing) supports the precise specification of requirements by means of so called Fit tables, which express relevant usage scenarios in a tabular format, easily understood also by the customer. Fit tables can be turned into executable test cases through the creation of pieces of glue code, called fixtures.

The remote trust problem aims to address the issue of verifying the execution of a program running on an un-trusted host which communicates regularly with a trusted server. One proposed solution to this problem relies on a centralized scheme using assertions and replication to withhold usable services from a tampered client. We show how to extend such a scheme to a distributed trusted hardware such as tamper-resistant smartcards. We compared the performance and security of the proposed distributed system to the original centralized scheme on a case study. Our results indicate that, compared to a centralized scheme, our distributed trust scheme has dramatically lower network traffic, and smaller memory and computational requirements on the trusted server.

An appropriate translation of the data model is central to any language migration effort. Finding a mapping between original and target data models may be challenging for legacy languages (e.g., Assembly) which lack a structured data model and rely instead on explicit programmer control of the overlay of variables. Before legacy applications written in languages with an unstructured data model can be migrated to modern languages, a structured data model must be inferred. This paper describes a set of source transformations used to create such a model as part of a migration of eight million lines of code to Java. The original application is written in a proprietary language supporting variable layout by memory relocation.

Proponents of design notations tailored for specific application domains or reference architectures, often available in the form of UML stereotypes, motivate them by improved understandability and modifiability. However, empirical studies that tested such claims report contradictory results, where the most intuitive notations are not always the best performing ones. This indicates the possible existence of relevant influencing factors, other than the design notation itself. In this work we report the results of a family of three experiments performed at different locations and with different subjects, in which we assessed the effectiveness of UML stereotypes for Web design in support to comprehension tasks. Replications with different subjects allowed us to investigate whether subjects' ability and experience play any role in the comprehension of stereotyped diagrams. We observed different behaviors of users with different degrees of ability and experience, which suggests alternative comprehension strategies of (and tool support for) different categories of users.

Aspect oriented programming (AOP) is a new programming paradigm that offers a novel modularization unit for the crosscutting concerns. Functionalities originally spread across several modules and tangled with each other can be factored out into a single, separate unit, called an aspect. We investigated automated techniques that can be used to support the migration of existing Object Oriented Programming (OOP) code to AOP. To migrate an application to the new paradigm, a preliminary identification of the cross-cutting concerns is required (aspect mining). Then refactoring is applied to transform the scattered concerns into aspects. The proposed methods have been assessed on case studies for a total of more than half a million lines of code.

Abstract Web application design requires the modeling of multiple, separate concerns, such as the navigational structure, the business logic and the data persistence. To this aim, several methodologies have been conceived. One of them, the Web Application Extension (WAE), extends the UML notation by means of stereotypes and tagged values intended to capture Web‐specific concepts (e.g., the navigational structure). Although the WAE methodology is nowadays quite mature and ready for industrial adoption, the question whether it is able to actually facilitate the task of developers and maintainers has still to be empirically investigated. This paper reports and discusses the results from a controlled experiment on the benefits associated with the use of the WAE notation in the execution of comprehension tasks, carried out before maintenance. The WAE notation was compared against the use of pure unified modified language. Results indicate that the use of the WAE notation significantly improves the level of comprehension, although it does not increase the time needed to perform the comprehension task in a significant way. Copyright © 2007 John Wiley & Sons, Ltd.

Aspect-oriented programming (AOP) provides mechanisms for the separation of crosscutting concerns - functionalities scattered through the system and tangled with the base code. Existing systems are a natural testbed for the AOP approach since they often contain several crosscutting concerns which could not be modularized using traditional programming constructs. This paper presents an automated approach to the problem of migrating systems developed according to the object-oriented programming (OOP) paradigm into aspect-oriented programming (AOP). A simple set of six refactorings has been defined to transform OOP to AOP and has been implemented in the AOP-migrator tool, an Eclipse plug-in. A set of enabling transformations from OOP to OOP complement the initial set of refactorings. The paper presents the results of four case studies, which use the approach to migrate selected crosscutting concerns from medium-sized Java programs (in the range of 10K to 40K lines of code) into equivalent programs in AspectJ. The case study results show the feasibility of the migration and indicate the importance of the enabling transformations as a preprocessing step

This paper presents a human-guided automated approach to refactoring object oriented programs to the aspect oriented paradigm. The approach is based upon the iterative application of four steps: discovery, enabling, selection, and refactoring. After discovering potentially applicable refactorings, the enabling step transforms the code to improve refactorability. During the selection phase the particular refactorings to apply are chosen. Finally, the refactoring phase transforms the code by moving the selected code to a new aspect. This paper presents the results of an evaluation in which one of the crosscutting concerns of a 40,000 LoC program (JHotDraw) is refactored.

Aspect oriented programming aims at addressing the problem of the crosscutting concerns, i.e., those functionalities that are scattered among several modules in a given system. Aspects can be defined to modularize such concerns. In this work, we focus on a specific kind of crosscutting concerns, the scattered implementation of methods declared by interfaces that do not belong to the principal decomposition. We call such interfaces aspectizable. All the aspectizable interfaces identified within a large number of classes from the Java Standard Library and from three Java applications have been automatically migrated to aspects. To assess the effects of the migration on the internal and external quality attributes of these systems, we collected a set of metrics and we conducted an empirical study, in which some maintenance tasks were executed on the two alternative versions (with and without aspects) of the same system. In this paper, we report the results of such a comparison.

Separation of concerns and modularization are the cornerstones of software engineering. However, when a system is decomposed into units, functionalities often emerge which cannot be assigned to a single element of the decomposition. The implementation of interfaces represents a typical instance of this problem. In fact, the code that defines the interface methods is often scattered across several classes in the system and tangled with the original code. Aspect oriented programming provides mechanisms for the dynamic and static composition of transversal functionalities, that can be used to factor out the implementation of interfaces. We describe a technique for the identification of those interface implementations that are most likely to represent crosscutting concerns. Moreover, the code transformation (refactoring) to migrate such interfaces to aspects is also presented. Experimental results validate the approach.