Detecting vulnerabilities is vital for software security, yet deep learning-based vulnerability detectors (DLVD) face a data shortage, which limits their effectiveness. Data augmentation can potentially alleviate the data shortage, but augmenting vulnerable code is challenging and requires a generative solution that maintains vulnerability. Previous works have only focused on generating samples that contain single statements or specific types of vulnerabilities. Recently, large language models (LLMs) have been used to solve various code generation and comprehension tasks with inspiring results, especially when fused with retrieval augmented generation (RAG). Therefore, we propose VulScribeR , a novel LLM-based solution that leverages carefully curated prompt templates to augment vulnerable datasets. More specifically, we explore three strategies to augment both single and multi-statement vulnerabilities, with LLMs, namely Mutation, Injection, and Extension. Our extensive evaluation across four vulnerability datasets and DLVD models, using three LLMs, show that our approach beats two SOTA methods VulGen and VGX, and Random Oversampling (ROS) by 27.48%, 27.93%, and 15.41% in F1-score with 5K generated vulnerable samples on average, and 53.84%, 54.10%, 69.90%, and 40.93% with 15K generated vulnerable samples. Our approach demonstrates its feasibility for large-scale data augmentation by generating 1K samples at as cheap as US$1.88.

In the last decade, Android applications have emerged as a primary interface in consumer technology. With approximately 2.5 billion mobile devices running Android globally, security threats to the Android ecosystem due to vulnerabilities in it become increasingly broadly consequential via user applications (i.e., Android apps). This necessitates efficient methods for defending them against those vulnerabilities. Taint analysis, a popular and fundamental security defense technique, assesses the flow of sensitive information within an app between sources (e.g., reading from user inputs) and sinks (e.g., writing to databases). However, traditional taint analysis is notably resource-intensive. Performing a comprehensive analysis on a single app given a complete list of potential sources and sinks can take hours, a situation exacerbated by the frequent updates typical in mobile app development. In this article, we propose EvoTaint , an incremental taint analysis , tailored to fit and exploit the evolving nature of Android apps. It aims to substantially reduce the time cost of conventional static taint analysis against an evolved version of a given app by narrowing down the analysis scope from the entire app to only the parts that are changed or impacted by the changes in the evolved version. We have implemented EvoTaint as a practical, open-source tool and evaluated it on 100 Android apps each with 2, 3, or even 5 versions considered. Our results demonstrated a significant (51.8–68.9%) reduction in the time cost of static taint analysis of each of the 1–4 evolved versions on average, without compromising the accuracy of the analysis results (i.e., taint flow paths), compared to using the conventional approach treating each version as a separate/standalone app. Our further analysis aimed to clarify why and when EvoTaint performs favorably. It revealed that the time efficiency gains of incremental taint analysis are strongly correlated with the ratio of changed methods and the proportion of sources/sinks affected by these changes during app evolution.

The Android ecosystem has experienced rapid growth, resulting in a diverse range of platforms and devices. This expansion has also brought about compatibility issues that negatively impact user experiences and hinder app development productivity. Existing relevant studies are focused on and limited to the “static” sense of those issues (in terms of potentialities and proneness), while only addressing compatibility issues that possibly occur during app executions. In this article, we present an extensive and longitudinal study on app compatibility issues that are disparate from yet complementary to prior studies, characterizing the incompatibilities based on actual , exercised observations and evidence at both installation and run-time. With a dataset of 74,545 benign apps and 56,919 malicious apps over a span of 12 years (2010 through 2021) and 10 Android versions, we extensively examine the prevalence and symptoms/effects and causes of, as well as the contributing factors to, installation-time and run-time compatibility issues. Our study reveals 12 major novel findings regarding Android app incompatibilities. First (Findings 1, 2), installation-time incompatibilities persisted significantly over the 12 years, even more so in malware than benign apps. Second (Findings 7, 8), run-time compatibility issues were also seen persistently over time but only on specific Android platforms (such as API 26,27, etc.) and much less by malware than benign apps. Third (Findings 5, 6, 11, 12), there is a significant (moderate/stronger) correlation between an app’s specified minSdkVersion and its incompatibilities (over all symptoms and/or with respect to one of its dominating symptom), with stronger correlations seen in malware than in benign apps, for both installation-time and run-time incompatibilities. Similar observations hold (although with much stronger correlation in absolute terms) when considering, instead of the minSdkVersion itself, the gap between the app’s minSdkVersion and the SDK API level of the platform the app is installed to or runs on. Last (Findings 3, 4, 9, 10), installation-time incompatibilities are primarily caused by the utilization of architecture-incompatible native libraries within apps, while run-time incompatibilities are mainly attributed to API changes during the evolution of the Android SDK; the symptoms of run-time failures seen by malware are much more diverse than by benign apps. In addition to these insights, we provide practical recommendations for both app developers and end users on how to effectively address compatibility issues in Android apps, as well as how to devise effective defenses against malware from the compatibility perspectives.

Mobile apps are known to distribute different versions across geographic regions to accommodate local regulations and market preferences. While prior research has examined metadata-level differences such as permissions and privacy policies, there lacks systematic investigation into code-level geographic variations that may impact security. In this paper, we present the first comprehensive study of geo-feature differences (GFDs) in Android apps at the code implementation level. We develop Freelens, a novel framework that overcomes key technical challenges including code obfuscation and analysis scalability to identify and characterize security-relevant variations across regions. Using Freelens, we conducted a large-scale study of 21,120 Android apps distributed across ten countries with diverse levels of Internet freedom. Our findings reveal that GFDs are widespread, with significant variations in advertising, data handling, and authentication mechanisms. These differences frequently compromise security baselines and introduce disparities in privacy protections across regions. The study highlights a rising trend in GFD prevalence, emphasizing the urgency for harmonized privacy and security standards. Based on our empirical findings, we also provide actionable insights for developers, platform providers, and regulators to ensure equitable user protections.

Artificial intelligence and recent advances in deep learning architectures, including transformer networks and large language models, change the way people think and act to solve problems. Software engineering, as an increasingly complex process to design, develop, test, deploy, and maintain large-scale software systems for solving real-world challenges, is profoundly affected by many revolutionary artificial intelligence tools in general and machine learning in particular. In this roadmap for artificial intelligence in software engineering, we highlight the recent deep impact of artificial intelligence on software engineering by discussing successful stories of applications of artificial intelligence to classic and new software development challenges. We identify the new challenges that the software engineering community has to address in the coming years to successfully apply artificial intelligence in software engineering, and we share our research roadmap toward the effective use of artificial intelligence in the software engineering profession, while still protecting fundamental human values. We spotlight three main areas that challenge the research in software engineering: the use of generative artificial intelligence and large language models for engineering large software systems, the need of large and unbiased datasets and benchmarks for training and evaluating deep learning and large language models for software engineering, and the need of a new code of digital ethics to apply artificial intelligence in software engineering.

A defining, unique aspect of distributed systems lies in interprocess communication (IPC) through which distributed components interact and collaborate toward the holistic system behaviors. This highly decoupled construction intuitively contributes to the scalability, performance, and resiliency advantages of distributed software, but also adds largely to their greater complexity, compared to centralized software. Yet despite the importance of IPC in distributed systems, little is known about how to quantify IPC-induced behaviors in these systems through IPC measurement and how such behaviors may be related to the quality of distributed software . To answer these questions, in this article, we present DistMeasure , a framework for measuring distributed software systems via the lens of IPC hence enabling the study of its correlation with distributed system quality. Underlying DistMeasure is a novel set of IPC metrics that focus on gauging the coupling and cohesion of distributed processes. Through these metrics, DistMeasure quantifies relevant runtime characteristics of distributed systems and their quality relevance, covering a range of quality aspects each via respective direct quality metrics. Further, DistMeasure enables predictive assessment of distributed system quality in those aspects via learning-based anomaly detection with respect to the corresponding quality metrics based on their significant correlations with related IPC metrics. Using DistMeasure , we demonstrated the practicality and usefulness of IPC measurement against 11 real-world distributed systems and their diverse execution scenarios. Among other findings, our results revealed that IPC has a strong correlation with distributed system complexity, performance efficiency, and security. Higher IPC coupling between distributed processes tended to be negatively indicative of distributed software quality, while more cohesive processes have positive quality implications. Yet overall IPC-induced behaviors are largely independent of the system scale, and higher (lower) process coupling does not necessarily come with lower (higher) process cohesion. We also show promising merits (with 98% precision/recall/F1) of IPC measurement (e.g., class-level coupling and process-level cohesion) for predictive anomaly assessment of various aspects (e.g., attack surface and performance efficiency) of distributed system quality.

With the rise of large language models, such as ChatGPT, non-decisional models have been applied to various tasks. Moreover, ChatGPT has drawn attention to the traditional decision-centric task of Android malware detection. Despite effective detection methods proposed by scholars, they face low interpretability issues. Specifically, while these methods excel in classifying applications as benign or malicious and can detect malicious behavior, they often fail to provide detailed explanations for the decisions they make. This challenge raises concerns about the reliability of existing detection schemes and questions their true ability to understand complex data. In this study, we investigate the influence of the non-decisional model, ChatGPT, on the traditional decision-centric task of Android malware detection. We choose three state-of-the-art solutions, Drebin , \(XM_{AL}\) , and MaMaDroid , conduct a series of experiments on publicly available datasets, and carry out a comprehensive comparison and analysis. Our findings indicate that these decision-driven solutions primarily rely on statistical patterns within datasets to make decisions, rather than genuinely understanding the underlying data. In contrast, ChatGPT, as a non-decisional model, excels in providing comprehensive analysis reports, substantially enhancing interpretability. Furthermore, we conduct surveys among experienced developers. The result highlights developers’ preference for ChatGPT, as it offers in-depth insights and enhances efficiency and understanding of challenges. Meanwhile, these studies and analyses offer profound insights, presenting developers with a novel perspective on Android malware detection—enhancing the reliability of detection results from a non-decisional perspective.

With the emergence of the open API ecosystem, third-party developers can publish their APIs on the API marketplace, significantly facilitating the development of cutting-edge features and services. The RapidAPI platform is currently the largest API marketplace and it provides over 40,000 APIs, which have been used by more than 4 million developers. However, such open API also raises security and privacy concerns associated with APIs hosted on the platform. In this work, we perform the first large-scale analysis of 32,089 APIs on the RapidAPI platform. By searching in the GitHub code and Android apps, we find that 3,533 RapidAPI keys, which are important and used in API request authorization, have been leaked in the wild. These keys can be exploited to launch various attacks, such as Resource Exhaustion Running, Theft of Service, Data Manipulation, and User Data Breach attacks. We also explore risks in API metadata that can be abused by adversaries. Due to the lack of a strict certification system, adversaries can manipulate the API metadata to perform typosquatting attacks on API URLs, impersonate other developers or renowned companies, and publish spamming APIs on the platform. Lastly, we analyze the privacy non-compliance of APIs and applications, e.g., Android apps, that call these APIs with data collection. We find that 1,709 APIs collect sensitive data and 94% of them dont provide a complete privacy policy. For the Android apps that call these APIs, 50% of them in our study have privacy non-compliance issues.

Accompanying the successes of learning-based defensive software vulnerability analyses is the lack of large and quality sets of labeled vulnerable program samples, which impedes further advancement of those defenses. Existing automated sample generation approaches have shown potentials yet still fall short of practical expectations due to the high noise in the generated samples. This paper proposes VGX, a new technique aimed for large-scale generation of high-quality vulnerability datasets. Given a normal program, VGX identifies the code contexts in which vulnerabilities can be injected, using a customized Transformer featured with a new value-flow-based position encoding and pre-trained against new objectives particularly for learning code structure and context. Then, VGX materializes vulnerability-injection code editing in the identified contexts using patterns of such edits obtained from both historical fixes and human knowledge about real-world vulnerabilities.

Fuzzing is one of the major techniques for uncovering vulnerabilities in smart contracts. The effectiveness of fuzzing is significantly affected by its throughput but unfortunately existing fuzzers for smart contracts have low throughput due to the slow execution of EVM, the delay introduced by the consensus protocols, the limited parallelization capability of CPUs, and the overhead caused by the instrumented EVM. To tackle this critical issue, in this paper, we take the first step to leverage GPU’s parallel computing power to boost the throughput of smart contract fuzzing. More precisely, by converting the fuzzing workload to a SIMD task, we can activate thousands of GPU cores to test the smart contract simultaneously. To achieve this purpose, we design new solutions to address three major challenges, namely developing incremental storage to reduce GPU memory cost, proposing a stateful bitmap to embed transaction dependency to the feedback metric, and designing a parallel feedback algorithm to rule out undesired seeds that cause redundant overlaps. We implement a prototype named Mau, which first transforms the bytecode of a smart contract to a SIMD application in PTX assembly and then runs it parallelly on the GPU. We evaluate Mau using both a large and small benchmark. The experimental results demonstrate that the throughput of Mau reaches 162.37K execs/sec and 328.06K execs/sec, which leads to an 8.69-15.38X improvement to the state-of-the-art tool. Moreover, the high throughput empowers Mau to detect 1.01-2.50X more bugs and obtain 1.03–4.71X more code coverage than baselines.

Amazon Alexa is one of the largest Voice Personal Assistant (VPA) platforms and it allows third-party developers to publish their voice apps, named skills, to the Alexa skill store. To satisfy the needs of European users, Amazon Alexa has established multiple skill marketplaces in Europe and allows developers to publish skills in their native languages. Skills in European marketplaces are required to comply with GDPR (General Data Protection Regulation), which imposes strict obligations on data collection and processing. Skills that involve data collection should provide a privacy policy to disclose the data practice to users and meet GDPR requirements.

For many years now, modern software is known to be developed in multiple languages (hence termed as multilingual or multi-language software). Yet, to date, we still only have very limited knowledge about how multilingual software systems are constructed. For instance, it is not yet really clear how different languages are used, selected together, and why they have been so in multilingual software development. Given the fact that using multiple languages in a single software project has become a norm, understanding language use and selection (i.e., language profile ) as a basic element of the multilingual construction in contemporary software engineering is an essential first step. In this article, we set out to fill this gap with a large-scale characterization study on language use and selection in open-source multilingual software. We start with presenting an updated overview of language use in 7,113 GitHub projects spanning the 5 past years by characterizing overall statistics of language profiles, followed by a deeper look into the functionality relevance/justification of language selection in these projects through association rule mining. We proceed with an evolutionary characterization of 1,000 GitHub projects for each of the 10 past years to provide a longitudinal view of how language use and selection have changed over the years, as well as how the association between functionality and language selection has been evolving. Among many other findings, our study revealed a growing trend of using three to five languages in one multilingual software project and the noticeable stableness of top language selections. We found a non-trivial association between language selection and certain functionality domains, which was less stable than that with individual languages over time. In a historical context, we also have observed major shifts in these characteristics of multilingual systems both in contrast to earlier peer studies and along the evolutionary timeline. Our findings offer essential knowledge on the multilingual construction in modern software development. Based on our results, we also provide insights and actionable suggestions for both researchers and developers of multilingual systems.

With the emergence of smartphones, Android has become a widely used mobile operating system. However, it is vulnerable when encountering various types of attacks. Every day, new malware threatens the security of users’ devices and private data. Many methods have been proposed to classify malicious applications, utilizing static or dynamic analysis for classification. However, previous methods still suffer from unsatisfactory performance due to two challenges. First, they are unable to address the imbalanced data distribution problem, leading to poor performance for malware families with few members. Second, they are unable to address the zero-day malware (zero-day malware refers to malicious applications that exploit unknown vulnerabilities) classification problem. In this article, we introduce an innovative meta -learning approach for m ulti-family A ndroid m alware c lassification named Meta-MAMC , which uses meta-learning technology to learn meta-knowledge (i.e., the similarities and differences among different malware families) of few-family samples and combines new sampling algorithms to solve the above challenges. Meta-MAMC integrates (i) the meta-knowledge contained within the dataset to guide models in learning to identify unknown malware; and (ii) more accurate and diverse tasks based on novel sampling strategies, as well as directly adapting meta-learning to a new few-sample and zero-sample task to classify families. We have evaluated Meta-MAMC on two popular datasets and a corpus of real-world Android applications. The results demonstrate its efficacy in accurately classifying malicious applications belonging to certain malware families, even achieving 100% classification in some families.

Fragmentation is a serious problem in the Android ecosystem, which is mainly caused by the fast evolution of the system itself and the various system customizations. Many efforts have attempted to mitigate its impact via approaches to automatically pinpointing compatibility issues in Android apps. We conducted a literature review to identify all the currently available approaches to addressing this issue. Within the nine identified approaches, the four issue detection tools and one incompatible API harvesting tool could be successfully executed. We tried to reproduce them based on their original datasets and then empirically compared those approaches against common datasets. Our experimental results show that existing tool capabilities are quite distinct with only a small overlap in the compatibility issues being identified. Moreover, these detection tools commonly detect compatibility issues via two separate steps including incompatible APIs gathering and compatibility issues (induced by the incorrect invocations of the identified incompatible APIs) determination. To help developers better identify compatibility issues in Android apps, we developed a new approach, AndroMevol , to systematically spot incompatible APIs as they play a crucial role in issue detection. AndroMevol was able to pinpoint 397,678 incompatible APIs against the full history of the official Android framework and 52 customized Android frameworks spanning five popular device manufacturers. Our approach could enhance the ability of the state-of-the-art detection tools by identifying many more incompatible APIs that may cause compatibility issues in Android apps and foster more advanced approaches to pinpointing all types of compatibility issues.

The prosperity of software applications brings fierce market competition to developers. Employing third-party libraries (TPLs) to add new features to projects under development and to reduce the time to market has become a popular way in the community. However, given the tremendous TPLs ready for use, it is challenging for developers to effectively and efficiently identify the most suitable TPLs. To tackle this obstacle, we propose an innovative approach named PyRec to recommend potentially useful TPLs to developers for their projects. Taking Python project development as a use case, PyRec embeds Python projects, TPLs, contextual information, and relations between those entities into a knowledge graph. Then, it employs a graph neural network to capture useful information from the graph to make TPL recommendations. Different from existing approaches, PyRec can make full use of not only project-library interaction information but also contextual information to make more accurate TPL recommendations. Comprehensive evaluations are conducted based on 12,421 Python projects involving 963 TPLs, 9,675 extra entities, 121,474 library usage records, and 73,277 contextual records. Compared with five representative approaches, PyRec improves the recommendation performance significantly in all cases.

Developing software projects that incorporate multiple languages has been a prevalent practice for many years. However, theissuesencountered by developers during the development process, the underlyingchallengescausing these issues, and thesolutionsprovided to developers remain unknown. In this paper, our objective is to provide answers to these questions by conducting a study on developer discussions on Stack Overflow (SO). Through a manual analysis of 586 highly relevant posts spanning 14 years, we revealed that multilingual development is a highly and sustainably active topic on SO, with older questions becoming inactive and newer ones getting first asked (and then mostly remaining active for more than one year). From these posts, we observed a diverse array of issues (11 categories), primarily centered around interfacing and data handling across different languages. Our analysis suggests that error/exception handling issues were the most difficult to resolve among those issue categories, while security related issues were most likely to receive an accepted answer. The primary challenge faced by developers was the complexity and diversity inherent in building multilingual code and ensuring interoperability. Additionally, developers often struggled due to a lack of technical expertise on the varied features of different programming languages (e.g., threading and memory management mechanisms). In addition, properly handling message passing across languages constituted a key challenge with using implicit language interfacing. Notably, Stack Overflow emerged as a crucial source of solutions to these challenges, with the majority (73%) of the posts receiving accepted answers, most within a week (36.5% within 24 hours and 25% in the following six days). Based on our analysis results, we have formulated actionable insights and recommendations that can be utilized by researchers and developers in this field.

Given the widespread use of Python and its sustaining impact, the security and reliability of the Python runtime system is highly and broadly critical. Yet with real-world bugs in Python runtimes being continuously and increasingly reported, technique/tool support for automated detection of such bugs is still largely lacking. In this paper, we present PyRTFuzz, a novel fuzzing technique/tool for holistically testing Python runtimes including the language interpreter and its runtime libraries. PyRTFuzz combines generationand mutation-based fuzzing at the compiler- and application-testing level, respectively, as enabled by static/dynamic analysis for extracting runtime API descriptions, a declarative, specification language for valid and diverse Python code generation, and a custom type-guided mutation strategy for format/structure-aware application input generation. We implemented PyRTFuzz for the primary Python implementation (CPython) and applied it to three versions of the runtime. Our experiments revealed 61 new, demonstrably exploitable bugs including those in the interpreter and most in the runtime libraries. Our results also demonstrated the promising scalability and cost-effectiveness of PyRTFuzz and its great potential for further bug discovery. The two-level collaborative fuzzing methodology instantiated in PyRTFuzz may also apply to other language runtimes especially those of interpreted languages.

The Amazon Alexa marketplace is the largest Voice Personal Assistant (VPA) platform with over 100,000 voice applications (i.e., skills) published to the skills store. In an effort to maintain the quality and trustworthiness of voice-apps, Amazon Alexa has implemented a set of policy requirements to be adhered to by third-party skill developers. However, recent works reveal the prevalence of policy-violating skills in the current skills store. To understand the causes of policy violations in skills, we first conduct a user study with 34 third-party skill developers focusing on whether they are aware of the various policy requirements defined by the Amazon Alexa platform. Our user study results show that there is a notable gap between VPA's policy requirements and skill developers' practices. As a result, it is inevitable that policy-violating skills will be published.

WebAssembly (Wasm) smart contracts have shown growing popularity across blockchains (e.g., EOSIO and NEAR) recently. Wasm smart contracts have been suffering from various attacks exploiting their vulnerabilities. Even worse, few developers released the source code of their Wasm smart contracts for security review, raising the bar for uncovering vulnerable contracts. Although a few approaches have been proposed to detect vulnerable Wasm smart contracts, they have several major limitations, e.g., low code coverage, low accuracy and lack of scalability, unable to produce exploit payloads, etc. To fill the gap, we design and implement WASAI,a new concolic fuzzer for uncovering vulnerabilities in Wasm smart contract. We conduct extensive experiments to evaluate WASAI,and the results show that it outperforms the state-of-the-art methods. WASAI achieves 2x code coverage than the baselines and surpasses them in detection accuracy, with an F1-measure of 99.2%. Applying WASAI to all deployed smart contracts in the wild, we find that over 707 smart contracts are vulnerable. One Fake EOS vulnerability reported to the EOSIO ecosystem was recently assigned a CVE identifier (CVE-2022-27134).

Building new, powerful data-driven defenses against prevalent software vulnerabilities needs sizable, quality vulnerability datasets, so does large-scale benchmarking of existing defense solutions. Automatic data generation would promisingly meet the need, yet there is little work aimed to generate much-needed quality vulnerable samples. Meanwhile, existing similar and adaptable techniques suffer critical limitations for that purpose. In this paper, we present VULGEN, the first injection-based vulnerability-generation technique that is not limited to a particular class of vulnerabilities. VULGEN combines the strengths of deterministic (pattern-based) and probabilistic (deep-learning/DL-based) program transformation approaches while mutually overcoming respective weaknesses. This is achieved through close collaborations between pattern mining/application and DL-based injection localization, which separates the concerns with how and where to inject. By leveraging large, pretrained programming language modeling and only learning locations, VULGEN mitigates its own needs for quality vulnerability data (for training the localization model). Extensive evaluations show that VULGEN significantly outperforms a state-of-the-art (SOTA) pattern-based peer technique as well as both Transformer- and GNN-based approaches in terms of the percentages of generated samples that are vulnerable and those also exactly matching the ground truth (by 38.0-430.1% and 16.3-158.2%, respectively). The VULGEN-generated samples led to substantial performance improvements for two SOTA DL-based vulnerability detectors (by up to 31.8% higher in F1), close to those brought by the ground-truth real-world samples and much higher than those by the same numbers of existing synthetic samples.

Developing a software project using multiple languages together has been a dominant practice for years. Yet it remains unclear what issues developers encounter during the development, which challenges cause the issues, and what solutions developers receive. In this paper, we aim to answer these questions via a study on developer discussions on Stack Overflow. By manually analyzing 586 highly relevant posts spanning 14 years, we observed a large variety (11 categories) of issues, dominated by those with interfacing and data handling among different languages. Behind these issues, we found that a major challenge developers faced is the diversity and complexity in multilingual code building and interoperability. Another key challenge lies in developers' lack of particular technical background on the diverse features of various languages (e.g., threading and memory management mechanisms). Meanwhile, Stack Overflow itself served as a key source of solutions to these challenges-the majority (73%) of the posts received accepted answers eventually, and most in a week (36.5% within 24 hours and 25% in the next 6 days). Based on our findings on these issues, challenges, and solutions, we provide actionable insights and suggestions for both multi-language software researchers and developers.

Security of Android devices is now paramount, given their wide adoption among consumers. As researchers develop tools for statically or dynamically detecting suspicious apps, malware writers regularly update their attack mechanisms to hide malicious behavior implementation. This poses two problems to current research techniques: static analysis approaches, given their over-approximations, can report an overwhelming number of false alarms, while dynamic approaches will miss those behaviors that are hidden through evasion techniques. We propose in this work a static approach specifically targeted at highlighting hidden sensitive operations (HSOs), mainly sensitive data flows. The prototype version of HiSenDroid has been evaluated on a large-scale dataset of thousands of malware and goodware samples on which it successfully revealed anti-analysis code snippets aiming at evading detection by dynamic analysis. We further experimentally show that, with FlowDroid, some of the hidden sensitive behaviors would eventually lead to private data leaks. Those leaks would have been hard to spot either manually among the large number of false positives reported by the state-of-the-art static analyzers, or by dynamic tools. Overall, by putting the light on hidden sensitive operations, HiSenDroid helps security analysts in validating potentially sensitive data operations, which would be previously unnoticed.

Artificial intelligence (AI) for software engineering (SE) tasks has recently achieved promising performance. In this article, we investigate to what extent the pre-trained language model truly understands those SE tasks such as code search, code summarization, etc. We conduct a comprehensive empirical study on a board set of AI for SE (AI4SE) tasks by feeding them with variant inputs: 1) with various masking rates and 2) with sufficient input subset method. Then, the trained models are evaluated on different SE tasks, including code search, code summarization, and duplicate bug report detection. Our experimental results show that pre-trained language models are insensitive to the given input, thus they achieve similar performance in these three SE tasks. We refer to this phenomenon asoverinterpretation, where a model confidently makes a decision without salient features, or where a model finds some irrelevant relationships between the final decision and the dataset. Our study investigates two approaches to mitigate the overinterpretation phenomenon: whole word mask strategy and ensembling. To the best of our knowledge, we are thefirstto reveal this overinterpretation phenomenon to the AI4SE community, which is an important reminder for researchers to design the input for the models and calls for necessary future work in understanding and implementing AI4SE tasks.

Open science is a practice that makes scientific research publicly accessible to anyone, hence is highly beneficial. Given the benefits, the software engineering (SE) community has been diligently advocating open science policies during peer reviews and publication processes. However, to this date, there has been few studies that look into the status and issues of open science in SE from a systematic perspective. In this paper, we set out to start filling this gap. Given the great breadth of SE in general, we constrained our scope to a particular topic area in SE as an example case. Recently, an increasing number of deep learning (DL) approaches have been explored in SE, includingDL-based software vulnerability detection, a popular, fast-growing topic that addresses an important problem in software security. We exhaustively searched the literature in this area and identified 55 relevant works that propose a DL-based vulnerability detection approach. This was then followed by comprehensively investigating the four integral aspects of open science:availability,executability,reproducibility, andreplicability. Among other findings, our study revealed that only a small percentage (25.5%) of the studied approaches provided publiclyavailabletools. Some of these available tools did not provide sufficient documentation and complete implementation, making them notexecutableor notreproducible. The uses of balanced or artificially generated datasets caused significantly overrated performance of the respective techniques, making most of them notreplicable. Based on our empirical results, we made actionable suggestions on improving the state of open science in each of the four aspects. We note that our results and recommendations on most of these aspects (availability,executability,reproducibility) are not tied to the nature of the chosen topic (DL-based vulnerability detection) hence are likely applicable to other SE topic areas. We also believe our results and recommendations onreplicabilityto be applicable to other DL-based topics in SE as they are not tied to (the particular application of DL in) detecting software vulnerabilities.

To prevent unauthorized apps from retrieving the sensitive data, Android framework enforces a permission based access control. However, it has long been known that, to bypass the access control, unauthorized apps can intercept the Intent objects which are sent by authorized apps and carry the retrieved sensitive data. We find that there is a new (previously unknown) attack surface in Android framework that can be exploited by unauthorized apps to violate the access control. Specifically, we discover that part of Intent objects that are sent by Android framework and carry sensitive data can be received by unauthorized apps, resulting in the leak of sensitive data. In this paper, we conduct the first systematic investigation on the new attack surface namely the Intent based leak of sensitive data in Android framework. To automatically uncover such kind of vulnerability in Android framework, we design and develop a new tool named LeakDetector, which finds the Intent objects sent by Android framework that can be received by unauthorized apps and carry the sensitive data. Applying LeakDetector to 10 commercial Android systems, we find that it can effectively uncover the Intent based leak of sensitive data in Android framework. Specifically, we discover 36 exploitable cases of such kind of data leak, which can be abused by unauthorized apps to steal the sensitive data, violating the access control. At the time of writing, 16 of them have been confirmed by Google, Samsung, and Xiaomi, and we received bug bounty rewards from these mobile vendors.

WebAssembly (Wasm) smart contracts have shown growing popularity across blockchains (e.g., EOSIO) recently. Similar to Ethereum smart contracts, Wasm smart contracts suffer from various attacks exploiting their vulnerabilities. Even worse, few developers released the source code of their Wasm smart contracts for security review, raising the bar for uncovering vulnerable contracts. Although a few approaches have been proposed to detect vulnerable Wasm smart contracts, they have several major limitations, e.g., low code coverage, low accuracy and lack of scalability, unable to produce exploit payloads, etc. To fill the gap, in this paper, we design and develop WASAI, a new concolic fuzzer for uncovering vulnerabilities in Wasm smart contract after tackling several challenging issues. We conduct extensive experiments to evaluate WASAI, and the results show that it outperforms the state-of-the-art methods. For example, it achieves 2x code coverage than the baselines and surpasses them in detection accuracy, with an F1-measure of 99.2%. Moreover, WASAI can handle complicated contracts (e.g., contracts with obfuscation and sophisticated verification). Applying WASAI to 991 deployed smart contracts in the wild, we find that over 70% of smart contracts are vulnerable. By the time of this study, over 300 vulnerable contracts have not been patched and are still operating on the EOSIO Mainnet. One fake EOS vulnerability reported to the EOSIO ecosystem was recently assigned a CVE identifier (CVE-2022-27134).

Fragmentation is a serious problem in the Android ecosystem. This problem is mainly caused by the fast evolution of the system itself and the various customizations independently maintained by different smartphone manufacturers. Many efforts have attempted to mitigate its impact via approaches to automatically pinpoint compatibility issues in Android apps. Unfortunately, at this stage, it is still unknown if this objective has been fulfilled, and the existing approaches can indeed be replicated and reliably leveraged to pinpoint compatibility issues in the wild. We, therefore, propose to fill this gap by first conducting a literature review within this topic to identify all the available approaches. Among the nine identified approaches, we then try our best to reproduce them based on their original datasets. After that, we go one step further to empirically compare those approaches against common datasets with real-world apps containing compatibility issues. Experimental results show that existing tools can indeed be reproduced, but their capabilities are quite distinct, as confirmed by the fact that there is only a small overlap of the results reported by the selected tools. This evidence suggests that more efforts should be spent by our community to achieve sound compatibility issues detection.

More and more Android apps implement their functionalities in native code, so does malware. Although various approaches have been designed to analyze the native code used by apps, they usually generate incomplete and biased results due to their limitations in obtaining and analyzing high-fidelity execution traces and memory data with low overheads. To fill the gap, in this paper, we propose and develop a novel hardware-assisted analyzer for native code in apps. We leverage ETM, a hardware feature of ARM platform, and eBPF, a kernel component of Android system, to collect real execution traces and relevant memory data of target apps, and design new methods to scrutinize native code according to the collected data. To show the unique capability of NCScope, we apply it to four applications that cannot be accomplished by existing tools, including systematic studies on self-protection and anti-analysis mechanisms implemented in native code of apps, analysis of memory corruption in native code, and identification of performance differences between functions in native code. The results uncover that only 26.8% of the analyzed financial apps implement self-protection methods in native code, implying that the security of financial apps is far from expected. Meanwhile, 78.3% of the malicious apps under analysis have anti-analysis behaviors, suggesting that NCScope is very useful to malware analysis. Moreover, NCScope can effectively detect bugs in native code and identify performance differences.

Software construction using multiple languages has long been a norm, yet it is still unclear if multilingual code construction has significant security implications and real security consequences. This paper aims to address this question with a large-scale study of popular multi-language projects on GitHub and their evolution histories, enabled by our novel techniques for multilingual code characterization. We found statistically significant associations between the proneness of multilingual code to vulnerabilities (in general and of specific categories) and its language selection. We also found this association is correlated with that of the language interfacing mechanism, not that of individual languages. We validated our statistical findings with in-depth case studies on actual vulnerabilities, explained via the mechanism and language selection. Our results call for immediate actions to assess and defend against multilingual vulnerabilities, for which we provide practical recommendations.

Today’s software systems are mostly developed in multiple languages (i.e., multi-language software), yet tool support for understanding and assuring these systems is rare. To facilitate future research on multi-language software engineering, this paper presents PolyFax, a toolkit that offers automated means for dataset collection from GitHub and two analysis utilities--a vulnerability-fixing commit categorization tool (VCC) and a language interfacing mechanism identification/categorization tool (LIC). The VCC tool immediately assists with assessing the vulnerability proneness of a given multi-language project based on its version histories, while the LIC tool enables dissection of the most important aspect of the construction of multi-language systems. Application of PolyFax to 7,113 multi-language projects with 12.6 million commits showed its practical usefulness in terms of promising efficiency and accuracy for studying multi-language software.

The availability of large-scale, realistic vulnerability datasets is essential both for benchmarking existing techniques and for developing effective new data-driven approaches for software security. Yet such datasets are critically lacking. A promising solution is to generate such datasets by injecting vulnerabilities into real-world programs, which are richly available. Thus, in this paper, we explore the feasibility of vulnerability injection through neural code editing. With a synthetic dataset and a real-world one, we investigate the potential and gaps of three state-of-the-art neural code editors for vulnerability injection. We find that the studied editors have critical limitations on the real-world dataset, where the best accuracy is only 10.03%, versus 79.40% on the synthetic dataset. While the graph-based editors are more effective (successfully injecting vulnerabilities in up to 34.93% of real-world testing samples) than the sequence-based one (0 success), they still suffer from complex code structures and fall short for long edits due to their insufficient designs of the preprocessing and deep learning (DL) models. We reveal the promise of neural code editing for generating realistic vulnerable samples, as they help boost the effectiveness of DL-based vulnerability detectors by up to 49.51% in terms of F1 score. We also provide insights into the gaps in current editors (e.g., they are good at deleting but not at replacing code) and actionable suggestions for addressing them (e.g., designing effective editing primitives).

Analyzing multilingual code holistically is key to systematic quality assurance of real-world software which is mostly developed in multiple computer languages. Toward such analyses, state-of-the-art approaches propose an almost-fully language-agnostic methodology and apply it to dynamic dependence analysis/slicing of multilingual code, showing great promises. We investigated this methodology through a technical analysis followed by a replication study applying it to 10 real-world multilingual projects of diverse language combinations. Our results revealed critical practicality (i.e., having the levels of efficiency/scalability, precision, and extensibility to various language combinations for practical use) challenges to the methodology. Based on the results, we reflect on the underlying pitfalls of the language-agnostic design that leads to such challenges. Finally, looking forward to the prospects of dynamic analysis for multilingual code, we identify a new research direction towards better practicality and precision while not sacrificing extensibility much, as supported by preliminary results. The key takeaway is that pursuing fully language-agnostic analysis may be both impractical and unnecessary, and striving for a better balance between language independence and practicality may be more fruitful.

A play test is the process in which testers play video games for software quality assurance. Manual testing is expensive and time-consuming, especially when there are many mobile games to test and every game version requires extensive testing. Current testing frameworks (e.g., Android Monkey) are limited as they adopt no domain knowledge to play games. Learning-based tools (e.g., Wuji) require tremendous manual effort and ML expertise of developers. This paper presents LIT-a lightweight approach to generalize play test tactics from manual testing, and to adopt the tactics for automatic testing. Lit has two phases: tactic generalization and tactic concretization. In Phase I, when a human tester plays an Android game$G$for a while (e.g., eight minutes), Lit records the tester's inputs and related scenes. Based on the collected data, Lit infers a set of context-aware, abstract play test tactics that describe under what circumstances, what actions can be taken. In Phase II, LIttests$G$based on the generalized tactics. Namely, given a randomly generated game scene, Lit tentatively matches that scene with the abstract context of any inferred tactic; if the match succeeds, Lit customizes the tactic to generate an action for playtest. Our evaluation with nine games shows Lit to outperform two state-of-the-art tools and a reinforcement learning (RL)-based tool, by covering more code and triggering more errors. Lit complements existing tools and helps developers test various casual games (e.g., match3, shooting, and puzzles).

Cryptographic protocols are often expected to be provably secure. However, this security guarantee often falls short in practice due to various implementation flaws. We propose a new paradigm calledcryptographic program analysis (CPA)which prescribes the use of program analysis to detect these implementation flaws at compile time. The principal insight of the CPA is that many of these flaws in cryptographic implementations can be mapped to the violation ofmeta-level propertiesof implementations. A program property that is necessary to realize a cryptographic property is referred to as meta-level property. We show that violations of these meta-level properties can be identified at compile-time that can serve as sufficient evidence of the encompassing flaws. We investigated existing literature on cryptographic implementation flaws and derived 25 corresponding meta-level properties. To instantiate the abstract paradigm of CPA, we develop a specification language based on deterministic finite automaton (DFA) and show that most of the meta-level properties can be expressed in terms of our language. We then develop a tool calledTaintCryptwhich uses static taint analysis to identify meta-level property violations of C/C++ cryptographic implementations at compile-time. We demonstrate the efficacy ofTaintCryptby analyzing open-source C/C++ cryptographic libraries (e.g., OpenSSL) and observe thatTaintCryptcould have helped to avoid several high-profile flaws. We also evaluatedTaintCrypton 5 popular applications and libraries, which generated new security insights. The experimental evaluation on large-scale projects indicates the scalability of our approach.

As modern software systems are increasingly developed for running in distributed environments, it is crucial to provide fundamental techniques such as dependence analysis for checking, diagnosing, and evolving those systems. However, traditional dependence analysis is either inapplicable or of very limited utility for distributed programs due to the decoupled components of these programs which run in concurrent processes at physically separated machines. Motivated by the need for dependence analysis of distributed software and the diverse cost-effectiveness needs of dependence-based applications, this paper presents D2ABS, a framework of dynamic dependence analysis for distributed programs. By partially ordering distributed method execution events and inferring causality from the ordered events, D2ABS computes method-level dependencies both within and across process boundaries. Further, by exploiting message-passing semantics across processes, and incorporating static dependencies and statement coverage within individual components, D2ABS offers three additional instantiations that trade efficiency for better precision. We present the design of the D2ABS framework and evaluate the four instantiations of D2ABS on distributed systems of various architectures and scales using our implementation for Java. Our empirical results show that D2ABS is significantly more effective than existing options while offering varying levels of cost-effectiveness tradeoffs. As our framework essentially computes whole-system run-time dependencies, it naturally empowers a range of other dependence-based applications.

Data-driven approaches have proven to be promising in mobile software analysis, yet these approaches rely on sizable and quality datasets. For Android app analysis in particular, there have been several well-known datasets that are widely used by the community. However, there is still a lack of such datasets that represent the run-time behaviors of apps- existing datasets are largely static, whereas run-time datasets are essential for data-driven dynamic and hybrid analysis of apps. In this paper, we present AndroCT, a large-scale dataset on the run-time traces of function calls in 35,974 benign and malicious Android apps from ten historical years (2010 through 2019). These call traces were produced by running each sample app against automatically generated test inputs for ten minutes. Moreover, each app was exercised both on an emulator and a real device, and the traces were separately curated. AndroCT has been used to build a novel dynamic profile of Android apps that has enabled several effective techniques and informative empirical studies concerning Android app security. We describe what this dataset includes, how it was created and stored, and how it has been used in past and would be used in the future.

Bug reports (BR) contain vital information that can help triaging teams prioritize and assign bugs to developers who will provide the fixes. However, studies have shown that BR fields often contain incorrect information that need to be reassigned, which delays the bug fixing process. There exist approaches for predicting whether a BR field should be reassigned or not. These studies use mainly BR descriptions and traditional machine learning algorithms (SVM, KNN, etc.). As such, they do not fully benefit from the sequential order of information in BR data, such as function call sequences in BR stack traces, which may be valuable for improving the prediction accuracy. In this paper, we propose a novel approach, called EnHMM, for predicting the reassignment of BR fields using ensemble Hidden Markov Models (HMMs), trained on stack traces. EnHMM leverages the natural ability of HMMs to represent sequential data to model the temporal order of function calls in BR stack traces. When applied to Eclipse and Gnome BR repositories, EnHMM achieves an average precision, recall, and F-measure of 54%, 76%, and 60% on Eclipse dataset and 41%, 69%, and 51% on Gnome dataset. We also found that EnHMM improves over the best single HMM by 36% for Eclipse and 76% for Gnome. Finally, when comparing EnHMM to Im.ML.KNN, a recent approach in the field, we found that the average F-measure score of EnHMM improves the average F-measure of Im.ML.KNN by 6.80% and improves the average recall of Im.ML.KNN by 36.09%. However, the average precision of EnHMM is lower than that of Im.ML.KNN (53.93% as opposed to 56.71%).

Data-oriented attacks manipulate non-control data to alter a program’s benign behavior without violating its control-flow integrity. It has been shown that such attacks can cause significant damage even in the presence of control-flow defense mechanisms. However, these threats have not been adequately addressed. In this survey article, we first map data-oriented exploits, including Data-Oriented Programming (DOP) and Block-Oriented Programming (BOP) attacks, to their assumptions/requirements and attack capabilities. Then, we compare known defenses against these attacks, in terms of approach, detection capabilities, overhead, and compatibility. It is generally believed that control flows may not be useful for data-oriented security. However, data-oriented attacks (especially DOP attacks) may generate side effects on control-flow behaviors in multiple dimensions (i.e., incompatible branch behaviors and frequency anomalies). We also characterize control-flow anomalies caused by data-oriented attacks. In the end, we discuss challenges for building deployable data-oriented defenses and open research questions.

Distributed software systems are increasingly developed and deployed today. Many of these systems are supposed to run continuously. Given their critical roles in our society and daily lives, assuring the quality of distributed systems is crucial. Analyzing runtime program dependencies has long been a fundamental technique underlying numerous tool support for software quality assurance. Yet conventional approaches to dynamic dependence analysis face severe scalability barriers when they are applied to real-world distributed systems, due to the unbounded executions to be analyzed in addition to common efficiency challenges suffered by dynamic analysis in general. In this article, we present S EADS , a distributed , online , and cost-effective dynamic dependence analysis framework that aims at scaling the analysis to real-world distributed systems. The analysis itself is distributed to exploit the distributed computing resources (e.g., a cluster) of the system under analysis; it works online to overcome the problem with unbounded execution traces while running continuously with the system being analyzed to provide timely querying of analysis results (i.e., runtime dependence set of any given query). Most importantly, given a user-specified time budget, the analysis automatically adjusts itself to better cost-effectiveness tradeoffs (than otherwise) while respecting the budget by changing various analysis parameters according to the time being spent by the dependence analysis. At the core of the automatic adjustment is our application of a reinforcement learning method for the decision making—deciding which configuration to adjust to according to the current configuration and its associated analysis cost with respect to the user budget. We have implemented S EADS for Java and applied it to eight real-world distributed systems with continuous executions. Our empirical results revealed the efficiency and scalability advantages of our framework over a conventional dynamic analysis, at least for dynamic dependence computation at method level. While we demonstrate it in the context of dynamic dependence analysis in this article, the methodology for achieving and maintaining scalability and greater cost-effectiveness against continuously running systems is more broadly applicable to other dynamic analyses.

Malware detection at scale in the Android realm is often carried out using machine learning techniques. State-of-the-art approaches such as DREBIN and MaMaDroid are reported to yield high detection rates when assessed against well-known datasets. Unfortunately, such datasets may include a large portion of duplicated samples, which may bias recorded experimental results and insights. In this article, we perform extensive experiments to measure the performance gap that occurs when datasets are de-duplicated. Our experimental results reveal that duplication in published datasets has a limited impact on supervised malware classification models. This observation contrasts with the finding of Allamanis on the general case of machine learning bias for big code. Our experiments, however, show that sample duplication more substantially affects unsupervised learning models (e.g., malware family clustering). Nevertheless, we argue that our fellow researchers and practitioners should always take sample duplication into consideration when performing machine-learning-based (via either supervised or unsupervised learning) Android malware detections, no matter how significant the impact might be.

With the rise of the mobile computing market, Android has received tremendous attention from both academia and industry. Application programming in Android is known to have unique characteristics, and Android apps be particularly vulnerable to various security attacks. In response, numerous solutions for particular security issues have been proposed. However, there is little broad understanding about Android app code structure and behaviors along with their implications for app analysis and security defense, especially in an evolutionary perspective. To mitigate this gap, we present a longitudinal characterization study of Android apps to systematically investigate how they are built and execute over time. Through lightweight static analysis and method-level tracing, we examined the code and execution of 17,664 apps sampled from the apps developed in each of eight past years, with respect to metrics in three complementary dimensions. Our study revealed that (1) apps functionalities heavily rely on the Android framework/SDK, and the reliance continues to grow, (2)Activitycomponents constantly dominated over other types of components and were responsible for the invocation of most lifecycle callbacks, (3) event-handling callbacks consistently focused more on user-interface events than system events, (4) the overall use of callbacks has been slowly diminishing over time, (5) the majority of exercised inter-component communications (ICCs) did not carry any data payloads, and (6) sensitive data sources and sinks targeted only one/two dominant categories of information or operations, and the ranking of source/sink categories remained quite stable throughout the eight years. We discuss the implications of our empirical findings for cost-effective app analysis and security defense for Android, and make cost-effectiveness improvement recommendations accordingly.

We envision visual semantics learning (VSL), a novel methodology that derives high-level functional description of given software from its visual (graphical) outputs. By visual semantics, we mean the semantic description about the software’s behaviors that are exhibited in its visual outputs. VSL works by composing this description based on visual element labels extracted from these outputs through image/video understanding and natural language generation. The result of VSL can then support tasks that may benefit from the high-level functional description. Just like a developer relies on program understanding to conduct many of such tasks, automatically understanding software (i.e., by machine rather than by human developers) is necessary to eventually enable fully automated software engineering. Apparently, VSL only works with software that does produce visual outputs that meaningfully demonstrate the software’s behaviors. Nevertheless, learning visual semantics would be a useful first step towards automated software understanding. We outline the design of our approach to VSL and present early results demonstrating its merits.

We present Dads, the first distributed, online, scalable, and cost-effective dynamic slicer for continuously-running distributed programs with respect to user-specified budget constraints. Dads is distributed by design to exploit distributed and parallel computing resources. With an online analysis, it avoids tracing hence the associated time and space costs. Most importantly, Dads achieves and maintains practical scalability and cost-effectiveness tradeoffs according to a given budget on analysis time by continually and automatically adjusting the configuration of its analysis algorithm on the fly via reinforcement learning. Against eight real-world Java distributed systems, we empirically demonstrated the scalability and cost-effectiveness merits of Dads. The open-source tool package of Dads with a demo video is publicly available.

Data dependence analysis underlies various applications in software quality assurance, yet existing frameworks/tools for this analysis commonly suffer scalability challenges. We present PCA, a static interprocedural data dependence analyzer for real-world C programs. PCA performs interprocedural points-to and data-flow analyses with a lightweight design. Most of all, it features a partial call-path (PCA) analysis that consists of optimization options to further speed up data dependence computation. As an example application of it, PCA readily supports memory leak detection, for which it helps achieve close or better performance and precision relative to the same application based on a state-of-the-art value flow analysis. In particular, it found four more memory leaks in an industry-scale system which have been fixed by the developers. Through the data dependence it computes, PCA can enable other applications (e.g., impact analysis and taint analysis).

In this paper, we present the first exploratory study of deprecated Python library APIs to understand the status quo of API deprecation in the realm of Python libraries. Specifically, we aim to comprehend how deprecated library APIs are declared and documented in practice by their maintainers, and how library users react to them. By thoroughly looking into six reputed Python libraries and 1,200 GitHub projects, we experimentally observe that API deprecation is poorly handled by library contributors, which subsequently introduce difficulties for Python developers to resolve the usage of deprecated library APIs. This empirical evidence suggests that our community should take immediate actions to appropriately handle the deprecation of Python library APIs.

Impact analysis as a critical step in software evolution assists developers with decision making as regards whether and where to apply code changes in evolving software. Dynamic approaches to this analysis particularly focus on the effects of potential code changes to a program with respect to its concrete executions. Given the existence of a number of prior approaches to dynamic impact analysis as opposed to a lack of systematic understanding of their performance, the first comprehensive study of the predictive accuracy of dynamic impact analysis was conducted, comparing the performance of representative techniques in this area against various kinds of realized code changes. This paper reflects on the progress in dynamic impact analysis, concerning the impact of that earlier study on later research. We also situate dynamic impact analysis within the current research and practice on impact analysis in general, and envision relevant future research vectors in this area.

We present preliminary results of a study on memory vulnerability detectors based on (static and/or dynamic) program analysis. Against a public suite of 520 C/C++ programs as benchmarks which cover 14 different vulnerability categories, we measured the performance of five state-of-the-art detectors in terms of effectiveness and efficiency. Our study revealed that with respect to the particular set of benchmarks we chose: (1) the effectiveness of these studied detectors varied widely: 66.7% to 100% precision, 0% to 100% recall, and 0% to 100% F1 per category, indicating most of the techniques worked extremely well on certain kinds of vulnerabilities yet quite poorly on others, (2) these detectors were generally quite efficient: despite a few outliers, the average (per benchmark) time costs were around one second, (3) except for between the most and least accurate detectors, other pairs of detectors did not have statistically significant and large differences in accuracy in our pair-wise statistical testing. We also share insights into the failures and successes of these detectors obtained from our case studies.

Logging is an essential software practice that is used by developers to debug, diagnose and audit software systems. Despite the advantages of logging, poor logging practices can potentially leak sensitive data. The problem of data leakage is more severe in applications that run on mobile devices, since these devices carry sensitive identification information ranging from physical device identifiers (e.g., IMEI MAC address) to communications network identifiers (e.g., SIM, IP, Bluetooth ID), and application-specific identifiers related to the location and the users' accounts. This preliminary study explores the impact of logging practices on data leakage of such sensitive information. Particularly, we want to investigate whether log-related statements inserted into an application code could lead to data leakage. While studying logging practices in mobile applications is an active research area, to our knowledge, this is the first study that explores the interplay between logging and security in the context of mobile applications for Android. We propose an approach called MobiLogLeak, an approach that identifies log statements in deployed apps that leak sensitive data. MobiLogLeak relies on taint flow analysis. Among 5,000 Android apps that we studied, we found that 200 apps leak sensitive data through logging.

Malware collusion is a technique utilized by attackers to evade standard detection. It is a new threat where two or more applications, appearing benign, communicate to perform a malicious task. Most proposed approaches aim at detecting stand-alone malicious applications. We point out the need for analyzing data flows across multiple Android apps, a problem referred to as end-to-end flow analysis. In this work, we present a flow analysis for app pairs that computes the risk level associated with their potential communications. Our approach statically analyzes the sensitivity and context of each inter-app flow based on inter-component communication (ICC) between communicating apps, and defines fine-grained security policies for inter-app ICC risk classification. We perform an empirical study on 7,251 apps from the Google Play store to identify the apps that communicate with each other via ICC channels. Our results report four times fewer warnings on our dataset of 197 real app pairs communicating via explicit external ICCs than the state-of-the-art permission-based collusion detection.

Machine learning–based classification dominates current malware detection approaches for Android. However, due to the evolution of both the Android platform and its user apps, existing such techniques are widely limited by their reliance on new malware samples, which may not be timely available, and constant retraining, which is often very costly. As a result, new and emerging malware slips through, as seen from the continued surging of malware in the wild. Thus, a more practical detector needs not only to be accurate on particular datasets but, more critically, to be able to sustain its capabilities over time without frequent retraining. In this article, we propose and study the sustainability problem for learning-based app classifiers. We define sustainability metrics and compare them among five state-of-the-art malware detectors for Android. We further developed DroidSpan , a novel classification system based on a new behavioral profile for Android apps that captures sensitive access distribution from lightweight profiling. We evaluated the sustainability of DroidSpan versus the five detectors as baselines on longitudinal datasets across the past eight years, which include 13,627 benign apps and 12,755 malware. Through our extensive experiments, we showed that DroidSpan significantly outperformed all the baselines in substainability at reasonable costs, by 6%–32% for same-period detection and 21%–37% for over-time detection. The main takeaway , which also explains the superiority of DroidSpan , is that the use of features consistently differentiating malware from benign apps over time is essential for sustainable learning-based malware detection, and that these features can be learned from studies on app evolution.

The rapid expansion of the Android ecosystem is accompanied by continuing diversification of platforms and devices, resulting in increasing incompatibility issues which damage user experiences and impede app development productivity. In this paper, we conducted a large-scale, longitudinal study of compatibility issues in 62,894 benign apps developed in the past eight years, to understand the symptoms and causes of these issues. We further investigated the incompatibilities that are actually exercised at runtime through the system logs and execution traces of 15,045 apps. Our study revealed that, among others, (1) compatibility issues were prevalent and persistent at both installation and run time, with greater prevalence of run-time incompatibilities, (2) there were no certain Android versions that consistently saw more or less app incompatibilities than others, (3) installation-time incompatibilities were strongly correlated with the minSdkVersion specified in apps, while run-time incompatibilities were most significantly correlated with the underlying platform’s API level, and (4) installation-time incompatibilities were mostly due to apps’ use of architecture-incompatible native libraries, while run-time incompatibilities were mostly due to API changes during SDK evolution. We offered further insights into app incompatibilities, as well as recommendations on dealing with the issues for bother developers and end users of Android apps.

Due to the increasing demands for computational scalability and performance, more distributed software systems are being developed than single-process programs. As an important step in software quality assurance, software measurement provides essential means and evidences in quality assessment hence incentives and guidance for quality improvement. However, despite the rich literature on software measurement in general, existing measures are mostly defined for single-process programs only or limited to conventional metrics. In this paper, we propose a novel set of metrics for common distributed systems, with a focus on their interprocess communications (IPC), a vital aspect of their run-time behaviors. We demonstrated the practicality of characterizing IPC dynamics and complexity via the proposed IPC metrics, by computing the measures against nine real-world distributed systems and their varied executions. To demonstrate the practical usefulness of IPC measurements, we extensively investigated how the proposed metrics may help understand and analyze various quality factors of distributed systems, ranging from maintainability and stability to security and performance, on the same nine distributed systems and their executions. We found that higher IPC coupling tended to be generally detrimental to most of the quality aspects while interprocess sharing of common functionalities should be promoted due to its understandability and security benefits.

Android app testing is challenging and time-consuming because fully testing all feasible execution paths is difficult. Nowadays apps are usually tested in two ways: human testing or automated testing. Prior work compared different automated tools. However, some fundamental questions are still unexplored, including (1) how automated testing behaves differently from human testing, and (2) whether automated testing can fully or partially substitute human testing.

As in other software domains, information flow security is a fundamental aspect of code security in distributed systems. However, most existing solutions to information flow security are limited to centralized software. For distributed systems, such solutions face multiple challenges, including technique applicability, tool portability, and analysis scalability. To overcome these challenges, we present DistTaint, a dynamic information flow (taint) analyzer for distributed systems. By partial-ordering method-execution events, DistTaint infers implicit dependencies in distributed programs, so as to resolve the applicability challenge. It resolves the portability challenge by working fully at application level, without customizing the runtime platform. To achieve scalability, it reduces analysis costs using a multi-phase analysis, where the pre-analysis phase generates method-level results to narrow down the scope of the following statement-level analysis. We evaluated DistTaint against eight real-world distributed systems. Empirical results showed DistTaint’s applicability to, portability with, and scalability for industry-scale distributed systems, along with its capability of discovering known and unknown vulnerabilities. A demo video for DistTaint can be downloaded from https://www.dropbox.com/l/scl/AAAkrm4p63Ffx0rZqblY3zlLFuaohbRxs0 or viewed here https://youtu.be/fy4yMIaKzPE online. The tool package is here: https://www.dropbox.com/sh/kfr9ixucyny1jp2/AAC00aI-I8Od4ywZCqwZ1uaa?dl=0

Most existing Android malware detection and categorization techniques are static approaches, which suffer from evasion attacks, such as obfuscation. By analyzing program behaviors, dynamic approaches are potentially more resilient against these attacks. Yet existing dynamic approaches mostly rely on characterizing system calls which are subject to system-call obfuscation. This paper presents DroidCat, a novel dynamic app classification technique, to complement existing approaches. By using a diverse set of dynamic features based on method calls and inter-component communication (ICC) Intents without involving permission, app resources, or system calls while fully handling reflection, DroidCat achieves superior robustness than static approaches as well as dynamic approaches relying on system calls. The features were distilled from a behavioral characterization study of benign versus malicious apps. Through three complementary evaluation studies with 34 343 apps from various sources and spanning the past nine years, we demonstrated the stability of DroidCat in achieving high classification performance and superior accuracy compared with the two state-of-the-art peer techniques that represent both static and dynamic approaches. Overall, DroidCat achieved 97% F1-measure accuracy consistently for classifying apps evolving over the nine years, detecting or categorizing malware, 16%-27% higher than any of the two baselines compared. Furthermore, our experiments with obfuscated benchmarks confirmed higher robustness of DroidCat over these baseline techniques. We also investigated the effects of various design decisions on DroidCat's effectiveness and the most important features for our dynamic classification. We found that features capturing app execution structure such as the distribution of method calls over user code and libraries are much more important than typical security features such as sensitive flows.

Today, computing on various Android devices is pervasive. However, growing security vulnerabilities and attacks in the Android ecosystem constitute various threats through user apps. Taint analysis is a common technique for defending against these threats, yet it suffers from challenges in attaining practical simultaneous scalability and effectiveness. This paper presents a novel approach to fast and precise taint checking, called incremental taint analysis, by exploiting the evolving nature of Android apps. The analysis narrows down the search space of taint checking from an entire app, as conventionally addressed, to the parts of the program that are different from its previous versions. This technique improves the overall efficiency of checking multiple versions of the app as it evolves. We have implemented the techniques as a tool prototype, EvoTaint, and evaluated our analysis by applying it to real-world evolving Android apps. Our preliminary results show that the incremental approach largely reduced the cost of taint analysis, by 78.6% on average, yet without sacrificing the analysis effectiveness, relative to a representative precise taint analysis as the baseline.

Impact analysis determines the effects that program entities of interest, or changes to them, may have on the rest of the program for software measurement, maintenance, and evolution tasks. Dynamic impact analysis could be one major approach to impact analysis that computes smaller impact setsthan static alternatives for concrete sets of executions. However, existing dynamic approaches often produce impact sets that are too large to be useful, hindering their adoption in practice. To address this problem, we propose to exploit static program dependencies to drastically prune false-positive impacts that are not exercised by the set of executions utilized by the analysis, via hybrid dependence approximation. Further, we present a novel dynamic impact analysis called Diver which leverages both the information provided by the dependence graph and method-execution events to identify runtimemethod-level dependencies, hence dynamic impact sets, much more precisely without reducing safety and at acceptable costs. We evaluate Diver on ten Java subjects of various sizes and application domains against both arbitrary queries covering entire programs and practical queries based on changes actually committed by developers to actively evolving software repositories. Our extensive empirical studies show that Diver can significantly improve the precision of impact prediction, with 100-186 percent increase, with respect to a representative existing alternative thus provide a far more effective option for dynamic impact prediction. Following a similar rationale to Diver, we further developed and evaluated an online dynamic impact analysis called DiverOnline which produces impact sets immediately upon the termination of program execution. Our results show that compared to the offline approach, for the same precision, the online approach can reduce the time by 50 percent on average for answering all possible queries in the given program at once albeit at the price of possibly significant increase in runtime overhead. For users interested in one specific query only, the online approach may compute the impact set for that query during runtime without much slowing down normal program operation. Further, the online analysis, which does not incur any space cost beyond the static-analysis phase, may be favored against the offline approach when trace storage and/or related file-system resource consumption becomes a serious challenge or even stopper for adopting dynamic impact prediction. Therefore, the online and offline analysis together offer complementary options to practitioners accommodating varied application/task scenarios and diverse budget constraints.

Inter-Component Communication (ICC) enables useful interactions between mobile apps. However, misuse of ICC exposes users to serious threats such as intent hijacking/spoofing and app collusions, allowing malicious apps to access privileged user data via another app. Unfortunately, existing ICC analyses are largely incompetent in both accuracy and scale. This poster points out the need and technical challenges of prioritized analysis of inter-app ICC risks. In this poster, we propose MR-Droid, a MapReduce-based computing framework for accurate and scalable inter-app ICC analysis in Android. MR-Droid extracts data-flow features between multiple communicating apps and the target apps to build a large-scale ICC graph. Our approach is to leverage the ICC graph to provide contexts for inter-app communications to produce precise alerts and prioritize risk assessments. This process requires large app-pair data, which is enabled by our MapReduce-based program analysis. Our initial extensive experiments on 11,996 apps from 24 app categories (13 million pairs) demonstrate the scalability of our approach.

Most existing research for Android focuses on particular security issues, yet there is little broad understanding of Android application run-time characteristics and their implications. To mitigate this gap, we present the first systematic dynamic characterization study of Android apps that targets a broad understanding of application behaviors in Android. Through lightweight method-level profiling, we collected 59GB traces of method calls and Intent-based inter-component communication (ICC) from 125 popular Android apps and 62 pairs among them that enabled an intensive empirical investigation of their run-time behaviors. Our study revealed that, among other findings, (1) the application executions were overwhelmingly dominated by the Android framework, (2) Activity components dominated over other types of components and were responsible for most lifecycle callbacks (3) most event handlers dealt with user interactions as opposed to system events, (4) the majority of exercised ICCs did not carry any data payloads, and (5) sensitive data sources and sinks targeted only one/two dominant categories of information or operations. We also discuss the implications of our results for cost-effective program analysis and security defense for Android.

As the Android app market keeps growing, there is a pressing need for automated tool supports to empower Android developers to produce quality apps with higher productivity. Yet existing tools for Android mostly aim at security and privacy protection, primarily targeting end users and security analysts. Towards filling this gap, we present DROIDFAX, a toolkit that targets the developers to help them comprehensively understand Android apps regarding their code structure and behavioral traits. To that end, DROIDFAX features a systematic app characterization in multiple dimensions and views, through lightweight code analysis and profiling of both ordinary method calls (including those via reflection and exceptional control flows) and inter-component communications (including those within and across apps). The toolkit also includes a statement coverage tracker that works directly on bytecode and a dedicated tracer of events occurred during app executions. Applying DROIDFAX in two use cases has resulted in important findings about app behavioral patterns and an advanced security defense technique for Android. Empirical results also showed promising efficiency and scalability of DROIDFAX for practical adoption. A demo video for DROIDFAX can be viewed here or downloaded here.

We describe a set of artifacts for dynamic analysis of Android apps, including a dataset used in a dynamic characterization study, source code used for performing the study, an Android inter-app benchmark suite, and definition of Android behavioral metrics.

Inter-component communication (ICC) serves as a key element of any Android app's implementation. Specifically, an Android app uses Intents as the main mechanism for ICC to complete tasks such as switching between different user interfaces, starting background services, communicating to other apps on the Android device, and saving or retrieving data from device storage. Thus, dissecting how an app uses ICCs to accomplish its tasks is fundamental to understanding the app's underlying behaviors. Existing works on Android ICCs focus on resolving Intents and/or mapping ICCs for security analysis purposes. While the ICC analysis result is potentially informative, it is difficult to digest on its own and has not been utilized for app/ICC comprehension. Also, the result is based on static analysis thus does not inform of run-time app behaviors due to ICCs. We propose the first approach to dissecting Android ICCs via interactive, dynamic visualizations empowered by static and dynamic ICC analysis combined. Through multiple semantically linked views and in-situ interaction features, our approach enables real-time visual explorations of ICCs as they are triggered by user inputs to the app under analysis. It conveys rich ICC information while managing limited visual space through various visualization design strategies. Our case studies with a number of most commonly used apps have showed promising merits of the approach for both deep ICC comprehension and security vulnerability inspection, as well as practical scalability. Our tool prototype of the approach has enabled quick revelation of inefficient, intrusive, and malicious behaviors in several popular apps that were normally hidden to users.

Dynamic impact analysis is a fundamental technique for understanding the impact of specific program entities, or changes to them, on the rest of the program for concrete executions. However, existing techniques are either inapplicable or of very limited utility for distributed programs running in multiple concurrent processes. This paper presents DistIA, a dynamic analysis of distributed systems that predicts impacts propagated both within and across process boundaries by partially ordering distributed method-execution events, inferring causality from the ordered events, and exploiting message-passing semantics. We applied DistIA to large distributed systems of various architectures and sizes, for which it on average finishes the entire analysis within one minute and safely reduces impact-set sizes by over 43% relative to existing options with runtime overhead less than 8%. Moreover, two case studies initially demonstrate the precision of DistIA and its utility in distributed system understanding. While conservative thus subject to false positives, DistIA balances precision and efficiency to offer cost-effective options for evolving distributed programs.

Impact analysis not only assists developers with change planning and management, but also facilitates a range of other client analyses, such as testing and debugging. In particular, for developers working in the context of specific program executions, dynamic impact analysis is usually more desirable than static approaches, as it produces more manageable and relevant results with respect to those concrete executions. However, existing techniques for this analysis mostly lie on two extremes: either fast, but too imprecise, or more precise, yet overly expensive. In practice, both more cost-effective techniques and variable cost-effectiveness trade-offs are in demand to fit a variety of usage scenarios and budgets of impact analysis. This article aims to fill the gap between these two extremes with an array of cost-effective analyses and, more broadly, to explore the cost and effectiveness dimensions in the design space of impact analysis. We present the development and evaluation of D ia P ro , a framework that unifies a series of impact analyses, including three new hybrid techniques that combine static and dynamic analyses. Harnessing both static dependencies and multiple forms of dynamic data including method-execution events, statement coverage, and dynamic points-to sets, D ia P ro prunes false-positive impacts with varying strength for variant effectiveness and overheads. The framework also facilitates an in-depth examination of the effects of various program information on the cost-effectiveness of impact analysis. We applied D ia P ro to ten Java applications in diverse scales and domains, evaluating it thoroughly on both arbitrary and repository-based queries from those applications. We show that the three new analyses are all significantly more effective than existing alternatives while remaining efficient, and the D ia P ro framework, as a whole, provides flexible cost-effectiveness choices for impact analysis with the best options for variable needs and budgets. Our study results also suggest that hybrid techniques tend to be much more cost-effective than purely dynamic approaches, in general, and that statement coverage has mostly stronger effects than dynamic points-to sets on the cost-effectiveness of dynamic impact analysis, while static dependencies have even stronger effects than both forms of dynamic data.

Software is constantly changing. To ensure the quality of this process, when preparing to change a program, developers must first identify the main consequences and risks of modifying the program locations they intend to change. This activity is called change-impact analysis. However, existing impact analysis suffers from two major problems: coarse granularity and large size of the resulting impact sets. Finer-grained analyses such as slicing give more detailed impact sets which, however, are also even larger in size. While various impact-set reduction approaches have been proposed at different levels of granularity, the challenge persists as very-large impact sets are still produced, impeding the adoption of impact analysis due to the great costs of inspecting those impact sets. To address these challenges, we present a novel dynamic-analysis technique called SensA which combines sensitivity analysis and execution differencing. SensA not only provides fine-grained (statement-level) impact sets but also prioritizes potential impacts via semantic-dependence quantification for program slices. We evaluated the benefits of impact prioritization using SensA with respect to static and dynamic forward slicing via an extensive empirical study of open-source Java applications and three case studies. Our results show that SensA can offer much better cost-effectiveness than slicing in assisting developers with impact inspection and fault cause-effect understanding.

While empowering a wide range of software engineering tasks, the traditional fine-grained software dependence (TSD) model can face great scalability challenges that hinder its applications. Many dependence abstraction approaches have been proposed, yet most of them either target very specific clients or model partial dependencies only, while others have not been fully evaluated for their accuracy with respect to the TSD model, especially in approximating forward dependencies on object-oriented programs. To fill this gap, we present a new dependence abstraction called the method dependence graph (MDG) that approximates the TSD model at method level, and compare it against a recent TSD abstraction, called the Static-Exectue-After (SEA), concerning forward-dependence approximation. We also evaluate the cost-effectiveness of both approaches in the application context of impact analysis. Our results show that the MDG can approximate TSD safely, for method-level forward dependence at least, with little loss of precision yet huge gain in efficiency, and for the same purpose, while both are safe, the MDG can achieve significantly higher precision than SEA at practical costs.

Dynamic impact analysis can greatly assist developers with managing software changes by focusing their attention on the effects of potential changes relative to concrete program executions. While dependence-based dynamic impact analysis (DDIA) provides finer-grained results than traceability-based approaches, traditional DDIA techniques often produce imprecise results, incurring excessive costs thus hindering their adoption in many practical situations. In this paper, we present the design and evaluation of a DDIA framework and its three new instances that offer not only much more precise impact sets but also flexible cost-effectiveness options to meet diverse application needs such as different budgets and levels of detail of results. By exploiting both static dependencies and various dynamic information including method-execution traces, statement coverage, and dynamic points-to data, our techniques achieve that goal at reasonable costs according to our experiment results. Our study also suggests that statement coverage has generally stronger effects on the precision and cost-effectiveness of DDIA than dynamic points-to data.

We present the design and implementation of TRACERJD, a toolkit devoted to dynamic dependence analysis via fine-grained whole-program dependence tracing. TRACERJD features a generic framework for efficient offline analysis of dynamic dependencies, including those due to exception-driven control flows. Underlying the framework is a hierarchical trace indexing scheme by which TRACERJD maintains the relationships among execution events at multiple levels of granularity while capturing those events at runtime. Built on this framework, several application tools are provided as well, including a dynamic slicer and a performance profiler. These example applications also demonstrate the flexibility and ease with which a variety of client analyses can be built based on the framework. We tested our toolkit on four Java subjects, for which the results suggest promising efficiency of TRACERJD for its practical use in various dependence-based tasks.

Impact analysis determines the effects that the behavior of program entities, or changes to them, can have on the rest of the system. Dynamic impact analysis is one practical form that computes smaller impact sets than static alternatives for concrete sets of executions. However, existing dynamic approaches can still produce impact sets that are too large to be useful. To address this problem, we present a novel dynamic impact analysis called DIVER that exploits static dependencies to identify runtime impacts much more precisely without reducing safety and at acceptable costs. Our preliminary empirical evaluation shows that DIVER can significantly increase the precision of dynamic impact analysis.

Sensitivity analysis determines how a system responds to stimuli variations, which can benefit important software-engineering tasks such as change-impact analysis. We present SENSA, a novel dynamic-analysis technique and tool that combines sensitivity analysis and execution differencing to estimate the dependencies among statements that occur in practice. In addition to identifying dependencies, SENSA quantifies them to estimate how much or how likely a statement depends on another. Quantifying dependencies helps developers prioritize and focus their inspection of code relationships. To assess the benefits of quantifying dependencies with SENSA, we applied it to various statements across Java subjects to find and prioritize the potential impacts of changing those statements. We found that SENSA predicts the actual impacts of changes to those statements more accurately than static and dynamic forward slicing. Our SENSA prototype tool is freely available for download.

Dynamic slicing is a practical and popular analysis technique used in various software-engineering tasks. Dynamic slicing is known to be incomplete because it analyzes only a subset of all possible executions of a program. However, it is less known that its results may inaccurately represent the dependencies that occur in those executions. Some researchers have identified this problem and developed extensions such as relevant slicing, which incorporates static information. Yet, dynamic slicing continues to be widely used, even though the extent of its inaccuracy is not well understood, which can affect the benefits of this analysis. In this paper, we present an approach to assess the accuracy of forward dynamic slices, which are used in software maintenance and evolution tasks. Because finding all actual dependencies is an undecidable problem, our approach instead computes bounds of the precision and recall of forward dynamic slices. Our approach uses sensitivity analysis and execution differencing to find a subset of all program statements that truly depend at runtime on another statement. Using this approach, we studied the accuracy of many forward dynamic slices from a variety of Java applications. Our results show that forward dynamic slicing can have low recall -- for dependencies in the analyzed executions -- and some potential imprecision. We also conducted a case study that shows how this inaccuracy affects a software maintenance task. To the best of our knowledge, ours is the first work that quantifies the intrinsic limitations of dynamic slicing.

Program slicing is a popular but imprecise technique for identifying which parts of a program affect or are affected by a particular value. A major reason for this imprecision is that slicing reports all program statements possibly affected by a value, regardless of how relevant to that value they really are. In this paper, we introduce quantitative slicing (q-slicing), a novel approach that quantifies the relevance of each statement in a slice. Q-slicing helps users and tools focus their attention first on the parts of slices that matter the most. We present two methods for quantifying slices and we show the promise of q-slicing for a particular application: predicting the impacts of changes.

We report the impact of display characteristics (stereo and size) on task performance in diffusion magnetic resonance imaging (DMRI) in a user study with 12 participants. The hypotheses were that (1) adding stereo and increasing display size would improve task accuracy and reduce completion time, and (2) the greater the complexity of a spatial task, the greater the benefits of an improved display. Thus we expected to see greater performance gains when detailed visual reasoning was required. Participants used dense streamtube visualizations to perform five representative tasks: (1) determine the higher average fractional anisotropy (FA) values between two regions, (2) find the endpoints of fiber tracts, (3) name a bundle, (4) mark a brain lesion, and (5) judge if tracts belong to the same bundle. Contrary to our hypotheses, we found the task completion time was not improved by the use of the larger display and that performance accuracy was hurt rather than helped by the introduction of stereo in our study with dense DMRI data. Bigger was not always better. Thus cautious should be taken when selecting displays for scientific visualization applications. We explored the results further using the body-scale unit and subjective size and stereo experiences.