The Android ecosystem has experienced rapid growth, resulting in a diverse range of platforms and devices. This expansion has also brought about compatibility issues that negatively impact user experiences and hinder app development productivity. Existing relevant studies are focused on and limited to the “static” sense of those issues (in terms of potentialities and proneness), while only addressing compatibility issues that possibly occur during app executions. In this article, we present an extensive and longitudinal study on app compatibility issues that are disparate from yet complementary to prior studies, characterizing the incompatibilities based on actual , exercised observations and evidence at both installation and run-time. With a dataset of 74,545 benign apps and 56,919 malicious apps over a span of 12 years (2010 through 2021) and 10 Android versions, we extensively examine the prevalence and symptoms/effects and causes of, as well as the contributing factors to, installation-time and run-time compatibility issues. Our study reveals 12 major novel findings regarding Android app incompatibilities. First (Findings 1, 2), installation-time incompatibilities persisted significantly over the 12 years, even more so in malware than benign apps. Second (Findings 7, 8), run-time compatibility issues were also seen persistently over time but only on specific Android platforms (such as API 26,27, etc.) and much less by malware than benign apps. Third (Findings 5, 6, 11, 12), there is a significant (moderate/stronger) correlation between an app’s specified minSdkVersion and its incompatibilities (over all symptoms and/or with respect to one of its dominating symptom), with stronger correlations seen in malware than in benign apps, for both installation-time and run-time incompatibilities. Similar observations hold (although with much stronger correlation in absolute terms) when considering, instead of the minSdkVersion itself, the gap between the app’s minSdkVersion and the SDK API level of the platform the app is installed to or runs on. Last (Findings 3, 4, 9, 10), installation-time incompatibilities are primarily caused by the utilization of architecture-incompatible native libraries within apps, while run-time incompatibilities are mainly attributed to API changes during the evolution of the Android SDK; the symptoms of run-time failures seen by malware are much more diverse than by benign apps. In addition to these insights, we provide practical recommendations for both app developers and end users on how to effectively address compatibility issues in Android apps, as well as how to devise effective defenses against malware from the compatibility perspectives.

A defining, unique aspect of distributed systems lies in interprocess communication (IPC) through which distributed components interact and collaborate toward the holistic system behaviors. This highly decoupled construction intuitively contributes to the scalability, performance, and resiliency advantages of distributed software, but also adds largely to their greater complexity, compared to centralized software. Yet despite the importance of IPC in distributed systems, little is known about how to quantify IPC-induced behaviors in these systems through IPC measurement and how such behaviors may be related to the quality of distributed software . To answer these questions, in this article, we present DistMeasure , a framework for measuring distributed software systems via the lens of IPC hence enabling the study of its correlation with distributed system quality. Underlying DistMeasure is a novel set of IPC metrics that focus on gauging the coupling and cohesion of distributed processes. Through these metrics, DistMeasure quantifies relevant runtime characteristics of distributed systems and their quality relevance, covering a range of quality aspects each via respective direct quality metrics. Further, DistMeasure enables predictive assessment of distributed system quality in those aspects via learning-based anomaly detection with respect to the corresponding quality metrics based on their significant correlations with related IPC metrics. Using DistMeasure , we demonstrated the practicality and usefulness of IPC measurement against 11 real-world distributed systems and their diverse execution scenarios. Among other findings, our results revealed that IPC has a strong correlation with distributed system complexity, performance efficiency, and security. Higher IPC coupling between distributed processes tended to be negatively indicative of distributed software quality, while more cohesive processes have positive quality implications. Yet overall IPC-induced behaviors are largely independent of the system scale, and higher (lower) process coupling does not necessarily come with lower (higher) process cohesion. We also show promising merits (with 98% precision/recall/F1) of IPC measurement (e.g., class-level coupling and process-level cohesion) for predictive anomaly assessment of various aspects (e.g., attack surface and performance efficiency) of distributed system quality.

As modern software systems are increasingly developed for running in distributed environments, it is crucial to provide fundamental techniques such as dependence analysis for checking, diagnosing, and evolving those systems. However, traditional dependence analysis is either inapplicable or of very limited utility for distributed programs due to the decoupled components of these programs which run in concurrent processes at physically separated machines. Motivated by the need for dependence analysis of distributed software and the diverse cost-effectiveness needs of dependence-based applications, this paper presents D2ABS, a framework of dynamic dependence analysis for distributed programs. By partially ordering distributed method execution events and inferring causality from the ordered events, D2ABS computes method-level dependencies both within and across process boundaries. Further, by exploiting message-passing semantics across processes, and incorporating static dependencies and statement coverage within individual components, D2ABS offers three additional instantiations that trade efficiency for better precision. We present the design of the D2ABS framework and evaluate the four instantiations of D2ABS on distributed systems of various architectures and scales using our implementation for Java. Our empirical results show that D2ABS is significantly more effective than existing options while offering varying levels of cost-effectiveness tradeoffs. As our framework essentially computes whole-system run-time dependencies, it naturally empowers a range of other dependence-based applications.

Data-driven approaches have proven to be promising in mobile software analysis, yet these approaches rely on sizable and quality datasets. For Android app analysis in particular, there have been several well-known datasets that are widely used by the community. However, there is still a lack of such datasets that represent the run-time behaviors of apps- existing datasets are largely static, whereas run-time datasets are essential for data-driven dynamic and hybrid analysis of apps. In this paper, we present AndroCT, a large-scale dataset on the run-time traces of function calls in 35,974 benign and malicious Android apps from ten historical years (2010 through 2019). These call traces were produced by running each sample app against automatically generated test inputs for ten minutes. Moreover, each app was exercised both on an emulator and a real device, and the traces were separately curated. AndroCT has been used to build a novel dynamic profile of Android apps that has enabled several effective techniques and informative empirical studies concerning Android app security. We describe what this dataset includes, how it was created and stored, and how it has been used in past and would be used in the future.

Distributed software systems are increasingly developed and deployed today. Many of these systems are supposed to run continuously. Given their critical roles in our society and daily lives, assuring the quality of distributed systems is crucial. Analyzing runtime program dependencies has long been a fundamental technique underlying numerous tool support for software quality assurance. Yet conventional approaches to dynamic dependence analysis face severe scalability barriers when they are applied to real-world distributed systems, due to the unbounded executions to be analyzed in addition to common efficiency challenges suffered by dynamic analysis in general. In this article, we present S EADS , a distributed , online , and cost-effective dynamic dependence analysis framework that aims at scaling the analysis to real-world distributed systems. The analysis itself is distributed to exploit the distributed computing resources (e.g., a cluster) of the system under analysis; it works online to overcome the problem with unbounded execution traces while running continuously with the system being analyzed to provide timely querying of analysis results (i.e., runtime dependence set of any given query). Most importantly, given a user-specified time budget, the analysis automatically adjusts itself to better cost-effectiveness tradeoffs (than otherwise) while respecting the budget by changing various analysis parameters according to the time being spent by the dependence analysis. At the core of the automatic adjustment is our application of a reinforcement learning method for the decision making—deciding which configuration to adjust to according to the current configuration and its associated analysis cost with respect to the user budget. We have implemented S EADS for Java and applied it to eight real-world distributed systems with continuous executions. Our empirical results revealed the efficiency and scalability advantages of our framework over a conventional dynamic analysis, at least for dynamic dependence computation at method level. While we demonstrate it in the context of dynamic dependence analysis in this article, the methodology for achieving and maintaining scalability and greater cost-effectiveness against continuously running systems is more broadly applicable to other dynamic analyses.

We present Dads, the first distributed, online, scalable, and cost-effective dynamic slicer for continuously-running distributed programs with respect to user-specified budget constraints. Dads is distributed by design to exploit distributed and parallel computing resources. With an online analysis, it avoids tracing hence the associated time and space costs. Most importantly, Dads achieves and maintains practical scalability and cost-effectiveness tradeoffs according to a given budget on analysis time by continually and automatically adjusting the configuration of its analysis algorithm on the fly via reinforcement learning. Against eight real-world Java distributed systems, we empirically demonstrated the scalability and cost-effectiveness merits of Dads. The open-source tool package of Dads with a demo video is publicly available.

The rapid expansion of the Android ecosystem is accompanied by continuing diversification of platforms and devices, resulting in increasing incompatibility issues which damage user experiences and impede app development productivity. In this paper, we conducted a large-scale, longitudinal study of compatibility issues in 62,894 benign apps developed in the past eight years, to understand the symptoms and causes of these issues. We further investigated the incompatibilities that are actually exercised at runtime through the system logs and execution traces of 15,045 apps. Our study revealed that, among others, (1) compatibility issues were prevalent and persistent at both installation and run time, with greater prevalence of run-time incompatibilities, (2) there were no certain Android versions that consistently saw more or less app incompatibilities than others, (3) installation-time incompatibilities were strongly correlated with the minSdkVersion specified in apps, while run-time incompatibilities were most significantly correlated with the underlying platform’s API level, and (4) installation-time incompatibilities were mostly due to apps’ use of architecture-incompatible native libraries, while run-time incompatibilities were mostly due to API changes during SDK evolution. We offered further insights into app incompatibilities, as well as recommendations on dealing with the issues for bother developers and end users of Android apps.

It is particularly challenging to defend common distributed systems against security vulnerabilities because of the complexity and their large sizes. However, traditional solutions, that attack the information flow security problem, often fail for large, complex real-world distributed systems due to scalability problems. The problem would be even exacerbated for the online defense of continuously-running systems. My proposed research consists of three connected themes. First, I have developed metrics to help users understand and analyze the security characteristics of distributed systems at runtime in relation to their coupling measures. Then, I have also developed a highly scalable, cost-effective dynamic information flow analysis approach for distributed systems. It can detect implicit dependencies and find real security vulnerabilities in industrial distributed systems with practical portability and scalability. In order to thoroughly solve the scalability problem in general scenarios, I am developing a self-adaptive dynamic dependency analysis framework to monitor security issues during continuous running. In this proposal, I outline the three projects in a related manner as to how they consistently target the central objective of my thesis research.

Due to the increasing demands for computational scalability and performance, more distributed software systems are being developed than single-process programs. As an important step in software quality assurance, software measurement provides essential means and evidences in quality assessment hence incentives and guidance for quality improvement. However, despite the rich literature on software measurement in general, existing measures are mostly defined for single-process programs only or limited to conventional metrics. In this paper, we propose a novel set of metrics for common distributed systems, with a focus on their interprocess communications (IPC), a vital aspect of their run-time behaviors. We demonstrated the practicality of characterizing IPC dynamics and complexity via the proposed IPC metrics, by computing the measures against nine real-world distributed systems and their varied executions. To demonstrate the practical usefulness of IPC measurements, we extensively investigated how the proposed metrics may help understand and analyze various quality factors of distributed systems, ranging from maintainability and stability to security and performance, on the same nine distributed systems and their executions. We found that higher IPC coupling tended to be generally detrimental to most of the quality aspects while interprocess sharing of common functionalities should be promoted due to its understandability and security benefits.

To protect the privacy and search sensitive data leaks, we must solve multiple challenges (e.g., applicability, portability, and scalability) for developing an appropriate taint analysis for distributed systems.We hence present DistTaint, a dynamic taint analysis for distributed systems against these challenges. It could infer implicit dependencies from partial-ordering method events in executions to resolve the applicability challenge. DistTaint fully works at application-level without any customization of platforms to overcome the portability challenge. It exploits a multi-phase analysis to achieve scalability. By proposing a pre-analysis, DistTaint narrows down the following fine-grained analysis’ scope to reduce the overall cost significantly. Empirical results showed DistTaint’s practical applicability, portability, and scalability to industry-scale distributed programs, and its capability of discovering security vulnerabilities in real-world distributed systems. The tool package can be downloaded here: https://www.dropbox.com/sh/kfr9ixucyny1jp2/AAC00aI-I8O-d4ywZCqwZ1uaa?dl=0

As in other software domains, information flow security is a fundamental aspect of code security in distributed systems. However, most existing solutions to information flow security are limited to centralized software. For distributed systems, such solutions face multiple challenges, including technique applicability, tool portability, and analysis scalability. To overcome these challenges, we present DistTaint, a dynamic information flow (taint) analyzer for distributed systems. By partial-ordering method-execution events, DistTaint infers implicit dependencies in distributed programs, so as to resolve the applicability challenge. It resolves the portability challenge by working fully at application level, without customizing the runtime platform. To achieve scalability, it reduces analysis costs using a multi-phase analysis, where the pre-analysis phase generates method-level results to narrow down the scope of the following statement-level analysis. We evaluated DistTaint against eight real-world distributed systems. Empirical results showed DistTaint’s applicability to, portability with, and scalability for industry-scale distributed systems, along with its capability of discovering known and unknown vulnerabilities. A demo video for DistTaint can be downloaded from https://www.dropbox.com/l/scl/AAAkrm4p63Ffx0rZqblY3zlLFuaohbRxs0 or viewed here https://youtu.be/fy4yMIaKzPE online. The tool package is here: https://www.dropbox.com/sh/kfr9ixucyny1jp2/AAC00aI-I8Od4ywZCqwZ1uaa?dl=0

This paper proposes a novel framework that introduces noise correction techniques to further improve label quality after ground truth inference in crowdsourcing. In the framework, an adaptive voting noise correction algorithm (AVNC) is proposed to identify and correct the most likely noises with the help of estimated qualities of labelers provided by the ground truth inference. The experimental results on two real-world datasets show that (1) the framework can improve label quality regardless of inference algorithms, especially under the circumstance that each example has a few noisy labels; and (2) since the algorithm AVNC considers both the number of and the probability of potential noises, it outperforms a baseline noise correction algorithm.