Machine Learning (ML) systems have seen widespread use for automated decision making. Testing is essential to ensure the quality of these systems, especially safety-critical autonomous systems in the automotive domain. ML systems introduce new challenges with the potential to affect test maintenance, the process of updating test cases to match the evolving system. We conducted an exploratory case study in the automotive domain to identify factors that affect test maintenance for ML systems, as well as to make recommendations to improve the maintenance process. Based on interview and artifact analysis, we identified 14 factors affecting maintenance, including five especially relevant for ML systems—with the most important relating to non-determinism and large input spaces. We also proposed ten recommendations for improving test maintenance, including four targeting ML systems—in particular, emphasizing the use of test oracles tolerant to acceptable non-determinism. The study’s findings expand our knowledge of test maintenance for an emerging class of systems, benefiting the practitioners testing these systems.

Abstract Federated learning enables multiple users to build a joint model by sharing their model updates (gradients), while their raw data remains local on their devices. In contrast to the common belief that this provides privacy benefits, we here add to the very recent results on privacy risks when sharing gradients. Specifically, we investigate Label Leakage from Gradients (LLG), a novel attack to extract the labels of the users’ training data from their shared gradients. The attack exploits the direction and magnitude of gradients to determine the presence or absence of any label. LLG is simple yet effective, capable of leaking potential sensitive information represented by labels, and scales well to arbitrary batch sizes and multiple classes. We mathematically and empirically demonstrate the validity of the attack under different settings. Moreover, empirical results show that LLG successfully extracts labels with high accuracy at the early stages of model training. We also discuss different defense mechanisms against such leakage. Our findings suggest that gradient compression is a practical technique to mitigate the attack.

Decentralized online social networks enhance users’ privacy by empowering them to control their data. However, these networks mostly lack for practical solutions for building recommender systems in a privacy-preserving manner that help to improve the network’s services. Association rule mining is one of the basic building blocks for many recommender systems. In this paper, we propose an efficient approach enabling rule mining on distributed data. We leverage the Metropolis-Hasting random walk sampling and distributed FP-Growth mining algorithm to maintain the users’ privacy. We evaluate our approach on three real-world datasets. Results reveal that the approach achieves high average precision scores () for as low as 1% sample size in well-connected social networks with remarkable reduction in communication and computational costs.

Anonymity is an essential asset for a variety of communication systems, like humans' communication, the internet of things, and sensor networks. Establishing and maintaining such communication systems requires the exchange of information about their participants (called subjects). However, protecting anonymity reduces the availability of subject information, as these can be leveraged to break anonymity. Additionally, established techniques for providing anonymity often reduce the efficiency of communication networks. In this paper, we model four mechanisms to share routing information and discuss them with respect to their influence on anonymity and efficiency. While there is no “one fits all” solution, there are suitable trade-offs to establish routing information complying with the technical capabilities of the subjects. Distributed solutions like decentralized lookup tables reduce routing information in messages at the cost of local memory consumption; other mechanisms like multi-layer encrypted path information come with higher communication overhead but reduce memory consumption for each subject.

Empowered by the high connectivity of manifold devices in today's world, distributed machine learning enables multiple, distributed users to build a joint model by sharing their gradients over a network. In this paper, we highlight the privacy risk of sharing gradients by proposing LLG, an algorithm to disclose the labels of the users' training data from their shared gradients. We conduct an empirical analysis on two datasets to demonstrate the validity of our algorithm. Results show that our approach effectively extracts the labels with high accuracy in different scenarios.

Internet voting can afford more inclusive and inexpensive elections. The flip side is that the integrity of the election can be compromised by adversarial attacks and malfunctioning voting infrastructure. Individual verifiability aims to protect against such risks by letting voters verify that their votes are correctly registered in the electronic ballot box. Therefore, voters need to carry out additional tasks making human factors crucial for security. In this article, we establish a categorization of individually verifiable Internet voting schemes based on voter interactions. For each category in our proposed categorization, we evaluate a voting scheme in a user study with a total of 100 participants. In our study, we assessed usability, user experience, trust, and further qualitative data to gain deeper insights into voting schemes. Based on our results, we conclude with recommendations for developers and policymakers to inform the choices and design of individually verifiable Internet voting schemes.

Quantitative analysis and forecasting of software vulnerability discoveries is important for patching cost and time estimation, and as input to security metrics and risk assessment methodologies. However, as of now, quantitative studies (a) require considerable manual effort, (b) make use of noisy datasets, and (c) are especially challenging to reproduce. In this poster abstract we describe our ongoing work towards quantitative analysis of vulnerabilities in Debian GNU/Linux packages. We focus on the challenges of making the process as automated and reproducible as possible, while collecting good-quality data necessary for the analysis. We then state a number of interesting hypotheses that can be investigated, and present preliminary results.

Today's commercial online social networks (COSNs) are under continuous debate for their lack of proper data and privacy protection; meanwhile, privacy-preserving online social networks (PPOSNs) struggle to attract a critical number of users. In this paper, we propose a hybrid solution that combines the best of both worlds in a seamless experience. Our hybrid online social network (HOSN) concept enables users to gradually transition parts of their online social experience to a more privacy- preserving network, while also considering the needs of the commercial providers. A reference implementation for Twitter with a detailed technical discussion proves the viability of this concept.

Emerging proactive applications need user data to power their underlying AI algorithms. However, both the training and inference tasks are typically performed in the provider's cloud, leading to multiple privacy issues to the data subject. Current privacy-preserving concepts are neither practicable for AI algorithms nor promise mutual benefits for both parties. To address these issues, we propose {P}Net---a novel two-level approach for privacy-preserving personalization, which exploits 'divisible' AI algorithms and anonymous inter-person similarity measurements. In short, {P}Net splits the training task into a cloud-based general model learning process and 'local' personalization steps in which the general 'black-box' model is subsequently adapted to individuals (level 1). Based on anonymously-contributed model modifications (patches) resulting from the first level, {P}Net further allows new users to request a community model---representing the general model personalized by crowd-sourced patches from other similar users (level 2). Our experiments show that {P}Net outperforms existing techniques especially when only a few user data is labeled. With {P}Net, new users can now directly benefit from personalized applications in a practical and privacy-preserving way with reduced labeling effort.

Many recommender systems use association rules mining, a technique that captures relations between user interests and recommends new probable ones accordingly. Applying association rule mining causes privacy concerns as user interests may contain sensitive personal information (e.g., political views). This potentially even inhibits the user from providing information in the first place. Current distributed privacy-preserving association rules mining (PPARM) approaches use cryptographic primitives that come with high computational and communication costs, rendering PPARM unsuitable for large-scale applications such as social networks. We propose improvements in the efficiency and privacy of PPARM approaches by minimizing the required data. We propose and compare sampling strategies to sample the data based on social graphs in a privacy-preserving manner. The results on real-world datasets show that our sampling-based approach can achieve a high average precision score with as low as 50% sampling rate and, therefore, with a 50% reduction of communication cost.

Emerging connected devices lead to ubiquitous communication in which anonymity and efficiency gain additional importance. In this paper, we show that current measures for sender anonymity are not sufficient and propose a new approach based on DCnets. The novel ADCnet mechanism establishes local DCnet groups that communicate asymmetrically and hide senders with lower communication overhead in comparison to cover traffic-based anonymization and classical DCnets. This paper presents concepts for the initialization and the group formation of ADCnets. The novel mechanism of ADCnets is evaluated w.r.t. anonymity and efficiency. We show that ADCnets provide DCnet-like anonymity while massively improving efficiency.

The ever-growing number of cyber attacks originating from botnets has made them one of the biggest threat to the Internet ecosystem. Especially P2P-based botnets like ZeroAccess and Sality require special attention as they have been proven to be very resilient against takedown attempts. To identify weaknesses and to prepare takedowns more carefully it is thus a necessity to monitor them by crawling and deploying sensor nodes. This in turn provokes botmasters to come up with monitoring countermeasures to protect their assets. Most existing anti-monitoring countermeasures focus mainly on the detection of crawlers and not on the detection of sensors deployed in a botnet. In this paper, we propose two sensor detection mechanisms called SensorRanker and SensorBuster. We evaluate these mechanisms in two real world botnets, Sality and ZeroAccess. Our results indicate that SensorRanker and SensorBuster are able to detect up to 17 sensors deployed in Sality and four within ZeroAccess.

Online Social Networks (OSNs) are the core of most communications nowadays, leading to possibly sensitive information exchange. Privacy is an important building block of free societies, and thus, for OSNs. OSNs function as group communication systems and can be build in centralised and distributed styles. Privacy can be achieved in distributed systems as all participants contribute to privacy. Peer-to-peer-based group communication systems achieve this privacy improvement partially, at the cost of additional messaging overhead. In this paper, we introduce ant colony optimisation to reduce the messaging overhead of anonymous communication systems, bridging the gap between privacy and efficiency. We apply our adapted privacy sensitive ant colony optimization to improve routing paths by encouraging re-usage and aggregation. Our first results indicate a 9-13% lower messaging overhead compared to the state of the art, while maintaining privacy.

Anonymity can protect from political repression in Online Social Networks (OSNs) as well as from undesired profiling, e.g., by advertisement companies, in todays' Internet. P2P-based anonymous publish-subscribe (pub-sub) is a highly-scalable approach to protect anonymity while enabling efficient many-to-many communication between services and users. However, churn and the resulting overlay degradation in P2P-based pub-sub systems require repairs and optimizations to maintain anonymity and efficiency. This paper analyzes attacks on such repair and optimization functions to disclose participants. For that, we apply a strong attacker model that combines large-scale traffic monitoring with malicious insiders. Furthermore, we propose and evaluate heuristic countermeasures. Our findings indicate that some attacks can be mitigated at reasonable costs. However, churn seems to remain a major threat to anonymity.