Deep neural networks (DNNs) are increasingly being deployed in high-stakes applications, from self-driving cars to biometric authentication. However, their unpredictable and unreliable behaviors in real-world settings require new approaches to characterize and ensure their reliability. This paper introduces DeepProv, a novel and customizable system designed to capture and characterize the runtime behavior of DNNs during inference by using their underlying graph structure. Inspired by system audit provenance graphs, DeepProv models the computational information flow of a DNN's inference process through Inference Provenance Graphs (IPGs). These graphs provide a detailed structural representation of the behavior of DNN, allowing both empirical and structural analysis. DeepProv uses these insights to systematically repair DNNs for specific objectives, such as improving robustness, privacy, or fairness. We instantiate DeepProv with adversarial robustness as the goal of model repair and conduct extensive case studies to evaluate its effectiveness. Our results demonstrate its effectiveness and scalability across diverse classification tasks, attack scenarios, and model complexities. DeepProv automatically identifies repair actions at the node and edge-level within IPGs, significantly enhancing the robustness of the model. In particular, applying DeepProv repair strategies to just a single layer of a DNN yields an average 55% improvement in adversarial accuracy. Moreover, DeepProv complements existing defenses, achieving substantial gains in adversarial robustness. Beyond robustness, we demonstrate the broader potential of DeepProv as an adaptable system to characterize DNN behavior in other critical areas, such as privacy auditing and fairness analysis.

Relying on untrusted data exposes machine learning models to backdoor attacks, where adversaries poison training data to embed hidden behaviors.Existing defenses struggle against increasingly stealthy attacks, particularly clean-label backdoor attacks, due to their inability to monitor fine-grained impact of individual training samples on model updates.In this paper, we present PoisonSpot, a novel system that precisely detects clean-label backdoor attacks by using fine-grained training provenance tracking, inspired by dynamic taint tracking.PoisonSpot captures and analyzes the impact of individual training samples on model parameter updates throughout the training process.By attributing poisoning scores to suspect samples based on their impact lineage, PoisonSpot allows for accurate identification and rejection of samples carrying backdoor triggers.We evaluate PoisonSpot on multiple benchmark datasets and attack scenarios, demonstrating its superior performance compared to the state-of-the-art clean-label backdoor poisoning defense.Poi-sonSpot consistently achieves high true positive rates, low false positive rates, and effectively mitigates backdoor attacks, even under adaptive adversarial strategies.Furthermore, PoisonSpot operates efficiently in various training settings, including retraining and fine-tuning regimes, demonstrating its robustness and scalability.

In membership inference attacks (MIAs), an adversary observes the predictions of a model to determine whether a sample is part of the model’s training data. Existing MIA defenses conceal the presence of a target sample through strong regularization, knowledge distillation, confidence masking, or differential privacy. We propose MIAShield, a new MIA defense based on preemptive exclusion of member samples instead of masking the presence of a member. MIAShield departs from prior defenses in that it weakens the strong membership signal that stems from the presence of a target sample by preemptively excluding it at prediction time without compromising model utility. To that end, we design and evaluate a suite of preemptive exclusion oracles leveraging model confidence, exact/approximate sample signature, and learning-based exclusion of member data points. To be practical, MIAShield splits a training data into disjoint subsets and trains each subset to build an ensemble of models. The disjointedness of subsets ensures that a target sample belongs to only one subset, which isolates the sample to facilitate the preemptive exclusion goal. We evaluate MIAShield on three benchmark image classification datasets. We show that MIAShield effectively mitigates membership inference (near random guess) for a wide range of MIAs; achieves far better privacy/utility trade-off compared with state-of-the-art defenses; and remains resilient in the face of adaptive attacks.

The widespread usage of machine learning (ML) in a myriad of domains has raised questions about its trustworthiness in high-stakes environments. Part of the quest for trustworthy ML is assessing robustness to test-time adversarial examples. Inline with the trustworthy ML goal, a useful input to potentially aid robustness evaluation is feature-based explanations of model predictions. In this paper, we present a novel approach, called EG-Booster, that leverages techniques from explainable ML to guide adversarial example crafting for improved robustness evaluation of ML models. The key insight in EG-Booster is the use of feature-based explanations of model predictions to guide adversarial example crafting by adding consequential perturbations (likely to result in model evasion) and avoiding non-consequential perturbations (unlikely to contribute to evasion). EG-Booster is agnostic to model architecture, threat model, and supports diverse distance metrics used in the literature. We evaluate EG-Booster using image classification benchmark datasets: MNIST and CIFAR10. Our findings suggest that EG-Booster significantly improves the evasion rate of state-of-the-art attacks while performing a smaller number of perturbations. Through extensive experiments that cover four white-box and three black-box attacks, we demonstrate the effectiveness of EG-Booster against two undefended neural networks trained on MNIST and CIFAR10, and an adversarially-trained ResNet model trained on CIFAR10. Furthermore, we introduce a stability assessment metric and evaluate the reliability of our explanation-based attack boosting approach by tracking the similarity between the model's predictions across multiple runs of EG-Booster. Our results over 10 separate runs suggest that EG-Booster's output is stable across distinct runs. Combined with state-of-the-art attacks, we hope EG-Booster will be used towards improved robustness assessment of ML models against evasion attacks.

Differential Privacy (DP) has emerged as a rigorous formalism to quantify privacy protection provided by an algorithm that operates on privacy sensitive data. In machine learning (ML), DP has been employed to limit inference/disclosure of training examples. Prior work leveraged DP across the ML pipeline, albeit in isolation, often focusing on mechanisms such as gradient perturbation.

Robustness to adversarial examples of machine learning models remains an open topic of research. Attacks often succeed by repeatedly probing a fixed target model with adversarial examples purposely crafted to fool it. In this paper, we introduce Morphence, an approach that shifts the defense landscape by making a model a moving target against adversarial examples. By regularly moving the decision function of a model, Morphence makes it significantly challenging for repeated or correlated attacks to succeed. Morphence deploys a pool of models generated from a base model in a manner that introduces sufficient randomness when it responds to prediction queries. To ensure repeated or correlated attacks fail, the deployed pool of models automatically expires after a query budget is reached and the model pool is seamlessly replaced by a new model pool generated in advance. We evaluate Morphence on two benchmark image classification datasets (MNIST and CIFAR10) against five reference attacks (2 white-box and 3 black-box). In all cases, Morphence consistently outperforms the thus-far effective defense, adversarial training, even in the face of strong white-box attacks, while preserving accuracy on clean data and reducing attack transferability.

Cyber threat intelligence (CTI) is being used to search for indicators of attacks that might have compromised an enterprise network for a long time without being discovered. To have a more effective analysis, CTI open standards have incorporated descriptive relationships showing how the indicators or observables are related to each other. However, these relationships are either completely overlooked in information gathering or not used for threat hunting. In this paper, we propose a system, called POIROT, which uses these correlations to uncover the steps of a successful attack campaign. We use kernel audits as a reliable source that covers all causal relations and information flows among system entities and model threat hunting as an inexact graph pattern matching problem. Our technical approach is based on a novel similarity metric which assesses an alignment between a query graph constructed out of CTI correlations and a provenance graph constructed out of kernel audit log records. We evaluate POIROT on publicly released real-world incident reports as well as reports of an adversarial engagement designed by DARPA, including ten distinct attack campaigns against different OS platforms such as Linux, FreeBSD, and Windows. Our evaluation results show that POIROT is capable of searching inside graphs containing millions of nodes and pinpoint the attacks in a few minutes, and the results serve to illustrate that CTI correlations could be used as robust and reliable artifacts for threat hunting.

In this paper, we present HOLMES, a system that implements a new approach to the detection of Advanced and Persistent Threats (APTs). HOLMES is inspired by several case studies of real-world APTs that highlight some common goals of APT actors. In a nutshell, HOLMES aims to produce a detection signal that indicates the presence of a coordinated set of activities that are part of an APT campaign. One of the main challenges addressed by our approach involves developing a suite of techniques that make the detection signal robust and reliable. At a high-level, the techniques we develop effectively leverage the correlation between suspicious information flows that arise during an attacker campaign. In addition to its detection capability, HOLMES is also able to generate a high-level graph that summarizes the attacker's actions in real-time. This graph can be used by an analyst for an effective cyber response. An evaluation of our approach against some real-world APTs indicates that HOLMES can detect APT campaigns with high precision and low false alarm rate. The compact high-level graphs produced by HOLMES effectively summarizes an ongoing attack campaign and can assist real-time cyber-response operations.

Web-borne malware continues to be a major threat on the Web. At the core of malware infection are for-crime toolkits that exploit vulnerabilities in browsers and their extensions. When a victim host gets infected, the infection dynamics is often buried in benign traffic, which makes the task of inferring malicious behavior a non-trivial exercise. In this paper, we leverage web conversation graph analytics to tap into the rich dynamics of the interaction between a victim and malicious host(s) without the need for analyzing exploit payload. Based on insights derived from infection graph analytics, we formulate the malware detection challenge as a graph-analytics based learning problem. The key insight of our approach is the payload-agnostic abstraction and comprehensive analytics of malware infection dynamics pre-, during-, and post-infection. Our technique leverages 3 years of infection intelligence spanning 9 popular exploit kit families. Our approach is implemented in a tool called DynaMiner and evaluated on infection and benign HTTP traffic. DynaMiner achieves a 97.3% true positive rate with false positive rate of 1.5%. Our forensic and live case studies suggest the effectiveness of comprehensive graph abstraction malware infection. In some instances, DynaMiner detected unknown malware 11 days earlier than existing AV engines.

We tackle the problem of automated exploit generation for web applications. In this regard, we present an approach that significantly improves the state-of-art in web injection vulnerability identification and exploit generation. Our approach for exploit generation tackles various challenges associated with typical web application characteristics: their multi-module nature, interposed user input, and multi-tier architectures using a database backend. Our approach develops precise models of application workflows, database schemas, and native functions to achieve high quality exploit generation. We implemented our approach in a tool called Chainsaw. Chainsaw was used to analyze 9 open source applications and generated over 199 first- and second-order injection exploits combined, significantly outperforming several related approaches.

Organized cybercrime on the Internet is proliferating due to exploit kits. Attacks launched through these kits include drive-by-downloads, spam and denial-of-service. In this paper, we tackle the problem of detecting whether a given URL is hosted by an exploit kit. Through an extensive analysis of the workflows of about 40 different exploit kits, we develop an approach that uses machine learning to detect whether a given URL is hosting an exploit kit. Central to our approach is the design of distinguishing features that are drawn from the analysis of attack-centric and self-defense behaviors of exploit kits. This design is based on observations drawn from exploit kits that we installed in a laboratory setting as well as live exploit kits that were hosted on the Web. We discuss the design and implementation of a system called WEBWINNOW that is based on this approach. Extensive experiments with real world malicious URLs reveal that WEBWINNOW is highly effective in the detection of malicious URLs hosted by exploit kits with very low false-positives.

Most existing work to thwart malicious web pages capture maliciousness via discriminative artifacts, learn a model, and detect by leveraging static and/or dynamic analysis. Unfortunately, there is a two-sided evolution of the artifacts of web pages. On one hand, cybercriminals constantly revamp attack payloads in malicious web pages. On the other hand, benign web pages evolve to improve content rendering and interaction with users. Consequently, the onceprecise detection techniques suffer from limitations to cope with the evolution, resulting in malicious web pages that escape detection. In this paper, we present EINSPECT, an evolution-aware and learning-based approach to address evolution of web page artifacts to more precisely analyze and detect malicious web pages. EINSPECT continuously tunes its detection models to automatically decide the best interplay of features and learning algorithms to embrace the evolution of web page artifacts into the analysis and detection. We have implemented and evaluated our approach and the results show that EINSPECT is able to improve the effectiveness of analysis and detection ofmalicious web pages while aligning the detection models with the continuous evolution of web page artifacts.

This paper presents a web-based tool to supplement defense against security misconfiguration vulnerabilities in web applications. The tool automatically audits security configuration settings of server environments in web application development and deployment. It also offers features to automatically adjust security configuration settings and quantitatively rates level of safety for server environments before deploying web applications. Using the tool, we were able to evaluate eleven server packages for Apache, PHP and MySQL across three operating system platforms. Our evaluation revealed that the tool is able to audit current security configuration settings and alert users to fix the server environment to achieve the level of safety of security configuration with respect to recommended configurations for real-life web application deployment.

Intrusion detection systems are deployed on hosts in a computing infrastructure to tackle undesired events in the course of usage of the systems. One of the promising domains of applying intrusion detection is the healthcare domain. A typical healthcare scenario is characterized by high degree of mobility, frequent interruptions and above all demands access to sensitive medical records by concerned stakeholders. Migrating this set of concerns in pervasive healthcare environments where the traditional characteristics are more intensified in terms of uncertainty, one ends up with more challenges on security due to nature of pervasive devices and wireless communication media along with classic security problems for desktop based systems. Despite evolution of automated healthcare services and sophistication of attacks against such services, there is a reasonable lack of techniques, tools and experimental setups for protecting hosts against intrusive actions. This paper presents a contribution to provide a host-based, anomaly modeling and detection approach based on data mining techniques for pervasive healthcare systems. The technique maintains normal usage profile of pervasive healthcare applications and inspects current workflow against normal usage profile so as to classify it as anomalous or normal. The technique is implemented as a prototype with sample data set and the results obtained revealed that the technique is able to perform classification of anomalous activities.

Emerging technologies like mobile and wireless communication are offering promising opportunities to enable mobile healthcare delivery to citizens. But, enhancing the quality of service of such systems demands systematic improvement of existing service infrastructure. In this paper, we describe a context information refinement architecture proposed to address the shortcomings of existing works in relation to context information refinement in pervasive medical systems. The shortcomings are lack of adequate consideration for: quality parameters of context information, relevance of context information and particular requirements of the pervasive healthcare domain. The proposed architecture facilitates and coordinates the refinement of context information starting from acquisition of context information up until the refined context information is delivered to the target application in a pervasive medical system. We developed a prototype that implements the core components of the proposed architecture and evaluated it with real-life pervasive healthcare scenario and proved the validity of the architecture using a real-life scenario.

ICT seems well understood as a tool and an infrastructure for delivering information and services for the society and for allowing communications through interactions among the service users -mostly, the digital society. Using ICT for ensuring better life requires far more than good infrastructure, ICT know-how and the various techniques and tools in place. If ICT has to address the real problems of the society, it should be at a rescue being environment-friendly, with real and tangible impact, sustainable, seamless, down to the grass-roots and above all with reproducible experiences. In this paper, we introduce a different perspective of looking into and using ICT, which we call ICT for Good (ICT4G). It is about using ICT for addressing problems of societies with low ICT penetration and changing a society's life for the better. More specifically, based on our observation of current promises ICT gives to society, we discuss ICT4G's distinguishing aspects, opportunities it offers, challenges it imposes along with preliminary roadmap for its realization. A high-level correlation of what we pointed out with a relevant case study (i.e., the eGIF4M1) is presented.