Adversarial patches pose a significant threat to computer vision models' integrity, decreasing the accuracy of various tasks, including object detection (OD). Most existing OD defenses exhibit a trade-off between enhancing the model's adversarial robustness and maintaining its performance on benign images. We propose KDAT (knowledge distillation with adversarial tuning), a novel mechanism that enhances the robustness of an OD model without compromising its performance on benign images or its inference time. Our method combines the knowledge distillation (KD) technique with the adversarial tuning concept to teach the model to match the predictions of adversarial images with those of their corresponding benign ones. To match these predictions, we designed four unique loss components, allowing the student model to effectively distill the knowledge of different features from various parts of the teacher model. Our extensive evaluation on the COCO and INRIA datasets demonstrates KDAT's ability to improve the performance of Faster R-CNN and DETR on benign images by 2-4 mAP% and adversarial examples by 10-15 mAP%, outperforming other state-of-the-art (SOTA) defenses. Furthermore, our additional physical evaluation on the Superstore dataset demonstrates KDAT's SOTA adversarial robustness against printed patches (improvement of 22 mAP% compared to the undefended model).

In fully managed serverless environments, cloud service providers handle the underlying infrastructure, reducing application developers’ operational and maintenance efforts. However, these environments limit the use of traditional cybersecurity frameworks and tools, compromising observability and situational awareness capabilities for security tasks (e.g., risk assessment, incident response). Additionally, existing security frameworks for serverless applications often lack generalizability across architectures and require specialized expertise. In this paper, we propose a three-layer security stack for fully managed serverless applications. The first layer establishes a foundational generic ontology that models serverless application resources and their interactions using API logs. In the second layer, the ontology is leveraged via perimeterless pipeline, to map the logs into a unified application activity KG, and in the third layer, two situational awareness tools that utilize the graph-based representation are implemented: (1) an incident response dashboard that leverages the ontology to visualize and examine application activity logs in the context of cybersecurity alerts; our user study showed that this dashboard enabled participants to respond 10% more accurately and almost twice as fast than the examined baseline tool, and (2) a criticality of asset (CoA) risk assessment framework that enables efficient expert-based prioritization in cybersecurity contexts; our expert-based questionnaire demonstrated strong agreement, achieving a Kendall-W score of 0.7179.

Serverless Function-as-a-Service (FaaS) environments enable developers to build and run cloud applications without the need to manage the underlying servers and computing infrastructure, allowing them to focus on implementing the application logic. Such environments contain numerous functions and dynamic resources, e.g., APIs and databases, making it challenging to gain insight and context of internal events i.e., recognize modules. Module in a serverless application is a set of functions and resources, that represents a functional unit that shares logical context. This paper presents SMART, a method for automatic analysis and recognition of modules for managed serverless applications. The proposed method creates an event-based graph by analyzing the standard serverless logs that document events involving the application’s functions and resources and utilizes well-known community detection algorithms (such as Louvain), with graph centrality metrics (such as degree centrality) to recognize the modules. SMART enables high-level visibility of the application’s structure and logical context which can facilitate security analysis and contribute to improved decision-making of incident response handlers, who typically do not have direct access to the application’s design and code, which can lead to challenges in fully understanding the system’s intricacies. We focused on the popular Amazon Web Services (AWS) Lambda serverless computing platform and evaluated the proposed method on three different demo applications (Airline Booking, VOD, and E-commerce). We compared SMART’s performance to four overlapping community detection algorithms and showed that it outperformed them in the task of module recognition, with a maximum improvement of 61% on the omega index metric compared to the Speaker-Listener Label Propagation algorithm. In addition, we demonstrate that the use of large language models (LLMs) with the knowledge gained by SMART can enrich security analysis insights.

Recent studies have demonstrated external information leakage (EIL) attacks which allow an attacker to infer various sensitive implicit properties related to a machine learning (ML) model’s training data. Most of those attacks assumed 1) a white-box scenario in which the attacker has complete access to the ML model, its structure, and its parameters, or 2) a black-box (alternatively gray-box) scenario with non-realistic requirements such as a high query budget or high computational resources for the attacker. In this paper, we propose two practical query-based (i.e., black-box) EIL attacks that target face recognition ML models and allow an attacker to infer sensitive implicit properties, such as the facial characteristics, gender, ethnicity, income level, and average age of the individuals in the training data, with a limited number of queries. The first proposed attack, referred to as the random noise injection (RNI) attack, exploits the effect of injecting random noise into input samples on the target model’s predictions. The second proposed attack, referred to as the property substitute model (PSM) attack, creates a substitute model for each property value examined, whose predictions are compared to the target model’s predictions. Our comprehensive evaluation (a total of 730 experiments) performed on the CelebA dataset shows that the proposed attacks outperform existing EIL attacks and successfully infer private information, posing a threat to the privacy and security of the face recognition models.

Phishing attacks are continually changing, so machine learning detection models must be continuously updated by collecting new URL data from users without compromising their privacy. Existing approaches for the detection of phishing URLs have unrealistic assumptions: (1) representative URL datasets are available in a centralized location, (2) users’ URL entries are labeled, (3) users’ unique behavioral patterns are ignored, and (4) users’ data are identically and independently distributed (IID data). This paper presents a semi-supervised, transfer-learning (TL), federated-learning (FL) approach for detecting phishing URL attacks, a novel approach that does not hold the above assumptions. We train a bidirectional long short-term memory (Bi-LSTM) autoencoder network across multiple decentralized edge devices (using FL) containing unlabeled data samples without sharing them (the process is privacy-preserving). A centralized server collects the updated Bi-LSTM autoencoder networks from the users' devices and aggregates them into a global Bi-LSTM autoencoder network using the FedAVG algorithm. The server then performs TL in order to use the autoencoder that learns the patterns from the global Bi-LSTM autoencoder networks and induces a classification model. The method is evaluated using three benchmark datasets and compared to state-of-the-art URL phishing detection methods that utilize centralized learning (CL) and FL. Our experiments show that our proposed approach achieves higher results based on the F1 score compared to the state-of-the-art method.

Utilizing existing methods for bias detection in machine learning (ML) models is challenging since each method: 1) explores a different ethical aspect of bias, which may result in contradictory output among the different methods; 2) provides output in a different range/scale and therefore cannot be compared with other methods; and 3) requires different input, thereby requiring a human expert's involvement to adjust each method according to the model examined. In this article, we present BENN, a novel bias estimation method that uses a pretrained unsupervised deep neural network. Given an ML model and data samples, BENN provides a bias estimation for every feature based on the examined model's predictions. We evaluated BENN using three benchmark datasets, one proprietary churn prediction model used by a European telecommunications company, and a synthetic dataset that includes both a biased feature and a fair one. BENN's results were compared with an ensemble of 21 existing bias estimation methods. The evaluation results show that BENN provides bias estimations that are aligned with those of the ensemble while offering significant advantages, including the fact that it is a generic approach (i.e., can be applied to any ML model) and does not require a domain expert.

Deep neural networks (DNNs) perform well on samples from the training distribution. However, DNNs deployed in the real world are exposed to out-of-distribution (OOD) samples, which refer to the samples from distributions that differ from the training distribution. OOD detection is indispensable to the DNNs as OOD samples can cause unexpected behaviors for them. This paper empirically explores the effectiveness of weight pruning of DNNs for OOD detection in a post-hoc setting (i.e., performing OOD detection based on pretrained DNN models). We conduct experiments on image, text, and tabular datasets to thoroughly evaluate OOD detection performance of weight-pruned DNNs. Our experimental results bring the following three novel findings: (i) Weight pruning improves OOD detection per-formance more significantly with a Mahalanobis distance-based detection approach, which performs OOD detection on DNN hidden representations using the Mahalanobis distance, than with logit-based detection approaches. (ii) Weight-pruned DNNs tend to extract global features of inputs, which improves the OOD detection on samples much dissimilar to the in-distribution samples. (iii) The weights that are useless for classification are often useful for OOD detection, and thus weight importance should not be quantified as the sensitivity of weights only to classification error. On the basis of these findings, we advocate practical techniques of DNN weight pruning that enable weight-pruned DNNs to maintain both OOD detection and classification capabilities.

A machine learning (ML) fairness estimator, which is used to assess an ML model’s fairness, should satisfy several conditions when used in real-life settings. Specifically, it should: i) support a comprehensive fairness evaluation that explores all ethical aspects; ii) be flexible and support different ML model settings; iii) enable comparison between different evaluations and ML models; and iv) provide reasoning and explanations for the fairness assessments produced. Existing methods do not sufficiently satisfy all of the above conditions. In this paper, we present FEPC (Fairness Estimation using Prototypes and Critics for tabular data), a novel method for fairness assessment that provides explanations and reasoning for its assessments by using an adversarial attack and customized fairness measurement. Given an ML model and data records, FEPC performs a comprehensive fairness evaluation and produces a fairness assessment for each examined feature. FEPC was evaluated using two benchmark datasets (ProPublica COMPAS and Statlog datasets) and a synthetic dataset containing two features, one of which is biased and one of which is fair, and compared to existing fairness assessment methods. The evaluation demonstrates that FEPC satisfies all of the conditions, making it suitable for real-life settings, and outperforms existing methods.

Machine learning (ML) models are commonly used to detect hate speech, which is considered one of the main challenges of online social networks. However, ML models have been shown to be vulnerable to well-crafted input samples referred to as adversarial examples. In this paper, we present an adversarial attack against hate speech detection models and explore the attack’s ability to: (1) prevent the detection of a hateful user, which should result in termination of the user’s account, and (2) classify normal users as hateful, which may lead to the termination of a legitimate user’s account. The attack is targeted at ML models that are trained on tabular, heterogeneous datasets (such as the datasets used for hate speech detection) and attempts to determine the minimal number of the most influential mutable features that should be altered in order to create a successful adversarial example. To demonstrate and evaluate the attack, we used the open and publicly available “Hateful Users on Twitter” dataset. We show that under a black-box assumption (i.e., the attacker does not have any knowledge on the attacked model), the attack has a 75% success rate, whereas under a white-box assumption (i.e., the attacker has full knowledge on the attacked model), the attack has an 88% success rate.