Autonomous vehicles must excel in safety-critical perception tasks, especially in adverse weather conditions. In addition, transitional weather shifts in nature, such as sunny to rainy, rainy to cloudy, etc., pose abrupt illumination changes that can distort object boundaries and degrade segmentation performance. Existing research focuses mainly on segmentation in clear and discrete weather conditions, leaving a gap in addressing the issues of transitional weather scenarios. Hence, we propose a novel method called causal road and rest segmentation (CaRS) that utilizes causal intervention to mitigate the confounding bias due to transitional weather changes. We use dual complementary attention modules, one for causal and another for confounding feature extraction. These modules complement each other and are fine-tuned via an adversarial min-max approach to reduce confounding bias and enhance segmentation performance. Also, our CaRS method concurrently performs road semantic segmentation and instance segmentation of vehicles and pedestrians. Further, we introduce a transitional weather-driving dataset for segmentation (TWDS16) using a spurious correlation generator that leverages data interpolation to produce 16 weather transitions. We evaluate the performance of CaRS on TWDS16, along with three other benchmark datasets, namely, Foggy Cityscapes, RainCityscapes, and BDD100K. The experimental results validate the efficacy of the proposed method in mitigating confounding influences, leading to improved mIoU for semantic segmentation and mAP for instance segmentation across diverse datasets.

Federated learning (FL) has emerged as a promising solution to enable distributed learning without sharing sensitive data. However, FL is vulnerable to data poisoning attacks, where malicious clients inject malicious data during training to compromise the global model. Existing FL defenses suffer from the assumptions of independent and identically distributed (IID) model updates, asymptotic optimal error rate bounds, and strong convexity in the optimization problem. Hence, we propose a novel framework called Federated Learning Optimal Transport (FLOT) that leverages the Wasserstein barycentric technique to obtain a global model from a set of locally trained non-IID models on client devices. In addition, we introduce a loss function-based rejection (LFR) mechanism to suppress malicious updates and a dynamic weighting scheme to optimize the Wasserstein barycentric aggregation function. We provide the theoretical proof of the Byzantine resilience and convergence of FLOT to highlight its efficacy. We evaluate FLOT on four benchmark datasets: GTSRB, KBTS, CIFAR10, and EMNIST. The experimental results underscore the practical significance of FLOT as an effective defense mechanism against data poisoning attacks in FL while maintaining high accuracy and scalability. Also, we observe that FLOT serves as a robust client selection technique under no attack, which demonstrates its effectiveness.

Ensuring auditability and verifiability in Federated Learning (FL) is both challenging and essential to guarantee that local data remains untampered and client updates are trustworthy. Recent FL frameworks assess client contributions through a trusted central server using various client selection and aggregation techniques. However, reliance on a central server can create a single point of failure, making it vulnerable to privacy-centric attacks and limiting its ability to audit and verify client-side data contributions due to restricted access. In addition, data quality and fairness evaluations are often inadequate, failing to distinguish between high-impact contributions and those from low-quality or poisoned data. To address these challenges, we propose Federated Auditable and Verifiable Data valuation (FAVD), a privacy-preserving method that ensures auditability and verifiability of client contributions through data valuation, independent of any central authority or predefined training algorithm. FAVD utilizes shared local data density functions to construct a global density function, aligning data contributions and facilitating effective valuation prior to local model training. This proactive approach improves transparency in data valuation and ensures that only benign updates are generated, even in the presence of malicious data. Further, to mitigate privacy risks associated with sharing data density functions, we add Gaussian noise to each client’s local density function before sharing it with the server. We theoretically demonstrate the convergence, auditability, and verifiability of FAVD, along with its resilience against data poisoning threats. Our experiments on five diverse benchmarks, including three medical datasets, show that FAVD achieves significant performance gains, accurate data valuation, and fair client contributions under threat, highlighting its reliability as a trustworthy FL approach.

Federated learning (FL) has emerged as a powerful collaborative learning approach that enables client devices to train a joint machine learning model without sharing private data. However, the decentralized nature of FL makes it highly vulnerable to adversarial attacks from multiple sources. There are diverse FL data poisoning and model poisoning attack methods in the literature. Nevertheless, most of them focus only on the attack’s impact and do not consider the attack budget and attack visibility. These factors are essential to effectively comprehend the adversary’s rationale in designing an attack. Hence, our work highlights the significance of considering these factors by providing an attacker perspective in designing an attack with a low budget, low visibility, and high impact. Also, existing attacks that use total neuron replacement and randomly selected neuron replacement approaches only cater to these factors partially. Therefore, we propose a novel federated learning minimal model replacement attack (FL-MMR) that uses optimal transport (OT) for minimal neural alignment between a surrogate poisoned model and the benign model. Later, we optimize the attack budget in a three-fold adaptive fashion by considering critical learning periods and introducing the replacement map. In addition, we comprehensively evaluate our attack under three threat scenarios using three large-scale datasets: GTSRB, CIFAR10, and EMNIST. We observed that our FL-MMR attack drops global accuracy to$\approx 35\%$less with merely 0.54% total attack budget and lower attack visibility than other attacks. The results confirm that our method aligns closely with the attacker’s viewpoint compared to other methods.

Federated Learning (FL) is a collaborative learning paradigm enabling participants to collectively train a shared machine learning model while preserving the privacy of their sensitive data. Nevertheless, the inherent decentralized and data-opaque characteristics of FL render its susceptibility to data poisoning attacks. These attacks introduce malformed or malicious inputs during local model training, subsequently influencing the global model and resulting in erroneous predictions. Current FL defense strategies against data poisoning attacks either involve a trade-off between accuracy and robustness or necessitate the presence of a uniformly distributed root dataset at the server. To overcome these limitations, we present FedZZ, which harnesses a zone-based deviating update (ZBDU) mechanism to effectively counter data poisoning attacks in FL. The ZBDU approach identifies the clusters of benign clients whose collective updates exhibit notable deviations from those of malicious clients engaged in data poisoning attack. Further, we introduce a precision-guided methodology that actively characterizes these client clusters (zones), which in turn aids in recognizing and discarding malicious updates at the server. Our evaluation of FedZZ across two widely recognized datasets: CIFAR10 and EMNIST, demonstrate its efficacy in mitigating data poisoning attacks, surpassing the performance of prevailing state-of-the-art methodologies in both single and multi-client attack scenarios and varying attack volumes. Notably, FedZZ also functions as a robust client selection strategy, even in highly non-IID and attack-free scenarios. Moreover, in the face of escalating poisoning rates, the model accuracy attained by FedZZ displays superior resilience compared to existing techniques. For instance, when confronted with a 50% presence of malicious clients, FedZZ sustains an accuracy of 67.43%, while the accuracy of the second-best solution, FL-Defender, diminishes to 43.36%.

Federated Learning (FL) facilitates clients to collaborate on training a shared machine learning model without exposing individual private data. Nonetheless, FL remains susceptible to utility and privacy attacks, notably evasion data poisoning and model inversion attacks, compromising the system's efficiency and data privacy. Existing FL defenses are often specialized to a particular single attack, lacking generality and a comprehensive defender's perspective. To address these challenges, we introduce Federated Cryptography Defense (FCD), a unified single framework aligning with the defender's perspective. FCD employs row-wise transposition cipher based data encryption with a secret key to counter both evasion black-box data poisoning and model inversion attacks. The crux of FCD lies in transferring the entire learning process into an encrypted data space and using a novel distillation loss guided by the Kullback-Leibler (KL) divergence. This measure compares the probability distributions of the local pretrained teacher model's predictions on normal data and the local student model's predictions on the same data in FCD's encrypted form. By working within this encrypted space, FCD eliminates the need for decryption at the server, resulting in reduced computational complexity. We demonstrate the practical feasibility of FCD and apply it to defend against evasion utility attack on benchmark datasets (GTSRB, KBTS, CIFAR10, and EMNIST). We further extend FCD for defending against model inversion attack in split FL on the CIFAR100 dataset. Our experiments across the diverse attack and FL settings demonstrate practical feasibility and robustness against utility evasion (impact > 30) and privacy attacks (MSE > 73) compared to the second best method.

Navigating safely and dependably through challenging weather conditions poses a significant hurdle for autonomous vehicles (AVs). While state-of-the-art object detection models have demonstrated superior performance on standard benchmark datasets, their accuracy is compromised by visual variations introduced by adverse weather conditions. In addition, we naturally observe the continuous shifts between discrete weather conditions (cloudy to rainy, rainy to sunny, etc.), with variation in different levels of adversity. The current object detection research predominantly concentrates on identifying objects in discrete weather conditions (extremely cloudy, rainy, etc.). However, there is a lack of emphasis on continuous shifts between these stationary weather conditions. In response to this challenge, we introduce a pioneering solution, the Multi-Scale Adaptive Transformer (mSAT). This innovative approach amalgamates a Domain Adaptive Network (DAN), adept at identifying continuous weather-invariant features across various scales, with a transformer network tailored for object detection. Our method is evaluated on the AIWD6 dataset, showcasing its efficacy in addressing the impact of adverse weather conditions on object detection. Our approach effectively mitigates the domain discrepancy, enabling adaptation to various continuous weather shifts. Later, we introduce three novel metrics for evaluating object detection performance on continuous weather data along with standard metrics. Our proposed mSAT, designed to operate on various intensity levels of weather with unlabeled target data, achieves 74.6 mAP on the AIWD6 dataset. Experimental results demonstrate that our model adapts to continuous weather shifts and effectively performs object detection.

Federated learning (FL) has emerged as a powerful machine learning technique that enables the development of models from decentralized data sources. However, the decentralized nature of FL makes it vulnerable to adversarial attacks. In this survey, we provide a comprehensive overview of the impact of malicious attacks on FL by covering various aspects such as attack budget, visibility, and generalizability, among others. Previous surveys have primarily focused on the multiple types of attacks and defenses but failed to consider the impact of these attacks in terms of their budget, visibility, and generalizability. This survey aims to fill this gap by providing a comprehensive understanding of the attacks' effect by identifying FL attacks with low budgets, low visibility, and high impact. Additionally, we address the recent advancements in the field of adversarial defenses in FL and highlight the challenges in securing FL. The contribution of this survey is threefold: first, it provides a comprehensive and up-to-date overview of the current state of FL attacks and defenses. Second, it highlights the critical importance of considering the impact, budget, and visibility of FL attacks. Finally, we provide ten case studies and potential future directions towards improving the security and privacy of FL systems.

Driving in adverse weather conditions is a key challenge for autonomous vehicles (AV). Typical scene perception models perform poorly in rainy, foggy, snowy, and cloudy conditions. In addition, we observe transition states between extremes (cloudy to rainy, rainy to sunny, etc.) in nature with variations in adversity. It is crucial to define and understand these transition states in order to develop robust AV perception models. Existing research works on classification focused on identifying extreme weather conditions. However, there is a lack of emphasis on the transition between these extreme weather scenes. Hence, this paper proposes an approach to define and understand six intermediate weather transition states: sunny to rainy, rainy to sunny, and others. Firstly, we propose a way to interpolate the intermediate weather transition data using a variational autoencoder and extract its spatial features using VGG. Further, we model the temporal distribution of these spatial features using a gated recurrent unit to classify the corresponding transition state. Also, we introduce a large-scale dataset called the AIWD6: Adverse Intermediate Weather Driving dataset, generated for three different time intervals. Experimental results on the AIWD6 dataset demonstrate that our model efficiently generates weather transition conditions for AV technology. Also, the spatio-temporal deep neural network can effectively classify the adverse weather transition states for different time intervals.