Deep reinforcement learning (DRL) algorithms have been widely explored to enhance connected and automated vehicle (CAV) decision-making in complex driving environments such as freeway weaving segments, with various methods being developed to model vehicle interactions. However, different vehicle interaction characteristics caused by heterogenous driving intention and varying relative positions, were not modeled in existing DRL algorithms. In this research, we present an innovative approach that incorporates different interaction types between CAVs and their neighboring vehicles into the decision-making process. Using Relational Graph Attention Networks (R-GAT), this method captures the heterogeneous interaction dynamics by applying a novel interaction type-specific transformation to vehicle interaction features (e.g., relative time headway) and modified graph attention mechanism. Afterward, the R-GAT is integrated with a Deep Q-network (DQN) to make driving decisions for CAVs. Experimental results demonstrated that the proposed model outperforms traditional Graph Attention Networks (GAT) and DQN in various traffic scenarios with higher mean rewards, reducing 30%-48% conflicts in dense traffic. The proposed model is also evaluated by comparing the model-controlled driving behavior with field trajectories, and it was proven that the algorithm is capable of completing the driving tasks at weaving segments with a more stable speed profile than the human driver trajectories. The model can be transferred to various driving tasks at different locations by modifying the graph features and DRL settings, highlighting its potential for broader applications in CAV decision-making problems.

We introduce a real-time crash safety system capable of predicting crashes 5-10 minutes in advance, using advanced machine learning techniques. The system integrates primary, severity, and secondary crash predictions into a unified framework, employing a multi-layered prediction strategy that enhances proactive traffic safety management by providing robust and actionable alerts. It uses a comprehensive segment-specific approach, utilizing well-established machine learning models suited to the unique characteristics of different road segment types, such as basic, merge, diverge, and ramp. Additionally, route-specific models capture regional variations in traffic patterns and environmental factors, ensuring that the system adapts to the unique characteristics of different routes. Performance evaluations demonstrate that the system achieves high sensitivity (0.798-0.918) and low false alarm rates (0.088-0.208), depending on the segment type. The use of the continuity concept ensures the consistency of high crash likelihood situations, which improves the reliability of predictions. The results show that the system can accurately predict crashes and provide proper warnings. It can make traffic operators to take proactive actions such as adjusting traffic management measures. By predicting both the occurrence and severity of crashes, as well as the likelihood of secondary crashes, the system enhances the overall efficiency and effectiveness of traffic crash management. This comprehensive approach to crash prediction represents a significant advancement in proactive traffic management and road safety enhancement.

Diffusion models (DMs) embark a new era of generative modeling and offer more opportunities for efficient generating high-quality and realistic data samples. However, their widespread use has also brought forth new challenges in model security, which motivates the creation of more effective adversarial attackers on DMs to understand its vulnerability. We propose CAAT, a simple but generic and efficient approach that does not require costly training to effectively fool latent diffusion models (LDMs). The approach is based on the observation that cross-attention layers exhibits higher sensitivity to gradient change, allowing for leveraging subtle perturbations on published images to significantly corrupt the generated images. We show that a subtle perturbation on an image can significantly impact the cross-attention layers, thus changing the mapping between text and image during the fine-tuning of customized diffusion models. Extensive experiments demonstrate that CAAT is compatible with diverse diffusion models and out-performs baseline attack methods in a more effective (more noise) and efficient (twice as fast as Anti-DreamBooth and Mist) manner.

We consider the problem of real-time log anomaly detection for distributed system with deep neural networks by unsupervised learning. There are two challenges in this problem, including detection accuracy and analysis efficacy. To tackle these two challenges, we propose GLAD, a simple yet effective approach mining for anomalies in distributed systems. To ensure detection accuracy, we exploit the gradient features in a well-calibrated deep neural network and analyze anomalous pattern within log files. To improve the analysis efficacy, we further integrate one-class support vector machine (SVM) into anomalous analysis, which significantly reduces the cost of anomaly decision boundary delineation. This effective integration successfully solves both accuracy and efficacy in real-time log anomaly detection. Also, since anomalous analysis is based upon unsupervised learning, it significantly reduces the extra data labeling cost. We conduct a series of experiments to justify that GLAD has the best comprehensive performance balanced between accuracy and efficiency, which implies the advantage in tackling practical problems. The results also reveal that GLAD enables effective anomaly mining and consistently outperforms state-of-the-art methods on both recall and F1 scores.

Large-scale real-time crash prediction is critical to traffic operation and management, but very challenging, even for machine learning models because the observation data are not independent and identically distributed (non-IID) due to its spatial heterogeneity. Data discontinuity, computational inefficiency, and higher false alarm rates are some other challenges in real-world and real-time crash prediction. To address these issues, we propose a generic framework, which blends spatial ensemble learning and knowledge distillation. Spatial ensemble learning leverages individual segment-level modeling to alleviate the non-IID issue for more accurate crash prediction, while knowledge distillation reduces the model size and improves false alarm rate. We justify the effectiveness of the proposed method using real-world and real-time scenario and comparing the results with the up-to-date benchmark models. Our method successfully improves sensitivity and false alarm rate to 90.35% and 24.21%. With this more accurate prediction model, we analyze the prediction variability across this region. We found that: 1) across segments, false alarm rate exhibits more heterogenous pattern than sensitivity; 2) rear-end crashes are more accurately predicted compared to sideswipe/angle crashes; 3) urban segments show better prediction performance compared to rural segments. 4) developing models with desired accuracy requires special attention in higher traffic fluctuation segments. These observations are very useful to develop more accurate prediction model and traffic safety decision making. To the best of our knowledge, this is one of the pioneering studies to integrate spatial ensemble learning and knowledge distillation to predict large-scale real-time crashes and apply it to analyze crash prediction variability.

We study the problem of semantic segmentation calibration. Lots of solutions have been proposed to approach model miscalibration of confidence in image classification. However, to date, confidence calibration research on se-mantic segmentation is still limited. We provide a system-atic study on the calibration of semantic segmentation models and propose a simple yet effective approach. First, we find that model capacity, crop size, multi-scale testing, and prediction correctness have impact on calibration. Among them, prediction correctness, especially misprediction, is more important to miscalibration due to over-confidence. Next, we propose a simple, unifying, and effective approach, namely selective scaling, by separating correct/incorrect prediction for scaling and more focusing on misprediction logit smoothing. Then, we study popular existing cali-bration methods and compare them with selective scaling on semantic segmentation calibration. We conduct exten-sive experiments with a variety of benchmarks on both in-domain and domain-shift calibration and show that selective scaling consistently outperforms other methods.

Logs are valuable resources that record the health status of systems. Analyzing logs to uncover and investigate abnormal behaviors has become an essential approach of ensuring system security. However, some potential anomalies may be missed when performing log anomaly detection, and even a seemingly insignificant abnormal behavior can lead to a series of severe anomalies or continuous negative impact on the performance of systems. Therefore, the reliability and security of systems are facing significant challenges. To address this issue, in this study, we propose TransFlowLog, an Encoder-Decoder architecture-based approach for log anomaly detection. It utilizes the Transformer Encoder, a state-of-the-art sequence modeling technique, to comprehensively understand contextual relationships using self-attention mechanism. Moreover, we introduce the Interflow Decoder, which considers information exchange between channels in embedded log sequences. The Interflow Decoder enhances the features encoded by the Transformer Encoder in both sequence and channel dimensions, thereby capturing interdependence between different channels. Comparative experiments conducted on three real-world datasets demonstrate the effectiveness of the proposed method, as it achieves higher F1-score and reduces the number of false negatives.

Syslog-based anomaly detection is crucial for protecting the systems from malicious attacks or malfunctions. System logs are semi-structured text messages printed by logging statements to record the system’s run-time status, involving rich semantic information. However, the existing BERT-based log anomaly detection method is based on the log key sequence, does not consider the semantics of the log data, and discards the variable part, resulting in a high rate of missed detection. In this paper, we propose SemLog, a self-supervised framework for log anomaly detection based on BERT. By incorporating log semantics and variables and employing multi-feature fusion, we mitigate the independent assumption issue in the Masked Language Modeling model. The experimental results on three benchmarks show that SemLog achieves high performance compared with the state-of-the-art approaches for anomaly detection.

Out-of-distribution detection is critical to a reliable application of deep neural networks. To reduce model uncertainty, we propose a simple yet effective approach of ensemble knowledge distillation. We blend ensemble model and knowledge distillation to improve model generalization on indomain recognition, thereby yielding accurate and robust out-of-distribution detection. The former effectively expands data recognition feature space, while the latter further regularizes the model through knowledge distillation, enhancing in-domain feature recognition. This effective integration successfully yields lower model uncertainty on in-domain feature recognition and improves anomaly detection in a more scalable manner. We justify our approach through extensive experiments on various benchmarks, demonstrating its significant improvement in out- of-distribution detection. We validate our approach with a variety of up-to-date DNNs, like Vision Transformer.

Deep neural networks (DNNs) have been widely employed to solve log anomaly detection and outperform a range of conventional methods. They have attained such striking success because they can usually explore and extract semantic information from a large volume of log data, which helps to infer complex log anomaly patterns more accurately. Despite its success in generalization accuracy, this data-driven approach can still suffer from a high vulnerability to adversarial attacks , which severely limits its practical use. To address this issue, several studies have proposed anomaly detectors to equip neural networks to improve their robustness. These anomaly detectors are built based on effective adversarial attack methods. Therefore, effective adversarial attack approaches are important for developing more efficient anomaly detectors, thereby improving neural network robustness. In this study, we propose two strong and effective black-box attackers, an attention-based and a gradient-based attacker, to defeat three target systems: MLP, AutoEncoder , and DeepLog. Our approach facilitates the generation of more effective adversarial examples with the help of the analysis of vulnerable logkeys. The proposed attention-based attacker leverages attention weights to achieve vulnerable logkeys and derive adversarial examples, which are implemented using our previously developed attention-based convolutional neural network model . The proposed gradient-based attacker calculates gradients based on potential vulnerable logkeys to seek an optimal adversarial sample. The experimental results showed that these two approaches significantly outperformed the state-of-the-art attacker model log anomaly mask (LAM). In particular, owing to its optimization, the proposed gradient-based attacker approach can significantly increase the misclassification rate on three target models, yields a 70% successful attack rate on DeepLog and greatly exceeds the baseline by 52%.

Abstract With versatility and complexity of computer systems, warning and errors are inevitable. To effectively monitor system’s status, system logs are critical. To detect anomalies in system logs, deep learning is a promising way to go. However, abnormal system logs in the real world are often difficult to collect, and effectively and accurately categorize the logs is an even time-consuming project. Thus, the data incompleteness is not conducive to the deep learning for this practical application. In this paper, we put forward a novel semi-supervised dual branch model that alleviate the need for large scale labeled logs for training a deep system log anomaly detector. Specifically, our model consists of two homogeneous networks that share the same parameters, one is called weak augmented teacher model and the other is termed as strong augmented student model. In the teacher model, the log features are augmented with small Gaussian noise, while in the student model, the strong augmentation is injected to force the model to learn a more robust feature representation with the guidance of teacher model provided soft labels. Furthermore, to further utilize unlabeled samples effectively, we propose a flexible label screening strategy that takes into account the confidence and stability of pseudo-labels. Experimental results show favorable effect of our model on prevalent HDFS and Hadoop Application datasets. Precisely, with only 30% training data labeled, our model can achieve the comparable results as the fully supervised version.

An accurate and efficient forecasting system is imperative to the prevention of emerging infectious diseases such as COVID-19 in public health. This system requires accurate transient modeling, lower computation cost, and fewer observation data. To tackle these three challenges, we propose a novel deep learning approach using black-box knowledge distillation for both accurate and efficient transmission dynamics prediction in a practical manner. First, we leverage mixture models to develop an accurate, comprehensive, yet impractical simulation system. Next, we use simulated observation sequences to query the simulation system to retrieve simulated projection sequences as knowledge. Then, with the obtained query data, sequence mixup is proposed to improve query efficiency, increase knowledge diversity, and boost distillation model accuracy. Finally, we train a student deep neural network with the retrieved and mixed observation-projection sequences for practical use. The case study on COVID-19 justifies that our approach accurately projects infections with much lower computation cost when observation data are limited.

We study how to train a student deep neural network for visual recognition by distilling knowledge from a blackbox teacher model in a data-efficient manner. Progress on this problem can significantly reduce the dependence on large-scale datasets for learning high-performing visual recognition models. There are two major challenges. One is that the number of queries into the teacher model should be minimized to save computational and/or financial costs. The other is that the number of images used for the knowledge distillation should be small; otherwise, it violates our expectation of reducing the dependence on large-scale datasets. To tackle these challenges, we propose an approach that blends mixup and active learning. The former effectively augments the few unlabeled images by a big pool of synthetic images sampled from the convex hull of the original images, and the latter actively chooses from the pool hard examples for the student neural network and query their labels from the teacher model. We validate our approach with extensive experiments.