Emerging from the pairwise attention in conventional Transformers, there is a growing interest in sparse attention mechanisms that align more closely with localized, contextual learning in the biological brain. Existing studies such as the Coordination method employ iterative cross-attention mechanisms with a bottleneck to enable the sparse association of inputs. However, these methods are parameter inefficient and fail in more complex relational reasoning tasks. To this end, we propose Associative Transformer (AiT) to enhance the association among sparsely attended input tokens, improving parameter efficiency and performance in various vision tasks such as classification and relational reasoning. AiT leverages a learnable explicit memory comprising specialized priors that guide bottleneck attentions to facilitate the extraction of diverse localized tokens. Moreover, AiT employs an associative memory-based token reconstruction using a Hopfield energy function. The extensive empirical experiments demonstrate that AiT requires significantly fewer parameters and attention layers outperforming a broad range of sparse Transformer models. Additionally, AiT outperforms the SOTA sparse Transformer models including the Coordination method on the Sort-of-CLEVR dataset.

Visual Question Answering (VQA) based on multi-modal data facilitates real-life applications such as home robots and medical diagnoses. One significant challenge is to devise a robust decentralized learning framework for various client models where centralized data collection is refrained due to confidentiality concerns. This work aims to tackle privacy-preserving VQA by decoupling a multi-modal model into representation modules and a contrastive module, leveraging inter-module gradients sharing and inter-client weight sharing. To this end, we propose Bidirectional Contrastive Split Learning (BiCSL) to train a global multi-modal model on the entire data distribution of decentralized clients. We employ the contrastive loss that enables a more efficient self-supervised learning of decentralized modules. Comprehensive experiments are conducted on the VQA-v2 dataset based on five SOTA VQA models, demonstrating the effectiveness of the proposed method. Furthermore, we inspect BiCSL's robustness against a dual-key backdoor attack on VQA. Consequently, BiCSL shows significantly enhanced resilience when exposed to the multi-modal adversarial attack compared to the centralized learning method, which provides a promising approach to decentralized multi-modal learning.

The commercialization of 5G has been initiated for a while. Furthermore, millimeter wave (mmWave) has been introduced to small cells with small coverage due to its strong linearity and non-winding characteristics. On the other hand, in connected autonomous vehicles (CAV s), where various traffic systems can cooperatively perform recognition, decision-making, and execution, communication is assumed to be always connected. Therefore, to use low latency mm Wave for high-speed moving CAV, existing beamforming cannot follow them at high speed. This paper proposes an improved beam tracking algorithm for high-speed CAVs, which can be evaluated in a more general environment using a traffic simulator. We proposed an adaptive algorithm for a general road environment by increasing the number of beam searches and search dimensions.

Trojan attacks embed perturbations in input data leading to malicious behavior in neural network models. A combination of various Trojans in different modalities enables an adversary to mount a sophisticated attack on multimodal learning such as Visual Question Answering (VQA). However, multimodal Trojans in conventional methods are susceptible to parameter adjustment during processes such as fine-tuning. To this end, we propose an instance-level multimodal Trojan attack on VQA that efficiently adapts to fine-tuned models through a dual-modality adversarial learning method. This method compromises two specific neurons in a specific perturbation layer in the pretrained model to produce overly large neuron activations. Then, a malicious correlation between these overactive neurons and the malicious output of a fine-tuned model is established through adversarial learning. Extensive experiments are conducted using the VQA-v2 dataset, based on a wide range of metrics including sample efficiency, stealthiness, and robustness. The proposed attack demonstrates enhanced performance with diverse vision and text Trojans tailored for each sample. We demonstrate that the proposed attack can be efficiently adapted to different fine-tuned models, by injecting only a few shots of Trojan samples. Moreover, we investigate the attack performance under conventional defenses, where the defenses cannot effectively mitigate the attack.

Wireless Ad Hoc Federated Learning (WAFL) has been proposed to allow fully distributed collaborative learning in a device-to-device communication without depending on any centralized mechanisms. The main focus of previous studies has been to generalize models of users over label distribution skew cases in not independent and identically distributed (Non- IID) scenarios. However, a generalized WAFL model does not always provide correct answers to each individual over label preference skew, which needs personalization. We proposed a personalization method of WAFL for label preference skew in a previous work, which divide a local model into public and private parameters by layer. In this paper, we propose more fine- grained parameter decoupling approach, Neuron Personalization. We have carried out evaluations using modified CIFAR10 dataset and Pascal VOC dataset. The results indicate that the Neuron Personalization have a good effects to keeping the balance between generalization and personalization for label preference skew.

Object detection is vital for various applications like autonomous vehicles and surveillance. Mission-oriented applications, such as retail, manufacturing, agriculture, healthcare, and robotics, require additional tuning for specific target images, often containing privacy-sensitive data unsuitable for cloud storage. With the deployment of AI chips for edge devices, training object detection models on-device becomes feasible, allowing collaborative model training among multiple devices via device-to-device communication. This paper proposes Wireless Ad Hoc Federated Learning for Detection Transformers (DETR), introducing three parameter-exchange methods: full-parameter exchange (FPE), transformer-layer exchange (TLE), and head ex-change (HE) for the distributed environment. This paper analyzes their impact on prediction accuracy and communication load. Experiments demonstrate that WAFL-DETR-TLE outperforms others, covering both IID and non-IID label distribution scenarios across various network topologies.

Anomaly detection is an important function in IoT applications for finding outliers caused by abnormal events. Anomaly detection sometimes comes with high-frequency data sampling which should be carried out at Edge devices rather than Cloud. In this paper, we consider the case that multiple IoT devices are installed in a single remote site and that they collaboratively detect anomalies from the observations with device-to-device communications. For this, we propose a fully distributed collaborative scheme for training distributed anomaly detectors. We introduce the concept of Global Anomaly which sample is not only rare to the local device but rare to all the devices in the target domain. We also propose a distributed threshold-finding algorithm for Global Anomaly detection. With our standard benchmark-based evaluation, we have confirmed that our scheme trained anomaly detectors perfectly across the devices. We have also confirmed that the devices collaboratively found thresholds for Global Anomaly detection with low false positive rates while achieving high true positive rates with few exceptions.

Electricity thefts are conventionally manually detected by inspections, accusations, and the failure of meters. However, the recent evolution of machine learning may allow the automatic detection of electricity theft only from the patterns of meter readings. Electric consumption heavily relies on many factors, e.g., the lifestyle of the day and the weather, and thus the accuracy of detection is questioned. We propose an electricity theft detection framework for smart homes with knowledge-based synthetic attack data. This allows training of the attack classifier only from the legitimate power consumption data, i.e, without attack actions and associated labels. We identified five attack patterns as the knowledge which consisted of smart attacks and legacy attacks. We have conducted comprehensive evaluations with nine machine learning models using the Almanac of Minutely Power dataset version 2 (AMPds2) dataset fine-grained time-series data of a smart home. We found that Gradient Boosting-based algorithms achieved the best, and Random Forest performed alternatively with almost 100% accuracy for detecting and classifying legacy attacks. Some smart attacks were not detected, but those algorithms achieved good performance in detection and classification.

Cyber-attacks on industrial control systems (ICSs) may cause huge damage to our society and our lives. RS-485 is a backbone network for many ICSs deployed worldwide as a standard. Attack detection in the RS-485 network has been studied in the past. However, the operator still needs to identify and eliminate the attacker in the network after detected, which may require a huge downtime of the system. We propose an attacker localization framework for RS-485 networks. This framework uses (1) a current transformer for monitoring the analog signals of the communication line and (2) machine learning for detecting and localizing the attacker. We have carried out a performance evaluation on a 200-meter scale testbed and found that regression-based localization model performed the best with an averaging aggregator. It could estimate the location of the attacker with about 100% accuracy if we could obtain 6 or 10 attacker points in the training dataset. It could also estimate the location with 93%-96% accuracy with only 4 attacker training points, which would be still practically useful for finding the attacker in RS-485 network.

The rapidly increasing number of smart devices on the Internet necessitates an efficient inspection system for safeguarding our networks from suspicious activities such as Address Resolution Protocol (ARP) probes. In this research, we analyze sequence data of ARP traffic on LAN based on the numerical count and degree of its packets. A dynamic threshold is employed to detect underlying suspicious activities, which are further converted into feature vectors to train an unsupervised autoencoder neural network. Then, we leverage K-means clustering to separate the extracted latent features of suspicious activities from the autoencoder into various patterns.

In cooperative ITS, security and privacy protection are essential. Cooperative Awareness Message (CAM) is a basic V2V message standard, and misbehavior detection is critical for protection against attacking CAMs from the inside system, in addition to node authentication by Public Key Infrastructure (PKI). On the contrary, pseudonym IDs, which have been introduced to protect privacy from tracking, make it challenging to perform misbehavior detection. In this study, we improve the performance of misbehavior detection using observation data of other vehicles. This is referred to as collective perception message (CPM), which is becoming the new standard in European countries. We have experimented using realistic traffic scenarios and succeeded in reducing the rate of rejecting valid CAMs (false positive) by approximately 15 percentage points while maintaining the rate of correctly detecting attacks (true positive).

Model poisoning attacks on federated learning intrude in the entire system via compromising an edge model, resulting in malfunctioning of machine learning models. Such compromised models are tampered with to perform adversary-desired behaviors. In particular, we considered a semi-targeted situation where the source class is predetermined however the target class is not. The goal is to cause the global classifier to misclassify data of the source class. Though approaches such as label flipping have been adopted to inject poisoned parameters into federated learning, it has been shown that their performances are usually class-sensitive varying with different target classes applied. Typically, an attack can become less effective when shifting to a different target class. To overcome this challenge, we propose the Attacking Distance-aware Attack (ADA) to enhance a poisoning attack by finding the optimized target class in the feature space. Moreover, we studied a more challenging situation where an adversary had limited prior knowledge about a client's data. To tackle this problem, ADA deduces pair-wise distances between different classes in the latent feature space from shared model parameters based on the backward error analysis. We performed extensive empirical evaluations on ADA by varying the factor of attacking frequency in three different image classification tasks. As a result, ADA succeeded in increasing the attack performance by 1.8 times in the most challenging case with an attacking frequency of 0.01.

Remotely connected devices have been adopted in several industrial control systems (ICS) recently due to the advancement in the Industrial Internet of Things (IIoT). This led to new security vulnerabilities because of the expansion of the attack surface. Moreover, cybersecurity incidents in critical infrastructures are increasing. In the ICS, RS-485 cables are widely used in its network for serial communication between each component. However, almost 30 years ago, most of the industrial network protocols implemented over RS-485 such as Modbus were designed without security features. Therefore, anomaly detection is required in industrial control networks to secure communication in the systems. The goal of this paper is to study unsupervised anomaly detection in RS-485 traffic using autoencoders. Five threat scenarios in the physical layer of the industrial control network are proposed. The novelty of our method is that RS-485 traffic is collected indirectly by an analog-to-digital converter. In the experiments, multilayer perceptron (MLP), 1D convolutional, Long Short-Term Memory (LSTM) autoencoders are trained to detect anomalies. The results show that three autoencoders effectively detect anomalous traffic with F1-scores of 0.963, 0.949, and 0.928 respectively. Due to the indirect traffic collection, our method can be practically applied in the industrial control network.

With increasingly more sophisticated phishing campaigns in recent years, phishing emails lure people using more legitimate-looking personal contexts. To tackle this problem, instead of traditional heuristics-based algorithms, more adaptive detection systems such as natural language processing (NLP)powered approaches are essential to understanding phishing text representations. Nevertheless, concerns surrounding the collection of phishing data that might cover confidential information hinder the effectiveness of model learning. We propose a decentralized phishing email detection framework called Federated Phish Bowl (FedPB) which facilitates collaborative phishing detection with privacy. In particular, we devise a knowledge-sharing mechanism with federated learning (FL). Using long short-term memory (LSTM) for phishing detection, the framework adapts by sharing a global word embedding matrix across the clients, with each client running its local model with Non-IID data. We collected the most recent phishing samples to study the effectiveness of the proposed method using different client numbers and data distributions. The results show that FedPB can attain a competitive performance with a centralized phishing detector, with generality to various cases of FL retaining a prediction accuracy of 83%.

In the last few decades, Federated Learning (FL) is proposed in order to perform Machine Learning (ML) tasks in a distributed manner while protecting users' privacy and data. However, most of the traditional FL methods rely on centralized entities while in many real-life situations, there isn't any central server which can orchestrate the training procedure. Moreover, it is also easy to be ignored by researchers that the network topology is likely to be changing all the time in some scenarios such as Ad-hoc networks. Besides, how to deal with the unbalanced data which are not independently identically distributed (IID) collected by devices is also an important open problem. As a consequence, in this paper, we propose a peer-to-peer federated learning algorithm with ad-hoc network, along with five model aggregation strategies. We tested our algorithm on self-made Non-IID datasets. After 5000 epochs, the average accuracy of devices reaches 69% under the best strategies on unbalanced CIFAR10 dataset, improving 28% from the self-training cases without P2P communication. The results indicate that our strategies can reduce the negative effect caused by Non- IID datasets even with ad-hoc networks.

In this article, we presented a visualization method for representing network traffic features using raw data of it. The raw network traffic data was divided into regulated segments. By employing a supervised neural network and an expert-knowledge based labeling method, model training was conducted based on a dataset covering two weeks' network traffic, where the first week's data was employed as the training set and the second week's data was used as the validation set. At last, we achieved validation precision scores of 0.980 for detecting the ARP flooding, 0.800 and 0.815 for detecting the malicious SMB and TCP SYN flooding respectively.

With society's increasing dependence on the Internet, more private data is transmitted through networks every day. Unfortunately, this traffic is susceptible to a wide range of threats and vulnerabilities, including phishing attacks that trick users into compromising their systems or revealing sensitive personal information. In this research, we proposed a deep learning approach to detect malware using data collected from a web crawler that systematically sent requests to benign and malicious websites on the Internet. After applying procedures to segment the network flows and extract features, we used these extracted high-level network traffic features to train a deep neural network to recognize benign and malicious flows. Finally, we evaluated our malware detection approach against various metrics, including precision, recall, and f1 score. The achieved f1 score was 0.924, validating the overall performance of the detection scheme.

An attack on deep learning systems where intelligent machines collaborate to solve problems could cause a node in the network to make a mistake on a critical judgment. At the same time, the security and privacy concerns of AI have galvanized the attention of experts from multiple disciplines. In this research, we successfully mounted adversarial attacks on a federated learning (FL) environment using three different datasets. The attacks leveraged generative adversarial networks (GANs) to affect the learning process and strive to reconstruct the private data of users by learning hidden features from shared local model parameters. The attack was target-oriented drawing data with distinct class distribution from the CIFAR-10, MNIST, and Fashion-MNIST respectively. Moreover, by measuring the Euclidean distance between the real data and the reconstructed adversarial samples, we evaluated the performance of the adversary in the learning processes in various scenarios. At last, we successfully reconstructed the real data of the victim from the shared global model parameters with all the applied datasets.

Cooperative intelligent transportation systems (ITS) are used by autonomous vehicles to communicate with surrounding autonomous vehicles and roadside units (RSU). Current C-ITS applications focus primarily on real-time information sharing, such as cooperative perception. In addition to realtime information sharing, self-driving cars need to coordinate their action plans to achieve higher safety and efficiency. For this reason, this study defines a vehicles future action plan/path and designs a cooperative path-planning model at intersections using future path sharing based on the future path information of multiple vehicles. The notion is that when the RSU detects a potential conflict of vehicle paths or an acceleration opportunity according to the shared future paths, it will generate a coordinated path update that adjusts the speeds of the vehicles. We implemented the proposed method using the open-source Autoware autonomous driving software and evaluated it with the LGSVL autonomous vehicle simulator. We conducted simulation experiments with two vehicles at a blind intersection scenario, finding that each car can travel safely and more efficiently by planning a path that reflects the action plans of all vehicles involved. The time consumed by introducing the RSU is 23.0 % and 28.1 % shorter than that of the stand-alone autonomous driving case at the intersection.

In recent years, Wi-Fi has been used as a means of near-field high-speed communication across personal computers, smartphones and IoT devices such as hospital healthcare devices. Meanwhile, there have been many attempts to exploit equipment leveraging Wi-Fi. Among those exploits and attacks, an attack called channel-based man-in-the-middle (MITM) attack is a serious threat, since it can be used to exploit WPA2, which is a standard encryption and authentication scheme currently and widely in use. In this paper, we propose a scan-based self anomaly detection (SSAD), which is a client-side solution to detect and mitigate channel-based man-in-the-middle attacks using access point (AP) scans. SSAD enables wireless devices to verify the authenticity of wireless access points without the support of the access points, but running anomaly detection by themselves. This characteristic of SSAD, independent from access points, is especially favorable to mobile clients such as smartphones and IoT devices since they usually connect to multiple wireless access points. We implemented SSAD into an open source Wi-Fi client software and evaluated the effectiveness. With our experiments in some operational fields, we achieved 99% detection rate if an attacker was in the same room of a legitimate AP, and over 91% detection rate if an attacker and a legitimate AP were in different rooms.

The Controller Area Network (CAN) bus system works inside connected cars as a central system for communication between electronic control units (ECUs). Despite its central importance, the CAN does not support an authentication mechanism, i.e., CAN messages are broadcast without basic security features. As a result, it is easy for attackers to launch attacks at the CAN bus network system. Attackers can compromise the CAN bus system in several ways: denial of service, fuzzing, spoofing, etc. It is imperative to devise methodologies to protect modern cars against the aforementioned attacks. In this paper, we propose a Long Short-Term Memory (LSTM)-based Intrusion Detection System (IDS) to detect and mitigate the CAN bus network attacks. We first inject attacks at the CAN bus system in a car that we have at our disposal to generate the attack dataset, which we use to test and train our model. Our results demonstrate that our classifier is efficient in detecting the CAN attacks. We achieved a detection accuracy of 99.9949%.

The modern car is increasingly connected. That connection is magnified by the presence of a large number of electronic control units (ECUs). The communication between the ECUs of a modern car is assured by the Controller Area Network (CAN) bus system. Despite its importance, the CAN bus system is bereft of security mechanisms making it vulnerable to numerous security attacks. When an attacker succeeds in compromising the ECUs, they can take control and stop the engine, disable the brakes, turn the lights on/off, etc. An intrusion detection system (IDS) can be deployed as an appropriate security measure to detect the malicious network traffic in the CAN bus system. In this paper, we propose a Convolutional Neural Network (CNN)-based network attacks IDS for protecting the CAN bus system. For efficiency reasons, we generated our own datasets from three car models. Our experiment results demonstrate that our classifier is efficient for detecting the CAN bus system attacks, and it performs with a high accuracy of 99.99% and a detection rate of 0.99.

With the approach of 5G Society, more and more devices have been connected to the Internet, where information is stored, analyzed, and shared. Federated learning allows participants to train a machine learning model through sharing the parameters of it based on local training, instead of raw private data at local. In this research, we propose the implementation of the blockchain in federated learning for local parameters evaluation and global parameter aggregation, thus alleviating the influence of end-point adversarial training data. Besides, all updates of local parameters are encrypted and stored in a block of the blockchain after the consensus by the committee. We evaluate the performance of the scheme when adopting various types of corruption to the adversary's dataset, including noise with various degrees and circle occlusion with various diameters. At last, it shows robust and resilient performance compared with the traditional federated learning, achieving a validation accuracy rate of 0.957 when adding noise with a degree of 1.0, and one of 0.944 when adopting circle occlusion with a diameter of 28 pixels for the classification.

Traditional approaches to cybersecurity issues usually protect users from attacks after the occurrence of specific types of attacks. Besides, patterns of recent cyberattacks tend to be changeable, which add up to unpredictability of them. On the other hand, machine learning, as a new method used to detect intrusion, is attracting more and more attention. Moreover, through the sharing of local training data, the centralized learning approach has proven to improve a model's performance. In this research, a segmented federated learning is proposed, different from a collaborative learning based on single global model in a traditional federated learning model, it keeps multiple global models which allow each segment of participants to conduct collaborative learning separately and rearranges the segmentation of participants dynamically as well. Furthermore, these multiple global models interact with each other for updating parameters, thus being adaptable to various participants' LANs. A dataset covering two months' traffic data from 20 participants' LANs in the LAN-Security Monitoring Project is used. We adopt three types of knowledge-based methods for labeling network events and train a CNN model based on the dataset. At last, we achieve validation accuracies of 0.923, 0.813 and 0.877 individually with these labeling methods.

There is an apparently increasing trend of cyber attacks towards LANs in recent years. It is getting more important to monitor the behaviors in LAN and detect intrusions rapidly and accurately. However, there are few studies for the behavior of LAN-internal communications. These works are faced with problems including (1) the lack of popular datasets especially captured from real-world LAN-internal communications, and (2) the lack of well-designed feature extraction for LAN communications. In this paper we propose (1) LAN traffic dataset with protocol based features and labels, and (2) XGBoost based misuse intrusion detection for LAN. After deploying 45 monitoring devices in distributed LANs in 10 countries, we detect malicious hosts from total 52,463 hosts appearing during Nov.1st, 2019 to May.5th, 2020 by extracting their behavioral features on each protocol. Evaluation results demonstrate that our misuse detection performs 97.5% in overall precision and 97.5% in overall recall. Besides, we also discovered that ARP, MDNS and NBNS are the top 3 protocols that influence the intrusion detection most in LAN.

In recent years, in the Internet, it is common to encrypt communication lines for the assumption that the contents of communication are eavesdropped, but even if the communication lines are secure, there are many cases in which the possibility of the contents of communication being leaked to a third party by a side-channel attack is not taken into account. Although it is important that the contents of all communication are not known by the third party, the information related to privacy may be leaked unintentionally by only encrypting traffics. In this study, we made smart speakers, an IoT device that has started to penetrate into our daily lives, to perform eight kinds of activities, and used their traffic data to estimate their activities with CNN, and we were able to estimate the activities with 98% accuracy. As a counter measure, we propose a method to reduce the accuracy of estimation by adding dummy packets to their communication traffic as noise. While adding random noise only reduced the accuracy of our machine learning model to 0.5 with 800 [packets/100msec] of noise, by adding well-designed noise, we were able to reduce the accuracy to 0.28 with 200 [packets/100msec] of noise of the same model. In this study, we made smart speakers, an IoT device that has started to penetrate into our daily lives, to perform eight kinds of activities, and used their traffic data to estimate their activities with CNN, and we were able to estimate the activities with 98% accuracy. As a counter measure, we propose a method to reduce the accuracy of estimation by adding dummy packets to their communication traffic as noise. While adding random noise only reduced the accuracy of our machine learning model to 0.5 with 800 [packets/100msec] of noise, by adding well-designed noise, we were able to reduce the accuracy to 0.28 with 200 [packets/100msec] of noise of the same model. While adding random noise only reduced the accuracy of our machine learning model to 0.5 with 800 [packets/100msec] of noise, by adding well-designed noise, we were able to reduce the accuracy to 0.28 with 200 [packets/100msec] of noise of the same model.

Presents the introductory welcome message from the conference proceedings. May include the conference officers' congratulations to all involved with the conference event and publication of the proceedings record.

Wireless mesh networks suffer from intermittent connectivity, and thus hop-by-hop reliability and parallel message propagation, which DTN researches have explored, can be applied to allow scalable message propagation over such challenged network environments. We implemented those communication schemes onto UTMesh - 50-node scale wireless mesh network testbed, and studied the delivery patterns. On the evaluation result with UTMesh, we have confirmed (1) that hop-by-hop reliability scheme achieves scalable message propagation (e.g., 23 hops), and (2) that message propagation speed increases as the redundancy-level increases. We have also observed that the smallest hop count path does not always achieve the fastest message delivery. This was probably because longer distant links were unstable and message paths over short distant links provided faster propagation.