As modern communication technologies advance, the Industrial Internet of Things (IIoT) is progressively evolving towards greater intelligence. The extensive implementation of smart grids has significantly affected IIoT factories. Data aggregation is commonly used to protect the factory's privacy. However, existing multi-dimensional data aggregation schemes lack flexibility and are vulnerable to internal attacks, where private data from certain smart devices may be decrypted by insiders. Moreover, replacing related devices necessitates updating the keys of the entire system, which incurs heavy overhead. To address these issues, a multi-dimensional privacy-preserving data aggregation scheme based on homomorphic proxy re-encryption (MPDA-HPR) is proposed. Using a modified Paillier encryption algorithm supported by proxy re-encryption and super-increasing sequences, the proposed scheme enhances flexibility and scalability. Security analyses demonstrate that the proposed scheme can withstand various security threats and effectively preserve the privacy of devices. Finally, the prototype is implemented and evaluated, demonstrating that the proposed scheme is robust, efficient, and feature-rich.

Training language models from scratch presents a critical challenge in Natural Language Processing (NLP), primarily due to the computational demands of pre-trained Large Language Models, which are predominantly trained on English corpora using extensive resources. While offering viable solutions, existing alternatives still rely heavily on high-performance hardware. This work introduces a different approach to reducing the algorithmic complexity of Transformer-based architectures through the U-Net Encapsulated Transformer (UET), which applies dimensionality reduction to token embeddings. The UET architecture enables the development of language models with significantly reduced parameter sizes for a given set of hyperparameters. Alternatively, it allows researchers to design models of comparable size but with a substantially greater number of Transformer blocks, enhancing model depth and potential capacity. This study also outlines practical methodologies for training language models in resource-constrained environments. Experimental results illustrate the potential of the UET architecture in achieving reasonable performance under resource-constrained conditions, highlighting its promise as an accessible alternative for language model development. This work could broaden the accessibility of NLP research, empowering researchers with hardware constraints to contribute to language model development.

Anomaly detection is to identify abnormal events against normal ones within surveillance videos mainly collected in ground-based settings. Recently, the demand for processing drone-collected data is rapidly growing with the expanding range of drone applications. However, as most aerial videos collected by flying drones contain dynamic backgrounds and others, it is necessary to deal with their spatio-temporal features in detecting anomalies. This study presents a transformer-based video anomaly detection method whereby we investigate a challenging aerial dataset and three benchmark ground-based datasets. A multi-stage transformer is leveraged as an encoder to generate multi-scale feature maps, which are then conveyed to a hierarchical spatio-temporal transformer, that is linked to a decoder and used to capture spatial and temporal information by utilizing a joint attention mechanism. Extensive evaluations including several ablation studies suggest that this network outperforms the state-of-the-art methods. We expect the proposed transformer for U-net can find diverse applications in the video processing area. Code and pre-trained models are publicly available at https://github.com/vt-le/HSTforU .

The widespread availability of similarity queries over trajectory data has led to numerous real-world applications, such as traffic management and path planning. With the proliferation of trajectory data, data owners often outsource storage and computation tasks to the cloud due to limited computing and storage resources. However, this scenario raises sharp security concerns, where it is critical to ensure both the integrity of query results and privacy during query processing. Furthermore, most existing works assume a single-user setting where all query users share the same key, which may lead to query privacy leakage. Therefore, in this article, we take the first step in studying the issue of multiuser and secure trajectory similarity range query (MSRQ). Specifically, inspired by the M-tree, we propose a secure index based on a distributed two-trapdoor public-key cryptosystem (DT-PKC), called M*-tree, and devise secure protocols to support multiuser query processing. We also carefully design a filtering strategy and verification scheme to ensure fast search and integrity guarantees. Finally, we theoretically analyze the security and complexity and empirically evaluate the performance and feasibility of our proposed approach.

Vehicular intrusion detection systems (VIDSs) play a crucial role in protecting the security of vehicular ad hoc networks (VANETs). Recently, numerous researchers have proposed effective vehicular intrusion detection systems to protect the security of VANETs. However, some existing vehicle intrusion systems are susceptible to catastrophic forgetfulness in the process of implementing incremental updates to target novel attacks. Other solutions lack consideration for the continuous updating capabilities of vehicle intrusion detection systems. It is worth mentioning that in the real world, the network attacks suffered by vehicles are not constant, and fixed intrusion detection systems may struggle to detect new network attacks effectively. To address these challenges, we propose an incremental learning-based vehicular intrusion detection scheme that supports continuous updating of the intrusion detection system. Specifically, we design a sample gradient optimization algorithm to enhance the data quality of training samples. Additionally, we utilize locally stored historical data to balance the number of old attack classes for model distillation, thus mitigating the problem of forgetting the old classes as the model learns new classes. The comprehensive experimental results on the CICIDS2017, TON_IOT, and Veremi datasets demonstrate that the proposed vehicular intrusion detection system maintains superior detection accuracy during continuous updating and surpasses the state-of-the-art solution.

The rapid development of network technology has allowed numerous vehicular applications to be deployed in vehicles, thereby enriching the driving experience of users. However, the openness of vehicular networks enables attackers to launch network attacks on vehicles through network ports, leading to the destruction of vehicular networks. To develop an intrusion detection system suitable for distributed vehicular networks, researchers have utilized federated learning to train detection models. Nevertheless, most federated learning-based vehicular intrusion detection systems seldom consider rapidly updating the detection model and fail to detect unknown attacks effectively. In this study, we propose a federated learning-based vehicular intrusion detection system that fully considers the traffic characteristics of multiple network regions and selects representative clients to participate in model aggregation, thereby accelerating the convergence of the global model. Furthermore, to enhance the robustness of the detection system, we utilize extreme value theory and multilayer activation vectors to construct an unknown attack discriminator that can determine whether a network flow is an unknown attack. Comprehensive experiments on three open datasets demonstrate that the proposed intrusion detection system can quickly update and effectively identify known/unknown attacks in open vehicular networks

A 3D avatar typically has one of six cardinal facial expressions. To simulate realistic emotional variation, we should be able to render a facial transition between two arbitrary expressions. This study presents a new framework for instruction-driven facial expression generation that produces a 3D face and, starting from an image of the face, transforms the facial expression from one designated facial expression to another. The Instruction-driven Facial Expression Decomposer (IFED) module is introduced to facilitate multimodal data learning and capture the correlation between textual descriptions and facial expression features. Subsequently, we propose the Instruction to Facial Expression Transition (I2FET) method, which leverages IFED and a vertex reconstruction loss function to refine the semantic comprehension of latent vectors, thus generating a facial expression sequence according to the given instruction. Lastly, we present the Facial Expression Transition model to generate smooth transitions between facial expressions. Extensive evaluation suggests that the proposed model outperforms state-of-the-art methods on the CK+ and CelebV-HQ datasets. The results show that our framework can generate facial expression trajectories according to text instruction. Considering that text prompts allow us to make diverse descriptions of human emotional states, the repertoire of facial expressions and the transitions between them can be expanded greatly. We expect our framework to find various practical applications. More information about our project can be found athttps://vohoanganh.github.io/tg3dfet/

Recommending a tempting sequence of items according to a user’s previous history of purchases and clicks, for instance, in the online shopping portals is challenging. And yet it is a crucial task for all service providers. One of the core components in the recommender systems is a sequential model with which an input sequence is transformed into the predicted items. Among many, deep neural networks, such as RNN, LSTM, and Transformer, have been favored for this purpose. However, improving the performance of these models remains an important task. To address this, we propose a novel sequential model by combining a U-net and Transformer, called U-net Encapsulated Transformer. This hybrid architecture places a transformer model between a convolutional encoder and its decoder, wherein each convolutional layer processes 1D signal, i.e. text. The primary benefit of this structure is that the computational burden is reduced since the embedding size of the input to the Transformer is drastically decreased as the signal has to go through the multi-layer convolutional encoder. This solution leverages recommendation lists for action predictions and uses user feedback as direct rewards for updating the model. In addition, the loss function mechanism is improved by including contrastive and reinforcement learning losses. Evaluation of the proposed model, including extensive ablation study, is carried out on four standard benchmark datasets, such as RC15, RetailRocket, MovieLens-1M, and Amazon-Beauty, demonstrating superior performance compared to state-of-the-art methods.