Neural Architecture Search (NAS) has emerged as a crucial method for automating the design of deep learning models. Despite its potential, NAS frequently requires substantial computational and hardware resources. To mitigate these challenges, transferable NAS (TNAS) has been introduced, leveraging prior NAS results to enhance performance on new tasks. However, existing methods largely focus on knowledge transfer within identical neural search spaces, overlooking the potential for cross-domain transferability. Motivated by this gap, we explore evolutionary TNAS across heterogeneous search spaces by learning common neural representations. In particular, we introduce a novel approach that encodes both operational and topological information of neural architectures into a unified sequence using a simple tokenizer. This sequence is then processed by a variational auto-encoder, with a Transformer-based encoder to capture rich neural representations and a decoder that reconstructs the original sequence. By utilizing these latent representations, we further establish an inter-domain mapping that acts as a bridge, enabling effective explicit solution transfer among diverse search spaces to enhance the evolutionary NAS process. To harness this capability, we develop an evolutionary sequential transfer optimization approach that transfers knowledge during population initialization, providing both flexibility and adaptability. To the best of our knowledge, this work serves as the first attempt in the literature exploring evolutionary TNAS across diverse spaces. Moreover, we demonstrate the utility of our method through comprehensive empirical studies using different architecture spaces, including NAS-Bench-101, NAS-Bench-201, and the DARTS search space. Our results show that the proposed method significantly enhances the adaptability and performance of NAS across varied domains.

House layout plays a crucial role in housing planning and design. In recent years, automated generation of house layouts has gained significant attention. The objective is to automatically generate floorplans that meet specific requirements under given constraints. In this paper, we present an extension and improvement of the existing diffusion model, focusing on the generation of housing layouts in more complex constrained scenarios. Firstly, we introduce the consideration of relative room area ranking as a new problem scenario. Secondly, we propose an indirect encoding approach that represents the ranking relationship of room relative areas as a directed graph to address potential issues arising from embedding ranking features. Finally, we introduce a corresponding attention module to capture the newly added constraint relationships. Moreover, we evaluate the proposed approach using a range of metrics and the RPLAN dataset. Our proposed method demonstrates improved accuracy in considering variations in room sizes and provides more reasonable layout. The obtained results also indicate that our enhanced approach exhibits better compatibility and Spearman correlation for relative room area ranking compared to the state-of-the-art methods.

Prompt-based learning has improved the few-shot learning performance of large-scale Pre-trained Language Models (PLMs). Yet, it is challenging to deploy large-scale PLMs in resource-constrained environments for online applications. Knowledge Distillation (KD) is a promising approach for PLM compression. However, distilling prompt-tuned PLMs in the few-shot learning setting is a non-trivial problem due to the lack of task-specific training data and KD techniques for the new prompting paradigm. We propose Prompt-Distiller, the first few-shot KD algorithm for prompt-tuned PLMs, which forces the student model to learn from both its pre-trained and prompt-tuned teacher models to alleviate the model overfitting problem. We further design a contrastive learning technique to learn higher-order dependencies from intermediate-layer representations of teacher models, considering different knowledge capacities of teacher and student models. Extensive experiments over various datasets show that Prompt-Distiller consistently outperforms baselines by a large margin.

Neural architecture search (NAS) has been a popular research topic for designing deep neural networks (DNNs) automatically. It is able to improve the design efficiency of neural architectures significantly for given learning tasks. Recently, instead of conducting architecture search in the original neural architecture space, many NAS approaches have been proposed to learn continuous representations from neural architectures for architecture search or estimation. In particular, Neural Architecture Optimization (NAO) is a representative method which encodes neural architectures as continuous representations by an auto-encoder and then performs continuous optimization in the encoded space with gradient-based methods. However, as NAO only considers the top-ranked architectures in learning the continuous representation, it could fail to construct a satisfied continuous optimization space which contains the expected high-quality neural architectures. Taking this cue, in this paper we propose a two-stage NAO (TNAO) to learn a more completed continuous representation of neural architectures which could provide a better optimization space for NAS. Specifically, by designing a pipeline that separates the training and search stages, we first build the training set via random sampling from the entire neural architecture search space, which is with the aim of collecting the well-distributed neural architectures for training. Moreover, to exploit the architectural semantic information with limited data effectively, we propose an improved Transformer auto-encoder for learning the continuous representation, which is supervised by ranking information of the neural architecture performance. Lastly, towards more effective optimization of neural architectures, we adopt a population-based swarm intelligence algorithm, i.e. competitive swarm optimization (CSO), with a newly designed remapping scoring scheme. To evaluate the efficiency of the proposed TNAO, comprehensive experimental studies are conducted on two common search spaces, i.e., NAS-Bench-101 and NAS-Bench-201. The architecture with the top 0.02% performance is discovered on NAS-Bench-101 and the best architecture in the CIFAR-10 dataset is obtained on NAS-Bench-201.

Federated learning allows a large number of resource-constrained clients to train a globally-shared model together without sharing local data. These clients usually have only a few classes (categories) of data for training, where the data distribution is non-iid (not independent identically distributed). In this article, we put forward the concept ofcategory privacyfor the first time to indicatewhich classes of data a client has, which is an important but ignored privacy goal in the federated learning with non-iid data. Although secure aggregation protocols are designed for federated learning to protect the input privacy of clients, we perform the first systematic study oncategory inference attackand demonstrate that these protocols cannot fully protect category privacy. We design a differential selection strategy and two de-noising approaches to achieve the attack goal successfully. In our evaluation, we apply the attack to non-iid federated learning settings with various datasets. On MNIST, CIFAR-10, AG_news, and DBPedia dataset, our attack achieves$>90\%$accuracy measured in F1-score in most cases. We further consider a possible detection method and propose two strategies to make the attack more inconspicuous.

Neural architecture search (NAS) has attracted much attention in recent years. It automates the neural network construction for different tasks, which is traditionally addressed manually. In the literature, evolutionary optimization (EO) has been proposed for NAS due to its strong global search capability. However, despite the success enjoyed by EO, it is worth noting that existing EO algorithms for NAS are often very computationally expensive, which makes these algorithms unpractical in reality. Keeping this in mind, in this article, we propose an efficient memetic algorithm (MA) for automated convolutional neural network (CNN) architecture search. In contrast to existing EO algorithms for CNN architecture design, a new cell-based architecture search space, and new global and local search operators are proposed for CNN architecture search. To further improve the efficiency of our proposed algorithm, we develop a one-epoch-based performance estimation strategy without any pretrained models to evaluate each found architecture on the training datasets. To investigate the performance of the proposed method, comprehensive empirical studies are conducted against 34 state-of-the-art peer algorithms, including manual algorithms, reinforcement learning (RL) algorithms, gradient-based algorithms, and evolutionary algorithms (EAs), on widely used CIFAR10 and CIFAR100 datasets. The obtained results confirmed the efficacy of the proposed approach for automated CNN architecture design.

The federated learning provides an effective solution to train collaborative models over a large scale of participated Industrial Internet of Things (IIoT) applications with the help of a global server, building an intelligent life. However, the federated learning is vulnerable to the backdoor attack from strong malicious participants. The backdoor attack is inconspicuous and may result in devastating consequences. To resist the attack on IIoT applications, we propose the federated backdoor filter defense that can identify backdoor inputs and restore the data to availability by theblur-label-flippingstrategy. We build multiple filters with eXplainable AI models on the server and send them to clients randomly, preventing advanced attackers from evading the defense. Our backdoor filters show significant backdoor recognition with the accuracy up to 99%. After the implementation of the blur-label-flipping strategy, victim's local model on suspicious backdoor samples can achieve the accuracy up to 88%.

Federated learning (FL) enables a large number of clients to collaboratively train a global model through sharing their gradients in each synchronized epoch of local training. However, a centralized server used to aggregate these gradients can be compromised and forge the result in order to violate privacy or launch other attacks, which incurs the need to verify the integrity of aggregation. In this work, we explore how to design communication-efficient and fast verifiable aggregation in FL. We propose VeriFL, a verifiable aggregation protocol, with O(N) (dimension-independent) communication and O(N+ d) computation for verification in each epoch, where N is the number of clients and d is the dimension of gradient vectors. Since d can be large in some real-world FL applications (e.g., 100K), our dimension-independent communication is especially desirable for clients with limited bandwidth and high-dimensional gradients. In addition, the proposed protocol can be used in the FL setting where secure aggregation is needed or there is a subset of clients dropping out of protocol execution. Experimental results indicate that our protocol is efficient in these settings.

A NoSQL, also called a "Non-Relational" or "Not only SQL," database system provides an approach to data management and database design for very large sets of distributed data and real-time web applications. A NoSQL database system is also a popular data storage for information retrieval because it supports better scalability, availability, and faster data access while comparing with traditional relational database management systems (RDBMS). What the RDBMS data needs is predictable as its data is stored in structured tables by defining the relationship between the different columns. In contrary the data in NoSQL databases does not need to be stored in a structured or fixed fashion. When performance and real-time access are more concerned than consistency, such as indexing and retrieving large numbers of records, NoSQL databases are more suitable than relational databases. With their obvious advantages in better performance, scalability, and flexibility, NoSQL databases have been adopted lately by many small businesses as they are moving their increasing business data into the clouds. However, the research on the security of a specific NoSQL database system or NoSQL database systems in general is very limited. Although there are many storage advantages in NoSQL databases, the need of quick and easy access to data has been seriously affected by the security issue of NoSQL databases. This paper examines the maturity of security measures for MongoDB, a typical NoSQL database system, with aspects in both attack and defense at the code level. The experimental testing on NoSQL injections is performed with JavaScript and PHP. After the demonstration on how a server-side JavaScript injection attack against a NoSQL database system reveals the customer's private data, two methods are discussed in preventing this type of security problems from happening. It is believed that our study will help database developers not only realizing that NoSQL database systems are not designed with security as a priority but also learning how to build a security layer to their organizations' NoSQL applications to avoid NoSQL injections.

A robot autonomous tracking system is developed in this paper through integrating vision and distance sensory information. A neural network is utilized to fuse the image processing results and sonar measurements, and select a correct motion command for the follower robot. The network architecture, data pre-processing, sample design and training results are presented. The simulation and experimental results validate the proposed leader-follower robot formation control strategy.