Multi-hop question answering (QA) requires the model to integrate multiple evidential fragments scattered in the document to infer the answer. At present, while graph neural network-based (GNN-based) approaches have exhibited promising performance in multi-hop QA tasks, some underlying issues have also emerged. The first is that GNN is insensitive to crucial dependencies between "levels" in the graph. The second is that reasoning messages on the graph require multiple hops to reach the answer entity, and this jumping between nodes induced by reasoning significantly disperses attention. To address these issues, we propose a novel model of Subgraph-based Attention Network (SuAN) for multi-hop QA, which achieves controlled message passing in GNN through subgraph decomposition and constrained reasoning order, more in line with the multi-hop reasoning process. In SuAN, we decompose the graph into a series of subgraphs composed of first-order neighboring nodes based on semantics. Subsequently, we update node representations on these subgraphs according to a pre-defined reasoning order. This approach not only narrows the gap between the human reasoning process and the message passing mechanism of GNN but also enables the full exploitation of the implicit or explicit hierarchical property inherent in the graph structure. The experimental results on HotpotQA demonstrate the effectiveness of the proposed approach and help the baseline model achieve significant improvements.

Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

Initially developed for natural language processing (NLP), Transformer model is now widely used for speech processing tasks such as speaker recognition, due to its powerful sequence modeling capabilities. However, conventional self-attention mechanisms are originally designed for modeling textual sequence without considering the characteristics of speech and speaker modeling. Besides, different Transformer variants for speaker recognition have not been well studied. In this work, we propose a novel multi-view self-attention mechanism and present an empirical study of different Transformer variants with or without the proposed attention mechanism for speaker recognition. Specifically, to balance the capabilities of capturing global dependencies and modeling the locality, we propose a multi-view self-attention mechanism for speaker Transformer, in which different attention heads can attend to different ranges of the receptive field. Furthermore, we introduce and compare five Transformer variants with different network architectures, embedding locations, and pooling methods to learn speaker embeddings. Experimental results on the VoxCeleb1 and VoxCeleb2 datasets show that the proposed multi-view self-attention mechanism achieves improvement in the performance of speaker recognition, and the proposed speaker Transformer network attains excellent results compared with state-of-the-art models.

Self-supervised speech representation learning has shown promising results in various speech processing tasks.However, the pre-trained models, e.g., HuBERT, are storage-intensive Transformers, limiting their scope of applications under lowresource settings.To this end, we propose LightHuBERT, a once-for-all Transformer compression framework, to find the desired architectures automatically by pruning structured parameters.More precisely, we create a Transformer-based supernet that is nested with thousands of weight-sharing subnets and design a two-stage distillation strategy to leverage the contextualized latent representations from HuBERT.Experiments on automatic speech recognition (ASR) and the SU-PERB benchmark show the proposed LightHuBERT enables over 10 9 architectures concerning the embedding dimension, attention dimension, head number, feed-forward network ratio, and network depth.LightHuBERT outperforms the original HuBERT on ASR and five SUPERB tasks with the Hu-BERT size, achieves comparable performance to the teacher model in most tasks with a reduction of 29% parameters, and obtains a 3.5× compression ratio in three SUPERB tasks, e.g., automatic speaker verification, keyword spotting, and intent classification, with a slight accuracy loss.The code and pre-trained models are available at https://github.com/mechanicalsea/lighthubert.

Convolutional neural networks (CNNs), such as the time-delay neural network (TDNN), have shown their remarkable capability in learning speaker embedding. However, they meanwhile bring a huge computational cost in storage size, processing, and memory. Discovering the specialized CNN that meets a specific constraint requires a substantial effort of human experts. Compared with hand-designed approaches, neural architecture search (NAS) appears as a practical technique in automating the manual architecture design process and has attracted increasing interest in spoken language processing tasks such as speaker recognition. In this paper, we propose EfficientTDNN, an efficient architecture search framework consisting of a TDNN-based supernet and a TDNN-NAS algorithm. The proposed supernet introduces temporal convolution of different ranges of the receptive field and feature aggregation of various resolutions from different layers to TDNN. On top of it, the TDNN-NAS algorithm quickly searches for the desired TDNN architecture via weight-sharing subnets, which surprisingly reduces computation while handling the vast number of devices with various resources requirements. Experimental results on the VoxCeleb dataset show the proposed EfficientTDNN enables approximate$10^{13}$architectures concerning depth, kernel, and width. Considering different computation constraints, it achieves a 2.20% equal error rate (EER) with 204 M multiply-accumulate operations (MACs), 1.41% EER with 571 M MACs as well as 0.94% EER with 1.45 G MACs. Comprehensive investigations suggest that the trained supernet generalizes subnets not sampled during training and obtains a favorable trade-off between accuracy and efficiency.

Cyber-physical systems (CPS) have been deployed in many areas and have reached unprecedented levels of performance and efficiency. However, the security and privacy problems in CPS have not been properly addressed, e.g., the monitored source location can be inferred by an attacker, which can substantially undermine the reliability of CPS. Unfortunately, the existing techniques to protect against the leakage of the source location do not achieve an acceptable balance among the source location privacy, transmission delay, and energy consumption to guarantee high reliability. To address this issue, we propose an attacker location evaluation-based fake source scheduling (FSSE) for source location privacy in CPS to enhance the privacy level and maintain the system performance. The proposed FSSE contains two main phases. The first, backbone construction, is dependent on the probability of capture derived from the communication information of self and neighboring nodes. This phase aims to build a backbone to form a baseline with respect to the source location privacy and transmission delay. The second phase is fake message scheduling, which is established to provide a trade-off among privacy, transmission delay, and communication overhead in terms of the hypothesized location of the attacker by using stochastic processes. Through analysis and simulation, we demonstrate that the proposed method has a more stable privacy level and more efficient transmission delay and energy consumption than the three compared algorithms, i.e., phantom routing, tree-based diversionary routing, and dynamic fake source selection.