Transformers have been widely adopted in natural language processing, computer vision, and other domains due to their exceptional performance across a variety of tasks. However, the computational cost of Transformers is prohibitively high, particularly when handling long input sequences, significantly increasing both training and inference time. Although various token pruning methods have been proposed to reduce the computational burden of Transformers, most approaches overlook critical differences in sequences in terms of length and complexity, leading to suboptimal compression efficiency. In this paper, we propose AD-TP, an Attribution-Driven Adaptive Token Pruning method designed to retain only the most informative tokens. We analyze the performance of using accumulated attention values to measure token importance and find that attention values do not accurately reflect the actual contribution of each token to text understanding. Additionally, we observe significant variations in the length and complexity of different sequences within the dataset. Based on these insights, we adopt Integrated Gradients to evaluate token importance and introduce a lightweight adaptive token retainer module that dynamically generates pruning configurations for each input sequence. In addition, we incorporate both teacher supervision and self-supervised learning objectives to enhance the training efficiency, accuracy, and robustness of the model. Experiments conducted on GLUE, SQuAD, and 20News demonstrate that AD-TP outperforms state-of-the-art token pruning and model compression methods in both accuracy and computational efficiency. On GLUE, AD-TP reduces FLOPs by an average of 7.8× while improving performance by 0.6%.

Dialogue State Tracking (DST) is a critical component in Task-Oriented Dialogue (TOD) systems, responsible for generating the dialogue state at each turn. Current approaches often struggle with complicated conversational contexts, primarily due to issues of information redundancy and insufficiency, which adversely affects accuracy. To address these challenges, we propose a novel approach termed Selection based on Integrated Information (SII). This method comprises three key components: an Information Integrator, which distills core information from the dialogue; an Information Selector, which identifies the most pertinent core information for each slot; and a State Predictor, which executes predictions based on the selected information. By focusing on selected information, SII demonstrates enhanced performance, achieving joint goal accuracies of 55.44% and 59.89% on the MultiWOZ2.0 and MultiWOZ2.1 datasets, respectively.

In emergencies, negative emotion has a significant impact on decision-making during crowd evacuation. Psychological studies suggest that negative emotion in decision-making has a double-edged sword effect. Excessive negative emotion has adverse impacts, such as causing crowd chaos and congestion. Conversely, moderate negative emotion has a positive effect by speeding up crowd movement. However, current researches mainly focus on one aspect which is how to reduce the negative effects of negative emotion on crowd evacuation, while overlooking the benefits of negative emotion. How to fully explore the double-edged sword effect of negative emotion and regulate negative emotion to improve the efficiency of crowd evacuation is still an open issue. To achieve this, we propose the NegEmotion model which considers the positive impact of negative emotion on crowd evacuation, and regulates crowd emotion by controlling knowledge spreading according to Siminov’s psychological principle. In this model, the knowledge spreading network (KSN) and the stress emotional contagion network (SECN) are constructed. Based on these networks, we study the evolution process of knowledge spreading and stress emotional contagion, respectively. Next, we formulate the emotional regulation as an optimization problem to maximize the efficiency of crowd evacuation. Then, a heuristic algorithm is used to solve for the optimal emotional regulation strategy. Finally, a crowd simulation system is implemented to verify the effectiveness of our NegEmotion model. The experimental results show that our method is effective to improve the efficiency of crowd evacuation.

The convolutional neural network (CNN) is an important deep learning method, which is widely used in many fields. However, it is very time consuming to implement the CNN where convolution usually takes most of the time. There are many zero values in feature maps and filters, which leads to redundant calculations and memory accesses if dense methods are used to compute convolution. Many works recently have made use of sparsity to skip the calculations for zero values to reduce the inference time of the CNN. On the graphics processing unit platform, current works cannot fully exploit the sparsity of the feature map and achieve satisfactory performance. Therefore, we design a new parallel strategy to transform the feature map into a new storage format to avoid the redundant computation of zero values on graphics processing units. Also considering the sparsity in the feature map, we propose a fused storage format to combine the convolution operation with the following pooling operation, to further improve the performance. We carry out experiments with mainstream CNN models and achieve better performance compared with cuDNN and cuSPARSE. For VGG-19, ResNet-50, DenseNet-121, and RegNetX-16GF, 1.97×, 2.23×, 2.74×, and 1.58× speedups respectively are obtained over cuDNN. The speedups over cuSPARSE respectively are 2.10×, 1.83×, 2.35×, and 1.35× when only using the first method.

Chinese Named Entity Recognition (NER) is an essential task in natural language processing, and its performance directly impacts the downstream tasks. The main challenges in Chinese NER are the high dependence of named entities on context and the lack of word boundary information. Therefore, how to integrate relevant knowledge into the corresponding entity has become the primary task for Chinese NER. Both the lattice LSTM model and the WC-LSTM model did not make excellent use of contextual information. Additionally, the lattice LSTM model had a complex structure and did not exploit the word information well. To address the preceding problems, we propose a Chinese NER method based on the deep neural network with multiple ways of embedding fusion. First, we use a convolutional neural network to combine the contextual information of the input sequence and apply a self-attention mechanism to integrate lexicon knowledge, compensating for the lack of word boundaries. The word feature, context feature, bigram feature, and bigram context feature are obtained for each character. Second, four different features are used to fuse information at the embedding layer. As a result, four different word embeddings are obtained through cascading. Last, the fused feature information is input to the encoding and decoding layer. Experiments on three datasets show that our model can effectively improve the performance of Chinese NER.

Most of the syntax-based metrics obtain the similarity by comparing the sub-structures extracted from the trees of hypothesis and reference. These sub-structures cannot represent all the information in the trees because their lengths are limited. To sufficiently use the reference syntax information, a new automatic evaluation metric is proposed based on the dependency parsing model. First, a dependency parsing model is trained using the reference dependency tree for each sentence. Then, the hypothesis is parsed by this dependency parsing model and the corresponding hypothesis dependency tree is generated. The quality of hypothesis can be judged by the quality of the hypothesis dependency tree. Unigram F-score is included in the new metric so that lexicon similarity is obtained. According to experimental results, the proposed metric can perform better than METEOR and BLEU on system level and get comparable results with METEOR on sentence level. To further improve the performance, we also propose a combined metric which gets the best performance on the sentence level and on the system level.