Integrating external knowledge with traditional spoken language understanding (SLU) models can effectively mitigate the ambiguity in user utterances in real-world scenarios. Knowledge graph, as a common source of external knowledge, encapsulates entities enriched with diverse attribute information. Nevertheless, existing models consider all entities as relevant, which introduces significant noise into the input. Additionally, not all attribute information of the entities is essential, resulting in considerable noise and redundancy. In this article, we propose a Noise-Removal of Knowledge-Enhanced (NRKE) framework for SLU, which involves two different types of denoising. The first approach involves hard denoising via entity selection, where we leverage a small clean dataset and introduce a BERT-based auxiliary model to filter out entities unrelated to user utterances, effectively eliminating noisy entities. In addition, we further refine entity selection by incorporating Large Language Models (LLMs) to assist in filtering out entities unrelated to user utterances. The second method involves soft denoising through the selection of entity attribute information. This approach utilizes a keywords-based local semantic selection that gives greater weight to relevant local semantics associated with specific keywords. This allows us to capture task-related information from the chosen entities, thereby minimizing noise and redundancy. To evaluate the generalization capability of existing knowledge-enhanced SLU models, we construct a new dataset named KGCAIS. The experimental results show that our NRKE achieves better performance than the competing models on both the PROSLU and KGCAIS datasets.

Large Language Models (LLMs) have demonstrated impressive capabilities in language generation and general task performance.However, their application to spoken language understanding (SLU) remains challenging-particularly for token-level tasks, where the autoregressive nature of LLMs often leads to misalignment issues.They also struggle to capture nuanced interrelations in semanticlevel tasks through direct fine-tuning alone.To address these challenges, we propose the Entitylevel Language Model (ECLM) framework, which reformulates slot-filling as an entity recognition task and introduces a novel concept, Chain of Intent, to enable step-by-step multiintent recognition.Experimental results show that ECLM significantly outperforms strong baselines such as Uni-MIS, achieving gains of 3.7% on MixATIS and 3.1% on MixSNIPS.Compared to standard supervised fine-tuning of LLMs, ECLM further achieves improvements of 8.5% and 21.2% on these datasets, respectively.Our code is available at https: //github.com/SJY8460/ECLM.

Decoder-only Large Language Models (LLMs) have demonstrated exceptional performance in language generation, exhibiting broad capabilities across various tasks. However, the application to label-sensitive language understanding tasks remains challenging due to the limitations of their autoregressive architecture, which restricts the sharing of token information within a sentence. In this paper, we address the Multi-Intent Detection (MID) task and introduce MIDLM, a bidirectional LLM framework that incorporates intent number detection and multi-intent selection. This framework allows autoregressive LLMs to leverage bidirectional information awareness through post-training, eliminating the need for training the models from scratch. Comprehensive evaluations across 8 datasets show that MIDLM consistently outperforms both existing vanilla models and pretrained baselines, demonstrating its superior performance in the MID task.

Task-oriented dialogue summarization aims to automatically extract key information to generate domain summaries to improve service efficiency and quality. Task-oriented dialogue is inherently logical and surrounds specific topic. How to effectively capture the dialogue topic and the most salient information becomes one of the major challenges of this task. In this paper, we propose a task-oriented dialogue summarization model utilizing a multi-level encoder with global topic (MLEGT). This model discovers temporal and spatial dependencies through a multi-level encoder, while simultaneously obtaining a simple yet effective global topic in the process of graph construction. It not only captures the crucial details but also breaks the long-range limit and discover the intrinsic structure across the dialogue. Then, a global topic guided pointer mechanism is added into the summrizer to capture ignored but important information, which is helpful to improve the accuracy of the summary. A comprehensive study of two public datasets, including a real-world large-scale Chinese police interrogation dataset and a medical reporting service dataset proves the superiority of our method on several strong baselines.

In real-world scenarios, due to data scarcity, cross-domain slot filling in spoken language understanding remains a significant challenge. Previous works focus on supplementing sequence labeling models with slot meta-information or metric learning. They have poor generalization capabilities lacking specific domain knowledge. To enhance generalization, recent studies introduce implicit general knowledge to enhance the performance of slots lacking domain-specific knowledge by further pretraining or larger-parameter generative models. However, this knowledge is domain-agnostic and difficult to provide comprehensive knowledge for target domain. Therefore, we propose a two-stage data generation strategy, utilizing powerful LLMs to synthesize samples to introduce knowledge for each slot of the data-scarce target domain. More importantly, we employ a data curation mechanism based on confidence and uncertainty to identify and filter out low-quality samples to obtain a high-quality synthetic dataset. Extensive experimental results demonstrate the effectiveness and generality of our approach.

So far, multi-intent spoken language understanding (SLU) has become a research hotspot in the field of natural language processing (NLP) due to its ability to recognize and extract multiple intents expressed and annotate corresponding sequence slot tags within a single utterance. Previous research has primarily concentrated on the token-level intent-slot interaction to model joint intent detection and slot filling, which resulted in a failure to fully utilize anisotropic intent-guiding information during joint training. In this work, we present a novel architecture by modeling the multi-intent SLU as a multi-view intent-slot interaction. The architecture resolves the kernel bottleneck of unified multi-intent SLU by effectively modeling the intent-slot relations with utterance, chunk, and token-level interaction. We further develop a neural framework, namely Uni-MIS, in which the unified multi-intent SLU is modeled as a three-view intent-slot interaction fusion to better capture the interaction information after special encoding. A chunk-level intent detection decoder is used to sufficiently capture the multi-intent, and an adaptive intent-slot graph network is used to capture the fine-grained intent information to guide final slot filling. We perform extensive experiments on two widely used benchmark datasets for multi-intent SLU, where our model bets on all the current strong baselines, pushing the state-of-the-art performance of unified multi-intent SLU. Additionally, the ChatGPT benchmark that we have developed demonstrates that there is a considerable amount of potential research value in the field of multi-intent SLU.

Dialogue intent classification is a pivotal task in natural language understanding, crucial for effective human-computer interactions. Despite significant progress in structural modeling of dialogues and texts, existing research still has several limitations: intention labels are treated as independent entities, ignoring the hierarchical relationships and dependencies among them and making it hard to make accurate predictions on fine-grained labels. In this paper, we propose a Hierarchical Label-aware Dialogue Intent Classification model (HLDIC) for dialogue intent classification. Specifically, we leverage the hierarchical relationships of labels by introducing a coarse-grained label classification auxiliary task. A hierarchical adaptive attention mechanism is proposed, which employs gate mechanisms to guide the model in recognizing keywords and vital adjacent pairs. To further explore label dependencies, a hierarchy-aware mechanism is proposed to use a mask matrix to allow the model to focus on the correct fine-grained labels within the corresponding coarse-grained labels and partially suppress the noise from other coarse-grained labels. Experimental results on public CCL2018-Task1 corpus show the superior performance of HLDIC.

Out-of-domain (OOD) detection plays an important role in spoken language understanding (SLU). It can help dialog systems reduce confusion between in-domain (ID) and OOD utterances. Many dialog systems train their model to achieve this goal by collecting annotated OOD and ID data. However, acquiring large-scale OOD datasets can be costly. Recent generative adversarial networks (GANs) based OOD detection methods aim to mitigate this problem. However, their performance in low-resource scenarios remains limited due to a lack of diversity in generated samples and the information contained in the distribution of real samples doesn’t get fully exploited. To address these issues, we propose an Anchor-guided GAN with Contrastive Loss (AGCL) for low-resource OOD detection. In this model, two distinct anchor distributions are established as ground-truth distributions to guide GAN training, which prevents the model from collapsing to a narrow criterion. Furthermore, we introduce an extra contrastive loss for the generator to increase the distinction between the features of generated OOD samples and the limited real OOD samples provided by the dataset, thereby enhancing their diversity. This modification subsequently results in better performance of the anchor-guided GAN. Experimental results demonstrate that our proposed method outperforms existing methods in low-resource scenarios.

Cross-domain slot filling is a widely explored problem in spoken language understanding (SLU), which requires the model to transfer between different domains under data sparsity conditions. Dominant two-step hierarchical models first extract slot entities and then calculate the similarity score between slot description-based prototypes and the last hidden layer of the slot entity, selecting the closest prototype as the predicted slot type. However, these models only use slot descriptions as prototypes, which lacks robustness. Moreover, these approaches have less regard for the inherent knowledge in the slot entity embedding to suffer from the issue of overfitting. In this letter, we propose a Robust Multi-prototypes Aware Integration (RMAI) method for zero-shot cross-domain slot filling. In RMAI, more robust slot entity-based prototypes and inherent knowledge in the slot entity embedding are utilized to improve the classification performance and alleviate the risk of overfitting. Furthermore, a multi-prototypes aware integration approach is proposed to effectively integrate both our proposed slot entity-based prototypes and the slot description-based prototypes. Experimental results on the SNIPS dataset demonstrate the well performance of RMAI.

Multi-intent spoken language understanding (SLU) that can handle multiple intents in an utterance has attracted increasing attention. Previous studies treat the slot filling task as a token-level sequence labeling task, which results in a lack of entity-related information. In our paper, we propose anEntity-LevelSlotFilling (ELSF) framework for joint multiple intent detection and slot filling. In our framework, two entity-oriented auxiliary tasks, entity boundary detection and entity type assignment, are introduced as the regularization to capture the entity boundary and the context of type, respectively. Besides, to better utilize the entity interaction, we design an effective entity-level coordination mechanism for modeling the interaction in both entity-entity and intent-entity relationships. Experiments on five datasets demonstrate the effectiveness and generalizability of our ELSF.

Emotion Recognition in Conversation (ERC) has considerable prospects due to its wide range of applications. Most existing works integrate speaker information statically and capture a relatively consistent atmosphere in conversation. However, these works poorly track the emotional state dynamics of each party in a conversation and focus on emotion consistency. The speakers’ emotional states are independent but influence each other during the conversation. To address the above issues, we propose a Speaker Dynamics Tracking Network (SDTN) for ERC. Specifically, SDTN can dynamically track the local and global speaker states during emotional flow in conversation and capture implicit stimulation of emotional shift. Extensive experiments on MELD and EmoryNLP datasets demonstrate the superiority and effectiveness of our proposed SDTN model, and confirm that every designed module consistently benefits the performance.

Sequential Recommendation (SR) models user dynamics and predicts the next preferred items based on the user history. Existing SR methods model the ‘was interacted before’ item-item transitions observed in sequences, which can be viewed as an item relationship. However, there are multiple auxiliary item relationships, e.g., items from similar brands and with similar contents in real-world scenarios. Auxiliary item relationships describe item-item affinities in multiple different semantics and alleviate the long-lasting cold start problem in the recommendation. However, it remains a significant challenge to model auxiliary item relationships in SR.To simultaneously model high-order item-item transitions in sequences and auxiliary item relationships, we propose a Multi-relational Transformer capable of modeling auxiliary item relationships for SR (MT4SR). Specifically, we propose a novel self-attention module, which incorporates arbitrary item relationships and weights item relationships accordingly. Second, we regularize intra-sequence item relationships with a novel regularization module to supervise attentions computations. Third, for inter-sequence item relationship pairs, we introduce a novel inter-sequence related items modeling module. Finally, we conduct experiments on four benchmark datasets and demonstrate the effectiveness of MT4SR over state-of-the-art methods and the improvements on the cold start problem. The code is available in https://github.com/zfan20/MT4SR.

Dialogue intent classification is a fundamental and essential task in dialogue systems. Although sentence-level and document-level text classification have made dramatic progress in recent years with the help of deep learning technology, dialogue-level classification remains challenging. Dialogue has unique characteristics that distinguish it from other types of text. Dialogue is interactive, with feedback between speakers, and turn-taking. These unique features suggest that model architecture should take dialogue structure into account to learn a better representation. In this paper we propose an Adjacency Pairs-Aware Hierarchical Attention Network (AP-HAN) for dialogue intent classification. A dialogue reconstruction strategy is designed to match the question and answer utterances properly and then make the dialogue to be presented as a sequence of adjacent pairs. Then, the adjacency pairs features are incorporated into the hierarchical attention network. Experimental results on public CCL2018-Task1 corpus show the better performance of the proposed model.

Intent detection and slot filling are two important tasks for spoken language understanding. Considering the close relation between them, most existing methods joint them by sharing parameters or establishing explicit connection between them for potentially benefiting each other. However, most of them only consider single directional connection and ignore their cross-impact between them. Moreover, these joint methods treat the predicted labels as the gold labels, which may cause error propagation. In this paper, we propose a two-stage Graph Attention Interactive Refine (GAIR) framework. In stage one, the basic SLU model predicts the coarse intent and slots. In stage two, we select the top-k candidate labels from stage one and construct a graph to make full advantage of intent and slot filling information. By constructing such graph, our framework can establish a bidirectional connection between two tasks and refine the coarse result, which can better take full use of cross-impact between two tasks. Moreover, contextual regularization is introduced for better alleviating error propagation. Experiments on two datasets show that our model achieves the state-of-the-arts performance.

Multi-intent spoken language understanding (SLU) that can handle an utterance containing multiple intents is more practical and attracts increasing attention. However, existing state-of-the-art models are either too coarse-grained (Utterance-level) or too fine-grained (Token-level) in intent detection, and thus may fail to recognize the intent transition point and the correct intents in an utterance. In this paper, we propose a Chunk-Level Intent Detection (CLID) framework, where we introduce a sliding window-based self-attention (SWSA) scheme for regional chunk intent detection. Based on the SWSA, an auxiliary task is introduced to identify the intent transition point in an utterance and obtain sub-utterances with a single intent. The intent of each sub-utterance is then predicted by assembling the intent predictions of the chunks (in a sliding window manner) within it. We conduct experiments on two public datasets, MixATIS and MixSNIPS, and the results show that our model achieves state-of-the-art performance.

In domain classification for spoken language understanding, correct detection of out-of-domain (OOD) utterances is crucial because it reduces confusion and unnecessary interaction costs between users and the systems. In the situation where both in-domain (ID) and OOD samples are available, our goal is to take advantage of OOD samples under the GAN-based framework for OOD detection. We propose a GAN-based OOD detector with OOD prior distribution and weighted loss (WOODP-GAN). The model consists of a GAN-based detector with OOD prior distribution for generating effective pseudo OOD samples, and a weighted loss function for balancing the loss of fake OOD samples against real OOD samples in the discriminator. Extensive experiments show our proposed WOODP-GAN model outperforms the existing methods in the benchmark dataset CLINC150.

Host load prediction is one of the key research issues in Cloud computing. However, due to the drastic fluctuation of the host load in the Cloud, accurately predicting the host load remains a challenge. In this paper, a discriminative model (SVM) is employed to improve upon the accuracy of host load prediction in a Cloud data center. A rich set of features are generated by function based methods and incorporated into discriminative modelling. The performance of our proposed method is empirically evaluated using a one-month trace of a Google data center with over 12000 heterogeneous hosts. The results show that the proposed method achieves a better prediction performance than some state-of-the-art methods.

In order to better understand and describe tasks and improve the ability of Cloud, the analyzing of tasks inessential. A coarse-grained analysis, cluster analysis, anointer-cluster analysis are used to model tasks for the analysis of a one-month trace of a Google MapReduce cluster across about 12,000 machines. In this paper, we consider the k value which is central to the performance of k-means algorithm can effect on modelling. Besides, we also take the selection of attributes into account which are used as the dimension when tasks are classified. Experiment results by using different type of task attributes and k value show the well performance odour approach.

This paper presents a novel scheme for accelerating ray traversal computation in ray tracing. By the scheme, a pre-computed stage is applied to constructing what is called a traversal field for each rigid object that records the destinations for all possible incoming rays. The field data, which could be efficiently compressed offline, is stored in a small number of big rectangles called ray-relays that enclose each approximate convex segment of an object. In the ray-tracing stage, the records on relays are retrieved in a constant time, so that a ray traversal is implemented as a simple texture lookup on GPU. Thus, the performance of our approach is only related to the number of relays rather than scene size, while the number of relays is quite small. In addition, because the traversal fields only depend on the internal construction of each convex segment, they can be used to ray trace objects undergoing rigid motions at a negligible extra cost. Experimental results show that interactive rates could be achieved for dynamic scenes with the effects of specular reflections and refractions on an ordinary desk PC with GPU.