Sentence representation learning (SRL) aims to learn sentence embeddings that conform to the semantic information of sentences.In recent years, fine-tuning methods based on pre-trained models and contrastive learning frameworks have significantly advanced the quality of sentence representations.However, within the semantic space of SRL models, both word embeddings and sentence representations derived from word embeddings exhibit substantial redundant information, which can adversely affect the precision of sentence representations.Existing approaches predominantly optimize training strategies to alleviate the redundancy problem, lacking fine-grained guidance on reducing redundant representations.This paper proposes a novel approach that dynamically identifies and reduces redundant information in a dimensional perspective, training the SRL model to redistribute semantics on different dimensions, and entailing better sentence representations.Extensive experiments across seven semantic text similarity benchmarks demonstrate the effectiveness and generality of the proposed method.A comprehensive analysis of the experimental results is conducted and the code/data will be released.

Simile tasks are challenging in natural language processing (NLP) because models require adequate world knowledge to produce predictions. In recent years, pre-trained language models (PLMs) have succeeded in NLP since they learn generic knowledge from a large corpus. The knowledge embedded in PLMs can be used for different kinds of Simile tasks. However, previous work usually explored one type of simile knowledge for a specific simile task, how to fully utilize different types of knowledge embedded in the PLMs requires further exploration. This paper proposes a self-verified method for exploring simile knowledge from PLMs, which allows the PLMs to leverage one type of simile knowledge to self-validate another. To this end, we first enhance PLMs with a novel multi-level simile recognition (MLSR) task that trains PLMs to evaluate the quality of similes. Then the PLMs leverage this evaluation score to assist the simile interpretation and generation tasks. In this way, we connect different types of simile knowledge in PLMs and make better use of them. Experiments on different pre-trained models and multiple publicly available datasets show that our method works for different kinds of PLMs and can explore more accurate simile knowledge for PLMs. Our code/data will be released on GitHub.

Document-grounded dialogue (DGD) uses documents as external knowledge for dialogue generation. Correctly understanding the dialogue context is crucial for selecting knowledge from the document and generating proper responses. In this article, we propose using a dialogue policy to help the dialogue understanding in DGD. Our dialogue policy consists of two kinds of guiding signals: utterance function and topic transfer intent. The utterance function reflects the purpose and style of an utterance, and the topic transfer intent reflects the topic and content of an utterance. We propose a novel framework exploiting our dialogue policy for two core tasks in DGD, namely, knowledge selection (KS) and response generation (RG). The framework consists of two modules: the policy planner leverages policy-aware dialogue representation to select knowledge and predict the policy of the response; the generator uses policy/knowledge-aware dialogue representation for response generation. Our policy-driven model gets state-of-the-art performance on three public benchmarks, and we provide a detailed analysis of the experimental results. Our code/data will be released on GitHub.

This paper introduces a novel Self-supervised Fine-grained Dialogue Evaluation framework (SelF-Eval). The core idea is to model the correlation between turn quality and the entire dialogue quality. We first propose a novel automatic data construction method that can automatically assign fine-grained scores for arbitrarily dialogue data. Then we train SelF-Eval with a multi-level contrastive learning schema which helps to distinguish different score levels. Experimental results on multiple benchmarks show that SelF-Eval is highly consistent with human evaluations and better than the state-of-the-art models. We give a detailed analysis of the experiments in this paper. Our code is available on GitHub.

Incorporating external knowledge into dialogue generation has been proven to benefit the performance of an open-domain Dialogue System (DS), such as generating informative or stylized responses, controlling conversation topics. In this article, we study the open-domain DS that uses unstructured text as external knowledge sources ( U nstructured T ext E nhanced D ialogue S ystem ( UTEDS )). The existence of unstructured text entails distinctions between UTEDS and traditional data-driven DS and we aim at analyzing these differences. We first give the definition of the UTEDS related concepts, then summarize the recently released datasets and models. We categorize UTEDS into Retrieval and Generative models and introduce them from the perspective of model components. The retrieval models consist of Fusion, Matching, and Ranking modules, while the generative models comprise Dialogue and Knowledge Encoding, Knowledge Selection (KS), and Response Generation modules. We further summarize the evaluation methods utilized in UTEDS and analyze the current models’ performance. At last, we discuss the future development trends of UTEDS, hoping to inspire new research in this field.

Link prediction predicts the likelihood of a future association between two nodes in a network. It plays an important role in mining and analyzing the evolution of networks and it is the fundament of many applications, including bioinformatics, e-commerce, security domain and co-authorship networks. The past few decades has witnessed the development of link prediction. In this paper, we propose a link prediction method VirtualLink2vec, which learns features of target link by concatenating representations from two different aspects as the link's full representation. Comprehensive experiments are conducted on six networks. Compared with heuristics and latent feature methods, our results show that VirtualLink2vec has the ability to reserve rich information related to positive link and negative link with less consumption of time and computing resources. In the meantime, our method consistently performs well across networks with different structures and characteristics.

Answer selection is an important subtask of the question answering domain in natural language processing(NLP) applications. In this task, attention mechanism is a widely used technique which focuses on the context information and interrelationship between different words in the sentences to allocate different weight and enhance feature. However, the natural characteristics of words themselves are not fully excavated, thus the performance may be limited to a certain extent. In this paper, we propose a novel Hierarchical Multidimensional Attention (HMDA) model to address this issue. Especially, HMDA proposes a new kind of attention mechanism, word-attention, a true individual attention which can enhance the implied meaning of the word itself to extract features from word level which are more unique. Then HMDA uses global co-attention to better utilize word-attention and capture more common similar features. In order to utilize this attention-based semantic information on different granularities differently, HMDA designs a multi-layer structure which makes full use of all attention mechanisms by embedding attention features to model hierarchically. HMDA obtains various fine-grained information between question and candidate answers and avoids information loss. Empirically, we demonstrate that our proposed model can consistently outperform the state-of-the-art baselines under different evaluation metrics on all TrecQA, WikiQA and InsuranceQA datasets.

Selecting features that represent a particular corpus is important to the success of many natural language processing tasks. However, the previous attention-based work only focused on feature augmentation in the lexical level, lacking the exploration of feature enhancement in the sentence level. In this paper, we exploit a novel feature generation method for information retrieval, denoted by Multi-perspective Feature Generation Network (MFGN). We first add features at sentence level from multiple perspectives to enhance sentence representation, then the word level features make second interactive confirmation from two perspectives. After two-level processing for different granularity characteristics, the model complete the retrieval process. It is more in line with the cognitive process of human beings to grasp the overall meaning of sentences and then carefully study the potential meaning of specific words. In the process of feature generation and extraction, text interacts from multiple perspectives, takes into account the information of text itself and the information of text interaction in different levels and multiple granularities. Different kinds of characteristics are distilled according to specific tasks, enhancing the practicability and robustness of the model. MFGN relies solely on the text itself, requires no external knowledge or feature engineering. Experimental results show that our approach outperforms previous work on multiple well-known Retrieval-based question answering datasets. Through the analysis of the experiments, we prove that MFGN provides excellent retrieval and interpretative abilities.