Recently, dense retrieval method has significantly outperformed sparse retrieval technology. It becomes a main-stream approach of relevant passage retrieving task. Dense retrieval tasks encode query and passage into dense representation space and apply contrastive learning to obtain more representative vectors, then retrieve the most similar passage by the inner product of the dense vectors. However, previous achievements rely on extremely large batch size, epoch and large-scaled pre-trained models, which results in tremendous computational resource consumption. Also, the expensive equipment prerequisite and low training efficiency constrain the development of dense retrieval community. Therefore, we present an alternative solution to improve training efficiency and quality by syntactical contrastive learning methods with specially designed masking strategy. To alleviate the computational consumption problem, this paper proposes query-based and passage-based masking strategies to obtain syntactical-isolated representations. Besides, instead of only considering query-to-passage similarity while conducting contrastive learning, we additionally consider query-to-query and passage-to-passage similarity when training the dual-encoder retriever. The experiments show that the proposed approach achieved competitive results in small batch size and epoch comparing to previous state-of-the-art dense retrieval methods, and also to strong baseline.

Abstract The text matching is a basic task of NLP and is important for tasks such as text retrieval, question answering, and so forth. The development of pre‐trained language models has promoted the progress of text matching tasks. However, due to the natural particularity of Chinese characters and expressions, the Chinese text matching tasks still have problems such as word segmentation difficulty, serious semantic loss, and model instability. In this paper, we propose the DAINet model, which includes DMM, AA and IO modules. We use the Dynamic Multi‐Mask module (DMM) to enhance the completeness of word segmentation. Then we use the Augmented Adversarial module (AA) to further extraction of semantic information. Finally, we use the Integrated Output module (IO) for a more stable output. We conducted experiments on LCQMC, BQ and Xiaobu datasets and compared the results with seven strong baseline models. The results showed that DAINet model made great improvement, including improving ACC value of BQ dataset to , AUC value to , ACC value of LCQMC dataset to and AUC value to . The ACC value of Xiaobu dataset was improved to and the AUC value was improved to . Further ablation experiment results show that the proposed DMM, AA and IO modules have good adaptability and improvement over existing models.

Dense text retrieval has become a widely used paradigm for recalling existing language models , and efficient dense text retrieval is essential for obtaining sufficiently accurate candidate samples. However, the existing methods for dense text retrieval, which typically use dual-encoder architectures to match similar queries and documents, suffer from a lack of information interaction at low data volumes, resulting in suboptimal performance. Additionally, existing research relies on negative sampling techniques that do not take into account the negative effects of single negative sampling bias on the robustness of the model. These limitations hinder the development of more robust dense text retrieval models . In this paper, we propose a multi-view contrast learning architecture, named MvCR, to address these issues. MvCR improves the performance of dense text retrieval by performing contrast learning with multiple views while significantly increasing the model’s ability to discriminate between positive and negative samples. Additionally, we propose a data augmentation method that focuses on increasing the number of hard negative samples with accurate and semantic matching features. The experimental results have shown that MvCR can perform as well as strong baseline models even when the data volume is small. Furthermore, MvCR achieved better results on two popular retrieval benchmarks with comparable amounts of data. Specifically, MRR@10 was 39 . 1 ( + 0 . 9 % ) and Recall@50 was 87 . 8 ( + 1 . 4 % ) on the MS-MARCO dataset. And Recall@5 increased to 77 . 2 ( + 1 . 8 % ) and Recall@50 increased to 85 . 3 ( + 1 . 0 % ) on the Natural Questions dataset.

Because social networks have become a vital part of people's lives, cyberbullying becomes the most common risk encountered by young people on social networking platforms and raised serious concerns in society. Over the past few decades, most existing work on cyberbullying has focused on text analysis. Yet, the cyberbullying develops into multi-objective, multi-channel, and multi-form. Traditional text analysis methods cannot satisfy the diversity of bullying data in social networks. To deal with the new type of cyberbullying, we propose a multi-modal detection framework that takes into multi-modal information(e.g., image, video, comments, time) on social networks. Specifically, we not only extract textual features but also use the hierarchical attention networks to capture the session feature in social networks and encode several media information(e.g., video, image). Based on these features, we model the multi-modal cyberbullying detection framework to solve the new form of cyberbullying. Experimental analysis on two real-world datasets shows that our framework outperforms several existing state-of-the-art models.

Service discovery aims to address the problem of service information explosion and find and locate services that meet the needs of service requesters. Because service description information is mostly composed of short text with noise and has the characteristics of semantic sparseness, it is difficult to extract the implied context information of service description. This paper proposes a service discovery framework based on Knowledge graphs and neural Similarity Network (KSN). Which uses knowledge graphs to connect entities to obtain rich external information to enhance the semantic information of service descriptions. convolutional neural network and similarity network is utilized to extract context information. Through extensive experiments on a real service data set show that KSN is superior to existing web service discovery methods in terms of multiple evaluation metrics.