The task of text semantic matching focuses on measuring the semantic similarity between two texts and is widely applied in search and ranking scenarios. In recent years, pre-trained foundation models based on the Transformer architecture have demonstrated powerful semantic representation capabilities. The pipeline of fine-tuning pre-trained foundation models on downstream semantic matching tasks has achieved promising results and widespread adoption. However, practical downstream scenarios often face severe challenges in terms of data quality and quantity. Ensuring high-quality and large quantities of samples is often difficult. Current research on enhancing pre-trained models for few-shot text semantic matching tasks is still not advanced enough. Therefore, this paper focuses on providing a general enhancement scheme for few-shot text semantic matching tasks. Specifically, we propose an Enhancing Transformer-based Semantic Matching method for few-shot learning through weakly contrastive pre-training, which is named as SEMFormer. First, starting from the token-level and structural-level perspectives, we design a simple and low-cost data augmentation method to construct weakly supervised samples. Then, we use the global semantic representations to construct a contrastive objective from the relation-aspect perspective. Next, we design a contrastive objective based on the alignment-aspect, aiming to achieve effective semantic matching by optimizing the bidirectional semantic awareness between texts. We conducted comprehensive experiments based on five Chinese and English datasets. The experimental results validated that our proposed weakly contrastive pre-training augmentation method significantly improves model performance. Further experiments confirmed the effectiveness of our design. The source code is available at: https://github.com/llm-ml/SEMFormer.

Real recommendation systems contain various features, which are often high-dimensional, sparse, and difficult to learn effectively. In addition to numerical features, user reviews contain rich semantic information including user preferences, which are used as auxiliary features by researchers. The methods of supplementing data features based on reviews have certain effects. However, most of them simply concatenate review representations and other features together, without considering that the text representation contains a lot of noise information. In addition, the important intentions contained in user reviews are not modeled effectively. In order to solve the above problems, we propose a novel Review-based Multi-intention Contrastive Learning (RMCL) method. In detail, RMCL proposes an intention representation method based on mixed Gaussian distribution hypothesis. Further, RMCL adopts a multi-intention contrastive strategy, which establishes a fine-grained connection between user reviews and item reviews. Extensive experiments on five real-world datasets demonstrate significant improvements of our proposed RMCL model over the state-of-the-art methods.

The automatic generation of music comments is of great significance for increasing the popularity of music and the music platform's activity.In human music comments, there exists high distinction and diverse perspectives for the same song.In other words, for a song, different comments stem from different musical perspectives.However, to date, this characteristic has not been considered well in research on automatic comment generation.The existing methods tend to generate common and meaningless comments.In this paper, we propose an effective multiperspective strategy to enhance the diversity of the generated comments.The experiment results on two music comment datasets show that our proposed model can effectively generate a series of diverse music comments based on different perspectives, which outperforms state-of-the-art baselines by a substantial margin. 1