Graph encoding and the attention mechanism enable Graph Transformers (GTs) to extract features of graph structures. However, employing graph encoding and Transformer-based attention mechanism may lead to two defects: high computing complexity and sensitivity to graph structures. Therefore, this paper designs a graph feature extractor with a pure or lightweight attention mechanism that does not rely on graph encoding or feature projection matrices, called Collaborative Filtering Transformer (CFT). CFT uses the pure attention mechanism as the main structure of the encoder and introduces non-interaction information by constructing preference sentences. The core of CFT is that its encoder does not contain any collaborative information, but only plays a role in generating associations between different nodes, so that a node on the graph can notice its non-neighboring nodes. The utilization of collaborative information is only achieved through optimizing the loss function. In addition, through theoretical analysis and experimental verification, we prove that adding path-based graph encoding in CFT has a negative effect on the feature extraction process of the attention mechanism. Furthermore, experiments show that during the optimization process, the proposed pure attention mechanism can always assign higher attention scores to nodes with interactions, while making the attention scores between nodes without interactions approach zero. Finally, our model achieves the best performance when compared with the latest methods on five real-world datasets, and compared to the second-best baseline, the recommendation performance is improved by up to 20.60%. Moreover, CFT achieves considerably high training efficiency across all five datasets, with training time comparable to that of simple matrix factorization-based baselines.

Click-Through Rate (CTR) prediction is a critical task in commercial recommender systems. Feature embedding is the cornerstone of CTR models. Most previous work on CTR prediction to improve the model performance focused on modelling feature interactions. However, the quality of embeddings and the effective information of embeddings carried are overlooked. As a result, performing feature interactions on inferior embeddings will be a bottleneck to capture informative knowledge. To address this problem, we optimize embeddings from the perspective of expanding the weight gap and masking noisy information of the embedding vector. We first propose a novel module named Simple Stacked Gate (SSG), which selects the salient information of feature embeddings and enhances the weight gap between it and other information. Based on this, we further propose an Automatic Stacked Gate (AutoSG) framework, which utilizes Neural Architecture Search (NAS) method to automatically identify and mask noisy information of the embedding vector obtained after the SSG module. Both SSG and AutoSG can be seen as plug-and-play components and added to any existing deep CTR models flexibly. Extensive experiments on four benchmark datasets demonstrate that these two methods can further improve the performance of the backbone CTR models. Even only half of the bit information retained in the optimized embeddings, better performance can still be achieved compared to the original model. The implementation of our work can be found at https://github.com/SG-CTR/Stacked-Gate.

Graph Convolutional Neural Networks (GCNs) have performed well in many recommendation scenarios. In spite of this, recommendation models based on GCNs still face problems such as insufficient information mining and high complexity for some existing models. To address the above problems, we propose a Graph Convolutional Neural Network for Recommendation Based on Community Detection and the Combination of Multiple Heterogeneous Graphs (GCN-CMHG). This model uses the community detection algorithm to detect the communities in the user-item interaction heterogeneous graph (UIIHG), Finds the regional central nodes of communities, and then creates edges between the regional central node of each community and all other nodes in the UIIHG to construct the heterogeneous partial adjacent graph. Then, a Heterogeneous Partial Adjacent Auxiliary (HPAA) layer is designed to aggregate information on the heterogeneous partial adjacent graph. HPAA layer expands the influence of distant nodes on target nodes, enables target nodes to receive global information, and enhances the ability of GCN-CMHG to mine information. Specially, due to the low complexity of HPAA layer and the abandonment of redundant information, GCN-CMHG is easier to implement and train. Under the exact same experimental setting, GCN-CMHG’s time consumption is only about 1/10 of another model based on GCN called Graph Convolutional Neural Network for Recommendation Based on the Combination of Multiple Heterogeneous Graphs (GCN-MHG). Experiments on multiple real-world datasets show that GCN-CMHG achieves better results compared with several advanced models. The implementation of our work can be found at https://github.com/GCNRSs/GCN-CMHG.