Sequential recommendation has attracted a lot of attention from both academia and industry, however the privacy risks associated with gathering and transferring users’ personal interaction data are often underestimated or ignored. Existing privacy-preserving studies are mainly applied to traditional collaborative filtering or matrix factorization rather than sequential recommendation. Moreover, these studies are mostly based on differential privacy or federated learning, which often lead to significant performance degradation, or have high requirements for communication. In this work, we address privacy-preserving from a different perspective. Unlike existing research, we capture collaborative signals of neighbor interaction sequences and directly inject indistinguishable items into the target sequence before the recommendation process begins, thereby increasing the perplexity of the target sequence. Even if the target interaction sequence is obtained by attackers, it is difficult to discern which ones are the actual user interaction records. To achieve this goal, we introduce a novel sequential recommender system called CoLlaborative-cOnfusion seqUential recommenDer (CLOUD) , which incorporates a collaborative confusion mechanism to modify the raw interaction sequences before conducting recommendation. Specifically, CLOUD first calculates the similarity between the target interaction sequence and other neighbor sequences to find similar sequences. Then, CLOUD considers the shared representation of the target sequence and similar sequences to determine the operation to be performed: keep, delete, or insert. A copy mechanism is designed to make items from similar sequences have a higher probability to be inserted into the target sequence. Finally, the modified sequence is used to train the recommender and predict the next item. We conduct extensive experiments on three benchmark datasets. The experimental results show that CLOUD achieves a maximum modification rate of 66.57% on interaction sequences and obtains over 99% recommendation accuracy compared to the state-of-the-art sequential recommendation methods. This proves that CLOUD can effectively protect user privacy at minimal recommendation performance cost, which provides a new solution for privacy-preserving for sequential recommendation. Our implementation is available at https://github.com/weiwang0927/CLOUD .

Modern recommender systems heavily rely on high-quality representations learned from high-dimensional sparse data. While significant efforts have been invested in designing powerful algorithms for extracting user preferences, the factors contributing to good representations have remained relatively unexplored. In this work, we shed light on an issue in the existing pairwise learning paradigm (i.e., embedding collapse), that the representations tend to span a subspace of the whole embedding space, leading to a suboptimal solution and reducing the model capacity. Specifically, we show that alignment of positive pairs is equivalent to a low-pass filter causing users and items to collapse to a constant vector. While negative sampling can partially mitigate this issue by acting as a high-pass filter to balance the spectrum, leading to an incomplete collapse. To tackle this issue, we present a novel learning paradigm DirectSpec, which directly optimizes the spectrum distribution to ensure that users and items effectively span the entire embedding space. We demonstrate that many self-supervised learning algorithms without explicit negative sampling can be considered as special cases of DirectSpec. Furthermore, we show that optimizing the spectrum inappropriately could also be detrimental to data representation, where the key lies in a dynamic balance between alignment of positive pairs and spectrum balancing. Finally, we propose an enhanced and practical implementation DirectSpec + to balance the embedding spectrum more adaptively and effectively. We implement DirectSpec + on two popular recommender models: matrix factorization and LightGCN. Our experimental results demonstrate its effectiveness and efficiency over competitive baselines.

Real-world problems in e-commerce and other practical services are framed as nonstationary and contextual multi-armed bandit (MAB) problems, driving the active development of MAB policies. Current MAB policies leverage nonlinear regression models, such as neural networks and Gaussian processes (GP). However, these regression models lack a recursive learning mechanism in nonstationary environments. As a result, MAB policies must consider all training data amassed over an extended period of time, leading to escalating computational complexity in sequential decision-making as the volume of training data grows. To address this online performance issue, we propose an online nonstationary and nonlinear contextual MAB policy using a GP regression model that employs Random Fourier Features and weighted Recursive Least Squares with regularization. The proposed policy enables fast decision-making based on recursive learning and accurately corrects the estimation error of the predictive distribution of the GP caused by recursive weighting the regularization at any given time. Simulation results of the nonstationary and nonlinear contextual MAB problem show that the proposed policy significantly reduces the computation time while maintaining the cumulative reward, and the proposed error correction method resolves the trade-off between the estimation accuracy of the prediction distribution and the computation time.

It has been shown that the effectiveness of graph convolutional network (GCN) for recommendation is attributed to the spectral graph filtering. Most GCN-based methods consist of a graph filter or followed by a low-rank mapping optimized based on supervised training. However, we show two limitations suppressing the power of graph filtering: (1) Lack of generality. Due to the varied noise distribution, graph filters fail to denoise sparse data where noise is scattered across all frequencies, while supervised training results in worse performance on dense data where noise is concentrated in middle frequencies that can be removed by graph filters without training. (2) Lack of expressive power. We theoretically show that linear GCN (LGCN) that is effective on collaborative filtering (CF) cannot generate arbitrary embeddings, implying the possibility that optimal data representation might be unreachable.

While Graph Convolutional Networks (GCNs) have shown great potential in recommender systems and collaborative filtering (CF), they suffer from expensive computational complexity and poor scalability. On top of that, recent works mostly combine GCNs with other advanced algorithms which further sacrifice model efficiency and scalability. In this work, we unveil the redundancy of existing GCN-based methods in three aspects: (1) Feature redundancy . By reviewing GCNs from a spectral perspective, we show that most spectral graph features are noisy for recommendation, while stacking graph convolution layers can suppress but cannot completely remove the noisy features, which we mostly summarize from our previous work; (2) Structure redundancy . By providing a deep insight into how user/item representations are generated, we show that what makes them distinctive lies in the spectral graph features, while the core idea of GCNs (i.e., neighborhood aggregation) is not the reason making GCNs effective; and (3) Distribution redundancy . Following observations from (1), we further show that the number of required spectral features is closely related to the spectral distribution, where important information tends to be concentrated in more (fewer) spectral features on a flatter (sharper) distribution. To make important information be concentrated in as few features as possible, we sharpen the spectral distribution by increasing the node similarity without changing the original data, thereby reducing the computational cost. To remove these three kinds of redundancies, we propose a Simplified Graph Denoising Encoder (SGDE) only exploiting the top- K singular vectors without explicitly aggregating neighborhood, which significantly reduces the complexity of GCN-based methods. We further propose a scalable contrastive learning framework to alleviate data sparsity and to boost model robustness and generalization, leading to significant improvement. Extensive experiments on three real-world datasets show that our proposed SGDE not only achieves state-of-the-art but also shows higher scalability and efficiency than our previously proposed GDE as well as traditional and GCN-based CF methods.

Out-of-Domain (OOD) text detection has attracted significant research interest. However, conventional approaches primarily employ Cross-Entropy loss during upstream encoder training and seldom focus on optimizing discriminative In-Domain (IND) and OOD representations. To fill this gap, we introduce a novel method that applies supervised contrastive learning (SCL) to IND data for upstream representation optimization. This effectively brings the embeddings of semantically similar texts together while pushing dissimilar ones further apart, leading to more compact and distinct IND representations. This optimization subsequently improves the differentiation between IND and OOD representations, thereby enhancing the detection effect in downstream tasks. To further strengthen the ability of SCL to consolidate IND embedding clusters, and to improve the generalizability of the encoder, we propose a method that generates two different variations of the same text as "views". This is achieved by applying a twice "dropped-out" on the embeddings before performing SCL. Extensive experiments indicate that our method not only outperforms state-of-the-art approaches, but also reduces the requirement for training a large 354M-parameter model down to a more efficient 110M-parameter model, highlighting its superiority in both effectiveness and computational economy.

Selecting an optimal recommendation method is crucial for an electronic commerce (EC) site. However, the effectiveness of recommendation methods cannot be known in advance. Although continuous comparative evaluation in an actual environment is essential, it results in opportunity loss. To overcome this problem, opportunity loss reduction has been studied as a multi-armed bandit (MAB) problem, and adaptive meta-recommender systems (meta-RS) were devised to automatically and continuously select the best recommendation method according to a policy. The following three factors cause opportunity loss: the context of the recommendation method, temporal variation, and response time. However, studies are yet to formulate an MAB policy that considers all three factors. Thus, reducing opportunity loss remains a problem. We propose an MAB policy that considers all three causes of opportunity loss by using a Kalman filter for a linear Gaussian state space model. We conducted extensive experiments to select the best recommendation method using data from a real EC site. The results revealed that the proposed policy is highly effective for reducing opportunity loss of the meta-RS during evaluation and increasing cumulative clicks compared with baseline policies.

With the tremendous success of Graph Convolutional Networks (GCNs), they have been widely applied to recommender systems and have shown promising performance. However, most GCN-based methods rigorously stick to a common GCN learning paradigm and suffer from two limitations: (1) the limited scalability due to the high computational cost and slow training convergence; (2) the notorious over-smoothing issue which reduces performance as stacking graph convolution layers. We argue that the above limitations are due to the lack of a deep understanding of GCN-based methods. To this end, we first investigate what design makes GCN effective for recommendation. By simplifying LightGCN, we show the close connection between GCN-based and low-rank methods such as Singular Value Decomposition (SVD) and Matrix Factorization (MF), where stacking graph convolution layers is to learn a low-rank representation by emphasizing (suppressing) components with larger (smaller) singular values. Based on this observation, we replace the core design of GCN-based methods with a flexible truncated SVD and propose a simplified GCN learning paradigm dubbed SVD-GCN, which only exploits K-largest singular vectors for recommendation. To alleviate the over-smoothing issue, we propose a renormalization trick to adjust the singular value gap, resulting in significant improvement. Extensive experiments on three real-world datasets show that our proposed SVD-GCN not only significantly outperforms state-of-the-arts but also achieves over 100x and 10x speedups over LightGCN and MF, respectively.

As much as Graph Convolutional Networks (GCNs) have shown tremendous success in recommender systems and collaborative filtering (CF), the mechanism of how they, especially the core components (\textiti.e., neighborhood aggregation) contribute to recommendation has not been well studied. To unveil the effectiveness of GCNs for recommendation, we first analyze them in a spectral perspective and discover two important findings: (1) only a small portion of spectral graph features that emphasize the neighborhood smoothness and difference contribute to the recommendation accuracy, whereas most graph information can be considered as noise that even reduces the performance, and (2) repetition of the neighborhood aggregation emphasizes smoothed features and filters out noise information in an ineffective way. Based on the two findings above, we propose a new GCN learning scheme for recommendation by replacing neihgborhood aggregation with a simple yet effective Graph Denoising Encoder (GDE), which acts as a band pass filter to capture important graph features. We show that our proposed method alleviates the over-smoothing and is comparable to an indefinite-layer GCN that can take any-hop neighborhood into consideration. Finally, we dynamically adjust the gradients over the negative samples to expedite model training without introducing additional complexity. Extensive experiments on five real-world datasets show that our proposed method not only outperforms state-of-the-arts but also achieves 12x speedup over LightGCN.

Product reviews are more and more important for potential customers to decide on their purchases in electronic commerce nowadays. The helpful vote is a critical indicator of how much impact the review has on other customers. Therefore, the prediction of helpful votes is an essential task. Linear and Tobit Regression are general methods of the prediction. Those methods share the same objective function and come from the initial assumption that the helpful votes on any dataset follow a normal distribution. However, the assumption is not usually confirmed, and the distribution of the helpful votes often follows other distributions. Consequently, the prediction results might not be fully appropriate. This paper proposes a model that follows the best approximate distribution of helpful votes to predict the number of helpful votes. On top of that, considering the elapsed time since reviews were written, we propose an adaptive window size sampling method to evaluate the model on review datasets sorted chronologically. To validate the proposed model, we conducted extensive experiments on real-world datasets. Experimental results illustrate the validity of the proposed model.

Prediction of bus travel time and/or delay time is a useful tool for passengers who want to plan their journey, e.g., when they should leave from the origin bus stop, what they will do after arriving at the destination bus stop, and so on. Many studies have tackled this task using probe data and/or the real time data provided by automatic vehicle location (AVL) systems. Most of them only targeted a small number of routes, short time periods, e.g. less than one week, and used few machine learning models to evaluate their methods. However, different routes generally show different characteristics. In fact, there are big differences between urban routes and rural routes. Furthermore, the performance of machine learning models also varies according to the data dealt with by the models. In this paper, we propose prediction models for bus delay over all intervals between pairs of adjacent bus stops. To build the models, we use one month of bus probe data, which includes more than 80 routes, and apply several machine learning models: linear regression (LR), artificial neural network (ANN), support vector regression (SVR), random forest (RF), and gradient boosting decision tree (GBDT). Experimental results demonstrate the superiority of the GBDT-based prediction model and the effects of considering travel time over prior intervals.

Government 2.0 activities have become very attractive and popular these days. Using platforms to support the activities, anyone can anytime report issues or complaints in a city with their photographs and geographical information on the Web, and share them with other people. Since a variety of reports are posted, officials in the city management section have to check the importance of each report and sort out their priorities to the reports. However, it is not easy task to judge the importance of the reports. When several officials work on the task, the agreement rate of their judgments is not always high. Even if the task is done by only one official, his/her judgment sometimes varies on a similar report. To remedy this low agreement rate problem of human judgments, we propose a method of detecting signs of danger or unsafe problems described in citizens' reports. The proposed method uses a machine learning technique with word feature selection. Experimental results clearly explain the low agreement rate of human judgments, and illustrate that the proposed machine learning method has much higher performance than human judgments.

Today, the use of learning analytics is becoming more crucial in the learning environment for the purpose of understanding and optimizing students' learning situations. The purpose of this paper is to examine the impacts of Teacher Interventions (TIs) on students' attitudes and achievements involved with the lesson by analyzing their freestyle comment data after every lesson. The current study proposes a new method for building an accessible prediction model, which represents students' activities, situations and viewpoints; the method classifies words in the student comments into six attribute types and indicates the most important types that affect the prediction results. Further, the prediction results are compared with the topic-based statistical method that uses Latent Dirichlet Allocation and Support Vector Machine models. The results proved that there were positive correlations between TIs and the quality of writing comments that affect on improving the prediction results.

Government 2.0 activities have become very attractive and popular these days. Using platforms to support the activities such as FixMyStreet, SeeClickFix, or CitySourced, anyone can anytime report issues or complaints in a city with their photographs and geographical information on the Web, and share them with other people. On the other hand, unlike telephone calls, the concreteness of a report depends on its reporter; the actual status and demand to the status may not be described clearly or either one may be miss-described in the report. It may accordingly happen that officials in the city management section can not understand the actual status or a demand to the status from the report. To solve the problems, it is indispensable to complement missing information and estimate the actual status or the demand to the status from ambiguous information in the report. This paper proposes novel methods to detect segments related to an actual status and the demand to the status in a report. The methods combine empirical rules with several machine learning techniques that actively use dependency relation between words. Experimental results illustrate the validity of the proposed methods.

Software libraries have become more and more complex in recent years. Developers usually have to rely on search engines to find API documents and then select suitable APIs to do relevant development when working on unfamiliar functions. However, the traditional search engines do not focus on searching APIs that make this process inconvenient and time consuming. Although a lot of efforts have been made on API understanding and code search in industry and academia, work and tools that can recommend API methods to users based on their description of API's functionality are still very limited.

Learning analytics is valuable sources of understanding students' behavior and giving feedback to them so that we can improve their learning activities. Analyzing comment data written by students after each lesson helps to grasp their learning attitudes and situations. They can be a powerful source of data for all forms of assessment. In the current study, we break down student comments into different topics by employing two topic models: Probabilistic Latent Semantic Analysis (PLSA), and Latent Dirichlet Allocation (LDA), to discover the topics that help to predict final student grades as their performance. The objectives of this paper are twofold: First, determine how the three time-series items: P-, C- and N-comments and the difficulty of a subject affect the prediction results of final student grades. Second, evaluate the reliability of predicting student grades by considering the differences between prediction results of two consecutive lessons. The results obtained can help to understand student behavior during the period of the semester, grasp prediction error occurred in each lesson, and achieve further improvement of the student grade prediction.

Assessment of learning progress and learning gain play a pivotal role in education fields. New technologies like comment data mining promote the use of new types of contents; student comments highly reflect student learning attitudes and activities compared to more traditional methods and they can be a powerful source of data for all forms of assessment. A teacher just asks students after every lesson to freely describe and write about their learning situations and behaviors. This paper proposes new methods based on a statistical latent class "Topics" for the task of student grade prediction; our methods convert student comments using latent semantic analysis (LSA) and probabilistic latent semantic analysis (PLSA), and generate prediction models using support vector machine (SVM) and artificial neural network (ANN) to predict student final grades. The experimental results show that our methods can accurately predict student grades based on comment data.

Predicting students' academic performance has long been an important research topic in many academic disciplines. The prediction will help the tutors identify the weak students and help them score better marks; these steps were taken to improve the performance of the students. The present study uses free style comments written by students after each lesson. These comments reflect their learning attitudes to the lesson, understanding of subjects, difficulties to learn, and learning activities in the classroom. (Goda and Mine, 2011) proposed PCN method to estimate students' learning situations from their comments freely written by themselves. This paper uses C (Current) method from the PCN method. The C method only uses comments with C item that focuses on students' understanding and achievements during the class period. The aims of this study are, by applying the method to the students' comments, to clarify relationships between student's behaviour and their success, and to develop a model of students' performance predictors. To this end, we use Latent Semantic Analyses (LSA) and K-means clustering techniques. The results of this study reported a model of students' academic performance predictors by analysing their comment data as variables of predictors.

Predicting students' academic achievement with high accuracy has an important vital role in many academic disciplines. Most recent studies indicate the important role of the data type selection. They also attempt to understand individual students more deeply by analyzing questionnaire for a particular purpose. The present study uses free-style comments written by students after each lesson, to predict their performance. These comments reflect their learning attitudes to the lesson, understanding of subjects, difficulties to learn, and learning activities in the classroom. To reveal the high accuracy of predicting student's grade, we employ (LSA) latent semantic analysis technique to extract semantic information from students' comments by using statistically derived conceptual indices instead of individual words, then apply (ANN) artificial neural network model to the analyzed comments for predicting students' performance. We chose five grades instead of the mark itself to predict student's final result. Our proposed method averagely achieves 82.6% and 76.1% prediction accuracy and F-measure of students' grades, respectively.

According to user's intent, Broder classified Web queries into three classes: navigational, informational and transactional. However, current search engines do not provide relevant search results based on the user's intent. In this paper, we propose a user intent based searching mechanism (UIBS for short) in order to enable precise discovery of a Web site (for navigational searching) as well as aggregating of their information (for informational searching) and performing further activities (for transactional searching). We first propose a Web site capability description interface for the explicit description of a Web site's capability and a search interface that enables explicit description of user's search intents, preferences, and query on the information. And then we describe the process of the query engine which provides relevant search results based on the user's intent, preferences, and requirements. Finally, a test example is given to show the availability of the proposed UIBS mechanism.

In Web retrieval, it is often the case that the results given by search engines are not just what we want. To solve this problem there have been many studies on improving queries to be submitted to search engines. However, they are still insufficient due to lack of consideration on time information. To remedy this we propose a user-schedule-based Web page recommendation method. The method makes use of a user's schedule to make the recommendation suite to his/her requests. In addition, an algorithm for extracting related words is introduced as a key technique in this method. Some preliminary experiments show very promising results in recommending Web pages relevant to users requests. We also confirmed that this algorithm outperforms a method using mutual information.

The semantic Web and Web services technologies have provided both new possibilities and challenges to automatic information processing. There are a lot of researches on applying these new technologies to current personal Web information retrieval systems, but no research addresses the semantic issues from the whole life cycle and architecture point of view. Web services provide a new way for accessing Web resources, but until now, they have been managed separately from conventional Web contents resources. In this paper, we point out new system requirements and propose a conceptual architecture for a personal semantic Web information retrieval system. It incorporates semantic Web, Web sendees, and multiagent technologies to enable not only precise location of Web resources but also the automatic or semiautomatic integration of hybrid Web contents and Web service resources.

How to help learners plan a navigation path on the Web is an important issue in Web-based learning/education. Our approach to this is to allow the learners to preview a sequence of Web pages as a navigation path plan. In this paper, we introduce an assistant system that enables learners to plan the navigation path in a self-directed way before navigating hyperspace provided by Web-based learning resources. It also realizes an adaptive previewing in accordance with their planning context. Since the adaptive previewing gives the learners in advance an overview of the contents to be learned, their learning in hyperspace can be improved.