Rapid advances in positioning and monitoring technologies have significantly enhanced the ability to track dynamic moving objects for studying movement patterns. As an emerging field in spatiotemporal data mining, a well-established taxonomy of movement patterns facilitates tasks that mine movement patterns. In this study, we proposed a novel classification framework with the 5W1H1R (who, what, when, where, why, how, and relationships) principle and a bottom-up multi-level cognitive model to support the taxonomy of movement patterns. Guided by first principles thinking and combinatorics, we differentiated basic patterns categorized along spatial, temporal, and motion attribute dimensions from compound patterns composed of these basic patterns. We summarized five key constraints to refine movement patterns over recently studied pattern types. We validated our framework in three domains and compared it with four existing frameworks in five aspects. The results demonstrate the broader coverage, extensibility, and adaptability of the framework. Our classification framework can adapt to various moving objects, application domains, and movement data at different scales and resolutions. It can serve as a conceptual and ontological foundation for guiding the mining and analysis of movement patterns and, especially, for developing models to detect multimodal movement patterns.

Trajectory clustering is a fundamental yet challenging data mining task that aims to group similar trajectories. Due to the inherent nature and implicit patterns of trajectories, existing methods often struggle to cluster trajectories with varying densities and noise, and automatically determine cluster numbers. We propose an adaptive two-stage trajectory cluster (ATSTC) algorithm considering the intra-cluster trajectory distance distributions. In the first stage, peak trajectories with the highest local densities, and their adaptively estimated k-nearest neighbors, are initialized as candidate clusters. Trajectories in multiple peak neighborhoods are assigned to clusters with minimal relative distances while remaining trajectories are merged with the nearest cluster or labeled as noise depending on the resultant changes in intra-cluster distance standard deviation. In the second stage, a hierarchical agglomerative strategy is employed to merge clusters by analyzing changes in the average distance and standard deviation of intra-cluster trajectories before and after merging. Experiments on four simulated datasets, with comparisons to eight baselines, demonstrate the superior performance (e.g. ARI) of ATSTC in detecting trajectory clusters under different scenarios. Case studies involving route extraction from ship trajectories and bird tracks with clusters of different sizes, densities, and noise underscore the potential and efficacy of ATSTC in real-world applications.

Abstract Session-based recommendation aims to generate recommendations for the next item of users’ interest based on a given session. In this manuscript, we develop intention enhanced mixed attentive model () to generate session-based recommendations using two important factors: temporal patterns and estimates of users’ intentions. Unlike existing methods which primarily leverage complicated gated recurrent units to model the temporal patterns, models the temporal patterns using a light-weight while effective position-sensitive attention mechanism. In , we also leverage the estimate of users’ prospective preferences to signify important items, and generate better recommendations. Our experimental results demonstrate that models significantly outperform the state-of-the-art methods in six benchmark datasets, with an improvement as much as 19.2%. In addition, our run-time performance comparison demonstrates that during testing, models are much more efficient than the best baseline method, with a significant average speedup of 47.7 folds.

Self-attention (SA) mechanisms have been widely used in developing sequential recommendation (SR) methods, and demonstrated state-of-the-art performance. However, in this article, we show that self-attentive SR methods substantially suffer from the over-smoothing issue that item embeddings within a sequence become increasingly similar across attention blocks. As widely demonstrated in the literature, this issue could lead to a loss of information in individual items, and significantly degrade models’ scalability and performance. To address the over-smoothing issue, in this article, we view items within a sequence constituting a star graph and develop a method, denoted as \(\mathop{\mathtt{MSSG}}\limits\) , for SR. Different from existing self-attentive methods, \(\mathop{\mathtt{MSSG}}\limits\) introduces an additional internal node to specifically capture the global information within the sequence, and does not require information propagation among items. This design fundamentally addresses the over-smoothing issue and enables \(\mathop{\mathtt{MSSG}}\limits\) a linear time complexity with respect to the sequence length. We compare \(\mathop{\mathtt{MSSG}}\limits\) with eleven state-of-the-art baseline methods on six public benchmark datasets. Our experimental results demonstrate that \(\mathop{\mathtt{MSSG}}\limits\) significantly outperforms the baseline methods, with an improvement of as much as 10.10%. Our analysis shows the superior scalability of \(\mathop{\mathtt{MSSG}}\limits\) over the state-of-the-art self-attentive methods. Our complexity analysis and runtime performance comparison together show that \(\mathop{\mathtt{MSSG}}\limits\) is both theoretically and practically more efficient than self-attentive methods. Our analysis of the attention weights learned in SA-based methods indicates that on sparse recommendation data, modeling dependencies in all item pairs using the SA mechanism yields limited information gain, and thus, might not benefit the recommendation performance. Our source code and data are publicly accessible through GitHub .

Sequential recommendation aims to identify and recommend the next few items of users’ interest. It becomes an effective tool to help users select their favorite items from a variety of options. A key challenge in sequential recommendation is to learn the patterns and dynamics, which are most pertinent to inform future interactions of users. With the prosperity of deep learning, many deep models, particularly based on recurrent neural networks [1] and with attention mechanisms [2] , [3] , have been developed for sequential recommendation purposes. However, our analysis demonstrates that, these deep models, particularly those with attention mechanisms, may not always learn meaningful attention weights from the extremely sparse recommendation data, and thus, could degrade the recommendation performance. Therefore, in this study, instead of deep models, we develop novel, effective and efficient hybrid associations models (HAM) to better learn from the sparse and limited recommendation data. This study has been published in IEEE Transactions on Knowledge and Data Engineering. Please refer to the full manuscript [4] for more details.

Next-basket recommendation considers the problem of recommending a set of items into the next basket that users will purchase as a whole. In this paper, we develop a novel mixed model with preferences, popularities and transitions ($\mathop {\mathtt {M^2}}\limits$) for the next-basket recommendation. This method models three important factors in next-basket generation process: 1) users’ general preferences, 2) items’ global popularities and 3) transition patterns among items. Unlike existing recurrent neural network-based approaches,$\mathop {\mathtt {M^2}}\limits$does not use the complicated networks to model the transitions among items, or generate embeddings for users. Instead, it has a simple encoder-decoder based approach ($\mathop {\mathtt {ed\text{-}Trans}}\limits$) to better model the transition patterns among items. We compared$\mathop {\mathtt {M^2}}\limits$with different combinations of the factors with 5 state-of-the-art next-basket recommendation methods on 4 public benchmark datasets in recommending the first, second and third next basket. Our experimental results demonstrate that$\mathop {\mathtt {M^2}}\limits$significantly outperforms the state-of-the-art methods on all the datasets in all the tasks, with an improvement of up to 22.1%. In addition, our ablation study demonstrates that the$\mathop {\mathtt {ed\text{-}Trans}}\limits$is more effective than recurrent neural networks in terms of the recommendation performance. We also have a thorough discussion on various experimental protocols and evaluation metrics for next-basket recommendation evaluation.

Sequential recommendation aims to identify and recommend the next few items for a user that the user is most likely to purchase/review, given the user's purchase/rating trajectories. It becomes an effective tool to help users select favorite items from a variety of options. In this manuscript, we developed hybrid associations models (HAM) to generate sequential recommendations. using three factors: 1) users' long-term preferences, 2) sequential, high-order and low-order association patterns in the users' most recent purchases/ratings, and 3) synergies among those items. HAM uses simplistic pooling to represent a set of items in the associations, and element-wise product to represent item synergies of arbitrary orders. We compared HAM models with the most recent, state-of-the-art methods on six public benchmark datasets in three different experimental settings. Our experimental results demonstrate that HAM models significantly outperform the state of the art in all the experimental settings. with an improvement as much as 46.6%. In addition, our run-time performance comparison in testing demonstrates that HAM models are much more efficient than the state-of-the-art methods. and are able to achieve significant speedup as much as 139.7 folds.

Over the past decade low graduation and retention rates has plagued higher education institutions. To assist students in choosing a sequence of courses, choosing majors and successful academic pathways; many institutions provide several on-site academic advising services supported by data driven educational technologies. Accurate performance prediction can serve as the backbone for degree planning software, personalized advising systems and early warning systems that can identify students at-risk of dropping from their field of study. In this work, we present a deep learning based recommender system approach called Neural Collaborative Filtering (NCF) for predicting the grade a student will earn in a course that he/she plans to take in the next-term. Prior grade prediction methods are based on matrix factorization (MF) where students and courses are represented in a latent "knowledge" space. The deep learning inspired approach provides added flexibility in learning the latent spaces in comparison to MF approaches. The proposed approach also incorporates instructor information besides student and course information. Moreover, for proper analysis of the learned model parameters, we assume the embeddings obtained for students, courses and instructors should be non-negative. This non-negative NCF model referred by NCFnn model adds a rectified linear units (ReLU) on the embedding layer of NCF. The experimental results on datasets from George Mason University, a large, public university in the United States, demonstrate that the proposed NCF approaches significantly outperform competitive baselines across different test sets.

The past decade has seen a growth in the development and deployment of educational technologies for assisting college-going students in choosing majors, selecting courses and acquiring feedback based on past academic performance. Grade prediction methods seek to estimate a grade that a student may achieve in a course that she may take in the future (e.g., next term). Accurate and timely prediction of students' academic grades is important for developing effective degree planners and early warning systems, and ultimately improving educational outcomes. Existing grade prediction methods mostly focus on modeling the knowledge components associated with each course and student, and often overlook other factors such as the difficulty of each knowledge component, course instructors, student interest, capabilities and effort. In this paper, we propose additive latent effect models that incorporate these factors to predict the student next-term grades. Specifically, the proposed models take into account four factors: (i) student's academic level, (ii) course instructors, (iii) student global latent factor, and (iv) latent knowledge factors. We compared the new models with several state-of-the-art methods on students of various characteristics (e.g., whether a student transferred in or not). The experimental results demonstrate that the proposed methods significantly outperform the baselines on grade prediction problem. Moreover, we perform a thorough analysis on the importance of different factors and how these factors can practically assist students in course selection, and finally improve their academic performance.

Sparse models have demonstrated substantial success in applications for data analysis such as clustering, classification and denoising. However, most of the current work is built upon the assumption that data is distributed in a union of subspaces, whereas limited work has been conducted on nonlinear datasets where data reside in a union of manifolds rather than a union of subspaces. To understand data nonlinearity using sparse models, in this paper, we propose to exploit the self-representation property of nonlinear data in an implicit feature space using kernel methods. We propose a kernelized sparse self-representation model, denoted as KSSR, and a novel Kernelized Fast Iterative Soft-Thresholding Algorithm, denoted as K-FISTA, to recover the underlying nonlinear structure among the data. We evaluate our method for clustering problems on both synthetic and real-world datasets, and demonstrate its superior performance compared to the other state-of-the-art methods. We also apply our method for collaborative filtering in recommender systems, and demonstrate its great potential for novel applications beyond clustering.

The effectiveness of existing top-N recommendation methods decreases as the sparsity of the datasets increases. To alleviate this problem, we present an item-based method for generating top-N recommendations that learns the item-item similarity matrix as the product of two low dimensional latent factor matrices. These matrices are learned using a structural equation modeling approach, wherein the value being estimated is not used for its own estimation. A comprehensive set of experiments on multiple datasets at three different sparsity levels indicate that the proposed methods can handle sparse datasets effectively and outperforms other state-of-the-art top-N recommendation methods. The experimental results also show that the relative performance gains compared to competing methods increase as the data gets sparser.

The increasing amount of side information associated with the items in E-commerce applications has provided a very rich source of information that, once properly exploited and incorporated, can significantly improve the performance of the conventional recommender systems. This paper focuses on developing effective algorithms that utilize item side information for top-N recommender systems. A set of sparse linear methods with side information (SSLIM) is proposed, which involve a regularized optimization process to learn a sparse aggregation coefficient matrix based on both user-item purchase profiles and item side information. This aggregation coefficient matrix is used within an item-based recommendation framework to generate recommendations for the users. Our experimental results demonstrate that SSLIM outperforms other methods in effectively utilizing side information and achieving performance improvement.

This paper focuses on developing effective and efficient algorithms for top-N recommender systems. A novel Sparse Linear Method (SLIM) is proposed, which generates top-N recommendations by aggregating from user purchase/rating profiles. A sparse aggregation coefficient matrix W is learned from SLIM by solving an ℓ1-norm and ℓ2-norm regularized optimization problem. W is demonstrated to produce high quality recommendations and its sparsity allows SLIM to generate recommendations very fast. A comprehensive set of experiments is conducted by comparing the SLIM method and other state-of-the-art top-N recommendation methods. The experiments show that SLIM achieves significant improvements both in run time performance and recommendation quality over the best existing methods.

Extracting semantic relations between entities is an important step towards automatic text understanding. In this paper, we propose a novel Semi-supervised Convolution Graph Kernel (SCGK) method for semantic Relation Extraction (RE) from natural language. By encoding English sentences as dependence graphs among words, SCGK computes kernels (similarities) between sentences using a convolution strategy, i.e., calculating similarities over all possible short single paths from two dependence graphs. Furthermore, SCGK adds three semi-supervised strategies in the kernel calculation to incorporate soft-matches between (1) words, (2) grammatical dependencies, and (3) entire sentences, respectively. From a large unannotated corpus, these semi-supervision steps learn to capture contextual semantic patterns of elements in natural sentences, which therefore alleviate the lack of annotated examples in most RE corpora. Through convolutions and multi-level semi-supervisions, SCGK provides a powerful model to encode both syntactic and semantic evidence existing in natural English sentences, which effectively recovers the target relational patterns of interest. We perform extensive experiments on five RE benchmark datasets which aim to identify interaction relations from biomedical literature. Our results demonstrate that SCGK achieves the state-of-the-art performance on the task of semantic relation extraction.

This paper focuses on developing classification algorithms for problems in which there is a need to predict the class based on multiple observations (examples) of the same phenomenon (class). These problems give rise to a new classification problem, referred to as set classification, that requires the prediction of a set of instances given the prior knowledge that all the instances of the set belong to the same unknown class. This problem falls under the general class of problems whose instances have class label dependencies. Four methods for solving the set classification problem are developed and studied. The first is based on a straightforward extension of the traditional classification paradigm whereas the other three are designed to explicitly take into account the known dependencies among the instances of the unlabeled set during learning or classification. A comprehensive experimental evaluation of the various methods and their underlying parameters shows that some of them lead to significant gains in performance.