Recent advances have shown that sequential fine-tuning (SeqFT) of pre-trained vision transformers (ViTs), followed by classifier refinement using approximate distributions of class features, can be an effective strategy for class-incremental learning (CIL). However, this approach is susceptible to distribution drift, caused by the sequential optimization of shared backbone parameters. This results in a mismatch between the distributions of the previously learned classes and that of the updated model, ultimately degrading the effectiveness of classifier performance over time. To address this issue, we introduce a latent space transition operator and propose Sequential Learning with Drift Compensation (SLDC). SLDC aims to align feature distributions across tasks to mitigate the impact of drift. First, we present a linear variant of SLDC, which learns a linear operator by solving a regularized least-squares problem that maps features before and after fine-tuning. Next, we extend this with a weakly nonlinear SLDC variant, which assumes that the ideal transition operator lies between purely linear and fully nonlinear transformations. This is implemented using learnable, weakly nonlinear mappings that balance flexibility and generalization. To further reduce representation drift, we apply knowledge distillation (KD) in both algorithmic variants. Extensive experiments on standard CIL benchmarks demonstrate that SLDC significantly improves the performance of SeqFT. Notably, by combining KD to address representation drift with SLDC to compensate distribution drift, SeqFT achieves performance comparable to joint training across all evaluated datasets.

Single-Source Personalized PageRank (SSPPR) is a fundamental problem in social network analytics, yet maintaining accurate SSPPR query results in evolving social networks poses significant challenges, especially for real-time applications. Existing approaches often overlook the role of subgraphs and struggle with frequent graph updates, resulting in inefficiency regarding dynamic scenarios. In this study, we define a novel personalized PageRank query, n-steps SSPPR, designed to address the challenges of dynamic environments. To support this query, we propose a baseline solution, Pn-FORA, as a foundational approach. While effective, Pn-FORA is inefficient due to its computationally expensive information update scheme. To overcome these limitations, we propose a multithreaded framework for processing massive-scale n-steps SSPPR queries in real-time over evolving graphs. Central to our framework is the Global Walk Synchronization (GWS) method, ensuring the accuracy of SSPPR scores by synchronizing walk information across nodes as the graph evolves. To further enhance GWS, we introduce an influence-aware graph representation to optimize update propagation. Furthermore, we develop a dynamic workload balancing strategy and precision-aware concurrency controls, which achieve an effective balance between efficiency and accuracy. Extensive experiments on real-world datasets demonstrate that our approach significantly outperforms existing methods, offering superior scalability and efficiency for real-time n-steps SSPPR query processing over large-scale social networks. The source code of our implementation is publicly available at https://github.com/SujunShuai/Work2023.

Next POInt-of-Interest (POI) recommendation predicts a user's next move and facilitates location-based services such as navigation and travel planning. SOTA methods fuse each POI and its contexts (e.g., time, category, and region) into a single representation to model sequential user movement. This hinders the effective utilization of context information, and diverse user preferences are also neglected. To tackle these limitations, we propose Disentangled and Personalized Representation Learning (DPRL) as a novel method for next POI recommendation. DPRL decouples POIs and contexts during representation learning, capturing their sequential regularities independently using separate recurrent neural networks (RNNs). To model the preference of each user, DPRL adopts an aggregation mechanism that integrates dynamic user preferences and spatial-temporal factors into the learned representations. We compare DPRL with 16 state-of-the-art baselines. The results show that DPRL outperforms all baselines and achieves an average accuracy improvement of 10.53% over the best-performing baseline.

Road trajectory generation creates synthetic yet realistic trajectories to tackle data collection costs and privacy concerns. Existing methods generate a trajectory either segment-by-segment using sequence models or holistically in one step using diffusion models. Sequence-based models have good regularity and consistency (i.e., resemble the input trajectories) but lack diversity, while diffusion-based models enhance diversity but sacrifice regularity and consistency. To combine the merits of existing methods, we propose Seed, by bridging sequence and diffusion models for trajectory generation. In particular, Seed adopts a conditional diffusion structure, where a Transformer models the movement of each trajectory along the road segments, and conditioned on the Transformer's output, a diffusion model recovers the next road segment from random noise. The rationale is that the Transformer captures sequential patterns for regularity and consistency, while the diffusion model introduces diversity by recovering from noise. We use a trajectory reconstruction task to train Seed, and design a curriculum learning strategy to accelerate convergence. We compare Seed with 8 state-of-the-art trajectory generation methods on 3 datasets, and the results show that Seed improves the best-performing baseline by over 50%.

Next point-of-interest (POI) recommendationpredicts user’s next movement and facilitates location-based applications such as destination suggestion and travel planning. State-of-the-art (SOTA) methods learn an adaptive graph from user trajectories and compute POI representations using graph neural networks (GNNs). However, a single graph cannot capture thediverse dependenciesamong the POIs (e.g., geographical proximity and transition frequency). To tackle this limitation, we propose theAdaptiveGraphContrastiveLearning(AGCL) framework. AGCL constructs multiple adaptive graphs, each modeling a kind of POI dependency and producing one POI representation; and the POI representations from different graphs are merged into amulti-facet representationthat encodes comprehensive information. To train the POI representations, we tailor agraph-based contrastive learning, which encourages the representations of similar POIs to align and dissimilar POIs to differentiate. Moreover, to learn the sequential regularities of user trajectories, we design an attention mechanism to integrate spatial-temporal information into the POI representations. An explicitspatial-temporal biasis also employed to adjust the predictions for enhanced accuracy. We compare AGCL with 10 state-of-the-art baselines on 3 datasets. The results show that AGCL outperforms all baselines and achieves an improvement of 10.14% over the best performing baseline in average accuracy.

Strong priors are imposed on the search space of differentiable architecture search (DARTS), such that cells of the same type share the same topological structure and each intermediate node retains two operators from distinct nodes. While these priors reduce optimization difficulties and improve the applicability of searched architectures, they hinder the subsequent development of automated machine learning (auto-ML) and prevent the optimization algorithm from exploring more powerful neural networks through improved architectural flexibility. This article aims to reduce these prior constraints by eliminating restrictions on cell topology and modifying the discretization mechanism for super-networks. Specifically, the flexible DARTS (FX-DARTS) method, which leverages an entropy-based super-network shrinking (ESS) framework, is presented to address the challenges arising from the elimination of prior constraints. Notably, FX-DARTS enables the derivation of neural architectures without strict prior rules while maintaining the stability in the enlarged search space. Experimental results on image classification benchmarks demonstrate that FX-DARTS is capable of exploring a set of neural architectures with competitive trade-offs between performance and computational complexity within a single search procedure.

Neural architecture search (NAS) has emerged as a transformative approach for automating the design of neural networks, demonstrating exceptional performance across a variety of tasks. Numerous NAS methods aim to optimize neural architectures within discrete or continuous search spaces, but each method possesses its own inherent limitations. Additionally, the search efficiency is notably impeded by suboptimal encoding methods, presenting an ongoing challenge. In response to these obstacles, this paper introduces a novel approach, evolutionary neural architecture optimization (ENAO), which optimizes architectures in an approximate continuous search space. ENAO begins with training a deep generative model to embed discrete architectures into a condensed latent space, leveraging unsupervised representation learning. Subsequently, evolutionary algorithm is employed to refine neural architectures within this approximate continuous latent space. Empirical comparisons against several NAS benchmarks underscore the effectiveness of the ENAO method. Thanks to its foundation in deep unsupervised representation learning, ENAO demonstrates a distinguished ability to identify high-quality architectures with fewer evaluations and achieve state-of-the-art result in NAS-Bench-201 dataset. Overall, the ENAO method is a promising approach for optimizing neural network architectures in an approximate continuous search space with evolutionary algorithms and may be a useful tool for researchers and practitioners in the field of NAS.

Bayesian inference has provided the continual learning (CL) with an elegant framework where past experiences and new knowledge are consolidated into the posterior constantly. Typical approaches rely on Bayesian neural networks whose parameters are updated by variational inference, namely, maximizing the evidence lower bound of log-likelihood. In this paper, we discuss the effects of local reparameterization on the optimization of such networks in the context of CL. The empirical results show that it does not only increase the inference speed of neural networks, but also enhance the CL performance in some scenarios. Additionally, motivated by the observation that variance matrices have low-rank structures, we propose the d-tied variational continual learning (d-tied-VCL) to improve the parameter efficiency of variational continual learning (VCL). Experiments on random classification, per-muted MNIST, and split CIFAR100 show that even VCL with rank-1 variance matrices achieves competitive performance.

Traffic flow forecasting plays a vital role in the transportation domain. Existing studies usually manually construct correlation graphs and design sophisticated models for learning spatial and temporal features to predict future traffic states. However, manually constructed correlation graphs cannot accurately extract the complex patterns hidden in the traffic data. In addition, it is challenging for the prediction model to fit traffic data due to its irregularly-shaped distribution. To solve the above-mentioned problems, in this paper, we propose a novel learning-based method to learn a spatial-temporal correlation graph, which could make good use of the traffic flow data. Moreover, we propose First-Order Gradient Supervision (FOGS), a novel method for traffic flow forecasting. FOGS utilizes first-order gradients, rather than specific flows, to train prediction model, which effectively avoids the problem of fitting irregularly-shaped distributions. Comprehensive numerical evaluations on four real-world datasets reveal that the proposed methods achieve state-of-the-art performance and significantly outperform the benchmarks.

Next Point-of Interest (POI) recommendation plays an important role in location-based applications, which aims to recommend the next POIs to users that they are most likely to visit based on their historical trajectories. Existing methods usually use rich side information, or customized POI graphs to capture the sequential patterns among POIs. However, the graphs only focus on connectivity between POIs. Few studies propose to explicitly learn a weighted POI graph, which could reflect the transition patterns among POIs and show the importance of its different neighbors for each POI. In addition, these approaches simply utilize the user characteristics for personalized POI recommendation without sufficient consideration. To this end, we construct a novel User-POI Knowledge Graph with strong representation ability, called Spatial-Temporal Knowledge Graph (STKG). STKG is used to learn the representations of each node (i.e., user, POI) and each edge. Then, we design a similarity function to construct our POI transition graph based on the learned representations. To incorporate the learned graph into sequential model, we propose a novel network Graph-Flashback for recommendation. Graph-Flashback applies a simplified Graph Convolution Network (GCN) on the POI transition graph to enrich the representation of each POI. Further, we define a similarity function to consider both spatiotemporal information and user preference in modelling sequential regularity. Experimental results on two real-world datasets show that our proposed method achieves the state-of-the-art performance and significantly outperforms all existing solutions.

In the aforementioned letter, the authors indicate that with a$2M + 1$sensor uniform linear array (ULA), up to$2M - 1$sources can actually be localized by the proposed algorithm. In this comment, however, we prove that the algorithm will no longer be valid if the number of sources exceeds$M$. A numerical simulation is performed to verify this conclusion.