Recent advances in generative artificial intelligence, particularly large language models (LLMs), have opened new opportunities for enhancing recommender systems (RecSys). Most existing LLM-based RecSys approaches operate in a discrete space, using vector-quantized tokenizers to align with the inherent discrete nature of language models. However, these quantization methods often result in lossy tokenization and suboptimal learning, primarily due to inaccurate gradient propagation caused by the non-differentiable argmin operation in standard vector quantization. Inspired by the emerging trend of embracing continuous tokens in language models, we propose ContRec, a novel framework that seamlessly integrates continuous tokens into LLM-based RecSys. Specifically, ContRec consists of two key modules: a σ-VAE Tokenizer, which encodes users/items with continuous tokens; and a Dispersive Diffusion module, which captures implicit user preference. The tokenizer is trained with a continuous Variational Auto-Encoder (VAE) objective, where three effective techniques are adopted to avoid representation collapse. By conditioning on the previously generated tokens of the LLM backbone during user modeling, the Dispersive Diffusion module performs a conditional diffusion process with a novel Dispersive Loss, enabling high-quality user preference generation through next-token diffusion. Finally, ContRec leverages both the textual reasoning output from the LLM and the latent representations produced by the diffusion model for Top-K item retrieval, thereby delivering comprehensive recommendation results. Extensive experiments on four datasets demonstrate that ContRec consistently outperforms both traditional and state-of-the-art LLM-based recommender systems. Our results highlight the potential of continuous tokenization and generative modeling for advancing the next generation of recommender systems.

Sequential recommendation methods are crucial in modern recommender systems for their remarkable capability to understand a user’s changing interests based on past interactions. However, a significant challenge faced by current methods (e.g., RNN- or Transformer-based models) is to effectively and efficiently capture users’ preferences by modeling long behavior sequences, which impedes their various applications like short video platforms where user interactions are numerous. Recently, an emerging architecture named Mamba , built on state space models (SSM) with efficient hardware-aware designs, has showcased the tremendous potential for sequence modeling, presenting a compelling avenue for addressing the challenge effectively. Inspired by this, we propose a novel generic and efficient framework ( SSD4Rec ) for sequential recommendations, which explores the seamless adaptation of Mamba for recommendations. Specifically, SSD4Rec marks the long-length item sequences with sequence registers and processes the item representations with a novel Masked Bidirectional Structured State Space Duality block. This not only allows for hardware-aware matrix multiplication but also empowers outstanding capabilities in variable-length and long-range sequence modeling. Extensive evaluations on four benchmark datasets demonstrate that the proposed model achieves state-of-the-art performance while maintaining near-linear scalability with user sequence length. Our implementation based on PyTorch is available at https://github.com/ZhangYifeng1995/SSD4Rec .

Benchmarking the capabilities and limitations of large language models (LLMs) in graph-related tasks is becoming an increasingly popular and crucial area of research. Recent studies have shown that LLMs exhibit a preliminary ability to understand graph structures and node features. However, the potential of LLMs in graph pattern mining remains largely unexplored. This is a key component in fields such as computational chemistry, biology, and social network analysis. To bridge this gap, this work introduces a comprehensive benchmark to assess LLMs' capabilities in graph pattern tasks. We have developed a benchmark that evaluates whether LLMs can understand graph patterns based on either terminological or topological descriptions. Additionally, our benchmark tests the LLMs' capacity to autonomously discover graph patterns from data. The benchmark encompasses both synthetic and real datasets, and a variety of models, with a total of 11 tasks and 7 models. Our experimental framework is designed for easy expansion to accommodate new models and datasets. Our findings reveal that: (1) LLMs have preliminary abilities to understand graph patterns, with O1-mini outperforming in the majority of tasks; (2) Formatting input graph data to align with the knowledge acquired during pretraining can enhance performance; (3) LLMs employ diverse potential algorithms to solve one task, with performance varying based on their execution capabilities.

With the advancement of web techniques, they have significantly revolutionized various aspects of people's lives. Despite the importance of the web, many tasks performed on it are repetitive and time-consuming, negatively impacting the overall quality of life. To efficiently handle these tedious daily tasks, one of the most promising approaches is to advance autonomous agents to incorporate human-like intelligence based on Artificial Intelligence (AI) techniques, referred to as AI Agents. AI Agents offer significant advantages in handling such tasks since they can operate continuously without fatigue or performance degradation. Therefore, leveraging AI Agents - termed WebAgents in the context of web - to automatically assist people in handling tedious daily tasks can dramatically enhance productivity and efficiency. Recently, Large Foundation Models (LFMs) containing billions of parameters have exhibited human-like language understanding and reasoning capabilities, showing proficiency in performing various complex tasks. This naturally raises the question: 'Can LFMs be utilized to develop powerful AI Agents that automatically handle web tasks, providing significant convenience to users?' To fully explore the potential of LFMs, extensive research has emerged on WebAgents designed to complete daily web tasks according to user instructions, significantly enhancing the convenience of daily human life. In this survey, we comprehensively review existing research studies on WebAgents across three key aspects: architectures, training, and trustworthiness. Additionally, several promising directions for future research are explored to provide deeper insights.

Electric vehicles (EVs) are driving green and low-carbon transport in modern cities. It makes charging station occupancy prediction (CSOP) critual for intelligent transportation systems (ITS) to achieve a balance between the supply and demand in resolving the dynamics between EVs and changing stations. Even though several Big Data-based solutions have been discussed, they are still struggling to collaboratively utilize heterogeneous data and distributed computing resources located at both physically and logicially isolated charging stations to better support context-driven CSOP. To addres this challenge, we propose an Asynchronous Federated Meta-learning Mechanism (AFML) for CSOP, which can train a meta-model with strong adaptation ability in an asynchronous and collaborative manner. In general, it incorporates an adaptive reptile algorithm (AR) and an weighted aggregation strategy (WA) to jointly ensure the training efficiency and model adaptivity. Evaluations on real-world CSOP datasets demonstrate that compared to the second best method, AFML can significantly improve forecasting accuracy by 14%, accelerate model convergence by 9% and enhance model generalizability by 10%, illustrating its merits in support CSOP to embrace a smart and sustainable city.

There is a growing interest in utilizing large language models (LLMs) to advance next-generation Recommender Systems (RecSys), driven by their outstanding language understanding and reasoning capabilities. In this scenario, tokenizing users and items becomes essential for ensuring seamless alignment of LLMs with recommendations. While studies have made progress in representing users and items using textual contents or latent representations, challenges remain in capturing high-order collaborative knowledge into discrete tokens compatible with LLMs and generalizing to unseen users/items. To address these challenges, we propose a novel framework called TokenRec, which introduces an effective ID tokenization strategy and an efficient retrieval paradigm for LLM-based recommendations. Our tokenization strategy involves quantizing the masked user/item representations learned from collaborative filtering into discrete tokens, thus achieving smooth incorporation of high-order collaborative knowledge and generalizable tokenization of users and items for LLM-based RecSys. Meanwhile, our generative retrieval paradigm is designed to efficiently recommend top-K items for users, eliminating the need for the time-consuming auto-regressive decoding and beam search processes used by LLMs, thus significantly reducing inference time. Comprehensive experiments validate the effectiveness of the proposed methods, demonstrating that TokenRec outperforms competitive benchmarks, including both traditional recommender systems and emerging LLM-based recommender systems. Codes and data are available athttps://github.com/Quhaoh233/TokenRec.

Recent booming successes of electric vehicles (EVs) motivate emerging exploration of spatio-temporal EV charging demand forecasting to inform policy making. Recent studies have contributed to remarkable accuracy improvement by developing deep learning methods. However, when they access massive amounts of data and frequently exchange data through the Internet of Things (IoT), data silos and inefficient training emerge as main challenges. To tackle these challenges, this study proposes an integrated approach for regional EV charging demand forecasting, named FMGCN, which consists of two modules, namely 1) Spatio-temporal Learning module, which introduces spatial and temporal attentions to capture the underlying charging patterns between different regions and cities effectively; and 2) Distributed Pretraining module, which incorporates Federated Learning and Meta-Learning to enhance the adaptivity and generalisability of the forecasting model. A comprehensive evaluation based on a real-world dataset of 25,246 public EV charging piles shows that the proposed model outperforms other representative models with 1) an average improvement of 29.9% in forecasting errors; 2) an acceleration of 65% in convergence speed; and 3) a sound adaptability to support varying charging demand.

Along with the proliferation of electric vehicles (EVs), optimizing the use of EV charging space can significantly alleviate the growing load on intelligent transportation systems. As the foundation to achieve such an optimization, a spatiotemporal method for EV charging demand prediction in urban areas is required. Although several solutions have been proposed by using data-driven deep learning methods, it can be that these performance-oriented approaches may struggle to correctly understand the underlying factors influencing charging demand, particularly charging prices. A representative case that highlights the challenge faced by existing methods is their potential misinterpretation of high prices during peak times, leading to an incorrect assumption that higher prices correspond to increased demand. To address the challenges associated with training an accurate and reliable prediction model for EV charging demand, this paper proposes a novel approach called PAG, which leverages the integration of graph and temporal attention mechanisms for effective feature extraction and introduces physics-informed meta-learning in the pre-training step to facilitate prior knowledge learning. Evaluation results on a dataset of 18,061 EV charging piles in Shenzhen, China, show that the proposed approach can achieve state-of-the-art forecasting performance and the ability to understand the adaptive changes in charging demands caused by price fluctuations.

In a city, the usage optimization of parking spaces with a near real-time response to car drivers can significantly reduce the unnecessary cruising for parking and the additional congestion of regional traffic. As the foundation to achieve such an optimization, a parking occupancy prediction method is required to address the emerging challenges of training a simple but effective model. To fill the gap, this paper proposes a novel approach that enables the integration of Time Series Decomposition (TSD), Gated Recurrent Unit (GRU), and First-order Model-agnostic Meta-learning (FOMAML) for feature engineering, model building, and model pre-training, respectively. Moreover, as shown by a detailed evaluation, such an integration strengthens the proposed approach, named Meta TSD-GRU, which outperforms other state-of-the-art methods with 1) prediction errors reduced by about 45% on average, 2) the speed of model adaptation and convergence improved about 2 and 102 times against the methods with and without pre-training, respectively, and 3) the generalizability of the model enhanced to handle various time intervals of forecasting and types of parking lots under a consistent and stable performance.

Federated learning (FL) is a novel decentralized machine learning mechanism, which bridges data silos to train a global model by utilizing data and computation power of local clients in a privacy-preserving way. Moreover, to handle heterogeneous data across domains, tasks and partities, federated meta-learning (FML) has been proposed, which leverages the fast adaptation of meta-learning for transferable and customizable models. Nevertheless, most of the existing studies focus on providing personalized models for different users, leaving other important issues not well solved, especially, incentive mechanisms, which are used as a common foundation for FML to attrack and maintain high-quality and reputable clients. To fill the gap, this paper proposes a learning-based incentive mechanism for FML to motivate local clients to participate in the data federation. First, we propose to reward clients according to the amount of data they contribute to the model training. Then, to analyze the behaviors of model owner and local clients, we formulate the incentivized training task as a Stackelberg game, and design a method based on reinforcement learning (RL) to learn optimal pricing and participating strategies for the task publisher and the local clients, respectively. Lastly, extensive experiments are conducted to demonstrate the efficiency and effectiveness of the proposed RL-based incentive mechanism, which assists multiple parties to reach the equilibrium of the game designed for FML.