The rapid growth in demand for large language models (LLMs) has strained cloud-edge infrastructure. While edges offer low latency and clouds provide vast resources, scheduling LLM requests efficiently remains a major challenge due to their unpredictable processing times, which leads to Headof-Line (HOL) blocking that degrades system throughput and responsiveness. To address this, we introduce the Online CloudEdge Collaborative Request Scheduling (OCE-CRS) framework. OCE-CRS models the proactive scheduling of LLM requests as a contextual combinatorial bandit problem. At its core is our novel Combinatorial Neural Delayed Upper Confidence Bound (CN DUCB) algorithm, which learns to predict request processing times from the semantic content of the request prompt alone. This enables an inspired policy based on Shortest Job First (SJF) that prioritizes shorter jobs for edge execution, simultaneously maximizing throughput and mitigating HOL blocking. To prevent time-consuming neural network training from blocking scheduling decisions, we employ an asynchronous mechanism. This decouples model updates from the real-time scheduling loop, effectively handling the resultant delayed feedback where observations from past rounds are used in later training steps. We provide a theoretical sublinear regret bound for our algorithm. Extensive experiments validate that OCE-CRS significantly improves throughput, Job Completion Time (JCT), and queueing delay, demonstrating superior performance and robustness in both static and continuous batching environments.

The multi-armed bandit (MAB) models have attracted significant research attention due to their applicability and effectiveness in various real-world scenarios such as resource allocation in uncertain environments, online advertising, and dynamic pricing. As an important branch, the adversarial multi-armed bandit problems with delayed feedback have been proposed and studied by many researchers recently where a conceptual adversary strategically selects the reward distributions associated with each arm to challenge the learning algorithm and the agent experiences a bunch of delays in receiving the corresponding reward feedback from different users after taking an action on them. However, the existing models restrict the feedback to being generated from only one user, which makes models inapplicable to the prevailing scenarios of multiple users (e.g. ad recommendation for a group of users). In this paper, we consider that the delayed feedback results are from multiple users and are unrestricted on internal distribution while the feedback delay is arbitrary and unknown to the player in advance. Also, for different users in a round, the delays in feedback have no assumption of latent correlation. Thus, we formulate an adversarial multi-armed bandit problem with multi-user delayed feedback and design a modified EXP3 algorithm named MUD-EXP3, which makes a decision at each round by considering the importance-weighted estimator of the received feedback from different users. On the premise of known terminal round index$T$, the number of users$M$, the number of arms$N$, and upper bound of delay$d_{max}$, we prove a regret of$\mathcal {O}(\sqrt{TM^{2}\ln {N}(N\mathrm{e}+4d_{max})})$. Furthermore, for the more common case of unknown$T$, an adaptive algorithm named AMUD-EXP3 is proposed with a sublinear regret concerning$T$. Finally, extensive experiments are conducted to indicate the correctness and effectiveness of our algorithms in dynamic environments.

Influence Maximization (IM) is a classical combinatorial optimization problem, which can be widely used in mobile networks, social computing, and recommendation systems. It aims at selecting a small number of users such that maximizing the influence spread across the online social network. Because of its potential commercial and academic value, there are a lot of researchers focusing on studying the IM problem from different perspectives. The main challenge comes from the NP-hardness of the IM problem and #P-hardness of estimating the influence spread, thus traditional algorithms for overcoming them can be categorized into two classes: heuristic algorithms and approximation algorithms. However, there is no theoretical guarantee for heuristic algorithms, and the theoretical design is close to the limit. Therefore, it is almost impossible to further optimize and improve their performance. With the rapid development of artificial intelligence, technologies based on Machine Learning (ML) have achieved remarkable achievements in many fields. In view of this, in recent years, a number of new methods have emerged to solve combinatorial optimization problems by using ML-based techniques. These methods have the advantages of fast solving speed and strong generalization ability to unknown graphs, which provide a brand-new direction for solving combinatorial optimization problems. Therefore, we abandon the traditional algorithms based on iterative search and review the recent development of ML-based methods, especially Deep Reinforcement Learning, to solve the IM problem and other variants in social networks. We focus on summarizing the relevant background knowledge, basic principles, common methods, and applied research. Finally, the challenges that need to be solved urgently in future IM research are pointed out.

To walk around the city, individuals use mobile mapping services, and such services mostly suggest shortest routes. To go beyond recommending such walkable routes, we propose a new framework for automatic wayfinding for pedestrians. This framework tackles two main drawbacks from which past work suffers, namely coarse-grained representation of space and absence of contextual dynamics. We model the human tendency to regularize space by borrowing a spatial representation, Space Syntax, from the discipline of Architecture. Moreover, the proposed framework accounts for contextual dynamics of individual streets by predicting the popularity of each street under different contexts (e.g., at a given time, with a certain weather condition). Using Foursquare check-ins (i.e., whereabouts of the users of the popular location-based service) and publicly available weather data, we validate our framework in the entire city of Barcelona. We find that, with paths slightly longer than the shortest ones, our framework is able to accommodate our mental topography and effectively capture contextual changes.