Travel-time prediction constitutes a task of high importance in transportation networks, with web mapping services like Google Maps regularly serving vast quantities of travel time queries from users and enterprises alike. Further, such a task requires accounting for complex spatiotemporal interactions (modelling both the topological properties of the road network and anticipating events---such as rush hours---that may occur in the future). Hence, it is an ideal target for graph representation learning at scale. Here we present a graph neural network estimator for estimated time of arrival (ETA) which we have deployed in production at Google Maps. While our main architecture consists of standard GNN building blocks, we further detail the usage of training schedule methods such as MetaGradients in order to make our model robust and production-ready. We also provide prescriptive studies: ablating on various architectural decisions and training regimes, and qualitative analyses on real-world situations where our model provides a competitive edge. Our GNN proved powerful when deployed, significantly reducing negative ETA outcomes in several regions compared to the previous production baseline (40+% in cities like Sydney).

With the rapid development of Internet services, mobile communications, and IoT applications, Location-Based Service (LBS) has become an indispensable part in our daily life in recent years. However, when users benefit from LBSs, the collection and analysis of users’ location data and trajectory information may jeopardize their privacy. To address this problem, a new privacy-preserving method based on historical proximity locations is proposed. The main idea of this approach is to substitute one existing historical adjacent location around the user for his/her current location and then submit the selected location to the LBS server. This method ensures that the user can obtain location-based services without submitting the real location information to the untrusted LBS server, which can improve the privacy-preserving level while reducing the calculation and communication overhead on the server side. Furthermore, our scheme can not only provide privacy preservation in snapshot queries but also protect trajectory privacy in continuous LBSs. Compared with other location privacy-preserving methods such as k -anonymity and dummy location, our scheme improves the quality of LBS and query efficiency while keeping a satisfactory privacy level.

Cellular network configuration plays a critical role n network performance. In current practice, network configuration depends heavily on field experience of engineers and often remains static for a long period of time. This practice is far from optimal. To address this limitation, online-learning-based approaches have great potentials to automate and optimize network configuration. Learning-based approaches face the challenges of learning a highly complex function for each base station and balancing the fundamental exploration-exploitation tradeoff while minimizing the exploration cost. Fortunately, in cellular networks, base stations (BSs) often have similarities even though they are not identical. To leverage such similarities, we propose kernel-based multi-BS contextual bandit algorithm based on multi-task learning. In the algorithm, we leverage the similarity among different BSs defined by conditional kernel embedding. We present theoretical analysis of the proposed algorithm in terms of regret and multi-task-learning efficiency. We evaluate the effectiveness of our algorithm based on a simulator built by real traces.

In this paper, we propose and study opportunistic contextual bandits - a special case of contextual bandits where the exploration cost varies under different environmental conditions, such as network load or return variation in recommendations. When the exploration cost is low, so is the actual regret of pulling a sub-optimal arm (e.g., trying a suboptimal recommendation). Therefore, intuitively, we could explore more when the exploration cost is relatively low and exploit more when the exploration cost is relatively high. Inspired by this intuition, for opportunistic contextual bandits with Linear payoffs, we propose an Adaptive Upper-Confidence-Bound algorithm (AdaLinUCB) to adaptively balance the exploration-exploitation trade-off for opportunistic learning. We prove that AdaLinUCB achieves O((log T)^2) problem-dependent regret upper bound, which has a smaller coefficient than that of the traditional LinUCB algorithm. Moreover, based on both synthetic and real-world dataset, we show that AdaLinUCB significantly outperforms other contextual bandit algorithms, under large exploration cost fluctuations.

Cellular network performance depends heavily on the configuration of its network parameters. Current practice of parameter configuration relies largely on expert experience, which is often suboptimal, time-consuming, and error-prone. Therefore, it is desirable to automate this process to improve the accuracy and efficiency via learning-based approaches. However, such approaches need to address several challenges in real operational networks: the lack of diverse historical data, a limited amount of experiment budget set by network operators, and highly complex and unknown network performance functions. To address those challenges, we propose a collaborative learning approach to leverage data from different cells to boost the learning efficiency and to improve network performance. Specifically, we formulate the problem as a transferable contextual bandit problem, and prove that by transfer learning, one could significantly reduce the regret bound. Based on the theoretical result, we further develop a practical algorithm that decomposes a cell's policy into a common homogeneous policy learned using all cells' data and a cell-specific policy that captures each individual cell's heterogeneous behavior. We evaluate our proposed algorithm via a simulator constructed using real network data and demonstrates faster convergence compared to baselines. More importantly, a live field test is also conducted on a real metropolitan cellular network consisting 1700+ cells to optimize five parameters for two weeks. Our proposed algorithm shows a significant performance improvement of 20%.

In a vehicular edge computing (VEC) system, some vehicles with surplus computing resources can provide computation task offloading opportunities for other vehicles or pedestrians. However, the vehicular network is highly dynamic, with fast varying channel states and computation loads. These dynamics are difficult to model or to predict, but they have a major impact on the quality of service (QoS) of task offloading, including delay performance and service reliability. Meanwhile, the computing resources in VEC are often redundant due to the high density of vehicles. To improve the QoS of VEC and exploit the abundant computing resources on vehicles, we propose a learning-based task replication algorithm (LTRA) based on combinatorial multi-armed bandit (CMAB) theory, in order to minimize the average offloading delay. LTRA enables multiple vehicles to process the replicas of the same task simultaneously, and vehicles that require computing services can learn the delay performance of other vehicles while offloading tasks. We take the occurrence time of vehicles into consideration, and redesign the utility function of existing CMAB algorithm, so that LTRA can adapt to the time varying network topology of VEC. We use a realistic highway scenario to evaluate the delay performance and service reliability of LTRA through simulations, and show that compared with single task offloading, LTRA can improve the task completion ratio with deadline 0.6s from 80% to 98%.

Vehicular cloud computing (VCC) is proposed to effectively utilize and share the computing and storage resources on vehicles. However, due to the mobility of vehicles, the network topology, the wireless channel states and the available computing resources vary rapidly and are difficult to predict. In this work, we develop a learning-based task offloading framework using the multi-armed bandit (MAB) theory, which enables vehicles to learn the potential task offloading performance of its neighboring vehicles with excessive computing resources, namely service vehicles (SeVs), and minimizes the average offloading delay. We propose an adaptive volatile upper confidence bound (AVUCB) algorithm and augment it with load-awareness and occurrence-awareness, by redesigning the utility function of the classic MAB algorithms. The proposed AVUCB algorithm can effectively adapt to the dynamic vehicular environment, balance the tradeoff between exploration and exploitation in the learning process, and converge fast to the optimal SeV with theoretical performance guarantee. Simulations under both synthetic scenario and a realistic highway scenario are carried out, showing that the proposed algorithm achieves close-to- optimal delay performance.

In this paper, we propose and study opportunistic bandits - a new variant of bandits where the regret of pulling a suboptimal arm varies under different environmental conditions, such as network load or produce price. When the load/price is low, so is the cost/regret of pulling a suboptimal arm (e.g., trying a suboptimal network configuration). Therefore, intuitively, we could explore more when the load/price is low and exploit more when the load/price is high. Inspired by this intuition, we propose an Adaptive Upper-Confidence-Bound (AdaUCB) algorithm to adaptively balance the exploration-exploitation tradeoff for opportunistic bandits. We prove that AdaUCB achieves O(log T) regret with a smaller coefficient than the traditional UCB algorithm. Furthermore, AdaUCB achieves O(1) regret with respect to T if the exploration cost is zero when the load level is below a certain threshold. Last, based on both synthetic data and real-world traces, experimental results show that AdaUCB significantly outperforms other bandit algorithms, such as UCB and TS (Thompson Sampling), under large load/price fluctuations.

Cellular network configuration is critical for network performance. Current practice is mostly based on field experience and manual adjustment. The process is labor-intensive, error-prone, and far from optimal. To automate and optimize cellular network configuration, in this paper, we propose an online-learning-based joint-optimization approach that addresses a few specific challenges: limited data availability, convoluted sample data, highly complex optimization due to interactions among neighboring cells, and the need to adapt to network dynamics. In our approach, to learn an appropriate utility function for a cell, we develop a neural-network-based model that addresses the convoluted sample data issue and achieves good accuracy based on data aggregation. Based on the utility function learned, we formulate a global network configuration optimization problem. To solve this high-dimensional nonconcave maximization problem, we design a Gibbs-sampling-based algorithm that converges to an optimal solution when a technical parameter is small enough. Furthermore, we design an online scheme that updates the learned utility function and solves the corresponding maximization problem efficiently to adapt to network dynamics. To illustrate the idea, we use the case study of pilot power configuration. Numerical results illustrate the effectiveness of the proposed approach.

In vehicular cloud computing (VCC) systems, the computational resources of moving vehicles are exploited and managed by infrastructures, e.g., roadside units, to provide computational services. The offloading of computational tasks and collection of results rely on successful transmissions between vehicles and infrastructures during encounters. In this paper, we investigate how to provide timely computational services in VCC systems. In particular, we seek to minimize the deadline violation probability given a set of tasks to be executed in vehicular clouds. Due to the uncertainty of vehicle movements, the task replication methodology is leveraged which allows one task to be executed by several vehicles, and thus trading computational resources for delay reduction. The optimal task replication policy is of key interest. We first formulate the problem as a finite-horizon sampled-time Markov decision problem and obtain the optimal policy by value iterations. To conquer the complexity issue, we propose the balanced-task-assignment (BETA) policy which is proved optimal and has a clear structure: it always assigns the task with the minimum number of replicas. Moreover, a tight closed-form performance upper bound for the BETA policy is derived, which indicates that the deadline violation probability follows the Rayleigh distribution approximately. Applying the vehicle speed-density relationship in the traffic flow theory, we find that vehicle mobility benefits VCC systems more compared with road traffic systems, by showing that the optimum vehicle speed to minimize the deadline violation probability is larger than the critical vehicle speed in traffic theory which maximizes traffic flow efficiency.

Cellular network configuration is critical for network performance. Current practice is labor-intensive, error-prone, and far from optimal. To automate efficient cellular network configuration, in this work, we propose an online-learning-based joint-optimization approach that addresses a few specific challenges: limited data availability, convoluted sample data, highly complex optimization due to interactions among neighboring cells, and the need to adapt to network dynamics. In our approach, to learn an appropriate utility function for a cell, we develop a neural-network-based model that addresses the convoluted sample data issue and achieves good accuracy based on data aggregation. Based on the utility function learned, we formulate a global network configuration optimization problem. To solve this high-dimensional non-concave maximization problem, we design a Gibbs-sampling-based algorithm that converges to an optimal solution when a technical parameter is small enough. Furthermore, we design an online scheme that updates the learned utility function and solves the corresponding maximization problem efficiently to adapt to network dynamics. To illustrate the idea, we use the case study of pilot power configuration. Numerical results illustrate the effectiveness of the proposed approach.

Mobile edge computing is a novel technique in which mobile devices offload computation-intensive tasks with stringent delay requirements to the edge cloud. However, the limited computational resource in the edge cloud may result in the Quality of Service degradation. In this paper, we address this issue by coordinating the heterogeneous cloud which includes the edge cloud and the remote cloud. Considering the offloading of delay-bounded tasks, we study into the scheduling of heterogeneous cloud in order to maximize the probability that tasks can have the delay requirements met. The problem formulation is proved to be concave, and an optimal algorithm is proposed accordingly. The optimal policy with heterogeneous cloud is notably different from the policy merely using the edge cloud. With only the edge cloud, the system serves tasks with loose delay bounds and drops tasks with stringent delay bounds when the traffic load is heavy. However, with the heterogeneous cloud, tasks with stringent delay bounds are offloaded to the edge cloud and tasks with loose delay bounds are offloaded to the remote cloud. In numerical results, the probability that the delay bounds of tasks are satisfied can be improved by about 40% with the assistance of the remote cloud.

To reduce the energy consumption of various information and communication systems, sleeping mechanism design is considered to be a key problem. Prior work has derived optimal single server sleeping policies only for non-bursty, memoryless Poisson arrivals. In this paper, for the first time, we derive the optimal sleep operation for a single server facing bursty traffic arrivals. Specifically, we model job arrivals as a discrete-time interrupted Bernoulli process (IBP) which models bursty traffic arrivals. Key factors including the switching and working energy consumption costs as well as a delay penalty are accounted for in our model. As the arrival process state (busy or quiet) cannot be directly observed by the server, we formulate the problem as a POMDP (partially observable Markov decision process), and show that it can be tractably solved as a belief-MDP by considering the time interval since the last observed arrival t. We prove that the optimal sleeping policy is hysteretic and the numerical results reveal that the optimal policy is a t-based two- threshold policy, where the sleeping thresholds change with t. The simulation results show that our policy outperforms the previously derived Poisson-optimal policy and that the system cost decreases with the burstiness of traffic.

Edge cloud is a promising architecture in order to address the latency problem in mobile cloud computing. However, as compared with remote clouds, edge clouds have limited computational resources, and higher operating costs. In this paper, we design policies which carry out the assignment of tasks that are generated at the mobile subscribers with edge clouds in an online fashion. The proposed policies achieve an optimal power-delay trade-off in the system. Here, the delay experienced by a mobile computing task includes the time spent waiting for transmission to the edge cloud, and the execution time at the edge cloud servers. We perform a theoretical analysis after modeling the system as a continuous-time queueing system. The contribution of this paper is two-fold: Firstly, the algorithm to determine the optimal policy is obtained by proposing an equivalent discrete-time Markov decision process. Secondly, an easily implementable index policy is proposed by analyzing the dual of the original problem. Extensive simulations illustrate the effectiveness of the proposed policies.

Base station (BS) sleeping is an effective way to improve the energy-efficiency of cellular networks. However, it may bring extra user-perceived delay. We conduct a theoretical study into the impact of BS sleeping on both energy-efficiency and user-perceived delay. We consider hysteresis sleep and three typical wake-up schemes, namely single sleep, multiple sleep, and N-limited schemes. We model the system as an M/G/1 vacation queue, which captures the setup time, the mode-changing cost, as well as the counting or detection cost during the sleep mode. Closed-form expressions for the average power and the Laplace-Stieltjes transform of delay distribution are obtained. The impacts of system parameters on these expressions are analyzed. We then formulate an optimization problem to design delay-constrained energy-optimal BS sleeping policies. We show that the optimal solutions possess a special structure, thereby allowing us to obtain them explicitly or numerically by simple bisection search. In addition, the relationship between the optimal power consumption and the mean delay constraint is analyzed, so as to answer the fundamental question: how much energy can be saved by trading off a certain amount of delay? It is shown that this optimal relationship is linear only when the delay constraint is lower than a threshold. Numerical studies are also conducted, where the impact of detection or counting cost during the sleep mode is explored, and the delay distribution under the optimal policy is obtained.

In cyber-physical systems such as in-vehicle wireless sensor networks, a large number of sensor nodes continually generate measurements that should be received by other nodes such as actuators in a regular fashion. Meanwhile, energy-efficiency is also important in wireless sensor networks. Motivated by these, we develop scheduling policies which are energy efficient and simultaneously maintain “regular” deliveries of packets. A tradeoff parameter is introduced to balance these two conflicting objectives. We employ a Markov Decision Process (MDP) model where the state of each client is the time-since-last-delivery of its packet, and reduce it into an equivalent finite-state MDP problem. Although this equivalent problem can be solved by standard dynamic programming techniques, it suffers from a high-computational complexity. Thus we further pose the problem as a restless multi-armed bandit problem and employ the low-complexity Whittle Index policy. It is shown that this problem is indexable and the Whittle indexes are derived. Also, we prove the Whittle Index policy is asymptotically optimal and validate its optimality via extensive simulations.

A problem of much current practical interest is the replacement of the wiring infrastructure connecting approximately 200 sensor and actuator nodes in automobiles by an access point. This is motivated by the considerable savings in automobile weight, simplification of manufacturability, and future upgradability. A key issue is how to schedule the nodes on the shared access point so as to provide regular packet delivery. In this and other similar applications, the mean of the inter-delivery times of packets, i.e., throughput, is not sufficient to guarantee service-regularity. The time-averaged variance of the inter-delivery times of packets is also an important metric. So motivated, we consider a wireless network where an Access Point schedules real-time generated packets to nodes over a fading wireless channel. We are interested in designing simple policies which achieve optimal mean-variance tradeoff in interdelivery times of packets by minimizing the sum of time-averaged means and variances over all clients. Our goal is to explore the full range of the Pareto frontier of all weighted linear combinations of mean and variance so that one can fully exploit the design possibilities. We transform this problem into a Markov decision process and show that the problem of choosing which node's packet to transmit in each slot can be formulated as a bandit problem. We establish that this problem is indexable and explicitly derive the Whittle indices. The resulting Index policy is optimal in certain cases. We also provide upper and lower bounds on the cost for any policy. Extensive simulations show that Index policies perform better than previously proposed policies.

In cyber-physical systems such as automobiles, measurement data from sensor nodes should be delivered to other consumer nodes such as actuators in a regular fashion. But, in practical systems over unreliable media such as wireless, it is a significant challenge to guarantee small enough inter-delivery times for different clients with heterogeneous channel conditions and inter-delivery requirements. In this paper, we design scheduling policies aiming at satisfying the inter-delivery requirements of such clients. We formulate the problem as a risk-sensitive Markov Decision Process (MDP). Although the resulting problem involves an infinite state space, we first prove that there is an equivalent MDP involving only a finite number of states. Then we prove the existence of a stationary optimal policy and establish an algorithm to compute it in a finite number of steps.

Base station (BS) sleeping operation is one of the effective ways to save energy consumption of cellular networks, but it may lead to longer delay to the customers. The fundamental question then arises: How much energy can be traded off by a tolerable delay? In this paper, we characterize the fundamental tradeoffs between total energy consumption and overall delay in a BS with sleep mode operations by queueing models. Here, the BS total energy consumption includes not only the transmitting power but also basic power (for baseband processing, power amplifier, etc.) and switch-over power of the BS working mode, and the overall delay includes not only transmission delay but also queueing delay. Specifically, the BS is modeled as an M/G/1 vacation queue with setup and close-down times, where the BS enters sleep mode if no customers arrive during the close-down (hysteretic) time after the queue becomes empty. When asleep, the BS stays in sleep mode until the queue builds up to N customers during the sleep period ( N-Policy) . Several closed-form formulas are derived to demonstrate the tradeoffs between the energy consumption and the mean delay for different wake-up policies by changing the close-down time, setup time, and the parameter N. It is shown that the relationship between the energy consumption and the mean delay is linear in terms of mean close-down time, but non-linear in terms of N. The explicit relationship between total power consumption and average delay with varying service rate is also analyzed theoretically, indicating that sacrificing delay cannot always be traded off for energy saving. In other words, larger N may lead to lower energy consumption, but there exists an optimal N* that minimizes the mean delay and energy consumption at the same time. We also investigate the maximum delay (delay bound) for certain percentage of service and find that the delay bound is nearly linear in mean delay in the cases tested. Therefore, similar tradeoffs exist between energy consumption and the delay bound. In summary, the closed-form energy-delay tradeoffs cast light on designing BS sleeping and wake-up control policies that aim to save energy while maintaining acceptable quality of service.

Reasonable scanning task scheduling strategy can improve the scanning efficiency of vulnerability detection systems at a large extent. Task scheduling problem has been proved to be an NP-complete problem. Based on the request and characteristics of vulnerability detection technology, this paper establishes a distributed scanning task scheduling model, and describes a scanning task distributing algorithm. Genetic algorithms (GA) and ant colony algorithm (ACA) have been widely used to solve various types of NP problem. So far, they have been used to research scheduling algorithms, but there are still some defects. In order to overcome the shortcomings, after having studied the suggested algorithms for scanning task scheduling, a new algorithm that combines genetic algorithm and ant colony algorithm is proposed. Compared with the third reference, the algorithm has better load-balancing rate, stability and convergence. The effectiveness of the algorithm is verified by simulation experiment.

In power grid, we focus on one kind of high performance computing applications, that is, power system computing and Simulation (PCS) applications. PCS applications are always broken down into several sub-tasks depending on each other, which can be represented as a DAG. Genetic algorithm (GA) has been widely used to solve the dependent tasks scheduling. However the conventional GA is too slow to be used in power grid due to its time-consuming iteration. This paper applies an improved genetic algorithm (IGA) to dependent tasks scheduling in power grid. Based on the characteristic of Power Grid scheduling, we design detail the chromosome presentation, fitness function, and evolutionary process. And, this algorithm increases search efficiency with limited number of iteration by improving the evolutionary process while meeting a feasible result. An simulation study was conducted to evaluate the performance of the algorithm. It showed the general suitability of the presented algorithm within power grid.