In this paper, we address the problem of Device-to-Device communication (D2D) in the next generations of cellular networks, where the number of D2D pairs can grow large and, hence, improving spectral efficiency becomes a crucial design factor. More specifically, decentralized channel selection and power control by D2D pairs for interference mitigation without inflicting a heavy controlling overhead on the network become significant challenges in allocating resources. To this end, we introduce an online distributed reinforcement learning algorithm at D2D pairs to maximize network throughput, while guaranteeing both D2D users’ and cellular users’ (CUs) Quality of Service (QoS) under the dynamic wireless channel environment. To track and evaluate the performance of the proposed online algorithm, we define three metrics, i.e., D2D collision probability, D2D access rate and time-average network throughput. The simulation results confirm the convergence property of the proposed algorithm and shows improved performance in terms of three defined metrics as compared to the celebrated Q-learning-based method.

Mobile edge computing (MEC) is a promising technique which migrates computational intensive tasks from smart devices to edge servers, so as to increase the computational capacity of smart devices while saving the battery energy. In this paper, we consider a MEC network system where the smart device with energy harvesting module and electricity storage chooses an offloading rate to offload computational task to one of edge servers. We formulate the offloading problem as a joint minimization problem of energy consumption and delay in the long-term, while meeting the smart device’s quality of experience constraints. The challenge is that the varying renewable energy generation and the energy consumed by the current action will affect the next battery level. To address this problem, we use a reinforcement learning model to account for future dynamics in the environment. To this end, we develop an algorithm based on a deep Q noisy neural network that adjusts automatically the noise level at smart devices for exploration, and hence, replaces the epsilon-greedy policy, traditionally used in the Q-learning algorithm. Simulation results show that the proposed algorithm achieves lower energy consumption and better quality of experience, as compared to the celebrated deep Q-learning algorithm and the random scheme.

In this paper, we consider a multi-access edge computing (MEC) network with one base station (BS) and multiple users. A number of edge computing servers with limited computing and storage capability are attached to the BS to execute the computation tasks offloaded by the users. We develop an online mirror-prox optimization (OMO) algorithm to minimize the overall network delay for task computation. Solving the underlying optimization problem distributively across the users and over a long time horizon to obtain globally optimal decisions at individual users is challenging due to having to cope with time-varying cost function and constraints with unknown statistics. To make the proposed algorithm perform in a distributed and globally optimal manner at users, the BS broadcasts information based on the current states of the servers to individual users. We evaluate the performance of the algorithm using two performance metrics which are dynamic regret, assessing the closeness of the achievable cost against the dynamic optimal value, and aggregate violation, measuring the asymptotic satisfaction of the constraints. The simulation results indicate the effectiveness of the proposed algorithm in the long term and achieve considerable efficiency improvement in lower battery consumption at users' devices.

This work deals with the task offloading problem for multiple cellular edge devices in a multi-access edge computing (MEC) infrastructure attached to a base-station (BS). In order to minimize the overall task computing-communication delay through coping with time-varying cost and constraint functions with unknown statistics on-the-go, we propose a novel distributed bandit optimization (DBO) algorithm which runs based on the projected dual gradient iterations and a single broadcast communicating the MEC states to the SDs at the end of each time-slot. To track the performance of the proposed online learning algorithm over time, we define a dynamic regret to assess the closeness of the underlying delay cost of the DBO to a clairvoyant dynamic optimum and an aggregate violation metric to evaluate the asymptotic satisfaction of the constraints. We derive lower and upper bounds for dynamic regret as well as an upper-bound for the aggregate violation and show that the upper-bounds are sub-linear under sub-linear accumulated hindsight variations. The simulation results and comparisons confirm the effectiveness of the proposed algorithm in the long run.

In this paper, we address the problem of overall delay minimization in a cellular multi-access edge computing (MEC) network, where servers with limited computing and storage resources are co-located with a base station (BS) to execute the computation tasks offloaded by the users. We formulate this problem as a distributed mirror prox (DMP) optimization at individual users and the MEC servers' local controller (LC) over a long-time horizon and develop an online algorithm to ensure queue stability of the overall network in the long run. Both, the cost and the constraint functions are time-varying with unknown statistics. We evaluate the performance of the proposed algorithm using two performance metrics: the dynamic regret to assess the closeness of the achievable cost against the dynamic optimal value; and the aggregate violation to measure the asymptotic satisfaction of the constraints. Given the optimum hindsight variation is sub-linear, we prove that both of the dynamic regret and the aggregate violation are sub-linear in the long run. The simulation results confirm the superiority of the proposed DMP algorithm over the stochastic dual gradient in terms of delay minimization, dynamic regret, aggregate violation and energy efficiency in battery-powered user devices.

This paper focuses on a practically very important problem of matching a real-world product photo to exactly the same item(s) in online shopping sites. The task is extremely challenging because the user photos (i.e., the queries in this scenario) are often captured in uncontrolled environments, while the product images in online shops are mostly taken by professionals with clean backgrounds and perfect lighting conditions. To tackle the problem, we study deep network architectures and training schemes, with the goal of learning a robust deep feature representation that is able to bridge the domain gap between the user photos and the online product images. Our contributions are two-fold. First, we propose an alternative of the popular contrastive loss used in siamese deep networks, namely robust contrastive loss, where we "relax" the penalty on positive pairs to alleviate over-fitting. Second, a multi-task fine-tuning approach is introduced to learn a better feature representation, which not only incorporates knowledge from the provided training photo pairs, but also explores additional information from the large ImageNet dataset to regularize the fine-tuning procedure. Experiments on two challenging real-world datasets demonstrate that both the robust contrastive loss and the multi-task fine-tuning approach are effective, leading to very promising results with a time cost suitable for real-time retrieval.