Many extensions of the Multi-Armed Bandit (MAB) problem were studied recently offering a strong theoretical basis for applications that require active learning. The Budget-Constrained and Deadline-Driven MAB with Delay (BD-MAB) variation introduces a novel scenario in which a player must pull from K arms, each associated with reward, delay, and cost distributions. To the best of our knowledge, this is the first work that combines budget constraints with time delays in an MAB problem. The model involves three phases. The first phase comprises pulling arms, while incurring random cost, and observing rewards. This phase ends when the budget is depleted. The second phase includes observing some of the delayed rewards. The third phase starts at a fixed termination deadline, marking the end of reward observation, where any rewards returned after this phase are not observed and are considered dead. We present a novel solution to this problem by developing a new Upper Confidence Bound (UCB)-based algorithm. The name of the algorithm is Budget-Constrained and Deadline-Driven UCB with Delay (BD-UCB) algorithm. We provide extensive regret analysis that confirms the efficiency of our approach in managing these complexities. Lastly, we provide numerical simulations which further demonstrate the effectiveness of our proposed solution.

Deep Neural Network (DNN) models have been widely utilized in various applications. However, the growing complexity of DNNs has led to increased challenges and prolonged training durations. Despite the availability of high-performance computing systems, certain DNNs still require several days for successful training. This study aims to address this issue by proposing a method for significantly reducing the training time of deep learning models while maintaining test accuracy. Existing approaches primarily concentrate on optimizing training efficiency through computational and communication overlap/scheduling. In contrast, this research takes a step further by inspiring transfer learning. Transfer learning is a useful way to quickly retrain a model on new data without having to retrain the entire network. During transfer learning, the first layers of the network are frozen while leaving the end layers open to modification. By doing so, computation and communication requirements in these frozen layers are eliminated. This intelligent approach involves freezing some of the specific DNN layers and allocating resources to the remaining active layers during the training process, thereby minimizing DNN training time. To achieve this objective, we propose an intelligently freezing DNN using pipeline parallelism. Through trace-based simulation results, our scheme has demonstrated its effectiveness in efficiently reducing the time cost of a training iteration.

Stochastic Multi-Armed Bandit (MAB) has recently been studied widely due to its vast range of applications. The classic model considers the reward of a pulled arm to be observed after a time delay that is sampled from a random distribution assigned for each arm. In this paper, we propose an extended framework in which pulling an arm gives both an instant (short-term) reward and a delayed (long-term) reward at the same time. The distributions of reward values for short-term and long-term rewards are related with a previously known relationship. The distribution of time delay for an arm is independent of the reward distributions of the arm. In our work, we devise three UCB-based algorithms, where two of them are near-optimal-regret algorithms for this new model, with the corresponding regret analysis for each one of them. Additionally, the random distributions for time delay values are allowed to yield infinite time, which corresponds to a case where the arm only gives a short-term reward. Finally, we evaluate our algorithms and compare this paradigm with previously known models on both a synthetic data set and a real data set that would reflect one of the potential applications of this model.

Transfer learning refers to transferring the knowledge of a specific domain to a related domain. In cases where the source and the target learner have similar distribution and parameters, transfer learning can reduce the cost of learning and the construction of the target learner and improve the performance of the target learner. In wireless ad-hoc networks, the users connect to networks based on the service location, and various network channels with different levels of quality-of-service (QoS) are available. The wireless channels represent specific ranges of radio frequencies. When the users move from one location to another, the mobile application may switch channels for good quality of service. This paper predicts the wireless channel based on the user's location. Since channel prediction based on location is feasible in one city, the knowledge of channel prediction in one city can be transferred to another city. Thus, transfer learning is applicable and effective in such applications. The paper uses two cities' wireless mapping datasets to predict network channels and uses transfer learning to predict one city's network channels based on the other city's model. Experiments using different initial learning rates during training and different source and target domain data ratios show that transfer learning is feasible for network prediction among different cities.

Cyber-security research often focuses on attack-defense games where a strategic attacker seeks to destroy the defender's targets or kill him. In such a game there is a defender who just try to protect himself. In the real world, players can choose to protect themselves as well as kill their opponents to maximize their overall gain. In this case, the player allocates their budget for both defending and attacking actions and decides how well to attack and how well to defend against others. Players should allocate their budgets appropriately for each action throughout multiple rounds when playing such a game. The probabilities of surviving and killing in each round are determined by what happened in the previous rounds and the amount of the remaining budget. Players can continue playing until they die or run out of money. It is possible for the player not to be aware of everything his opponent does. Despite knowing that his opponent is playing according to one of the possible types, he cannot see which action exactly he is taking. Likewise, it may be the case that the player only sees the opponent's action, but does not know what its objective is. To meet this challenge, this paper develops a game where players decide how to allocate resources when they have partial information. For a model with complete information, equilibrium can be found and, as an extension, models with incomplete and imperfect information are also discussed. Our simulation examines how utility changes based on prior beliefs, total budgets, costs, and uncertainty.

One of the main hindrances that wireless sensor networks (WSNs) face is the battery-powered sensors that need to be charged from time to time. Recently, the approach of having mobile chargers (MCs) that travel to the static sensors of the network and transfer energy wirelessly and efficiently has become a promising solution to that hindrance. An optimization problem, called the mobile charger coverage problem, arises naturally to keep all of the sensors alive with an objective of determining both the minimum number of MCs required to meet the sensor recharge frequency and the schedule of these MCs. It is shown that this optimization problem becomes NP-hard in a high-dimensional space, including 2-D space with homogeneous recharging frequency. On the other hand, it is shown that a polynomial-time algorithm exists for sensors with a homogeneous recharge frequency on a 1-dimensional space (line or ring). In this paper, we seek to find a delicate border between the tractable and intractable problem space. Specifically, we study the special case of heterogeneous sensors that take frequencies of 1’s and 2’s (lifetimes of 1 and 0.5 time units) on a line, conjecture its NP-hardness, propose a novel brute-force optimal algorithm, and present a linear-time greedy algorithm that gives a 1.5-approximation solution for the problem. A comprehensive simulation is conducted to verify the efficiency of using our proposed algorithms.