In this article, we investigate the joint task offload for Device-to-Device (D2D) communications with energy harvesting and power control requirements. Utilizing D2D communication for data offloading effectively decreases the load on cellular Base Stations (BSs). Thus, we investigate the challenge of optimizing connection density in D2D networks with multiple connections and investigate scenarios involving delays and energy harvesting, aiming for simultaneous transmissions and efficient resource utilization accordingly. The BS assigns each device a task, and the under-resourced devices aim to build a connection with one helper and offload part of the task. This scheduling policy includes optimal task assignment and practical helper choice. We formulate this problem as finding a semi-matching over a bipartite graph derived from multiple connections and distinct constraints, i.e., power budget and time constraint. Furthermore, we compared our proposed solution, Energy Harvesting Task Offloading Semi-Matching (EHTO-SM), with predefined baselines. Numerical results confirm that the proposed task allocation scheme provides users with high-quality services and demonstrates the effectiveness and dynamic resource adaptability in multiuser settings in various scenarios.

Recently, vehicular over-the-air (OTA) updates have been raised as an intelligent solution for the autonomous vehicle revolution. It is critical to download important vehicular safety and stability updates onboard. The forthcoming six-generation (6G) systems might consider OTA a cach service delivered by base stations or roadside units (RSUs). This paper introduces the practical implementation of OTA updates using roadside units and dual-band WiGig/Wi-Fi communications. The system's performance is investigated against different blocking vehicles (small, medium, and large), simulating realistic scenarios. Experimental results ensure the 60 GHz WiGig performance efficiency regarding download speed and response time.

Recently, Narrowband Internet of Things (NB-IoT) systems gained a significant focus as a promising direction for massive connectivity issues in forthcoming wireless communication systems. Thus, this paper investigates the challenge of maximizing connection density in NB-IoT networks, considering a downlink Non-Orthogonal Multiple Access (NOMA) scenario for simultaneous transmissions and efficient resource utilization. The base station assigns each device to one of the accessible Physical Resource Blocks (PRBs). This scheduling policy includes effective device clustering and optimal NOMA power assignment. We formulate this problem as finding semi-matching over a bipartite graph derived from multiple PRBs and distinct constraints, i.e., power budget, admitted PRB, interference, and quality of service constraints. Furthermore, we compared our solutions NOMA-Semi Matching 1 (NOMA-SM1) and NOMA-Semi Matching 2 (NOMA-SM2) with previous solutions. Numerical Simulations confirm that the proposed semi-matching aided approaches attain a better theoretical bound and superior performance.

This paper uses an aerial mounted RIS (A-RIS) to assist WiGig base station (BS) in serving mobile equipments (MEs) located within hotspot zones. The aerial should cover numerous large-capacity hotspots in this context while anticipating its flying/hovering energy expenditures. Hence, two advanced multi armed bandit (MAB) approaches, i.e. perturbed history exploration (PHE) and mini-max optimal Thompson sampling (MOTS), are envisioned as applicable self-learning methodologies to deal with such a problem effectively. Simulation results ensure the excellent performance of the envisioned schemes over naive upper confidence bound (UCB) and Thompson sampling (TS) algorithms and traditional heuristic solutions.

As vehicular communication networks embrace metaverse beyond 5G/6G systems, the rich content update via the least interfered subchannel of the optimal frequency band in a hybrid band vehicle to everything (V2X) setting emerges as a challenging optimization problem. We model this problem as a tradeoff between multi-band VR/AR devices attempting to perform metaverse scenes and environmental updates to metaverse roadside units (MRSUs) while minimizing energy consumption. Due to the computational hardness of this optimization, we formulate an opportunistic band selection problem using a multi-armed bandit (MAB) that provides a good quality solution in real-time without computationally burdening the already stretched augmented/virtual reality (AR/VR) units acting as transmitting nodes. The opportunistic use of scheduling rich content updates at traffic signals and stand-still scenarios maps well with the formulated bandit problem. We propose a Dual-Objective Minimax Optimal Stochastic Strategy (DOMOSS) as a natural solution to this problem. Through extensive computer-based simulations, we demonstrate the effectiveness of our proposal in contrast to baselines and comparable solutions. We also verify the quality of our solution and the convergence of the proposed strategy.

Intelligent transportation systems have become increasingly important for efficient traffic management and road safety. Vehicle classification is a fundamental task in these systems, enabling various applications such as traffic monitoring, congestion management, and accident prevention. Traditional methods for vehicle classification heavily rely on visual or sensor-based data, such as images or radar signals. However, these methods may encounter limitations in adverse weather conditions, poor lighting, or occlusion scenarios. To address these limitations, this paper introduces a novel approach for vehicle classification using seismic data, which captures the vibrations generated by vehicles and is less susceptible to environmental factors. The proposed approach leverages the fractional wavelet domain to extract both time and frequency signatures of vehicles from the seismic data effectively. These signatures are then used to train a recurrent neural network that captures the temporal features of the input, thereby facilitating vehicle classification. Additionally, the proposed approach incorporates various modules to improve the generalization and robustness of the deep model against noise. The implementation of the proposed approach on realistic data demonstrated its ability to classify vehicles with an accuracy of 98%, improving upon the state-of-the-art techniques.

Recently emerging WiGig systems experience limited coverage and signal strength fluctuations due to strict line-of-sight (LoS) connectivity requirements. In this paper, we address these shortcomings of WiGig communication by exploiting two emerging technologies in tandem, namely the reconfigurable intelligent surface (RIS) and unmanned aerial vehicles (UAVs). In ultra-dense traffic sites (referred to as hotspots) where WiGig nodes or User Devices (UDs) experience complex propagation and non-line-of-sight (non-LoS) environment, we envision the deployment of a UAV-mounted RIS system to complement the WiGig base station (WGBS) to deliver services to the UDs. However, commercially available UAVs have limited energy (i.e., constrained flight time). Therefore, the trajectory of our considered UAV needs to be locally estimated to enable it to serve multiple hotspots while minimizing its energy consumption within the WGBS coverage boundaries. Since this tradeoff problem is computationally expensive for the resource-constrained UAV, we argue that sequential learning can be a lightweight yet effective solution to locally solve the problem with a low impact on the available energy on the UAV. We formally formulate this problem as a contextual multi-armed bandit (CMAB) game. Then, we develop the linear randomized upper confidence bound (Lin-RUCB) algorithm to solve the problem effectively. We regard the UAV as the bandit learner, which attempts to maximize its attainable rate (i.e., the reward) by serving distinct hotspots in its trajectory that we treat as the arms of the considered bandit. The context is defined as the hotspots’ locations provided using GPS (global positioning system) service and the reward history of each hotspot. Our proposal accounts for the energy expenditure of the UAV in moving from one hotspot to another within its battery charge lifetime. We evaluate the performance of our proposal via extensive simulations that exhibit the superiority of our proposed Lin-RUCB algorithm over benchmarking methods.

This paper proposes reinforcement learning (RL)-based solutions for unmanned aerial vehicle (UAV) data offloading in B5G mmWave-enabled communications. This is particularly useful for ad-hoc transmission scenarios within environments experiencing connectivity issues with the main servicing network as in disaster-stricken areas. Double deep Q-network and multiarmed bandit-based algorithms are proposed to tackle the joint problem of UAV-positioning and Rx-node activation and power allocation for data offloading in downlink NOMA transmissions. Numerical simulations are performed to ensure the proposed RL-based algorithms can adequately provide high data transfer rates, along with random and exhaustive search solutions as benchmarks for lower and upper bounds on the achievable sum-rate levels.

Abstract Millimetre wave (mmWave) communications, that is, 30 to 300 GHz, have intermittent short‐range transmissions, so the use of reconfigurable intelligent surface (RIS) seems to be a promising solution to extend its coverage. However, optimizing phase shifts (PSs) of both mmWave base station (BS) and RIS to maximize the received spectral efficiency at the intended receiver seems challenging due to massive antenna elements usage. In this paper, an online learning approach is proposed to address this problem, where it is considered a two‐phase multi‐armed bandit (MAB) game. In the first phase, the PS vector of the mmWave BS is adjusted, and based on it, the PS vector of the RIS is calibrated in the second phase and vice versa over the time horizon. The minimax optimal stochastic strategy (MOSS) MAB algorithm is utilized to implement the proposed two‐phase MAB approach efficiently. Furthermore, to relax the problem of estimating the channel state information (CSI) of both mmWave BS and RIS, codebook‐based PSs are considered. Finally, numerical analysis confirms the superior performance of the proposed scheme against the optimal performance under different scenarios.

To handle the exponentially growing service expectations from mobile users and circumvent the band switching slow rate, device-to-device (D2D) communication is receiving much research attention in the Internet of Things (IoT). While the emerging D2D nodes can support heterogeneous frequency bands [radio frequency (RF) including 2.4 GHz/5 GHz wireless local area network (WLAN), 38-GHz millimeter wave (mmWave), and visible light communication (VLC)], the physical constraints (e.g., blocking) require the user devices to dynamically switch between the bands in order to avoid the loss of connectivity and throughput degradation. In this article, we investigate an effective online link selection in hybrid RF-VLC scenarios for direct user data handling. First, we model the multiband selection issue as a multiarmed bandit (MAB) problem. The source/relay node acts as a player who gambles to maximize its long-term feedback/reward via selecting suitable arms, i.e., available bands (WLAN, mmWave, or VLC). Then, we propose an online, energy-aware band selection (EABS) methodology by leveraging three theoretically guaranteed MAB techniques [upper confidence bound (UCB), Thompson sampling (TS), and minimax optimal stochastic strategy (MOSS)] to derive optimal band selection policies. Based on these adopted policies, we propose three algorithms, namely, EABS-UCB, EABS-TS, and EABS-MOSS, to implement the EABS strategy, respectively. Extensive simulations demonstrate our proposed algorithms’ superior performance compared to the traditional link selection schemes regarding energy efficiency, average throughput, and convergence rate. In particular, EABS-MOSS emerges as the best algorithm as it exhibits near-optimal performance due to its flexibility to both stochastic and adversarial environments.

We consider the expert advice problem with a low rank but noisy loss sequence, where a loss vector $l_{t} \in [-1,1]^N$ in each round $t$ is of the form $l_{t} = U v_{t} + \epsilon_{t}$ for some fixed but unknown $N \times d$ matrix $U$ called the kernel, some $d$-dimensional seed vector $v_{t} \in \mathbb{R}^{d}$, and some additional noisy term $\epsilon_t \in \mathbb{R}^{N}$ whose norm is bounded by $\epsilon$. This is a generalization of the works of Hazan et al. and Barman et al., where the former only treats noiseless loss and the latter assumes that the kernel is known in advance. In this paper, we propose an algorithm, where we re-construct the kernel under the assumptions, that the low rank loss is noised and there is no prior information about kernel. In this algorithm, we approximate the kernel by choosing a set of loss vectors with a high degree of independence from each other, and we give a regret bound of $O(d\sqrt{T}+d^{4/3}(N\epsilon)^{1/3}\sqrt{T})$. Moreover, even if in experiment, the proposed algorithm performs better than Hazan’s algorithm and Hedge algorithm.

We consider a variant of the online semi-definite programming problem: The decision space consists of positive semi-definite matrices with bounded diagonal entries and bounded $\Gamma$-trace norm, which is a generalization of the trace norm defined by a positive definite matrix $\Gamma$. To solve this problem, we propose a follow-the-regularized-leader algorithm with a novel regularizer, which is a generalisation of the log-determinant function parameterized by the matrix $\Gamma$. Then we apply our algorithm to online binary matrix completion (OBMC) with side information and online similarity prediction with side information, and improve mistake bounds by logarithmic factors. In particular, for OBMC our mistake bound is optimal.

While hybrid-band wireless systems recently gained prominence to achieve high capacity, selecting the best channel in these systems in real-time is still a formidable research challenge that requires further investigations. In this paper, we address this challenge in terms of an optimization problem, which is reformu-lated as a stochastic multi-armed bandit (MAB). Then, we introduce online learning-based solutions to solve the MAB problem for the multi-band/channel selection (MBS). Improved variants of the upper confidence bound (UCB) scheme are investigated and modified to be energy-aware. Hence, we propose Energy-Aware Randomized UCB-MBS (EA-RUCB-MBS) and Energy-Aware Kullback-Leibler UCB-MBS (EA-KLUCB-MBS) methods, which demonstrate near-optimal results. Also, EA-KLUCB-MBS exhibits the fastest convergence, while the convergence of EA-RUCB-MBS is similar to that of the original UCB. Based on extensive simulation results, we evaluate the performance of our proposed algorithms against benchmark MBS schemes including UCB and Thompson sampling (TS).

We propose a new formulation of multiple-instance learning (MIL), in which a unit of data consists of a set of instances called a bag. The goal is to find a good classifier of bags based on the similarity with a "shapelet" (or pattern), where the similarity of a bag with a shapelet is the maximum similarity of instances in the bag. In previous work, some of the training instances have been chosen as shapelets with no theoretical justification. In our formulation, we use all possible, and thus infinitely many, shapelets, resulting in a richer class of classifiers. We show that the formulation is tractable, that is, it can be reduced through linear programming boosting (LPBoost) to difference of convex (DC) programs of finite (actually polynomial) size. Our theoretical result also gives justification to the heuristics of some previous work. The time complexity of the proposed algorithm highly depends on the size of the set of all instances in the training sample. To apply to the data containing a large number of instances, we also propose a heuristic option of the algorithm without the loss of the theoretical guarantee. Our empirical study demonstrates that our algorithm uniformly works for shapelet learning tasks on time-series classification and various MIL tasks with comparable accuracy to the existing methods. Moreover, we show that the proposed heuristics allow us to achieve the result in reasonable computational time.

Extended learning support systems for all-age education requires inclusive user interface design, especially for elderly users. A dual-tablet user interface with simplified visual layers and more intuitive operations was proposed aiming to reduce the physical and mental loads of elderly learners. An initial prototype with basic functions of viewing learning material was developed based on a cross-platform framework. Two preliminary user experiments participated by elderly volunteers were carried out for formative evaluations, in order to improve the usability of the interface design iteratively. The prototype was modified based on the participants' comments and observation of their operations during the experiments. Additional findings of the elderly users' preference and tendency were discussed for further development.

To improve designs of e-learning materials, it is necessary to know which word or figure a learner felt "difficult" in the materials. In this pilot study, we measured electroencephalography (EEG) and eye gaze data of learners and analyzed to estimate which area they had difficulty to learn. The developed system realized simultaneous measurements of physiological data and subjective evaluations during learning. Using this system, we observed specific EEG activity in difficult pages. Integrating of eye gaze and EEG measurements raised a possibility to determine where a learner felt "difficult" in a page of learning materials. From these results, we could suggest that the multimodal measurements of EEG and eye gaze would lead to effective improvement of learning materials. For future study, more data collection using various materials and learners with different backgrounds is necessary. This study could lead to establishing a method to improve e-learning materials based on learners' mental states.

We present a short survey on recent results on combinatorial online prediction in the adversarial setting.

We consider a job scheduling problem under precedence constraints, a classical problem for a single processor and multiple jobs to be done. The goal is, given processing time of n fixed jobs and precedence constraints over jobs, to find a permutation of n jobs that minimizes the total flow time, i.e., the sum of total wait time and processing times of all jobs, while satisfying the precedence constraints. The problem is an integer program and is NP-hard in general. We propose a decision diagram pi-MDD, for solving the scheduling problem exactly. Our diagram is suitable for solving linear optimization over permutations with precedence constraints. We show the effectiveness of our approach on the experiments on large scale artificial scheduling problems.

The problem of learning a sparse model is conceptually interpreted as the process of identifying active features/samples and then optimizing the model over them. Recently introduced safe screening allows us to identify a part of non-active features/samples. So far, safe screening has been individually studied either for feature screening or for sample screening. In this paper, we introduce a new approach for safely screening features and samples simultaneously by alternatively iterating feature and sample screening steps. A significant advantage of considering them simultaneously rather than individually is that they have a synergy effect in the sense that the results of the previous safe feature screening can be exploited for improving the next safe sample screening performances, and vice-versa. We first theoretically investigate the synergy effect, and then illustrate the practical advantage through intensive numerical experiments for problems with large numbers of features and samples.

We consider the learning problem under an online Markov decision process (MDP) aimed at learning the time-dependent decision-making policy of an agent that minimizes the regret-the difference from the best fixed policy. The difficulty of online MDP learning is that the reward function changes over time. In this letter, we show that a simple online policy gradient algorithm achieves regret O(√T) for T steps under a certain concavity assumption and O(log T) under a strong concavity assumption. To the best of our knowledge, this is the first work to present an online MDP algorithm that can handle continuous state, action, and parameter spaces with guarantee. We also illustrate the behavior of the proposed online policy gradient method through experiments.

We consider an online density estimation problem for the Bradley-Terry model, where each model parameter defines the probability of a match result between any pair in a set of n teams. The problem is hard because the loss function (i.e., the negative log-likelihood function in our problem setting) is not convex. To avoid the non-convexity, we can change parameters so that the loss function becomes convex with respect to the new parameter. But then the radius K of the reparameterized domain may be infinite, where K depends on the outcome sequence. So we put a mild assumption that guarantees that K is finite. We can thus employ standard online convex optimization algorithms, namely OGD and ONS, over the reparameterized domain, and get regret bounds O(n^\frac12(\ln K)\sqrtT) and O(n^\frac32K\ln T), respectively, where T is the horizon of the game. The bounds roughly means that OGD is better when K is large while ONS is better when K is small. But how large can K be? We show that K can be as large as Θ(T^n-1), which implies that the worst case regret bounds of OGD and ONS are O(n^\frac32\sqrtT\ln T) and \tildeO(n^\frac32(T)^n-1), respectively. We then propose a version of Follow the Regularized Leader, whose regret bound is close to the minimum of those of OGD and ONS. In other words, our algorithm is competitive with both for a wide range of values of K. In particular, our algorithm achieves the worst case regret bound O(n^\frac52T^\frac13 \ln T), which is slightly better than OGD with respect to T. In addition, our algorithm works without the knowledge K, which is a practical advantage.

We study the problem of classification when only a dissimilarity function between objects is accessible. That is, data samples are represented not by feature vectors but in terms of their pairwise dissimilarities. We establish sufficient conditions for dissimilarity functions to allow building accurate classifiers. The theory immediately suggests a learning paradigm: construct an ensemble of simple classifiers, each depending on a pair of examples; then find a convex combination of them to achieve a large margin. We next develop a practical algorithm referred to as dissimilarity-based boosting (DBoost) for learning with dissimilarity functions under theoretical guidance. Experiments on a variety of databases demonstrate that the DBoost algorithm is promising for several dissimilarity measures widely used in practice.

We investigate improvements of AdaBoost that can exploit the fact that the weak hypotheses are one-sided, i.e. either all its positive (or negative) predictions are correct. In particular, for any set of m labeled examples consistent with a disjunction of k literals (which are one-sided in this case), AdaBoost constructs a consistent hypothesis by using O(k2 log m) iterations. On the other hand, a greedy set covering algorithm finds a consistent hypothesis of size O(k log m). Our primary question is whether there is a simple boosting algorithm that performs as well as the greedy set covering. We first show that InfoBoost, a modification of AdaBoost pro- posed by Aslam for a different purpose, does perform as well as the greedy set covering algorithm. We then show that AdaBoost requires Ω(k2 log m) iterations for learning k-literal disjunctions. We achieve this with an adversary construction and as well as in simple experiments based on artificial data. Further we give a vari- ant called SemiBoost that can handle the degenerate case when the given examples all have the same label. We conclude by showing that SemiBoost can be used to produce small conjunctions as well.