In large-scale Internet of Things (IoT) deployments, efficiently allocating computing resources to IoT devices, while preserving the integrity and utility of their data, remains a critical challenge. This paper introduces a novel online probabilistic model designed to handle uncertainties in both demand and resource availability within IoT networks, where the computing tasks of requesting devices (RDs) are fulfilled by serving devices (SDs). The proposed model integrates stochastic elements and formulates an optimization problem that aims to minimize the number of active serving devices required for task offloading, subject to the constraints of available computing resources. To further enhance decision-making, the model incorporates the concept ofValue of Information (VoI)to ensure that the informational utility of each device’s data remains above a predefined threshold during task processing. The optimization problem is addressed using a heuristic algorithm. In scenarios where no serving device is immediately available, tasks are temporarily stored in a buffer and deferred to the next time slot, with their waiting time being tracked. This task allocation process is inspired by bin-packing algorithms, which are known for their efficiency in resource management and task scheduling. Moreover, the paper evaluates the performance of the proposed solution under worst-case conditions through feasibility analysis, thereby demonstrating its robustness. Two buffering strategies, First-In First-Out (FIFO) and Last-In First-Out (LIFO), are also examined to model task retrieval and execution behavior. Results show that adopting the FIFO strategy can reduce the average waiting time by approximately 50%. Overall, the proposed framework provides a reliable and scalable task computing service, with each serving device capable of supporting, on average, four requesting devices under typical operating conditions.

Artificial intelligence (AI), and more specifically deep learning techniques, have demonstrated strong capabilities in processing wireless signals, enabling automatic modulation classification (AMC). However, existing AI-based AMC methods often produce unreliable predictions and lack robustness to out-of-distribution (OOD) inputs, which limits their deployment in real-world scenarios. This study aims to address the gap in simulations to real deployments by fitting predictions to OOD scenarios. We propose an uncertainty-aware AMC framework based on supervised contrastive learning (SupCon). The proposed framework aims to enhance classification reliability, particularly under OOD conditions. In this framework, a ResNet-based representation learning model that consists of a feature extractor and a projection head is first trained using a combination of SupCon loss and cross-entropy (CE) loss to produce class-discriminative embeddings. By decoupling the shared representation learning model from task-specific classifiers, the framework enables a modular and computationally efficient uncertainty estimation strategy, avoiding redundant computation during inference. The learned embeddings are then processed by a set of class-wise binary classifiers that provide both classification and sample-wise uncertainty estimates. A rejection mechanism is incorporated to improve decision reliability by discarding uncertain predictions. We evaluate the proposed framework on the RadioML 2018 dataset. Experimental results show that our approach significantly improves representation quality and classification reliability compared to conventional supervised learning. Specifically, the classification accuracy increases from 63.7% to 93.6% under in-distribution conditions, and from 30.8% to 92.5% under 50% OOD contamination, while maintaining over 85% recall on accepted predictions.

In this paper, we propose a novel joint task offloading and user scheduling (JTO-US) framework for 5G mobile edge computing (MEC) systems under security threats from jamming attacks. The goal is to minimize the delay and the ratio of dropped tasks, taking into account both communication and computation delays. The system model includes a 5G network equipped with MEC servers and an adversarial on-off jammer that disrupts communication. The proposed framework optimally schedules tasks and users to minimize the impact of jamming while ensuring that high-priority tasks are processed efficiently. Genetic algorithm (GA) is used to solve the optimization problem, and the results are compared with benchmark methods such as GA without considering jamming effect, Shortest Job First (SJF), and Shortest Deadline First (SDF). The simulation results demonstrate that the proposed JTO-US framework achieves the lowest drop ratio in the presence of the jammer and effectively manages priority tasks, outperforming existing methods. Particularly, when the jamming probability is 0.8, the proposed framework mitigates the jammer's impact by reducing the drop ratio to 63%, compared to 89% achieved by the next best method.

This paper presents a novel heuristic deep reinforcement learning (HDRL) framework designed to optimize reconfigurable intelligent surface (RIS) phase shifts in secure satellite communication systems utilizing rate splitting multiple access (RSMA). The proposed HDRL approach addresses the challenges of large action spaces inherent in deep reinforcement learning by integrating heuristic algorithms, thus improving exploration efficiency and leading to faster convergence toward optimal solutions. We validate the effectiveness of HDRL through comprehensive simulations, demonstrating its superiority over traditional algorithms, including random phase shift, greedy algorithm, exhaustive search, and Deep Q-Network (DQN), in terms of secure sum rate and computational efficiency. Additionally, we compare the performance of RSMA with non-orthogonal multiple access (NOMA), highlighting that RSMA, particularly when implemented with an increased number of RIS elements, significantly enhances secure communication performance. The results indicate that HDRL is a powerful tool for improving the security and reliability of RSMA satellite communication systems, offering a practical balance between performance and computational demands.

In mmWave wireless networks, signal blockages present a significant challenge due to the susceptibility to environmental moving obstructions. Recently, the availability of visual data has been leveraged to enhance blockage prediction accuracy in mmWave networks. In this work, we propose a Vision Transformer (ViT)-based approach for visual-aided blockage prediction that intelligently switches between mmWave and Sub-6 GHz frequencies to maximize network throughput and maintain reliable connectivity. Given the computational demands of processing visual data, we implement our solution within a hierarchical fog-cloud computing architecture, where fog nodes collaborate with cloud servers to efficiently manage computational tasks. This structure incorporates a generative AI-based compression technique that significantly reduces the volume of visual data transmitted between fog nodes and cloud centers. Our proposed method is tested with the real-world DeepSense 6G dataset, and according to the simulation results, it achieves a blockage prediction accuracy of$\mathbf{9 2. 7 8 \%}$while reducing bandwidth usage by 70.31 %.

This paper proposes a novel semi-self sensing hybrid reconfigurable intelligent surface (SS-HRIS) in terahertz (THz) bands, where the RIS is equipped with reflecting elements divided between passive and active elements in addition to sensing elements. SS-HRIS along with integrated sensing and communications (ISAC) can help to mitigate the multipath attenuation that is abundant in THz bands. In our proposed scheme, sensors are configured at the SS-HRIS to receive the radar echo signal from a target. A joint base station (BS) beamforming and HRIS precoding matrix optimization problem is proposed to maximize the sum rate of communication users while maintaining satisfactory sensing performance measured by the Cramér- Rao bound (CRB) for estimating the direction of angles of arrival (AoA) of the echo signal and thermal noise at the target. The CRB expression is first derived and the sum rate maximization problem is formulated subject to communication and sensing performance constraints. To solve the complex non-convex optimization problem, deep deterministic policy gradient (DDPG)- based deep reinforcement learning (DRL) algorithm is proposed, where the reward function, the action space and the state space are modeled. Simulation results show that the proposed DDPG- based DRL algorithm converges well and achieves better performance than several baselines, such as the soft actor-critic (SAC), proximal policy optimization (PPO), greedy algorithm and random BS beamforming and HRIS precoding matrix schemes. Moreover, it demonstrates that adopting HRIS significantly enhances the achievable sum rate compared to passive RIS and random BS beamforming and HRIS precoding matrix schemes.

Satellite-based communication systems are integral to delivering high-speed data services in aviation, particularly for business aviation operations requiring global connectivity. These systems, however, are challenged by a multitude of interdependent factors such as satellite handovers, congestion, flight maneuvers and seasonal trends, making network performance prediction a complex task. No established methodologies currently exist for network performance prediction in avionic communication systems. This paper addresses the gap by proposing machine learning (ML)-based approaches for pre-flight network performance predictions. The proposed models predict performance along a given flight path, taking as input positional and network-related information and outputting the predicted performance for each position. In business aviation, flight crews typically have multiple flight plans to choose from for each city pair, allowing them to select the most optimal option. This approach enables proactive decision-making, such as selecting optimal flight paths prior to departure.

Emerging services such as augmented reality (AR) and virtual reality (VR) have increased the volume of data transmitted in wireless communication systems, revealing the limitations of traditional Shannon theory. To address these limitations, semantic communication has been proposed as a solution that prioritizes the meaning of messages over the exact transmission of bits. This paper explores semantic communication for text data transmission in end-to-end (E2E) systems through a novel approach called KG-LLM semantic communication, which integrates knowledge graph (KG) extraction and large language model (LLM) coding. In this method, the transmitter first utilizes a KG to extract key entities and relationships from sentences. The extracted information is then encoded using an LLM to obtain the semantic meaning. On the receiver side, messages are decoded using another LLM, while a bidirectional encoder representations from transformers (i.e., BERT) model further refines the reconstructed sentences for improved semantic similarity. The KG-LLM semantic communication method reduces the transmitted text data volume by $30 \%$ through KG-based compression and achieves $84 \%$ semantic similarity between the original and received messages. This demonstrates the KG-LLM methods efficiency and robustness in semantic communication systems, outperforming the deep learning-based semantic communication model (DeepSC), which achieves only $63 \%$.

Cell-free Integrated Sensing and Communication (ISAC) aims to revolutionize 6th Generation (6G) networks. By combining distributed access points with ISAC capabilities, it boosts spectral efficiency, situational awareness, and communication reliability. Channel estimation is a critical step in cell-free ISAC systems to ensure reliable communication, but its performance is usually limited by challenges such as pilot contamination and noisy channel estimates. This paper presents a novel framework leveraging sensing information as a key input within a Conditional Denoising Diffusion Model (CDDM). In this framework, we integrate CDDM with a Multimodal Transformer (MMT) to enhance channel estimation in ISAC-enabled cell-free systems. The MMT encoder effectively captures inter-modal relationships between sensing and location data, enabling the CDDM to iteratively denoise and refine channel estimates. Simulation results demonstrate that the proposed approach achieves significant performance gains. As compared with Least Squares (LS) and Minimum Mean Squared Error (MMSE) estimators, the proposed model achieves normalized mean squared error (NMSE) improvements of 8 dB and 9 dB, respectively. Moreover, we achieve a 27.8% NMSE improvement compared to the traditional denoising diffusion model (TDDM), which does not incorporate sensing channel information. Additionally, the model exhibits higher robustness against pilot contamination and maintains high accuracy under challenging conditions, such as low signal-to-noise ratios (SNRs). According to the simulation results, the model performs well for users near sensing targets by leveraging the correlation between sensing and communication channels.

Reconfigurable intelligent surfaces (RIS) have emerged as a promising technology for enhancing wireless communication by dynamically controlling signal propagation in the environment. However, their efficient deployment relies on accurate channel state information (CSI), which leads to high channel estimation overhead due to their passive nature and the large number of reflective elements. In this work, we solve this challenge by proposing a novel framework that leverages a pre-trained open-source foundation model (FM) named large wireless model (LWM) to process wireless channels and generate versatile and contextualized channel embeddings. These embeddings are then used for the joint optimization of the BS beamforming and RIS configurations. To be more specific, for joint optimization, we design a deep reinforcement learning (DRL) model to automatically select the BS beamforming vector and RIS phase-shift matrix, aiming to maximize the spectral efficiency (SE). This work shows that a pre-trained FM for radio signal understanding can be fine-tuned and integrated with DRL for effective decision-making in wireless networks. It highlights the potential of modality-specific FMs in real-world network optimization. According to the simulation results, the proposed method outperforms the DRL-based approach and beam sweeping-based approach, achieving 9.89% and 43.66% higher SE, respectively.

This paper investigates the problem of Cramér-Rao Bound (CRB) minimization in a semi-self sensing (SS) active reconfigurable intelligent surface (RIS)-enabled integrated sensing and communication (ISAC) system in millimeter-wave (mmWave). Unlike conventional RIS, the proposed active SS-RIS architecture incorporates both reflecting and sensing elements, enabling direct radar echo reception while enhancing communication performance. A joint optimization problem is formulated to design the base station (BS) beamforming and active RIS precoding matrix with the objective of minimizing the CRB of the target's angle-of-arrival (AoA) estimation, subject to communication quality-of-service (QoS) constraints and power limitations at both the BS and RIS. Given the inherent non-convexity and computational complexity of the optimization problem, a twin delayed DDPG (TD3)-based deep reinforcement learning (DRL) framework is proposed to efficiently optimize both BS beamforming and active SS-RIS precoding matrix. Simulation results demonstrate that the proposed TD3-based approach significantly outperforms other machine learning (ML) benchmark schemes, such as deep deterministic policy gradient (DDPG) and proximal policy optimization (PPO), in terms of sensing accuracy (CRB minimization) given the present constraints. Moreover, we have shown that increasing the number of sensing elements in the SS-RIS offers substantial gains, confirming the effectiveness of active SS-RIS in enhancing both sensing and communication performance in ISAC systems.

Intent-based network automation is a promising tool that enables easier network management; however, certain challenges must be addressed effectively. These are: 1) processing intents, i.e., identification of logic and necessary parameters to fulfill an intent, 2) validating an intent to align it with current network status, and 3) satisfying intents via network optimizing applications. This paper addresses these points via a three-fold strategy to introduce intent-based automation for modern 5G architectures. First, intents are processed via a lightweight Large Language Model (LLM). Secondly, once an intent is processed, it is validated against future incoming traffic volume profiles (high or low). Finally, a series of network optimization applications has been developed. With their machine learning-based functionalities, they can improve certain key performance indicators such as throughput, delay, and energy efficiency. In the final stage, using an attention-based hierarchical reinforcement learning algorithm, these applications are optimally initiated to satisfy the intent of an operator. Our simulations show that the proposed method can achieve at least a 12% increase in throughput, a 17.1% increase in energy efficiency, and a 26.5% decrease in network delay compared to the baseline algorithms.

Mobile traffic prediction is an important enabler for optimizing resource allocation and improving energy efficiency in mobile wireless networks. Building on the advanced contextual understanding and generative capabilities of large language models (LLMs), this work introduces a context-aware wireless traffic prediction framework powered by LLMs. To further enhance prediction accuracy, we leverage in-context learning (ICL) and develop a novel two-step demonstration selection strategy, optimizing the performance of LLM-based predictions. The initial step involves selecting ICL demonstrations using the effectiveness rule, followed by a second step that determines whether the chosen demonstrations should be utilized, based on the informativeness rule. We also provide an analytical framework for both informativeness and effectiveness rules. The effectiveness of the proposed framework is demonstrated with a real-world fifth-generation (5G) dataset with different application scenarios. According to the numerical results, the proposed framework shows lower mean squared error and higherR2-Scores compared to the zero-shot prediction method and other demonstration selection methods, such as constant ICL demonstration selection and distance-only-based ICL demonstration selection.

The integration of unmanned aerial vehicles (UAVs) with mobile edge computing (MEC) and Internet of Things (IoT) technology is crucial for efficient resource management and sustainable agricultural productivity in smart frames. This paper addresses the critical need for optimizing task offloading in secure UAV-assisted smart farm networks, aiming to reduce total delay and energy consumption while maintaining robust security in data communications. We propose a multi-agent deep reinforcement learning (DRL)-based approach using a deep double Q-network (DDQN) with an action mask (AM), designed to manage task offloading dynamically and efficiently. Simulation results demonstrate the superior performance of our method in managing task offloading, highlighting significant improvements in operational efficiency, such as reduced delay and energy consumption. This aligns with the goal of developing sustainable and energy-efficient solutions for next-generation network infrastructures, making our approach an advanced solution for performance and sustainability in smart farming applications.

Federated learning (FL) is a promising technique for learning-based functions in wireless networks, thanks to its distributed implementation capability. On the other hand, distributed learning may increase the risk of exposure to malicious attacks where attacks on a local model may spread to other models by parameter exchange. Meanwhile, such attacks can be hard to detect due to the dynamic wireless environment, especially considering local models can be heterogeneous with non-independent and identically distributed (non-IID) data. Therefore, it is critical to evaluate the effect of malicious attacks and develop advanced defense techniques for FL-enabled wireless networks. In this work, we introduce a federated deep reinforcement learning-based cell sleep control scenario that enhances the energy efficiency of the network. We propose multiple intelligent attacks targeting the learning-based approach and we propose defense methods to mitigate such attacks. In particular, we have designed two attack models, generative adversarial network (GAN)-enhanced model poisoning attack and regularization-based model poisoning attack. As a counteraction, we have proposed two defense schemes, autoencoder-based defense, and knowledge distillation (KD)-enabled defense. The autoencoder-based defense method leverages an autoencoder to identify the malicious participants and only aggregate the parameters of benign local models during the global aggregation, while KD-based defense protects the model from attacks by controlling the knowledge transferred between the global model and local models. The simulation results demonstrate that the proposed attacks can degrade the network performance by 34% and 77%, and lead to lower throughput and energy efficiency. On the other hand, our proposed defense schemes can effectively protect the system from attacks. The system performance can be recovered to approximately 95% of a secure system by using the proposed KD-based defense.

Integrated sensing and communications is a key enabler for the 6G wireless communication systems. The multiple sensing modalities will allow the base station to have a more accurate representation of the environment, leading to context-aware communications. Some widely equipped sensors such as cameras and RADAR sensors can provide some environmental perceptions. However, they are not enough to generate precise environmental representations, especially in adverse weather conditions. On the other hand, the LiDAR sensors provide more accurate representations, however, their widespread adoption is hindered by their high cost. This paper proposes a novel approach to enhance the wireless communication systems by synthesizing LiDAR point clouds from images and RADAR data. Specifically, it uses a multimodal transformer architecture and pre-trained encoding models to enable an accurate LiDAR generation. The proposed framework is evaluated on the DeepSense 6G dataset, which is a real-world dataset curated for context-aware wireless applications. Our results demonstrate the efficacy of the proposed approach in accurately generating LiDAR point clouds. We achieve a modified mean squared error of 10.39 with {256, 128, 64, 64} convolutional filters in the LiDAR decoder, as compared to 38.58 achieved for the all-zeroes benchmark. Visual examination of the images indicates that our model can successfully capture the majority of structures present in the LiDAR point cloud for diverse environments. By integrating LiDAR synthesis with existing sensing modalities, our method can enhance the performance of various wireless applications, including beam and blockage prediction.

In 5G networks, network slicing has emerged as a pivotal paradigm to address diverse user demands and service requirements. To meet the requirements, reinforcement learning (RL) algorithms have been utilized widely, but this method has the problem of overestimation and exploration-exploitation trade-offs. To tackle these problems, this paper explores the application of self-play ensemble Q-learning, an extended version of the RL-based technique. Self-play ensemble Q-learning utilizes multiple Q-tables with various exploration-exploitation rates leading to different observations for choosing the most suitable action for each state. Moreover, through self-play, each model endeavors to enhance its performance compared to its previous iterations, boosting system efficiency, and decreasing the effect of overestimation. For performance evaluation, we consider three RL-based algorithms; self-play ensemble Q-learning, double Q-learning, and Q-learning, and compare their performance under different network traffic. Through simulations, we demonstrate the effectiveness of self-play ensemble Q-learning in meeting the diverse demands within 21.92% in latency, 24.22% in throughput, and 23.63% in packet drop rate in comparison with the baseline methods. Furthermore, we evaluate the robustness of self-play ensemble Q-learning and double Q-learning in situations where one of the Q-tables is affected by a malicious user. Our results depicted that the self-play ensemble Q-learning method is more robust against adversarial users and prevents a noticeable drop in system performance, mitigating the impact of users manipulating policies.

Large language models (LLMs), especially generative pre-trained transformers (GPTs), have recently demonstrated outstanding ability in information comprehension and problem-solving. This has motivated many studies in applying LLMs to wireless communication networks. In this paper, we propose a pre-trained LLM-empowered framework to perform fully automatic network intrusion detection. Three in-context learning methods are designed and compared to enhance the performance of LLMs. With experiments on a real network intrusion detection dataset, in-context learning proves to be highly beneficial in improving the task processing performance in a way that no further training or fine-tuning of LLMs is required. We show that for GPT-4, testing accuracy and F1-Score can be improved by 90%. Moreover, pre-trained LLMs demonstrate big potential in performing wireless communication-related tasks. Specifically, the proposed framework can reach an accuracy and F1-Score of over 95% on different types of attacks with GPT-4 using only 10 in-context learning examples.

Jamming attacks present a crucial threat to the quality of service in wireless networks, disrupting essential features including reliability, latency, and effective rate specifically in mission-critical applications. This paper introduces and evaluates a NOMA-based model to improve the robustness of wireless networks against jamming attacks. To investigate and address the consequences of a reactive jammer in this context, its effect on substantial network metrics such as reliability, average transmission delay, and the effective sum rate (ESR) under finite blocklength transmissions are mathematically computed, taking by considering the detection probability of the jammer. Furthermore, the effect of UEs' allocated power and blocklength on the network metrics is explored. Contrary to the existing literature, results show that gNB can mitigate the impact of reactive jamming by decreasing transmit power, making the transmissions covert at the jammer side. Finally, an optimization problem is formulated to maximize the ESR under reliability, delay, and transmit power constraints. It is shown that by adjusting the allocated transmit power to UEs by gNB, the gNB can bypass the jammer effect to fulfill the 0.99999 reliability and the latency of 5ms without the need for packet re-transmission.

Radio access technology is crucial in both 5G and 6G cellular networks, providing differentiated services that demand reliability, low latency, and high throughput. To meet these requirements, machine learning (ML) has demonstrated considerable progress by facilitating resource allocation. However, these ML techniques can be susceptible to attacks, and the jamming attack is one of the most considered attacks in the literature, disrupting network functionality by sending interference signals. This paper, to the best of our knowledge for the first time, examines the vulnerability of radio access networks (RANs) to jamming attacks on resource allocation of a transfer reinforcement learning (TRL) based system and provides a mitigation approach to such attacks. A system model is presented for RAN slicing, followed by an introduction of the TRL algorithm for resource allocation. Afterward, we investigate covert patterned jamming attack (CPJA) on the TRL algorithm in downlink communication which decreases system throughput by 17% and 38.14% in the expert and learner agents and increases latency by 7.36% and 9.37% respectively. In addition, we propose a neural network (NN) solution to mitigate the CPJA trained on the network side and provide the trained NN model to the users' equipment (UEs) to eliminate interference from the signal by the filter. The trained NN is applied to predict the future activity of the interference generated by the attacker. Attack mitigation reduces the impact of the attack while the system's throughput suffers a 6% and 1.8% degradation, and its latency increases by 6.5% and 3.83% compared to the original system for expert and learner agents, respectively.

Federated learning (FL) allows distributed partic-ipants to train machine learning models in a decentralized manner. It can be used for radio signal classification with multiple receivers due to its benefits in terms of privacy and scalability. However, the existing FL algorithms usually suffer from slow and unstable convergence and are vulnerable to poisoning attacks from malicious participants. In this work, we aim to design a versatile FL framework that simultaneously promotes the performance of the model both in a secure system and under attack. To this end, we leverage attention mechanisms as a defense against attacks in FL and propose a robust FL algorithm by integrating the attention mechanisms into the global model aggregation step. To be more specific, two attention models are combined to calculate the amount of attention cast on each participant. It will then be used to determine the weights of local models during the global aggregation. The proposed algorithm is verified on a real-world dataset and it outperforms existing algorithms, both in secure systems and in systems under data poisoning attacks.

Evaluating service connectivity of OpenWiFi to endusers calls for meticulous analysis of traffic patterns to identify any unusual behavior. This work proposes an automated approach to annotate the partially labeled OpenWiFi data more effectively, enabling discrimination of anomalous behavior from normal traffic behavior. Our framework comprises feature selection, semi-supervised learning (SSL), deep semi-supervised transfer learning (DSSTL), and K-Means clustering analysis for forecasting pseudo labels on unlabeled data, utilizing limited labeled information to extract relevant Key Performance Indicators (KPIs). We utilized one-dimensional convolutional neural network (1D-CNN) and Long Short-Term Memory (LSTM), enhanced by semi-supervised learning (SSL) and unsupervised K-Means to generate pseudo labels as anomaly or normal for unlabeled data. DSSTL involves domain adaptation by combining SSL, transfer learning, and K-Means clustering algorithm, thereby generalizing the forecasting quality of pseudo labels over conventional SSL and clustering approach. Our findings showcase DSSTL-based 1D-CNN with Mean Square Error (MSE) loss outperforms the analysis for SSL and DSSTL-based LSTM in acquiring pseudo labels for all proportions of unlabeled data, as indicated by Calinski and Harabasz (CH) score. It is noticeable that DSSTL-based 1D-CNN with MSE achieves a CH score that increases by 94% corresponding to 25% over 30% unlabeled samples, regardless of the increasing trend in CH score with an increase in proportions of unlabeled samples.

Extended Reality (XR) services will revolutionize applications over $5^{\text {th }}$ and $\mathbf{6}^{\text {th }}$ generation wireless networks by providing seamless virtual and augmented reality experiences. These applications impose significant challenges on network infrastructure, which can be addressed by machine learning algorithms due to their adaptability. This paper presents a Multi-Agent Reinforcement Learning (MARL) solution for optimizing codec parameters of XR traffic, comparing it to the Adjust Packet Size (APS) algorithm. Our cooperative multi-agent system uses an Optimistic Mixture of Q-Values ($\mathbf{O Q M I X}$) approach for handling Cloud Gaming (CG), Augmented Reality (AR), and Virtual Reality (VR) traffic. Enhancements include an attention mechanism and slate-Markov Decision Process (MDP) for improved action selection. Simulations show our solution outperforms APS with average gains of $30.1 \%, 15.6 \%, 16.5 \% 50.3 \%$ in XR index, jitter, delay, and Packet Loss Ratio (PLR), respectively. APS tends to increase throughput but also packet losses, whereas oQMIX reduces PLR, delay, and jitter while maintaining goodput.

In precision farming, timely information is vital for effective decision-making in irrigation and pest control processes. Unmanned Aerial Vehicles (UAVs) and Multi-access Edge Computing (MEC) servers optimize data collection from ground sensors. However, integrating sensors, controllers, and actuators complicates data freshness in closed-loop communication. Computational resource optimization is also crucial, especially in energy-constrained equipment and competitive resource scenarios. This work optimizes data freshness and task turnaround time (TAT) with a UAV trajectory and MEC offloading strategy. Analysis includes assessing delays in uplink, downlink, and queues. This gains significance under dynamic network conditions and variable resources. A hierarchical deep reinforcement approach is proposed. Numerical results indicate the proposed solution outperforms baselines, improving data freshness by up to $31 \%$ and simultaneously decreasing TAT by $34 \%$.

The Open Radio Access Network (Open RAN) vision is based on the three principles of (i) open interfaces; (ii) cloudification; and (iii) automation through closed-loop control. It is a network architecture paradigm embodied and augmented through technical reference specifications of the 3GPP and the O-RAN Alliance. At the centre of Open RAN are open, programmable, and virtualized components, connected to each other through open interfaces that enable closed-loop, data-driven, and intelligent control. For instance, the O-RAN Alliance introduced two RAN Intelligent Controllers (or RICs) that connect through open interfaces to the disaggregated components of the RAN, and implement control loops that run at different time scales.

Innovation and standardization in 5G have brought advancements to every facet of the cellular architecture. This ranges from the introduction of new frequency bands and signaling technologies for the radio access network (RAN), to a core network underpinned by micro-services and network function virtualization (NFV). However, like any emerging technology, the pace of real-world deployments does not instantly match the pace of innovation. To address this discrepancy, one of the key aspects under continuous development is the RAN with the aim of making it more open, adaptive, functional, and easy to manage. In this paper, we highlight the transformative potential of embracingnovel cellular architecturesby transitioning from conventional systems to the progressive principles of Open RAN. This promises to make 6G networks more agile, cost-effective, energy-efficient, and resilient. It opens up a plethora of novel use cases, ranging from ubiquitous support for autonomous devices to cost-effective expansions in regions previously underserved. The principles of Open RAN encompass: (i) a disaggregated architecture with modular and standardized interfaces; (ii) cloudification, programmability and orchestration; and (iii) AI-enabled data-centric closed-loop control and automation. We first discuss the transformative role Open RAN principles have played in the 5G era. Then, we adopt a system-level approach and describe how these Open RAN principles will support 6G RAN and architecture innovation. We qualitatively discuss potential performance gains that Open RAN principles yield for specific 6G use cases. For each principle, we outline the steps that research, development and standardization communities ought to take to make Open RAN principles central to next-generation cellular network designs.

Edge computing offers a promising solution to enhance network reliability. In this study, we investigate the integration of mobile edge computing (MEC) technology and unmanned aerial vehicles (UAVs) within the context of smart agriculture. Smart agriculture relies on resource-constrained Internet of Things (IoT) devices for local environmental monitoring and data collection. These IoT devices send the collected data to UAVs for analysis. A central theme of this work is the focus on the applications generated by each UAV and the consideration of their topology to derive our optimization algorithm. To tackle these challenges, we propose harnessing the computational and power resources of UAVs and MEC at the network’s edge to offload and execute resource-intensive tasks in UAV-MEC-assisted networks. Our research focuses on the joint optimization of power allocation and task offloading in these wireless networks. Central to our investigation is the problem of minimizing the energy-time cost (ETC) for the UAVs, considering the interdependencies among tasks. To address this complex problem efficiently, we introduce graph convolutional neural networks (GCNs) and reinforcement learning (RL)-based techniques. We employ a directed acyclic graph (DAG) to model task interdependencies, with GCNs characterizing the DAG. Our approach incorporates an actor-critic method with embedding layers, trained using the compound-action actor-critic (CA2C) algorithm. Our findings reveal a significant improvement in minimizing both delay and energy consumption, with a 27% percent reduction in delay and a 45% reduction in consumed energy for executing complex, interdependent tasks.

Sentiment Analysis Systems (SASs) are data-driven Artificial Intelligence (AI) systems that assign one or more numbers to convey the polarity and emotional intensity of a given piece of text. However, like other automatic machine learning systems, SASs can exhibit model uncertainty, resulting in drastic swings in output with even small changes in input. This issue becomes more problematic when inputs involve protected attributes like gender or race, as it can be perceived as bias or unfairness. To address this, we propose a novel method to assess and rate SASs. We perturb inputs in a controlled causal setting to test if the output sentiment is sensitive to protected attributes while keeping other components of the textual input, such as chosen emotion words, fixed. Based on the results, we assign labels (ratings) at both fine-grained and overall levels to indicate the robustness of the SAS to input changes. The ratings can help decision-makers improve online content by reducing hate speech, often fueled by biases related to protected attributes such as gender and race. These ratings provide a principled basis for comparing SASs and making informed choices based on their behavior. The ratings also benefit all users, especially developers who reuse off-the-shelf SASs to build larger AI systems but do not have access to their code or training data to compare.

In this paper, we propose a novel cooperative resource sharing technique in multi-tier edge slicing networks which is robust to imperfect channel state information (CSI) caused by user equipments’ (UEs) mobility. Due to the mobility of UEs, the dynamic requirements of their tasks, and the limited resources of the network, we propose a smart joint dynamic pricing and resources sharing (SJDPRS) scheme that can incentivize the infrastructure provider (InP) and mobile network operators (MNOs). Aiming to maximize the profits of UEs, MNOs and the InP under the task fulfillment constraints, we formulate an optimization problem by deploying the multi-objective optimization method where in addition to the resource allocation variables, the price values are also the optimization variables. To solve the problem, we adopt a new deep reinforcement learning (DRL) method based on a carefully designed reward function. The simulation results indicate that the proposed resource sharing scenario can increase total profits for the UEs, MNOs, and InP in comparison to non-cooperative case, while also providing almost complete fairness among the players. In particular, as compared to the baselines and benchmarks, the profits for each network component (MNO, InP, and UEs), under fairness considerations, are enhanced by 75%, 79%, and 76%, respectively.

Network slicing envisions the 5th generation (5G) mobile network resource allocation to be based on different requirements for different services, such as Ultra-Reliable Low Latency Communication (URLLC) and Enhanced Mobile Broadband (eMBB). Open Radio Access Network (O-RAN), proposes an open and disaggregated concept of RAN by modulizing the functionalities into independent components. Network slicing for O-RAN can significantly improve performance. Therefore, an advanced resource allocation solution for network slicing in O-RAN is proposed in this study by applying Reinforcement Learning (RL). This research demonstrates an RL compatible simplified edge network simulator with three components, user equipment(UE), Edge O-Cloud, and Regional O-Cloud. This simulator is later used to discover how to improve throughput for targeted network slice(s) by dynamically allocating unused bandwidth from other slices. Increasing the throughput for certain network slicing can also benefit the end users with a higher average data rate, peak rate, or shorter transmission time. The results show that the RL model can provide eMBB traffic with a high peak rate and shorter transmission time for URLLC compared to balanced and eMBB focus baselines.

In 5G non-standalone mode, traffic steering is a critical technique to take full advantage of 5G new radio while optimizing dual connectivity of 5G and LTE networks in multiple radio access technology (RAT). An intelligent traffic steering mechanism can play an important role to maintain seamless user experience by choosing appropriate RAT (5G or LTE) dynamically for a specific user traffic flow with certain QoS requirements. In this paper, we propose a novel traffic steering mechanism based on Deep Q-learning that can automate traffic steering decisions in a dynamic environment having multiple RATs, and maintain diverse QoS requirements for different traffic classes. The proposed method is compared with two baseline algorithms: a heuristic-based algorithm and Q-learning-based traffic steering. Compared to the Q-learning and heuristic baselines, our results show that the proposed algorithm achieves better performance in terms of 6% and 10% higher average system throughput, and 23% and 33% lower network delay, respectively.

Proper and efficient beam selection is of great importance to unleash the full potential of mmWave communications. Traditionally, each candidate beam is evaluated using reference signals (beam sweeping), however, the exhaustive search method can be time-consuming with high signaling overhead. To avoid such problems, in B5G and 6G, sensing information is considered to be used, as in Integrated Sensing and Communication (ISAC) solutions, and Machine Learning (ML) methods can be applied to map sensing data inputs to an optimal beam index. When using sensing information sources external to the Radio Access Network (RAN) in a multi-vendor disaggregated environment, those methods need to account for issues such as privacy and data ownership. In this work, we apply multi-modal sensing information to the beam selection task. Specifically, we propose a multi-modal sensing-aided ML strategy based on Split Learning (SL) that can cope with deployment challenges in novel RAN architectures. Moreover, the method is applied to single and multi-level beam selection decisions, where the latter considers the case of hierarchical codebook structures. With the proposed approach, accuracy levels above 90% can be achieved while overhead diminishes by 85% or more. SL achieves comparable performance with centralized learning-based strategies, with the added value of accounting for privacy and data ownership issues. We also show that sensing-aided ML-based beam selection decisions in multi-level codebooks are more effective when applied to their first level.

As wireless networks continue to evolve, machine learning (ML) algorithms are used to address communication challenges and meet various service requirements. While ML methods are promising, they can be prone to malicious attacks, which may degrade user experience and network performance. Specifically, the security challenges of radio access networks (RANs) are highlighted due to frequent interactions with a large number of users, and evaluating these attacks is critical for securing wireless communications. In this paper, for the first time, we investigate the vulnerability of transfer reinforcement learning (TRL) algorithms for resource allocation in 5G RAN slicing. In particular, we first present the system model for RAN slicing, and then the TRL algorithm is introduced for resource allocation. Afterward, we investigate three types of attack methods on the TRL algorithm. The simulations indicate that the attack on an expert agent can affect the performance of a learner agent since the expert shares knowledge with the learner. We show that the effect of the black-box policy poisoning attack on the learner increases latency by 18.31% and reduces throughput by 13.80%, while white-box policy poisoning attacks result in a 48.96% increase in latency and an 87.02% reduction in throughput.

The growing adoption of mmWave frequency bands to realize the full potential of 5G, turns beamforming into a key enabler for current and next-generation wireless technologies. Many mmWave networks rely on beam selection with Grid-of-Beams (GoB) approach to handle user-beam association. In beam selection with GoB, users select the appropriate beam from a set of pre-defined beams and the overhead during the beam selection process is a common challenge in this area. In this paper, we propose an Advantage Actor Critic (A2C) learning-based framework to improve the GoB and the beam selection process, as well as optimize transmission power in a mmWave network. The proposed beam selection technique allows performance improvement while considering transmission power improves Energy Efficiency (EE) and ensures the coverage is maintained in the network. We further investigate how the proposed algorithm can be deployed in a Service Management and Orchestration (SMO) platform. Our simulations show that A2C-based joint optimization of beam selection and transmission power is more effective than using Equally Spaced Beams (ESB) and fixed power strategy, or optimization of beam selection and transmission power disjointly. Compared to the ESB and fixed transmission power strategy, the proposed approach achieves more than twice the average EE in the scenarios under test and is closer to the maximum theoretical EE.

In 5G non-standalone mode, an intelligent traffic steering mechanism can vastly aid in ensuring a smooth user experience by selecting the best radio access technology (RAT) from a multi-RAT environment for a specific traffic flow. In this paper, we propose a novel load-aware traffic steering algorithm based on hierarchical reinforcement learning (HRL) while satisfying the diverse quality of service requirements of different traffic types. HRL can significantly increase system performance using a bi-level architecture having a meta-controller and a controller. In our proposed method, the meta-controller provides an appropriate threshold for load balancing, while the controller performs traffic admission to an appropriate RAT in the lower level. Simulation results show that HRL outperforms a Deep Q-Learning (DQN) and a threshold-based heuristic baseline with 8.49%, 12.52% higher average system throughput and 27.74%, 39.13% lower network delay, respectively.

Performance degradation of OpenWiFi traffic is a significant problem while provisioning service to a dense area consisting of thousands of clients operating at the same time. Among various vulnerabilities of poor performance in Wireless Local Area Networks (WLANs), deviation of a traffic pattern from normal indicates probable presence of anomaly or outlier. Adoption of machine learning algorithms including unsupervised and supervised models, the complexity of detection of a anomalous traffic along with respective root cause is untangled with augmented machine learning to involve domain knowledge input. In this paper, to cope with unlabeled data in an OpenWiFi setting, the following systematic work flow is proposed to augment machine learning-based anomaly detection. First, a combination of two unsupervised clustering algorithms is used to segregate the anomalies from normal distribution of traffic. The anomalous instances are confirmed via domain knowledge input. Next, supervised models are trained to detect anomalies in a different domain (Wireless Sensor Networks) with labeled data. Following upon feature extraction to obtain the same number of dimensions in the OpenWiFi data as the labeled dataset, trained supervised classifiers are used to detect anomalies in the OpenWiFi system. Furthermore, the impact of oversampling methods have also been investigated. Through numerical results we show the impact of the proposed supervised models in terms of decision-making metrics and the shift in performances from the standpoint of imbalanced distribution of rare-class classification problem.

A deep reinforcement learning technique is presented for task offloading decision-making algorithms for a multi-access edge computing (MEC) assisted unmanned aerial vehicle (UAV) network in a smart farm Internet of Things (IoT) environment. The task offloading technique uses financial concepts such as cost functions and conditional variable at risk (CVaR) in order to quantify the damage that may be caused by each risky action. The approach was able to quantify potential risks to train the reinforcement learning agent to avoid risky behaviors that will lead to irreversible consequences for the farm. Such consequences include an undetected fire, pest infestation, or a UAV being unusable. The proposed CVaR-based technique was compared to other deep reinforcement learning techniques and two fixed rule-based techniques. The simulation results show that the CVaR-based risk quantifying method eliminated the most dangerous risk, which was exceeding the deadline for a fire detection task. As a result, it reduced the total number of deadline violations with a negligible increase in energy consumption.

IEEE 802.11be -Extremely High Throughput-, commercially known as Wireless-Fidelity (Wi-Fi) 7 is the newest IEEE 802.11 amendment that comes to address the increasingly throughput hungry services such as Ultra High Definition (4K/8K) Video and Virtual/Augmented Reality (VR/AR). To do so, IEEE 802.11be presents a set of novel features that will boost the Wi-Fi technology to its edge. Among them, Multi-Link Operation (MLO) devices are anticipated to become a reality, leaving Single-Link Operation (SLO) Wi-Fi in the past. To achieve superior throughput and very low latency, a careful design approach must be taken, on how the incoming traffic is distributed in MLO capable devices. In this paper, we present a Reinforcement Learning (RL) algorithm named Multi-Headed Recurrent Soft-Actor Critic (MH-RSAC) to distribute incoming traffic in 802.11be MLO capable networks. Moreover, we compare our results with two non-RL baselines previously proposed in the literature named: Single Link Less Congested Interface (SLCI) and Multi-Link Congestion-aware Load balancing at flow arrivals (MCAA). Simulation results reveal that the MH-RSAC algorithm is able to obtain gains in terms of Throughput Drop Ratio (TDR) up to 35.2% and 6% when compared with the SLCI and MCAA algorithms, respectively. Finally, we observed that our scheme is able to respond more efficiently to high throughput and dynamic traffic such as VR and Web Browsing (WB) when compared with the baselines. Results showed an improvement of the MH-RSAC scheme in terms of Flow Satisfaction (FS) of up to 25.6% and 6% over the the SCLI and MCAA algorithms.

Dense and unplanned IEEE 802.11 Wireless Fidelity (Wi-Fi) deployments and the continuous increase of throughput and latency stringent services for users have led to machine learning algorithms to be considered as promising techniques in the industry and the academia. Specifically, the ongoing IEEE 802.11be EHT —Extremely High Throughput, known as Wi-Fi 7— amendment propose, for the first time, Multi-Link Operation (MLO). Among others, this new feature will increase the complexity of channel selection due the novel multiple interfaces proposal. In this paper, we present a Parallel Transfer Reinforcement Learning (PTRL)-based cooperative Multi-Agent Reinforcement Learning (MARL) algorithm named Parallel Transfer Reinforcement Learning Optimistic-Weighted Value Decomposition Networks (oVDN) to improve intelligent channel selection in IEEE 802.11be MLO-capable networks. Additionally, we compare the impact of different parallel transfer learning alternatives and a centralized non-transfer MARL baseline. Two PTRL methods are presented: Multi-Agent System (MAS) Joint Q-function Transfer, where the joint Q-function is transferred and MAS Best/Worst Experience Transfer where the best and worst experiences are transferred among MASs. Simulation results show that oVDNg–only the best experiences are utilized– is the best algorithm variant. Moreover, oVDNgoffers a gain up to 3%, 7.2% and 11% when compared with VDN, VDN-nonQ and non-PTRL baselines. Furthermore, oVDNgexperienced a reward convergence gain in the 5 GHz interface of 33.3% over oVDNband oVDN where only worst and both types of experiences are considered, respectively. Finally, our best PTRL alternative showed an improvement over the non-PTRL baseline in terms of speed of convergence up to 40 episodes and reward up to 135%.

Using Machine Learning (ML) techniques for the next generation wireless networks have shown promising results in the recent years, due to high learning and adaptation capability of ML algorithms. More specifically, ML techniques have been used for load balancing in Self-Organizing Networks (SON). In the context of load balancing and ML, several studies propose network management automation (NMA) from the perspective of a single and centralized agent. However, a single agent domain does not consider the interaction among the agents. In this paper, we propose a more realistic load balancing approach using novel Multi-Agent Deep Deterministic Policy Gradient with Adaptive Policies (MADDPG-AP) scheme that considers throughput, resource block utilization and latency in the network. We compare our proposal with a single-agent RL algorithm named Clipped Double Q-Learning (CDQL) . Simulation results reveal a significant improvement in latency, packet loss ratio and convergence time.

To implement network slicing in 5G, resource allocation is a key function to allocate limited network resources such as radio and computation resources to multiple slices. However, the joint resource allocation also leads to a higher complexity in the network management. In this work, we propose a knowledge transfer based resource allocation (KTRA) method to jointly allocate radio and computation resources for 5G RAN slicing. Compared with existing works, the main difference is that the proposed KTRA method has a knowledge transfer capability. It is designed to use the prior knowledge of similar tasks to improve performance of the target task, e.g., faster convergence speed or higher average reward. The proposed KTRA is compared with Q-learning based resource allocation (QLRA), and KTRA method presents a 18.4% lower URLLC delay and a 30.1% higher eMBB throughput as well as a faster convergence speed.

5G New Radio proposes the usage of frequencies above 10 GHz to speed up LTE's existent maximum data rates. However, the effective size of 5G antennas and consequently its repercussions in the signal degradation in urban scenarios makes it a challenge to maintain stable coverage and connectivity. In order to obtain the best from both technologies, recent dual connectivity solutions have proved their capabilities to improve performance when compared with coexistent standalone 5G and 4G technologies. Reinforcement learning (RL) has shown its huge potential in wireless scenarios where parameter learning is required given the dynamic nature of such context. In this paper, we propose two reinforcement learning algorithms: a single agent RL algorithm named Clipped Double Q-Learning (CDQL) and a hierarchical Deep Q-Learning (HiDQL) to improve Multiple Radio Access Technology (multi-RAT) dual-connectivity handover. We compare our proposal with two baselines: a fixed parameter and a dynamic parameter solution. Simulation results reveal significant improvements in terms of latency with a gain of 47.6% and 26.1% for Digital-Analog beamforming (BF), 17.1% and 21.6% for Hybrid-Analog BF, and 24.7% and 39% for Analog-Analog BF when comparing the RL-schemes HiDQL and CDQL with the with the existent solutions, HiDQL presented a slower convergence time, however obtained a more optimal solution than CDQL. Additionally, we foresee the advantages of utilizing context-information as geo-location of the UEs to reduce the beam exploration sector, and thus improving further multi-RAT handover latency results.

User location is a piece of critical information for network management and control. However, location uncertainty is unavoidable in certain settings leading to localization errors. In this paper, we consider the user location uncertainty in the mmWave networks, and investigate joint vision-aided sensing and communications using deep reinforcement learning-based beam management for future 6G networks. In particular, we first extract pixel characteristic-based features from satellite images to improve localization accuracy. Then we propose a UK-medoids based method for user clustering with location uncertainty, and the clustering results are consequently used for the beam management. Finally, we apply the DRL algorithm for intra-beam radio resource allocation. The simulations first show that our proposed vision-aided method can substantially reduce the localization error. The proposed UK-medoids and DRL based scheme (UKM-DRL) is compared with two other schemes: K-means based clustering and DRL based resource allocation (K-DRL) and UK-means based clustering and DRL based resource allocation (UK-DRL). The proposed method has 17.2% higher throughput and 7.7% lower delay than UK-DRL, and more than doubled throughput and 55.8% lower delay than K-DRL.

Recently, open radio access network (O-RAN) has become a promising technology to provide an open environment for network vendors and operators. Coordinating the x-applications (xAPPs) is critical to increase flexibility and guarantee high overall network performance in O-RAN. Meanwhile, federated reinforcement learning has been proposed as a promising technique to enhance the collaboration among distributed reinforcement learning agents and improve learning efficiency. In this paper, we propose a federated deep reinforcement learning algorithm to coordinate multiple independent xAPPs in O-RAN for network slicing. We design two xAPPs, namely a power control xAPP and a slice-based resource allocation xAPP, and we use a federated learning model to coordinate two xAPP agents to enhance learning efficiency and improve network performance. Compared with conventional deep reinforcement learning, our proposed algorithm can achieve 11% higher throughput for enhanced mobile broadband (eMBB) slices and 33% lower delay for ultra-reliable low-latency communication (URLLC) slices.

Reconfigurable intelligent surface (RIS) is emerging as a promising technology to boost the energy efficiency (EE) of 5G beyond and 6G networks. Inspired by this potential, in this paper, we investigate the RIS-assisted energy-efficient radio access networks (RAN). In particular, we combine RIS with sleep control techniques, and develop a hierarchical reinforcement learning (HRL) algorithm for network management. In HRL, the meta-controller decides the on/off status of the small base stations (SBSs) in heterogeneous networks, while the sub-controller can change the transmission power levels of SBSs to save energy. The simulations show that the RIS-assisted sleep control can achieve significantly lower energy consumption, higher throughput, and more than doubled energy efficiency than no- RIS conditions.

Unmanned aerial vehicles (UAVs) with mounted base stations are a promising technology for monitoring smart farms. They can provide communication and computation services to extensive agricultural regions. With the assistance of a Multi-Access Edge Computing infrastructure, an aerial base station (ABS) network can provide an energy-efficient solution for smart farms that need to process deadline critical tasks fed by IoT devices deployed on the field. In this paper, we introduce a multi-objective maximization problem and a Q-Learning based method which aim to process these tasks before their deadline while considering the UAVs’ hover time. We also present three heuristic baselines to evaluate the performance of our approaches. In addition, we introduce an integer linear programming (ILP) model to define the upper bound of our objective function. The results show that Q-Learning outperforms the baselines in terms of remaining energy levels and percentage of delay violations.

Data is the fuel of data science and machine learning techniques for smart grid applications, similar to many other fields. However, the availability of data can be an issue due to privacy concerns, data size, data quality, and so on. To this end, in this paper, we propose a Variational AutoEncoder Generative Adversarial Network (VAE-GAN) as a smart grid data generative model which is capable of learning various types of data distributions and generating plausible samples from the same distribution without performing any prior analysis on the data before the training phase. We compared the Kullback–Leibler (KL) divergence, maximum mean discrepancy (MMD), and Wasserstein distance between the synthetic data (electrical load and PV production) distribution generated by the proposed model, vanilla GAN network, and the real data distribution, to evaluate the performance of our model. Furthermore, we used five key statistical parameters to describe the smart grid data distribution and compared them between synthetic data generated by both models and real data. Experiments indicate that the proposed synthetic data generative model outperforms the vanilla GAN network. The distribution of VAE-GAN synthetic data is the most comparable to that of real data.

Recently, the concept of open radio access network (O-RAN) has been proposed, which aims to adopt intelligence and openness in the next generation radio access networks (RAN). It provides standardized interfaces and the ability to host network applications from third-party vendors by x-applications (xAPPs), which enables higher flexibility for network management. However, this may lead to conflicts in network function implementations, especially when these functions are implemented by different vendors. In this paper, we aim to mitigate the conflicts between xAPPs for near-real-time (near-RT) radio intelligent controller (RIC) of O-RAN. In particular, we propose a team learning algorithm to enhance the performance of the network by increasing cooperation between xAPPs. We compare the team learning approach with independent deep Q-learning where network functions individually optimize resources. Our simulations show that team learning has better network performance under various user mobility and traffic loads. With 6 Mbps traffic load and 20 m/s user movement speed, team learning achieves 8% higher throughput and 64.8% lower PDR.

Millimeter Wave (mmWave) is an important part of 5G new radio (NR), in which highly directional beams are adapted to compensate for the substantial propagation loss based on UE locations. However, the location information may have some errors such as GPS errors. In any case, some uncertainty, and localization error is unavoidable in most settings. Applying these distorted locations for clustering will increase the error of beam management. Meanwhile, the traffic demand may change dynamically in the wireless environment. Therefore, a scheme that can handle both the uncertainty of localization and dynamic radio resource allocation is needed. In this paper, we propose a UK-means-based clustering and deep reinforcement learning-based resource allocation algorithm (UK-DRL) for radio resource allocation and beam management in 5G mm Wave networks. We first apply UK-means as the clustering algorithm to mitigate the localization uncertainty, then deep reinforcement learning (DRL) is adopted to dynamically allocate radio resources. Finally, we compare the UK-DRL with K-means-based clustering and DRL-based resource allocation algorithm (K-DRL), the simulations show that our proposed UK-DRL-based method achieves 150% higher throughput and 61.5% lower delay compared with K-DRL when traffic load is 4Mbps.

Reconfigurable Intelligent Surfaces (RIS) are planar structures connected to electronic circuitry, which can be employed to steer the electromagnetic signals in a controlled manner. Through this, the signal quality and the effective data rate can be substantially improved. While the benefits of RIS-assisted wireless communications have been investigated for various scenarios, some aspects of the network design, such as coverage, optimal placement of RIS, etc., often require complex optimization and numerical simulations, since the achievable effective rate is difficult to predict. This problem becomes even more difficult in the presence of phase estimation errors or location uncertainty, which can lead to substantial performance degradation if neglected. Considering randomly distributed receivers within a ring-shaped RIS-assisted wireless network, this paper mainly investigates the effective rate by taking into account the above-mentioned impairments. Furthermore, exact closed-form expressions for the effective rate are derived in terms of Meijer’s G-function, which (i) reveals that the location and phase estimation uncertainty should be well considered in the deployment of RIS in wireless networks; and (ii) facilitates future network design and performance prediction.

Microgrids (MGs) are important players for the future transactive energy systems where a number of intelligent Internet of Things (IoT) devices interact for energy management in the smart grid. Although there have been many works on MG energy management, most studies assume a perfect communication environment, where communication failures are not considered. In this article, we consider the MG as a multiagent environment with IoT devices in which AI agents exchange information with their peers for collaboration. However, the collaboration information may be lost due to communication failures or packet loss. Such events may affect the operation of the whole MG. To this end, we propose a multiagent Bayesian deep reinforcement learning (BA-DRL) method for MG energy management under communication failures. We first define a multiagent partially observable Markov decision process (MA-POMDP) to describe agents under communication failures, in which each agent can update its beliefs on the actions of its peers. Then, we apply a double deep$Q$-learning (DDQN) architecture for$Q$-value estimation in BA-DRL, and propose a belief-based correlated equilibrium for the joint-action selection of multiagent BA-DRL. Finally, the simulation results show that BA-DRL is robust to both power supply uncertainty and communication failure uncertainty. BA-DRL has 4.1% and 10.3% higher reward than Nash deep$Q$-learning (Nash-DQN) and alternating direction method of multipliers (ADMM), respectively, under 1% communication failure probability.

As one of the key technologies in 5G networks, Carrier Aggregation (CA) is studied in this paper. In CA, Component Carriers (CCs) can be activated and deactivated depending on multiple factors, e.g., energy consumption and Quality of Service (QoS) demand of users. We propose CC management strategies where each User Equipment (UE) minimizes its average delay and at the same time minimizes its power consumption while considering that CCs can be activated and deactivated only at certain times, as in real-world CA implementations. We first model the problem as a centralized multi-objective optimum CC management problem. Since centralized approaches would impose a large overhead on the system, we then develop a semi-distributed solution by modeling the problem as a stochastic game and propose a multi-agent Double Deep Q-Network (DDQN) based CC management algorithm to solve the stochastic game. We finally compare the proposed approaches with single CC activation and all-CC activation baseline schemes. Simulation results show that our proposed algorithms outperform the all-CC algorithm in terms of UE power consumption and have the capability of transmitting a number of bits with delay close to the all-CC scheme. Meanwhile, our DDQN-based algorithm decreases the UE power consumption by about 20% with respect to the all-CC scheme.

Open Radio Access Networks (O-RAN) architecture is based on disaggregation, virtualization, openness, and intelligence. These features allow the RAN network functions (NFs) to be split into Central Unit (CU), Distributed Unit (DU), and Radio Unit (RU); and deployed on open hardware and cloud nodes as Virtualized Network Functions (VNFs) or Containerized Network Functions (CNFs). In this paper, we propose strategies for the placement of CU and DU network functions in the regional and edge O-Cloud nodes while jointly associating the users to RUs. The aim is to minimize the end-to-end delay of users and minimize the cost of O-RAN deployment. Thus, we first formulate the end-to-end delay, the cost, and the constraints. We then model the problem as a multi-objective optimization problem The optimization formulation consists of a huge number of constraints and variables. To provide a solution to the problem, we develop the corresponding Markov Decision Problem (MDP) and propose a Deep Q-Network (DQN)-based algorithm. The simulation results demonstrate that our proposed scheme reduces the average user delay up to 40% and the deployment cost up to 20% with respect to our baselines.

Carrier aggregation (CA) is considered a key enabler technology for delivering higher rates to users of LTE and 5G networks. However, the increased transmission rate comes with the price of higher energy consumption which stems from users continuously monitoring the control channel of the active component carriers (CCs) whether data transmission is ongoing or not. In order to reduce energy consumption, we exploit the activation-deactivation procedure at the medium access control (MAC) layer of LTE/5G network. In this paper, we propose a reinforcement learning-based algorithm to improve energy-efficiency by dynamically activating-deactivating secondary component carriers (SCCs) with awareness of the user traffic profiles. The proposed algorithm aims to predict the arrival of data and identify SCCs to activate for each user. In addition, a traffic splitting approach and an intelligent exploration strategy are proposed to balance users' load among CCs and improve the convergence of the algorithm, respectively. Results of the proposed algorithm are compared with three baseline algorithms. The first baseline always activates all CCs for each user, the second baseline activates one carrier only (i.e., the primary carrier) and the third baseline algorithm relies on a reactive method, where the activation-deactivation decision is performed after observing the arrival of data. Results show that Q-learning outperforms the baseline algorithms by achieving the highest sum throughput (and lowest average delay) with the lowest number of activated SCCs, which is obtained by learning to dynamically activate SCCs according to the traffic pattern. Hence, Q-learning is considered the most energy-efficient compared to the baseline algorithms.

Recently, there has been tremendous efforts by network operators and equipment vendors to adopt intelligence and openness in the next generation radio access network (RAN). The goal is to reach a RAN that can self-optimize in a highly complex setting with multiple platforms, technologies and vendors in a converged compute and connect architecture. In this paper, we propose two nested actor-critic learning based techniques to optimize the placement of resource allocation function, and as well, the decisions for resource allocation. By this, we investigate the impact of observability on the performance of the reinforcement learning based resource allocation. We show that when a network function (NF) is dynamically relocated based on service requirements, using reinforcement learning techniques, latency and throughput gains are obtained.

A smart home energy management system (HEMS) can contribute towards reducing the energy costs of customers; however, HEMS suffers from uncertainty in both energy generation and consumption patterns. In this paper, we propose a sequence to sequence (Seq2Seq) learning-based supply and load prediction along with reinforcement learning-based HEMS control. We investigate how the prediction method affects the HEMS operation. First, we use Seq2Seq learning to predict photovoltaic (PV) power and home devices' load. We then apply Q-learning for offline optimization of HEMS based on the prediction results. Finally, we test the online performance of the trained Q-learning scheme with actual PV and load data. The Seq2Seq learning is compared with VARMA, SVR, and LSTM in both prediction and operation levels. The simulation results show that Seq2Seq performs better with a lower prediction error and online operation performance.

Carrier Aggregation (CA) has been a breakthrough in LTE that led to increased throughput for users, and is still one of the key technologies in 5G that helps to enhance spectrum utilization. In CA, Component Carriers (CCs) are dynamically activated and deactivated depending on several performance factors. Optimal selection of CCs has been studied in the literature. However, the latency associated with activation and deactivation of CCs, control channel overhead for switching CCs, as well as the energy consumed for monitoring the active CCs have not been a part of the optimal CC selection problem. Nevertheless, those become stringent design constraints in practice. In this paper, we address optimal CC selection and resource allocation in 5G networks, where the above constraints are considered and the 5G network supports several service types with different 5G QoS Identifiers (5QI). The proposed optimum joint CC selection and Radio Resource Block (RB) allocation schemes maximize average throughput of users and satisfy QoS of users in terms of delay. In addition, the proposed schemes take CC activation and deactivation burden into consideration and aim to minimize the number of activations and deactivations. The simulation results demonstrate that our proposed solution outperforms the state of the art solution while satisfying the QoS requirements and creating close to 95.5% reduction on the number of CCs activations and deactivations.

With the growing momentum around Open RAN (O-RAN) initiatives, performing dynamic Function Splitting (FS) in disaggregated and virtualized Radio Access Networks (vRANs), in an efficient way, is becoming highly important. An equally important efficiency demand is emerging from the energy consumption dimension of the RAN hardware and software. Supplying the RAN with Renewable Energy Sources (RESs) promises to boost the energy-efficiency. Yet, FS in such a dynamic setting, calls for intelligent mechanisms that can adapt to the varying conditions of the RES supply and the traffic load on the mobile network. In this paper, we propose a reinforcement learning (RL)based dynamic function splitting (RLDFS) technique that decides on the function splits in an O-RAN to make the best use of RES supply and minimize operator costs. We also formulate an operational expenditure minimization problem. We evaluate the performance of the proposed approach on a real data set of solar irradiation and traffic rate variations. Our results show that the proposed RLDFS method makes effective use of RES and reduces the cost of an MNO. We also investigate the impact of the size of solar panels and batteries which may guide MNOs to decide on proper RES and battery sizing for their networks.

Appliance-level load forecasting plays a critical role in residential energy management, besides having significant importance for ancillary services performed by the utilities. In this paper, we propose to use an LSTM-based sequence-to-sequence (seq2seq) learning model that can capture the load profiles of appliances. We use a real dataset collected from four residential buildings and compare our proposed scheme with three other techniques, namely VARMA, Dilated One Dimensional Convolutional Neural Network, and an LSTM model. The results show that the proposed LSTM-based seq2seq model outperforms other techniques in terms of prediction error in most cases.

Demand for localization has been growing due to the increase in location-based services and high bandwidth applications requiring precise localization of users to improve resource management and beam forming. Outdoor localization has been traditionally done through Global Positioning System (GPS), however its performance degrades in urban settings due to obstruction and multi-path effects, creating the need for better localization techniques. We propose a technique using a cascaded approach composed of image classification and regional convolutional neural networks (CNN s) using LIDAR and received signal strength (RSS) data to predict the location of moving vehicles outdoors. We use simulated data in the mm Wave band that takes place in the neighbourhood of Rosslyn in Arlington, Virginia. Our results show an improvement in localization accuracy as a result of the hierarchical architecture, with a mean absolute error (MAE) of 6.55m for the proposed technique in comparison to a MAE of 9.82m using one CNN.

In recent years, long-term evolution (LTE) and 5G NR (5thGeneration New Radio) technologies have showed great potential to utilize Machine Learning (ML) algorithms in optimizing their operations, both thanks to the availability of fine-grained data from the field, as well as the need arising from growing complexity of networks. The aforementioned complexity sparked mobile operators’ attention as a way to reduce the capital expenditures (CAPEX) and the operational (OPEX) expenditures of their networks through network management automation (NMA). NMA falls under the umbrella of Self-Organizing Networks (SON) in which 3GPP has identified some challenges and opportunities in load balancing mechanisms for the Radio Access Networks (RANs). In the context of machine learning and load balancing, several studies have focused on maximizing the overall network throughput or the resource block utilization (RBU). In this paper, we propose a novel Clipped Double Q-Learning (CDQL)-based load balancing approach considering resource block utilization, latency and the Channel Quality Indicator (CQI). We compare our proposal with a traditional handover algorithm and a resource block utilization based handover mechanism. Simulation results reveal that our scheme is able to improve throughput, latency, jitter and packet loss ratio in comparison to the baseline algorithms.

5G is regarded as a revolutionary mobile network, which is expected to satisfy a vast number of novel services, ranging from remote health care to smart cities. However, heterogeneous Quality of Service (QoS) requirements of different services and limited spectrum make the radio resource allocation a challenging problem in 5G. In this paper, we propose a multi-agent reinforcement learning (MARL) method for radio resource slicing in 5G. We model each slice as an intelligent agent that competes for limited radio resources, and the correlated Q-learning is applied for inter-slice resource block (RB) allocation. The proposed correlated Q-learning based inter-slice RB allocation (COQRA) scheme is compared with Nash Q-learning (NQL), Latency-Reliability-Throughput Q-learning (LRTQ) methods, and the priority proportional fairness (PPF) algorithm. Our simulation results show that the proposed CO-QRA achieves 32.4% lower latency and 6.3% higher throughput when compared with LRTQ, and 5.8% lower latency and 5.9% higher throughput than NQL. Significantly higher throughput and lower packet drop rate (PDR) is observed in comparison to PPF.

In delay-sensitive industrial Internet of Things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such problems. In this paper, we first utilize single agent low-complex compound action actor-critic RL to cover both discrete and continuous actions and jointly minimize VNF cost and AoI in terms of network resources under end-to-end Quality of Service constraints. To surmount the single-agent capacity limitation for learning, we then extend our solution to a multi-agent DRL scheme in which agents collaborate with each other. Simulation results demonstrate that single-agent schemes significantly outperform the greedy algorithm in terms of average network cost and AoI. Moreover, multi-agent solution decreases the average cost by dividing the tasks between the agents. However, it needs more iterations to be learned due to the requirement on the agents' collaboration.

In this paper, we aim at interference mitigation in 5G millimeter-Wave (mm-Wave) communications by employing beamforming and Non-Orthogonal Multiple Access (NOMA) techniques with the aim of improving network's aggregate rate. Despite the potential capacity gains of mm-Wave and NOMA, many technical challenges might hinder that performance gain. In particular, the performance of Successive Interference Cancellation (SIC) diminishes rapidly as the number of users increases per beam, which leads to higher intra-beam interference. Furthermore, intersection regions between adjacent cells give rise to inter-beam inter-cell interference. To mitigate both interference levels, optimal selection of the number of beams in addition to best allocation of users to those beams is essential. In this paper, we address the problem of joint user-cell association and selection of number of beams for the purpose of maximizing the aggregate network capacity. We propose three machine learning-based algorithms; transfer Q-learning (TQL), Q-learning, and Best SINR association with Density-based Spatial Clustering of Applications with Noise (BSDC) algorithms and compare their performance under different scenarios. Under mobility, TQL and Q-learning demonstrate 12% rate improvement over BSDC at the highest offered traffic load. For stationary scenarios, Q-learning and BSDC outperform TQL, however TQL achieves about 29% convergence speedup compared to Q-learning.

Social networking research has primarily focused on public social networking services and applications, while rich social interactions in an enterprise setting and their related context has received less attention. In this paper, we focus on using the enterprise social context to augment traditional authentication tools. This is motivated by the emergence of smart mobile devices which introduce ease of remote access to work from almost anywhere and anytime, adding spatio-temporal dimension to the social context. However, it remains a challenge to efficiently manage access-controlled events by using different contextual properties. This paper analyzes specific actions under specific access-control rules to extract context-aware machine learning predictions. Such analysis includes the introduction of three contextual metrics: document shareability, valuation, and user cooperation. Furthermore, these socially-dependent metrics are combined with our Smart Enterprise Access Control (SEAC) technique to achieve authenticity precision of 99% while improving the corresponding efficiency trade-off associated with high and strict security.

To optimally cover users in millimeter-Wave (mmWave) networks, clustering is needed to identify the number and direction of beams. The mobility of users motivates the need for an online clustering scheme to maintain up-to-date beams towards those clusters. Furthermore, mobility of users leads to varying patterns of clusters (i.e., users move from the coverage of one beam to another), causing dynamic traffic load per beam. As such, efficient radio resource allocation and beam management is needed to address the dynamicity that arises from mobility of users and their traffic. In this paper, we consider the coexistence of Ultra-Reliable Low-Latency Communication (URLLC) and enhanced Mobile BroadBand (eMBB) users in 5G mmWave networks and propose a Quality-of-Service (QoS) aware clustering and resource allocation scheme. Specifically, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is used for online clustering of users and the selection of the number of beams. In addition, Long Short Term Memory (LSTM)-based Deep Reinforcement Learning (DRL) scheme is used for resource block allocation. The performance of the proposed scheme is compared to a baseline that uses K-means and priority-based proportional fairness for clustering and resource allocation, respectively. Our simulation results show that the proposed scheme outperforms the baseline algorithm in terms of latency, reliability, and rate of URLLC users as well as rate of eMBB users.

In this paper, we address inter-beam inter-cell interference mitigation in 5G networks that employ millimetre-wave (mmWave), beamforming and non-orthogonal multiple access (NOMA) techniques. Those techniques play a key role in improving network capacity and spectral efficiency by multiplexing users on both spatial and power domains. In addition, the coverage area of multiple beams from different cells can intersect, allowing more flexibility in user-cell association. However, the intersection of coverage areas also implies increased inter-beam inter-cell interference, i.e. interference among beams formed by nearby cells. Therefore, joint user-cell association and inter-beam power allocation stand as a promising solution to mitigate inter-beam, inter-cell interference. In this paper, we consider a 5G mmWave network and propose a reinforcement learning algorithm to perform joint user-cell association and inter-beam power allocation to maximize the sum rate of the network. The proposed algorithm is compared to a uniform power allocation that equally divides power among beams per cell. Simulation results present a performance enhancement of 13 - 30% in network's sum-rate corresponding to the lowest and highest traffic loads, respectively.

The traffic generated by video streaming applications constitutes a large portion of the Internet traffic carried over today’s networks. Video streaming demands low latency and high bandwidth. In particular, the transmission of high-quality (high-resolution) streaming video may put the network under pressure. Therefore, high-quality video traffic requires network managers to implement smart and fast routing decisions. Software Defined Networking (SDN) provides a global view and centralized control for the whole network which gives opportunities to dynamically manage networks. In this paper, we use an OpenFlow-based SDN environment and propose a dynamic routing scheme with online traffic estimation to increase the quality of high-quality video streaming and the throughput of the network. The traffic is clustered using an unsupervised machine learning algorithm, high-quality video flows are identified and routed over less congested paths. The whole design is tested in the Mininet simulator. Simulation results show that the proposed scheme improves the link utilization and reduces the amount of dropped frames as a result of excessive delay.

Next-generation wireless networks are moving rapidly towards supporting heterogeneous services that bring along several challenges in radio resource allocation. In this paper, we address the problem of multiplexing Ultra- Reliable Low- Latency Communication (URLLC) users and enhanced Mobile Broadband (eMBB) users on a shared channel of 5G New Radio (NR).We propose a joint power and resource allocation algorithm based on Q-learning. The proposed algorithm is crafted carefully to improve reliability and latency of URLLC users without hindering throughput of eMBB users. In particular, the algorithm rewards the actions that mitigate inter-cell interference as well as improve transmission and scheduling delays. We compare our results with a priority-based proportional fairness algorithm with fixed power allocation that relies on giving URLLC users priority in resource scheduling. Simulation results reveal that our algorithm is able to achieve 4% increase in reliability as well as lower latency results in high traffic load scenarios.

Smart grids require high reliability and sufficient bandwidth from wireless networks to support critical real-time applications and massive smart microgrid data. In general, smart microgrids need to guarantee delays at the order of a few μs for highly delay-sensitive data delivery; as well as delays within few seconds for regular data delivery. This paper presents a framework and its performance analysis for microgrid data aggregation where the microgrid is served by a wireless heterogeneous network. Using unsupervised machine learning, the framework introduces a multi-class and delay sensitivity-aware aggregation of microgrid data within the small cells of the heterogeneous network to ensure that clustering reduces the processing time for highly delay-sensitive messages. Thus, at each Transmission Time Interval (TTI), if there is queued delay-sensitive data, they are dequeued ahead of the delay-tolerant data at the scheduler. Through simulations, we show that the proposed approach successfully reduces the queuing delay by 93% for the packets of delay-sensitive (urgent) messages and the Packet Loss Rate (PLR) by 7% when compared to the benchmark where no aggregation mechanism exists prior to the small cell base stations.

Energy trading among microgrids has been emerging as a promising solution to implement community microgrids, also known as energy sharing communities. The key idea behind these communities is to share the surplus energy in one microgrid with another microgrid that has higher demand than its generation. The objective of these transactions can be monetary as well as optimizing a system parameter. In this paper, we focus on energy trading for the purpose of power loss minimization. We assume microgrids form coalitions to avoid exporting energy from the utility grid or a distant microgrid which might cause higher line losses due to increased distance. We propose a novel Bayesian Reinforcement Learning (BRL) based algorithm, which allows the microgrids to reduce the overall power loss. We compare this scheme with a coalitional game theory-based approach, Q-learning based approach, random coalition formation approach, as well as with a case that has no coalitions. Our results show that more than 50% reduction in power loss compared to no coalitions and less power loss than the other approaches is achieved. We also show power loss can be further reduced by proper sizing of the storage unit.

Next-generation wireless networks will be supporting mission critical services such as safety related applications of connected autonomous vehicles, and real-time control of medical and industrial systems, as well as serving traditional mobile users. In mission critical services, high-reliability and low-latency requirements should be satisfied. In this paper, we aim to reduce the latency of uplink scheduling of a network of Mission Critical Devices (MCDs) while maintaining fairness among other users, served by a dense small cell network. We propose a Deep Reinforcement Learning algorithm, namely Delay Minimizing Deep Q-Learning (DMDQ), that combines Long Short-term Memory with Q-learning. The problem is cast as a resource block allocation for delay minimization. The proposed algorithm is compared to a tabular Q-learning approach and a simple Round Robin (RR) algorithm in terms of latency, throughput, fairness and convergence. Our performance results show that DMDQ outperforms both schemes in terms of latency and offers high fairness. The Q-learning approach achieves slightly higher throughput than DMDQ however DMDQ convergences faster.

Rising energy costs, losses in the present-day electricity grid, risks from nuclear power generation, and global environmental changes are motivating a transformation of the conventional ways of generating electricity. Globally, there is a desire to rely more on renewable energy resources (RERs) for electricity generation. RERs reduce greenhouse gas emissions and may have economic benefits, e.g., through applying demand side management with dynamic pricing so as to shift loads from fossil fuel-based generators to RERs. The electricity grid is presently evolving toward an intelligent grid, the so-called smart grid (SG). One of the major goals of the future SG is to move toward 100% electricity generation from RERs, i.e., toward a 100% renewable grid. However, the disparate, intermittent, and typically widely geographically distributed nature of RERs complicates the integration of RERs into the SG. Moreover, individual RERs have generally lower capacity than conventional fossil fuel-based plants, and these RERs are based on a wide spectrum of different technologies. In this article, we give an overview of recent efforts that aim to integrate RERs into the SG. We outline the integration of RERs into the SG along with their supporting communication networks. We also discuss ongoing projects that seek to integrate RERs into the SG around the globe. Finally, we outline future research directions on integrating RERs into the SG.

The pervasiveness of smartphones equipped with various built-in sensors combined with the capability of serving multiple applications that could access social network information introduces next generation soft biometrics tools that could be used to verify a user's identity through their social behavior. Smart mobile devices can provide multi-modal data acquisition from various social networking applications, and when aggregated, these data can help form highly identifiable behaviometric information. Continuous identification and authentication of users through monitoring social behavior improves detection of identity spoofing. In this paper, we propose a social behaviometric framework to cope with identity spoofing on smartphones. The proposed framework consists of a front-end client module that acquires and provides social networking data to the back-end module which runs online machine learning procedures and provides analytics as a service to the front-end in order to verify user identity through social interactions. We evaluate the performance of the proposed framework by using real data collected from participants, and inject noisy behavioral patterns to simulate identity spoofing scenarios. Performance results show that under anomalous behavioral patterns, the proposed system can identify genuine users with up to 97% success ratio using an aggregated behavior pattern on five different social network applications.

The papers in this special section focus on the deployment of information and communication technology (ICT) in smart grids as it relates to renewable energy resource management. The successful integration of renewable energy into the power grid is expected to reduce the dependence of the grid on the fossil fuels. The potential renewable energy resources include light, wind, vibration, heat, biofuel, biomass, and tides. It is envisaged that the use of renewable energy will reduce the use of traditional energy resources, such as nuclear, oil, and gas, in the future and this trend will continue in order to reduce the emission of greenhouse gases. The abundance of these renewable distributed energy resources (DERs) at the consumer side may help to develop distributed renewable energy generation at a large scale. The DERs will likely be an integral part of the future electric grid, i.e., the smart grid [4]–[6]. A prominent feature of the smart grid is that it allows for two-way communication between the utility and its customers through ICTs.

With a wide variety of mobile applications and social networks, there is a growing demand for anytime, anywhere access to high quality video and other multimedia content. Fifth generation wireless networks are aiming to keep up with user demands by incorporating new spectrum bands at high frequencies, which results in shrinking of cell sizes and eventually leads to small cells. Dense deployment of small cells and base stations open up new opportunities for faster and energy-efficient access to content. Content caching at small cell base stations, for the purpose of reducing delay, has been studied in several previous works. In our previous work, we used Integer Linear Programming (ILP) models to show that content caching can also reduce energy consumption of User Equipment (UE). In particular, selecting a subset of small cell base stations to cache the content, reduces uplink energy budget, cuts down caching energy budget and simplifies management of distributed caches. Yet, ILP based methods are powerful for obtaining benchmark results but they are computationally intensive for large size problems. In this paper, we propose a particle swarm optimization (PSO) based scheme to deploy an overlay small cell content-caching network with the aim of reducing UE energy consumption. We show that PSO is computationally efficient while it slightly increases energy consumption with respect to the ILP-based methods.

The widespread use of mobile technology has accelerated the popularity of social networking services, and has made these services convenient to access. This paper presents a behaviometric mobile application, namely TrackMaison (Track My activity in social networks). TrackMaison keeps track of social network service usage of smartphone users through data usage, location, usage frequency and session duration of five popular social network services. The data collected by the mobile application is presented to the smartphone user and is analyzed to aid in understanding mobile social network service usage. Furthermore, we introduce the social activity rate and sociability factor metrics where the former is a function of a user's relative data usage rate in social network services and the latter is a function of a user's relative session durations in social networks. By using TrackMaison tool, we identify three user behavior types. Those are active user profile, moderately active user profile and low active user profile. Through analysis of real data, we advocate that continuous identification/authentication of mobile device users is possible by using the introduced sociability metrics. We further present a case study on various Instagram user profiles, and show that low active profiles can be identified with negligible false acceptance rates (FAR) whereas a highly active user can be identified with a FAR as low as 3%.

Traditional wireless networks mainly rely on macro cell deployments, meanwhile with the advances in forth generation networks, the recent architectures of LTE and LTE-A support Heterogeneous Networks (HetNets) that employ a mix of macro and small cells. Small cells aim at increasing coverage and capacity. Coverage both at cell edges and indoor environments can be significantly improved by relays and small cells. Capacity is inherently limited because of the limited spectrum, and although 4G wireless networks have been able to provide a considerable amount of increase in capacity, it has always been challenging to keep up with the growing user demands. In particular, the high volume of traffic resulting from video uploads or downloads is the major reason for the ever growing user demand. In the Internet, content caching at locations closer to the users have been a successful approach to enhance resource utilization. Very recently, content caching within the wireless network has been considered for 4G networks. In this paper, we propose an Integer Linear Programming (ILP)-based energy-efficient content placement approach for small cells. The proposed model, namely minimize Uplink Power and Caching Power (minUPCA), jointly minimizes uplink and caching powers. We compare the performance of minUPCA with a scheme that only aims to minimize uplink power. Our results show that minUPCA provides a compromise between the uplink energy budget of the User Equipment (UE) and the caching energy budget of the Small Cell Base Station (SCBS).

Environmental monitoring applications for wireless sensor and actor networks rely on position estimation in order to process or evaluate the observed data. The absence of efficient positioning techniques for sensor nodes operating in harsh environments calls for novel approaches. While monitoring the Amazon river, unprecedented characteristics of the river and its surroundings challenge the node communications and drifting of the nodes makes it difficult to use the existing positioning methods. To address these challenges, we propose a multi-hop localization technique that takes advantage of sensor mobility with local information exchange. The collected information is used to enrich the environmental data with location information. The maximum hop distance for actor affiliation is also adapted according to network characteristics to improve energy consumption behavior. The motion of the sensor nodes follows the advection of the fluid parcels in the river, which is modeled as a combination of a central streamline with a meandering motion around the rough surface. This translates into a stretching topology with correlated motion for sensor nodes. Through extensive simulations, we show that the nodes can be efficiently positioned using the proposed approach, as our technique is compliant with the movement patterns of the sensor nodes in the realistic mobility model of the river.

Cloud data centers are significant players in the electricity market due to their high energy consumption. It is highly desired to operate cloud data centers on only renewable energy such as solar, wind or tidal to reduce their cost and emissions. In the literature, utilization of renewable energy is maximized through workload migration towards data centers that are forecasted to have surplus renewable energy which is also known as “follow the sun chase the wind” approach. In practice, it is rather difficult and unrealistic to shift workloads based on instantaneous output of renewable generation. Generally forecast tools are used which can only provide a rough estimate of the generation capacity. In this paper, we use a more realistic approach and propose an overlay architecture to form energy circles of cloud data centers depending on their load and renewable energy futures contracts. A futures contract is an electricity purchase agreement between the data center operator and the renewable energy generator to purchase electricity in the future with today's price. Futures contracts are electricity market mechanisms that reduce the cost related risks for both parties and are seen as tools to scale-up renewable generation. On the other hand, fluctuating loads of cloud data centers may leave some contract capacity left unused or exceed the capacity. In this case, electricity will be purchased at current market price from the renewable generator or the utility grid where in both cases the electricity bill will increase. Hence, a mechanism to utilize the unused capacity in the contracts of peer data centers and shifting workloads towards available capacity can reduce bills as well as increase the utilization of renewable energy. Our proposed energy circles approach aims to group cloud data centers to achieve those goals.

Wireless Energy Transfer (WET) promises charging wireless sensor networks, cell phones and on-body medical devices without the need of battery replacement nor plugging in to the mains. Magnetic induction and electromagnetic radiation are two alternative technologies for WET. Magnetic induction based WET is a mature technology while electromagnetic radiation based WET has been recently studied for WSNs or RFID tags in many studies. On the other hand, powering cell phones, PDAs or other User Equipment (UE) from ambient electromagnetic signals has unique challenges and is an emerging field of study. In this paper, we consider Radio Frequency WET (RF-WET) for prolonging UE lifetime in a Heterogeneous wireless network (HetNet). In a HetNet, coverage and capacity of the macro cell is augmented by small cells such as picocells, femtocells or Wi-Fi hotspots. In this paper, we assume small cell base stations and dedicated Energy Transmission Towers (ETTs) work together towards supplying power to the UEs. Power is supplied in the same frequency band with the communications in a time-sharing manner. We propose an ILP model where a mix of Picocell Base Stations (PBSs) and ETTs are placed such that the harvested energy is maximized while the number of ETTs and the number of actively power transmitting PBSs are minimized. We show that extended range of PBSs aid in increasing the amount of energy harvested by the UEs while as the number of serviced UEs increase the overall power harvesting capacity of the system improves.

Communication technologies are striving to provide ubiquitous and cable-free communication services to users while user devices are still limited with their batteries and need wires to recharge their batteries. The recent advances in Wireless Power Transfer (WPT) are promising to charge wireless sensor networks and on-body medical devices without the need of wires or battery replacement. One natural way of scavenging energy from the environment and providing ubiquitous power is electromagnetic radiation based WPT. Recently powering up Wireless Sensor Networks (WSNs) or RFID tags via omnidirectional radiation and beamforming has been studied in several studies. Yet, the potential of exploiting wireless networks to power User Equipment (UE) such as mobile phones or PDAs has been less explored. Long distances between wireless towers and UEs as well as their relatively low transmit power are among the major bottlenecks for WPT in wireless networks. In this paper, we consider dedicated energy transmission units (DETUs) to provide power to UEs. We show that although certain amount of power can be harvested by UEs, the cost of deploying DETUs dominates the design decision. As a transitional solution, power from relays, small cell towers and WiFi hotspots can be exploited. However when WPT is in-band with information transfer there may be interruption in connectivity. We discuss the challenges of WPT in wireless networks and propose several future directions.

Wireless Sensor Networks (WSNs) are anticipated to become the preferred tools of choice for monitoring and controlling power utility assets in the smart grid due to their versatility. However, in some smart grid monitoring applications, data generation rates could fluctuate rapidly due to the sudden occurrence of critical faults or failures in the monitored equipment. As a consequence, critical data could experience excessive delays because of this increase in the packet arrival rates. In this paper, we present an Adaptive Guaranteed Time Slot (GTS) allocation scheme (AGTS) for IEEE 802.15.4-based WSNs used in high traffic intensity smart grid monitoring applications. AGTS scheme can adaptively reduce the end-to-end delay and flexibly tune the GTS to provide the required Quality of Service (QoS) differentiation to delay critical smart grid monitoring applications. The proposed scheme can adaptively allocate the needed GTS to nodes transmitting high priority traffic or draw back the unneeded GTS.

The use of Wireless Sensor Networks (WSNs) to monitor and control power utility assets in the smart grid is gaining increasing popularity due to their various desirable features. WSNs with multihop cluster tree topologies solve the limited coverage problem of sensor nodes. However, in smart grid monitoring applications, data rates could increase suddenly due to the occurrence of critical faults in the monitored environment. Critical data transmission could experience excessive delays because of this increase in the packet arrival rates. Therefore, there should be an optimum operating point in the network where the network could handle high packet arrival rates and maintain low latency at the same time. In this paper, we present an optimization scheme that can achieve low latency while maintaining high reliability values. Furthermore, we design our scheme to provide Quality of Service (QoS) differentiation to high priority and delay critical data. Results show that our proposed scheme significantly reduces the delay while providing high reliability and incurring low energy consumption.

In critical infrastructure monitoring applications, the packet arrival rates of a Wireless Sensor Network (WSN) may abruptly increase when cascaded failures are observed in the monitored environment. WSNs with cluster-tree topologies could experience excessive delays because of this increase in packet arrival rates. Therefore, there should be an optimum operating point in the network where the network could accommodate high packet arrival rates with low latency. In this paper, we propose an adaptive scheme that can achieve low latency while maintaining high reliability values in cluster-tree based WSNs. Furthermore, our scheme provides Quality of Service (QoS) differentiation to high priority data. Analytical and simulation results show that our scheme significantly reduces the delay while maintaining high reliability and energy efficiency values.

Energy routing has not been feasible in the traditional power grid due to real-time nature of the electrical services. Electricity is generated and used almost in real time where the balance between the two is maintained by regulation services. Additionally, small number of fixed storage units are utilized to store some portion of the generated energy. In the future electricity grids, Mobile Energy Buffers (MEB) together with local energy buffering capabilities, will be the enabler of energy routing. In this paper, we propose an energy routing framework for low and medium voltage electricity distribution systems that house prioritized MEBs, local buffers and Delay-Tolerant electrical Loads (DTL). We model the distribution system as a token-based system where energy transfer between MEBs and local storage units rely on the availability of tokens that are generated by DTLs. Coordination of token advertisement, storage interest and token access is maintained by machine-to-machine communications. The underlying communication technology can be PowerLine Communications (PLC) or a medium-range wireless communication technology. We provide a mathematical analysis of the token-network with DTLs and prioritized MEBs. We show that coordination and prioritization allow lower blocking rates for high priority MEBs. Our analysis provides valuable insights for utility planning decisions.

The Quality of Service (QoS) in smart grid communications especially in monitoring smart grid assets is becoming significantly important for emerging smart grid applications. Wireless Sensor Networks (WSNs) are expected to be widely utilized in a broad range of smart grid applications due to their numerous advantages along with their successful adoption in various critical areas including military and health. WSNs protocols are not designed to provide QoS provisioning for monitoring applications. Thus, the use of WSNs in transmitting delay-critical data from smart grid assets calls for data prioritization and delay-mitigation schemes. In this paper, we propose a delay-responsive, cross layer scheme with linear backoff (LDRX) mechanism to address delay and service requirements of the smart grid monitoring applications. The LDRX scheme is designed to operate in cluster-tree WSN topology that is suitable for monitoring wide areas such as electrical substations or large installations. We show that LDRX has greater impact on delay reduction compared to previously proposed WSNs delay reduction schemes.

Wireless Sensor Networks (WSNs) provide wide reach and coverage at low-cost which enable them to be utilized in various fields such as health, smart grid, industrial facilities and defense. One of the fundamental limitations of WSNs in long-lasting applications is the network lifetime. To overcome the battery constraint of sensor nodes, duty cycling, energy-efficient protocols and energy harvesting have been considered widely in the literature. A recently emerging energy harvesting technique, namely Radio Frequency (RF)-based wireless energy transfer promises to extend the lifetime of Wireless Rechargeable Sensor Networks (WRSN) with no dependency on intermittent ambient energy resources. In RF-based wireless energy transfer, deploying power transmitters to fixed locations is costly due to range limitations of wireless power. For this reason, mobile power transmitters that visit a few selected locations; i.e. landmarks are employed. Furthermore, in WSNs sensors are expected to perform certain tasks or missions during their lifetime. The achievement of each mission provides certain profits. In this paper, we aim to optimally select the landmarks for sensor nodes that participate in profit maximizing missions. We propose an Integer Linear Programming (ILP) model, namely Mission-Aware Placement of Wireless Power Transmitters (MAPIT) that optimizes the placement of RF-based chargers in the WRSN by maximizing the number of nodes receiving power from a landmark and those that contribute the maximum profit by achieving a mission. We show that the profit increases for low landmark limit since the number of nodes receiving power from a landmark increases under less landmarks. On the other hand, profit reduces by increased number of missions since the nodes participating to missions become spatially diverse.

Sensor network web services have recently emerged as promising tools to provide remote management, data collection and querying capabilities for sensor networks. They can be utilized in a large number of fields among which Demand-Side Energy Management (DSEM) is an important application area that has become possible with the smart electrical power grid. DSEM applications generally aim to reduce the cost and the amount of power consumption. In the traditional power grid, DSEM has not been implemented widely due to the large number of households and lack of fine-grained automation tools. However by employing intelligent devices and implementing communication infrastructure among these devices, the smart grid will renovate the existing power grid and it will enable a wide variety of DSEM applications. In this paper, we analyze various DSEM scenarios that become available with sensor network web services. We assume a smart home with a Wireless Sensor Network (WSN) where the sensors are mounted on the appliances and they are able to run web services. The web server retrieves data from the appliances via the web services running on the sensor nodes. These data can be stored in a database after processing, where the database can be accessed by the utility, as well as the inhabitants of the smart home. We show that our implementation is efficient in terms of running time. Moreover, the message sizes and the implementation code is quite small which makes it suitable for the memory-limited sensor nodes. Furthermore, we show the application scenarios introduced in the paper provide energy saving for the smart home.

New services and applications that employ the advances in the Information and Communication Technologies (ICT) to the electrical power grid are rapidly emerging and consequently the traditional power grid is evolving into a smart grid. In the smart grid, communication among the supplier controlled generation units, utility administered transmission and distribution system and the consumer devices is providing new opportunities for improving the resilience and the efficiency of the grid. Resilience is a significant issue due to increasing demand, and in contrast, diminishing fossil fuels. Moreover, in the near future, resilience is expected to become a more significant concern especially due to the additional loads of the Plug-In Hybrid Electrical Vehicles (PHEVs). PHEVs are expected be widely adopted as passenger cars and as commercial vehicle fleets since they have low carbon emissions and low operating costs. On the other hand, their load on the power grid should be managed so that they do not cause failures. In this paper, we propose the Communication-based PHEV Load Management (Co-PLaM) scheme to control the load of the PHEVs. In our scheme, utilities provision a certain amount of energy for each distribution system based on the predicted supply level. The provisioned energy is communicated to the Substation Control Center (SCC) where each charging request is either accepted or rejected based on the utility set limits. Then, these decisions are sent to the smart charging stations through a Wireless Mesh Network (WMN) that uses IEEE 802.11s. In this paper, we simulate the Co-PLaM scheme and also mathematically analyze the blocking probability of the system. We show the performance of WMN in terms of delivery ratio, delay and jitter. Furthermore, we provide the blocking results and show the required additional capacity to supply all the PHEV loads without causing grid failures.

Information and Communication Technologies (ICT) are playing a key role in converting the traditional power grid into a smart power grid, and hence, they provide a number of opportunities to develop novel applications for the new cyber-physical power infrastructure. Interconnection of the smart appliances, consumer devices, Plug-In Hybrid Electric Vehicles (PHEV) and local renewable energy generation resources with the smart grid enables energy and demand management for the cyber-physical power infrastructure. In this paper, we employ a Home Gateway and Controller (HGC) device that communicates with the PHEV and controls its charging and discharging profile. HGC can also communicate with the controller of the solar power generation unit in the smart home, and it can schedule the consumption of the smart appliances accordingly. Moreover, since PHEVs draw large amount of electricity, simultaneous charging in a neighborhood can overload the utility transformers in the distribution substations and risk the resilience of the power grid. To avoid this, HGC communicates with the other HGC devices in the neighborhood and coordinates PHEV loads. Our simulation results show that, efficiency of a PHEV as a storage unit increases as it is plugged for longer periods. Moreover, when renewable energy resources are not available, a larger portion of the PHEV battery can be used for storing energy during off-peak hours, and discharging during peak hours to accommodate the household demand. Thus, we show that HGC is able to provide savings for the consumers and it can also coordinate the power supply such that the availability of solar power increases the efficiency and reduces the utilization of PHEV battery.

Smart grids aim to integrate recent advances in communications and information technologies to renovate the existing power grid. In smart grids, consumers can generate energy and sell it to the utilities. Moreover, they can avoid consumption during peak hours which helps reducing the peak load on the grid. Energy-aware homes can aid consumers to manage their demand and supply profile. In this paper, we propose Appliance Coordination with Feed In (ACORD-FI) scheme for such energy-aware smart homes. We show that ACORD-FI decreases the cost of energy consumption of home appliances, significantly.

Coexistence of Plug-in Hybrid Vehicles (PHEVs) with the emerging smart grids has been recently an attractive and equally challenging research topic. The existing electricity grids are rapidly evolving into smart grids by utilizing the advances in Information and Communication Technologies (ICT). Meanwhile, advances in Lithium-Ion (Li-ion) battery technologies have made manufacturing of PHEVs cost-wise effective, and PHEVs are expected to be widely adopted in the following years. PHEVs have several benefits over conventional vehicles such as, less fuel dependency, lower operating costs and lower amount of CO2emissions. On the other hand, unless PHEVs are powered by off the grid renewable energy resources, they will be drawing electricity from the grid to charge their batteries and they will increase the load on the grid. In the worst case, when the Time Of Charging (TOC) coincides with the critical peak periods, the grid may experience overall or partial failure. For most of the cases, TOC may be during the peak hours when the price of electricity is high. To avoid endangering grid resilience and to avoid high costs, a charging strategy and communication with the smart grid is essential. In this paper, we propose a prediction-based charging scheme which receives dynamic pricing information by wireless communications, predicts the market prices during the charging period and determines an appropriate TOC with low cost. Our prediction-based charging scheme is based-on a simple, light-weight classification technique which is suitable for implementation on a vehicle or a charging station. We show that prediction-based charging provides less operating cost and less CO2emissions.

The electricity grid is undergoing a major renovation and becoming a smart grid by integrating the advances in Information and Communication Technologies (ICT). Current applications in energy generation, power distribution and its consumption need improvement in several ways, such as, making efficient use of green energy, increasing automation in distribution and enabling residential energy management. The existing grid does not provide sufficient mechanisms to manage the residential electricity consumption. However, interconnecting consumer devices with the home area networks, and at the same time, communicating with the utility networks through a home gateway facilitate residential energy management in smart grids. Residential energy management uses utility-driven price signals which vary depending on the time of the day. This is called as Time Of Use (TOU) pricing. In TOU pricing, electricity consumption during peak hours costs more than electricity consumption during off-peak hours. TOU prices reflect the variation in the actual cost of power during one day. Utilities run bas plants to supply power for the base load. In peak hours, demands of the consumers rise, and utilities bring peaker plants online to supply additional power. Peaker plants have higher operating costs and higher GreenHouse Gas (GHG) emission rates than base plants. Therefore, reducing peak load decreases the expenses for energy generation and it decreases the GHG emissions. Wireless sensor networks can play a key role in reducing the demand of the consumers in peak hours. In this paper, we employ TOU-aware energy management in a smart home with wireless sensor home area network and analyze the impact of this schemes on the peak load. We show that our scheme decreases the use of the appliances in peak hours and reduces the energy bills for consumers.

Underwater mobile acoustic sensor networks are promising tools for the exploration of the oceans. These networks require new robust solutions for fundamental issues such as: localization service for data tagging and networking protocols for communication. All these tasks are closely related with connectivity, coverage and deployment of the network. A realistic mobility model that can capture the physical movement of the sensor nodes with ocean currents gives better understanding on the above problems. In this paper, we propose a novel physically-inspired mobility model which is representative of underwater environments. We study how the model affects a range-based localization protocol, and its impact on the coverage and connectivity of the network under different deployment scenarios.