In this article, we propose a novel modification to the Hegselmann-Krause (H-K) model that introduces dynamic, agent-specific confidence intervals, driven by the concept of social confirmation bias. Each agent’s confidence interval is dynamically adjusted based on their conformity resistance (CR) and the deviation of their opinion from the average opinion of the network. The proposed model reflects heterogeneity in social influence by allowing for varying confidence intervals among agents. This leads to more realistic modeling of opinion formation, where the system can evolve to form a limited number of opinion clusters, depending on the initial distribution of opinions and CR parameters. We demonstrate that the system converges to either consensus or multiple distinct clusters, governed by nonlinear dynamical equations. In addition, the stability of the proposed model is analyzed and the conditions under which the system reaches different numbers of clusters are studied. Our results show that the adaptive model more accurately captures the diversity of social interactions than previous H-K models with dynamic confidence intervals. Furthermore, the nonlinear nature of the model presents new challenges for convergence analysis, which we address by studying the properties of fixed points and cluster formations.

This paper proposes a novel control design for voltage tracking of an islanded AC microgrid in the presence of nonlinear loads and parametric uncertainties at the primary level of control. The proposed method is based on the Tube-Based Robust Model Predictive Control (RMPC), an online optimization-based method which can handle the constraints and uncertainties as well. The challenge with this method is the conservativeness imposed by designing the tube based on the worst-case scenario of the uncertainties. This weakness is amended in this paper by employing a combination of a learning-based Gaussian Process (GP) regression and Tube-Based RMPC. The advantage of using GP is that both the mean and variance of the loads are predicted at each iteration based on the real data, and the resulted values of mean and the bound of confidence are utilized to design the tube in Tube-Based RMPC. The theoretical results are also provided to prove the recursive feasibility and stability of the proposed learning based Tube-Based RMPC. Finally, the simulation results are carried out on both single and multiple DG (Distributed Generation) units.Note to Practitioners—In this paper, we present a new way to control the voltage in an islanded microgrid to improve Power Quality (PQ). The method we propose is based on an online optimization technique called Tube-Based Robust Model Predictive Control. It can handle uncertainties and disturbances that occur when the microgrid operates independently, ensuring the voltage remains stable. However, there’s a challenge with this method. It tends to be too cautious because it assumes the worst-case scenario for uncertainties. To make the control more efficient, we improve it by combining a learning-based technique called Gaussian Process regression with Tube-Based RMPC. The advantage of using GP is that it predicts the uncertainty of the electrical devices based on real data. We use these predictions to design the control in Tube-Based RMPC more accurately. We also provide theoretical results to show that our new learning-based control is reliable and stable. We tested our approach through computer simulations on different scenarios with one or multiple power sources in the microgrid. The results show the effectiveness of our control design in regulating the voltage even with uncertain and nonlinear loads. Overall, this paper suggests a practical and reliable way to control the voltage in an independent microgrid using a combination of online optimization and learning techniques.

Designing an optimal Demand Response (DR) program necessitates having information on the utility of users. However, that is not easily accessible since the users may not be willing to report their personal information. In this paper, we propose an incentive-based DR mechanism for Commercial and Industrial (C$\&$I) customers, leveraging contract theory to address the challenges of incomplete information and ensure the participation of customers in the DR program. Designing a DR mechanism includes specifying an incentive reward, penalty, and the corresponding demand reduction as functions of the customer’s utility information. These functions are obtained through an optimization problem that maximizes the grid operator’s profit, ensures truthful reporting of personal information and participation of the customers in the DR program. After formulating the optimal mechanism, the main technical challenge is reformulating the obtained non-convex and computationally inefficient optimization problem to a tractable and convex one. The extensive numerical evaluations demonstrate the effectiveness of our proposed framework in comparison to existing benchmark schemes.Note to Practitioners—Imbalances between supply and demand, particularly during the hot summer months, can result in issues such as power outages. This problem is detrimental for both grid operators and customers alike. DR programs have been implemented in numerous countries to address these challenges. C$\&$I customers, due to the schedulable nature of their loads, make for ideal targets for these programs. However, the main obstacle to implementing such programs is customer participation. Incentive rewards offered in these programs for load reduction are fixed for all customers, discouraging those with low rewards from participating and imposing high costs on grid operators for those with high rewards. On the other side, designing unique rewards and penalties for each customer necessitates information about their preferences, which grid operators cannot access. In this paper, we propose an incentive mechanism based on the contract between the grid operator and C$\&$I customers that ensures customers truthfully report their preferences while simultaneously maximizing grid operator profit and guaranteeing the participation of all customers. The proposed DR program can be implemented for large non-residential customers in order to reduce the peak load of the grid.

In this article, we investigate the Nash-seeking problem of a set of agents, playing an infinite network aggregative Markov game. In particular, we focus on a noncooperative framework where each agent selfishly aims at maximizing its long-term average reward without having explicit information on the model of the environment dynamics and its own reward function. The main contribution of this article is to develop a continuous multiagent reinforcement learning (MARL) algorithm for the Nash-seeking problem in infinite dynamic games with convergence guarantee. To this end, we propose an actor-critic MARL algorithm based on expected policy gradient (EPG) with two general function approximators to estimate the value function and the Nash policy of the agents. We consider continuous state and action spaces and adopt a newly proposed EPG to alleviate the variance of the gradient approximation. Based on such formulation and under some conventional assumptions (e.g., using linear function approximators), we prove that the policies of the agents converge to the unique Nash equilibrium (NE) of the game. Furthermore, an estimation error analysis is conducted to investigate the effects of the error arising from function approximation. As a case study, the framework is applied on a cloud radio access network (C-RAN) by modeling the remote radio heads (RRHs) as the agents and the congestion of baseband units (BBUs) as the dynamics of the environment.

This article studies the problem of noncooperative multiagent reinforcement learning (MARL), where selfish agents play a general-sum Markov game. We consider the framework where no agent has explicit information on the model of dynamic environment, the model of other agents, and even on its own cost function. We propose an actor–critic MARL to learn the Nash equilibrium (NE) policy of the agents. The main contribution of this article is to extend the NE seeking methods to incomplete information stochastic nonzero sum games. Based on such formulation and under some conventional assumptions, we prove that by applying linear function approximators, the policies of agents converge to an approximation of the first-order NE point of the game. Finally, as a case study, the framework is applied to a Cloud Radio Access Network.

We propose an incentive mechanism for the sponsored content provider (CP) market in which the communication of users can be represented by a graph, and the private information of the users is assumed to have a continuous distribution function. The CP stipulates incentive rewards to encourage users to reveal their private information truthfully and increase their content demand, which leads to an increase in the advertising revenue. We prove that all users gain a nonnegative utility and disclose their private information truthfully. Moreover, we study the effectiveness and scalability of the proposed mechanism in a case study with different network structures.

Due to the increasing popularity of Electric Vehicles (EVs) and infrastructural limitations, it is vital to manage traffic and Charging Stations (CSs) crowdedness. In Bakhshayesh and Kebriaei (2022), the problem of choosing the route and CSs of EVs is modeled as a non-cooperative game of selfish EVs with probabilistic decision strategies. In this paper, we have proposed a linear pricing policy that ensures global efficiency of the obtained Nash strategies of EVs in Bakhshayesh and Kebriaei (2022) for the Smart City Coordinator (SCC). We model the problem as a hierarchical game with a SCC as the leader and EVs as the followers. The leader aims to design optimal price functions of CSs and Traffic Coordinator (TC) and impose them on the EVs to maximize the social profits of CSs and TC. In response, the followers play a non-cooperative game with coupling constraints to optimally decide on their route and charging destination. Thus, we have a Stackelberg Game (SG) between SCC and EVs and also a Nash game among the EVs in the lower level. Compared to the conventional Nash-based pricing policies, our proposed functional SG formulation enables the possibility of simultaneous and global-optimum management of traffic and CSs’ crowdedness. Moreover, we have proposed a two-level decentralized algorithm that preserves the privacy of EVs and have considered a decentralized computation for equilibrium seeking of followers based on the Alternating Direction Method of Multipliers (ADMM) method. Finally, we carry out simulation studies on the transportation network of Sioux Falls City to compare and evaluate the proposed method.

The novel concept of Internet of Nano Things (IoNT) brings even larger groups of nanomachines collaborating to achieve more complex tasks in the military, medical, and security fields. Moreover, Internet of Bio-Nano Things (IoBNT) is an emerging technology defining the seamless connection of nanomachines and biological entities with each other where they can access the traditional wireless communication networks to provide novel Internet of Things (IoT) applications, such as health monitoring, healthcare, and targeted therapy. The exchange of information between biological cells is based on the synthesis, transformation, emission, propagation, and reception of molecules. This information exchange is recently classified in telecommunications as molecular communication which is a biologically inspired technique to communicate in very small dimension networks. In nanonetworks, a high number of nanothings will operate in the same medium and they interfere with each other. Multiuser interference will create significant limits. Hence, we introduce to use the direction of releasing molecules as a new property to convey information. Releasing the molecules to the specific directions enhances the performance of molecular communication systems due to multiusers interference mitigation. Hence, adjacent transmitters can convey information in different directions, simultaneously. Then, we propose the binary direction shift keying (BDSK) modulation scheme where the transmitter pumps molecules in two different directions. Next, we obtain the error probability and achievable bit rate of BDSK modulation. Finally, we evaluate the performance of BDSK modulation by numerical results. The result of this article can be useful for molecular communication relay systems where relay nodes convey information to the different destinations.

Increasing the penetration of electric vehicles (EVs) in public transportation, which is also sped up by governments’ carbon net-zero policies, will significantly increase the demand for electricity. Therefore, when we face with a large population of selfish EV users, we need a coordination mechanism to manage both the traffic congestion and electricity resource limitations. This paper introduces a novel aggregative game model where heterogeneous EVs simultaneously plan their parking lot as their destination (usually accompanied by battery charging) and the route to the destination. The cost function of users consists of factors such as traveling time, variable costs of congestion and electricity demand, and tolling which is imposed to satisfy coupling constraints such as roads’ capacity and stations’ power capacity. Since the users are selfish and do not reveal their objectives and personal constraints, we propose a privacy preserving decentralized algorithm with a traffic coordinator and multiple stations’ coordinators for generalized Nash equilibrium (GNE) seeking of the game model. Only aggregate information such as traffic on the road and stations’ energy demand are available to the traffic coordinator and charging stations’ coordinators, respectively. We show that the proposed aggregative game admits a unique variational generalized Nash equilibrium (v-GNE). Then, using the theory of variational inequality (VI), we show that the proposed decentralized algorithm converges to the unique v-GNE of the game. Finally, we carry out comprehensive simulation studies on a simulated Savannah city model to compare and evaluate the proposed method.

In a blockchain network, to mine new blocks like in cryptocurrencies or secure IoT networks, each node or player specifies the amount of computational power as its strategy by compromising between the cost and expected utility. Since the strategies of all players affect the expected utility of others through the probability of success, in this article, we first formulate the mining competition among the players in a blockchain network as a noncooperative game. The existence and uniqueness of the Nash equilibrium (NE) point of the game are proven. We consider a gradient learning strategy for the players while preserving their private information as a bounded rational learning model. Furthermore, the convergence of this learning strategy to the E-NE point of the game is studied analytically using the concept of the mean field (MF) game theory. While conventional analytical tools face problems in dealing with a large number of participants, which is a key feature in many IoT networks, deploying the MF game theory facilitates analyzing the behavior of a large population of players by encapsulating the network behavior in an MF term. As the number of players becomes larger, the accuracy of the MF method becomes greater. Moreover, in the MF approach, no information exchange among the agents is needed for optimal decision making and the privacy of the players is preserved. The minimal information exchange is also a proper motivation for using the MF approach in the IoT networks.

In the reverse Stackelberg mechanism, by considering a decision function for the leader rather than a decision value in the conventional Stackelberg game, the leader can explore a wider decision space. This flexibility can result in realizing the globally optimal solution of the leader's objective function, while controlling the reaction function of the followers, simultaneously. We consider an aggregator who purchases energy from the wholesale energy market. The aggregator acts as the leader for a group of plugged in electric vehicles (PEVs) and determines the price of energy versus consumption at each hour a day as its decision function. In the followers level, since the optimal charging strategies of the PEVs are coupled through the electricity price, the PEVs in a group are considered to cooperate in finding their Nash-Pareto-optimal charging strategy, by minimizing a social cost function. For a large number of PEVs, the cooperative cost minimization of PEVs can be modeled as a cooperative mean-field (MF) game. We propose a decentralized MF optimal control algorithm and prove that the algorithm converges to leader-follower MF εN-Nash equilibrium point of the game. Furthermore, a decentralized reverse Stackelberg algorithm is implemented to achieve the optimal linear price function of the leader. Simulation results and comparison with benchmark methods are performed to demonstrate the advantages of the proposed method.

Network function virtualization (NFV) is referred to the technology in which softwarized network functions virtually run on commodity servers. Such functions are called virtual network functions (VNFs). A specific service is composed of a set of VNFs. This is a paradigm shift for service provisioning in telecom networks which introduces new design and implementation challenges. One of such challenges is to meet the reliability requirement of the requested services considering the reliability of the commodity servers. NFV placement which is the problem of assigning commodity servers to the VNFs becomes crucial under such circumstances. To address such an issue, in this paper, we employ Deep Reinforcement Learning (Deep-RL) to model NFV placement problem considering the reliability requirement of the services. The output of the introduced model determines optimal placement in each state. Numerical evaluations show that the introduced model can significantly improve the performance of the network operator.

Perimeter traffic flow control, based on the macroscopic fundamental diagram (MFD), has been introduced for traffic control and congestion management in large-scale networks. The perimeter controller is a set of traffic signals on the border between the regions manipulating the transfer flows with the aim to maximize the number of trips that reach their destinations. This paper tackles the optimal perimeter control of MFD systems for two-region urban networks which model heterogeneously congested cities. The modeling of the system results in nonlinear state dynamics, a non-quadratic cost function, and constraints on control actions and traffic states. We prove the existence of the optimal controller, analytically derive the optimal control policy, and introduce a numerical method to solve the optimal control policy. Based on the indirect optimal approach, HJB equation, and Pontryagin's maximum principal, we demonstrate that the optimal controller is in the form of Bang-Bang control. We apply the Chebyshev pseudospectral method to solve the two-point boundary value problem (TPBVP) for the proposed constrained optimal control problem. Consequently, the TPBVP is reduced to determination of the solution of a nonlinear system with algebraic equations. A numerical study is performed to measure the effectiveness of the proposed method.

In this paper, a distributed, minimum overhead power control algorithm for underlay cognitive radio networks (CRNs) having multiple primary and secondary users is proposed. The problem is formulated as a noncooperative game and a learning algorithm is proposed for optimizing the power allocation of secondary users. In the considered network, secondary users (SUs) do not have full information on the interference and power control strategies of other SUs. As a result, they update their strategy using a simple feedback from the primary user base station that provides the total interference. Although there is no cooperation among secondary users, it is shown that, under incomplete information, the proposed learning algorithm converges to the strategy of the players in the Nash equilibrium of the complete information case. The Nash equilibrium point is derived analytically, and then it is demonstrated that, although each user individually tries to maximize its own payoff, at the end, the proposed algorithm will converge to the complete information game Nash equilibrium point. It is also shown that the algorithm will be capable of adapting to a time-varying environment if some conditions on the SUs' processing power are satisfied. This is due to the slotted time assumption of the algorithm. Simulation results are then used to corroborate the analytical derivations.

Leveraging the path diversity in heterogeneous wireless networks by Multi-path TCP (MPTCP) not only depends on end-user's decisions but also on other competitors who are looking for maximizing their benefits. The incomplete information repeated Bayesian game is proposed to enhance the MPTCP throughput in a resource-shared wireless network context. Mobile nodes establish the initial path in the first stage by choosing the best opening sub-flows connection. Moreover, by receiving feedbacks regarding the preference of other opponents the repeated Bayesian game in the second stage improves the achievable throughput via selecting the best combination of networks. The numerical and simulation results demonstrate that the proposed algorithm could at least achieve (17%-24%) more throughput than WiFi in the initial path selection and (11%-14%) more throughput than MPTCP.

We develop a robust one-leader multi-follower dynamic linear quadratic game for 5G wireless networks with a cloud radio access network setting. The network consists of a baseband unit (BBU) owner and multiple remote radio heads (RRHs). The users' traffic from the RRH is routed to BBUs based on their price and congestion which is modeled as a dynamic difference equation, considering the effect of decisions taken by the RRHs, price, and uncertainties. In the proposed dynamic game, the BBU owner is the leader and the RRHs are the followers. The uncertainties raised from compression noise and link imperfection, such as packet loss, is modeled as an exogenous disturbance to the system. The equilibrium point of the proposed robust hierarchical dynamic game is achieved, and the conditions for the existence and uniqueness of the equilibrium point are provided in two cases, including unconstrained problem and also by considering the fronthaul constraints. Finally, the simulation results show the advantages in terms of cumulative payoff compared with other existing baseline schemes.

In this paper, a novel approach for interference pricing and power control in the uplink of a two-tier small cell network is proposed. To model this problem, a Stackelberg game is formulated in which the macrocell base station (MBS) and the femtocell user equipments (FUEs) are the players that seek to maximize their utility. In this game, the MBS optimizes its revenue which depends on the interference quota sold to the FUEs while the FUEs optimize the utility that captures the tradeoff between rate and payment to the MBS. Here, the MBS (leader) must choose an optimal price in order to manage the interference level from the FUEs (followers). To solve this game, a two-step distributed interference price bargaining algorithm is proposed. Using a number of techniques, the convergence of the proposed algorithm to a Stackelberg equilibrium is shown analytically. Simulation results show that this approach converges for a wide range of channel power gains while maintaining a certain energy efficiency level for the transmitting users.

In an incomplete information game, a big challenge is to find the best way of exploiting available information for optimal decision making of the agents. In this paper, two decision making methods, namely model-based and learning-based bidding strategies, are proposed and compared, for repeated Cournot competition of the generators in a day-ahead electricity market. The sum of the rivals' offered quantities (SROQ) is considered as the state of the agent and its value is estimated using an adaptive expectation method. In the model-based approach, the convergence of the agents' strategies to the Nash equilibrium point is also studied in two different cases. In the learning-based approach, the optimal bidding strategy is learned through combination of state estimation and a reinforcement learning method. Using the estimated state (SROQ), the optimal decision is learned through a fuzzy Q-learning algorithm. Through a case study, which is performed on the three-bus benchmark Cournot model, the convergence of the generators' bids to the Nash-Cournot equilibrium is examined.

We consider a cellular based primary network coexisting with a secondary network. The primary network consists of multiple service providers (SP) and there are several independent users in the secondary network. The SP are operating on the different frequency spectrum and a group of secondary users intend to share these spectrum with the primary services. This situation is formulated as a bandwidth auction game where each user bids a demand curve and each SP offers a supply curve. We consider two cases of complete information case and incomplete information case or learning games. For two cases, we derive the optimal strategies of the players and the distributed algorithms are presented to obtain the solution of these dynamic games.

In this paper, an agent matching method for bilateral contracts in a multi‐agent market is proposed. Each agent has a hierarchical representation of its trading commodity attributes by a tree structure of fuzzy attributes. Using this structure, the similarity between the trees of each pair of buyer and seller is computed using a new ordered fuzzy similarity algorithm. Then, using the concept of Stackelberg equilibrium in a leader–follower game, matchmaking is performed among the sellers and buyers. The fuzzy similarities of each agent with others in its personal viewpoint have been used as its payoffs in a bimatrix game. Through a case study for bilateral contracts of energy, the capabilities of the proposed agent‐based system are illustrated.