We consider a setting in which the agent aims to maximize the expected cumulative reward, subject to a constraint that the entropic risk of the total utility exceeds a given threshold. Unlike the risk-neutral case, standard primal-dual approaches fail to directly yield regret and violation bounds, as value iteration with respect to a combined state-action value function is not applicable in the risk-sensitive setting. To address this, we adopt the Optimized Certainty Equivalent (OCE) representation of the entropic risk measure and reformulate the problem by augmenting the state space with a continuous budget variable. We then propose a primal-dual algorithm tailored to this augmented formulation. In contrast to the standard approach for risk-neutral CMDPs, our method incorporates a truncated dual update to account for the possible absence of strong duality. We show that the proposed algorithm achieves regret of $\tilde{\mathcal{O}}\big(V_{g,\max}K^{3/4} + \sqrt{H^4 S^2 A \log(1/\delta)}K^{3/4}\big)$ and constraint violation of $\tilde{\mathcal{O}}\big(V_{g,\max} \sqrt{ {H^3 S^2 A \log(1/\delta)}}K^{3/4} \big)$ with probability at least $1-\delta$, where $S$ and $A$ denote the cardinalities of the state and action spaces, respectively, $H$ is the episode length, $K$ is the number of episodes, $\alpha < 0$ is the risk-aversion parameter, and $V_{g,\max} = \frac{1}{|\alpha|}(\exp(|\alpha|H) - 1)$. To the best of our knowledge, this is the first result establishing sublinear regret and violation bounds for the risk-sensitive CMDP problem.

In Reinforcement Learning (RL), tasks with instantaneous hard constraints present significant challenges, particularly when the decision space is non-convex or non-star-convex. This issue is especially relevant in domains like autonomous vehicles and robotics, where constraints such as collision avoidance often take a non-convex form. In this paper, we establish a regret bound of $\tilde{\mathcal{O}}((1 + \tfrac{1}{\tau}) \sqrt{\log(\frac{1}{\tau}) d^3 H^4 K})$, applicable to both star-convex and non-star-convex cases, where $d$ is the feature dimension, $H$ the episode length, $K$ the number of episodes, and $\tau$ the safety threshold. Moreover, the violation of safety constraints is zero with high probability throughout the learning process. A key technical challenge in these settings is bounding the covering number of the value-function class, which is essential for achieving value-aware uniform concentration in model-free function approximation. For the star-convex setting, we develop a novel technique called *Objective–Constraint Decomposition* (OCD) to properly bound the covering number. This result also resolves an error in a previous work on constrained RL. In non-star-convex scenarios, where the covering number can become infinitely large, we propose a two-phase algorithm, Non-Convex Safe Least Squares Value Iteration (NCS-LSVI), which first reduces uncertainty about the safe set by playing a known safe policy. After that, it carefully balances exploration and exploitation to achieve the regret bound. Finally, numerical simulations on an autonomous driving scenario demonstrate the effectiveness of NCS-LSVI.

In the ever-evolving landscape of Deep Neural Networks (DNN) hardware acceleration, unlocking the true potential of systolic array accelerators has long been hindered by the daunting challenges of expertise and time investment. Large Language Models (LLMs) offer a promising solution for automating code generation, which is key to unlocking unprecedented efficiency and performance in various domains, including hardware descriptive code. The generative power of LLMs can enable the effective utilization of preexisting designs and dedicated hardware generators. However, the successful application of LLMs to hardware accelerator design is contingent upon the availability of specialized datasets tailored for this purpose. To bridge this gap, we introduce the Systolic Array-based Accelerator DataSet (SA-DS). SA-DS comprises a diverse collection of spatial array designs following the standardized Berkeley’s Gemmini accelerator generator template, enabling design reuse, adaptation, and customization. SA-DS is intended to spark LLM-centered research on DNN hardware accelerator architecture. We envision that SA-DS provides a framework that will shape the course of DNN hardware acceleration research for generations to come. SA-DS is open-sourced under the permissive MIT license at https://github.com/ACADLab/SA-DS.

Mobile Robots (MRs), typically equipped with single-antenna radios, face many challenges in maintaining reliable connectivity established by multiple wireless access points (APs). These challenges include the absence of direct line-of-sight (LoS), ineffective beam searching due to the time-varying channel, and interference constraints. This paper presents REMARKABLE, an online learning based adaptive beam selection strategy for robot connectivity that trains kernelized bandit model directly in real-world settings of a factory floor. REMARKABLE employs reconfigurable intelligent surfaces (RISs) with passive reflective elements to create beamforming toward target robots, eliminating the need for multiple APs. We develop a method to create a beamforming codebook, reducing the search space complexity. We also develop a reconfigurable rotational mechanism to expand RIS coverage by rotating its projection plane. To address non-stationary conditions, we adopt the bandit over bandit idea that employs adaptive restarts, allowing the system to forget outdated observations and safely relearn the optimal interference-constrained beam. We show that our approach achieves a dynamic regret and the violation bound of Õ(T3/4B1/4) where T is the total time, and B is the total variation budget which captures the total changes in the environment without even assuming the knowledge of B. Finally, experimental validation with custom-designed RIS hardware and mobile robots demonstrates 46.8% faster beam selection and 94.2% accuracy, outperforming classical methods across diverse mobility settings.

Constrained decision-making is essential for designing safe policies in real-world control systems, yet simulated environments often fail to capture real-world adversities. We consider the problem of learning a policy that will maximize the cumulative reward while satisfying a constraint, even when there is a mismatch between the real model and an accessible simulator/nominal model. In particular, we consider the robust constrained Markov decision problem (RCMDP) where an agent needs to maximize the reward and satisfy the constraint against the worst possible stochastic model under the uncertainty set centered around an unknown nominal model. Primal-dual methods, effective for standard constrained MDP (CMDP), are not applicable here because of the lack of the strong duality property. Further, one cannot apply the standard robust value-iteration based approach on the composite value function, either, as the worst-case models may be different for the reward value function and the constraint value function. We propose a novel technique that effectively minimizes the constraint value function--to satisfy the constraints; on the other hand, when all the constraints are satisfied, it can simply maximize the robust reward value function. We prove that such an algorithm finds a policy with at most $\epsilon$ sub-optimality and a feasible policy after $O(\epsilon^{-2})$ iterations. In contrast to the state-of-the-art method, we do not need to employ a binary search; thus, we reduce the computation time and achieve a better performance, especially for continuous state-space.

We study the reinforcement learning (RL) problem in a constrained Markov decision process (CMDP), where an agent explores the environment to maximize the expected cumulative reward while satisfying a single constraint on the expected total utility value in every episode. While this problem is well understood in the tabular setting, theoretical results for function approximation remain scarce. This paper closes the gap by proposing an RL algorithm for linear CMDPs that achieves $\widetilde{\mathcal{O}}(\sqrt{K})$ regret with an episode-wise zero-violation guarantee. Furthermore, our method is computationally efficient, scaling polynomially with problem-dependent parameters while remaining independent of the state space size. Our results significantly improve upon recent linear CMDP algorithms, which either violate the constraint or incur exponential computational costs.

Join-the-shortest queue (JSQ) and its variants have often been used in solving load balancing problems. The aim of such policies is to minimize the average system occupation, e.g., the customer's system time. In this paper, we extend the load balancing setting to include constraints that may be imposed, e.g., due to the communication network. First, we cast the problem in the framework of constrained MDPs: this permits us to address both action-dependent constraints, such as, e.g, bandwidth limitation, and state-dependent constraints, such as, e.g., minimum queue utilization. Hence, unlike the state-of-the-art approaches in load balancing, we derive new policies that satisfy the constraints while minimizing system occupancy. Extensive numerical simulations have evaluated their performance under various system settings.

We study the constrained Markov decision processes (CMDPs), in which an agent aims to maximize the expected cumulative reward subject to a constraint on the expected total value of a utility function. Existing approaches have primarily focused on \emph{soft} constraint violation, which allows compensation across episodes, making it easier to satisfy the constraints. In contrast, we consider a stronger \emph{hard} constraint violation metric, where only positive constraint violations are accumulated. Our main result is the development of the \emph{first model-free}, \emph{simulator-free} algorithm that achieves a sub-linear regret and a sub-linear hard constraint violation simultaneously, even in \emph{large-scale} systems. In particular, we show that $\tilde{\mathcal{O}}(\sqrt{d^3H^4K})$ regret and $\tilde{\mathcal{O}}(\sqrt{d^3H^4K})$ hard constraint violation bounds can be achieved, where $K$ is the number of episodes, $d$ is the dimension of the feature mapping, $H$ is the length of the episode. Our results are achieved via novel adaptations of the primal-dual LSVI-UCB algorithm, i.e., it searches for the dual variable that balances between regret and constraint violation within every episode, rather than updating it at the end of each episode. This turns out to be crucial for our theoretical guarantees when dealing with hard constraint violations.

Fairness plays a crucial role in various multi-agent systems (e.g., communication networks, financial markets, etc.). Many multi-agent dynamical interactions can be cast as Markov Decision Processes (MDPs). While existing research has focused on studying fairness in known environments, the exploration of fairness in such systems for unknown environments remains open. In this paper, we propose a Reinforcement Learning (RL) approach to achieve fairness in multi-agent finite-horizon episodic MDPs. Instead of maximizing the sum of individual agents' value functions, we introduce a fairness function that ensures equitable rewards across agents. Since the classical Bellman's equation does not hold when the sum of individual value functions is not maximized, we cannot use traditional approaches. Instead, in order to explore, we maintain a confidence bound of the unknown environment and then propose an online convex optimization based approach to obtain a policy constrained to this confidence region. We show that such an approach achieves sub-linear regret in terms of the number of episodes. Additionally, we provide a probably approximately correct (PAC) guarantee based on the obtained regret bound. We also propose an offline RL algorithm and bound the optimality gap with respect to the optimal fair solution. To mitigate computational complexity, we introduce a policy-gradient type method for the fair objective. Simulation experiments also demonstrate the efficacy of our approach.

We propose WSAC (Weighted Safe Actor-Critic), a novel algorithm for Safe Offline Reinforcement Learning (RL) under functional approximation, which can robustly optimize policies to improve upon an arbitrary reference policy with limited data coverage. WSAC is designed as a two-player Stackelberg game to optimize a refined objective function. The actor optimizes the policy against two adversarially trained value critics with small importance-weighted Bellman errors, which focus on scenarios where the actor's performance is inferior to the reference policy. In theory, we demonstrate that when the actor employs a no-regret optimization oracle, WSAC achieves a number of guarantees: $(i)$ For the first time in the safe offline RL setting, we establish that WSAC can produce a policy that outperforms {\bf any} reference policy while maintaining the same level of safety, which is critical to designing a safe algorithm for offline RL. $(ii)$ WSAC achieves the optimal statistical convergence rate of $1/\sqrt{N}$ to the reference policy, where $N$ is the size of the offline dataset. $(iii)$ We theoretically show that WSAC guarantees a safe policy improvement across a broad range of hyperparameters that control the degree of pessimism, indicating its practical robustness. Additionally, we offer a practical version of WSAC and compare it with existing state-of-the-art safe offline RL algorithms in several continuous control environments. WSAC outperforms all baselines across a range of tasks, supporting the theoretical results.

This article presents real-time Pareto-optimal scheduling for bidirectional electric vehicle (EV) charging in a commercial charging station with on-site renewable energy and battery energy storage to optimize several objectives. To incorporate the inherent uncertainty in the model, mixture density neural networks are presented to estimate the parameters of the probability distribution of demands and deadlines using a negative-log-likelihood loss function. From the joint distribution of demands and deadlines, future EV charging requests are estimated. Furthermore, we formulate the control problem as a multiobjective stochastic convex optimization problem from the perspective of the charging station operator, which simultaneously aims to minimize the total cost of charging, frequent change in charging rates, maximum demand of the charging station and battery degradation costs subject to various system constraints. We empirically evaluate the proposed scheduling policy for optimality gap, competitive ratio, and robustness, and show that the proposed scheduling policy reduces cost by about$30 \%$over the benchmark scheduling policies.

We study model-free reinforcement learning (RL) algorithms in episodic non-stationary constrained Markov decision processes (CMDPs), in which an agent aims to maximize the expected cumulative reward subject to a cumulative constraint on the expected utility (cost). In the non-stationary environment, the reward, utility functions, and the transition kernels can vary arbitrarily over time as long as the cumulative variations do not exceed certain variation budgets. We propose the first model-free, simulator-free RL algorithms with sublinear regret and zero constraint violation for non-stationary CMDPs in both tabular and linear function approximation settings with provable performance guarantees. Our results on regret bound and constraint violation for the tabular case match the corresponding best results for stationary CMDPs when the total budget is known. Additionally, we present a general framework for addressing with the well-known challenges associated with analyzing non-stationary CMDPs, without requiring prior knowledge of the variation budget. We apply the approach for both tabular and linear approximation settings.

Network sharing is widely accepted as a cost effective approach for mobile network deployment. It remains uncertain, however, how regulators will evaluate network sharing agreements (NSA) for future networks in the context of the current competition law. For example, 5G mobile network operators (MNOs) seeking to enter NSAs may risk legal challenges, as regulators have not given MNOs sufficient guidance for self-evaluation of their NSAs. One way for MNOs to reduce the risk of legal challenge is to avoid sharing variable costs in the NSA. However, constraining costs to be non-variable (i.e., fixed) rules out the use of most pricing mechanisms that have been widely adopted for dynamic resource trading between MNOs. In this article, we propose a network sharing framework to allow dynamic resource sharing without the use of resource pricing. To incentivize sharing without pricing, our framework presents sharing as a means for MNOs to differentiate services and better compete in the service market for profit. We evaluate our framework in a duopoly market model and demonstrate the economic and regulatory viability of our framework.

We study the constrained reinforcement learning problem, in which an agent aims to maximize the expected cumulative reward subject to a constraint on the expected total value of a utility function. In contrast to existing model-based approaches or model-free methods accompanied with a `simulator’, we aim to develop the first \emph{model-free}, \emph{simulator-free} algorithm that achieves a sublinear regret and a sublinear constraint violation even in \emph{large-scale} systems. To this end, we consider the episodic constrained Markov decision processes with linear function approximation, where the transition dynamics and the reward function can be represented as a linear function of some known feature mapping. We show that $\tilde{\mathcal{O}}(\sqrt{d^3H^3T})$ regret and $\tilde{\mathcal{O}}(\sqrt{d^3H^3T})$ constraint violation bounds can be achieved, where $d$ is the dimension of the feature mapping, $H$ is the length of the episode, and $T$ is the total number of steps. Our bounds are attained without explicitly estimating the unknown transition model or requiring a simulator, and they depend on the state space only through the dimension of the feature mapping. Hence our bounds hold even when the number of states goes to infinity. Our main results are achieved via novel adaptations of the standard LSVI-UCB algorithms. In particular, we first introduce primal-dual optimization into the LSVI-UCB algorithm to balance between regret and constraint violation. More importantly, we replace the standard greedy selection with respect to the state-action function with a soft-max policy. This turns out to be key in establishing uniform concentration (a critical step for provably efficient model-free exploration) for the constrained case via its approximation-smoothness trade-off. Finally, we also show that one can achieve an even zero constraint violation for large enough $T$ by trading the regret a little bit but still maintaining the same order with respect to $T$.

To fully utilize the abundant spectrum resources in millimeter wave (mmWave), Beam Alignment (BA) is necessary for large antenna arrays to achieve large array gains. In practical dynamic wireless environments, channel modeling is challenging due to time-varying and multipath effects. In this paper, we formulate the beam alignment problem as a nonstationary online learning problem with the objective to maximize the received signal strength under interference constraint. In particular, we employ the non-stationary kernelized bandit to leverage the correlation among beams and model the complex beamforming and multipath channel functions. Furthermore, to mitigate interference to other user equipment, we leverage the primal-dual method to design a constrained UCB-type kernelized bandit algorithm. Our theoretical analysis indicates that the proposed algorithm can adaptively adjust the beam in time-varying environments, such that both the cumulative regret of the received signal and constraint violations have sublinear bounds with respect to time. This result is of independent interest for applications such as adaptive pricing and news ranking. In addition, the algorithm assumes the channel is a black-box function and does not require any prior knowledge for dynamic channel modeling, and thus is applicable in a variety of scenarios. We further show that if the information about the channel variation is known, the algorithm will have better theoretical guarantees and performance. Finally, we conduct simulations to highlight the effectiveness of the proposed algorithm.

Autonomous vehicles (AVs) are attractive for ride service providers (RSPs) in part because they eliminate the need to compete for human drivers. We investigate a scenario where two RSPs with AVs compete for customers. We model the problem as a game where the RSPs select prices for each origin-destination pair over multiple time periods in an underlying graph representing the customers’ desired trips. Each RSP also decides the number of AVs to be stationed at each node at each time period to serve the customers’ demands. The number of customers who avail service of a RSP depends on the price selected by the RSP and its competitor. Since the strategy choices available to a RSP depends on its competitor, we seek to compute a Generalized Nash equilibrium (GNE). We show that there may be multiple GNEs. However, when a RSP selects prices in order to deter its competitor when it is not serving a source-destination pair, the game has a potential function and admits a unique GNE. We also compare the competitive prices with a monopoly price where only one RSP is in the market. Numerically, we show that if a network consists of two equal size spatial clusters of demand where the demand between clusters is low, the RSPs may partition the market, i.e, one cluster is served by only one RSP. Hence, the competitive price may become close to the monopoly price.

We consider an intersection zone where autonomous vehicles (AVs) and human-driven vehicles (HDVs) can be simulteneously present. As a new vehicle arrives, the traffic controller needs to decide and suggest an optimal sequence of the vehicles which will exit the intersection zone. The traffic controller can inform the time at which an AV can cross the intersection; however, the traffic controller can not communicate with the HDVs, rather the HDVs can only be controlled using the traffic lights. We formulate the problem as an integer constrained nonlinear optimization problem. Since the number of possible combinations increases exponentially with the number of vehicles in the traffic system, we relax the original problem and proposes an algorithm which gives the optimal solution of the relaxed problem and yet only scales linearly with the number of vehicles in the system. The numerical validation shows that our algorithm outperforms the First-In-First-Out (FIFO) algorithm.

Amazon EC2 provides two most popular pricing schemes–i) thecostlyon-demand instance where the job is guaranteed to be completed, and ii) thecheapspot instance where a job may be interrupted. We consider a user can select a combination of on-demand and spot instances to finish a task. Thus he needs to find the optimal bidding price for the spot-instance, and the portion of the job to be run on the on-demand instance. We formulate the problem as an optimization problem and seek to find the optimal solution. We consider three bidding strategies: one-time requests with expected guarantee, one-time requests with penalty for incomplete job and violating the deadline, and persistent requests. Even without a penalty on incomplete jobs, the optimization problem turns out to be non-convex. Nevertheless, we show that the portion of the job to be run on the on-demand instance is at most half. If the job has a higher execution time or smaller deadline, the bidding price is higher and vice versa. Additionally, the user never selects the on-demand instance if the execution time is smaller than the deadline. The numerical results illustrate the sensitivity of the effective portfolio to several of the parameters involved in the model. Our empirical analysis on the Amazon EC2 data shows that our strategies can be employed on the real instances, where the expected total cost of the proposed scheme decreases over 45 percent compared to the baseline strategy.

The Citizens Broadband Radio Service (CBRS) recently adopted in the U.S. enables two tiers of commercial users to share spectrum with a third tier of incumbent users. This sharing can be further assisted by Environmental Sensing Capability operators (ESCs), that monitor the spectrum occupancy to determine when use of the spectrum will not harm incumbents. Two key aspects of this framework that impact how firms may compete are the differences in information provided by different ESCs and the different tiers in which a firm may access the spectrum. We develop a game theoretic model that captures both of these features and analyze it to gain insight into their impact. Specifically, we consider a priority access (PA) tier firm has access to the both a licensed band and an unlicensed band, and a general authorized access (GAA) tier firm has access only to the unlicensed band. The PA tier and GAA tier firms compete for users. Our analysis reveals that the amount of unlicensed and licensed bandwidth must be chosen judiciously in order to maximize the social welfare. We also show that a limited amount of unlicensed access by the PA tier firm is beneficial to the user's surplus as well as to the social welfare.

We consider users who may have renewable energy harvesting devices or distributed generators. Such users can behave as consumers or producers (hence, we denote them as prosumers) at different time instances. We consider a tiered market where the grid selects a price function, which reveals price in the real time based on the total demand to the grid. In the real time, a prosumer can buy from another prosumer in an exchange market knowing the price from the grid. The exchange price is set by a platform and can be different for different sellers. A prosumer is a selfish entity, which selects the amount of energy it wants to buy either from the grid or from other prosumers or the amount of excess energy it wants to sell to other prosumers by maximizing its own payoff. However, the strategy and the payoff of a prosumer inherently depend on the strategy of other prosumers as a prosumer can only buy if the other prosumers are willing to sell. We formulate the problem as a coupled constrained game and seek to obtain the generalized Nash equilibrium. We show that the game is a concave potential game and show that there exists a unique generalized Nash equilibrium. We propose a distributed algorithm that converges to the exchange price, which clears the market and achieves the generalized Nash equilibrium. We, finally, show how the grid should select the price function in a day-ahead scenario by computing the estimated demand from the history. Our numerical result shows that the tiered market can reduce the peak load and increase the prosumers' total payoffs.

The success of modern ride-sharing platforms crucially depends on the profit of the ride-sharing fleet operating companies, and how efficiently the resources are managed. Further, ride-sharing allows sharing costs and, hence, reduces the congestion and emission by making better use of vehicle capacities. In this paper, we develop a distributed model-free, DeepPool, that uses deep Q-network (DQN) techniques to learn optimal dispatch policies by interacting with the environment. Further, DeepPool efficiently incorporates travel demand statistics and deep learning models to manage dispatching vehicles for improved ride sharing services. Using real-world dataset of taxi trip records in New York, DeepPool performs better than other strategies, proposed in the literature, that do not consider ride sharing or do not dispatch the vehicles to regions where the future demand is anticipated. Finally, DeepPool can adapt rapidly to dynamic environments since it is implemented in a distributed manner in which each vehicle solves its own DQN individually without coordination.

In cloud storage, the digital data is stored in logical storage pools, backed by heterogeneous physical storage media and computing infrastructure that are managed by a cloud service provider (CSP). One of the key advantages of cloud storage is its elastic pricing mechanism, in which the users need only pay for the resources/services they actually use, e.g., depending on the storage capacity consumed, the number of file accesses per month, and the negotiated service level agreement. To balance the tradeoff between service performance and cost, CSPs often employ different storage tiers, for instance, cold storage and hot storage. Storing data in hot storage incurs high storage cost yet delivers low access latency, whereas cold storage is able to inexpensively store massive amounts of data and thus provides lower cost with higher latency. In this paper, we address a major challenge confronting the CSPs utilizing such tiered storage architecture-how to maximize their overall profit over a variety of storage tiers that offer distinct characteristics, as well as file placement and access request scheduling policies. To this end, we propose a scheme where the CSP offers a two-stage auction process for: 1) requesting storage capacity and 2) requesting accesses with latency requirements. Our two-stage bidding scheme provides a hybrid storage and access optimization framework with the objective of maximizing the CSP's total net profit over four dimensions: file acceptance decision, placement of accepted files, file access decision and access request scheduling policy. The proposed optimization is a mixed-integer nonlinear program that is hard to solve. We propose an efficient heuristic to relax the integer optimization and to solve the resulting nonlinear stochastic programs. The algorithm is evaluated under different scenarios and with different storage system parameters, and insightful numerical results are reported by comparing the proposed approach with other profit-maximization models. We see a profit increase of over 60% of our proposed method compared to other baseline algorithms in certain simulation scenarios.

The recent framework for tiered spectrum sharing in the 3.5 GHz band establishes rules in which multiple firms called Environment Sensing Capability operators (ESCs) may measure spectrum occupancy and sell these measurements to other firms to help facilitate spectrum access. Motived by this we consider a scenario in which two spectrum access firms (SAs) seeks to access a shared band of spectrum and must in turn purchase spectrum measurements from one of two ESCs. Given the measurements they purchase, the SA firms then compete on price to serve customers in a shared band of spectrum. We study how differences in the quality and price of the spectrum measurements impact the resulting market equilibrium between the SAs and find that having different qualities of measurements available to different SAs can lead to better economic welfare.

We propose an online pricing mechanism for electric vehicle (EV) charging. A charging station decides prices for each arriving EV depending on the energy and the time within which the EV will be served (i.e. deadline). The user selects either one of the contracts by paying the prescribed price or rejects all depending on their utilities. The charging station has to select a price without knowing the future arrival times of the EVs and the utilities of the EV users. We show that there exists a social welfare pricing strategy, however, the above may not maximize the expected profit of the charging station and even the profit may be 0. We propose a fixed profit pricing strategy which provides a guaranteed fixed profit to the charging station. Numerically, we show that how the charging station can select a profit margin to trade-off between profit and the users' surpluses. We also show empirically that since our proposed mechanism also controls the deadline of the vehicles compared to the existing pricing mechanisms, hence, the number of charging spots required can be lower.

We consider a secondary spectrum market where primaries set prices for their unused channels. The payoff of a primary then depends on the availability of channels for its competitors, which a primary might not have information about. We study a model where a primary can acquire this competitor's channel state information (C-CSI) at a cost. We formulate a game between two primaries, where each primary decides whether to acquire the C-CSI or not and then selects its price based on that. We first characterize the Nash equilibrium of this game for a symmetric model where the C-CSI is perfect. We show that the payoff of a primary is independent of the C-CSI acquisition cost. We then generalize our analysis to allow for imperfect estimation and cases, where the two primaries have different C-CSI costs or different channel availabilities. Our results show interestingly that the payoff of a primary increases when there is estimation error. We also show that surprisingly the expected payoff of a primary may decrease when the C-CSI acquisition cost decreases or primaries have different availabilities.

In a secondary spectrum market primaries set prices for their unused channels to the secondaries. The payoff of a primary depends on the channel state information (CSI) of its competitors. We consider a model where a primary can acquire its competitors CSI at a cost. We formulate a game between two primaries where each primary decides whether to acquire its competitor's CSI or not and then selects its price based on that. Our result shows that no primary decide to acquire its competitor's CSI with an absolute certainty. When the cost of acquiring the CSI is above a threshold, there is a unique Nash Equilibrium (NE) where both the primaries remain uninformed of their respective competitor's CSI. When the cost is below the threshold, in the unique NE each primary randomizes between its decision to acquire the CSI or not. Our result reveals that irrespective of the cost of acquiring the CSI, the expected payoff of a primary remains the same.

We investigate a spectrum oligopoly market where each primary seeks to sell its channel to a secondary. Transmission rate of a channel evolves randomly. Each primary needs to select a price depending on the transmission rate of its channel. Each secondary selects a channel depending on the price and the transmission rate of the channel. We formulate the above problem as a noncooperative game. We show that there exists a unique Nash equilibrium (NE) and explicitly compute it. Under the NE strategy profile, a primary prices its channel to render the channel that provides high transmission rate more preferable; this negates the perception that prices ought to be selected to render channels equally preferable to the secondary regardless of their transmission rates. We show the loss of revenue in the asymptotic limit due to the noncooperation of primaries. In the repeated version of the game, we characterize a subgame perfect NE where a primary can attain a payoff arbitrarily close to the payoff it would obtain when primaries cooperate.

We propose power allocation among femto-base stations (femto-BSs) in a heterogeneous network (HetNet) based on non cooperative games. A minimum level of quality of service has to be guaranteed at macro-user terminals (macro-UTs). Femto-BSs are unaware of the exact values of the channel parameters between them and macro-UTs because of the lack of cooperation and fading. First, we consider the design criterion where the outage probability has to be below a certain threshold at macro-UTs. The equilibrium concept is based on the Normalized Nash Equilibrium (NNE) since it caters to the distributed setting. NNE is unique only for a few strictly concave utility functions in this case. We introduce the concept of Weakly Normalized Nash Equilibrium (WNNE) which keeps the most of the appealing features of NNE but can be extended to a wide class of utility functions and can be incorporated with low complexity. Finally, we consider the design criterion where the expected SINR at a macro-UT has to be greater than a threshold. In this case, the NNE is always unique for any strictly concave utility functions.

We investigate the economics of internet of things (IoT). An economic model of IoT consists of end users, advertisers and three different kinds of providers. We model different kinds of interaction among the providers as a combination of sequential and parallel non-cooperative games. We characterize the equilibrium pricing strategy and payoff of providers and corresponding demands of end users in each such setting. We quantify the impact of advertising revenue on the equilibrium pricing and demands, and compare the payoffs and demands for different interaction models.

We consider heterogeneous networks with multiple femtocells and macrocells. Femto-base stations (femto-BS) are constrained to allocate transmitting powers such that the total interference at each macro-user terminal (macro-UT) is below a given threshold. We formulate a power allocation problem as a concave game with femto-BSs as players and multiple macro-UTs enforcing coupled constraints. Equilibrium selection is based on the concept of normalized Nash equilibrium (NNE). When the interference at a femto-user terminal (femto-UT) from adjacent femto-BSs is negligible, for any strictly concave nondecreasing utility the NNE is unique and the NNE is the solution of a concave potential game. We also propose a distributed algorithm which converges to the unique NNE. When the interference is not negligible, an NNE may not be unique and the computation of NNE has exponential complexity. We introduce the concept of weakly normalized Nash equilibrium (WNNE) which keeps the most of NNEs' interesting properties but, in contrast to the latter, the WNNE can be determined with low complexity. We show the usefulness of the WNNE concept for the relevant case of Shannon capacity as femto-BS's utility.

We investigate a spectrum oligopoly where primary users allow secondary access in lieu of financial remuneration. Transmission qualities of the licensed bands fluctuate randomly. Each primary needs to select the price of its channel with the knowledge of its own channel state but not that of its competitors. Secondaries choose among the channels available on sale based on their states and prices. We formulate the price selection as a non-cooperative game and prove that a symmetric Nash equilibrium (NE) strategy profile exists uniquely. We explicitly compute this strategy profile and analytically and numerically evaluate its efficiency. Our structural results provide certain key insights about the unique symmetric NE.

Design of non-uniform linear antenna arrays is one of the most important electromagnetic optimization problems of current interest. In this article, an adaptive Differential Evolution (DE) algorithm has been used to optimize the spacing between the elements of the linear array to produce a radiation pattern with minimum side lobe level and null placement control. DE is arguably one of the best real parameter optimizers of current interest takes very few control parameters and is easy to implement in any programming language. In this study two very simple adaptation schemes are used to regulate the control parameters F and Cr, upon which the performance of DE is critically dependent. The adaptation schemes are based on the objective function values of the target vectors and donor vectors. The adaptive DE-variant has been used to solve three difficult instances of the design problem and the optimization goal in each example is easily achieved. The results of the proposed algorithm have been shown to meet or beat the recently published results obtained using other state-of-the-art metaheuristics like the Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Memetic Algorithms (MA), and Tabu Search (TS) in a statistically meaningful way.

An Optimal Control is a set of differential equations describing the path of the control variables that minimize the cost functional (function of both state and control variables). Direct solution methods for optimal control problems treat them from the perspective of global optimization: perform a global search for the control function that optimizes the required objective. Invasive Weed Optimization (IWO) technique is used here for optimal control. However, the direct solution method operates on discrete n-dimensional vectors, not on continuous functions, and becomes computationally unmanageable for large values of n. Thus, a parameterization technique is required, which can represent control functions using a small number of real-valued parameters. Typically, direct methods using evolutionary techniques parameterize control functions with a piecewise constant approximation. This has obvious limitations, both for accuracy in representing arbitrary functions, and for optimization efficiency. In this paper a new parameterization is introduced, using Bézier curves, which can accurately represent continuous control functions with only a few parameters. It is combined with Invasive Weed Optimization into a new evolutionary direct method for optimal control. The effectiveness of the new method is demonstrated by solving a wide range of optimal control problems.

Invasive Weed Optimization Algorithm IWO) is an ecologically inspired metaheuristic that mimics the process of weeds colonization and distribution and is capable of solving multi-dimensional, linear and nonlinear optimization problems with appreciable efficiency. In this article a modified version of IWO has been used for training the feed-forward Artificial Neural Networks (ANNs) by adjusting the weights and biases of the neural network. It has been found that modified IWO performs better than another very competitive real parameter optimizer called Differential Evolution (DE) and a few classical gradient-based optimization algorithms in context to the weight training of feed-forward ANNs in terms of learning rate and solution quality. Moreover, IWO can also be used in validation of reached optima and in the development of regularization terms and non-conventional transfer functions that do not necessarily provide gradient information