The planning problem constitutes a fundamental aspect of the autonomous driving framework. Recent strides in representation learning have empowered vehicles to comprehend their surrounding environments, thereby facilitating the integration of learning-based planning strategies. Among these approaches, Imitation Learning stands out due to its notable training efficiency. However, traditional Imitation Learning methodologies encounter challenges associated with the co-variate shift phenomenon. We propose Validity Learning on Failures, VL(on failure), as a remedy to address this issue. The essence of our method lies in deploying a pre-trained planner across diverse scenarios. Instances where the planner deviates from its immediate objectives, such as maintaining a safe distance from obstacles or adhering to traffic rules, are flagged as failures. The states corresponding to these failures are compiled into a new dataset, termed the failure dataset. Notably, the absence of expert annotations for this data precludes the applicability of standard imitation learning approaches. To facilitate learning from the closed-loop mistakes, we introduce the VL objective which aims to discern valid trajectories within the current environmental context. Experimental evaluations conducted on both reactive CARLA simulation and non-reactive log-replay simulations reveal substantial enhancements in closed-loop metrics such as Score, Progress, and Success Rate, underscoring the effectiveness of the proposed methodology. Further evaluations against the Bench2Drive benchmark demonstrate that VL(on failure) outperforms the state-of-the-art methods by a large margin.

Trajectory augmentation serves as a means to mitigate distributional shift in imitation learning. However, imitating trajectories that inadequately represent the original expert data can result in undesirable behaviors, particularly in safety-critical scenarios. We propose a trajectory augmentation method designed to maintain similarity with expert trajectory data. To accomplish this, we first cluster trajectories to identify minority yet safety-critical groups. Then, we combine the trajectories within the same cluster through geometrical transformation to create new trajectories. These trajectories are then added to the training dataset, provided that they meet our specified safety-related criteria. Our experiments exhibit that training an imitation learning model using these augmented trajectories can significantly improve closed-loop performance.

We study the data collection problem in an Internet of Things (IoT) network where an unmanned aerial vehicle (UAV) is utilized to aggregate data from a set of IoT devices. We formulate the scheduling and path planning problems for the UAV. The goal of the scheduling problem is to find the sequence of nodes that the UAV will visit to complete the data collection task in the shortest possible time, ensuring that it does not run out of energy during its mission. We express this problem as a mixed-integer nonlinear problem and propose an efficient algorithm to solve the aforementioned NP-hard problem in polynomial time. Path planning problem aims to find a collision-free path for the UAV. While the state-of-the-art schemes have focused on solving the path planning problem in static environments, we study the problem in a dynamic environment with moving obstacles. We develop an algorithm that works on both static and dynamic environments. Our method combines deep reinforcement learning (RL) with graph-based global path planning algorithms to find a collision-free path for the UAV. One important advantage of our RL-based method over the existing studies is its map independency, which allows us to transform the agent’s learning from one environment to another. Via simulation studies, we show that our method is significantly effective in improving the safety of the path planning algorithms in dynamic environments.

To prolong the lifetime of the unmanned aerial vehicles (UAVs), the UAVs need to fulfill their missions in the shortest possible time. In addition to this requirement, in many applications, the UAVs require a reliable internet connection during their flights. In this paper, we minimize the travel time of the UAVs, ensuring that a probabilistic connectivity constraint is satisfied. To solve this problem, we need a global model of the outage probability in the environment. Since the UAVs have different missions and fly over different areas, their collected data carry local information on the network's connectivity. As a result, the UAVs can not rely on their own experiences to build the global model. This issue affects the path planning of the UAVs. To address this concern, we utilize a two-step approach. In the first step, by using Federated Learning (FL), the UAVs collaboratively build a global model of the outage probability in the environment. In the second step, by using the global model obtained in the first step and rapidly-exploring random trees (RRTs), we propose an algorithm to optimize UAVs' paths. Simulation results show the effectiveness of this two-step approach for UAV networks.

This paper studies the trajectory optimization problem for an aerial vehicle with the mission of flying between a pair of given initial and final locations. The objective is to minimize the travel time of the aerial vehicle ensuring that the communication connectivity constraint required for the safe operation of the aerial vehicle is satisfied. We consider two different criteria for the connectivity constraint of the aerial vehicle which leads to two different scenarios. In the first scenario, we assume that the maximum continuous time duration that the aerial vehicle is out of the coverage of the ground base stations (GBSs) is limited to a given threshold. In the second scenario, however, we assume that the total time periods that the aerial vehicle is not covered by the GBSs is restricted. Based on these two constraints, we formulate two trajectory optimization problems. To solve these non-convex problems, we use an approach based on the double Q-learning method which is a model-free reinforcement learning technique and unlike the existing algorithms does not need perfect knowledge of the environment. Moreover, in contrast to the well-known Qlearning technique, our double Q-learning algorithm does not suffer from the over-estimation issue. Simulation results show that although our algorithm does not require prior information of the environment, it works well and shows near optimal performance.

Aerial vehicles have recently attracted significant attention in a variety of commercial and civilian applications due to their high mobility, flexible deployment and cost-effectiveness. To leverage these promising features, the aerial users have to satisfy two critical requirements: First, they have to maintain a reliable communication link to the ground base stations (GBSs) throughout their flights, to support command and control data flows. Second, the aerial vehicles have to minimize their propulsion power consumption to remain functional until the end of their mission. In this paper, we study the trajectory optimization problem for an aerial user flying over an area including a set of GBSs. The objective of this problem is to find the trajectory of the aerial user so that the total propulsion-related power consumption of the aerial user is minimized while a cellular-connectivity constraint is satisfied. This problem is a non-convex mixed integer non-linear problem and hence, it is challenging to find the solution. To deal with, first, the problem is relaxed and reformulated to a more mathematically tractable form. Then, using successive convex approximation (SCA) technique, an iterative algorithm is proposed to convert the problem into a sequence of convex problems which can be solved efficiently.

In this paper, the trajectory optimization problem for a multi-aerial base station (ABS) communication network is investigated. The objective is to find the trajectory of the ABSs so that the sum-rate of the users served by each ABS is maximized. To reach this goal, along with the optimal trajectory design, optimal power and sub-channel allocation is also of great importance to support the users with the highest possible data rates. To solve this complicated problem, we divide it into two sub-problems: ABS trajectory optimization sub-problem, and joint power and sub-channel assignment sub-problem. Then, based on the Q-learning method, we develop a distributed algorithm which solves these sub-problems efficiently, and does not need significant amount of information exchange between the ABSs and the core network. Simulation results show that although Q-learning is a model-free reinforcement learning technique, it has a remarkable capability to train the ABSs to optimize their trajectories based on the received reward signals, which carry decent information from the topology of the network.

This paper investigates the power minimization problem for the user terminals by application offloading in multi-cell multi-user OFDMA mobile cloud computing networks where some practical constraints such as backhaul capacity limitation, interference level on each channel and maximum tolerable delay as user's quality of service is taken into account. Furthermore, the mixed integer nonlinear problem is converted into a convex form using D.C. approximation. Moreover, to solve the optimization problem, we have proposed joint power allocation and decision making (J-PAD) algorithm which can make offloading decision and allocate power at the same time. Simulation results illustrate that by utilizing the J-PAD algorithm, in comparison with baselines, considerable power saving could be achieved e.g. about 30% for delays more than 100 ms.

We investigate the optimal power allocation problem for a femtocell network where the utilized air interface is single carrier frequency division multiple access (SC-FDMA). To overcome the interference, the receivers are equipped by frequency domain equalizers designed based on the zero forcing criterion. We derive the optimal power allocation when we protect the data rate of macro users by imposing a temperature limit on the interference arising from femto users to the macro base station. This constraint confines the cross-tier interference. We show that the optimization problem is convex and solve it using dual decomposition technique to obtain the optimal power of each femto user.