Unmanned aerial vehicles (UAVs) hold significant potential for sensing services in a large scope of area, thanks to their wide coverage and adaptable deployment. Considering the complex environment dynamics and limited sensing range, navigating multiple UAVs in a distributed way becomes challenging to implement cooperative data sensing and transmission tasks. In this paper, we optimize the trajectory design of UAVs by jointly considering the collected data volume, geographical fairness and limited energy reserve during their service period. To achieve the long-term serving objective, a memory augmented multi-agent deep reinforcement learning approach is presented to ensure energy-efficient distributed trajectory design with partial observations. Specifically, the intrinsic criterion is developed to enhance UAV spatial exploration when reaching the boundary of explored regions. Then, to address the information loss caused by incomplete observations, the spatial-temporal memory augmented actor-critic architecture is designed to extract historical contextual features for multi-UAV cooperative navigation. Furthermore, the prioritized experience replay mechanism is incorporated to enhance important experience exploitation for UAV collaboration. Extensive simulations using two real-world datasets in Shenzhen and Beijing demonstrate that the proposed method outperforms the state-of-the-art methods in terms of data collection ratio, geographical fairness, and energy consumption ratio.

This paper proposes a vehicle following control strategy based on Transformer-enabled offline reinforcement learning, effectively addressing the adaptability problems of traditional control methods in traffic scenarios. We design a Transformer encoding architecture capable of capturing temporal dependencies in the vehicle following, enhancing state representation. By combining policy optimization with behavior cloning, Expert driving knowledge is utilized to optimize the policies without the risks of environmental interactions. Additionally, a velocity-based dynamic safety gap model is constructed and corresponding reward function is designed to balance safety, comfort and efficiency. Quantitative assessments reveal that our proposed method outperforms traditional methodologies and existing learning methods in terms of safety and efficiency metrics, while maintaining equivalence with behavior cloning techniques regarding comfort indices.

Continuous sign language recognition seeks to identify unsegmented sign language from videos by means of a weakly supervised manner, providing only sentence-level labels. In sign language videos, the gestures are smooth and continuous, and the same word may also correspond to video clips of different scales. Therefore, this poses a challenge in accurately capturing complex temporal dependencies. For hearing-impaired service robots, continuous sign language recognition capability is particularly critical, as the robots need to understand the natural sign language expressions of hearing-impaired users in real time. Previous studies have shown that using methods with a time-invariant receptive field for temporal modeling can partially address this issue, but they are not well-suited to handle video clips of varying scales. In this study, we re-examined the temporal modeling schemes in recent CSLR works and proposed the Two-stage Temporal ConvTransformer (T2CT), which fully leverages the advantages of one-dimensional convolutional neural networks and Transformer encoders, adopting a two-stage structure to capture more comprehensive spatiotemporal features. In particular, each stage of the proposed T2CT consists of two parts: a Local Temporal Modeling Module to capture short-term temporal dependencies, and a Global Temporal Modeling Module for long-term temporal modeling. Experimental results on three challenging CSLR datasets demonstrate that the proposed T2CT achieves competitive performance.

Accurate prediction of lithium-ion battery capacity degradation and remaining useful life (RUL) is crucial for battery health management and the safe operation of equipment. However, the diversity of battery types and variations in usage environments pose challenges to data-driven predictive models. Traditional machine learning models often exhibit poor performance in terms of prediction and generalization capabilities. This paper introduces a time-series large model: ANVMD-Llama. The model employs Adaptive Noise Variational Mode Decomposition (ANVMD) to process battery aging data for RUL prediction. Initially, the adaptive noise variational mode decomposition optimizes the tokenization scheme of Lag-Llama, decomposing battery degradation data into multiscale modal components with distinct features to characterize degradation trends and fluctuation properties, aiding the model in understanding fluctuation patterns. Subsequently, ANVMD-Llama is pre-trained on a large dataset of diverse lithium-ion battery degradation data to learn capacity degradation patterns. The model is then fine-tuned using a small amount of data to update the top-level modules, achieving more accurate predictions. Finally, the experimental results demonstrate that the proposed model achieves accurate RUL prediction and exhibits strong transfer capability.

Accurate and effective prediction of battery remaining useful life (RUL) is crucial for the retirement planning of electric buses and the secondary utilization of battery packs. This study utilizes four years of operational data from nine electric buses to achieve precise RUL prediction for power batteries. First, considering the real-world operating characteristics of electric buses, this paper introduces a new RUL definition based on remaining mileage (RML) and remaining days of life (RDL) to characterize the remaining lifespan of battery packs. Subsequently, SOH labeling is conducted using charging data and filtering algorithms, followed by determining the end-of-life point of battery packs from SOH degradation trajectories. Finally, multi-time-scale features—including battery features, historical features, seasonal features, and discharging features are extracted from raw data, and the predictive performance of multiple ensemble learning models is compared. The results indicate that the AdaBoost model achieves the best performance in predicting RML and RDL, with a mean absolute error of 98 days and 16,852 km, respectively.

Mobile crowdsensing (MCS) by human participants and unmanned aerial vehicles (UAVs) is an emerging air-ground collaborative data collection paradigm by navigating a group of UAVs to collaborate with human participants to provide large-scale and fine-grained sensing services. In this paper, we aim to optimize the trajectory design of UAVs by jointly considering the collected data volume, geographical sensing fairness, and limited energy reserve during the serving period. To achieve the long-term serving objective, we propose a human participants distribution prediction based multi-agent deep reinforcement learning method for efficient UAV navigation to collaborate with human participants in performing MCS tasks. Specifically, we first introduce a region division based human participant spatial distribution prediction method to help UAVs to collaborate with human participants by the predictive mobility flows. Then, we present the multi-agent proximal policy optimization (MAPPO) based method for efficient UAV navigation decision-making. Extensive simulations and trajectory visualization using the real-world mobility dataset in KAIST show that the proposed method consistently outperforms the state-of-the-art in terms of the energy efficiency when varying the number of UAVs and human participants.

With the rapid development of vehicle-to-everything technology, vehicular edge caching has emerged as a crucial component for managing frequently accessed content at the network’s edge. However, due to vehicles’ high mobility, it is challenging to determine where and which content needs to be cached. To address this issue, a mobility and context-aware precache strategy is proposed to proactively prefetch and replace content in two steps. First, by integrating the traffic features from vehicles and roads, a spatial-temporal informer-based model is designed to predict long-term vehicle trajectories. Subsequently, a proactive context-aware precache strategy is proposed. By analyzing the context of different cache types, the required content can be further accurately estimated according to the cache type and workload. Extensive simulations based on real-world mobility scenarios are conducted to validate the performance of the proposed method. The results show that the proposed method can improve prediction accuracy and cache hit rate by 34.56% and 18.89%, and reduce mean response time and total energy cost by 6.1% and 2.65% compared to the existing precaching methods.

With the rapid growth of electric vehicle (EV) adoption, ensuring the reliability and safety of their battery systems is of significant importance. Current anomaly detection methods, which rely primarily on temporal dependencies, often overlook spatial patterns, leading to inaccurate detection. This paper introduces a novel autoencoder that incorporates a multi-head attention mechanism for voltage anomaly detection in EV batteries. First, comprehensive datasets from six real EVs operating under diverse and dynamic conditions were collected. Second, to address the complexity and variability of the data, various pre-processing techniques were employed, and the optimal method was selected to ensure data quality and consistency. Third, an improved deep learning autoencoder model was developed, incorporating multiple multi-head attention mechanisms to capture intricate patterns and temporal dependencies within the battery data. The effectiveness of the proposed method is verified by the tests on the comprehensive dataset of real electric vehicles operating under dynamic conditions.

The rapid and accurate detection of faults for lithium-ion batteries plays a critical role in ensuring the safe operation of electric vehicle systems. This paper proposes a fault detection method for electric vehicle batteries by exploiting the temporal smoothness and spatial similarity of battery pack data. Firstly, a temporal convolutional network (TCN) is utilized to learn the latent spatial and temporal features of the data. Then, a self-attention mechanism is employed to capture the correlations and importance between different features. Furthermore, an autoencoder is employed to reconstruct input data based on the extracted spatial-temporal features. This encoder-structured approach is trained only using normal data and the anomalies are detected as conspicuous differences between the input data and the reconstructed data. The detection performance of the proposed method is validated by utilizing real-world electric vehicle operational datasets.

Temperature heavily affects the lifespan and performance of batteries. High temperatures accelerate capacity degradation and can cause thermal runaway, highlighting the importance of battery cooling strategies for electric vehicles. This paper proposes the battery cooling strategy for electric vehicles with LiFePO4batteries using the twin delayed deep deterministic policy gradient (TD3) algorithm. Firstly, the electric-thermal-aging and active battery thermal management system models are introduced, and the thermal management problem is formulated as a continuous Markov decision process. Then, a reward function is designed based on prolonging battery life, reducing refrigeration cost, and maintaining battery temperature. Finally, a TD3 algorithm based on the double-delayed update mechanism is designed to obtain the optimal battery cooling strategy in the continuous state-action space. Simulation results demonstrate that the proposed strategy outperforms the traditional methods and closes to the optimal benchmark, offline dynamic programming, with battery capacity loss of less than 3.32% and an SoC consumption of less than 0.55%, significantly reducing the operational cost and mitigating the battery aging.

Microgrids have emerged as a promising solution to improve the resilience and reliability of power systems. However, unexpected faults can still pose significant challenges to the stable operation of microgrids. This paper proposes a stochastic programming-based strategy for mitigating faults in a DC microgrid. First, a scheduling strategy is implemented for normal operations and switched to a mitigation strategy upon detecting faults, which ensures optimal performance and resilience. Second, a scenario-based stochastic programming enhanced by the DBSCAN algorithm for scenario reduction is designed, which addresses the optimization problem efficiently. Comprehensive simulations are conducted to evaluate the proposed scheduling and mitigation strategies, demonstrating their advantages under both normal and fault conditions.

It is a significant challenge to reconcile the competing interests of individual microgrid units when energy is shared in a multi-energy system. Moreover, the majority of these systems solely focus on power sharing, without adequately addressing the associated carbon emissions. This paper puts forward a sharing approach that makes use of multi-agent collaboration to optimise energy sharing in multi-energy systems and achieve low-carbon operation. Firstly, a system model is constructed to describe the interrelationship between each microgrid and the energy-sharing platform. Subsequently, a shared pricing mechanism will be devised to incentivise each microgrid to prioritise its participation in the local sharing market. The shared pricing mechanism is employed to construct the microgrid utility maximisation problem and transform it into a Markov game process. A multi-agent cooperative approach is put forth as a means of optimising the energy-sharing strategy. Ultimately, the simulation results demonstrate that this energy-sharing strategy can satisfy the operational constraints, maintain equilibrium between supply and demand, reduce energy costs, and promote the economic and low-carbon operation of the microgrid.

In truck platoons with a bidirectional-leader topology, variations in channel conditions result in unreliability and high latency in vehicle-to-vehicle (V2V) communications. This paper proposes an adaptive relay selection strategy based on Q-learning (QL). The strategy ensures that all vehicles in the platoon receive safety messages from the lead vehicle quickly and reliably. Firstly, relay selection is modeled as a Markov decision process (MDP). The lead vehicle and the relays act as intelligent agents. Agents make decisions adaptively based on real-time state observations in a dynamic communication environment. Secondly, a reward function is designed based on platoon topology and channel state information statistics (CSI). The purpose is to drive the proposed strategy to learn the optimal strategy for message transmission under different environments. Lastly, the simulation results demonstrate the effectiveness and robustness of the proposed algorithm. In various channel attenuation environments, the strategy has been demonstrated to enhance the packet delivery ratio (PDR) for the platoon tail and significantly increase the platoon’s throughput.

Lithium-ion batteries are increasingly critical in portable and electrical technologies, making accurate estimations of their State of Health essential for extending battery life and ensuring safety. This study proposes a novel State of Health estimation model that integrates transfer learning with a Retentive Network architecture, addressing the limitations of traditional methods in data processing and feature extraction. By pre-training a model adaptable to various battery types, the approach leverages the Retentive Network’s powerful temporal data processing capabilities, enabling end-to-end application to raw data and reducing the need for complex preprocessing. The model utilizes a dual-stream parallel network architecture to extract valuable information from both charging and discharging data, significantly enhancing the accuracy of State of Health estimation. Furthermore, the model can quickly adapt to new or sparsely sampled battery types through pre-training and fine-tuning strategies, minimizing reliance on large labeled datasets. Compared with existing transfer learning approaches, the proposed model demonstrates superior performance in terms of accuracy and robustness, particularly in handling long-term degradation patterns. Extensive experiments on multiple public datasets show that the model consistently achieves a Mean Absolute Error below 0.90% and a Root Mean Square Error below 0.95%, confirming its effectiveness and precision. This research provides novel insights and methodologies for optimizing battery performance and improving management practices.

The energy scheduling strategy for microgrids based on reinforcement learning plays a very crucial role in realizing low-carbon and economic energy utilization. Considering the traditional reinforcement learning is difficult to meet the operational constraints of microgrids, this paper proposes a safety reinforcement learning method based on proximal policy optimization. Firstly, ladder-type carbon trading is introduced to strictly constrain the system carbon emission. Then, a safety proximal policy optimization method is proposed that incorporates a safety network to decouple the economic and safety factors in the traditional reward composition. Finally, experiments are conducted on real-world datasets to verify the effectiveness of the proposed algorithm. The experimental results illustrate that the proposed safety reinforcement learning method is able to minimize the economic cost and carbon emission of microgrid scheduling while strict guaranteeing safety compared to existing methods.

Predicting accurate capacity degradation and re-maining useful life (RUL) of lithium-lon battery is critical to health management and safe operation. However, variations in operating conditions and the variety of battery types present challenges to data-driven predictive models. Most data-driven methods rely on traditional machine learning models, which often have constrained predictive and generalization abilities. In this paper, a foundation model: Lag-Llama is used to predict capacity and RUL of battery with auto-correlation analysis. Firstly, the tokenization scheme of Lag-Llama is improved by auto-correlation analysis, which calculate the most probable periods in history capacity sequence. It is helpful for model to comprehend the capacity fluctuation pattern. Then, Lag-Llama is pre-trained to learn battery capacity degradation, and thus calculate the RUL. Additionally, the model is fine-tuned with a small amount of data to update the top-level module for application to the target cell. Finally, experimental results show that the proposed model exhibits accurate RUL prediction and strong transfer capability, within the average mean square error and absolute error less than 0.035 and 9 respectively.

Model predictive control is a real-time energy management method for hybrid energy storage systems, whose performance is closely related to the prediction horizon. However, a longer prediction horizon also means a higher computation burden and more predictive uncertainties. This paper proposed a predictive energy management strategy with an optimized prediction horizon for the hybrid energy storage system of electric vehicles. Firstly, the receding horizon optimization problem is formulated to minimize the battery degradation cost and traction electricity cost for the electric vehicle operation. Then, the optimal control sequence is solved to obtain the power allocation between the battery and the supercapacitor. Furthermore, the effect of different horizons on the optimization results is analyzed under diverse operating conditions, determining the optimal horizon to balance the system costs and computation burden. Compared with the short horizon, the optimal horizon can achieve 5.2%$\sim$8.5% performance improvement with the acceptable computation time approaching 1 s.

In the future 6G intelligent transportation system, the edge server will bring great convenience to the timely computing service for connected vehicles. To guarantee the quality of service, the time-critical services need to be migrated according to the future location of the vehicle. However, predicting vehicle mobility is challenging due to the time-varying of road traffic and the complex mobility patterns of vehicles. To address this issue, a spatial-temporal awareness proactive service migration strategy is proposed in this paper. First, a spatial-temporal neural network is designed to obtain accurate mobility by using gated recurrent units and graph convolutional layers extracting features from spatial road traffic and multi-time scales driving data. Then a proactive migration method is proposed to guarantee the reliability of services and reduce energy consumption. Considering the reliability of services and the real-time workload of servers, the migration problem is modeled as a multi-objective optimization problem, and the Lyapunov optimization method is utilized to obtain utility-optimal migration decisions. Extensive simulations based on real-world datasets are performed to validate the performance of the proposed method. The results show that the proposed method achieved 6% higher prediction accuracy, 10% lower dropping rate, and 10% lower energy consumption compared to state-of-the-art methods.

Battery degradation is a main hinder to extend the persistent lifespan of the portable heterogeneous computing device. Excessive energy consumption and prominent current fluctuations can lead to a sharp decline of battery endurance. To address this issue, a battery-aware workflow scheduling algorithm is proposed to maximize the battery lifetime and release the computing potential of the device fully. Firstly, a dynamic optimal budget strategy is developed to select the highest cost-effectiveness processors to meet the deadline of each task, accelerating the budget optimization by incorporating deep neural network. Second, an integer-programming greedy strategy is utilized to determine the start time of each task, minimizing the fluctuation of the battery supply current to mitigate the battery degradation. Finally, a long-term operation experiment and Monte Carlo experiments are performed on the battery simulator, SLIDE. The experimental results under real operating conditions for more than 1800 hours validate that the proposed scheduling algorithm can effectively extend the battery life by 7.31%-8.23%. The results on various parallel workflows illustrate that the proposed algorithm has comparable performance with speed improvement over the integer programming method.

High-reliability and low-latency communication is essential for timely information exchange in vehicle platooning. As a key enabler of this, the cellular vehicular-to-everything (C-V2X) network uses a sensing-based semi-persistent scheduling (SPS) protocol, where radio resources are reserved for a number of transmissions with reduced resource re-allocation and control overhead. However, consecutive access collisions may be caused by reservation conflict, which leads to long delay and threatens platoon’s stability and safety. In this paper, a coordinating resource reservation (CRR) protocol is proposed for vehicle platooning. By implementing error detection with coordination among platoon vehicles, the resource reservation is improved for reduced collisions and delay. Specifically, packet reception/loss information is sent out by platoon vehicles through their own packets. Such information is shared with transmitters and guides them to reserve new resources when access collision occurs. As a result, long delay is avoided while no extra feedback packet is introduced. Furthermore, Markov analysis is presented to evaluate the performance of SPS and the proposed CRR for vehicle platooning, providing the quantified performance gains. Finally, simulation results demonstrate the superiority of the proposed CRR in reducing packet loss and latency, compared with the legacy SPS and other state-of-the-art solutions.

The increasing number of connected vehicles exacerbates the scarcity of spectrum resources in vehicle-to-everything (V2X) communication. To optimize the utilization of wireless resources, it is crucial to allocate the limited spectrum blocks to each roadside unit (RSU) based on the real-time bandwidth demand of vehicles within their coverage. However, the complex mobility patterns of vehicles and dynamic traffic conditions make it challenging to accurately and promptly estimate the bandwidth demand. To address this issue, a spatial-temporal multi-attentional network (STMA-net) is designed to predict the future bandwidth demand of RSUs. Based on the predicted bandwidth demand, a prediction error-compensable proactive bandwidth allocation algorithm is proposed to adaptively allocate spectrum resources and narrow the discrepancy between predicted and actual demand. Experimental results with realistic traffic in Bologna demonstrate that the proposed STMA-net achieves 11.25% higher prediction accuracy compared to state-of-the-art methods. Furthermore, the proposed proactive bandwidth allocation method outperforms existing methods, providing the highest throughput and serving 5% more vehicles while reducing the service drop rate by an order of magnitude.

With the development of renewable energy, much attention has been paid to improving energy efficiency. This paper proposes a residential microgrid scheduling method to improve the energy utilization rate of residential buildings. Firstly, the power cost model for residential users and the specific charge-discharge models of electric vehicles and energy storage equipment is constructed. Then the whole energy scheduling process is formulated as a multi-variable mixed integer linear programming (MV-MILP) problem, whose objective is to minimize the power cost of end-users. The differential evolution algorithm is adopted to solve the problem, and the scaling factor adaptation is further used to accelerate the convergence. Simulation results demonstrate the effectiveness of the proposed scheduling method.

The driving performance of electric vehicles seriously degrades due to the deterioration of lithium-ion batteries at low temperatures. Preheating lithium-ion batteries can effectively improve the driving range of electric vehicles at subzero temperatures. In this paper, an optimal pulse heating strategy is proposed for low-temperature heating of lithiumion battery. Firstly, this paper establishes a coupling model to describe the electro-thermal-aging behavior of battery. Secondly, the heating time and capacity loss jointly form a multi-objective optimization problem with the current constraint. The optimization problem is solved by using the particle swarm optimization(PSO) algorithm and the effect of weighting coefficient on heating performance is discussed to obtain the optimal pulse current. The results show that the proposed strategy can effectively reduce heating time without causing serious capacity reduction.

For autonomous vehicles and smart transportation services, information exchange and fusion with low latency and high reliability is critical. The 3rd Generation Partnership Project has released the cellular vehicle-to-everything (C-V2X) Mode 4 to enable direct vehicle-to-vehicle communications regardless of the cellular coverage. Mode 4 uses the sensing-based semi-persistent resource scheduling (SPS) to support autonomous resource selection by vehicles. However, channel access collisions lead to packet losses, especially in crowded scenarios. Thus, an accurate analytical model is essential to quantify the system performance, reveal how to mitigate collision and ensure system reliability and scalability. This paper focuses on the analytical modeling of the SPS and derives the access collision ratio considering both the sensed and hidden terminals in V2X. Extended simulations are conducted to verify the correctness of the analytical framework. In addition, we investigate the impact of system parameters on performance, which provides important guidelines for improving the system configuration.

The application of nonorthogonal multiple access (NOMA) technology to New Radio (NR) Vehicles-to-Everything (V2X) can further reduce communication delay and improve the system capacity of the vehicular network. However, due to the high mobility of the vehicles, it is difficult for the base station to obtain the channel information between vehicles in real time for resource allocation and vehicle transmission power control. To this end, we propose a two-stage scheme of centralized resource allocation and distributed power control to meet the NR V2X Mode 1 requirement while NOMA technology is employed in each vehicle group. First, at the beginning of a transmission period, the base station allocates resources to vehicle groups. For this centralized manner, we propose a graph-based matching approach to allocate resources to improve system capacity. Then, each vehicle group controls and adjusts transmission power for NOMA communication in the period. For this distributed manner, we propose a noncooperative game to control the power of the vehicle groups and then analyze the performance of the noncooperative game. Afterward, we further put forward a cooperative game approach to control the power of the vehicle group to increase system capacity. Compared with the noncooperative game, the proposed scheme can increase communication capacity by up to 5% and reduce transmission power consumption by 36%.

Infrastructure-as-a-Service clouds in geo-distributed data centers offer various pricing options, including on-demand and reserved instances, which provide an elastic and cost-effective infrastructure to support High Performance Computing (HPC) applications. In this paper, we propose an instance reservation based cloud service framework, modeling the cost-minimizing reservation decision issue as an NP-hard integer programming problem for distributed data centers. To ease its computation complexity, two algorithms are proposed to minimize the HPC service cost with the worst-case performance guarantees: an offline heuristic-greedy algorithm, and a rolling-horizon based online algorithm when only short-term demand prediction is available. Facing fluctuating demands, instance reservation in a single data center may incur the highly underutilized capacity. To address this issue for further cost reduction, we extend the scheme with a novel cloud broker federation based resource sharing mechanism, reallocating already reserved but unused instances to computation-intensive and short-lived tasks for continuous execution without interruption. Extensive evaluations driven by large-scale trace-based datasets demonstrate that the proposed mechanism can effectively handle large volumes of service requests, saving considerable service costs with higher reservation resource utilization.

Traditional lighting systems suffer from the problem of low energy efficiency and low illumination quality due to its disappointing management. To address this issue, in this paper, a novel traffic-flow adaptive scheme of smart lighting systems is proposed on the basis of the cyber-physical cloud system. The cyber-physical cloud system consists of the digital twin and cyber-physical system. The operation of the lighting system is simulated in the counterpart twin system with the digital twin technology. The cyber-physical system realizes data collection, information interaction, analysis, and processing, as well as complex computation and remote control. The traffic adaptive scheme works according to the brightness sequence to improves the energy efficiency of the lighting system and provide higher illumination quality for drivers. Extensive simulation results verify the proposed control scheme could improve the energy efficiency of lighting systems.

Home energy management plays a key role in demand response for residential customers to reduce the total cost via scheduling household loads energy consumption. However, excessive energy consumption by customers will bring a great challenge to the stability of the grid. To address the challenge, a day-ahead multi-agent reinforcement learning method is proposed for home energy management under a peak power-limiting. We first formulate the total cost minimization problem as a Markov game, and then a novel household loads energy consumption scheduling algorithm is proposed based on Mutil-agent Deep Deterministic Policy Gradient (MADDPG). It is worth mentioning that the proposed algorithm can achieve cooperation between agents so that it can meet the peak power-limiting constraint. Simulation results are provided in this paper to show the effectiveness of the proposed method.

The vehicular service support with adaptability, real-time, and low delay is crucial for connected vehicles. However, due to limited coverage of mobile edge computing servers and data processing capability of connected vehicles, vehicular services need to be offloaded to the edge server and adaptively migrate as the connected vehicle moves. Aiming at the adaptive migration service, a deep Q-learning service migration decision algorithm is proposed in this paper. The proposed algorithm can dynamically adjust the vehicular service migration decision according to traffic information. Furthermore, a service migration framework consisting of neural networks is proposed in this paper to improve the adaptability and real-time performance of the algorithm. By using this framework, training and decision-making can be carried out simultaneously in different places. Finally, compared with the two existing algorithms, extensive simulations are conducted to verify the effectiveness of the proposed algorithm.

In geo-distributed cloud storage systems, data replication has been widely used to serve the ever more users around the world for high data reliability and availability. How to optimize the data replica placement has become one of the fundamental problems to reduce the inter-node traffic and the system overhead of accessing associated data items. In the big data era, traditional solutions may face the challenges of long running time and large overheads to handle the increasing scale of data items with time-varying user requests. Therefore, novel offline community discovery and online community adjustment schemes are proposed to solve the replica placement problem in a scalable and adaptive way. The offline scheme can find a replica placement solution based on the average read/write rates for a certain period of time. The scalability can be achieved as 1) the computation complexity is linear to the amount of data items and 2) the data-node communities can evolve in parallel for a distributed replica placement. Furthermore, the online scheme is adaptive to handle the bursty data requests, without the need to completely override the existing replica placement. Driven by real-world data traces, extensive performance evaluations demonstrate the effectiveness of our design to handle large-scale datasets.

The accurate estimation of adhesion force between wheels and rails is an important task as it helps to the wheel-slip prevention (WSP) system preventing the wheels from locking and reducing the stopping distance. Influenced by a changing external environment, the adhesion force estimation process is complex. Thus, an extended state observer (ESO) is proposed to accurately estimate the adhesion force of railway vehicles with the modeling deviation and measurement noise. With the estimated modeling error information, an auxiliary compensation part is designed to eliminate the steady-state estimation error causing by the modeling deviation. Further, a Fal function filter is added to the ESO to deal with the effect of measurement noise. The convergence of the proposed estimation method is analyzed theoretically. The effectiveness of the designed algorithm is corroborated by simulation comparisons to other standard approaches.

Precision stopping of trains requires special attention to brake control because a pneumatic brake system of a train is highly nonlinear, hybrid and uncertain. Existing solutions to automatic train stop control ignore the pneumatic brake system or simply treat it as a system delay, which is quite far apart from the real train stopping dynamics. Moreover, the service life of pneumatic brake systems decreases fast due to the frequent changes in output of existing controllers. Thus, an adaptive nonlinear sliding mode control method is developed in this paper, which has strong applicability to nonlinear and hybrid system control synthesis due to its natural variable structure characteristic. A nonlinear integral sliding surface with adaptive updating parameters is proposed to improve the control precision and the robustness. Extensive simulations are performed to validate the effectiveness of the proposed method. The results show that the proposed algorithm outperforms a PID control algorithm in terms of stopping error and expected lifetime of pneumatic brake systems.

Microgrids incorporated with distributed generation (DG) units and energy storage (ES) devices are expected to play more and more important roles in the future power systems. Yet, achieving efficient distributed economic dispatch in microgrids is a challenging issue due to the randomness and nonlinear characteristics of DG units and loads. This paper proposes a cooperative reinforcement learning algorithm for distributed economic dispatch in microgrids. Utilizing the learning algorithm can avoid the difficulty of stochastic modeling and high computational complexity. In the cooperative reinforcement learning algorithm, the function approximation is leveraged to deal with the large and continuous state spaces. And a diffusion strategy is incorporated to coordinate the actions of DG units and ES devices. Based on the proposed algorithm, each node in microgrids only needs to communicate with its local neighbors, without relying on any centralized controllers. Algorithm convergence is analyzed, and simulations based on real-world meteorological and load data are conducted to validate the performance of the proposed algorithm.

With the wide application of Lithium-ion (Li-ion) batteries in electric vehicles and unmanned aerial vehicles (UAVs), it is becoming more important and urgent to estimate the battery state to extend the operation range of electric vehicles or UAVs. Existing state of charge (SOC) estimation methods are highly model-based, which are difficult to be implemented in different scenarios. In this paper, we propose a generalized extended state observer (GESO) based SOC estimation method, where the accurate model is unnecessary, which can be effectively tracked by the observer. Thus, a first order RC model is utilized in GESO to capture the characteristics of Li-ion batteries. By appropriately designing a disturbance compensation gain, the GESO is applied for the nonintegral-chain system that is subject to uncertainties and nonlinear parameters of Li-ion batteries. Experiment results show that the proposed method has a good performance and robustness on SOC estimation of the battery. The SOC estimation results are found to be consistent with the reference SOC with less chattering than sliding mode observer, where the error is within 2% under both the known and unknown initial SOC value cases.

With the proliferation of renewable energy sources and the elevation of environmental concerns, it is expected that microgrids will become one of the major means for residential energy supply. However, the distributed nature of microgrid operation brings new technical challenges to energy management. Endowing wireless communication capability to the distributed generation (DG) units and energy storage (ES) devices in a microgrid is beneficial for their cooperation without a centralized controller. Yet, how to establish distributed energy management without \emph{a priori} statistical information for all the DG units and loads still requires extensive research. In this paper, a reinforcement learning algorithm with cooperative neural fitting iteration is proposed for distributed energy management in microgrids via wireless networks. The reinforcement learning algorithm leverages a distributed actor- critic structure to adopt the continuous states and action spaces of a microgrid. A diffusion strategy is incorporated in the reinforcement learning algorithm to coordinate the actions of DG units and ES devices by exchanging their evaluations and decisions via a wireless network. Simulation results based on realistic renewable power generation and load data are presented to evaluate the performance of the proposed algorithm.

Offloading computational intensive applications to cloud broker is a promising solution for overcoming the limitation of computational resources and energy of mobile devices. The cloud broker sends the jobs to the nearest local cloudlet to guarantee the least delay for the applications. Most of the existing studies consider a fixed amount of virtual machines (VMs) that the local cloudlet would like to rent. In reality, the jobs arriving at the cloud broker and the jobs submitted by the interior users to the cloudlet are both dynamic. In order to address this issue, we use the stochastic game approach in this paper to improve the long-term revenue of cloud broker and local cloudlet. We model the jobs arriving at the queue as a stochastic progress. Then, the cloud broker and the local cloudlet sent a number of jobs to the virtual machine pool in each time slot. Both of them ensure that the submitted jobs are completed, and the cloudlet guarantees that each of the jobs submitted by interior users is executed. We prove the existence of the equilibrium in this game and develop an algorithm to calculate the c-Nash equilibrium. The proposed approach is carefully evaluated based on usage the price information of Amazon EC2. The simulation results indicate that the stochastic game approach can improve the revenue for both cloud broker and local cloudlet.

In cloud market, cloud providers offer diverse service options, including on-demand instances and reserved instances. Generally reserved instance price is cheaper than on-demand instance, but excessive reservation may result in high capacity underutilization. To improve resource utilization rate and reduce providers' service cost, a steady broker federation is necessary to propose. Broker federation coordinate cloud providers service demands in geo-distributed data centers through deciding when and how many instances to reserve. Firstly, the service cost optimization problem is formulated as a nonlinear integer programming model. Then a hybrid algorithm combining ant colony with particle swarm optimization is presented to reduce computational complexity and providers service cost. Extensive simulations driven by large-scale Parallel Workloads Archive demonstrate the effectiveness and efficiency of the hybrid algorithm.

With the explosion of application of big data, it becomes inefficient and infeasible to process big data stream by using conventional data service infrastructure and management system. Cloud computing platform having multiple geo-distributed data centers is expected to be the most efficient platform to process the big data stream. In this paper, a multi-round combinational double auction based mechanism is proposed to allocate the resources of geo-distributed data centers to multiple users with large data stream processing tasks. This mechanism combines the advantages of combinatorial auction and double auction. In the proposed mechanism, the different types of VMs can be integrated into a bundle to be bid. The auction is double and conducted from both users and data centers. Different from existed double auctions, the QoS level is taken into consideration. In addition, the multiple rounds mode is adopted, so the failed users and data centers have the chance to adjust bids and asks to participate next auction round, increasing the ratio of successful transactions. Simulation results validate the effectiveness of the proposed mechanism.

Abstract Introducing orthogonal frequency division multiplexing (OFDM) into cognitive radio (CR) can potentially increase the spectrum efficiency, but it also leads to further challenges for the resource allocation of CR networks. In OFDM‐based cognitive relay networks, two of the most significant research issues are subcarrier selection and power allocation. In this paper, a non‐cooperative game model is proposed to maximize the system throughput by jointly optimizing subcarrier selection and power allocation. First, taking the direct and relay links into consideration, an equivalent channel gain is presented to simplify the cooperative relay model into a non‐relay model. Then, a variational inequality method is utilized to prove the existence and uniqueness of the Nash equilibrium solution of the proposed non‐cooperative game. Moreover, to compute the solution of the game, a suboptimal algorithm based on the Lagrange function and distributed iterative water‐filling algorithm is proposed. The proposed algorithm can jointly optimize the process of subcarrier selection and power allocation. Finally, simulation results are shown to demonstrate the effectiveness of the proposed joint subcarrier selection and power allocation scheme. Copyright © 2015 John Wiley & Sons, Ltd.

In cloud computing, Infrastructure-as-a-Service (IaaS) cloud providers can offer two types of purchasing plans for cloud users, including on-demand plan and reservation plan. Generally reservation price is cheaper than on-demand price, while reservation plan may cause highly underutilized capacity problem. How to joint optimize the service cost and the resource utilization for clouds is a critical issue. To address this issue, a novel steady broker federation is developed to coordinate service demands in this paper. And the optimal reservation problem can be formulated as a nonlinear integer programming model. Then a fine-grained heuristic algorithm is proposed to reduce its computational complexity and obtain quasi-optimal solutions. Numerical simulations driven by large-scale Parallel Workloads Archive demonstrate that the proposed approach can save considerable costs for cloud users and improves the resource utilization for IaaS cloud providers.

The traffic signal control is the basic method to solve the urban congestions problem coming with the accelerating urbanization. For large city, it is challenge to improve the traffic signal control flexibility to adapt the real-time traffic change while to decrease the computation complexity. This paper proposes a cooperative Q-learning with function approximation(CQFA) algorithm for vehicle to infrastructure (V2I) networks. By gathering the local intersection traffic information from V2I networks and employing cooperative behaviors with neighboring intersections, the algorithm can achieve the optimal policy without any central supervising agents. To address the curse of dimensionality effectively, the Q-learning function is approximated by using a fast gradient-descent function approximation method to pick out the optimal Q-learning action. The Q-learning with Function Approximation algorithm combining the cooperative mechanism balances the urban traffic flow and uses approximating strategy to decrease the computation dimensionality. It can improve the traffic throughout, reduce the average waiting time and avoid congestions. Simulation results verify the effectiveness of the proposed algorithm.

The Electronically Controlled Pneumatic (ECP) brake system is being applied to the heavy-haul trains for its control accracy and real time performance. A high power supply with high efficiency and reliability is essential to the safe operation of the ECP brake system. In this paper, a push-pull forward converter with a secondary full bridge rectifier is proposed for the ECP power supply. The voltage stress of the rectifier diodes can be reduced by a modified nodissipative snubber. Furthermore, hybrid control methods and criteria are utilized to improve the converter performance, including forced flux balancing circuit design, power supply output impedance and noise requirements. Finally, A 2.5kW prototype of the converter with efficiency up to 93% verifies the proposed circuit and theoretical analysis.

In wireless sensor networks, node energy resources are so limited that how to reduce energy consumption and prolong network lifetime become the primary factor that should be taken into account for the design of wireless sensor network routing protocols. In the single sink wireless sensor networks, the failure of sink will lead to entire network paralysis and reduce network reliability. In multi-sink wireless sensor networks, based on the mechanism of saving node energy and balancing flow of base stations, the zone of every sink in the network is divided. In each subnetwork, a location and energy based dynamical pre-clustering algorithm LEBDPC is proposed. LEBDPC not only balances energy consumption of nodes in the same cluster, but also energy consumption of nodes in different clusters. Simulation results show that compared with other clustering algorithms, LEBDPC algorithm can effectively reduce energy consumption and prolong network lifetime.