High gradient variance presents a significant obstacle to efficient post-training of large language models (LLMs) on memory-constrained devices. Existing practical strategies-such as reducing batch sizes or adopting gradient accumulation (GA)-suffer from an inherent trade-off: smaller batches exacerbate convergence issues due to increased gradient noise, while GA substantially prolongs training time owing to its sequential processing. In this work, we reveal that the Exponential Moving Average (EMA) in momentum-based optimizers exponentially discounts historical gradients, thereby limiting their effectiveness in stabilizing parameter updates, especially during post-training when parameter drift is minimal. Motivated by this, we propose integrating the core idea of GA directly into momentum updates via a novel Periodical Moving Average (PMA) mechanism, which structures training into fixed periods and replaces EMA with a uniform moving average within each period. We instantiate PMA within AdamW and Lion, resulting in the AdamW-PMA and Lion-PMA optimizers. Theoretical analysis establishes that AdamW-PMA matches the convergence guarantees of standard Adam. Extensive empirical evaluation on supervised fine-tuning and direct preference optimization tasks demonstrates that PMA-based methods achieve approximately $2\times$ faster training compared to GA, while yielding consistently better performance on downstream evaluations.

Distribution system state estimation (DSSE) plays a crucial role in the real-time monitoring, control, and operation of distribution networks. Besides intensive computational requirements, conventional DSSE methods need high-quality measurements to obtain accurate states, whereas missing values often occur due to sensor failures or communication delays. To address these challenging issues, a forecast-then-estimate framework of edge learning is proposed for DSSE, leveraging large language models (LLMs) to forecast missing measurements and provide pseudo-measurements. First, natural language-based prompts and measurement sequences are integrated by the proposed LLM to learn patterns from historical data and provide accurate forecasting results. Second, a convolutional layer-based neural network model is introduced to improve the robustness of state estimation under missing measurement. Third, to alleviate the overfitting of the deep-learning-based DSSE, it is reformulated as a multitask learning framework containing shared and task-specific layers. The uncertainty weighting algorithm is applied to find the optimal weights to balance different tasks. The numerical simulation on the Simbench case is used to demonstrate the effectiveness of the proposed forecast-then-estimate framework.

Abstract This paper presents a robust framework for handling a conic multiobjective linear optimization problem, where the objective and constraint functions are involving affinely parameterized data uncertainties. More precisely, we examine optimality conditions and calculate efficient solutions of the conic robust multiobjective linear problem. We provide necessary and sufficient linear conic criteria for efficiency of the underlying conic robust multiobjective linear program. It is shown that such optimality conditions can be expressed in terms of linear matrix inequalities and second-order conic conditions for a multiobjective semidefinite program and a multiobjective second order conic program, respectively. We show how efficient solutions of the conic robust multiobjective linear problem can be found via its conic programming reformulation problems including semidefinite programming and second-order cone programming problems. Numerical examples are also provided to illustrate that the proposed conic programming reformulation schemes can be employed to find efficient solutions for concrete problems including those arisen from practical applications.

The federated learning (FL) technique can provide a promising solution for the timely training of a deep learning model with the critical requirement of privacy protection. However, the existing FL frameworks still confront challenging issues including heterogeneous data sources, edge device heterogeneity, sensitive information leakage, nonconvex loss, and communication resource constraints which place obstacles in terms of practicality. In this article, first, a federated multitask learning (FedMTL) approach is introduced to reformulate the FL model as a multiobjective optimization problem which results in federated multigradient descent algorithm (FedMGDA) with a better model personalization against data heterogeneity and Byzantine attack. Second, a new semi-asynchronous model aggregation method is developed to asynchronously aggregate small partial clients for compensating impacts of the straggler and staleness. Third, a distributed differential privacy technique is applied to enhance the privacy protection of asynchronous FedMGDA with the convergence guarantee where the convergence analysis of differentially private asynchronous federated multiple gradient descent algorithm (DP-AsynFedMGDA) is studied for both the convex and the nonconvex loss functions. Empirical examples and comparative studies are presented to illustrate the effectiveness of the proposed DP-AsynFedMGDA.

Industrial manufacturing processes present unique challenges in implementing demand response due to their complex equipment interactions, diverse operation modes, and safety constraints. Conventional model-based optimization methods often struggle in this context, requiring complete mathematical models and accurate uncertainty distributions. In this article, we propose a model-free safe deep reinforcement learning approach for real-time scheduling of industrial manufacturing systems, ensuring constraint adherence and accommodating diverse equipment operation modes. Specifically, the approach formulates the industrial demand response problem as a constrained Markov decision process with hybrid action space, capturing the intricate interplay between variable-speed equipment and discrete actuators, while considering safety constraints. To enhance exploration and robustness, the proposed method combines Lagrange multipliers based on soft actor–critic to satisfy the constraints. The cross-attention mechanism is utilized to find the association rules between hybrid actions to solve the challenges posed by the spatial gradient of hybrid actions. The proposed approach is trained and tested on a real-world dataset, demonstrating its superior performance in achieving significant cost reductions for manufacturing while satisfying operational constraints. Furthermore, sensitivity analysis underpins robustness against the variable real-time prices, showcasing its industrial applicability.

The integration of photovoltaic (PV) systems into the grid has increased, resulting in various opportunities and challenges related to voltage fluctuation and the control of distributed energy resources (DERs). Smart inverters can protect against voltage violations by implementing active and reactive power control. Each smart inverter can be represented as an agent in deep reinforcement learning (DRL), which uses data to learn control actions for a given distribution network. The agent’s training must address the challenges of achieving stable learning and ensuring fair PV curtailment, while the voltage unbalance in a practical three-phase unbalanced distribution network must remain within the acceptable range. The proposed model-free method effectively addresses the problems by adopting a hybrid approach that combines gradient and evolutionary parameter update schemas to learn a robust policy for safe and fair PV curtailment. We successfully implemented the proposed method to achieve voltage regulation close to an ideal 1 per unit (p.u.) and a voltage unbalance factor within a 2% range. The PVC is guaranteed to be regulated fairly, implying that there will be a proportional but minimal to no reduction in low PV generation periods.

Abstract In this paper, we consider a polynomial problem with equilibrium constraints in which the constraint functions and the equilibrium constraints involve data uncertainties. Employing a robust optimization approach, we examine the uncertain equilibrium constrained polynomial optimization problem by establishing lower bound approximations and asymptotic convergences of bounded degree diagonally dominant sum-of-squares (DSOS), scaled diagonally dominant sum-of-squares (SDSOS) and sum-of-squares (SOS) polynomial relaxations for the robust equilibrium constrained polynomial optimization problem. We also provide numerical examples to illustrate how the optimal value of a robust equilibrium constrained problem can be calculated by solving associated relaxation problems. Furthermore, an application to electric vehicle charging scheduling problems under uncertain discharging supplies shows that for the lower relaxation degrees, the DSOS, SDSOS and SOS relaxations obtain reasonable charging costs and for the higher relaxation degrees, the SDSOS relaxation scheme has the best performance, making it desirable for practical applications.

Many machine learning models are naturally multitask, which may involve regression and classification tasks, in which they can be trained by the multitask network to yield a more generalized model with the aid of correlated features. When these learning models are deployed on Internet-of-Things devices, the computation efficiency and the privacy of the data can pose a significant challenge to developing a federated learning (FL) algorithm for both higher learning performance and better privacy protection. In this article, a new FL framework is proposed for a class of multitask learning problems with hard parameter-sharing model through which the learning tasks are reformulated as a multiobjective optimization problem for better performance. Specifically, the stochastic multiple gradient descent approach and differential privacy are integrated into this FL algorithm for achieving a Pareto optimality that obtains a good tradeoff among different learning tasks while providing data protection. The outstanding performance of this algorithm is demonstrated by the empirical experiments on multiMINIST, the Chinese city parking dataset, and Cityscapes dataset.

Traffic accident prediction plays a vital role in Intelligent Transportation Systems (ITS), where a large number of traffic streaming data are generated on a daily basis for spatiotemporal big data analysis. The rarity of accidents and the absent interconnection information make it hard for spatiotemporal modeling. Moreover, the inherent characteristic of the black box predictive model makes it difficult to interpret the reliability and effectiveness of the deep learning model. To address these issues, a novel self-explanatory spatial-temporal deep learning model–Attention Spatial-Temporal Multi-Graph Convolutional Network (ASTMGCN) is proposed for traffic accident prediction. The original recorded rare accident data is formulated as a multivariate irregularly interval-aligned dataset, and the temporal discretization method is used to transfer into regularly sampled time series. Multiple graphs are defined to construct edge features and represent spatial relationships when node-related information is missing. Multi-graph convolutional operators and attention mechanisms are integrated into a Sequence-to-Sequence (Seq2Seq) framework to effectively capture dynamic spatial and temporal features and correlations in multi-step prediction. Comparative experiments and interpretability analysis are conducted on a real-world data set, and results indicate that our model can not only yield superior prediction performance but also has the advantage of interpretability.

Location Dependent Mobile Crowdsensing (LDMC) often needs to collect data at different time points in various regions to ensure the coverage of sensing data. An incentive mechanism is needed to encourage participants to move to sparse areas and improve coverage. However, there are two problems: 1) most incentive mechanisms assume that the participants can get accurate information about tasks; 2) those mechanisms encourage participants through absolute utility so that the platform can obtain an improvement of incentive effect by increasing the reward. However, nodes usually get inaccurate information in reality. Moreover, behavioral economics finds that decision-making is often affected by relative utility rather than absolute utility. Path-dependence means that choices made on the basis of transitory conditions can persist long after those conditions change, which can solve the above problems. This study uses cognitive bias and the reference effect to explain the principle of path-dependence, and proposes a mechanism called Task Coverage promotion based on Path-dependence (TCPD). TCPD cultivates the cognitive bias of participants, causing an overestimation of expected utility. Then, it sets dynamic reference points to prevent participants from quitting early. The simulation results show that TCPD can improve the coverage and effectiveness of the platform.

Federated learning techniques provide a promising framework for collaboratively training a machine learning model without sharing users’ data, and delivering a security solution to guarantee privacy during the model training of IoT devices. Nonetheless, challenges posed by data heterogeneity and communication resource constraints make it difficult to develop an efficient federated learning algorithm in terms of the low order of convergence rate. It could significantly deteriorate the quality of service for critical machine learning tasks, e.g., facial recognition, which requires an edge-ready, low-power, low-latency training algorithm. To address these challenges, a communication-efficient federated learning approach is proposed in this paper where the momentum technique is leveraged to accelerate the convergence rate while largely reducing the communication requirements. First, a federated multi-task learning framework by which the learning tasks are reformulated by the multi-objective optimization problem is introduced to address the data heterogeneity. The multiple gradient descent algorithm is harnessed to find the common gradient descending direction for all participants so that the common features can be learned and no sacrifice on each clients’ performance. Second, to reduce communication costs, a local momentum technique with global information is developed to speed up the convergence rate, where the convergence analysis of the proposed method under non-convex case is studied. It is proved that the proposed local momentum can actually achieve the same acceleration as the global momentum, whereas it is more robust than algorithms that solely rely on the acceleration by the global momentum. Third, the generalization of the proposed acceleration approach is investigated which is demonstrated by the accelerated variation of FedAvg. Finally, the performance of the proposed method on the learning model accuracy, convergence rate, and robustness to data heterogeneity, is investigated by empirical experiments on four public datasets, while a real-world IoT platform is constructed to demonstrate the communication efficiency of the proposed method.

Real-time License Plate Recognition plays a significant role in traffic congestion control and road safety monitoring. Practically, a network camera may capture multiple license plates in one frame while the data collected by different network cameras cannot be shared due to privacy concern. In this paper, a federated learning framework is introduced to simultaneously detect multiple license plates over different network cameras through semantic communication. Specifically, to achieve a high efficiency of multiple license plates recognition in real time, the semantic segmentation model is applied to locally extract the important features of an image with multiple license plates. And then, an autoencoder is developed to carry out the semantic encoding which translates the meaningful information. Moreover, a multi-task learning approach for multiple license plates recognition is proposed through a multi-objective optimization technique which can train the license plate recognition model with stronger generalization. To improve the reliability, an asynchronous federated learning algorithm is also considered to ensure the training process can be tolerant to the transmission delay. Empirical experiments on the Chinese City Parking Dataset (CCPD) show that the proposed approach can effectively improve the recognition performance while providing robust service.

Fuel cell prognostic has been considered an efficient way for the health management of fuel cell system and contribute to its lifespan. Nevertheless, accurate fuel cell prognostic is a challenge task because the lifespan of fuel cell is influenced by the operating conditions and there could be reversible aging due to the electrochemical reaction nature of fuel cell. In addition, conventional data-driven prognostic method trains a prediction model for each working condition without leveraging the common aging features for various conditions and thus may yield a less generalized model. What's more, training a promising data-driven prognostic model require large amount of data, which is hard to be collected by one institution due to the high cost and long-term degradation nature of fuel cell, whereas collecting aging data from different entities face the challenges of privacy concern. To address the above-mentioned issue, this paper proposes a private reversible aging-aware method. To address the reversible aging issue, a hint function based method is proposed to add indication terms in the input feature to help the model better deal with the reversible aging. After that, a multitask learning approach is considered to enable the institution to learn shared representations of the aging data, where the multitask learning is reformulated by federated learning to protect the data privacy of each institution. Experiment on the PHM2014 data challenge prove the effectiveness and robustness of the proposed method.

The end-edge-cloud orchestration of the virtual power plant (VPP) enables the edge server to timely serve community users. By deploying the community energy storage system (CESS) and the community peer-to-peer (P2P) market, prosumers can form energy communities to achieve self-sufficiency of energy and independence from fuel-based power generators. This article proposed a two-stage community energy trading model under end-edge-cloud orchestration. The community P2P trading is the first stage where the edge server can execute the automatic bidding process for multiple buyers and sellers based on the real-time users’ energy profiles and the Bayesian-game-based pricing mechanism. The trading between the retailer and energy communities is the second stage where the edge server can dynamically update the optimal operation of the CESS based on the dynamic pricing mechanism. An original centralized optimization problem is decomposed into subproblems for each stakeholder and solved through the alternating direction method of multipliers (ADMM). Considering ADMM needs multiple information exchanges, a general form of the communication-censored ADMM for sharing problems is proposed to decrease the communication cost. Numerical simulations prove that the proposed mechanism can effectively increase transaction efficiency, avoid the new demand peak brought by the utilization of the CESS, and decrease the communication cost.

This paper investigates two accelerated primal-dual mirror dynamical approaches for smooth and nonsmooth convex optimization problems with affine and closed, convex set constraints. In the smooth case, an accelerated primal-dual mirror dynamical approach (APDMD) based on accelerated mirror descent and primal-dual framework is proposed and accelerated convergence properties of primal-dual gap, feasibility measure and the objective function value along with trajectories of APDMD are derived by the Lyapunov analysis method. Then, we extend APDMD into two distributed dynamical approaches to deal with two types of distributed smooth optimization problems, i.e., distributed constrained consensus problem (DCCP) and distributed extended monotropic optimization (DEMO) with accelerated convergence guarantees. Moreover, in the nonsmooth case, we propose a smoothing accelerated primal-dual mirror dynamical approach (SAPDMD) with the help of smoothing approximation technique and the above APDMD. We further also prove that primal-dual gap, objective function value and feasibility measure along with trajectories of SAPDMD have the same accelerated convergence properties as APDMD by choosing the appropriate smooth approximation parameters. Later, we propose two smoothing accelerated distributed dynamical approaches to deal with nonsmooth DEMO and DCCP to obtain accelerated and efficient solutions. Finally, numerical and comparative experiments are given to demonstrate the effectiveness and superiority of the proposed accelerated mirror dynamical approaches.

The integration of distributed energy resources (DER) into the smart grid through digitalization has transformed the power grid into a more decentralized system, enhancing energy efficiency and resilience during significant catastrophes. However, the integration of a large number of DERs into the transmission and distribution networks poses reliability challenges due to the intermittent nature of renewable energy sources. To tackle this, smart grids have been using advanced metering infrastructure (AMI) and Internet of Things (IoT) devices for over two decades to improve grid observability and enable near-real-time forecasting of continent-wide anomalies. Cloud-edge-end orchestrated computing can achieve hierarchical management and innovative operational strategies, such as multi-level control and optimization of all-grid-level DERs, voltage and frequency regulation using phasor measurement units (PMU), and special protection schemes (SPS) to detect and prevent potential faults or large-scale cyber-attacks.

The spread of electric vehicles (EVs) could reduce greenhouse gas emissions and achieve sustainable travel patterns. However, the rapidly increasing charging demand will bring challenges to the operation of charging stations and power systems. Therefore, a two-stage EV management scheme is introduced in this article to overcome these challenges and promote sustainable transport. A charging dispatch model based on fuzzy multicriteria decision making is proposed in the first stage, where users' preferences are in the form of intuitionistic fuzzy sets to address the fuzziness and uncertainty of subjective factors and human judgment. A$\sigma$-cut similarity matrix is proposed to increase the users' satisfaction by excluding options with lower similarity. In the second stage, a noncooperative game model is proposed to incentive EVs to participate in supplementary frequency regulation (SFR). A fuzzy set is employed to reflect users' willingness to adjust charging power. The existence and uniqueness of the Nash equilibrium are investigated. Moreover, a distributed proximal best response algorithm with linear convergence is employed to find Nash equilibrium. Numerical simulations indicate that the proposed method can reduce charging costs while meeting users' preferences and facilitate EVs to participate in SFR.

Many e-commerce platforms, such as AliExpress, run major promotion campaigns regularly. Before such a promotion, it is important to predict potential best sellers and their respective sales volumes so that the platform can arrange their supply chains and logistics accordingly. For items with a sufficiently long sales history, accurate sales forecast can be achieved through the traditional statistical forecasting techniques. Accurately predicting the sales volume of a new item, however, is rather challenging with existing methods; time series models tend to overfit due to the very limited historical sales records of the new item, whereas models that do not utilize historical information often fail to make accurate predictions, due to the lack of strong indicators of sales volume among the item's basic attributes. This article presents the solution deployed at Alibaba in 2019, which had been used in production to prepare for its annual "Double 11" promotion event whose total sales amount exceeded U.S. $ 38 billion in a single day. The main idea of the proposed solution is to predict the sales volume of each new item through its connections with older products with sufficiently long sales history. In other words, our solution considers the cross-selling effects between different products, which has been largely neglected in previous methods. Specifically, the proposed solution first constructs an item graph, in which each new item is connected to relevant older items. Then, a novel multitask graph convolutional neural network (GCN) is trained by a multiobjective optimization-based gradient surgery technique to predict the expected sales volumes of new items. The designs of both the item graph and the GCN exploit the fact that we only need to perform accurate sales forecasts for potential best-selling items in a major promotion, which helps reduce computational overhead. Extensive experiments on both proprietary AliExpress data and a public dataset demonstrate that the proposed solution achieves consistent performance gains compared to existing methods for sales forecast.

Many actual industrial production processes are dynamic and uncertain. When uncertain information are described by subjective experience and experts' knowledge based on scanty or vague information, fuzzy uncertainty exists. Fuzzy chance-constrained dynamic programming are applicable to industrial production modeling accompanied by fuzzy uncertainty and dynamics, where constraints need not or cannot be completely satisfied. In this article, a fuzzy chance-constrained dynamic optimization (FCCDO) formulation on the basis of credibility theory is established, in which, the credibility is used to measure the fuzzy uncertainty level of constraints. To solve the FCCDO problem (FCCDOP), an improved fuzzy simulation technique based on Hammersley sequence sampling is raised to transform fuzzy chance constraints to their deterministic equivalents, and then a data-driven state transition algorithm (DDSTA) using deep neural networks (DNNs) is put forward to achieve a stable, global and robust optimization performance. Finally, the successful applications of the FCCDO method to industrial studies demonstrate its advantages.

Graph neural architecture search (NAS) has gained great popularity in automatically designing powerful graph neural networks (GNNs) with superior learning abilities, significantly relieving human effort and expertise reliance. Despite the advanced performance of automated learning, existing graph NAS models mainly work on single-relational graphs, while the widespread multi-relational graphs in real-world applications, are not well addressed. Moreover, current search spaces of automated GNNs are generally coarse-grained by simply integrating typical GNN layers and hyper-parameters, resulting in severe limitations on search capacities and scopes for creating innovative GNN architectures. To tackle the limitations of single-relational setting and coarse-grained search space design in existing graph NAS, in this paper, we propose a novel framework of multi-relational graph neural architecture search, dubbed MR-GNAS, to automatically develop innovative and excellent multi-relational GNN architectures. Specifically, to enlarge search capacities and improve search flexibility, MR-GNAS contains a fine-grained search space that embraces the full-pipe multi-relational message passing schema, enabling expressive architecture search scopes. With the well-designed fine-grained search space, MR-GNAS constructs a relation-aware supernet with a tree topology, to jointly learn discriminative node and relation representations. By searching with a gradient-based strategy in the supernet, the proposed MR-GNAS could derive excellent multi-relational GNN architectures in multi-relational graph analysis. Extensive experiments on entity classification and link prediction tasks over multi-relational graphs illustrate the effectiveness and superiority of the proposed method.

Memristive LSTM networks have been proven as a powerful Neuromorphic Computing Architecture (NCA) for various time series forecasting tasks and are recognized as the next generation of AI. However, a lack of model explainability makes it hard to properly interpret forecasting results for existing memristive LSTM networks, which makes this NCA unreliable, unaccountable and untrustworthy. In this paper, an interpretable memristive (IM) LSTM network design is proposed for time series forecasting, where the mixture attention technique is embedded into IM-LSTM cells for characterizing the variable-wise feature and the temporal importance. The updating rules and training approach are also presented for this interpretable memristive LSTM network. We evaluate this approach on a probabilistic residential load forecasting task incorporating PV. By improving model interpretability, the most influential predictive factors can be verified by Built Environment domain experts, demonstrating the effectiveness of our design.

Automatic License plate recognition (ALPR) remains a challenging task in face of some difficulties such as multi-line character distribution and license plate (LP) deformation due to camera angles. Most existing ALPR methods either focus on single-line LP or perform horizontal multi-line LP detection and recognition with character-level annotations. In this paper, we propose a novel end-to-end irregular license plate recognition (EILPR) to detect and recognize the LP of multi-line text or arbitrary shooting angles, using only plate-level annotations for training. In EILPR, a coarse-to-fine strategy is adopted to extract the LP features accurately for sequence recognition. Firstly, a coarse rectangular box of the LP is located, along with the corresponding predicted LP class which is single-line or double-line. Then, considering the fact that a LP mainly generates perspective distortion in the image due to its rigid feature, we propose a new automatic perspective alignment network (APAN) to extract the fine LP features connecting the detection and recognition. For recognition, a location-aware 2D attention based recognition network is performed to recognize the multi-line and multinational LP based on the extracted features. Experiments on several datasets show that EILPR achieves the state-of-the-art performance, demonstrating the effectiveness of the proposed method.

This paper introduces a distributed compensation approach for the global optimization with separable objective functions and coupled constraints. By employing compensation variables, the global optimization problem can be solved without the information exchange of coupled constraints. The convergence analysis of the proposed algorithm is presented with the convergence condition through which a diminishing step-size with an upper bound can be determined. The convergence rate can be achieved at O(lnT/√T). Moreover, the equilibrium of this algorithm is proved to converge at the optimal solution of the global optimization problem. The effectiveness and the practicability of the proposed algorithm is demonstrated by the parameter optimization problem in smart building.

This paper presents the impulsive average quasi-consensus algorithm for distributed constrained convex optimization. First, the constrained optimization problem can be transformed into an unconstrained problem using the interior point method, and then a distributed algorithm is modeled by means of impulsive differential equation. In the framework of the continuous-time gradient method and algebraic graph theory, each agent can deal with one local objective function with local constraints. At the impulsive instants, each agent can communicate with its neighboring agents over the network. Under certain conditions, the impulsive average quasi-consensus is achieved. It is shown that the state of average quasi-consensus is the optimal solution of the aforementioned unconstrained optimization problem, and the state of each agent can also reach the neighborhood of the optimal solution. Finally, two numerical examples show the effectiveness of the proposed impulsive average quasi-consensus algorithm. Moreover, the feasibility of the approach is verified by an application to one sensor network localization problem.

Faster convergence is always sought in many applications. Designing fixed-time control has recently gained much attention since, for this type of control structure, the convergence time of the states does not depend on initial conditions, unlike other control methods providing faster convergence. This paper proposes a new distributed algorithm for second-order consensus in multiagent systems by using a full-order fixed-time convergent sliding surface. The stability analysis is performed using the Lyapunov function and bi-homogenous property. Moreover, the proposed control is smooth and free from any singularity. The robustness of the proposed scheme is verified both in the presence of Lipschitz disturbances and uncertainties in the network. The proposed method is compared with a state-of-the-art method to show the effectiveness.

In proof-of-work (PoW)-based blockchain networks, the miners contribute their distributed computation in solving a crypto-puzzle competition to win the reward. To secure stable profits, some miners organize mining pools and share the rewards from the pool in proportion to each miner's contribution. However, some miners may exhibit malicious behaviors which cause a waste of distributed computation resource, even posing a threat on the efficiency of blockchain networks. In this paper, we propose a new game-theoretic framework to incentivize miners mining honestly and help to bring about a higher total welfare of blockchain networks. We first formulate the mining process as a noncooperative iterated game. We then propose a mechanism in terms of zero-determinant strategies (ZD strategies) to encourage the cooperative mining and improve the efficiency of mining in PoW-based blockchain networks. In addition, we theoretically analyze the maximum system welfare of the target pool through the method of optimization. Numerical illustrations are also presented to support our theoretical results.

Semantic segmentation of point cloud data, an efficient 3-D scattered point representation, is a fundamental task for various applications, such as autonomous driving and 3-D telepresence. In recent years, deep learning techniques have achieved significant progress in semantic segmentation, especially in the 2-D image setting. However, due to the irregularity of point clouds, most of them cannot be applied to this special data representation directly. While recent works are able to handle the irregularity problem and maintain the permutation invariance, most of them fail to capture the valuable high-dimensional local feature in fine granularity. Inspired by fuzzy mathematical methods and the analysis on the drawbacks of current state-of-the-art works, in this article, we propose a novel deep neural model, Fuzzy3DSeg, that is able to directly feed in the point clouds while maintaining invariant to the permutation of the data feeding order. We deeply integrate the learning of the fuzzy neighborhood feature of each point into our network architecture, so as to perform operations on high-dimensional features. We demonstrate the effectiveness of this network architecture level integration, compared with methods of the fuzzy data preprocessing cascading neural network. Comprehensive experiments on two challenging datasets demonstrate that the proposed Fuzzy3DSeg significantly outperforms the state-of-the-art methods.

The group consensus problems of heterogeneous multiagent networks with input time delays are investigated in this paper. In this complex networks, the agents' dynamics are modeled by the first-order and the second-order multiagent systems, where a novel dynamic group consensus protocol is designed through the competitive relationship among the agents. By using matrix theory and the frequency-domain method, some algebraic criteria are theoretically established for reaching a group consensus in the following two cases: 1) with the identical and 2) different input time delays. Meanwhile, the conservative assumptions existed in the relevant literatures are absolutely relaxed, i.e., the in-degree balance and the geometric multiplicity of zero eigenvalue of Laplacian matrix are at least two. Finally, the effectiveness of our results is illustrated by numerical examples.

In recommender systems, the user-item interaction data is usually sparse and not sufficient for learning comprehensive user/item representations for recommendation. To address this problem, we propose a novel dual-bridging recommendation model (DBRec). DBRec performs latent user/item group discovery simultaneously with collaborative filtering, and interacts group information with users/items for bridging similar users/items. Therefore, a user's preference over an unobserved item, in DBRec, can be bridged by the users within the same group who have rated the item, or the user-rated items that share the same group with the unobserved item. In addition, we propose to jointly learn user-user group (item-item group) hierarchies, so that we can effectively discover latent groups and learn compact user/item representations. We jointly integrate collaborative filtering, latent group discovering and hierarchical modelling into a unified framework, so that all the model parameters can be learned toward the optimization of the objective function. We validate the effectiveness of the proposed model with two real datasets, and demonstrate its advantage over the state-of-the-art recommendation models with extensive experiments.

Based on the multiagent networks, this paper introduces a continuous-time algorithm to deal with distributed convex optimization. Using nonsmooth analysis and algebraic graph theory, the distributed network algorithm is modeled by the aid of a nonautonomous differential inclusion, and each agent exchanges information from the first-order and the second-order neighbors. For any initial point, the solution of the proposed network can reach consensus to the set of minimizers if the graph has a spanning tree. In contrast to the existing continuous-time algorithms for distributed optimization, the proposed model holds the least number of state variables and relaxes the strongly connected weighted-balanced topology to the weaker case. The modified form of the proposed continuous-time algorithm is also given, and it is proven that this algorithm is suitable for solving distributed problems if the undirected network is connected. Finally, two numerical examples and an optimal placement problem confirm the effectiveness of the proposed continuous-time algorithm.

In this brief, considering the inertial term into first order neural networks(NNs), an inertial NN(INN) modeled by means of a differential inclusion is proposed for solving linear complementarity problem with P0matrix. Compared with existing NNs, the presence of the inertial term allows us to overcome some drawbacks of many NNs, which are constructed based on the steepest descent method, and this model is more convenient for exploring different optimal solution. It is proved that the proposed NN is stable in the sense of Lyapunov and any equilibrium of our NN is the optimal solution of LCP with P0matrix. Simulation results on two numerical examples show the effectiveness and performance of the proposed neural network.

Demand response (DR) can provide a cost-effect approach for reducing peak loads while renewable energy sources (RES) can result in an environmental-friendly solution for solving the problem of power shortage. The increasingly integration of DR and renewable energy bring challenging issues for energy policy makers, and electricity market regulators in the main power grid. In this paper, a new two-stage stochastic game model is introduced to operate the electricity market, where Stochastic Stackelberg-Cournot-Nash (SSCN) equilibrium is applied to characterize the optimal energy bidding strategy of the forward market and the optimal energy trading strategy of the spot market. To obtain a SSCN equilibrium, sampling average approximation (SAA) technique is harnessed to address the stochastic game model in a distributed way. By this game model, the participation ratio of demand response can be significantly increased while the unreliability of power system caused by renewable energy resources can be considerably reduced. The effectiveness of proposed model is illustrated by extensive simulations.

Increasing penetration of plug-in electric vehicles (PEVs) has a substantial impact on the operation of power distribution networks. Given the fast-growing load demands from PEVs and unmatched infrastructure investment in transformer and feeder capacity, the PEV charging is subjected to both spatially and temporally security constraints beyond which the network failure may occur. This paper proposes a game-theory-based distributed charging control method to coordinate large-scale PEVs without compromising the security of the distribution network. Under a noncooperative game framework, a price-driven charging model is designed to minimize the cost of each individual PEV customer while satisfying the network loading constraints. Then, a Newton-type method is developed to find a better Nash equilibrium of the game model at a superlinear convergence rate. Furthermore, an accelerated gradient method is proposed to tackle the subproblem for each user's best response. The update of the user's best response is implemented in a distributed way in order to protect user's privacy. The convergence rate of the proposed algorithms is rigorously proved. The effectiveness and efficiency of the proposed methods are tested on the IEEE 13-bus system.

The resource allocation problem is studied and reformulated by a distributed interior point method via a -logarithmic barrier. By the facilitation of the graph Laplacian, a fully distributed continuous-time multiagent system is developed for solving the problem. Specifically, to avoid high singularity of the -logarithmic barrier at boundary, an adaptive parameter switching strategy is introduced into this dynamical multiagent system. The convergence rate of the distributed algorithm is obtained. Moreover, a novel distributed primal-dual dynamical multiagent system is designed in a smart grid scenario to seek the saddle point of dynamical economic dispatch, which coincides with the optimal solution. The dual decomposition technique is applied to transform the optimization problem into easily solvable resource allocation subproblems with local inequality constraints. The good performance of the new dynamical systems is, respectively, verified by a numerical example and the IEEE six-bus test system-based simulations.

This paper proposes two second-order continuous-time algorithms to solve the economic power dispatch problem in smart grids. The collective aim is to minimize a sum of generation cost function subject to the power demand and individual generator constraints. First, in the framework of nonsmooth analysis and algebraic graph theory, one distributed second-order algorithm is developed and guaranteed to find an optimal solution. As a result, the power demand constraints can be kept all the time under appropriate initial condition. The second algorithm is under a centralized framework, and the optimal solution is robust in the sense that different initial power conditions do not change the convergence of the optimal solution. Finally, simulation results based on five-unit system, IEEE 30-bus system, and IEEE 300-bus system show the effectiveness and performance of the proposed continuous-time algorithms. The examples also show that the convergence rate of second-order algorithm is faster than that of first-order distributed algorithm.

In this paper, a framework of convex optimization algorithm with a fixed time convergence rate is investigated. Given a strongly convex optimization problem, two control algorithms are developed to solve the problem within a fixed time of which the upper bound is theoretically obtained. Moreover, the fixed time convergence rate based algorithms are extended into the distributed manner which is applied to two typical distributed optimization problems including the resource allocation problem and the coordination optimization problem. Laplacian graph matrix is employed to the weighted gradient based and the coordination based distributed optimization algorithms. By developing the characteristic of the objective function, the upper bound of the fixed time convergence is derived. Two numerical examples are given to verify the main results.

There has been a desirable trend in recent years towards Electric Vehicles (EVs) contributing less air pollution and noise pollution than internal combustion engine vehicle. EV charging/discharging problem brings a new challenge to the power operation and control. In this paper, the charging/discharging problem is modelled by noncooperative game theory, the payoff function of this model not only maximizes the revenue of discharging activity, but also minimizes overall generation cost by decreasing electricity price on peak hours. The result of simulation illustrates that charging behaviors can shift the charging demand from peak hours to off-peak hours, while discharging behaviors can shave the peak loads in the parking periods.

A distributed control law based on Model predictive Control (MPC) scheme is proposed for secondary frequency control in a deregulated market, which utilizes Demand Response (DR) along with Automatic generation Control (AGC). The proposed strategy of combining DR and AGC is termed as Load frequency Control (LFC) in the paper. The main contribution of the paper focuses on developing a model for LFC, which combines DR as well as Governor Response (GR) as manipulated variables. The new model is then used in an embedded integrator based distributed MPC algorithm to optimally choose between the GR and DR for the frequency regulation within system's constraints and cost. The algorithm is tested on a system with two areas interconnected by means of a tie line and shows that by choosing DR the frequency response not only improves but also the cost of frequency regulation reduces.

Recently, projection neural network (PNN) was proposed for solving monotone variational inequalities (VIs) and related convex optimization problems. In this paper, considering the inertial term into first order PNNs, an inertial PNN (IPNN) is also proposed for solving VIs. Under certain conditions, the IPNN is proved to be stable, and can be applied to solve a broader class of constrained optimization problems related to VIs. Compared with existing neural networks (NNs), the presence of the inertial term allows us to overcome some drawbacks of many NNs, which are constructed based on the steepest descent method, and this model is more convenient for exploring different Karush-Kuhn-Tucker optimal solution for nonconvex optimization problems. Finally, simulation results on three numerical examples show the effectiveness and performance of the proposed NN.

The classical power allocation/reallocation faces difficult challenges in a future power grid with a great many distributed generators and fast power fluctuations caused by high percentage of renewable energy. To perform power reallocation fast in a future power grid with a large number of participants and disturbances, a hierarchical distributed scheme based on a partition framework is proposed. In the proposed scheme, the power grid is naturally partitioned into a certain number of regions, and the total energy demand in the power grid with disturbances is automatically estimated rather than given in advance. Besides, the centralized local optimizations in regions and the distributed global optimization among regions are coupled to solve the power reallocation problem, in which each region performs as a single agent. Thus, the agents in the proposed scheme are much fewer than the purely distributed ones, hence the communication load is greatly relieved and the reallocation process is significantly simplified. Effectiveness of the proposed scheme is verified by the cases.

Multienergy microgrids are a promising solution to improve overall energy (electricity, cooling, heating, etc.) efficiency. In this paper, a new optimal energy trading strategy is developed considering the risk from uncertain energy supply and demand in a set of individual multienergy microgrids. According to the historical data about energy supply of each microgrid, an aggregator aims to maximize each microgrid's profit while minimizing the risk of overbidding for renewable energy resources trading based microgrids. A novel two-stage stochastic game model with Cournot Nash pricing mechanism and the conditional value-at-risk criterion is proposed to characterize the payoff function of each microgrid. The sample average approximation (SAA) technique is employed to approximate the stochastic Nash equilibrium of the game model. The existence of the SAA Nash equilibrium is investigated and the corresponding Nash equilibrium seeking algorithm is also realized in a distributed manner. The proposed method is validated by numerical simulations on real-world data collected in Australia, and the results show that the SAA Nash equilibrium based strategy can effectively reduce the risk of not meeting the demand and improve the economic benefits for each microgrid.

This paper proposes a control framework, called pulse-modulated intermittent control, which unifies impulsive control and sampled control. Specifically, the concept of pulse function is introduced to characterize the control/rest intervals and the amplitude of the control. By choosing some specified functions as the pulse function, the proposed control scheme can be reduced to sampled control or impulsive control. The proposed control framework is applied to consensus problems of multiagent systems. Using discretization approaches and stability theory, several necessary and sufficient conditions are established to ensure the consensus of the controlled system. The results show that consensus depends not only on the network topology, the sampling period and the control gains, but also the pulse function. Moreover, a lower bound of the asymptotic convergence factor is derived as well. For a given pulse function and an undirected graph, an optimal control gain is designed to achieve the fastest convergence. In addition, impulsive control and sampled control are revisited in the proposed control framework. Finally, some numerical examples are given to verify the effectiveness of theoretical results.

This paper proposes an adaptive neural-fuzzy sliding-mode control method for uncertain nonlinear systems with actuator effectiveness faults and input saturation. The parameter dependence of the control scheme is removed from the bound of actuator faults by updating online. A neural-fuzzy model is developed to approximate the uncertain nonlinear terms and a sliding-mode online-updating controller is developed to estimate the bound of the actuator with no prior knowledge of the fault. The asymptotic stability is verified via the Lyapunov method in the presence of actuator faults and saturation. Furthermore, the adaptive neural-fuzzy control method is extended to the uncertain faulty nonlinear systems with integral sliding-mode manifold as well as other popular sliding-mode surfaces. A numerical example is presented to demonstrate the effectiveness of the derived results.

It has been estimated that the cumulative sales of Electric Vehicles (EVs) will be up to 5.9 million and the stock of EVs will be up to 20 million by 2020 [1]. As the number of EVs is expanding, there is a growing need for widely distributed, publicly accessible, EV charging facilities. The public EV Chargers (EVCs) are expected to be found and will be needed where there is on-street parking, at taxi stands, in parking lots at places of employment, hotels, airports, shopping centres, convenience shops, fast food restaurants, and coffee houses, etc. In this work, we aim to optimize the distribution of public EVCs across the city such that (i) the overall revenue generated by the EVCs is maximized, subject to (ii) the overall driver discomfort (e.g., queueing time) for EV charging is minimized. This is the first study on EVC distribution where EVCs are assumed to be installed in almost all regions across a city. The problem is formulated using a bilevel optimization model. We propose an alternating framework to solve it and have proved that a local minima is achievable. Moreover, this work introduces novel methods to extract information to understand the discomfort of petroleum car drivers, EV charging demands, parking time and parking fees across the city. The source data explored include the trajectories of taxis, the distribution of petroleum stations and various local features. The empirical study uses the real data sets from Shenzhen City, one of the largest cities in China. The extensive tests verify the superiority of the proposed bilevel optimization model in all aspects.

To reduce information exchange requirements in smart grids, an event-triggered communication-based distributed optimization is proposed for economic dispatch. In this work, the θ-logarithmic barrier-based method is employed to reformulate the economic dispatch problem, and the consensus-based approach is considered for developing fully distributed technology-enabled algorithms. Specifically, a novel distributed algorithm utilizes the minimum connected dominating set (CDS), which efficiently allocates the task of balancing supply and demand for the entire power network at the beginning of economic dispatch. Further, an event-triggered communication-based method for the incremental cost of each generator is able to reach a consensus, coinciding with the global optimality of the objective function. In addition, a fast gradient-based distributed optimization method is also designed to accelerate the convergence rate of the event-triggered distributed optimization. Simulations based on the IEEE 57-bus test system demonstrate the effectiveness and good performance of proposed algorithms.

This paper proposes a generalized Hopfield network for solving general constrained convex optimization problems. First, the existence and the uniqueness of solutions to the generalized Hopfield network in the Filippov sense are proved. Then, the Lie derivative is introduced to analyze the stability of the network using a differential inclusion. The optimality of the solution to the nonsmooth constrained optimization problems is shown to be guaranteed by the enhanced Fritz John conditions. The convergence rate of the generalized Hopfield network can be estimated by the second-order derivative of the energy function. The effectiveness of the proposed network is evaluated on several typical nonsmooth optimization problems and used to solve the hierarchical and distributed model predictive control four-tank benchmark.

In this paper a control strategy is proposed for Automatic Generation Control (AGC), which focuses on the interconnected system instead of individual areas and minimizes the cost of control while maintaining network constraints. A methodology is developed to maintain the network constraints by limiting the tie line flows within safe thermal limits when the generation and disturbances in the interconnected areas are utilized for AGC. Our contribution comes from extending the Economic AGC approach in [1] so that it is feasible for practical implementation. This is achieved by imposing constraints on a set of non physical auxiliary variables. The optimization function is extended to include the constraints on the auxiliary variables by using a logarithmic barrier function method. It is proved for a two area power system that the tie line flows attain the same values as the auxiliary variables under steady state conditions. A simulation study is presented to show the effectiveness of our approach.

In this paper, demand side management problem is reformulated by the jointly constrained noncooperative game. The corresponding networked optimization method that concentrates on seeking generalized Nash Equilibrium for noncooperative game is developed for the problem. Due to the large scale of users in demand side management, the noncooperative game based demand side management is divided into groups of sub games, which can be efficiently solved by Nikaido-Isoda function based Newton method. Simulation results verify that the effectiveness of the designed algorithm.

In this technical correspondence, we consider a distributed cooperative optimization problem encountered in a computational multiagent network with delay, where each agent has local access to its convex cost function, and jointly minimizes the cost function over the whole network. To solve this problem, we develop an algorithm that is based on dual averaging updates and delayed subgradient information, and analyze its convergence properties for a diminishing step-size by utilizing Bregman-distance functions. Moreover, we provide sharp bounds on the convergence rates as a function of the network size and topology embodied in the inverse spectral gap. Finally, we present a numerical example to evaluate our algorithm and compare its performance with several similar algorithms.

Concerning on optimal economic dispatch, interior point method via 6-logarithmic barrier is employed to reformulate the cost function of power generation. Fully distributed technology-enabled algorithm is developed to solve the economic dispatch. More specifically, the minimum connected dominating set based distributed algorithm aims at efficiently allocating the task of supply-demand balance for the whole power grid. A fast gradient based distributed optimization method is designed to fast converge to optimal solution. The simulations illustrate the effectiveness and good performance of our algorithms.

In this paper, global exponential synchronization of a class of discrete delayed complex networks with switching topology has been investigated by using Lyapunov-Ruzimiki method. The impulsive scheme is designed to work at the time instant of switching occurrence. A time-varying delay-dependent criterion for impulsive synchronization is given to ensure the delayed discrete complex networks switching topology tending to a synchronous state. Furthermore, a numerical simulation is given to illustrate the effectiveness of main results.

In this brief, based on the method of penalty functions, a recurrent neural network (NN) modeled by means of a differential inclusion is proposed for solving the bilevel linear programming problem (BLPP). Compared with the existing NNs for BLPP, the model has the least number of state variables and simple structure. Using nonsmooth analysis, the theory of differential inclusions, and Lyapunov-like method, the equilibrium point sequence of the proposed NNs can approximately converge to an optimal solution of BLPP under certain conditions. Finally, the numerical simulations of a supply chain distribution model have shown excellent performance of the proposed recurrent NNs.

This brief reports a retarded functional differential equation modeling tri-neuron network with time delay. The Bogdanov-Takens (B-T) bifurcation is investigated by using the center manifold reduction and the normal form method. We get the versal unfolding of the norm forms at the B-T singularity and show that the model can exhibit pitchfork, Hopf, homoclinic, and double-limit cycles bifurcations. Some numerical simulations are given to support the analytic results and explore chaotic dynamics. Finally, an algorithm is given to show that chaotic tri-neuron networks can be used for encrypting a color image.