Scaling LLM-based agents to long-horizon deep research is constrained by the context-noise trade-off, where linear history accumulation degrades reasoning and dilutes fine-grained evidence. To address this, we introduce the Cognitive Scaffold, a factorized memory architecture that decouples the cognitive state into a Fluid Working Context for immediate reasoning and a persistent Knowledge Graph for long-term retention. Unlike unstructured summarization, our framework employs a Rejection Sampling Fine-Tuning (RFT) pipeline to crystallize saturated context into structured event snapshots, strictly enforcing atomic constraints to preserve numerical values and entities. During reasoning, a thought-driven dual-path retrieval mechanism enables the agent to proactively recover precise evidence. Empirical evaluations on Xbench-DeepSearch, BrowseComp-ZH, and GAIA demonstrate that Cognitive Scaffold consistently outperforms baselines, achieving 74.7% Avg@3 and 87.0% Pass@3 on Xbench-DeepSearch, 48.5% Avg@3 and 65.9% Pass@3 on BrowseComp-ZH, and 72.8% Avg@3 and 88.3% Pass@3 on GAIA, while reducing compression hallucinations to 5.3%. We open-source our codebase to facilitate future research.

Independent learning (IL) is a popular approach for achieving scalability in large-scale multi-agent systems, yet it typically lacks global convergence guarantees. In this paper, we study two representative algorithms—independent $Q$-learning and independent natural actor-critic—within both value-based and policy-based frameworks, and provide the first finite-sample analysis for approximate global convergence. Our results show that IL can achieve global convergence up to a fixed error arising from agent interdependence, which characterizes the fundamental limit of IL in achieving true global convergence. To establish these results, we develop a novel approach by constructing a separable Markov decision process (MDP) for convergence analysis and then bounding the gap caused by the model discrepancy between this separable MDP and the original one. Finally, we present numerical experiments using a synthetic MDP and an electric vehicle charging example to demonstrate our findings and the practical applicability of IL.

The dynamic conditions and internal states of portable energy storage system (PESS), such as temperature, electricity price, state of charge (SOC), and state of health (SOH), significantly impact battery degradation. Current decision-making models for PESS operation often oversimplify the modeling of battery degradation. To address this, we introduce an environment-adaptive online learning framework that effectively integrates deep neural networks and reinforcement learning to exploit and explore external environments (i.e., electricity prices and temperature) and internal dynamics (i.e., battery degradation), providing decision support for PESS operation. This framework dynamically updates battery degradation and decision-making models in real-time, enhancing adaptive responses to external changes. Specifically, we developed a neural network based on porous electrode theory that considers multi-physical factors, such as charging power, initial and terminal SOC, SOH, and temperature to accurately assess battery degradation. This network is embedded within a deep reinforcement learning algorithm, enabling real-time, adaptive decision-making for PESS amidst varying environmental conditions. Furthermore, to navigate complex operational environments, a fine-tuning mechanism is incorporated into the degradation neural network. Application of this framework to the energy arbitrage of PESS in the California power grid demonstrates an average benefit increase of 37% compared to traditional degradation assessment models. Note to Practitioners—In this work, we develop a novel approach to addressing the critical issue of battery degradation in PESS. Existing models often oversimplify degradation, hindering accurate assessments of performance and lifespan projections. More recently, learning-based algorithms have demonstrated outstanding performance in both battery degradation modeling and real-time decision-making. In this sense, we introduce a sophisticated neural network model grounded in a porous electrode model. This model considers multi-physics factors involving charging/discharging power, initial and terminal SOC, SOH, and temperature. Complementing this, we integrate the aforementioned model into an online learning framework, enabling real-time decision-making for PESS. Furthermore, to tackle the challenges of complex operating environments, a fine-tuning mechanism is incorporated into the battery degradation neural network. We validate the effectiveness of the proposed methods through an energy arbitrage application of PESS and also reveal its potential for on-demand applications in energy and transportation systems. This note anticipates a positive impact on PESS management and contributes significantly to the evolution of energy storage systems, offering practitioners invaluable decision support for commercial applications involving battery sharing, trading, and renting.

We address the problem of under-utilization of resources in datacenters during cloud operations, specifically focusing on the challenge of online virtual machine (VM) scheduling. Rather than following the traditional approach of scheduling VMs based solely on their static flavors, we take into account their dynamic CPU utilization. We employ$\Gamma $-robustness theory to manage the dynamic nature and introduce a novel variant of bin packing - Probabilistic k-Bins Packing (PkBP), which theoretically protects the Physical Machines (PMs) from hotspots formation within a specified probability$\alpha $. We develop a scheduling algroithm named CloseRadiusFit and cold-start AI-based prediction algorithms for the online version of PkBP. To verify the quality of our approach towards the optimal solutions, we solve the Offline PkBP problem by designing a novel Mixed Integer Linear Programming (MILP) model and a combination of numerical upper and lower bounds. Our experimental results demonstrate that CloseRadiusFit achieves narrow gaps of 1.6% and 3.1% when compared to the lower and upper bounds, respectively. Note to Practitioners—A growing trend in the cloud industry involves overcommitting VMs on PMs. While this approach can ease the problem of low utilization of resources in datacenters, it also introduces a higher risk of hotspots due to resource contention and competition among VMs. In this work, we propose a novel method that leverages$\Gamma $-robustness theory and introduce effective heuristics to achieve ultimate utilization of datacenter resources while ensuring desirable service quality. We validate our approach using real-world production data from Huawei Cloud, improving resource utilization by 125% over traditional flavor-based allocation methods, while maintaining the occurrence of hotspots below 5% ($\alpha =0.05$). Our solution only requires VMs’ real utilization data that is typically already collected in cloud providers’ production environments. Therefore, with minimal modifications to the existing scheduling system, cloud providers can easily implement our solution and reap its benefits. Moreover, in cases of the absence of historical utilization data for VMs (cold-start), we use machine learning to predict VM utilization statistics for our approach.

Advancements in information technology and the exponential growth of data in energy systems present significant potential for intelligent and secure grid operations. However, variability in data quality remains a critical constraint. To address the lack of focus on the role of high-quality data in improving decision-making, this paper proposes a two-stage data purification framework. The first stage employs reinforcement learning-based valuation with refined policy strategies to quantify data quality, providing ranked references for decision-focused filtering in the second stage. Applied to stochastic optimization in wind-integrated unit commitment, the proposed method demonstrates its effectiveness on IEEE 30 and IEEE 118-bus systems by accurately identifying high-quality data and achieving economic benefits under varying uncertainty levels. This work aims to introduce a conceptual framework for data-centric decision-making improvement and provide methodological guidance for both academia and industry.

Battery storage is a key technology for distributed renewable energy integration. Wider applications of battery storage systems call for smarter and more flexible deployment models to improve their economic viability. Here we propose a hybrid energy storage system (HESS) model that flexibly coordinates both portable energy storage systems (PESSs) and stationary energy storage systems (SESSs) in a grid. PESSs are batteries and power conversion systems loaded on vehicles that travel between grid nodes with price differences to alleviate grid congestion. PESSs can charge/discharge at grid nodes or swap (part of) batteries with SESSs for profit maximization. We introduce a spatiotemporal decision-making framework for HESS including the planning of SESS and the on-demand dispatch of PESS. We propose a two-phase decision-making algorithm (TPDM), where the first phase uses a spatiotemporal cost-effectiveness aggregation method to determine the optimal SESS location; the second phase shapes a low-complexity solution space by arc destroying and repairing. The results show that HESS achieves significant arbitrage benefit improvement in 86.3% of the operating periods through a year compared with SESS and PESS alone. Compared with commercial solver, the proposed TPDM, on average, can reduce the computational time by 95.5% with an optimality of 1.04%.Note to Practitioners—Battery storage and electric vehicles (EVs) play a crucial role in renewable energy integration and in shaping a low-carbon and sustainable energy and transportation systems. To achieve efficient and scalable management of battery storage across energy and transportation systems, we incorporate the portable energy storage (i.e., batteries transported by vehicles) and stationary energy storage (i.e., batteries placed at grids), into a hybrid energy storage system (HESS), and develop efficient planning framework and scheduling algorithms. Specifically, the proposed methods can provide decision supports for the owners of battery assets to determine the optimal SESS location and for the high-quality coordination of battery charging, swapping, and routing in a HESS. Our methods also have potentials in the on-demand applications of battery storage and EVs across energy and transportation systems, such as ancillary services, grid investment deferral, and battery trading and sharing.

Smart grid is a typical application of industrial cyber-physical systems (ICPS) in the electric power industry. Due to the exposure to different kinds of uncertainties and unpredictable faults, how to reliably assess the short-term voltage stability (SVS) of smart grids to prevent the occurrence of large-scale blackouts is still of primary concern. To tackle this challenging problem, this article develops a novel machine learning scheme to achieve accurate and interpretable online SVS assessment in two steps. First, it utilizes time-series shapelet transform to extract key dynamics and convert the postfault time series into flat features. Second, it designs a graph convolutional network (GCN) to incorporate these features with topology information for SVS assessment. The GCN explores the spatial-temporal dynamics of power system via graph convolution and introduces a system layer to derive the final assessment result. Compared with conventional methods, this novel scheme makes full use of the spatial-temporal information in SVS dynamics, resulting in higher assessment accuracy and stronger adaptability. Besides, it is capable of discovering certain valuable underlying rules and patterns related to SVS. Test results on the IEEE 39-bus system and real-world Guangdong Power Grid in South China verify the effectiveness of the proposed scheme. Note to Practitioners—To achieve accurate and interpretable online short-term voltage stability (SVS) assessment in the challenging environment of smart grids, this article develops a novel machine learning scheme with full consideration of the spatial-temporal information in SVS dynamics. First, it utilizes the time series shapelet transform to convert the postfault time series into flat features. Second, it designs a graph convolutional network (GCN) to incorporate these features with topology information. The full consideration of spatial-temporal information in SVS dynamics can improve the assessment accuracy, and the integration of topology in the scheme can promote its adaptability to topology changes. Apart from the decent performances under changeable environments, the proposed scheme can provide certain valuable underlying rules and patterns related to SVS. Therefore, not only the proposed scheme for SVS assessment can work well in the practical challenging environment of smart grids, but also it helps the dispatchers in smart grids better understand and trust the proposed SVS assessment scheme.

We introduce an innovative solution approach to the challenging dynamic load-shedding problem which directly affects the stability of large power grid. Our proposed deep Q-network for load-shedding (DQN-LS) determines optimal load-shedding strategy to maintain power system stability by taking into account both spatial and temporal information of a dynamically operating power system, using a convolutional long-short-term memory (ConvLSTM) network to automatically capture dynamic features that are translation-invariant in short-term voltage instability, and by introducing a new design of the reward function. The overall goal for the proposed DQN-LS is to provide real-time, fast, and accurate load-shedding decisions to increase the quality and probability of voltage recovery. To demonstrate the efficacy of our proposed approach and its scalability to large-scale, complex dynamic problems, we utilize the China Southern Grid (CSG) to obtain our test results, which clearly show superior voltage recovery performance by employing the proposed DQN-LS under different and uncertain power system fault conditions. What we have developed and demonstrated in this study, in terms of the scale of the problem, the load-shedding performance obtained, and the DQN-LS approach, have not been demonstrated previously.

The COVID-19 pandemic has not only posed a significant threat to health, life, economy, and the whole society but also led to numerous new theoretical and practical challenges for automation science and engineering. The goal of this Special Issue is to bring together researchers and practitioners into a forum to show the state-of-the-art research and applications in responding to the challenges and opportunities of automation science and engineering to the pandemic, by presenting efficient scientific and engineering solutions, addressing the needs and difficulties for integration of new automation methodologies and technologies, and providing visions for future research and development.

To better manage patient flows, China has promoted a referral system across the country. Patients are encouraged to receive the initial diagnosis in community hospitals (CHs), and general hospitals (GHs) manage a slot reservation process to fulfill the needs of referral patients, who are in more severe conditions. According to the system practices, however, the reservation policy usually leads to either underutilized resources or unsatisfied referrals. This article aims to investigate a more effective method of allocating resources in GHs. We formulate the referral system as an appointment booking problem, considering the notion of the patient mix and system dynamics. The decision process of the referral system is captured by a discrete-time finite-horizon Markov decision process (MDP) model under a general framework. Theoretically, we analyze the structural properties of the MDP value functions to prove the monotonic properties of the optimal dynamic policy. The properties inspire us to design a heuristic policy called advanced referrals (ARs) policy, which offers resources to high-priority referrals earlier than regular patients. We prove that the AR policy is asymptotically optimal with infinite capacity and demand rates. Finally, we compare the performance of the AR policy with the optimal dynamic policy in numerical experiments, and also show that our policy outperforms fixed-reservation and first-come-first-serve policies which are widely used in practice. Note to Practitioners-In recent years, the healthcare system in China has been implementing the policy that general hospitals (GHs) reserve a fixed amount of slots for referrals from community hospitals (CHs), encouraging patients to choose CHs for initial diagnosis. However, the reservation policy ignores the demand uncertainty and system dynamics, which leads to circumstances where the reservations are either insufficient or underutilized most of the time. In the case of insufficient reservations, referrals will experience a treatment delay, while the underutilized slots lead to wastes of resources. In this work, we propose a more effective method that optimizes the allocation of GH resources between referrals and nonreferrals. In addition to analyzing the structure of the optimal dynamic policy, we design a heuristic policy that allows referrals to acquire resources earlier in time. This heuristic policy decides on a block time for the regular patients, and before the block time regular patients are not allowed to access the system, while referrals can get access to the resources freely. This policy is easy-to-implement and can better manage demand uncertainty. We provide an approach to calculate the policy and validate its performance both theoretically, and numerically.

In recent years, web-based appointment platforms develop rapidly, and many of them provide physician recommendation services to patients. Considering patients' heterogeneity in both illness and behavior, it is a great challenge to deliver personalized recommendations. Motivated by this healthcare application, this study incorporates patient choices and focuses on optimizing real-time personalized recommendation of physician assortment with limited resources, in order to satisfy patients' varying demands and improve the resource allocation efficiency. This work considers not only the influence of physician assortment to patient choice but also how physicians in the assortment are displayed on the webpage, i.e., the order of physicians. We adopt the location-based two-stage choice model to capture the patient behavior, in which patients randomly view the top physicians and then choose one among them. The recommendation problem is studied in both static and dynamic environments. In the static environment, we ignore resource capacities and optimize the recommendation of physician assortment as well as the displayed ranking. We propose a heuristic algorithm SORT for this static version of the problem and prove the lower bound of algorithm performance, along with the numerical performance validation. In the dynamic environment, we optimize the physician recommendations for a sequence of randomly arriving patients considering patients' heterogeneity in both matching degrees and choice probabilities. We propose a dynamic algorithm Adjust-exponential inventory balancing (Adjust-EIB) by incorporating our static algorithm SORT in the improved existing algorithm, which makes recommendation decisions based on the real-time remaining resources. We conduct a series of numerical experiments to compare our algorithm with several benchmarks. The numerical results show that Adjust-EIB outperforms the benchmark algorithms, especially in congested systems. We also conduct a case study with real-world data and verify the capability of our algorithm in improving real-world system efficiency. Note to Practitioners-This work is motivated by the increasing popularity of web-based appointment platforms. Considering the fact of limited physician resources, we address the issue of how to recommend physicians for patients in a personalized and real-time way on web-based appointment platform, in order to improve the matching degree between physicians and patients as well as the resource allocation efficiency. To the best of our knowledge, we propose one of the initial methods to recommend personalized physician rankings in dynamic environments with limited resources. We construct a specific model and propose algorithms to make physician recommendation decision, which performs better than other benchmark approaches based on the numerical analysis, and the case study with real-world system data indicates our method's capability of solving the practical problem. Our method is robust enough for the reason that the model allows arbitrary arrival pattern. The model and methods are specifically designed for the web-based appointment platforms, but it can also be easily applied in other application scenarios based on visual web pages, such as e-commerce.

There has been growing interest in healthcare delivery systems worldwide coupled with a recent influx of funding into the area. Due to rapid development in information and network technology, smartness and interconnectivity have become a central issue in healthcare delivery. Automation is important for healthcare delivery systems engineering. In recent years, the significant changes in healthcare delivery and the rapid development in data analytics, artificial intelligence, robotics, and wearable devices have generated numerous opportunities for innovation in automation for smart and interconnected healthcare delivery systems. In addition, many new challenges have emerged in order to apply and implement these innovations. Such opportunities and challenges have significantly expanded the scopes of traditional automation science and engineering. Therefore, to show the state-of-the-art research and applications in the general area of healthcare delivery systems automation and to address the needs and challenges for the integration of new automation technologies in healthcare delivery, this Special Issue serves as a forum to bring together researchers, clinicians, and healthcare practitioners to present efficient scientific and engineering solutions and to provide visions for future research and development.

Web-based appointment systems are emerging in healthcare industry providing patients with convenient and diversiform services, among which physician recommendation is becoming more and more popular tool to make assignments of physicians to patients. Motivated by a popular physician recommendation application on a web-based appointment system in China, this paper gives a pioneer work in modeling and solving the physician recommendation problem. The application delivers personalized recommendations of physician assortments to patients with heterogeneous illness conditions, and then, patients would select one physician for appointment according to their preferences. Capturing patient preferences is essential for physician recommendation delivery; however, it is also challenging due to the lack of data on patient preferences. In this paper, we formulate the physician recommendation problem based on which the preference learning algorithm is proposed that optimizes the recommendations and learns patient preferences at the same time. Since the illness conditions of patients are heterogeneous, the algorithm aims to make personalized recommendation for each patient. Besides demonstrating the effectiveness of algorithm performance in terms of regret bound, we also provide extensive numerical experiments to show the expected algorithm performance under heterogeneous reward scenarios and performance comparison with algorithms in the literature under fixed reward scenarios. We introduce the flexibility of adjusting preference estimate update interval into our algorithm and conclude that short update interval contributes to short-term performance while long update interval leads to good results in the long run. Furthermore, we analyze how preference bound helps the algorithm to make explorations, which constitute two major contributions of our algorithm. Finally, we discuss the relevance between patient preferences and physician utilization and present a utilization-balancing approach that is effective in numerical experiments.

Hospitals use appointment systems to manage patient access. Appointment rule, which consists of the length of the booking window, block capacity, and block service time, is critical to achieve efficiency and timely access to healthcare delivery. In this paper, we use a renewal process model to evaluate interday appointment planning and design improved appointment rules for hospitals, especially for those with limited or insufficient resources. We present an associated embedded Markov chain to derive the steady-state distribution. To balance the waiting time and probability of healthcare access, we propose three performance measures, namely, slot utilization, appointment success rate, and patient waiting time, for our evaluation. We then conduct a numerical study to examine the impact of each appointment rule parameter. Qualitative results show that extending the booking window does not significantly reduce system congestion, and a narrowed appointment block is a suitable design for highly in-demand doctors. We use our model and method to design an optimal appointment rule for an actual hospital in Beijing, China. The improved appointment rule is practical and useful for the decision-making of hospital managers. Note to Practitioners-An appointment system with a limited booking window length is a practical solution to reduce waiting time. However, potential patients will lose the opportunity to obtain access to healthcare systems, especially those with a high arrival rate. To balance the tradeoff between shortening the waiting time and increasing healthcare access probability, we attempt to evaluate and design an improved appointment rule that includes booking window length, block capacity, and block service time. Sensitivity analysis shows that extending the booking window does not significantly reduce system congestion, and a narrowed appointment block is a suitable design for highly in-demand doctors. On the basis of our model, we design an optimal appointment rule for an actual Chinese hospital. Results show that improvement can be significant (more than 60%, for example) depending on the parameters.

We propose a novel load shedding (LS) scheme against voltage instability using deep reinforcement learning (DRL). Both spatial and temporal information of a large power grid are used in the DRL control scheme. Specifically, both dynamic state variables and grid topology information are inputs to the learning controller. Within the DRL scheme, a deep learning neural network is designed and implemented to automatically extract translation-invariance information about voltage instability. The DRL load shedding controller interacts with system dynamics through a sequence of observations, actions and rewards to determine load shedding amounts in a manner that maximizes cumulative future reward, accomplishing coordination within the region rapidly to meet the online application requirements. The DRL based distributed LS scheme is performed to control the China Southern Power Grid (CSG) system. Our results show improved voltage recovery performance by load shedding using the proposed scheme under different unknown test scenarios.

With the increasing demand of patients for limited healthcare services, e.g., expert physicians in general hospitals and the long on-site waiting time of patients, a real-time admission control (AC) policy was developed considering both distinction and fairness among heterogeneous patients. The AC policy chooses the customer for the next service from the waiting area to minimize the expected total disutility according to the current system state. A continuous-time Markov decision process in a finite horizon was developed, and a myopic optimal policy was derived using a switching curve to discriminate patient admission based on patient types and waiting time. The effect of each parameter on the switching curve is analyzed, and managerial insights are discussed. In the numerical experiment, an empirical case in the Pediatric Department of Peking University Third Hospital is conducted using simulation models. Three commonly used scheduling policies and the myopic policy (MP) are tested in both Markovian settings and a real-world scenario. Simulation results confirm the effectiveness of the MP, because the total disutility of patients is significantly alleviated. Furthermore, the service experience of patients is expected to improve with the application of the AC policy.

The imbalanced development among different levels of healthcare facilities has become a major social issue in China's urban healthcare system, which has raised the irrational patient flow distribution on the levels of both intra-hospital and inter-healthcare facilities. In this research, we develop a methodology to find the optimal macrolevel patient flow distribution in terms of multidimension inputs and outputs for the two-level healthcare system. The proposed method integrates the discrete-event simulation (DES), the multiobjective optimization and the simulation budget allocation together to comprehensively improve the overall system performances by finding the approximate Pareto patient flow distribution in the hierarchical healthcare system. The multiobjective optimal computing budget allocation (MOCBA) is applied to improve the efficiency, where the nondominated probability is functioned as the fitness measurement to each design. A case study based on the real data is carried out to validate and implement the proposed method. The results of the case study show the recommended Pareto optimal patient flow distribution can improve the overall hierarchical system performances and our methodology are qualified as a quantitative decision tool for decision makers.

In this paper, we study the radiation treatment planning optimization for Volumetric-Modulated Arc Therapy (VMAT). A nonlinear mixed integer programming model is formulated, then the linearization technique is used, and the resulting mixed integer programming model is solved by a heuristic approach based on the Nested-Partitions framework. The approach partitions the feasible region iteratively and constructs a feasible solution by solving the LP relaxation of the original problem. We design two partition strategies: partition by column and expansion from center of aperture. Numerical results with clinical cases show the efficiency of the proposed model and algorithm.

In this paper a neural network-based approximate dynamic programming method, namely direct heuristic dynamic programming (direct HDP), is applied to power system stability control. Direct HDP makes use of learning and approximation to address nonlinear system control problems under uncertainty. The contribution of the paper includes a convergence proof of the direct HDP algorithm using an LQR framework. Under this setting, the paper proposes a direct HDP learning control algorithm for a static var compensator (SVC) supplementary damping control in a standard benchmark power system. The results are used to evaluate the online learning ability of the proposed direct HDP controller, and also to demonstrate that the learning controller does converge to the theoretical limit as derived.