We investigate a novel type of online sequential decision problem under uncertainty, namely mixed observability Markov decision process with time-varying interval-valued parameters (MOMDP-TVIVP). Such data-driven optimization problems with online learning widely have real-world applications (e.g., coordinating surveillance and intervention activities under limited resources for pandemic control). Solving MOMDP-TVIVP is a great challenge as online system identification and reoptimization based on newly observational data are required considering the unobserved states and time-varying parameters. Moreover, for many practical problems, the action and state spaces are intractably large for online optimization. To address this challenge, we propose a novel transfer reinforcement learning (TRL)-based algorithmic approach that ingrates transfer learning (TL) into deep reinforcement learning (DRL) in an offline-online scheme. To accelerate the online reoptimization, we pretrain a collection of promising networks and fine-tune them with newly acquired observational data of the system. The hallmark of our approach comes from combining the strong approximation ability of neural networks with the high flexibility of TL through efficiently adapting the previously learned policy to changes in system dynamics. Computational study under different uncertainty configurations and problem scales shows that our approach outperforms existing methods in solution optimality, robustness, efficiency, and scalability. We also demonstrate the value of fine-tuning by comparing TRL with DRL, in which at least 21% solution improvement can be yielded by TRL with fine-tuning for no more than 0.62% of time spent on pretraining in each period for problem instances with a continuous state-action space of modest dimensionality. A retrospective study on a pandemic control use case in Shanghai, China shows improved decision making via TRL in several public health metrics. Our approach is the first-ever endeavor of employing intensive neural network training in solving Markov decision processes requiring online system identification and reoptimization. History: Accepted by Paul Brooks, Area Editor for Applications in Biology, Medicine, & Healthcare. Funding: This work was supported in part by the National Natural Science Foundation of China [Grants 72371051 and 72201047] to the first and second authors and in part by the National Science Foundation [Grant 1825725] to the third author. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0236 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0236 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Substance use disorder, particularly opioid-related, is a serious public health challenge in the U.S. Accurately predicting opioid overdose events and stratifying the risk of having such an event are critical for healthcare providers to deliver effective interventions in patients with opioid overdose. Despite a large body of literature investigating various risk factors for the prediction, the existing research to date has not explicitly investigated and quantitatively modeled how an individual's past opioid overdose events affect future occurrences. In this paper, we proposed a covariate-adjusted triggering point process to simultaneously model the effect of various risk factors on opioid overdose events and the triggering mechanism among opioid overdose events. The prediction performance was assessed by the U.S. state-wise Medicaid reimbursement claims data. Compared with commonly used prediction models, the proposed model achieved the lowest Mean Absolute Errors and Mean Absolute Percentage Errors on 30-, 60-, 90, 120-, 150-, and 180-day-ahead predictions. In addition, our results showed the statistical significance of considering the triggering mechanism for recurrent opioid overdose events prediction. On average, around 47% of the event recurrence were explained by the triggering mechanism.

The inconvenience of charging is one of the major concern for potential electric vehicle (EV) users. In addition to building more charging facilities, electric vehicle charging assistance service has emerged for making EV charging more convenient to customers. In this paper, we consider an optimal EV charging station location problem with two types of customers. One is ordinary self-charging customers whereas the other is customers using a new service mode called valet-charging. We formulate the problem via bi-level location optimization model, where the lower level problem is a game model that characterizes customers’ station choice behaviors. To solve the hard nonlinear mixed-integer optimization problem, we design an adaptive large neighbourhood search (ALNS) algorithm for the upper level problem and a construct-improve heuristic for the lower level problem. We conduct numerical experiments to justify the efficiency of our solution method. We also conduct a need-inspired case study to derive practical insights which will help EV charging assistant service providers make strategic decisions.Note to Practitioners—The convenience of charging service is one major concern for EVs. In China, NIO Inc., NETA AUTO, and FAW-Volkswagen have started to provide valet-charging service. Charging station location problem becomes complicated while taking this service into account. We believe our work develops an effective tool for charging station planners to analyze station locations as well as the impact of valet charging services.

Anticholinergic (AC) drugs are commonly prescribed to older adults for treating diseases and chronic conditions, such as chronic obstructive pulmonary disease, urinary incontinence, gastrointestinal disorder, or simply pain and allergy. The high prevalence of AC drug use can have a detrimental effect on the mental health of older adults. We aim to improve the prediction of future trends of AC drug use at the individual level, with pharmacy refill data. The individual drug use data presents challenges in the modeling, such as data being discrete-valued with excess zeros and having significant unobserved heterogeneity in the trend pattern. To address these challenges, we propose a statistical model of hierarchical structure and an EM scheme for the model parameter estimation. We evaluate the proposed modeling approach through a numerical study with synthetic data and a case study with real-world pharmacy refill data. The simulation study show that our analysis method outperforms the existing ones (e.g., reducing MSE significantly), particularly in terms of accurately predicting the trend pattern. The real-world case study further verifies the out-performance and demonstrate the advantageous features of our method. We expect the prediction tool developed based on our study can assist pharmacists' decision on initiating or strengthening behavioral interventions with the hope of discontinuing AC drug misuse.

Healthcare capacity shortage contributes to poor access in many countries. Moreover, rapid urbanization often occurring in these countries has exacerbated the imbalance between healthcare capacity and need. One way to address the above challenge is expanding the total capacity and redistributing the capacity spatially. In this research, we studied the problem of locating new hospitals in a two-tier outpatient care system comprising multiple central and district hospitals, and upgrading existing district hospitals to central hospitals. We formulated the problem with a discrete location optimization model. To parameterize the optimization model, we used a multinomial logit model to characterize individual patients’ diverse hospital choice and to quantify the patient arrival rates at each hospital accordingly. To solve the hard nonlinear combinatorial optimization problem, we developed a queueing network model to approximate the impact of hospital locations on patient flows. We then proposed a multi-fidelity optimization approach, which involves both the aforementioned queuing network model as a surrogate and a self-developed stochastic simulation as the high-fidelity model. With a real-world case study of Shanghai, we demonstrated the changes in the care network and examined the impacts on the network design by population center emergence, governmental budget change and considering patients with different age groups or income levels. Note to Practitioners—Our work focuses on improving system-wide care access in a two-tier care network. We believe that our work can lead to effective development of a location analytics tool for city-wide healthcare system planners. We also think the importance of this study is further strengthened by the case studies based on real-world hospital choice experimental data from Shanghai, China, a region suffering from the imbalance between healthcare capacity and need. Our case studies are expected to make recommendations on care facility expansion and dispersion to better align with the spatial distribution of residential communities and patient hospital choice behavior.

While prior studies have designed incentive mechanisms to attract the public to share their collected data, they tend to ignore information asymmetry between data requesters and collectors. In reality, the sensing costs information (time cost, battery drainage, bandwidth occupation of mobile devices, and so on) is the private information of collectors, which is unknown by the data requester. In this article, we model the strategic interactions between health-data requester and collectors using a bilevel optimization model. Considering that the crowdsensing market is open and the participants are equal, we propose a Walrasian equilibrium-based pricing mechanism to coordinate the interest conflicts between health-data requesters and collectors. Specifically, based on the exchange economic theory, we transform the bilevel optimization problem into a social welfare maximization problem with the constraint condition that the balance between supply and demand, and dual decomposition is then employed to divide the social welfare maximization problem into a set of subproblems that can be solved by health-data requesters and collectors. We prove that the optimal task price is equal to the marginal utility generated by the collector’s health data. To avoid obtaining the collector’s private information, a distributed iterative algorithm is then designed to obtain the optimal task pricing strategy. Furthermore, we conduct computational experiments to evaluate the performance of the proposed pricing mechanism and analyze the effects of intrinsic rewards, sensing costs on optimal task prices, and collectors’ health-data supplies.

Ancillary service providers (ASPs) on an inpatient unit (IU), such as social workers, care managers, and physical and occupational therapists, play crucial roles in the inpatient discharge process. However, on a typical day, these ASPs must balance the needs of newly arriving, recurring, and outgoing patients (discharges). While prioritizing the discharges is shown to improve the hospital patient flow and bed utilization, the needs of the other patients cannot be neglected. We examine whether patient sequencing strategies for these ASPs can be found that maintain patient care and optimize patient flow. We approach this ASP workload sequencing problem by proposing a stochastic optimization model and developing a simulated annealing algorithm to derive sequencing strategies that are easy to implement and promising to all scenarios. Our experimental results suggest that patients with pressing constraints should be prioritized over discharges, with discharges then prioritized over other patients. As the complexity of the workload increases (a large number of patients and/or a high percentage of discharges), the focus should shift more towards discharges. We further evaluate the performance of our derived strategies against simpler realistic ASP sequencing strategies and classical machine scheduling rules. Our strategies consistently perform well on deviation from the optimal upstream boarding time and always ensure that patient constraints are met; other simpler strategies can outperform ours only at the cost of increases in cutoff and due-date violations. This paper reveals the importance of a systemic view (inpatient and upstream) on the part of the ASP when prioritizing patients in an IU.Note to Practitioners—This paper was motivated by the problem of sequencing patients daily for ancillary service providers (ASPs) on an inpatient unit (IU). This work is applicable to such ASPs as care managers and therapists. We derive patient sequencing strategies (rules), which they can apply to prioritize their patient workload in such a way that upstream patient boarding time is minimized without affecting the promptness of care for recurring and newly arriving patients. We develop a mathematical model that faithfully captures an ASP’s workload dynamics and a heuristic approach to derive easy-to-use, sequencing strategies. An experimental study using retrospective data and interviews with ASPs from a large teaching hospital in the Northeast U.S. led to a set of strategies to prioritize patients in order to minimize upstream boarding. This set is, however, sensitive to the total number of: 1) patients in the unit; 2) patients ready to be discharged; and 3) patients with tighter due dates. For instance, an increase in the number of patients and/or discharges tends to move the discharge patients ahead in the sequence. Among several realistic strategies we experimented with, we found one single strategy (newly arrived and recurring patients that have pressing constraints, then discharges, then all other patients) that appeared to perform close to our proposed set of strategies. However, care must be taken when implementing this single strategy, as missing a discharge cutoff for nonhome patients (as may occur) may lead to an additional overnight stay. Our findings also suggest that the ASP should consider the workload in the IU when sequencing discharges, instead of a myopic view of their own workload.

Spatially specific intervention is proven to be more cost-effective than the “one-fit-all” strategy in controlling infectious disease outbreaks. However, it presents decision challenges due to partially observable epidemic state information and imperfectly determined model parameters. With deployment of mobile sensor, additional information such as geo-tagged bacteria concentration can be acquired, which can help to improve the understanding of the epidemic state and model parameters. Thus, a mobile sensor dispatching problem deciding further sensing spots should be studied considering the surveillance capacity. To solve this problem, we develop a metapopulation model based predictive control approach for making intelligent public health intervention decisions and sensor spatial dispatch decisions for cholera outbreak control. This approach incorporates a mobile sensor deployment scheme considering the improvement on unmeasurable parameter estimation and epidemic state prediction. Then it optimizes spatially specific intervention strengths to mitigate the harm of the infected population with progressively gained understanding of the system via mobile sensing.

BACKGROUND: Cells operate in an uncertain environment, where critical cell decisions must be enacted in the presence of biochemical noise. Information theory can measure the extent to which such noise perturbs normal cellular function, in which cells must perceive environmental cues and relay signals accurately to make timely and informed decisions. Using multivariate response data can greatly improve estimates of the latent information content underlying important cell fates, like differentiation. RESULTS: We undertake an information theoretic analysis of two stochastic models concerning glioma differentiation therapy, an alternative cancer treatment modality whose underlying intracellular mechanisms remain poorly understood. Discernible changes in response dynamics, as captured by summary measures, were observed at low noise levels. Mitigating certain feedback mechanisms present in the signaling network improved information transmission overall, as did targeted subsampling and clustering of response dynamics. CONCLUSION: Computing the channel capacity of noisy signaling pathways present great probative value in uncovering the prevalent trends in noise-induced dynamics. Areas of high dynamical variation can provide concise snapshots of informative system behavior that may otherwise be overlooked. Through this approach, we can examine the delicate interplay between noise and information, from signal to response, through the observed behavior of relevant system components.

With its aging population, the United States is under increasing pressure to provide long-term care (LTC) coverage to its citizens and to manage their chronic health conditions. However, the research on LTC transition and utilization modeling remains in its infancy; needless to mention LTC resource allocation and transition pathway optimization. In this paper, we developed a parametric survival model to characterize nursing home (NH) length of stay (LOS), which incorporates information on transition history. In addition, the model addressed issues such as recurrent events and competing risks. To study the effect of covariates on LOS and to ensure the flexibility of the model in evaluating operational-level interventions, we elected to develop an accelerated failure time parametric survival model. We fit the model to care transition data collected from a large cohort of older adults receiving coordinated care in a Midwestern United States urban area. Through our study, we drew the following major conclusions: 1) transition history is a significant factor and a potential predictor of an individual's LOS in NH; 2) significance of frailty terms indicates that LOS estimates based on data with recurrent transition events can be significantly biased if not accounted for explicitly; and 3) the same clinical covariate can have opposite effects on NH LOS, depending on the destination care setting. Finally, we identified better-suited baseline hazard functions and frailty terms in each survival model from several representative candidates. Findings from our model can aid in operational-level NH care transition and utilization policy development. This paper also serves as the basis for extension into network-wide LTC transition models and utilization simulators.

In recent years, imbalanced utilization of medical resources is widely concerned within the tiered Chinese hospital system. Reverse referral, as a measure of promoting patient flows from upper level hospitals (ULHs) to lower level hospitals (LLHs), has demonstrated its advantages on alleviating ULH workload and balancing resource utilization between ULHs and LLHs. Nevertheless, it remains unclear on how to control the reverse referral decision process at the operational level. In this paper, we consider an ULH-dominant setting at which the LLH must accept patient referrals from the ULH whenever it has available beds. We focus our attention on an easy-to-implement threshold policy for the ULH to make the reverse referral decision. To investigate, we first formulate an analytically tractable queueing model for a simplified reverse referral process. We then investigate a more general patient flow control model, for which we analyze the patient population dynamics with a Markov chain process, and apply the concept of state-dependent Markovian arrival process to generate an infinitesimal generator of the system. We use RG factorization to compute the system performance measures. We next formulate a threshold optimization problem with the objective of maximizing the ULH profit. Simulation experiments are performed, which conclude that the threshold control policy is insensitive to the service time distribution. Finally, we report real-world inspired numerical studies, from which we generate insights into effective adjustment of the control threshold in response to the system parameters and discuss potential hindrance from the LLH incorrectly informing its real-time resource availability to the ULH. Our work is the first that applies systems engineering to the real-time reverse referral decision problem in China. It provides the novel perspective of resource balancing to patient flow control studies in the care transition management literature.

Significant uncertainty associated with Chinese urban logistics, caused by random vehicle operational restrictions due to severe weather (e.g., smog) in addiction to normal traffic variation makes the tactical production and distribution planning decisions quite challenging. In this paper, we propose a two-stage stochastic integer programming model for an optimal production distribution capacity planning problem under the aforementioned uncertainties. We aim to minimize both procurement spending and the expected operational cost under logistic uncertainty. Given the computational burden of solving the resultant stochastic integer program for real-world instances, we develop an improved stochastic branch-and-bound (SBB) algorithm embedding with Tabu search method. We conduct the numerical study to verify the superiority of the proposed algorithm. We also offer managerial insights to practitioners and policy recommendations to municipal governments based on the numerical study results.