Despite being a fundamental building block for reinforcement learning, Markov decision processes (MDPs) often suffer from ambiguity in model parameters. Robust MDPs are proposed to overcome this challenge by optimizing the worst-case performance under ambiguity. While robust MDPs can provide reliable policies with limited data, their worst-case performances are often overly conservative, and so they do not offer practical insights into the actual performance of these reliable policies. This paper proposes robust satisficing MDPs (RSMDPs), where the expected returns of feasible policies are softly-constrained to achieve a user-specified target under ambiguity. We derive a tractable reformulation for RSMDPs and develop a first-order method for solving large instances. Experimental results demonstrate that RSMDPs can prescribe policies to achieve their targets, which are much higher than the optimal worst-case returns computed by robust MDPs. Moreover, the average and percentile performances of our model are competitive among other models. We also demonstrate the scalability of the proposed algorithm compared with a state-of-the-art commercial solver.

We introduce a Python package called RSOME for modeling a wide spectrum of robust and distributionally robust optimization problems. RSOME serves as an open-source framework for modeling various optimization problems subject to distributional ambiguity in a highly readable and mathematically intuitive manner. It is versatile and fits well in the open-source software community in the sense that (i) it is consistent with NumPy arrays in indexing and slicing and; (ii) together with the rich Python libraries for machine learning, data analysis, and visualization, it is easy to implement data-driven models; and (iii) it provides convenient interfaces for users to switch and tune parameters among different solvers. History: Ted Ralphs, Area Editor for Software Tools. Funding: The research of Z. Chen is funded by the Strategic Research Grant [Project 7005792] from the City University of Hong Kong. The research of P. Xiong is supported by the Ministry of Education, Singapore, under its 2019 Academic Research Fund Tier 3 grant call [Grant MOE-2019-T3-1-010]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1291 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0146 ) at ( http://dx.doi.org/10.5281/zenodo.7463845 ).

We extend the notion of globalized robustness to consider distributional information beyond the support of the ambiguous probability distribution. We propose the globalized distributionally robust counterpart that disallows any (respectively, allows limited) constraint violation for distributions residing (respectively, not residing) in the ambiguity set. By varying its inputs, our proposal recovers several existing perceptions of parameter uncertainty. Focusing on the type 1 Wasserstein distance, we show that the globalized distributionally robust counterpart has an insightful interpretation in terms of shadow price of globalized robustness, and it can be seamlessly integrated with many popular optimization models under uncertainty without incurring any extra computational cost. Such computational attractiveness also holds for other ambiguity sets, including the ones based on probability metric, optimal transport, ϕ-divergences, or moment conditions, as well as the event-wise ambiguity set. Numerical studies on an adaptive network lot-sizing problem demonstrate the modeling flexibility of our proposal and its emphases on globalized robustness to constraint violation. History: Antonio Frangioni, Area Editor for Design & Analysis of Algorithms—Continuous. Funding: Z. Chen was supported by [General Research Fund Grant 9043424, NSFC/RGC Joint Research Scheme N_CityU105/21] from the Hong Kong Research Grants Council. S. Wang was supported by the National Natural Science Foundation of China [Grants 71922020, 72171221, and 71988101, entitled “Econometric Modeling and Economic Policy Studies”] and the Fundamental Research Funds for the Central Universities [Grant UCAS-E2ET0808X2]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0274 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0274 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

We study adjustable distributionally robust optimization problems, where their ambiguity sets can potentially encompass an infinite number of expectation constraints. Although such ambiguity sets have great modeling flexibility in characterizing uncertain probability distributions, the corresponding adjustable problems remain computationally intractable and challenging. To overcome this issue, we propose a greedy improvement procedure that consists of solving, via the (extended) linear decision rule approximation, a sequence of tractable subproblems—each of which considers a relaxed and finitely constrained ambiguity set that can be iteratively tightened to the infinitely constrained one. Through three numerical studies of adjustable distributionally robust optimization models, we show that our approach can yield improved solutions in a systematic way for both two-stage and multistage problems. History: Accepted by Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Funding: Financial support by the Early Career Scheme from the Hong Kong Research Grants Council [Project No. CityU 21502820], the CityU Start-Up Grant [Project No. 9610481], the CityU Strategic Research Grant [Project No. 7005688], the National Natural Science Foundation of China [Project No. 72032005], and Chow Sang Sang Group Research Fund sponsored by Chow Sang Sang Holdings International Limited [Project No. 9229076] is gratefully acknowledged. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2021.0181 ), as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0181 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

The p-hub center problem is a fundamental model for the strategic design of hub location. It aims at constructing p fully interconnected hubs and links from nodes to hubs so that the longest path between any two nodes is minimized. Existing literature on the p-hub center problem under uncertainty often assumes a joint distribution of travel times, which is difficult (if not impossible) to elicit precisely. In this paper, we bridge the gap by investigating two distributionally robust chance-constrained models that cover, respectively, an existing stochastic one under independent normal distribution and one that is based on the sample average approximation approach as a special case. We derive deterministic reformulations as a mixed-integer program wherein a large number of constraints can be dynamically added via a constraint-generation approach to accelerate computation. Counterparts of our models in the emerging robust satisficing framework are also discussed. Extensive numerical experiments demonstrate the encouraging out-of-sample performance of our proposed models as well as the effectiveness of the constraint-generation approach. History: Accepted by Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Funding: This work is partially supported by the National Natural Science Foundation of China [Grants 72101187 and 72021002] and Early Career Scheme from the Hong Kong Research Grants Council General Research Fund [Grant 9043424] and NSFC/RGC Joint Research Scheme N_CityU105/21. Y. Zhao is supported by the Ministry of Education, Singapore, under its 2019 Academic Research Fund Tier 3 grant call [Grant MOE-2019-T3-1-010]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0113 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0113 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .