Many critical business and societal decisions in areas such as supply chain and healthcare involve numerous potential actions, complex constraints, and goals that can be modeled as objective functions. Mathematical optimization, a core area in Operations Research (OR), provides robust, mathematically grounded methodologies to address such decisions and has shown tremendous benefits in many applications. However, its application requires the creation of accurate and efficient optimization models, necessitating rare expertise and considerable time, creating a barrier to widespread adoption in decision-making. Thus, it is a long-standing goal to make these capabilities widely accessible. The advent of Large Language Models (LLMs) has made advanced Artificial Intelligence (AI) capabilities widely accessible through natural language. LLMs can accelerate expert work in creating formal models like computer programs, and emerging research indicates they can also speed up the development of optimization models by OR experts. We, therefore, propose integrating and advancing LLM and optimization modeling to empower organizational decision-makers to model and solve such complex problems without requiring deep expertise in optimization. In this work, we present our vision for democratizing optimization modeling for organizational decision-making by such a combination of LLMs and optimization modeling. We identify a set of fundamental requirements for the vision's implementation and describe the state of the art through a literature survey and some experimentation. We show that a) LLMs already provide substantial novel capabilities relevant to realizing this vision, but that b) major research challenges remain to be addressed. We also propose possible research directions to overcome these gaps. We would like this work to serve as a call to action to bring together the LLM and OR optimization modeling communities to pursue this vision, thereby enabling much more widespread improved decision-making and increasing by orders of magnitude the benefits AI and OR can bring to enterprises and society.

Counterfactual explanations (CEs) play an important role in detecting bias and improving the explainability of data-driven classification models. A CE is a minimal perturbed data point for which the decision of the model changes. Most of the existing methods can only provide one CE, which may not be achievable for the user. In this work, we derive an iterative method to calculate robust CEs (i.e., CEs that remain valid even after the features are slightly perturbed). To this end, our method provides a whole region of CEs, allowing the user to choose a suitable recourse to obtain a desired outcome. We use algorithmic ideas from robust optimization and prove convergence results for the most common machine learning methods, including decision trees, tree ensembles, and neural networks. Our experiments show that our method can efficiently generate globally optimal robust CEs for a variety of common data sets and classification models. History: Accepted by Andrea Lodi, Area Editor for Design & Analysis of Algorithms—Discrete. Funding: This work was supported by the Nederlandse Organisatie voor Wetenschappelijk Onderzoek [Grant OCENW.GROOT.2019.015, Optimization for and with Machine Learning (OPTIMAL)]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoc.2023.0153 .

A notorious problem in queueing theory is to compute the worst possible performance of the GI/G/1 queue under mean-dispersion constraints for the interarrival- and service-time distributions. We address this extremal queue problem by measuring dispersion in terms of mean absolute deviation (MAD) instead of the more conventional variance, making available methods for distribution-free analysis. Combined with random walk theory, we obtain explicit expressions for the extremal interarrival- and service-time distributions and, hence, the best possible upper bounds for all moments of the waiting time. We also obtain tight lower bounds that, together with the upper bounds, provide robust performance intervals. We show that all bounds are computationally tractable and remain sharp also when the mean and MAD are not known precisely but are estimated based on available data instead. Summary of Contribution: Queueing theory is a classic OR topic with a central role for the GI/G/1 queue. Although this queueing system is conceptually simple, it is notoriously hard to determine the worst-case expected waiting time when only knowing the first two moments of the interarrival- and service-time distributions. In this setting, the exact form of the extremal distribution can only be determined numerically as the solution to a nonconvex nonlinear optimization problem. Our paper demonstrates that using mean absolute deviation (MAD) instead of variance alleviates the computational intractability of the extremal GI/G/1 queue problem, enabling us to state the worst-case distributions explicitly.

We derive computationally tractable formulations of the robust counterparts of convex quadratic and conic quadratic constraints that are concave in matrix-valued uncertain parameters. We do this for a broad range of uncertainty sets. Our results provide extensions to known results from the literature. We also consider hard quadratic constraints: those that are convex in uncertain matrix-valued parameters. For the robust counterpart of such constraints, we derive inner and outer tractable approximations. As an application, we show how to construct a natural uncertainty set based on a statistical confidence set around a sample mean vector and covariance matrix and use this to provide a tractable reformulation of the robust counterpart of an uncertain portfolio optimization problem. We also apply the results of this paper to norm approximation problems. Summary of Contribution: This paper develops new theoretical results and algorithms that extend the scope of a robust quadratic optimization problem. More specifically, we derive computationally tractable formulations of the robust counterparts of convex quadratic and conic quadratic constraints that are concave in matrix-valued uncertain parameters. We also consider hard quadratic constraints: those that are convex in uncertain matrix-valued parameters. For the robust counterpart of such constraints, we derive inner and outer tractable approximations.

Maximizing a convex function over convex constraints is an NP-hard problem in general. We prove that such a problem can be reformulated as an adjustable robust optimization (ARO) problem in which each adjustable variable corresponds to a unique constraint of the original problem. We use ARO techniques to obtain approximate solutions to the convex maximization problem. In order to demonstrate the complete approximation scheme, we distinguish the cases in which we have just one nonlinear constraint and multiple linear constraints. Concerning the first case, we give three examples in which one can analytically eliminate the adjustable variable and approximately solve the resulting static robust optimization problem efficiently. More specifically, we show that the norm constrained log-sum-exp (geometric) maximization problem can be approximated by (convex) exponential cone optimization techniques. Concerning the second case of multiple linear constraints, the equivalent ARO problem can be represented as an adjustable robust linear optimization problem. Using linear decision rules then returns a safe approximation of the constraints. The resulting problem is a convex optimization problem, and solving this problem gives an upper bound on the global optimum value of the original problem. By using the optimal linear decision rule, we obtain a lower bound solution as well. We derive the approximation problems explicitly for quadratic maximization, geometric maximization, and sum-of-max-linear-terms maximization problems with multiple linear constraints. Numerical experiments show that, contrary to the state-of-the-art solvers, we can approximate large-scale problems swiftly with tight bounds. In several cases, we have equal upper and lower bounds, which concludes that we have global optimality guarantees in these cases. Summary of Contribution: Maximizing a convex function over a convex set is a hard optimization problem. We reformulate this problem as an optimization problem under uncertainty, which allows us to transfer the hardness of this problem from its nonconvexity to the uncertainty of the new problem. The equivalent uncertain optimization problem can be relaxed tightly by using adjustable robust optimization techniques. In addition to building a new bridge between convex maximization and robust optimization, this approach also gives us strong algorithms that improve the state-of-the-art optimization solvers both in solution time and quality for various convex maximization problems.

In this paper, we focus on a subclass of quadratic optimization problems, that is, disjoint bilinear programming problems. We show that disjoint bilinear programming problems can be cast as two-stage robust linear optimization problems with fixed-recourse and right-hand-side uncertainty, and techniques for two-stage robust optimization can be used to solve the resulting problems. To this end, a scheme based on a blending of Fourier-Motzkin elimination and linear decision rules is used. Moreover, we show that the approximation via linear decision rules for the two-stage robust optimization reformulation is equivalent to applying a reformulation-linearization technique to the original disjoint bilinear problem. We then extend our approach to solve general bilinear problems. Numerical experiments on bimatrix games and concave quadratic minimization problems show that the proposed method is superior to the off-the-shelf solvers SCIP and CPLEX.

In this paper we consider uncertain second-order cone (SOC) and semidefinite programming (SDP) constraints with polyhedral uncertainty, which are in general computationally intractable. We propose to reformulate an uncertain SOC or SDP constraint as a set of adjustable robust linear optimization constraints withan ellipsoidal or semidefinite representable uncertainty set, respectively. The resulting adjustable problem can then (approximately) be solved by using adjustable robust linear optimization techniques. For example, we show that if linear decision rules are used, then the final robust counterpart consists of SOC or SDP constraints, respectively, which have the same computational complexity as the nominal version of the original constraints. We propose an efficient method to obtain good lower bounds. Moreover, we extend our approach to other classes of robust optimization problems, such as nonlinear problems that contain waitand-see variables or linear problems that contain bilinear uncertainty. Numerically, we apply our approach to reformulate the problem on finding the minimum volume circumscribing ellipsoid of a polytope, and solvethe resulting reformulation with linear and quadratic decision rules as well as Fourier-Motzkin elimination. We demonstrate the effectiveness and efficiency of the proposed approach by comparing it with the state-ofthe-art copositive approach. Moreover, we apply the proposed approach to a robust regression problem and a robust sensor network problem, and use linear decision rules to solve the resulting adjustable robust linear optimization problems, which solves the problem to (near) optimality.

Although robust optimization is a powerful technique in dealing with uncertainty in optimization, its solutions can be too conservative. More specifically, it can lead to an objective value much worse than the nominal solution or even to infeasibility of the robust problem. In practice, this can lead to robust solutions being disregarded in favor of the nominal solution. This conservatism is caused by both the constraint-wise approach of robust optimization and its core assumption that all constraints are hard for all scenarios in the uncertainty set. This paper seeks to alleviate this conservatism by proposing an alternative robust formulation that condenses all uncertainty into a single constraint, binding the worst-case expected violation in the original constraints from above. Using recent results in distributionally robust optimization, the proposed formulation is shown to be tractable for both right- and left-hand side uncertainty. A computational study is performed with problems from the NETLIB library. For some problems, the percentage of uncertainty is magnified fourfold in terms of increase in objective value of the standard robust solution compared with the nominal solution, whereas we find solutions that safeguard against over half the violation at only a 10th of the cost in objective value. For problems with an infeasible standard robust counterpart, the suggested approach is still applicable and finds both solutions that safeguard against most of the uncertainty at a low price in terms of objective value.

In radiation therapy planning, uncertainties in the definition of the target volume yield a risk of underdosing the tumor. The traditional corrective action in the context of external beam radiotherapy (EBRT) expands the clinical target volume (CTV) with an isotropic margin to obtain the planning target volume (PTV). However, the EBRT-based PTV concept is not directly applicable to brachytherapy (BT) since it can lead to undesirable dose escalation. Here, we present a treatment plan optimization model that uses worst-case robust optimization to account for delineation uncertainties in interstitial high-dose-rate BT of the prostate. A scenario-based method was developed that handles uncertainties in index sets. Heuristics were included to reduce the calculation times to acceptable proportions. The approach was extended to account for delineation uncertainties of an organ at risk (OAR) as well. The method was applied on data from prostate cancer patients and evaluated in terms of commonly used dosimetric performance criteria for the CTV and relevant OARs. The robust optimization approach was compared against the classical PTV margin concept and against a scenario-based CTV margin approach. The results show that the scenario-based margin and the robust optimization method are capable of reducing the risk of underdosage to the tumor. As expected, the scenario-based CTV margin approach leads to dose escalation within the target, whereas this can be prevented with the robust model. For cases where rectum sparing was a binding restriction, including uncertainties in rectum delineation in the planning model led to a reduced risk of a rectum overdose, and in some cases, to reduced targetcoverage. The online supplement is available at https://doi.org/10.1287/ijoc.2018.0815 .

We introduce a novel scheme based on a blending of Fourier-Motzkin elimination (FME) and adjustable robust optimization techniques to compute the maximum volume inscribed ellipsoid (MVE) in a polytopic projection. It is well-known that deriving an explicit description of a projected polytope is NP-hard. Our approach does not require an explicit description of the projection, and can easily be generalized to find a maximally sized convex body of a polytopic projection. Our obtained MVE is an inner approximation of the projected polytope, and its center is a centralized relative interior point of the projection. Since FME may produce many redundant constraints, we apply an LP-based procedure to keep the description of the projected polytopes at its minimal size. Furthermore, we propose an upper bounding scheme to evaluate the quality of the inner approximations. We test our approach on a simple polytope and a color tube design problem, and observe that as more auxiliary variables are eliminated, our inner approximations and upper bounds converge to optimal solutions. The online supplement is available at https://doi.org/10.1287/ijoc.2017.0763 .

Robust optimization is a methodology that can be applied to problems that are affected by uncertainty in their parameters. The classical robust counterpart of a problem requires the solution to be feasible for all uncertain parameter values in a so-called uncertainty set and offers no guarantees for parameter values outside this uncertainty set. The globalized robust counterpart (GRC) extends this idea by allowing controlled constraint violations in a larger uncertainty set. The constraint violations are controlled by the distance of the parameter from the original uncertainty set. We derive tractable GRCs that extend the initial GRCs in the literature: our GRC is applicable to nonlinear constraints instead of only linear or conic constraints, and the GRC is more flexible with respect to both the uncertainty set and distance measure function, which are used to control the constraint violations. In addition, we present a GRC approach that can be used to provide an extended trade-off overview between the objective value and several robustness measures.

In this paper we propose a methodology for constructing decision rules for integer and continuous decision variables in multiperiod robust linear optimization problems. This type of problem finds application in, for example, inventory management, lot sizing, and manpower management. We show that by iteratively splitting the uncertainty set into subsets, one can differentiate the later-period decisions based on the revealed uncertain parameters. At the same time, the problem’s computational complexity stays at the same level, as for the static robust problem. This also holds in the nonfixed recourse situation. In the fixed recourse situation our approach can be combined with linear decision rules for the continuous decision variables. We provide theoretical results on splitting the uncertainty set by identifying sets of uncertain parameter scenarios to be divided for an improvement in the worst-case objective value. Based on this theory, we propose several splitting heuristics. Numerical examples entailing a capital budgeting and a lot sizing problem illustrate the advantages of the proposed approach.

This paper proposes a new way to construct uncertainty sets for robust optimization. Our approach uses the available historical data for the uncertain parameters and is based on goodness-of-fit statistics. It guarantees that the probability the uncertain constraint holds is at least the prescribed value. Compared to existing safe approximation methods for chance constraints, our approach directly uses the historical data information and leads to tighter uncertainty sets and therefore to better objective values. This improvement is significant, especially when the number of uncertain parameters is low. Other advantages of our approach are that it can handle joint chance constraints easily, it can deal with uncertain parameters that are dependent, and it can be extended to nonlinear inequalities. Several numerical examples illustrate the validity of our approach.

In many fields, we come across problems where we want to optimize several conflicting objectives simultaneously. To find a good solution for such multiobjective optimization problems, an approximation of the Pareto set is often generated. In this paper, we consider the approximation of higher-dimensional convex Pareto sets using sandwich algorithms. We extend higher-dimensional sandwich algorithms in three different ways. First, we introduce the new concept of adding dummy points to the inner approximation of a Pareto set. By using these dummy points, we can determine accurate inner and outer approximations more efficiently, i.e., using less time-consuming optimizations. Second, we introduce a new method for the calculation of an error measure that is easy to interpret. Third, we show how transforming certain objective functions can improve the results of sandwich algorithms and extend their applicability to certain nonconvex problems. To show the effect of these enhancements, we make a numerical comparison using four test cases, including a four-dimensional case from the field of intensity-modulated radiation therapy. The results of the different cases show that we can achieve an accurate approximation using significantly fewer optimizations by using the enhancements.

In this paper, piecewise-linear upper and lower bounds for univariate convex functions are derived that are only based on function value information. These upper and lower bounds can be used to approximate univariate convex functions. Furthermore, new sandwich algorithms are proposed that iteratively add new input data points in a systematic way until a desired accuracy of the approximation is obtained. We show that our new algorithms that use only function value evaluations converge quadratically under certain conditions on the derivatives. Under other conditions, linear convergence can be shown. Some numerical examples that illustrate the usefulness of the algorithm, including a strategic investment model, are given.

The design of computer experiments is an important step in black-box evaluation and optimization processes. When dealing with multiple black-box functions the need often arises to construct designs for all black boxes jointly, instead of individually. These so-called nested designs are particularly useful as training and test sets for fitting and validating metamodels, respectively. Furthermore, nested designs can be used to deal with linking parameters and sequential evaluations. In this paper, we introduce one-dimensional nested maximin designs. We show how to nest two designs optimally and develop a heuristic to nest three and four designs. These nested maximin designs can be downloaded from the website http://www.spacefillingdesigns.nl . Furthermore, it is proven that the loss in space-fillingness, with respect to traditional maximin designs, is at most 14.64 and 19.21%, when nesting two and three designs, respectively.

Symbolic regression of input-output data conventionally treats data records equally. We suggest a framework for automatic assignment of weights to data samples, which takes into account the sample's relative importance. In this paper, we study the possibilities of improving symbolic regression on real-life data by incorporating weights into the fitness function. We introduce four weighting schemes defining the importance of a point relative to proximity, surrounding, remoteness, and nonlinear deviation fromknearest-in-the-input-space neighbors. For enhanced analysis and modeling of large imbalanced data sets we introduce a simple multidimensional iterative technique for subsampling. This technique allows a sensible partitioning (and compression) of data to nested subsets of an arbitrary size in such a way that the subsets are balanced with respect to either of the presented weighting schemes. For cases where a given input-output data set contains some redundancy, we suggest an approach to considerably improve the effectiveness of regression by applying more modeling effort to a smaller subset of the data set that has a similar information content. Such improvement is achieved due to better exploration of the search space of potential solutions at the same number of function evaluations. We compare different approaches to regression on five benchmark problems with a fixed budget allocation. We demonstrate that the significant improvement in the quality of the regression models can be obtained either with the weighted regression, exploratory regression using a compressed subset with a similar information content, or exploratory weighted regression on the compressed subset, which is weighted with one of the proposed weighting schemes.

This paper presents a novel approach to generate data-driven regression models that not only give reliable prediction of the observed data but also have smoother response surfaces and extra generalization capabilities with respect to extrapolation. These models are obtained as solutions of a genetic programming (GP) process, where selection is guided by a tradeoff between two competing objectives - numerical accuracy and the order of nonlinearity. The latter is a novel complexity measure that adopts the notion of the minimal degree of the best-fit polynomial, approximating an analytical function with a certain precision. Using nine regression problems, this paper presents and illustrates two different strategies for the use of the order of nonlinearity in symbolic regression via GP. The combination of optimization of the order of nonlinearity together with the numerical accuracy strongly outperforms ldquoconventionalrdquo optimization of a size-related expressional complexity and the accuracy with respect to extrapolative capabilities of solutions on all nine test problems. In addition to exploiting the new complexity measure, this paper also introduces a novel heuristic of alternating several optimization objectives in a 2-D optimization framework. Alternating the objectives at each generation in such a way allows us to exploit the effectiveness of 2-D optimization when more than two objectives are of interest (in this paper, these are accuracy, expressional complexity, and the order of nonlinearity). Results of the experiments on all test problems suggest that alternating the order of nonlinearity of GP individuals with their structural complexity produces solutions that are both compact and have smoother response surfaces, and, hence, contributes to better interpretability and understanding.

The Rummikub problem of finding the maximal number or value of the tiles that can be placed from your rack onto the table is very difficult, since the number of possible combinations are enormous. We show that this problem can be modeled as an integer linear programming problem. In this way solutions can be found in 1 s. We extend the model such that unnecessary changes of the existing sets on the table are minimized.