In binary classification problems, many performance metrics evaluate the probability that some error exceeds a threshold. Nevertheless, they focus more on the probability and fail to capture the magnitude of the error, which evaluates how large this error exceeds the threshold. Capturing the magnitude of error is desired in many applications. For example, in detecting disease and predicting credit default, the magnitude of error illustrates the confidence in making the wrong prediction. We propose a novel metric, the Fragility Index (FI), to evaluate the performance of binary classifiers by capturing the magnitude of the error. FI alleviates the risk of misclassification by penalizing the large error greatly, which is seldom considered by standard metrics. Moreover, to strengthen the generalization ability and handle unseen samples, we adopt the framework of distributionally robust optimization and robust satisficing, which allows us to derive and control the maximum degree of fragility of the classifier when the distribution of samples shifts. We show that FI can be easily calculated and optimized for common probabilistic distance measures. Experiments with real datasets demonstrate the new insights brought by FI and the advantages of classifiers selected under FI, which always improve the robustness and reduce the risk of large errors as compared to classifiers selected by alternative metrics.

In this paper, we propose a decision criterion that characterizes an enveloping bound on monetary risk measures and is computationally friendly. We start by extending the classical value at risk (VaR) measure. Whereas VaR evaluates the threshold loss value such that the loss from the risk position exceeding that threshold is at a given probability level, it fails to indicate a performance guarantee at other probability levels. We define the probabilistic enveloping measure (PEM) to establish the bound information for the tail probability of the loss at all levels. Using a set of normative properties, we then generalize the PEM to the risk enveloping measure (REM) such that the bound on the general monetary risk measures at all levels of risk aversion are captured. The coherent version of the REM (CREM) is also investigated. We demonstrate its applicability by showing how the coherent REM can be incorporated in distributionally robust optimization. Specifically, we apply the CREM criterion in surgery block allocation problems and provide a formulation that can be efficiently solved. Based on this application, we report favorable computational results from optimizing over the CREM criterion. Summary of Contribution: Our paper studies a fundamental problem in operations research: what criteria to optimize when uncertainties are involved. Extending from the classical chance constraint model, we propose a new decision criterion by an axiomatization approach. We then investigate the computing issue in the corresponding distributionally robust optimization problem. In particular, we provide solution methods for continuous and discrete optimization. After that, we apply it to a practical operations problem, surgery allocation decisions in healthcare management. The computational studies demonstrate the appealing performance of our proposed approach on this surgery allocation problem.

We consider a significant problem that arises in the planning of many projects. Project companies often use outsourced providers that require capacity reservations that must be contracted before task durations are realized. We model these decisions for a company that, given partially characterized distributional information, assumes the worst-case distribution for task durations. Once task durations are realized, the project company makes decisions about fast tracking and outsourced crashing, to minimize the total capacity reservation, fast tracking, crashing, and makespan penalty costs. We model the company’s objective using the target-based measure of minimizing an underperformance riskiness index. We allow for correlation in task performance, and for piecewise linear costs of crashing and makespan penalties. An optimal solution of the discrete, nonlinear model is possible for small to medium size projects. We compare the performance of our model against the best available benchmarks from the robust optimization literature, and show that it provides lower risk and greater robustness to distributional information. Our work thus enables more effective risk minimization in projects, and provides insights about how to make more robust capacity reservation decisions. Summary of Contribution: This work studies a financially significant planning problem that arises in project management. Companies that face uncertainties in project execution may need to reserve capacity with outsourced providers. Given that decision, they further need to plan their operational decisions to protect against a bad outcome. We model and solve this problem via adjustable distributionally robust optimization. While this problem involves two-stage decision making, which is computationally challenging in general, we develop a computationally efficient algorithm to find the exact optimal solution for instances of practical size.