We present a production LLM-based taxonomy classification system deployed at The Washington Post that tags news content across five schemas (Subject, Person, Company, Organization, Geography) using a proprietary taxonomy of ∼ 20,400 entries across seven hierarchical levels. For the Subject schema, we employ embedding-based candidate filtering followed by LLM selection. For other schemas, we combine LLM-based named entity extraction with fuzzy n-gram matching, followed by LLM selection. Comparison of post-production F1 scores against commercial vendor baselines demonstrates significant improvements across all five schemas, with the most substantial gain in Subject schema (+29.3%, p < 0.001). The system processes hundreds to thousands of articles and news items daily with a mean latency of 3–4 seconds per request and supports zero-downtime taxonomy updates.

We describe how The Washington Post deployed an offline reinforcement learning (RL) system to optimize paywall decisioning at production scale. We cast each non-subscriber article access attempt as a sequential decision with three actions: free access, registration wall, or subscription paywall, and learn policies from logged data collected via a small-traffic randomized controlled trial and subsequent production logging. We iterated from a tabular Q-learning baseline to a deep offline RL model trained with Conservative Q-Learning (CQL), using off-policy evaluation primarily to screen and rank candidates before online testing. The system was rolled out with guardrails and a persistent randomized holdout to manage risk in a revenue-critical setting. In year-long online experiments, the learned policies outperformed the legacy rules-based metering policy and improved a stakeholder-weighted value metric; the CQL policy delivered a +3% lift versus the randomized baseline while increasing subscriptions (+6%) and reducing the registration gap relative to earlier RL iterations. This case study highlights the practical steps needed to safely train, evaluate, and deploy offline RL for high-stakes personalization.

Publishers increasingly rely on access gates to drive registrations and subscriptions. Determining when to present these gates is a sequential decision problem well suited to reinforcement learning (RL). However, online exploration is costly and risky due to delayed conversion signals. We introduce Uncertainty-Aware Advantage-Weighted Actor–Critic (UA-AWAC), an offline RL method that learns from logged traffic to produce conversion-ready policies. UA-AWAC optimizes a multi-objective reward incorporating subscriptions, registrations, and engagement, while mitigating distribution shift through epistemic uncertainty modeling and pessimistic value targets. The policy is trained using advantage-weighted behavioral cloning with Kullback–Leibler (KL) regularization to remain close to historical gating behavior. Experiments show that UA-AWAC improves subscription rate by up to 10% and registration rate by 62% compared to baseline and state-of-the-art offline RL methods, demonstrating a practical and stable solution for intelligent content gating where exploration risks are high.

Mission Impact Assessment (MIA) is critical for enhancing system effectiveness and ensuring mission success. This article presents Interdependent Mission Impact Assessment ( iMIA ), an interdependent MIA framework that models relationships among mission components and enables probabilistic reasoning under uncertainty. Designed for AI-driven mission systems operating in dynamic, low-data, or poorly observable environments, iMIA addresses the limitations of traditional methods that often rely on overly confident assumptions about adversary behavior. While conventional Hypergame Theory (HGT) captures perceptual uncertainty from asymmetric or inaccurate views, it overlooks epistemic uncertainty arising from limited knowledge. To bridge this gap, we introduce a hybrid Subjective Logic (SL)-based HGT model (SLHG), integrating SL to represent epistemic uncertainty and HGT to account for misperceptions. This integration supports informed decision-making under both uncertain strategy beliefs and divergent environmental views. iMIA evaluates mission impact using multidimensional system quality metrics, security, trust, resilience, and agility, across diverse attacker–defender interactions. It identifies critical nodes influencing mission outcomes and quantifies performance gains from asset capacity reinforcement and asset vulnerability mitigation. Applied to a vehicle-assisted AI-based mission system, iMIA with SLHG improves performance by 16% in \(ASR\) , 20% in \(MTBF\) , 11% in \(TSA\) , and 14% in \(P_{ACC}\) . Designed for incremental development, iMIA supports continuous feedback and iterative refinement. Our results show that feedback-driven adjustments improve overall system performance by up to 18% in the accuracy performance.

An accurate assessment of a mission system’s performance, called mission impact assessment (MIA), can effectively identify and counteract any issues in the current system to avoid potential risks or vulnerabilities that may cause mission failure. Although the importance of MIA research has been recognized to mitigate the system’s potential risk for more than a decade, little research has shown a comprehensive MIA framework with experimental validation showing the inference performance of the MIA tools under realistic attack–defense interactions. To fill this gap, we propose an interdependent MIA, callediMIA, that can adequately capture the interrelationships of the key components in a mission system and environment and accurately infer a mission outcome (i.e., success or failure) under uncertainty. In the proposediMIA, we first consider Subjective Bayesian Networks to evaluate how successfully a given system introduces impacts on mission success under uncertainty due to a lack of evidence. In addition, we consider strategic attack-defense interactions based on a hypergame theory in which the attacker and defender can take actions under their perceived uncertainty. Our extensive experiments showed the outperformance of our proposediMIAup to 70% and 25% over the performance of the baseline and the state-of-the-art Bayesian network (BN)-based counterparts, respectively, in terms of inference accuracy in mission performance (e.g., mission outcomes, such as mission success or failure). We also provide in-depth sensitivity analyses to identify the key system components that should be considered more carefully to avoid mission failure.

Subjective Bayesian networks (SBN) integrate Bayesian Networks (BN) with Subjective Logic, enabling the representation of second-order uncertainty, denoting the uncertainty surrounding the probability distribution of an event. Although prior research predominantly centers on exact inference within the SBN framework, there is a notable dearth of exploration into the realm of approximate inference in SBN. Our work is specifically geared towards addressing this gap, focusing on the application of diverse sampling methodologies (i.e., forward and Gibbs sampling) for approximate inference in SBN. The primary contribution of this work lies not only in the introduction of approximate inference in SBN but also in the formulation of an “anytime” SBN inference algorithm. This implies that a best inference estimate can be obtained at any given moment, given trade-offs in the precision. Moreover, the allocation of computational resources is a customizable and potentially optimizable process. Through a rigorous series of experiments, we empirically demonstrate that the number of iterations to convergence decreases as we provide more samples for both forward and Gibbs sampling. Furthermore, we discover the difference between approximate and exact inference in belief ($\delta_{\text {belief }}$) and uncertainty ($\delta_{\text {uncertainty }}$) mass of subjective opinion becomes more unpredictable as the error gets large in BN probability. Lastly, in our experiments, we demonstrate the number of BN samples has a greater impact on $\delta_{\text {belief }}$ than the number of SBN iterations. These findings indicate that the family of greedy algorithms (based on local graded changes - such as gradients) can be a promising approach for finding optimal allocations of computational resources in this framework. The software assets produced and used in this work will be made available as an open source Python library.

A mission impact assessment (MIA) framework assesses a mission system’s performance and/or aims to identify risk factors of mission failure to take a mitigation strategy. The iMIA framework, unlike traditional MIA approaches, comprehensively addresses the interdependencies of key system components like asset vulnerabilities, attack behaviors, defense mechanisms, and service/task characteristics. This framework goes beyond conceptual models, providing a validated and detailed MIA framework covering from a conceptual model to detailed mission and system designs. Existing MIA approaches with Bayesian Networks (BNs) overlook inherent uncertainties arising from factors like insufficient evidence or data in real-world applications. To address decision-making challenges in the face of uncertainties, we introduce Subjective Bayesian Networks (SBNs). SBNs estimate uncertainties and interpret them using an expert’s domain knowledge or historical data, forming a subjective opinion through prior belief in SBNs. Using the SBNs, we enhance iMIA’s inference accuracy in a Federated Learning-based mission system (FLMS) for vehicular networks. Our experiments show thatiMIA’s SBN reasoning significantly improves mission outcome inference accuracy compared to conventional BN-based reasoning in existing MIA approaches. We also evaluate mission performance based on service availability and prediction accuracy from the FLMS.

Subjective Bayesian networks (SBN) combine Bayesian Networks (BN) with Subjective Logic in order to express the second-order uncertainty (i.e., the uncertainty about a probability distribution of an event – as opposed to the uncertainty about the event itself). While SBNs provide a strong formalism for treating the uncertainty in a higher level, the literature lacks support for extensive software implementations focused on compatibility with current software solutions or standards. Our work explores the structural congruence between BN and SBN (in terms of software data structure) and a semantic bijection between Subjective Logic opinions to Dirichlet distributions to introduce a SBN reasoning framework that targets on effectively reusing the existing BN solutions. Particularly, we developed two inference algorithms that apply a Monte Carlo method to existing BN inference algorithms (we chose the Junction Tree algorithm, for test), respectively for batch and interactive SBN reasoning. A method for translating an evidence in SBN to uncertain (virtual) evidence in BN is also presented. The main contribution of this paper is the introduction of a simple, yet flexible empirical estimation method and a software architecture that virtually adapts any BN inference algorithm to an approximate computational inference framework for SBN. We also developed a Java component to demonstrate the reusability, and we developed a case study of Mission Impact Assessment of Unmanned Aerial Vehicles transporting critical items between hospitals in order to illustrate the applicability in a knowledge engineering process.

Mission Impact Assessment (MIA) is a critical endeavor for evaluating the performance of mission systems, encompassing intricate elements such as assets, services, tasks, vulnerability, attacks, and defenses. This study introduces an innovative MIA framework that transcends existing methodologies by intricately modeling the interdependencies among these components. Additionally, we integrate hypergame theory to address the strategic dynamics of attack-defense interactions. To illustrate its practicality, we apply the framework to an Internet-of-Things (IoT)-based mission system tailored for accurate, time-sensitive object detection. Rigorous simulation experiments affirm the framework's robustness across a spectrum of scenarios. Our results prove that the developed MIA framework shows a sufficiently high inference accuracy (e.g., 80 %) even with a small portion of the training dataset (e.g., 20–50 %).

Intrusion Detection System (IDS) is designed to protect network assets by inspecting specified region or properties of network traffic. To represent hierarchical traffic flows comprehensively, this paper proposes a deep learning architecture for anomaly detection. We employ CNN model to automatically extract an adequate feature vector for each message composed of unidirectional sequence of packets. Based upon the CNN structure, we build BiLSTM with attention mechanism to summarize the sequential message vectors as a representative flow feature vector. Empirical evidence from reduced UNSW-NB15 dataset indicates the superior performance of the proposed method for detecting anomalous traffic flows.