With the advancements of artificial intelligence (AI), emerging scenarios involving close collaboration between AI and other unknown agents are becoming increasingly common. This requires sometimes training AI agents to collaborate with unknown agents in the absence of a reward function -- which may be unavailable to the AI agents or even undefined by the unknown agents themselves -- thus posing news challenges to existing learning algorithms that often require knowing the shared reward. In this paper, we show that effective teaming with unknown agents can be achieved in the absence of a reward function, through actively modeling other unknown agents and reasoning about their latent rewards from available interaction/observation history. In particular, we propose a novel framework that leverages a kernel density Bayesian inverse learning method for active reward/goal inference and prove that multi-agent reinforcement learning guided by the inferred reward signals can converge to an optimal policy teaming with unknown agents. The result enables us to develop an adaptive policy update strategy, through the use of a family of pre-trained, goal-conditioned policies, further eliminating the need for online retraining. The proposed solution is evaluated using a wide range of diverse unknown agents of latent and even non-stationary reward. Our solution significantly increases the teaming performance between AI and unknown agents in the absence of reward.

Cybersickness remains a major challenge in virtual and mixed reality (VR/MR), yet existing methods primarily focus on predicting its onset without offering formal guarantees regarding its occurrence or effective mitigation. As VR/MR applications expand into safety-critical domains like healthcare, defense, verifiable safety assurances become essential to protect users from adverse physiological and psychological effects. This paper introduces a probabilistic verification framework leveraging Bayesian Networks (BN) to explicitly model the interactions among system parameters, human physiological responses, and cybersickness severity. Unlike deep learning approaches that lack interpretability and formal verification capabilities, the proposed BN model explicitly captures how environmental and system-level factors (e.g., luminance, spectral entropy, and image gradient complexity via HoG features) influence physiological responses (e.g., heart rate, reaction time, eye tracking), ultimately affecting cybersickness severity. By learning the joint probability distribution of these factors, our approach provides rigorous formal guarantees on cybersickness risk under specified operational conditions. If these guarantees are not met, automated adaptive adjustments are recommended to restore safe conditions. Experimental validation involving physiological and systemlevel data demonstrates that Bayesian Networks provide an interpretable and efficient framework, uniquely enabling formal probabilistic verification of cybersickness risks. This capability makes the proposed approach particularly suitable for designing and deploying VR/MR systems with explicitly verified safety constraints.

Augmented reality (AR) systems are increasingly deployed in tactical environments, but their reliance on seamless human-computer interaction makes them vulnerable to cognitive attacks that manipulate a user's perception and severely compromise user decisionmaking. To address this challenge, we introduce the Perception Graph, a novel model designed to reason about human perception within these systems. Our model operates by first mimicking the human process of interpreting key information from an MR environment and then representing the outcomes using a semantically meaningful structure. We demonstrate how the model can compute a quantitative score that reflects the level of perception distortion, providing a robust and measurable method for detecting and analyzing the effects of such cognitive attacks.

Cognitive attacks in mixed reality (MR), e.g., latency perturbations that induce frame-time jitter, can divert visual attention and degrade task performance. We study 2D gaze prediction under such disturbances and propose a time-aware sequence model that handles irregular sampling by supplying elapsed times Δt between observations and conditions on sparse event/object context available at prediction time via learned token embeddings. Using time-based windows, we evaluate within-user and cross-user temporal generalization on MR recordings spanning multiple attack intensities. Results indicate accurate, time-robust gaze regression under latency perturbations, supporting adaptive MR interfaces in adversarial settings.

Ensuring the safety of neural network (NN) models in mixed reality (MR) systems is challenging due to adversarial manipulation of system parameters. We present PolySafe, which extends DeepPoly and Prover to validate safety of LSTM-based MR models. PolySafe unrolls temporal dependencies, introduces multi-plane abstractions for tighter bounds, and establishes probabilistic safety guarantees. It further includes an adaptive search that identifies minimal sets of critical parameters required to be constrained for defense. Evaluation on an MR engagement prediction model shows that PolySafe provides rigorous and actionable safety assurances for deployment.

Personal characteristics fundamentally shape virtual reality (VR) experiences, yet their integration into predictive models remains underexplored. This paper studies how to incorporate personal attributes (age, gender, prior VR experience) into Bayesian networks for cybersickness prediction via: (i) direct inclusion as root nodes, (ii) a two-stage model that learns a susceptibility score from personal attributes, and (iii) a stratified model. Using 26,040 samples from VR maze-navigation experiments, direct inclusion attains 82.53% accuracy (+14.02 percentage points over a 68.51% no-personal baseline). The two-stage approach reaches 77.32% while supporting cold-start prediction for unseen users, and stratified models achieve 73.62%. Using participant-level cross-validation to avoid subject leakage, we find that personalization consistently improves cybersickness prediction. These results argue that personal attributes should be treated as first-class signals in cybersickness models, with clear design trade-offs between maximal accuracy and deployability for unseen users, informing personalized VR systems and adaptive content delivery.

Gene regulatory networks (GRNs) regulate essential cellular functions, and their dysregulation contributes to diseases such as cancer and autoimmune disorders. Designing effective interventions is challenging due to (i) the adaptive resistance of cells to therapies and (ii) the limited knowledge of genes' states during the intervention process through gene expression data. To address these challenges, this paper develops a decentralized deep reinforcement learning framework for intervention in GRNs. The intervention process is formulated as an asymmetric two-player zero-sum game, where the history-dependent intervention policy is derived against a cell that has complete knowledge of gene states. The optimal intervention policy is expressed as a Nash equilibrium policy, and a deep policy gradient approach is developed to approximate this policy. The analytical results demonstrate that under non-aggressive cell responses, the proposed intervention policy achieves higher-than-expected gains, ensuring robustness even against the most complex adaptive cellular responses. Furthermore, if the true system state becomes fully observable, the proposed method converges to the full-state Nash equilibrium. Numerical experiments on two benchmark GRN models, p53-MDM2 and melanoma regulatory networks, validate the proposed method, demonstrating its superior adaptability under uncertainty compared to state-of-the-art intervention strategies.

Bayesian optimization (BO) has established itself as a leading strategy for efficiently optimizing expensive-to-evaluate functions. Existing BO methods mostly rely on Gaussian process (GP) surrogate models and are not applicable to (doubly-stochastic) Gaussian Cox processes, where the observation process is modulated by a latent intensity function modeled as a GP. In this paper, we propose a novel maximum *a posteriori* inference of Gaussian Cox processes. It leverages the Laplace approximation and change of kernel technique to transform the problem into a new reproducing kernel Hilbert space, where it becomes more tractable computationally. It enables us to obtain both a functional posterior of the latent intensity function and the covariance of the posterior, thus extending existing works that often focus on specific link functions or estimating the posterior mean. Using the result, we propose a BO framework based on the Gaussian Cox process model and further develop a Nyström approximation for efficient computation. Extensive evaluations on various synthetic and real-world datasets demonstrate significant improvement over state-of-the-art inference solutions for Gaussian Cox processes, as well as effective BO with a wide range of acquisition functions designed through the underlying Gaussian Cox process model.

The increasing interconnectivity in our infrastructure poses a significant security challenge, with external threats having the potential to penetrate and propagate throughout the network. Bayesian attack graphs have proven to be effective in capturing the propagation of attacks in complex interconnected networks. However, most existing security approaches fail to systematically account for the limitation of resources and uncertainty arising from the complexity of attacks and possible undetected compromises. To address these challenges, this paper proposes a partially observable Markov decision process (POMDP) model for network security under uncertainty. The POMDP model accounts for uncertainty in monitoring and defense processes, as well as the probabilistic attack propagation. This paper develops two security policies based on the optimal stationary defense policy for the underlying POMDP state process (i.e., a network with known compromises): the estimation‐based policy that performs the defense actions corresponding to the optimal minimum mean square error state estimation and the distribution‐based policy that utilizes the posterior distribution of network compromises to make defense decisions. Optimal monitoring policies are designed to specifically support each of the defense policies, allowing dynamic allocation of monitoring resources to capture network vulnerabilities/compromises. The performance of the proposed policies is examined in terms of robustness, accuracy, and uncertainty using various numerical experiments.

The complexity, scale, and uncertainty in regulatory networks (e.g., gene regulatory networks and microbial networks) regularly pose a huge uncertainty in their models. These uncertainties often cannot be entirely reduced using limited and costly data acquired from the normal condition of systems. Meanwhile, regulatory networks often suffer from the non-identifiability issue, which refers to scenarios where the true underlying network model cannot be clearly distinguished from other possible models. Perturbation or excitation is a well-known process in systems biology for acquiring targeted data to reveal the complex underlying mechanisms of regulatory networks and overcome the non-identifiability issue. We consider a general class of Boolean network models for capturing the activation and inactivation of components and their complex interactions. Assuming partial available knowledge about the interactions between components of the networks, this paper formulates the inference process through the maximum aposteriori (MAP) criterion. We develop a Bayesian lookahead policy that systematically perturbs regulatory networks to maximize the performance of MAP inference under the perturbed data. This is achieved by optimally formulating the perturbation process in a reinforcement learning context and deriving a scalable deep reinforcement learning perturbation policy to compute near-optimal Bayesian policy. The proposed method learns the perturbation policy through planning without the need for any real data. The high performance of the proposed approach is demonstrated by comprehensive numerical experiments using the well-known mammalian cell cycle and gut microbial community networks.

A major objective in genomics is to design interventions that can shift undesirable behaviors of such systems (i.e., those associated with cancers) into desirable ones. Several intervention policies have been developed in recent years, including dynamic and structural interventions. These techniques aim at making targeted changes to cell dynamics upon intervention, without considering the cell's defensive mechanisms to interventions. This simplified assumption often leads to early and short-term success of interventions, followed by partial or full recurrence of diseases. This is due to the fact that cells often have dynamic and intelligent responses to interventions through internal stimuli. This paper models gene regulatory networks (GRNs) using the Boolean network with perturbation. The dynamic and adaptive battle between intervention and the cell is modeled as a two-player zero-sum game, where intervention and the cell fight against each other with fully opposite objectives. An optimal intervention policy is obtained as a Nash equilibrium solution, through which the intervention is stochastic, ensuring the optimal solution to all potential cell responses. We analytically analyze the superiority of the proposed intervention policy against existing intervention techniques. Comprehensive numerical experiments using the p53-MDM2 negative feedback loop regulatory network and melanoma network demonstrate the high performance of the proposed method.

Bayesian optimization (BO) is a popular global optimization scheme for sample-efficient optimization in domains with expensive function evaluations. The existing BO techniques are capable of finding a single global optimum solution. However, finding a set of global and local optimum solutions is crucial in a wide range of real-world problems, as implementing some of the optimal solutions might not be feasible due to various practical restrictions (e.g., resource limitation, physical constraints, etc.). In such domains, if multiple solutions are known, the implementation can be quickly switched to another solution, and the best possible system performance can still be obtained. This paper develops a multimodal BO framework to effectively find a set of local/global solutions for expensive-to-evaluate multimodal objective functions. We consider the standard BO setting with Gaussian process regression representing the objective function. We analytically derive the joint distribution of the objective function and its first-order derivatives. This joint distribution is used in the body of the BO acquisition functions to search for local optima during the optimization process. We introduce variants of the well-known BO acquisition functions to the multimodal setting and demonstrate the performance of the proposed framework in locating a set of local optimum solutions using multiple optimization problems.

Abstract Early attack detection is essential to ensure the security of complex networks, especially those in critical infrastructures. This is particularly crucial in networks with multi-stage attacks, where multiple nodes are connected to external sources, through which attacks could enter and quickly spread to other network elements. Bayesian attack graphs (BAGs) are powerful models for security risk assessment and mitigation in complex networks, which provide the probabilistic model of attackers’ behavior and attack progression in the network. Most attack detection techniques developed for BAGs rely on the assumption that network compromises will be detected through routine monitoring, which is unrealistic given the ever-growing complexity of threats. This paper derives the optimal minimum mean square error (MMSE) attack detection and monitoring policy for the most general form of BAGs. By exploiting the structure of BAGs and their partial and imperfect monitoring capacity, the proposed detection policy achieves the MMSE optimality possible only for linear-Gaussian state space models using Kalman filtering. An adaptive resource monitoring policy is also introduced for monitoring nodes if the expected predictive error exceeds a user-defined value. Exact and efficient matrix-form computations of the proposed policies are provided, and their high performance is demonstrated in terms of the accuracy of attack detection and the most efficient use of available resources using synthetic Bayesian attack graphs with different topologies.

Traditional robot control or more general continuous control tasks often rely on carefully hand-crafted classic control methods. These models often lack the self-learning adaptability and intelligence to achieve human-level control. On the other hand, recent advancements in Reinforcement Learning (RL) present algorithms that have the capability of human-like learning. The integration of Deep Neural Networks (DNN) and RL thereby enables autonomous learning in robot control tasks. However, DNN-based RL brings both high-quality learning and high computation cost, which is no longer ideal for currently fast-growing edge computing scenarios.

In this paper, we propose BioHD, a novel genomic sequence searching platform based on Hyper-Dimensional Computing (HDC) for hardware-friendly computation. BioHD transforms inherent sequential processes of genome matching to highly-parallelizable computation tasks. We exploit HDC memorization to encode and represent the genome sequences using high-dimensional vectors. Then, it combines the genome sequences to generate an HDC reference library. During the sequence searching, BioHD performs exact or approximate similarity check of an encoded query with the HDC reference library. Our framework simplifies the required sequence matching operations while introducing a statistical model to control the alignment quality. To get actual advantage from BioHD inherent robustness and parallelism, we design a processing in-memory (PIM) architecture with massive parallelism and compatible with the existing crossbar memory. Our PIM architecture supports all essential BioHD operations natively in memory with minimal modification on the array. We evaluate BioHD accuracy and efficiency on a wide range of genomics data, including COVID-19 databases. Our results indicate that PIM provides 102.8× and 116.1× (9.3× and 13.2×) speedup and energy efficiency compared to the state-of-the-art pattern matching algorithm running on GeForce RTX 3060 Ti GPU (state-of-the-art PIM accelerator).

Data in many practical problems are acquired according to decisions or actions made by users or experts to achieve specific goals. For instance, policies in the mind of biologists during the intervention process in genomics and metagenomics are often reflected in available data in these domains, or data in cyber-physical systems are often acquired according to actions/decisions made by experts/engineers for purposes, such as control or stabilization. Quantification of experts' policies through available data, which is also known as reward function learning, has been discussed extensively in the literature in the context of inverse reinforcement learning (IRL). However, most of the available techniques come short to deal with practical problems due to the following main reasons: 1) lack of scalability: arising from incapability or poor performance of existing techniques in dealing with large systems and 2) lack of reliability: coming from the incapability of the existing techniques to properly learn the optimal reward function during the learning process. Toward this, in this brief, we propose a multifidelity Bayesian optimization (MFBO) framework that significantly scales the learning process of a wide range of existing IRL techniques. The proposed framework enables the incorporation of multiple approximators and efficiently takes their uncertainty and computational costs into account to balance exploration and exploitation during the learning process. The proposed framework's high performance is demonstrated through genomics, metagenomics, and sets of random simulated problems.

State-space models (SSMs) are a rich class of dynamical models with a wide range of applications in economics, healthcare, computational biology, robotics, and more. Proper analysis, control, learning, and decision-making in dynamical systems modeled by SSMs depend on the accuracy of the inferred/learned model. Most of the existing inference techniques for SSMs are capable of dealing with very small systems, unable to be applied to most of the large-scale practical problems. Toward this, this article introduces a two-stage Bayesian optimization (BO) framework for scalable and efficient inference in SSMs. The proposed framework maps the original large parameter space to a reduced space, containing a small linear combination of the original space. This reduced space, which captures the most variability in the inference function (e.g., log likelihood or log a posteriori), is obtained by eigenvalue decomposition of the covariance of gradients of the inference function approximated by a particle filtering scheme. Then, an exponential reduction in the search space of parameters during the inference process is achieved through the proposed two-stage BO policy, where the solution of the first-stage BO policy in the reduced space specifies the search space of the second-stage BO in the original space. The proposed framework's accuracy and speed are demonstrated through several experiments, including real metagenomics data from a gut microbial community.

Design is an inseparable part of most scientific and engineering tasks, including real and simulation-based experimental design processes and parameter/hyperparameter tuning/optimization. Several model-based experimental design techniques have been developed for design in domains with partial available knowledge about the underlying process. This article focuses on a powerful class of model-based experimental design called the mean objective cost of uncertainty (MOCU). The MOCU-based techniques are objective-based, meaning that they take the main objective of the process into account during the experimental design process. However, the lack of scalability of MOCU-based techniques prevents their application to most practical problems, including large discrete or combinatorial spaces. To achieve a scalable objective-based experimental design, this article proposes a graph-based MOCU-based Bayesian optimization framework. The correlations among samples in the large design space are accounted for using a graph-based Gaussian process, and an efficient closed-form sequential selection is achieved through the well-known expected improvement policy. The proposed framework's performance is assessed through the structural intervention in gene regulatory networks, aiming to make the network away from the states associated with cancer.

Partially-Observed Boolean dynamical systems (POBDS) are a general class of nonlinear state-space models consisting of a hidden Boolean state process observed through an arbitrary noisy mapping to a measurement space. The huge uncertainty present in systems/processes, along with the time-limit constraints, necessitate real-time or online joint state and parameter estimation of POBDS. In this manuscript, we present a real-time joint state and parameter estimation framework for POBDS. The proposed framework relies on a complete-sufficient statistic of parameters, where a joint state and parameter estimation is achieved based on the combination of online expectation-maximization method and the optimal MMSE state estimator for POBDS, called Boolean Kalman filter. The proposed method’s performance is assessed through a POBDS model for Boolean gene regulatory networks observed through noisy measurements.

We propose a novel methodology for fault detection and diagnosis in partially-observed Boolean dynamical systems (POBDS). These are stochastic, highly nonlinear, and derivativeless systems, rendering difficult the application of classical fault detection and diagnosis methods. The methodology comprises two main approaches. The first addresses the case when the normal mode of operation is known but not the fault modes. It applies an innovations filter (IF) to detect deviations from the nominal normal mode of operation. The second approach is applicable when the set of possible fault models is finite and known, in which case we employ a multiple model adaptive estimation (MMAE) approach based on a likelihood-ratio (LR) statistic. Unknown system parameters are estimated by an adaptive expectation-maximization (EM) algorithm. Particle filtering techniques are used to reduce the computational complexity in the case of systems with large state-spaces. The efficacy of the proposed methodology is demonstrated by numerical experiments with a large gene regulatory network (GRN) with stuck-at faults observed through a single noisy time series of RNA-seq gene expression measurements.

Nonlinear state-space models are ubiquitous in modeling real-world dynamical systems. Sequential Monte Carlo (SMC) techniques, also known as particle methods, are a well-known class of parameter estimation methods for this general class of state-space models. Existing SMC-based techniques rely on excessive sampling of the parameter space, which makes their computation intractable for large systems or tall data sets. Bayesian optimization techniques have been used for fast inference in state-space models with intractable likelihoods. These techniques aim to find the maximum of the likelihood function by sequential sampling of the parameter space through a single SMC approximator. Various SMC approximators with different fidelities and computational costs are often available for sample-based likelihood approximation. In this paper, we propose a multi-fidelity Bayesian optimization algorithm for the inference of general nonlinear state-space models (MFBO-SSM), which enables simultaneous sequential selection of parameters and approximators. The accuracy and speed of the algorithm are demonstrated by numerical experiments using synthetic gene expression data from a gene regulatory network model and real data from the VIX stock price index.

Control of gene regulatory networks (GRNs) to shift gene expression from undesirable states to desirable ones has received much attention in recent years. Most of the existing methods assume that the cost of intervention at each state and time point, referred to as the immediate cost function, is fully known. In this paper, we employ the Partially-Observed Boolean Dynamical System (POBDS) signal model for a time sequence of noisy expression measurement from a Boolean GRN and develop a Bayesian Inverse Reinforcement Learning (BIRL) approach to address the realistic case in which the only available knowledge regarding the immediate cost function is provided by the sequence of measurements and interventions recorded in an experimental setting by an expert. The Boolean Kalman Smoother (BKS) algorithm is used for optimally mapping the available gene-expression data into a sequence of Boolean states, and then the BIRL method is efficiently combined with the Q-learning algorithm for quantification of the immediate cost function. The performance of the proposed methodology is investigated by applying a state-feedback controller to two GRN models: a melanoma WNT5A Boolean network and a p53-MDM2 negative feedback loop Boolean network, when the cost of the undesirable states, and thus the identity of the undesirable genes, is learned using the proposed methodology.

We propose a Bayesian decision making framework for control of Markov Decision Processes (MDPs) with unknown dynamics and large, possibly continuous, state, action, and parameter spaces in data-poor environments. Most of the existing adaptive controllers for MDPs with unknown dynamics are based on the reinforcement learning framework and rely on large data sets acquired by sustained direct interaction with the system or via a simulator. This is not feasible in many applications, due to ethical, economic, and physical constraints. The proposed framework addresses the data poverty issue by decomposing the problem into an offline planning stage that does not rely on sustained direct interaction with the system or simulator and an online execution stage. In the offline process, parallel Gaussian process temporal difference (GPTD) learning techniques are employed for near-optimal Bayesian approximation of the expected discounted reward over a sample drawn from the prior distribution of unknown parameters. In the online stage, the action with the maximum expected return with respect to the posterior distribution of the parameters is selected. This is achieved by an approximation of the posterior distribution using a Markov Chain Monte Carlo (MCMC) algorithm, followed by constructing multiple Gaussian processes over the parameter space for efficient prediction of the means of the expected return at the MCMC sample. The effectiveness of the proposed framework is demonstrated using a simple dynamical system model with continuous state and action spaces, as well as a more complex model for a metastatic melanoma gene regulatory network observed through noisy synthetic gene expression data.

This paper is concerned with optimal estimation of the state of a Boolean dynamical systems observed through correlated noisy Boolean measurements. The optimal Minimum Mean-Square Error (MMSE) state estimator for general Partially-Observed Boolean Dynamical Systems (POBDS) can be computed via the Boolean Kalman Filter (BKF). However, thus far in the literature only the case of white observation noise has been considered. In this paper, we develop the optimal MMSE filter for a class of POBDS with correlated Boolean measurements. The performance of the proposed method is subsequently investigated using the p53-MDM2 negative feedback loop genetic network model.

BACKGROUND: Gene regulatory networks govern the function of key cellular processes, such as control of the cell cycle, response to stress, DNA repair mechanisms, and more. Boolean networks have been used successfully in modeling gene regulatory networks. In the Boolean network model, the transcriptional state of each gene is represented by 0 (inactive) or 1 (active), and the relationship among genes is represented by logical gates updated at discrete time points. However, the Boolean gene states are never observed directly, but only indirectly and incompletely through noisy measurements based on expression technologies such as cDNA microarrays, RNA-Seq, and cell imaging-based assays. The Partially-Observed Boolean Dynamical System (POBDS) signal model is distinct from other deterministic and stochastic Boolean network models in removing the requirement of a directly observable Boolean state vector and allowing uncertainty in the measurement process, addressing the scenario encountered in practice in transcriptomic analysis. RESULTS: BoolFilter is an R package that implements the POBDS model and associated algorithms for state and parameter estimation. It allows the user to estimate the Boolean states, network topology, and measurement parameters from time series of transcriptomic data using exact and approximated (particle) filters, as well as simulate the transcriptomic data for a given Boolean network model. Some of its infrastructure, such as the network interface, is the same as in the previously published R package for Boolean Networks BoolNet, which enhances compatibility and user accessibility to the new package. CONCLUSIONS: We introduce the R package BoolFilter for Partially-Observed Boolean Dynamical Systems (POBDS). The BoolFilter package provides a useful toolbox for the bioinformatics community, with state-of-the-art algorithms for simulation of time series transcriptomic data as well as the inverse process of system identification from data obtained with various expression technologies such as cDNA microarrays, RNA-Seq, and cell imaging-based assays.