Information Fusion seeks to combine observations from sensor data and/or human opinions to make a decision with reduced uncertainty. One common method for physics-based and human-derived information fusion (PHIF) that is well established is that of the analytical hierarchy processing (AHP). When decisions are not crisp, the Fuzzy AHP is a method towards establishing the boundary between decisions and the consistency among the decisions. In this paper, the Fuzzy AHP is utilized to assess information fusion designs developed from the Multisource AI Scorecard Table (MAST). MAST has devised a set of scores that can be ordinal (crisp), discrete (probabilities), and semantic (labels); and this paper explores a measure of uncertainty as from static fuzzy logic. The main finding is that assessment from different users with different ratings (i.e., Level 5 Fusion) can be enhanced by fuzzy AHP to check for consistency. If the combined PHIF results are not consistent, users would then need to subjectively identify weaknesses and limitations to continue the information fusion systems engineering design process.

We propose a neuro-symbolic approach for learning causal complex event models from multi-source data, integrating causal discovery and temporal logic. Given resource constraints, we employ signal-level fusion by averaging the data from different antennas of the same WiFi receiver, followed by downsampling to reduce computational overhead. We consider a dataset of WiFi Channel State Information capturing human activities alongside video data from which we extract atomic symbolic activities such as “moving the upper arm.” The extracted symbolic information is processed through LPCMCI (Latent PCMCI). This causal discovery method extends PCMCI (Peter and Clark Momentary Conditional Independence) to handle latent dependencies across multiple time steps while mitigating false discoveries due to auto-correlations. The resulting causal structure is then translated into a temporal logic formula, which serves as a symbolic constraint in a neuro-symbolic learning pipeline. To efficiently process and learn from these structured constraints under resource limitations, we leverage Spiking Neural Networks, which offer energy-efficient computation while preserving temporal dynamics.

This paper focuses on the multi-target tracking using the Stone Soup framework. In particular, we aim at evaluation of two multi-target tracking scenarios based on the simulated class-B dataset and ADS-B class-A dataset provided by OpenSky Network. The scenarios are evaluated w.r.t. selection of a local state estimator using a range of the Stone Soup metrics. Source code with scenario definitions and Stone Soup set-up are provided along with the paper.

Evaluating uncertainty in large language model (LLM) reasoning is challenging due to their vast parameter space, abstract knowledge representation, and limited transparency regarding training data. While normative formalisms, such as deductive logic, clearly define sound reasoning in the absence of uncertainty, reasoning under uncertainty admits multiple approaches, including probabilistic reasoning (e.g. Bayesian), belief function reasoning (e.g. Dempster-Shafer), or fuzzy logic, to name a few. This paper examines how LLMs handle uncertainty by analyzing outcomes based on an atomic fusion and reasoning problem. We establish a point of reference using the simplest of fusion topologies to facilitate transparency and understanding of how LLMs align with established theories. The reasoning approaches of different LLMs with varying complexities are compared to established normative frameworks, providing insights into which formalism best aligns with LLM reasoning and assessing its soundness and consistency. A deviation function for assessment is developed, and the results indicate that the tested LLMs' reasoning under uncertainty does not consistently align with established theories, even for the simplest information fusion topologies. These preliminary results form the basis for further investigations and LLM refinements.

Wi-Fi devices, akin to passive radars, can discern human activities within indoor settings due to the human body’s interaction with electromagnetic signals. Current Wi-Fi sensing applications predominantly employ data-driven learning techniques to associate the fluctuations in the physical properties of the communication channel with the human activity causing them. However, these techniques often lack the desired flexibility and transparency. This paper introduces DeepProbHAR, a neuro-symbolic architecture for Wi-Fi sensing, providing initial evidence that Wi-Fi signals can differentiate between simple movements, such as leg or arm movements, which are integral to human activities like running or walking. The neuro-symbolic approach affords gathering such evidence without needing additional specialised data collection or labelling. The training of DeepProbHAR is facilitated by declarative domain knowledge obtained from a camera feed and by fusing signals from various antennas of the Wi-Fi receivers. DeepProbHAR achieves results comparable to the state-of-the-art in human activity recognition. Moreover, as a by-product of the learning process, DeepProbHAR generates specialised classifiers for simple movements that match the accuracy of models trained on finely labelled datasets, which would be particularly costly.

This paper deals with state estimation of nonlinear stochastic dynamic models. In particular, the stochastic integration rule, which provides asymptotically unbiased estimates of the moments of nonlinearly transformed Gaussian random variables, is reviewed together with the recently introduced stochastic integration filter (SIF). Using SIF, the respective multi-step prediction and smoothing algorithms are developed in full and efficient square-root form. The stochastic-integration-rule-based algorithms are implemented in Python (within the Stone Soup framework) and in MATLAB® and are numerically evaluated and compared with the well-known unscented and extended Kalman filters using the Stone Soup defined tracking scenario.

The Unscented Kalman Filter (UKF) stands out as a versatile and dynamic algorithm, celebrated for its prowess in estimating the states of nonlinear dynamical systems within uncertain environments. However, the accuracy of UKF state estimations hinges significantly on the thoughtful selection of pivotal hyperparameters, $\alpha, \beta$, and $\kappa$, which are integral in shaping the distribution of sigma points around the current state estimate. Prevailing methods for tuning these parameters encompass heuristic approaches such as arbitrarily fix $\alpha$ at 0.001, $\kappa$ at 0, and $\beta$ at 2 for Gaussian noise, though the efficacy of such rules heavily hinges on the intricacies of the specific problem. Alternatively, the grid search technique seeks to optimize these hyperparameters, but it can become computationally burdensome, particularly when the search space is extensive and intricate. To navigate these hurdles, this paper introduces the RNN-UKF algorithm-a pioneering strategy that leverages recurrent neural networks (RNNs) to autonomously fine-tune UKF hyperparameters. The inherent adaptability of RNNs is harnessed to dynamically adjust the $\alpha, \beta$, and $\kappa$ parameters of the unscented transformation during each state estimation step, all aimed at minimizing the root mean squared error (RMSE). Demonstrated through numerical simulations, we provide compelling evidence that the RNN-UKF approach outperforms both heuristic rule of thumb and grid search techniques in terms of RMSE performance. Moreover, the RNN-UKF methodology showcases its superiority over the conventional extended Kalman filter (EKF) approach, particularly in scenarios characterized by substantial system noise.

Multimodal fusion is important for building intelligent systems that exploit patterns across diverse data sources for improved decision-making. However, the reliability and robustness of these systems in safety-critical domains are often compromised by the inherent noise and incompleteness of data. Probabilistic Circuits (PCs) have recently emerged as a promising approach for late (or decision) fusion. Their strength lies in being both expressive and capable of inferring source credibility due to their ability to tractably perform exact probabilistic inference. However, their ability to handle missing data and their reliability in practical scenarios remains underexplored. This work investigates the robustness of PCs as fusion functions in scenarios with missing and noisy data; particularly by examining their impact on the calibration and reliability of the resulting classifiers. Our findings show that PCs not only enable the modeling of complex correlations across modalities but also lead to calibrated and reliable classifiers, highlighting their potential as a robust fusion mechanism in multimodal systems.

Consistent bi-directional inferences are the key for many machine learning applications. Without consistency, inverse learning-based inferences can cause fuzzy images, erroneous control signals, and cascading failure in SCADA systems. Since standard deep neural networks (DNNs) are not inherently invertible to offer consistency, some past methods reconstruct DNN architecture analytically for one-to-one correspondence but compromise key features such as universal approximation. Other work maintains the capability of universal approximation in DNNs via iterative numerical approximation. However, these methods limit their applications significantly due to Lipschitz conditions and issues of numerical convergence. The dilemma of the analytical and numerical methods is the incompatibility between nonlinear layer compositions and bijective function construction for inverse modeling. Based on the observation, we propose decomposed-invertible-pathway DNNs (DipDNN). It relaxes the redundant reconstruction of nested DNN in the former methods and eases the Lipschitz constraint. As a result, we strictly guarantee the consistency of global inverse modeling without harming DNN's capability for universal approximation. As numerical stability and generalizability are keys for controlling critical infrastructures, we integrate contractive property with a parallel structure for inductive biases, leading to stable performance. Numerical results show that DipDNN performs significantly better than past methods, thanks to its enforcement of inverse consistency, numerical stability, and physical regularization.

Test and Evaluation for verification and validation (V&V) of sensor data fusion techniques utilize methods of uncertainty analysis. The Uncertainty Representation and Reasoning Evaluation Framework (URREF) ontology identifies many attributes of metrics (i.e., semantic meaning, object metrics, and subjective quality). With the growing interest in artificial intelligence (AI) due to large data corpus access, fast compute power, and machine/deep learning (ML/DL) techniques; V&V of these methods are needed. In this paper, the enhancement of the URREF to utilize a risk assessment for decision is demonstrated towards analysis/alignment of ML/DL methods that utilize multi-modal data fusion. Evidential reasoning is considered in the use case to provide data handing reliability source and processing credibility to measure decision risk in a maritime domain awareness scenario.

This paper presents an eXplainable Deep Reinforcement Learning (XDRL) based strategy for solving the proposed problem of fleet-level aircraft maintenance scheduling (AMS) optimization. The XDRL-AMS considers various factors such as the aircraft’s initial status, mission requirements, maintenance resource capacity, and operational constraints to create a maintenance schedule for a specified period. The schedule aims to balance both mission readiness and cost reduction. We developed an RL environment, called AMS-Gym, using the OpenAI Gym toolkit specifically designed for this problem. AMS-Gym is highly flexible, allowing for easy extension to more complex scenarios and incorporating additional explanatory capabilities. The explainable RL capability was achieved by utilizing a decomposed reward Deep Q-Network (drDQN) algorithm. In the context of the AMS scenario, the drDQN consists of two parts: (i) a DQN that aims to maximize the mission accomplishment objective, and (ii) a DQN that aims to minimize the maintenance cost objective. As a result, the proposed drDQN strategy can generate real-time aircraft maintenance decisions, explain why those decisions were selected, and present the tradeoffs between the chosen action and non-selected alternatives. Experiment results show that the proposed drDQN performs well, providing an approximate solution to the vanilla DQN with a simpler structure while offering the ability to explain its decisions. In addition, a web-based prototype with an intuitive textual and visual user interface was developed to demonstrate the feasibility of the drDQN approach.

ChatGPT, OpenAI’s chatbot, has gained consider-able attention since its launch in November 2022, owing to its ability to formulate articulated responses to text queries and comments relating to seemingly any conceivable subject. As impressive as the majority of interactions with ChatGPT are, this large language model has a number of acknowledged shortcomings, which in several cases, may be directly related to how ChatGPT handles uncertainty. The objective of this paper is to pave the way to formal analysis of ChatGPT uncertainty handling. To this end, the ability of the Uncertainty Representation and Reasoning Framework (URREF) ontology is assessed, to support such analysis. Elements of structured experiments for reproducible results are identified. The dataset built varies Information Criteria of Correctness, Non-specificity, Self-confidence, Relevance and Inconsistency, and the Source Criteria of Reliability, Competency and Type. ChatGPT’s answers are analyzed along Information Criteria of Correctness, Non-specificity and Self-confidence. Both generic and singular information are sequentially provided. The outcome of this preliminary study is twofold: Firstly, we validate that the experimental setup is efficient in capturing aspects of ChatGPT uncertainty handling. Secondly, we identify possible modifications to the URREF ontology that will be discussed and eventually implemented in URREF ontology Version 4.0 under development.

AI-based decision support solutions require life cycles that adequately address critical steps, such as (i) finding suitable machine learning (ML) methods for the problem at hand, (ii) preparing and executing adequate data acquisition processes and (iii) tractable evaluation of the overall solution. Understanding the data generating processes is key in achieving this. Training and test data can be seen as a result of a causal data generation process, a sampling process in which the data is collected from different sources that are influenced by multiple interdependent phenomena. This is represented by a Qualitative Model of Data Generation Processes (QM-DGP), a causal graphical model. QM-DGP facilitates analysis of the complexity of the underlying data generating processes that can inform the development of trustable ML-based solutions in multiple ways. Firstly, this analysis is the basis for the determination of the required complexity of the ML models. Secondly, it facilitates the determination of the quantities of training data supporting good learning results. Thirdly, it can provide guidance for a systematic simplification of the models, supporting tractable solutions without significantly reduced performance. The construction of QM-DGP and the analysis benefit from sound theoretical concepts, such as d-separation and I-Maps. Experimental results with simulated data indicate that the approach can be effective in predicting the required quantities of training data and the determination of the modelling complexity using different types of models.

The recent advancements in the Internet of Video Things (IoVT) and Edge-Fog-Cloud Computing paradigm make smart public safety surveillance (SPSS) a realistic solution for an effective public safety service in smart cities. Typically, a fully functional SPSS system requires multiple sensory inputs for situational awareness (SAW). As an essential component in the context of highly complex, dynamic, and heterogeneous smart city operations, SPSS is expected to be environment-resilient. Personal safety is among the top concerns of the residents in smart cities, and correspondingly pedestrian detectors are critical. Contemporary pedestrian detectors use optical cameras, whose accuracy is diminished in low-light environments, and they are rendered ineffective when obstacles block the direct line of sight to the camera. Complementary imaging sensors such as infrared have shown promise. This paper presents a full-spectrum, environment-resilient surveillance platform as an ultimate solution, which consists of multiple imaging units to cover a wide sensing spectrum. The initial hybrid pedestrian detection (HYPE) scheme is based on the fusion of data obtained from an IoVT network equipped with optical and thermal cameras. We demonstrate that training the YOLOv5 object detection model on a dataset of infrared images improves its accuracy in the detection of humans present in thermal surveillance images. A 41% decrease in objectness loss is achieved after transfer learning is performed.

In high-rate structural health monitoring, it is crucial to quickly and accurately assess the current state of a component under dynamic loads. State information is needed to make informed decisions about timely interventions to prevent damage and extend the structure’s life. In previous studies, a dynamic reproduction of projectiles in ballistic environments (DROPBEAR) testbed was used to evaluate the accuracy of state estimation techniques through dynamic analysis. This paper extends previous research by incorporating the local eigenvalue modification procedure (LEMP) and data fusion techniques to create a more robust state estimate using optimal sampling methodologies. The process of estimating the state involves taking a measured frequency response of the structure, proposing frequency response profiles, and accepting the most similar profile as the new mean for the position estimate distribution. Utilizing LEMP allows for a faster approximation of the proposed model with linear time complexity, making it suitable for 2D or sequential damage cases. The current study focuses on two proposed sampling methodology refinements: distilling the selection of candidate test models from the position distribution and applying a Kalman filter after the distribution update to find the mean. Both refinements were effective in improving the position estimate and the structural state accuracy, as shown by the time response assurance criterion and the signal-to-noise ratio with up to 17% improvement. These two metrics demonstrate the benefits of incorporating data fusion techniques into the high-rate state identification process.

In modern complex physical systems, advanced sensing technologies extend the sensor coverage but also increase the difficulties of improving system monitoring capabilities based on real-time data availability. Traditional model-based methods of sensor management are limited to specific systems/settings, which can be challenged when system knowledge is intractable. Fortunately, the large amount of data collected in real-time allows machine learning methods to be a complement. Especially, reinforcement learning-based control is recognized for its capability to dynamically interact with systems. However, the direct implementation of learning methods easily overfits and results in inaccurate physics modeling for sensor management. Although physical regularization is a popular direction to bridge the gap, learning-based sensor control still suffers from convergence failure under highly complex and uncertain scenarios. This paper develops physics-embedded and self-supervised reinforcement learning for sensor management using an intrinsic reward. Specifically, the intrinsic-motivated sensor management (IMSM) constructs the local surprise information from the physical latent features, which captures hidden states in observations, and thus intrinsically motivates the agent to speed-up exploration. We show that the designs can not only relieve the lack of consistency with underlying physics/physical dynamics, but also adapt the global objective of maximizing monitoring capabilities to local environment changes. We demonstrate its effectiveness by experiments on physical system sensor control. The proposed model is implemented for the sensor management of unmanned vehicles and sensor rescheduling in complex/settled power systems, with or without observability constraints. Numerical results show that our model provides consistently higher threat detection accuracy and better observability recovery, as compared to existing methods.

Context enhancement is critical for the environmental perception in night vision applications, especially for the dark night situation without sufficient illumination. In this article, we propose a thermal image translation method, which can translate thermal/infrared (IR) images into color visible (VI) images, called IR2VI. The IR2VI consists of two cascaded steps: translation from nighttime thermal IR images to gray-scale visible images (GVI), which is called IR-GVI; and the translation from GVI to color visible images (CVI), which is known as GVI-CVI in this article. For the first step, we develop the Texture-Net, a novel unsupervised image translation neural network based on generative adversarial networks. Texture-Net can learn the intrinsic characteristics from the GVI and integrate them into the IR image. In comparison with the state-of-the-art unsupervised image translation methods, the proposed Texture-Net is able to address some common challenges, e.g., incorrect mapping and lack of fine details, with a structure connection module and a region-of-interest focal loss. For the second step, we investigated the state-of-the-art gray-scale image colorization methods and integrate the deep convolutional neural network into the IR2VI framework. The results of the comprehensive evaluation experiments demonstrate the effectiveness of the proposed IR2VI image translation method. This solution will contribute to the environmental perception and understanding in varied night vision applications.

The International Society of Information Fusion (ISIF) Stone Soup project seeks to bring together advances in target tracking through an open-source repository of software libraries. Additionally, the ISIF uncertainty reasoning working group provides an open-source ontology. This paper seeks to demonstrate the correspondence between the open source tracking repository and the Uncertainty Representation and Reasoning Evaluation Framework (URREF) ontology. For example, many target tracking challenge problems propose a scenario for data fusion techniques to solve, from which various performance metrics are considered for evaluation. The Stone Soup framework has the MetricGenerator class and the URREF has the accuracy class. The example presented in the paper utilizes the cubature Kalman filter to determine the impact of corrupted measurements on the track accuracy as an instance of the Stone Soup and URREF metrics.

There have been many advances in image fusion to support multi-modal, multi-perspective, and multi-focal day-night robust surveillance. Contextual analysis of multi-sensor exploitation supports many information fusion systems for target tracking, situation awareness, and scene understanding. This paper highlights an example for electro-optical and infrared image fusion analysis and colorization in support of simultaneous tracking and identification for physics-based and human-derived information fusion (PHIF). The results provide an ongoing discussion and analysis of the importance of context as both a source to guide information fusion systems (e.g., sensor mode selection), but also the need to be able to provide users with robust systems over context changes (e.g., variations in illumination).

This paper considers a novel formulation of inverse reinforcement learning with behavioral economics constraints to address inverse sequential hypothesis testing (SHT) in Bayesian agents. The aim is to estimate the detection costs by observing the actions of the sequential hypothesis detector. Our methodology involves Bayesian revealed preferences from microeconomics and rational inattention from behavioral economics. First, we show that Bayesian agents optimally performing SHT are rationally inattentive utility maximizers. Using established results in Bayesian revealed preferences, we outline a feasibility test for a data analyst observing the Bayesian agents to estimate their detection costs. Numerical examples illustrate the performance of the inverse sequential hypothesis testing algorithm.

With ubiquitous data acquired from sensors, there is an ever increasing ability to abstract content from the combination of physics-based and human-derived information fusion (PHIF). The advancement of PHIF tools include graphical information fusion methods, target tracking techniques, and natural language understanding. Current discussions revolve around dynamic data-driven applications systems (DDDAS) which seeks to leverage high-dimensional modeling with real-time physical systems. An example of a model includes a learned dictionary that can be leveraged as information queries. In this paper, we discuss the DDDAS paradigm of sensor measurements, information processing, environmental modeling, and software implementation to deliver content for PHIF systems. Experimental results demonstrate the DDDAS-based Live Video Computing DataBase Modeling Systems (LVC-DBMS) approach affording data discovery and query-based flexibility for awareness to provide narratives of unknown situations.

Information fusion includes the integration of data for situational understanding. As a situation unfolds, maintaining awareness depends on diverse collections of data. In complex and dynamic scenarios, human operators face the difficult task of choosing which data to collect next. Hence, there is a need for multilayered fusion processes that exploit multiple models and levels of abstraction for understanding and sense-making Data collection has its roots in sensor management; however, there is an analogous need for data management - such as the incorporation of public domain data. Mature sensor management includes methods to utilize platform, sensor, and scene modeling so as to guide the user for future data collection. Additionally, these physics-based models could be a method to guide human-derived information models. Using the Data Fusion Information Group Model (DFIG), we develop an equivalent method for diffusion control. This paper focuses on recent techniques in statistical relational learning (SRL), Markov logic networks (MLN), and ontologies to support the control diffusion of data sensing to answer user queries.

This paper presents a comparative analysis of performances of two types of multi-target tracking algorithms: 1) the Joint Probabilistic Data Association Filter (JPDAF), and 2) classical Kalman Filter based algorithms for multi-target tracking improved with Quality Assessment of Data Association (QADA) method using optimal data association. The evaluation is based on Monte Carlo simulations for difficult maneuvering multiple-target tracking (MTT) problems in clutter.

The paper introduces a novel approach to an estimator design, the cooperative filter design, for state estimation of nonlinear systems. The approach is based on the idea of combining estimates of several different approximate (and thus sub-optimal) nonlinear filters, which are configured to perform the same task. Within the concept, two strategies are proposed, namely the cooperative estimation and cooperative monitoring. The strategies have the potential of an improvement of the estimation performance in terms of accuracy and consistency, which was confirmed by a numerical illustration.

The International Society of Information Fusion (ISIF) Evaluation Techniques for Uncertainty Representation Working Group (ETURWG) investigates the quantification and evaluation of all types of uncertainty regarding the inputs, reasoning and outputs of the information fusion process. The ETURWG is developing an Uncertainty Representation and Reasoning Framework (URREF) ontology for this purpose. This paper outlines a start towards the process of defining metrics for the URREF data criteria, which will align the URREF ontology with practical application. A criterion can be evaluated according to several metrics, and a metric can be applied to several criteria. As such, the ontology would have to reflect the nature of a many-to-many mapping between criteria and metrics. The main findings and suggestions of the paper advancing the use of URREF are: 1) The Weight of Information (WoI) is dependent on data criteria, which in turn depend on source criteria. 2) Criteria and metrics that apply to evidence (typically an input of the fusion system), could equally apply to the fusion system outputs or internal information, which in turn could form the inputs of another system. As such the word “Evidence” in the terms “Piece of Evidence” and “Weight of Evidence” should be replaced by the word “Information”. 3) Accuracy and precision and associated metrics are ubiquitous in the URREF ontology and can evaluate many parts of the fusion system. 4) The weight of information also assumes an important position in the ontology, as it depends on several source and data criteria.

The question addressed in this paper is “what” is to be evaluated by the Uncertainty Representation and Reasoning Evaluation Framework (URREF) ontology. We thus identify the elements composing uncertainty representation and reasoning approaches, which constitute various subjects being assessed. We distinguish between primary evaluation subjects (Uncertainty Representation and Reasoning components of the fusion algorithm), and secondary evaluation subjects (source of information, piece of information, fusion method and mathematical model). This paper proposes a list of source quality criteria to be added to the ontology and establishes formal links between the secondary and primary evaluation subjects. The key contribution of the paper is the update of the definitions of sub-criteria of the Expressiveness criterion together with suggestions for complementary concepts to be included in the ontology (type of scale, type of uncertainty expression). Conclusions are drawn to extend the work in using the expressiveness criterion for information fusion analysis.

A typical method to obtain valuable information is to extract the sentiment or opinion from a message. Machine learning technologies are widely used in sentiment classification because of their ability to “learn” from the training dataset to predict or support decision making with relatively high accuracy. However, when the dataset is large, some algorithms might not scale up well. In this paper, we aim to evaluate the scalability of Naïve Bayes classifier (NBC) in large datasets. Instead of using a standard library (e.g., Mahout), we implemented NBC to achieve fine-grain control of the analysis procedure. A Big Data analyzing system is also design for this study. The result is encouraging in that the accuracy of NBC is improved and approaches 82% when the dataset size increases. We have demonstrated that NBC is able to scale up to analyze the sentiment of millions movie reviews with increasing throughput.

Distributed image steganography (DIS) [8] is a new method of concealing secret information in several host images, leaving smaller traces than conventional steganographic techniques, and requiring a collection of affected images for secret information retrieval. Fusion system designs of the future will require enhanced security measures for distributed data communication. DIS, compared to other conventional steganographic techniques, can improve security and information hiding capacity because DIS leaves reduced signatures of hidden information in host images. The open literature does not offer effective detection methods and countermeasures for DIS, indicating that it can be potentially usable to criminals for unchallenged covert communication over the Internet and fusion architectures. In this paper, we explore a new information extraction method for both detecting and reversing DIS method by considering images as pseudo-random processes. The key idea is to estimate secret image as a random process, which is corrupted by a noise source (i.e. host image). The secret images may be nonlinear, non-Gaussian and nonstationary in nature, and can be disclosed by using some estimation techniques such as Kalman filtering. Our proposed method demonstrates great promise to reveal a secret image. Consequently, it is useful for intelligence gathering and information extraction in steganographic - images produced by DIS.

This paper presents a new probabilistic approach to determine survivability of reconfigurable systems as a system-level performance metric in an operational environment. In contrast to known methods of estimating survivability in terms of susceptibility and vulnerability, the proposed method (1) includes directional threats on various subsystems into the analysis and (2) provides a framework for operational information fusion processes to better sustain unpredictable or hostile environmental disturbances. In this paper, we distinguish the survivability and reliability metrics, where we demonstrate the importance of survivability metric in a dynamic information fusion process for an operational environment. We present our main result using a piping system of fluid flow; however, the concept easily extends to other flow systems, such as power networks, computer communication networks, and military reconfigurable information systems, etc. Survivability of these large scale reconfigurable networks depend on their capability of assessing directional threats, situation awareness, and their ability to dynamically adapt to new configurations. The proposed survivability method embedded in an information fusion environment can be used for real time dynamic reconfiguration of large scale systems, optimization and routing of data and information, and detect and mitigate hardware and software threats.

Sensor Resource Management (or process refinement) is a element of any information fusion system. Common Level 4 sensor management (SM) inter-relations to Level 1 target tracking and identification have been developed in the literature. During Fusion06, a panel discussion was held to explore the challenges and issues pertaining to the interaction between SM and situation and threat assessment. This paper summarizes the key tenants of the discussion to filter vast experiences of the invited panel experts. The common themes were: (1) Addressing the user in system management / control, (2) Determining a standard set of metrics for optimization, (3) Optimizing / evaluating fusion systems to deliver timely information needs, (4) Dynamic updating for planning mission time-horizons, (5) Joint optimization of objective functions at all levels (6) L2/3 situation entity definitions for knowledge discovery, modeling, and information projection (7) Addressing constraints for resource planning and scheduling

Summary form only given. Despite significant recent progress in automatic target exploitation (ATE) and recognition (ATR), current ATE systems do not meet the requirements of modern battlefield environments. Next generation ATE systems must actively manage sensor resources, aggregate sensed information across multiple platforms and diverse signaling modalities, and adapt to increasingly agile adversaries and operating conditions. Thus, the fundamental research challenge is to develop an integrated systems theory that jointly treats information fusion, control, and adaptation using multiple, dynamic multimodal sensor platforms in resource constrained environments.

Recent developments in image fusion have produced a variety of approaches like image overlay, image sharpening, and image cueing through pixel, feature, or region/shape combinations. The applicability of these approaches and techniques differ on the image content, contextual information, and generalized metrics of image fusion gain. An image fusion gain can be assessed relative to information gain or entropy reduction. In this paper, we are interested in exploring the techniques and data available with the image fusion toolbox (from www.imagefusion.org) to assess the use of relative entropy analysis for metric evaluation, image fusion gain calculations, and assessment of fused images as templates for automatic target recognition. Examples are demonstrated for medical (PET/MRI), (CT/MRI) and environment (visible/infrared) examples. A mutual information measure of the image fusion quality can be an effective tool to characterize image terrain, content, and contextual information to cue higher-level fusion algorithms.

This paper compares different defense strategies against various attacks utilizing a dynamic game theoretic data fusion framework for cyber network defense. In our game theoretic framework, Alerts generated by Intrusion Detection Sensors (IDSs) or Intrusion Prevention Sensors (IPSs) are fed into the data refinement (Level 0) and object assessment (L1) data fusion components. High-level situation/threat assessment (L2/L3) data fusion based on Markov game model and Hierarchical Entity Aggregation (HEA) are proposed to refine the primitive prediction generated by adaptive feature/pattern recognition and capture new unknown features. A Markov (Stochastic) game method is used to estimate the belief of each possible cyber attack pattern. Game theory captures the nature of cyber conflicts: determination of the attacking-force strategies is tightly coupled to determination of the defense-force strategies and vice versa. A software tool is developed to demonstrate and compare the performance of different defense strategies used in game theoretic high level information fusion for cyber network defense situations and a simulation example shows the enhanced understating of cyber-network defense.

This paper is concerned with tracking of ground targets on roads and investigates possible ways to improve target state estimation via fusing a target’s track with information about a road along which the target is believed to be traveling. A target track is estimated by a surveillance radar whereas a digital map provides the road network of a region under surveillance. When the information about roads is as accurate as (or even better than) radar measurements, it is desired naturally to incorporate such information (fusion) into target state estimation. In this paper, roads are modeled with analytic functions and its fusion with a target track is cast as linear or nonlinear state constraints in an optimization procedure. The constrained optimization is then solved with the Lagrangian multiplier, leading to a closed-form solution for linear constraints and an iterative solution for nonlinear constraints. Geometric interpretations of the solutions are provided for simple cases. Computer simulation results are presented to illustrate the algorithms.

Tracking maneuvering targets presents a great challenge to airborne surveillance radar signal processing and sensor systems management systems. Smears caused by an uncompensated maneuver (either translational or rotational) affect target identification (ID) with distorted target images. An unexpected maneuver introduces large position estimation errors to a tracker and in the worst case loss of track. On the other hand, a sensor manager relies upon an expected performance of a tracker to schedule its resources so as to maintain target ID/tracker performance. To aid a sensor management cost function, we present a simple target maneuver indicator (TMI) specifically for the operational condition of target maneuverability. It relates the slope of a target’s range-Doppler image to the underlying turn rate, if the target undergoes a maneuver. As an intermediate product of the range profile formation process, this approach provides an easy and quick indication of target maneuverability and, under favorable conditions, an estimate of such a maneuver (e.g., the turn rate), which can be incorporated into the tracking algorithm of the tracker..

Subsequent revisions to the Joint-Directors of Lab (JDL) model emphasize the differentiation between fusion (estimation) and sensor management (control). Two diverging groups include one pressing for fusion automation (JDL revisions) and one advocating the role of the user (user-fusion model). The center of debate is real-world delivery of fusion systems which requires presenting fusion results for knowledge representation (fusion estimation) and knowledge reasoning (control management). The purpose of the paper is to highlight the need of Users, with individual differences, facilitated by knowledge representations to reason about user situational awareness (SA). This paper includes: (1) Addressing the user in system management/control. (2) Assessing information quality (metrics) to support SA. (3) Evaluating Fusion systems to deliver user info needs, (4) Planning knowledge delivery for dynamic updating. (5) Designing SA interfaces to support user reasoning

Revisions to the JDL model by the current DFIG team (data fusion information group) include definitions for model usefulness that stressed various control functions of sensor, user, and mission (SUM) management. The purpose of the paper is to highlight issues and challenges to real world separation of control actions. This position paper highlights: 1) addressing the user in system management/control, 2) determining a standard set of metrics for optimization, 3) evaluating fusion systems to deliver timely info needs, 4) dynamic updating for planning mission time-horizons, 5) designing interfaces to support impact assessments

In [Simon and Chia, 2002], an analytic method was developed to incorporate linear state equality constraints into the Kalman filter. When the state constraint is nonlinear, linearization was employed to obtain an approximately linear constraint around the current state estimate. This linearized constrained Kalman filter is subject to approximation errors and may suffer from a lack of convergence. In this paper, we present a method that allows exact use of second-order nonlinear state constraints. It is based on a computational algorithm that iteratively finds the Lagrangian multiplier for the nonlinear constraints. The method therefore provides better approximation when higher order nonlinearities are encountered. Computer simulation results are presented to illustrate the algorithm