The first High-Level Information Fusion (HLIF) competition took place in 2024, proposing a challenge for supporting aircraft pilots handling “Notices to Air Missions”. Technical difficulties on both the input datasets and the competition architecture raised the bar to submissions. Only two solutions taking the form of self-contained software were submitted, and were compared with two references produced by the organizers. A detailed analysis shows that some misunderstanding of the underlying use-case objectives had a big impact on the choices made by the participants. Those misunderstandings were due to a lack of precisions in the problem descriptions, along with differences in contestants' backgrounds. Consequently, the competition cannot rank the available solutions. However, our analysis highlights some interesting epistemological aspects of such a setting, related to the design and use of ontologies, knowledge graphs and queries. This allows us to propose a set of advices for improving future occurrences of an HLIF competition, on the same use-case or on another one.

Evaluating uncertainty in large language model (LLM) reasoning is challenging due to their vast parameter space, abstract knowledge representation, and limited transparency regarding training data. While normative formalisms, such as deductive logic, clearly define sound reasoning in the absence of uncertainty, reasoning under uncertainty admits multiple approaches, including probabilistic reasoning (e.g. Bayesian), belief function reasoning (e.g. Dempster-Shafer), or fuzzy logic, to name a few. This paper examines how LLMs handle uncertainty by analyzing outcomes based on an atomic fusion and reasoning problem. We establish a point of reference using the simplest of fusion topologies to facilitate transparency and understanding of how LLMs align with established theories. The reasoning approaches of different LLMs with varying complexities are compared to established normative frameworks, providing insights into which formalism best aligns with LLM reasoning and assessing its soundness and consistency. A deviation function for assessment is developed, and the results indicate that the tested LLMs' reasoning under uncertainty does not consistently align with established theories, even for the simplest information fusion topologies. These preliminary results form the basis for further investigations and LLM refinements.

Large-scale high-level information fusion and data integration is a pressing need in several scientific domains. Recently, the biomedical community established BioCypher, a tool to help create large Semantic Knowledge Graphs (SKGs) in a simple and reproducible way. In this article, we introduce OntoWeaver, a companion tool to BioCypher that allows to easily extract tabular data into SKGs by using a simple declarative mapping. OntoWeaver allows implementing reproducible mappings from tabular data to SKGs with a simple declarative configuration. The use of OntoWeaver and BioCypher is demonstrated in two different use cases: cancer database integration and invasive species monitoring. We believe that OntoWeaver and BioCypher, both free and open-source software, can help several scientific communities working on SKGs and high-level information fusion problems.

This paper presents the HLIF2024 competition. HLIF2024 is the first competition for High-Level Information Fusion solutions, co-organized with the FUSION2024 conference. The paper presents the challenge use case associated with the competition: information extraction and fusion from Notices to Air Missions data. The challenge is to automate the processing of this data, in order to support pilots with their decision making tasks during flights. To simulate this, the challenge involves a fictitious flight during which information is requested by the pilot. To provide pertinent information to the pilots, the participants are asked to populate a given domain ontology that can then be queried to answer the challenge questions. The participants solutions are evaluated with regard to the correctness, completeness and preciseness of their answers to the questions. The motivation behind the use case is presented, together with the practice dataset that was built and provided to the competition participants. The evaluation methodology and metrics used to rank the participants solutions are detailed. The results of the competition will be presented at the FUSION2024 conference.

Test and Evaluation for verification and validation (V&V) of sensor data fusion techniques utilize methods of uncertainty analysis. The Uncertainty Representation and Reasoning Evaluation Framework (URREF) ontology identifies many attributes of metrics (i.e., semantic meaning, object metrics, and subjective quality). With the growing interest in artificial intelligence (AI) due to large data corpus access, fast compute power, and machine/deep learning (ML/DL) techniques; V&V of these methods are needed. In this paper, the enhancement of the URREF to utilize a risk assessment for decision is demonstrated towards analysis/alignment of ML/DL methods that utilize multi-modal data fusion. Evidential reasoning is considered in the use case to provide data handing reliability source and processing credibility to measure decision risk in a maritime domain awareness scenario.

ChatGPT, OpenAI’s chatbot, has gained consider-able attention since its launch in November 2022, owing to its ability to formulate articulated responses to text queries and comments relating to seemingly any conceivable subject. As impressive as the majority of interactions with ChatGPT are, this large language model has a number of acknowledged shortcomings, which in several cases, may be directly related to how ChatGPT handles uncertainty. The objective of this paper is to pave the way to formal analysis of ChatGPT uncertainty handling. To this end, the ability of the Uncertainty Representation and Reasoning Framework (URREF) ontology is assessed, to support such analysis. Elements of structured experiments for reproducible results are identified. The dataset built varies Information Criteria of Correctness, Non-specificity, Self-confidence, Relevance and Inconsistency, and the Source Criteria of Reliability, Competency and Type. ChatGPT’s answers are analyzed along Information Criteria of Correctness, Non-specificity and Self-confidence. Both generic and singular information are sequentially provided. The outcome of this preliminary study is twofold: Firstly, we validate that the experimental setup is efficient in capturing aspects of ChatGPT uncertainty handling. Secondly, we identify possible modifications to the URREF ontology that will be discussed and eventually implemented in URREF ontology Version 4.0 under development.

Social data analysis has gained prominence in a wide range of domains as it provides users with the opportunity to communicate and share posts and topics. Automated analysis and reasoning about such data potentially derive meaningful insights, with tremendous potential for applications. However, the sheer volume, noise, and high dynamics of social data impose challenges that hinder the efficacy of algorithms. Automated approaches and classification models require then significant resources to be developed and prove to be often relevant to only a limited number of tasks. Imperfections of inputs, precision of techniques and accuracy of results need to be accounted and assessed as the process runs. This paper discuses two use cases allowing the investigation of implicit and explicit uncertainty arising when processing data gleaned on social media. The objective of this paper is twofold. The first objective is to set up the ETUR use case on social media analysis by adopting two tasks on opinion mining for cyberspace surveillance and information extraction for crisis analysis, respectively. The second objective is to discuss an overall methodology allowing the identification and assessment of uncertainties underlying each task The paper introduces two illustrations of social data analysis, investigates various sources of uncertainty and describes a methodology to select criteria for uncertainty assessment.

Invasive species, such as jellyfish, cause economic losses in millions annually. Therefore, being able to accurately monitor and predict jellyfish is vital to several stakeholders (e.g. tourism, fishery, government). A potential tool to help these communities could be by combining a biophysical drift model with a processing chain for soft information fusion which would predict jellyfish occurrences. To guarantee accuracy, the model needs to be validated by actual data. This data can be gathered from citizens who reported jellyfish sightings on social media or a dedicated citizen science mobile app. As the information provided by citizens is spread among numerous atomic reports, we use a platform for soft information fusion to aggregate and fuse these reports into a single information network. The soft information fusion platform relies on the use of domain knowledge, provided through an ontology. The information network can then be queried to extract relevant features to validate the jellyfish drift model. Future work includes the initialisation of the model with soft information, as well as making use of the different levels of quality of the reports provided by citizens, in order to assess the quality of the fused information.

Content marketing objectives are to create and distribute valuable, relevant, and consistent content to attract and retain a clearly defined targeted audience. To earn credibility, the brand creates messages that are useful and make a positive difference in the lives of the prospects. As content items, at the most basic level, is information, this paper will present how semantic information fusion operator introduced by [17] becomes a cornerstone of an overall content marketing tool chain. After introducing Mathematical Morphology framework, this operator which relies on a conceptual graphs, will be sound defined. Some underlying morphological properties such as dilation or idempotence are demonstrated, and we explain how these properties can be used within semantic information fusion.

The development of mobile devices as well as social media platforms recently lead to the necessity of monitoring the latter during crisis and emergency situations. Paradoxically, the huge amount of information available through these new sources may lead to information gaps, within the Public Safety Organization operators' awareness. We describe some specific types of information gaps due first to imprecise or unreliable information and second to information overload. We then propose a set of tools aiming at reducing these information gaps and supporting the human operators in the social media generated information during crisis and emergency management. The first tool aims at geolocalising tweets relying on the content of the messages. The second tool provides sentiment analysis and clustering of multi-lingual messages and the third tool provides means for semantic information fusion and hypothesis evaluation relying on the contents and metadata of the tweets reporting about an event.

International audience

Most of studies in the field of information fusion focus on the production of high-level information from low-level data. The challenge is then to fuse this high-level information to produce a global and coherent information. Another approach consists in interpreting data as high-level information and fuse it at once. Our approach relies on the use of conceptual graphs model. The model is widely used for knowledge representation. We propose to go further and use it for information fusion. Conceptual graphs model contains aggregation operators such as join and maximal join. This paper is dedicated to the extension of the maximal join operator in order to manage heterogeneous information fusion. After describing the suitability of maximal join for high-level information fusion, we present the extension that we propose. The extension relies on relaxing the equality constraint on observations and on using fusion strategies. A case study illustrates our proposition.

The first challenge in crisis response management is the early damage and needs assessment based on all incoming information from various sources such as on field deployed sensors or human observations. Likewise, it is important for an efficient crisis response management to make sure that ambiguous terminology is clearly defined, and methodologies and indicators explained. To support the damage assessment, all the relevant items need to be formally modelled and represented in a machine-readable format. For that purpose, a formal ontology would provide the basis for the verbal description of the current situation understanding as well as the underlying semantics for any fusion related tasks and display operations.

One of the key issues of strategic intelligence within a crisis situation is to build an early assessment of the situation, based on a context sensitive information interpretation and through a well constructed situation representation. Our proposal is based on the conjunction of a conceptual modelling to represent situations out of document analysis and a reactive rule-based modelling to analyze them according to a domain knowledge and a goal. This paper focuses on this situation analysis process. But we present our global approach and sum-up the situation representation and its objectives. We introduce the complex event processing formalism used for the analysis and dynamic recognition of such situations. We illustrate our approach through a case study taken from what happened during the energy crisis in California in 2001