Subject indexing is vital for discovery but hard to sustain at scale and across languages. We release a large bilingual (English/German) corpus of catalog records annotated with the Integrated Authority File (GND), plus a machine-actionable GND taxonomy. The resource enables ontology-aware multi-label classification, mapping text to authority terms, and agent-assisted cataloging with reproducible, authority-grounded evaluation. We provide a brief statistical profile and qualitative error analyses of three systems. We invite the community to assess not only accuracy but usefulness and transparency, toward authority-anchored AI co-pilots that amplify catalogers’ work.

One of the main issues in thermoplastic polymer production plants is fouling in reactors. It happens when un-desired materials build up on the reactor surfaces, decreasing the effectiveness of heat transfer and raising operating expenses. It is possible to avoid excessive accumulation, enhance reactor performance, and more efficiently schedule maintenance by accurately forecasting the fouling factor over time. In order to predict the fouling factor using time series data, we created a fusion-based deep learning model in this study that incorporates LSTM, GRU, and Attention mechanism. The proposed model is designed to capture both long-term dependencies and short-term variations, while the Attention mechanism helps focus on critical time steps. Using actual data from a PE-EVA industrial plant from Versalis SpA, we trained and evaluated the model. The results showed strong performance, with an$\boldsymbol{R}^{2}$score of 0.87, MSE of 4.74 × 10−3, RMSE of 0.04691, and SMAPE of 12.53%. Our model outperformed the traditional time series models. These findings show that combining different deep learning models improves the accuracy and reliability of fouling factor forecasts.

Ontologies are nowadays widely used to organize information across specific domains, being effective due to their hierarchical structure and the ability to explicitly represent relationships between concepts. Knowledge engineering, like compiling companies’ vast bodies of knowledge into these structures, however, still represents a time-consuming, largely manually performed process, esp. with significant amounts of knowledge often only recorded within unstructured text documents. Since the recently introduced Large Language Models (LLMs) excel on text summarization, this raises the question whether these could be exploited within dedicated knowledge fusion architectures to assist human knowledge engineers by automatically suggesting relevant classes, instances and relations extracted from textual corpora. We therefore propose a novel approach that leverages the taxonomic structure of a partially defined ontology to prompt LLMs for hierarchical knowledge organization. Unlike conventional methods that rely solely on static ontologies, our methodology dynamically generates prompts based on the ontology’s existing class taxonomy, prompting the LLM to generate responses that extract supplementary information from unstructured documents. It thus introduces the concept of using ontologies as scaffolds for guiding LLMs, in order to realize a mutual interplay between structured ontological knowledge and the soft fusion capabilities of LLMs. We evaluate our proposed algorithm on a real-world case study, performing a knowledge fusion task on heterogeneous technical documentation from a medical prosthesis manufacturer.

Tracking a maneuvering target requires the modeling of the target’s movements by multiple pre-defined mathematical models. However, the uncertainty in the target’s dynamics can lead traditional model-based (MB) tracking algorithms to significant performance degradation when model mismatch occurs. To tackle this problem, we propose the use of a Recurrent Neural Network (RNN) for the purpose of learning complex target dynamics. Following the recent advances in state estimation provided by KalmanNet, a neural network-aided Kalman Filter, the proposed approach aims to exploit its tracking performance in a multiple model schema to compensate for model mismatch across maneuvers, leading to a more prompt response to motion switches. The results over a simulated set of maneuvering target trajectories demonstrate the potential of the proposed approach over the MB solution.

In this paper, we discuss and try a multi-image fusion approach with a multibranch Convolutional Neural Network (CNN) that implies a MultiHeadAttention (MHA) technique in the fusion center. This work studies the employment of the architecture on actual data containing different images of the same USB device. The images differ in the direction of the light at the moment of the acquisition. We observed that instead of employing a simple concatenation fusion of the outputs, the network architecture could employ a multibranch classification featurewise, which utilizes a multi-head attention mechanism instead of a channel attention one.

In recent years, segmentation algorithms utilizing deep learning have achieved outstanding performance in medical image segmentation. However, accurately delineating small anatomical structures continues to be a challenging task, even for the most advanced methods that produce impressive results. This challenge might arise from the use of small kernels and downsampling operations, which often emphasize complex high-level features at the expense of low-level details like edges. Inspired by recent research highlighting this challenge, we developed a novel architecture that combines the standard U Net with an additional branch harnessing the potential of large convolutional kernels. These large kernels are utilized in a decreasing-increasing manner over image features of the same size, guiding the network to focus on smaller parts. The proposed method demonstrated strong potential in segmenting small anatomical structures, surpassing our baseline and matching the performance of a robust state-of-the-art network across various datasets and domains, all while maintaining a relatively small number of parameters.

The majority of evidential fusion models presented in the literature is based on optimistic assumptions about the reliability of the models producing beliefs and assumes that they are equally reliable. At the same time, the belief models used in combination may have some limitations and may result in different reliabilities, which may decrease the performance of the combination. One way to confront this problem is to consider a discount rule utilizing reliability coefficients. One of the problems of using discounting is the way of modeling reliability coefficients. This paper proposes modeling reliability coefficients by considering a new effective measure of belief uncertainty. The new reliability coefficients are introduced in a multilayer decision fusion-based Convolutional Neural Network (CNN) architecture built within the Transferable Belief Model, as well as in a multimodal deep learning scenario. Case study results demonstrate the feasibility of representing reliability by the belief uncertainty measure considered.

The role of the military has been continuously evolving and expanding beyond traditional warfare. In addition to an increasingly asynchronous battlefield, today's military is involved in supporting peacekeeping, crisis management, and disaster relief, to name just a few. Such activities require intense contact with diverse non-military organizations such as government, police, emergency services, relief agencies, and religious leaders. As a result, the information needed for commanders and other decision-makers must come from a wide variety of sources in a wide variety of formats. Thus, any attempt at fusion of this information requires an underlying system concept that is able to standardize information from a variety of input sources, including device-derived information (sensors) and (multiple) natural language(s), as well as to understand complex context, both static and dynamic, and to be able to deal with measures of uncertainty which may vary from source-type to source-type. This paper presents several lessons learned from the ongoing work of a series of NATO Research Task Groups (which have focused on the changing data and information incorporating structured and unstructured human generated information with device-generated data needs for fusion in increasingly complex scenarios requiring interaction between devices, human-generated information and complex contextual backgrounds. We discuss the differing challenges facing the synergic information processing as scenario complexity grows.

In real-world monitoring tasks, a situation can be understood as a sequence of causally related events of interest. In road traffic control, such a situation could be a rear-end collision at the end of a traffic jam, which worsens congestion and requires clearing operations and potentially rerouting. Whereas conventional event sequence prediction focuses on sequences of individual events$\langle e_{1}, \ldots, e_{n}\rangle$, evolving situations thus can be conceived as sequences of states composed of multiple concurrent events, i.e., complex events:$\langle\{e_{1},\ldots, e_{m}\}, \ldots, \{e_{l},\ldots, e_{n}\}\rangle$. Situation (evolution) prediction thus requires learning a transition model for these complex events to provide the expectations for potential successor event types. In previous work, this was represented by a Markov Chain defined on the observed complex events. However, using the entire event composite as “atomic” situation state representation does not allow capturing patterns between its individual events (e.g., events of type “accident” share similar successor event types across different event composites), nor generalizing behaviors between similar event types or incorporating additional features. Hence, we propose a neural modeling approach to learn a distributed representation of a given situation dataset. By encoding the input states as conjunction of their individual comprised events, the devised model can learn associations (i.e., enable an “information flow”) between individual event types, allowing to capture similar behaviors across different situations. We test our approach on both synthetic and real-world datasets.

In everyday life as well as in asymmetric warfare domain, to achieve the intended goals, agents often do not make use of the designed and purpose-built tools, but some other tools whose features simply fit for the purpose. The present paper discusses the possibility of capturing and integrating relations and features from context that could drive the retrieval of possible candidate substitutes for the properly designed artifacts through Lexical Semantics Exploitation. Generative Lexicon theory assumes a structure (QualiaStructure) organizing the semantic content carried by lexical items through roles. Among them, theTelicrole exposes the function or purpose of the predicated entity and theConstitutiverole exposes its component parts. We argue that the typical function an entity has been thought for is related to its internal constituents. We also argue that a knowledge base and a proper metrics can be conveniently built extractingQualiaelements from suitable text corpora.

Context has received significant attention in recent years within the Information Fusion community as it can bring several advantages to information fusion processing by allowing for refining estimations, explaining observations and constraining processing and thereby improving the quality of inferences. At the same time context utilization involves concerns about the quality of the contextual information and its relationship with the quality of information obtained from observation and estimations that may be of low fidelity, contradictory, or redundant. Knowledge of the quality of this information and its effect on the quality of context characterization can improve contextual knowledge. At the same time, knowledge about a current context can improve the quality of observation and fusion results. This paper discusses the issues associated with understanding and evaluating Information Quality as well as Quality of Context, their relationships and their effect on fusion system performance.

Tracking objects in complex dynamic environments can be less challenging once their behavior is recognized. Inferring on targets' future actions based on their past can be addressed via probabilistic reasoning. Context information plays a crucial role in the reasoning process as it provides additional clues about targets' behavior. Combining context reasoning with target tracking continues to increase with the availability of supporting information. The framework here discussed views target's actions as a Hidden Markov Model (HMM) with relevant context associated with each node. Context is at each time step selected based on immediate and goal driven sets of actions. Inference in the HMM is conditioned on prior target's measurements and the belief state conditioned on context. This posterior is then compared with the target's state estimate in order to adjust the switching probability in the Interactive Multiple Models (IMM) tracking process.

Grid-based environment mapping and obstacle detection becomes increasingly more challenging when sensors' readings are highly contrasting. Without measurement prediction, commonly used approaches to grid fusion weight the sensor grids equally unless specified otherwise by the user. Empirically adjusted measurement weights are tailored only for certain scenarios and are not at all suited for a general purpose mapping. It therefore becomes apparent, that sensor weights need to be adjusted recursively during the map building process. We show that discrepancies between the grids can be exploited in such a manner where fusion of contradicting information will be less susceptible to sensor weighting and the accuracy of the mapped environment can be further improved. We present a realization of such a conflict resolution occupancy grid mapping, which combines grid-based mapping and situation assessment in a holistic approach.

Real-time detection, tracking, recognition, and activity understanding of moving objects from multiple sensors represent fundamental issues to be solved in order to develop surveillance systems that are able to autonomously monitor wide and complex environments. The algorithms that are needed span therefore from image processing to event detection and behaviour understanding, and each of them requires dedicated study and research. In this context, sensor fusion plays a pivotal role in managing the information and improving system performance. Here we present a novel fusion framework for combining the data coming from multiple and possibly heterogeneous sensors observing a surveillance area.

In this paper we propose an architecture for a surveillance application aimed to the automatic recognition of complex events. The main novelty of this work consists in the design of an effective and viable solution for the actual implementation of a complex automatic surveillance system that explicitly separates signal processing routines from the reasoning modules. We describe our ongoing efforts in the representation of the domain knowledge and in the development of a framework that allows the operator to easily check and update the systempsilas knowledge base. The taxonomical knowledge is expressed through ontologies (OWL), the event classification logic is expressed using a dedicated rule language (Jess), and the implementation is based on the java language.

We here present a multi-sensor data fusion architecture that takes into account the performance of video sensors in detecting moving targets for video surveillance purposes. Target detection and tracking is performed via classification by an ensemble of classifiers learned online using heterogeneous features for each target. A novel approach is then used to estimate the position of the target on the ground plane map by temporally fusing likelihood maps, then by approximating likelihoods analytically by a Gaussian function, and eventually projecting and fusing the likelihood functions. Experimental results are shown on real-world video sequences.

In this paper we propose a real-time algorithm for detecting and tracking moving objects in a video sequence. Based on the on-line boosting framework, our algorithm is able to detect an object as a member of a class, e.g. pedestrian, then a specific model for each instance of the class can be built on-line allowing at the same time robust tracking and recognition of the particular instance as it leaves and re-enters the scene. Promising experimental results have been performed on standard video sequences.

On-line boosting is a recent advancement in the field of machine learning that has opened a new spectrum of possibilities in many diverse fields. With respect to a static strong classifier, the on-line algorithm updates the ensemble using new incoming samples. This idea has been successfully exploited in tasks such as detection and tracking as a classification problem with good results. Our purpose is to provide an efficient and robust framework to build a cascade of on-line updated classifiers that, speeding up the application time, allows the employment of a higher number of features, thus achieving better detection performance.

In this paper, we investigate the problem of representing and maintaining rule knowledge for a video surveillance application. We focus on complex events representation which cannot be straightforwardly represented by canonical means. In particular, we highlight the ongoing efforts for a unifying framework for computable rule and taxonomical knowledge representation.

In this paper, we address the problem of representing domain knowledge for situation awareness in a security application. While ontologies are appropriate for describing taxonomical knowledge, they cannot express more complex knowledge such as entailments. In this paper, we describe how domain knowledge can be encoded through OWL ontologies and SWRL rules in order to reason about the entities and their interactions in a surveillance application. We describe how events can be described through ontologies and how video sequences can be annotated using the MPEG-7 standard.

Abstract: Ambient intelligence envisages an articulated, though transparent, interaction between the user and the environment. According to this grand vision, appliances and systems embedded in the environment have to react to the user's presence and provide services in a customized fashion. Therefore, ambient intelligence systems should be endowed with context awareness capabilities in order to provide the proper responses for each user. This paper specifically shows how the system can be instructed to recognize events occurring in the observed environment for security purposes.

In this correspondence, we address the problem of fusing data for object tracking for video surveillance. The fusion process is dynamically regulated to take into account the performance of the sensors in detecting and tracking the targets. This is performed through a function that adjusts the measurement error covariance associated with the position information of each target according to the quality of its segmentation. In this manner, localization errors due to incorrect segmentation of the blobs are reduced thus improving tracking accuracy. Experimental results on video sequences of outdoor environments show the effectiveness of the proposed approach.

The state of the art of surveillance systems include a large set of techniques for both low level and high level tasks. In particular, the research community has witnessed in the last decade a high proliferation of techniques that span from object detection and tracking to object recogni- tion and event understanding. Although some techniques have been proven to be very effective those tasks cannot be considered solved. Although more effort is needed in the event analysis field, a new problem arises from the develop- ment of large scale networked surveillance systems: infor- mation sharing. The way information is shared between the nodes of the surveillance network today represents a key- point issue. To provide a first and novel solution to such a problem, we propose an innovative system architecture for a video surveillance system with distributed processing over multiple processing units and with distributed communica- tion over multiple heterogeneous channels (wireless, satel- lite, local IP networks, etc.). In particular, a new real-time technique for changing the video transmission parameters (e.g., frame rate, spatial/color resolution, etc.) according to the bandwidth available will be here presented.

In this paper, we address the problem of identifying anomalous events in the context of a multi sensor surveillance system. Targets' trajectories are analyzed and compared to common patterns of activity represented as clusters of trajectories. Here we extend our previous work to cater for observations provided by multiple cameras observing the same scene. Data fusion is performed within the Dempster-Shafer theory of evidence framework. The proposed approach is validated through experimental results performed in the context of an automatic road traffic monitoring application

Video security systems are required to extract semantic information on the ongoing activities in a given environment, in order to detect suspicious or potentially dangerous behaviours. To achieve this goal we proposed a real-time trajectory clustering algorithm able to detect common patterns of activity; the output of this algorithm can be used to facilitate high level behaviour analysis and situation assessment. In this paper we extend our previous work using the Dempster-Shafer theory of evidence framework. This allows an explicit handling of evidence accrual and fusion that gives more meaningful data for high-level activity analysis modules.

In this paper we present a trajectory clustering method suited for video surveillance and monitoring systems. The clusters are dynamic and built in real-time as the trajectory data is acquired, without the need of an off-line processing step. We show how the obtained clusters can be successfully used both to give proper feedback to the low-level tracking system and to collect valuable information for the high-level event analysis modules.

In this paper, we provide an objective sensor performance evaluation approach to automatically carry out the sensor evaluation task in a multi-sensor system for video surveillance. That is, given a set of sensors monitoring the same area, the proposed technique estimates the accuracy of each sensor in detecting a given target. This measure depends on the conditions under which the sensor is operating and is computed per sensor and per target at every time instant. In this way, data coming from the different sensors may be weighted accordingly and fused. Promising experimental results are presented on real video sequences.

Moving toward the implementation of the intelligent building idea in the framework of ambient intelligence, a video security application for people detection, tracking, and counting in indoor environments is presented in this paper. In addition to security purposes, the system may be employed to estimate the number of accesses in public buildings, as well as the preferred followed routes. Computer vision techniques are used to analyze and process video streams acquired from multiple video cameras. Image segmentation is performed to detect moving regions and to calculate the number of people in the scene. Testing was performed on indoor video sequences with different illumination conditions.

An outdoor multi-camera video surveillance system operating under changing weather conditions is presented. A new confidence measure, appearance ratio (AR), is defined to evaluate automatically the sensors' performance for each time instant. By comparing their ARs, the system can select the most appropriate cameras to perform specific tasks. When redundant measurements are available for a target, the AR measures are used to perform a weighted fusion of them. Experimental results are presented on outdoor scenes under different weather conditions.

In this paper, a robust real-time face detection system based on the integration of different location methods is proposed. A hierarchical architecture composed of three levels is designed. At the first level, a change detection method is applied to detect blobs of moving objects (i.e., humans) in the scene. Then, the silhouette of each blob is analyzed to focalize the attention of the system on small image areas where the probability of finding human heads is high. At the second level, two different methods, i.e., the skin color and the principal component analysis, are applied to locate human faces. Finally, the higher level fuses the obtained location data to improve the face detection reliability. The system is tested in outdoor environments in the context of a video-based surveillance system.

A distributed sensor network (DSN) for video surveillance is presented. The system is able to manage heterogeneous sensors (e.g. optical, infrared, radar, etc.) to operate during night and day and in the presence of different weather conditions (e.g. fog, rain, etc.). Data fusion is therefore mandatory and exploited at different levels to integrate the information produced by the sensors. The architecture presented has low network requirements, is easily scalable, maintainable and allows an easy distribution of the system on a wide outdoor area.