Abstract Time series data is one of the most popular data modalities in critical domains such as industry and medicine. The demand for algorithms that not only exhibit high accuracy but also offer interpretability is crucial in such fields, as decisions made there bear significant consequences. In this paper, we present ProtoTSNet, a novel approach to interpretable classification of multivariate time series data, through substantial enhancements to the ProtoPNet architecture. Our method is tailored to overcome the unique challenges of time series analysis, including capturing dynamic patterns and handling varying feature significance. Central to our innovation is a modified convolutional encoder utilizing group convolutions, pre-trainable as part of an autoencoder and designed to preserve and quantify feature importance. We evaluated our model on 30 multivariate time series datasets from the UEA archive, comparing our approach with existing explainable methods as well as non-explainable baselines. Through comprehensive evaluation and ablation studies, we demonstrate that our approach achieves the best performance among ante-hoc explainable methods while maintaining competitive performance with non-explainable and post-hoc explainable approaches, providing interpretable results accessible to domain experts.

The digitization of cultural heritage collections has opened new directions for research, yet the lack of enriched metadata poses a substantial challenge to accessibility, interoperability, and cross-institutional collaboration. In several past years neural networks models such as YOLOv11 and Detectron2 have revolutionized visual data analysis, but their application to domain-specific cultural artifacts - such as manuscripts and incunabula - remains limited by the absence of methodologies that address structural feature extraction and semantic interoperability. In this position paper, we argue, that the integration of neural networks with semantic technologies represents a paradigm shift in cultural heritage digitization processes. We present the Metadata Enrichment Model (MEM), a conceptual framework designed to enrich metadata for digitized collections by combining fine-tuned computer vision models, large language models (LLMs) and structured knowledge graphs. The Multilayer Vision Mechanism (MVM) appears as the key innovation of MEM. This iterative process improves visual analysis by dynamically detecting nested features, such as text within seals or images within stamps. To expose MEM’s potential, we apply it to a dataset of digitized incunabula from the Jagiellonian Digital Library and release a manually annotated dataset of 105 manuscript pages. We examine the practical challenges of MEM’s usage in real-world GLAM institutions, including the need for domain-specific fine-tuning, the adjustment of enriched metadata with Linked Data standards and computational costs. We present MEM as a flexible and extensible methodology. This paper contributes to the discussion on how artificial intelligence and semantic web technologies can advance cultural heritage research, and also use these technologies in practice.

ABSTRACT This paper proposes a two‐stage framework that combines uncertainty reduction and predictive modelling to enhance the accuracy of battery Remaining Useful Life (RUL) prediction. In the first stage, a simplified fuzzy optimization learning model is introduced to mitigate uncertainty caused by abnormal capacity fluctuations in battery data. The proposed fuzzy model reconstructs degradation data into a consistent downward trend based on mid‐ and short‐term tendencies of the battery, alleviating abnormal variability and improving suitability for predictive modelling. In the second stage, uncertainty arising during the recursive prediction process of a standalone Transformer model was mitigated through the integration of a particle filter. This approach dynamically manages prediction errors using particles, effectively controlling cumulative errors and enhancing the stability and reliability of long‐term predictions. This methodology can lead to extended battery life and increased operational reliability through accurate RUL prediction. The proposed methodology is validated through experiments using NASA and CALCE battery datasets, demonstrating superior prediction accuracy and stability compared to conventional approaches by systematically reducing uncertainties.

This study proposes an open-environment, multi-label human risk classification framework, capable of identifying possible risks to which individuals appearing on input video data are exposed. The framework consists of an ensemble of models covering object detection, action recognition, context understanding and text classification tasks. Each model is evaluated separately in the context of home environments, with the overall framework performing well in each evaluation after fine tuning. The models were evaluated using a combination of several datasets, including Charades, ETRI-Activity3D, and custom video question answering and risk datasets. This study exploits the ability of large language models to interpret semantic visual features combined with textual input in order to understand the context in which the person is placed. The framework’s ability to output multiple risks and its cross-domain capabilities make it a powerful tool that can enhance current risk management systems in a variety of scenarios, such as homes, construction sites and industry.

Experiments in affective computing are based on stimulus datasets that, in the process of standardization, receive metadata describing which emotions each stimulus evokes. In this paper, we explore an approach to creating stimulus datasets for affective computing using generative adversarial networks (GANs). Traditional dataset preparation methods are costly and time consuming, prompting our investigation of alternatives. We conducted experiments with various GAN architectures, including Deep Convolutional GAN, Conditional GAN, Auxiliary Classifier GAN, Progressive Augmentation GAN, and Wasserstein GAN, alongside data augmentation and transfer learning techniques. Our findings highlight promising advances in the generation of emotionally evocative synthetic images, suggesting significant potential for future research and improvements in this domain.

Abstract This study evaluates quality management practices in Industry 4.0 in a specific case of steel manufacturing. We formulate a novel proposal based on Semantic Data Mining techniques a step towards knowledge‐driven decision support based on the industrial Six Sigma approach. In our research we combine machine learning classifiers, and explanation generation algorithms with the Six Sigma practice to automate the evaluation of quality of steel products and determine origins of their defects. We describe our original method, and provide evaluation of the results with real–life data from our industrial partner.

Current review papers in the area of Affective Computing and Affective Gaming point to a number of issues with using their methods in out-of-the-lab scenarios, making them virtually impossible to be deployed. On the contrary, we present a game that serves as a proof-of-concept designed to demonstrate that—being aware of all the limitations and addressing them accordingly—it is possible to create a product that works in-the-wild. A key contribution is the development of a dynamic game adaptation algorithm based on the real-time analysis of emotions from facial expressions. The obtained results are promising, indicating the success in delivering a good game experience.

Contemporary machine learning techniques are capable of extracting complex structure from data in a way that complements or exceeds manual examination, yet, as is welldocumented, many of these techniques suffer from a lack of interpretability. This paper extends previous work on explainable and interpretable machine learning, in particular on the ‘Knowledge-Augmented Clusters (KnAC)’ approach, allowing human users to benefit from uninterpretable ‘black box’ models to extract structure from datasets by clustering and to make this better understandable. One of the key functions of KnAC is to relate expert-annotated clusters to clusters that have been identified by a machine learning method, and then provide a comprehensible explanation, thus clarifying the relationships that KnAC discovered. Our novel contribution in this paper is to examine the usefulness of subgroup discovery as a way to generate comprehensible explanations within KnAC, and to compare this to the existing approach based on the XAI algorithm Anchors through a detailed evaluation. We find that the approach using subgroup discovery performs equally or better in our extensive experimentation testing this on six different datasets.

The field of Explainable Predictive Maintenance (PM) is concerned with developing methods that can clarify how AI systems operate in the PM domain. One of the challenges of creating maintenance plans is integrating AI output with human decision-making pro- cesses and expertise. For AI to be helpful and trustworthy, fault predictions must be contextualized and easily comprehensible to humans. This involves providing tailored explanations to different actors depending on their roles and needs. For example, engineers can be connected to technical installation blueprints, while man- agers can evaluate system downtime costs, and lawyers can assess safety-threatening failures' potential liability. In many industries, black-box AI systems analyze sensor data to predict failures by detecting anomalies and deviations from typical behavior with impressive accuracy. However, PM is just one part of a broader context that aims to identify the most probable causes, develop a recovery plan, and estimate remaining useful life while providing alternative solutions. Achieving this requires complex interactions among various actors in industrial and decision-making processes. Our tutorial explores current trends, and promising research directions in Explainable AI (XAI) relevant to Explainable Predictive Maintenance (XPM), and future challenges and open issues on this topic. We will also present three case studies that highlight XPM's challenges in bus and train operations and steel factories.

Abstract Pattern discovery in multidimensional data sets has been the subject of research for decades. There exists a wide spectrum of clustering algorithms that can be used for this purpose. However, their practical applications share a common post-clustering phase, which concerns expert-based interpretation and analysis of the obtained results. We argue that this can be the bottleneck in the process, especially in cases where domain knowledge exists prior to clustering. Such a situation requires not only a proper analysis of automatically discovered clusters but also conformance checking with existing knowledge. In this work, we present Knowledge Augmented Clustering (KnAC). Its main goal is to confront expert-based labelling with automated clustering for the sake of updating and refining the former. Our solution is not restricted to any existing clustering algorithm. Instead, KnAC can serve as an augmentation of an arbitrary clustering algorithm, making the approach robust and a model-agnostic improvement of any state-of-the-art clustering method. We demonstrate the feasibility of our method on artificially, reproducible examples and in a real life use case scenario. In both cases, we achieved better results than classic clustering algorithms without augmentation.

We present Loki, a semantic wiki designed to support the collaborative knowledge engineering process with the use of software engineering methods. Designed as a set of DokuWiki plug-ins, it provides a variety of knowledge representation methods, including semantic annotations, Prolog clauses, and business processes and rules oriented to specific tasks. Knowledge stored in Loki can be retrieved via SPARQL queries, in-line Semantic MediaWiki-like queries, or Prolog goals. Loki includes a number of useful features for a group of experts and knowledge engineers developing the wiki, such as knowledge visualization, ontology storage, or code hint and completion mechanism. Reasoning unit tests are also introduced to validate knowledge quality. The paper is complemented by the formulation of the collaborative knowledge engineering process and the description of experiments performed during Loki development to evaluate its functionality. Loki is available as free software at https://loki.re.

The field of Deep Visual Analytics (DVA) has recently arisen from the idea of developing Visual Interactive Systems supported by deep learning, in order to provide them with large-scale data processing capabilities and to unify their implementation across different data and domains. In this paper we present DeepVATS, an open-source tool that brings the field of DVA into time series data. DeepVATS trains, in a self-supervised way, a masked time series autoencoder that reconstructs patches of a time series, and projects the knowledge contained in the embeddings of that model in an interactive plot, from which time series patterns and anomalies emerge and can be easily spotted. The tool includes a back-end for data processing pipeline and model training, as well as a front-end with an interactive user interface. We report on results that validate the utility of DeepVATS, running experiments on both synthetic and real datasets. The code is publicly available on https://github.com/vrodriguezf/deepvats.

The development of predictive maintenance (PdM) solutions is one of the key challenges in the industry today. Manufacturing processes are usually well described by the law of physics and mathematical equations, but the irregularity and randomness of the asset degradation process make it a demanding task to model it. This makes physics-driven models insufficient for this kind of problem. On the other hand, data-driven models, mainly Artificial Intelligence (AI), are gaining much interest in research and applications due to their flexibility and robustness. A compromise between these two approaches are hybrid models that take into account the physics of the process and use modern AI methods to learn its behavior. The next challenge for AI models is to provide information on their reasoning to build understading and trustworthiness, which can be achieved through post-hoc Explainable AI (XAI) methods. In this paper, we use a Physics-Informed Autoencoder (PIAE) in a semi-supervised manner to learn the degradation process of work rolls in the cold- rolling process. We incorporate physics knowledge into the AI model by extending its input space and applying feature masking during the prediction phase. The results of the research show that such an architecture is capable of distinguising between low- and high-wear observations. Furthermore, we include the XAI layer in the model, which gives explanations for the prediction of the model through counterfactuals.

Anomaly detection is emerging trend in manufacturing processes and may be considered as part of the Industry 4.0 revolution. It can serve both as diagnostic tool in predictive maintenance task, as well as trace back mechanism for assessing quality of production or services. In this paper we describe and approach for explainable anomaly detection in industrial data which contains sequential and static features. We based our solution on modified autoencoder architecture with Long Short-Term Memory layers. To address a problem of explinability in deep learning and find origin of the anomalies we have engaged the SHAP method, which gives both local and global explanations of the model. Analysis of SHAP explanations allowed us to determine the source of majority of anomalies detected by deep learning model. We demonstrated the feasibility of our approach on synthetic, reproducible dataset and on real-life data gathered from hot rolling industrial process.

Explainable Artificial Intelligence (XAI) aims at introducing transparency and intelligibility into decision-making process of AI systems. Most of the work in this area is focused on supervised machine learning tasks such as classification and regression. Unsupervised algorithms such as clustering can also be explained with existing approaches. This is most often achieved by explaining a classifier trained on cluster data with cluster labels as a dependant variable. However, with such a transformation the information about cluster shape and distribution is lost, which may lead to wrong interpretation of explanations. In this paper, we introduce a method that aids end experts in cluster analysis with human-readable rule-based explanations. We use state-of-the-art explanation mechanism on the multidimensional bounding boxes that represent arbitrarily-shaped clusters. We demonstrate our approach on reproducible synthetic datasets.

In the paper we present an original semantic data mining solution for Internet of Things. We designed an ontology for sensors used in the monitoring of goods and vehicles. Moreover, we developed a practical implementation for this solution based on the state of the art semantic tools, namely RDF triple stores with dedicated interfaces. However, due to number of undocumented limitations of these tools, this process turned out to require number of refactorizations to deliver a working solution. We describe these challenges and solutions we found. We believe our experiences are valuable for researchers and practitioners in the field, as production-ready development of such semantic solutions is far more complex than lab experiments. In fact, our work was developed for the deployment in a production environment, with a close cooperation with an IT company funding the project.

This article tackles the problem of checking the conformance between a business process model and the data produced by its execution in cases where the data are not given as an event log, but as a set of time series containing the evolution of the variables involved in the process. Tasks in the process model are no longer restricted to the occurrence of a single event, and instead, they can be expressed as a set of temporal conditions about the values of the variables in the log. This causes a paradigm shift in conformance checking (and process mining at a more general level), and because of this, the formalization of both the data and the process model and the algorithms are here redesigned and adapted for this challenging perspective. To illustrate the effectiveness of our approach, an experimental evaluation on a real-world time-series log is carried out, highlighting the benefits of this change of paradigm.

Abstract Building mobile context‐aware systems is inherently complex and non‐trivial task. It consists of several phases starting from acquisition of context, through modeling to execution of contextual models. Today, such systems are mostly implemented on mobile platforms, that introduce specific requirements, such as intelligibility, robustness, privacy, and efficiency. Over the last decade, along with the rapid development of mobile industry, many approaches were developed that unevenly support these requirements. This is mainly caused by the fact that current modelling and reasoning methods are not crafted to operate in mobile environments. We argue that the use of rule‐based reasoning tailored to mobile environments is an optimal solution. Rules are based on symbolic knowledge representation, as such they meet the general tendency to enforce understandability, intelligibility, and controllability of artificial intelligence software, as stated in the recent European Union General Data Protection Regulation. To this goal, we introduce a lightweight rule engine dedicated for Android platform called HEARTDROID. It executes models in the HMR+ rule language that are capable of expressing uncertainty of knowledge, capturing dynamics of mobile environment and provide high level of intelligibility. We present a qualitative and quantitative comparison of HEARTDROID with the most popular rule engines available.

As technology becomes more ubiquitous and pervasive, special attention should be given to human-computer interaction, especially to the aspect related to the emotional states of the user. However, this approach assumes very specif c mode of data collection and storage. This data is used in the affective computing experiments for human emotion recognition. In the paper we describe a new software solution for mobile devices that allows for data acquisition from wristbands. The application reads physiological signals from wristbands and supports multiple recent devices. In our work we focus on the Heart Rate (HR) and Galvanic Skin Response (GSR) readings. The recorded data is conveniently stored in CSV files, ready for further interpretation. We provide the evaluation of our application with several experiments. The results indicate that the BandReader is a reliable software for data acquisition in affective computing scenarios.

Rules and processes constitute powerful representation forms.Although the same notions can be expressed in both of these representations, there is a significant difference in abstraction levels between processes and rules.In practice, rules are mostly used for the specification of rule task logic in processes.In this paper, we present various options where and how rules can be perceived in business processes.We introduce rule-based pattern perspective for process models, focusing on BPMN 2.0 Business Process Models.

Engaging users in threat reporting is important in order to improve threat monitoring in urban environments. Today, mobile applications are mostly used to provide basic reporting interfaces. With a rapid evolution of mobile devices, the idea of context awareness has gained a remarkable popularity in recent years. Modern smartphones and tablets are equipped with a variety of sensors including accelerometers, gyroscopes, pressure gauges, light and GPS sensors. Additionally, the devices become computationally powerful which allows for real-time processing of data gathered by their sensors. Universal access to the Internet via WiFi hot-spots and GSM network makes mobile devices perfect platforms for ubiquitous computing. Although there exist numerous frameworks for context-aware systems, they are usually dedicated to static, centralized, client-server architectures. There is still space for research in the field of context modeling and reasoning for mobile devices. In this paper, we propose a lightweight context-aware framework for mobile devices that uses data gathered by mobile device sensors and performs on-line reasoning about possible threats based on the information provided by the Social Threat Monitor system developed in the INDECT project.

Business Process models help to visualize the processes of an organization.There are open-source process execution engines that provide the environments for enacting process models.However, they lack semantization capabilities.In this paper, an overview of Business Process semantization techniques is provided.Moreover, we discuss the common architecture of the selected open-source process execution environments (Activiti, jBPM and Camunda) and provide the insights how they can be improved using semantization methods.We also present the use of the introduced techniques in the Prosecco (Processes Semantics Collaboration for Companies) research project.

Knowledge management in business information systems often requires a unified dictionary of business concepts, that allows for a transparent integration of such systems.Thanks to it sharing the conceptualization between users becomes possible, and better decision support facilities can be provided.The Prosecco project is a research and development project aims to address the needs and constraints of small and medium enterprises by designing methods that will significantly improve BPM systems.In this paper we focus on the development of ontology-based mechanisms allowing for creating taxonomies of business logic concepts unifying system objects.Building a taxonomy of business concepts shared by number of SMEs targeted in the project and then turning it into a formalized ontology integrating the software components is a major challenge.The paper demonstrates how this ontology is used to unify vocabulary of business processes and rules.The original contribution of this research discussed in the paper is the design and implementation of the ontology, and the demonstration of its practical use in the system.

In the paper a new approach to automating unit testing in software engineering is described. It is based on the use of a rule-based specification of a module. This specification is then used to automatically generate test cases, thus simplifying the testing process. The paper discusses the application of decision tables to capture the specification of the system. The tables are described in a formalized way using a design method for rule-based expert systems. They are then used to generate test cases for the JUnit software. This intelligent technique can speedup, automate and improve the software testing process.

In the paper a new architecture for the design and evaluation of business process models is proposed. The architecture is based on the use of a semantic wiki system. It supports a distributed and collaborative modeling approach. In such a case, models can be created by a possibly distributed team of analysts. Moreover, they can be gradually improved using an integrated quality metrics system. We give the main architectural assumptions for this approach. Requirements for the system are described along with a prototype implementation. This system offers a practical tool support for the automation of the design and evaluation of models in a distributed environment.

Formalized knowledge representation methods allow to build useful and semantically enriched knowledge bases which can be shared and reasoned upon. Unfortunately, knowledge acquisition for such formalized systems is often a time-consuming and tedious task. The process requires a domain expert to provide terminological knowledge, a knowledge engineer capable of modeling knowledge in a given formalism, and also a great amount of instance data to populate the knowledge base. We propose a CAPTCHA-like system called AceCAPTCHA in which users are asked questions in a controlled natural language. The questions are generated automatically based on a terminology stored in a knowledge base of the system, and the answers provided by users serve as instance data to populate it. The implementation uses AceWiki semantic wiki and a reasoning engine written in Prolog.

In this paper challenges of using Knowledge Engineering methods in the Software Engineering practice, with focus on the visual design methods for software applications, are discussed. The paper concerns practical design issues of rule-based models integrated with business applications built using the Model-View-Controller (MVC), or equivalent architectural pattern. The Unified Modeling Language (UML) constitutes a standardized notation among software engineers. Various attempts has been made to design the model in a way alternative to UML. Recently, the Business Rules approach has been proposed as a new way of capturing the functional requirements in a designer-friendly fashion. However, concepts on which the rule-based representation is based cannot be directly modeled in UML, because the semantics of UML diagrams does not correspond to rule semantics. Our research considers a method of designing rule bases which will be proper and consistent with the UML design. The approach has several important features: the core logic of the application is clearly identified and separated, and it is built in a declarative way, which makes the design transparent, easier to follow and develop. When rules are used to model the application logic, one has to keep in mind some common and well-known limitations of the existing rule tools. Our solution to these problems consists of using an expressive visual rule design formalism called XTT2, defining a direct translation between XTT2 and selected UML diagrams, implementing practical translators between the logical model and MOF-based UML model, and using the verification features provided with the XTT2 framework to assure the quality of the rule base. This solution seems to be superior both to existing visual UML rule notations, as well as visual design tools for rules. It does not extend any custom UML artifacts and can be used with standard UML tools.

Semantic wikis constitute an increasingly popular class of systems for collaborative knowledge engineering. We developed Loki, a semantic wiki that uses a logic-based knowledge representation. It is compatible with semantic annotations mechanism as well as Semantic Web languages. We integrated the system with a rule engine called Heart that supports inference with production rules. Several modes for modularized rule bases, suitable for the distributed rule bases present in a wiki, are considered. Embedding the rule engine enables strong reasoning and allows to run production rules over semantic knowledge bases. In the paper, we demonstrate the system concepts and functionality using an illustrative example.