Humans often use expressions with vague terms which play a fundamental role for effective communication. These expressions are successfully modeled with fuzzy technology, but they are not usually integrated yet with Natural Language Processing models and techniques. Large-scale pre-trained language models yield excellent results in many language tasks, but they have some drawbacks such as their lack of transparency and thorough temporal reasoning capabilities. Therefore, the use of such models may provoke inconsistent or incorrect dialogues in the context of conversational agents which were aimed at providing users of intelligent systems with interactive explanations. In this paper, we propose a model for fuzzy temporal reasoning to overcome some inconsistencies detected in pre-trained language models in a specific application domain of a conversational agent carefully designed for providing users with explanations which are endowed with a good balance between naturalness and fidelity. More precisely, starting from a knowledge graph that provides an intuitive representation of the entities and relations in the application domain, we describe how to map the temporal information onto a fuzzy temporal constraint network. This formalism allows to represent imprecise temporal information and provides mechanisms for checking consistency in conversations. In addition, as a proof of concept, we have developed TimeVersa, a conversational agent which integrates the proposed model into an application domain (i.e., a virtual assistant for tourists) that requires handling imprecise temporal constraints. We illustrate in a use case how the agent can identify temporal inconsistencies and answer queries related to temporal information properly. Results after a user study report that users' perception of consistency is significantly higher in a conversation with TimeVersa than in a similar conversation using the well-known GPT-3 Large Language Model, when vague temporal information is involved. The proposed approach is a step forward for developing conversational agents operating in application domains that require temporal reasoning under uncertainty.

In this work we link the understandability of machine learning models to the complexity of their SHapley Additive exPlanations (SHAP). Thanks to this reframing we introduce two novel metrics for understandability: SHAP Length and SHAP Interaction Length. These are model-agnostic, efficient, intuitive and theoretically grounded metrics that are anchored in well-established game-theoretic and psychological principles. We show how these metrics resonate with other model-specific ones and how they can enable a fairer comparison of epistemically different models in the context of Explainable Artificial Intelligence. In particular, we quantitatively explore the understandability-performance tradeoff of different models which are applied to both classification and regression problems. Reported results suggest the value of the new metrics in the context of automated machine learning and multi-objective optimisation.

Electronic health records provide rich, heterogeneous data about the evolution of the patients’ health status. However, such data need to be processed carefully, with the aim of extracting meaningful information for clinical decision support. In this paper, we leverage interpretable (deep) learning and signal processing tools to deal with multivariate time-series data collected from the Intensive Care Unit (ICU) of the University Hospital of Fuenlabrada (Madrid, Spain). The presence of antimicrobial multidrug-resistant (AMR) bacteria is one of the greatest threats to the health system in general and to the ICUs in particular due to the critical health status of the patients therein. Thus, early identification of bacteria at the ICU and early prediction of their antibiotic resistance are key for the patients’ prognosis. While intelligent data-based processing and learning schemes can contribute to this early prediction, their acceptance and deployment in the ICUs require the automatic schemes to be not only accurate but also understandable by clinicians. Accordingly, we have designed trustworthy intelligent models for the early prediction of AMR based on the combination of meaningful feature selection with interpretable recurrent neural networks. These models were created using irregularly sampled clinical measurements, both considering the health status of the patient and the global ICU environment. We explored several strategies to cope with strongly imbalance data, since only a few ICU patients are infected by AMR bacteria. It is worth noting that our approach exhibits a good balance between performance and interpretability, especially when considering the difficulty of the classification task at hand. A multitude of factors are involved in the emergence of AMR (several of them not fully understood), and the records only contain a subset of them. In addition, the limited number of patients, the imbalance between classes, and the irregularity of the data render the problem harder to solve. Our models are also enriched with SHAP post-hoc interpretability and validated by clinicians who considered model understandability and trustworthiness of paramount concern for pragmatic purposes. Moreover, we use linguistic fuzzy systems to provide clinicians with explanations in natural language. Such explanations are automatically generated from a pool of interpretable rules that describe the interaction among the most relevant features identified by SHAP. Notice that clinicians were especially satisfied with new insights provided by our models. Such insights helped them to trust the automatic schemes and use them to make (better) decisions to mitigate AMR spreading in the ICU. All in all, this work paves the way towards more comprehensible time-series analysis in the context of early AMR prediction in ICUs and reduces the time of detection of infectious diseases, opening the door to better hospital care.

The explanatory capacity of interpretable fuzzy rule-based classifiers is usually limited to offering explanations for the predicted class only. A lack of potentially useful explanations for non-predicted alternatives can be overcome by designing methods for the so-called counterfactual reasoning. Nevertheless, state-of-the-art methods for counterfactual explanation generation require special attention to human evaluation aspects, as the final decision upon the classification under consideration is left for the end user. In this paper, we first introduce novel methods for qualitative and quantitative counterfactual explanation generation. Then, we carry out a comparative analysis of qualitative explanation generation methods operating on (combinations of) linguistic terms as well as a quantitative method suggesting precise changes in feature values. Then, we propose a new metric for assessing the perceived complexity of the generated explanations. Further, we design and carry out two human evaluation experiments to assess the explanatory power of the aforementioned methods. As a major result, we show that the estimated explanation complexity correlates well with the informativeness, relevance, and readability of explanations perceived by the targeted study participants. This fact opens the door to using the new automatic complexity metric for guiding multi-objective evolutionary explainable fuzzy modeling in the near future.

Abstract In this work, we present a complete system to produce an automatic linguistic reporting about the customer activity patterns inside open malls, a mixed distribution of classical malls joined with the shops on the street. These reports can assist to design marketing campaigns by means of identifying the best places to catch the attention of customers. Activity patterns are estimated with process mining techniques and the key information of localization. Localization is obtained with a parallelized solution based on WiFi fingerprint system to speed up the solution. In agreement with the best practices for human evaluation of natural language generation systems, the linguistic quality of the generated report was evaluated by 41 experts who filled in an online questionnaire. Results are encouraging, since the average global score of the linguistic quality dimension is 6.17 (0.76 of standard deviation) in a 7-point Likert scale. This expresses a high degree of satisfaction of the generated reports and validates the adequacy of automatic natural language textual reports as a complementary tool to process model visualization.

We introduce a novel framework to deal with fairness, accountability and explainability of intelligent systems. This framework puts together several tools to deal with bias at the level of data, algorithms and human cognition. The framework makes use of intelligent classifiers endowed with fuzzy-grounded linguistic explainability. As a result, it facilitates the exhaustive comparison of (white/grey/black)-box modelling techniques in combination with different strategies for handling missing values and unbalanced datasets. The proposal is evaluated on a realworld dataset in the context of banking services and reported results are encouraging.

Fairness, Accountability, Transparency and Explainability have become strong requirements in most practical applications of Artificial Intelligence (AI). Fuzzy sets and systems are recognized world-wide because of their outstanding contribution to model AI systems with a good interpretability-accuracy trade-off. Accordingly, fuzzy sets and systems are at the core of the so-called Explainable AI. ExpliClas is a software as a service which paves the way for interpretable and self-explainable intelligent systems. Namely, this software provides users with both graphical visualizations and textual explanations associated with intelligent classifiers automatically learned from data. This paper presents the new functionality of ExpliClas regarding the generation, evaluation and explanation of fuzzy decision trees along with fuzzy inference-grams. This new functionality is validated with two well-known classification datasets (i.e., Wine and Pima), but also with a real-world beer-style classifier.

Artificial Intelligence (AI) is a first class citizen in the cities of the 21st century. In addition, trust, fairness, accountability, transparency and ethical issues are considered as hot topics regarding AI-based systems under the umbrella of Explainable AI (XAI). In this paper we have conducted an experimental study with 15 datasets to validate the feasibility of using a pool of gray-box classifiers (i.e., decision trees and fuzzy rule-based classifiers) to automatically explain a black-box classifier (i.e., Random Forest). Reported results validate our approach. They confirm the complementarity and diversity among the gray-box classifiers under study, which are able to provide users with plausible multi-modal explanations of the considered black-box classifier for all given datasets.

Data-driven classification algorithms have proven highly effective in a range of complex tasks. However, their output is sometimes questioned, as the reasoning behind it may remain unclear due to a high number of poorly interpretable parameters used during training. Evidence-based (factual) explanations for single classifications answer the question why a particular class is selected in terms of the given observations. On the contrary, counterfactual explanations pay attention to why the rest of classes are not selected. Accordingly, we hypothesize that providing classifiers with a combination of both factual and counterfactual explanations is likely to make them more trustworthy. In order to investigate how such explanations can be produced, we introduce a new method to generate factual and counterfactual explanations for the output of pretrained decision trees and fuzzy rule-based classifiers. Experimental results show that unification of factual and counterfactual explanations under the paradigm of fuzzy inference systems proves promising for explaining the reasoning of classification algorithms.

JFML is an open source Java library aimed at facilitating interoperability of fuzzy systems by implementing the IEEE Std 1855-2016 - the IEEE Standard for Fuzzy Markup Language (FML) that is sponsored by the IEEE Computational Intelligence Society. We developed a Python wrapper for JFML that enables to use all the functionalities of JFML through a Python 3.x module. The bridge between Python and Java is accomplished through the use of the Py4J framework. As a result, the possibility of using the IEEE standard for representing fuzzy systems is enlarged to a wider community of developers and knowledge engineers, with minimal code redundancy. Experiments show full interoperability between Python programs and JFML without any tangible overhead. We illustrate the use of Py4JFML in a beer style classification case study.

ExpliClas is a web service aimed at providing users with multimodal (textual + graphical) explanations related to Weka classifiers. In ExpliClas, two types of explanations are automatically generated. On the one hand, global explanations pay attention to the behavior of the classifier as a whole, i.e., they refer to a list of structural properties (number of classes, features, etc.) along with quality indicators such as accuracy or confusion matrix. On the other hand, local explanations go in depth with how the classifier deals with single instances. Current version of ExpliClas already explains classifications made by three different decision tree Weka implementations (J48, RepTree, and RandomTree) and one fuzzy algorithm (FURIA). In this paper, we describe ExpliClas in detail and illustrate its use with the Bank telemarketing dataset.

We present a data resource which can be useful for research purposes on language grounding tasks in the context of geographical referring expression generation.The resource is composed of two data sets that encompass 25 different geographical descriptors and a set of associated graphical representations, drawn as polygons on a map by two groups of human subjects: teenage students and expert meteorologists.

This paper presents an empirical research. It focuses on testing empirically the benefits of providing users, in a specific domain, with textual interpretation of the fuzzy inferences carried out by a fuzzy classifier for a given selection of samples. The hypothesis to test is as follows: “Users understand easier the decision made by a fuzzy system when they are provided with a textual interpretation of the fuzzy inference mechanism that the system carried out”. This hypothesis was successfully tested in a web survey. The application domain was leaf classification. The fuzzy classifiers were built with the GUAJE fuzzy modeling open source software which is aimed at generating interpretable fuzzy systems. The textual interpretation was handmade by an expert who followed the guidelines of the Natural Language Generation approach proposed by Reiter and Dale. Reported results encourage us to go on with a series of additional experiments devoted to deeply explore how Natural Language Generation techniques can contribute to facilitate the understanding of fuzzy systems.

The generation of text reports from numerical and symbolic data is getting the attention of many researchers. This paper presents an R package useful to develop computational systems able to generate linguistic descriptions of complex phenomena. It generates text reports from numerical and symbolic data related to the phenomena under consideration. This is an implementation of our previous research work that is supported by the computational theory of perceptions grounded in the fuzzy sets theory. Developing open source software while we follow the key issues (novelty, usability, interoperability, and relevance) will facilitate the adoption of this new discipline in the research community and industry. We present illustrative examples that show how to use this new R package. The examples reveal that the package is ready to become a relevant tool in the research field of text generation from data.

The existing studies of the interpretability of fuzzy systems have mainly focused on the analysis of the relation between a model of the fuzzy system and a human user considered as a beneficiary, i.e., a domain expert or a designer. Suggested by the concepts of “model” of formalized theories, “realization” of formalized language, and “interpretability” of a theory in another, the main contribution of this paper is the proposal of a new approach to the fuzzy system interpretability. This is grounded on the qualitative real-world semantics of words and relationships between the semantics of fuzzy system components and substructures of their real-world counterparts. Thus, we introduce a novel real-world-semantics-based approach. It is aimed at characterizing the so-called real-world-semantics-based interpretability of fuzzy systems. In addition, it considers the actual semantics of all fuzzy system components, including the inference engine. Moreover, this new approach opens the door to a new way to study the interpretability of fuzzy systems.

We present a novel heuristic approach that defines fuzzy geographical descriptors using data gathered from a survey with human subjects. The participants were asked to provide graphical interpretations of the descriptors `north' and `south' for the Galician region (Spain). Based on these interpretations, our approach builds fuzzy descriptors that are able to compute membership degrees for geographical locations. We evaluated our approach in terms of efficiency and precision. The fuzzy descriptors are meant to be used as the cornerstones of a geographical referring expression generation algorithm that is able to linguistically characterize geographical locations and regions. This work is also part of a general research effort that intends to establish a methodology which reunites the empirical studies traditionally practiced in data-to-text and the use of fuzzy sets to model imprecision and vagueness in words and expressions for text generation purposes.

Monitoring and analysis of complex phenomena attract the attention of both academy and industry.Dealing with data produced by complex phenomena requires the use of advance computational intelligence techniques.Namely, linguistic description of complex phenomena constitutes a mature research line.It is supported by the Computational Theory of Perceptions grounded on the Fuzzy Sets Theory.Its aim is the development of computational systems with the ability to generate vague descriptions of the world in a similar way how humans do.This is a human-centric and multi-disciplinary research work.Moreover, its success is a matter of careful design; thus, developers play a key role.The rLDCP R package was designed to facilitate the development of new applications.This demo introduces the use of rLDCP, for both beginners and advance developers, in practical use cases.

The number of applications for smartphones and tablets is growing exponentially in the last years. Many of these applications are supported by the so-called Location Based Services, which are expected to provide reliable real-time localization anytime and anywhere, no matter either outdoors or indoors. Even though outdoors world-wide localization has been successfully developed through the well-known Global Navigation Satellite System technology, its counterpart large-scale deployment indoors is not available yet. In previous work, we have already introduced a novel technology for indoor localization supported by a WiFi fingerprint approach. In this paper, we describe how to enhance such approach through the combination of hierarchical localization and fuzzy classifier ensembles. It has been tested and validated at the University of Edinburgh, yielding promising results.

Fuzzy systems have been used widely thanks to their ability to successfully solve a wide range of problems in different application fields. However, their replication and application require a high level of knowledge and experience. Furthermore, few researchers publish the software and/or source code associated with their proposals, which is a major obstacle to scientific progress in other disciplines and in industry. In recent years, most fuzzy system software has been developed in order to facilitate the use of fuzzy systems. Some software is commercially distributed, but most software is available as free and open-source software, reducing such obstacles and providing many advantages: quicker detection of errors, innovative applications, faster adoption of fuzzy systems, etc. In this paper, we present an overview of freely available and open-source fuzzy systems software in order to provide a well-established framework that helps researchers to find existing proposals easily and to develop well-founded future work. To accomplish this, we propose a two-level taxonomy, and we describe the main contributions related to each field. Moreover, we provide a snapshot of the status of the publications in this field according to the ISI Web of Knowledge. Finally, some considerations regarding recent trends and potential research directions are presented.

This work presents the full integration of fuzzy inference-grams (Fingrams) in KEEL to visual analysis of fuzzy association rules. Fingrams graphically represent fuzzy rule-based systems (FRBSs) in 2D graphs that illustrate the interaction among fuzzy rules in terms of rule cofiring, i.e., paying attention to rule pairs simultaneous fired by a given input. The new module allows to generate Fingrams for fuzzy association rules created in the suite KEEL, that can be afterwards analyzed to comprehend the system behavior and improve it. We sketch the use and potentials in an illustrative example built in KEEL over a real-world dataset including qualitative assessments of a set of design chairs.

This paper presents an open source software tool for generation, visual representation and expert analysis of the so-called fuzzy inference-grams (fingrams in short). Fingrams are extended to fuzzy association rules in this work, expanding the scope of this methodology. The tool automatically generates fingrams and displays them to the user for expert analysis. It takes as input a simple configuration file that may be provided by whatever fuzzy modeling tool (it is actually provided by two widely known tools, GUAJE and KEEL). Fingrams graphically represent multi-dimensional fuzzy systems in 2D. The first version of this new software handles three different kinds of fuzzy systems (fuzzy rule-based systems for classification and regression, but also fuzzy association rules). Fingrams illustrate the interaction among fuzzy rules in terms of rule co-firing, i.e., paying attention to pairs of rules fired at the same time by a given input. Their visualization looks like a social network where nodes represent fuzzy rules while edges show the degree of interaction among pairs of rules. Thus, both representation and analysis are supported by social network analysis techniques. The utility of the new software tool is illustrated in a case study on visual representation and analysis of association rules.

Since Zadeh’s proposal and Mamdani’s seminal ideas, interpretability is acknowledged as one of the most appreciated and valuable characteristics of fuzzy system identification methodologies. It represents the ability of fuzzy systems to formalize the behavior of a real system in a human understandable way, by means of a set of linguistic variables and rules with a high semantic expressivity close to natural language. Interpretability analysis involves two main points of view: readability of the knowledge base description (regarding complexity of fuzzy partitions and rules) and comprehensibility of the fuzzy system (regarding implicit and explicit semantics embedded in fuzzy partitions and rules, as well as the fuzzy reasoning method). Readability has been thoroughly treated by many authors who have proposed several criteria and metrics. Unfortunately, comprehensibility has usually been neglected because it involves some cognitive aspects related to human reasoning, which are very hard to formalize and to deal with. This paper proposes the creation of a new paradigm for fuzzy system comprehensibility analysis based on fuzzy systems’ inference maps, so-called fuzzy inference-grams (fingrams), by analogy with scientograms used for visualizing the structure of science. Fingrams show graphically the interaction between rules at the inference level in terms of co-fired rules, i.e., rules fired at the same time by a given input. The analysis of fingrams offers many possibilities: measuring the comprehensibility of fuzzy systems, detecting redundancies and/or inconsistencies among fuzzy rules, identifying the most significant rules, etc. Some of these capabilities are explored in this study for the case of fuzzy models and classifiers.

The so-called fuzzy inference-grams (fingrams) constitute a novel and powerful tool for expert analysis of fuzzy rule bases at inference level in terms of co-fired rules, i.e., rules fired at the same time by a given input vector. A fuzzy rule base can be seen as a population made up of a set of individual rules which are competing and collaborating among them with the aim of yielding both good generality-specificity and interpretability-accuracy trade-offs. Thus, system behavior turns up from the existing relations among rules which can be easily analyzed by looking at the rule base as a fingram, i.e., as a social network made of nodes representing fuzzy rules. This paper introduces a new software module for fingram generation and analysis which is provided with the last version of GUAJE, a free software tool devoted to the generation of understandable and accurate fuzzy systems. Notice that, fingram analysis consists of studying the interaction among nodes in the network for the purpose of understanding the structure and behavior of the fuzzy rule base under consideration. It is based on the basic principles of social network analysis which have been properly incorporated into GUAJE and adapted to the design of fuzzy systems.

This paper introduces a new system for people localization in indoor environments. It is mainly based on an intelligent classifier able to distinguish among a set of predefined and well identified topological locations. We adopt a WiFi signal strength fingerprint approach where most effort is done during the training phase, while online execution is fast and effective. Our proposal has been tested in a real environment with data collected in five different experimental sessions. Achieved results are encouraging since they overcome those ones provided by the well-known nearest neighbour fingerprint matching algorithm, that is usually considered as baseline for WiFi localization.

Assessing interpretability of fuzzy systems still remains an open and challenging problem. Defining a good index is extremely difficult mainly due to the inherent subjective nature of interpretability. It strongly depends on the background of the person who makes the assessment according to its own knowledge, but also taking into account its previous experience and preferences. Since looking for fuzzy systems with a good accuracy-interpretability trade-off is required for many applications, guiding the whole design process by a good quality index would be extremely appreciated. Such index must be aware of both accuracy and interpretability. This paper introduces a framework that makes possible defining an index easily adaptable to the context of each problem by means of incorporating user's preferences and quality criteria. To do so, all aspects related to interpretability are first identified and then combined into a decision hierarchy framework. It is derived from a previous experimental study based on a web poll. The top of the hierarchy represents the quality index while the bottom includes all fuzzy systems to be evaluated. It consists of k decision levels structured as suggested by the classical analytic hierarchy process (AHP) defined by Saaty. In addition, the aggregation process is made using the ordered weighted averaging (OWA) operators defined by Yager. Such AHP+OWA combination was already proposed by Yager for solving multi-criteria decision problems. A simple example shows how the proposed method becomes effective but also efficient when assessing several fuzzy systems in an automatic process. The index is easily adaptable for providing those rankings expected by different users.

This work presents a general framework for people indoor activity recognition. Firstly, a Wireless Fidelity (WiFi) localization system implemented as a Fuzzy Rule-based Classifier (FRBC) is used to obtain an approximate position at the level of discrete zones (office, corridor, meeting room, etc). Secondly, a Fuzzy Finite State Machine (FFSM) is used for human body posture recognition (seated, standing upright or walking). Finally, another FFSM combines both WiFi localization and posture recognition to obtain a robust, reliable, and easily understandable activity recognition system (working in the desk room, crossing the corridor, having a meeting, etc). Each user carries with a personal digital agenda (PDA) or smart-phone equipped with a WiFi interface for localization task and accelerometers for posture recognition. Our approach does not require adding new hardware to the experimental environment. It relies on the WiFi access points (APs) widely available in most public and private buildings. We include a practical experimentation where good results were achieved.

The goal of this paper is to study some of the most important WiFi signal variations, large and small scale variations and how they affect to WiFi localization systems. Moreover, the paper shows how to use soft computing techniques to deal with these uncertainties in WiFi localization systems. This work describes how to reduce uncertainty produced by small scale variations in indoor environments using fuzzy techniques. Some experimental results and conclusions are presented.

This work presents a new process for building comprehensible fuzzy systems for classification problems. Firstly, a feature selection procedure based on crisp decision trees is carried out. Secondly, strong fuzzy partitions are generated for all the selected inputs. Thirdly, a set of linguistic rules are defined combining the previously generated linguistic variables. Then, a linguistic simplification procedure guided by a novel interpretability index is applied to get a more compact and general set of rules without losing accuracy. Finally, an efficient and simple local search strategy increases the system accuracy while preserving the high interpretability. Results obtained in several benchmark classification problems are encouraging because they show the ability of the new methodology for generating highly interpretable fuzzy rule-based classifiers while yielding accuracy comparable to that achieved by other methods like neural networks and C4.5.

This work describes a new methodology for making easier the design process of interpretable knowledge bases. It considers both expert knowledge and knowledge extracted from data. The combination of both kinds of knowledge is likely to yield robust compact systems with a good trade-off between accuracy and interpretability. Fuzzy logic offers an integration framework where both types of knowledge are represented using the same formalism. However, as two knowledge bases may convey contradictions and/or redundancies, the integration process must be made carefully. Results obtained, in four well-known benchmark classification problems, show that our methodology leads to highly interpretable knowledge bases with a good accuracy, comparable to that achieved by other methodologies. © 2008 Wiley Periodicals, Inc.

This work shows how to achieve a good interpretability-accuracy trade-off through keeping the strong fuzzy partition property along the whole fuzzy modeling process. First, a small compact knowledge base is built. It is highly interpretable and reasonably accurate. Second, an optimization procedure, which only affects the fuzzy partitions defining the system variables, is carried out. It improves the system accuracy while preserving the system interpretability. Two optimization strategies are compared: Solis-Wetts, a local search based strategy; and Genetic Tuning, a global search based strategy. Results obtained in a well-known benchmark medical classification problem, related to breast cancer diagnosis, show that our methodology is able to achieve knowledge bases with high interpretability and accuracy comparable to that obtained by other methodologies.

This paper presents a user-friendly portable tool designed and developed in order to make easier knowledge extraction and representation for fuzzy logic based systems. KBCT is an open source software that could be executed under Linux or Windows operating systems. Main goal of KBCT is the generation or refinement of fuzzy knowledge bases with a particular interest of obtaining interpretable partitions and rules. The use of fuzzy logic simplifies the knowledge extraction process and increase interpretability of rules because of the fuzzy rule expression is closed to expert natural language. KBCT lets the user define expert variables and rules, but also provide induction capabilities for partitions and rules. Both types of knowledge, expert and induced, are integrated under the expert control. In addition to this, the user can check consistency and quality of rule base at any moment. A simplify option is implemented in order to allow the user to reduce the size of rule base. The main objective consists of ensuring interpretability, non-redundancy and consistency of the knowledge base along the whole process.