In this paper, we present a novel diffusion model-based monaural speech enhancement method. Our approach incorporates the separate estimation of speech spectra's magnitude and phase in two diffusion networks. Throughout the diffusion process, noise clips from real-world noise interferences are added gradually to the clean speech spectra and a noise-aware reverse process is proposed to learn how to generate both clean speech spectra and noise spectra. Furthermore, to fully leverage the intrinsic relationship between magnitude and phase, we introduce a complex-cycle-consistent (CCC) mechanism that uses the estimated magnitude to map the phase, and vice versa. We implement this algorithm within a phase-aware speech enhancement diffusion model (SEDM). We conduct extensive experiments on public datasets to demonstrate the effectiveness of our method, highlighting the significant benefits of exploiting the intrinsic relationship between phase and magnitude information to enhance speech. The comparison to conventional diffusion models demonstrates the superiority of SEDM.

Deepfake videos are synthetic media generated using advanced deep learning techniques that manipulate or replace the visual and audio content of an original recording, enabling the creation of highly realistic yet entirely fabricated audiovisual content. The proliferation of such manipulated media poses significant societal risks, including potential misinformation, reputation damage, psychological manipulation, and erosion of trust in digital visual communication. Recent deep learning methods for deepfake detection have emerged, leveraging sophisticated machine learning models that analyze multi-modal cues, including facial inconsistencies, unnatural temporal dynamics, and visual misalignments to distinguish between authentic and synthetic content. However, these state-of-the-art detection approaches often struggle with the domain-shift challenge, where models trained on specific deepfake datasets fail to generalize effectively when confronted with unseen generation techniques or evolving synthesis technologies. To address this critical limitation, we propose a self-supervised contrastive learning framework called CPDD, introducing contrast between features and prototypes of original data to alleviate domain-specific distractions (i.e., deepfake generative models or datasets). We calculate the cosine similarity between two features or prototypes to scale the original distance, clustering the features around closely related prototypes. This process encodes the semantic structures discovered through clustering into the learned embedding space. The extensive experiments show that, compared to various benchmark deepfake detection models and domain generalization techniques, the proposed model achieves state-of-the-art performance on the cross-domain deepfake detection task across a wide range of scenarios.

Fixed-wing aerial vehicles are equipped with functionalities such as ILS (instrument landing system), PAR (precision approach radar) and, DGPS (differential global positioning system), enabling fully automated landings. However, these systems impose significant costs on airport operations due to high installation and maintenance requirements. Moreover, since these navigation parameters come from ground or satellite signals, they are vulnerable to interference. A more cost-effective and independent alternative for guiding landing is a vision-based system that detects the runway and aligns the aircraft, reducing the pilot’s cognitive load. This paper proposes a novel framework that addresses three key challenges in developing autonomous vision-based landing systems. Firstly, to overcome the lack of aerial front-view video data, we created high-quality videos simulating landing approaches through the generator code available in the LARD (landing approach runway detection dataset) repository. Secondly, in contrast to former studies focusing on object detection for finding the runway, we chose the state-of-the-art model LoRAT to track runways within bounding boxes in each video frame. Thirdly, to align the aircraft with the designated landing runway, we extract runway keypoints from the resulting LoRAT frames and estimate the camera relative pose via the Perspective-n-Point algorithm. Our experimental results over a dataset of generated videos and original images from the LARD dataset consistently demonstrate the proposed framework’s highly accurate tracking and alignment capabilities. Our approach source code and the LoRAT model pre-trained with LARD videos are available at https:// github.com/ jpklock2/ visionbased-landing-guidance

The advance of foundation models (FM) makes it possible to avoid parametric tuning for transfer learning , taking advantage of pretrained feature spaces . In this study, we define a framework called IDEAL (Interpretable-by-design DEep learning ALgorithms) which tackles the problem of interpretable transfer learning by recasting the standard supervised classification problem into a function of similarity to a set of prototypes derived from the training data . This framework generalises previously-known prototypical approaches, such as ProtoPNet, xDNN and DNC, and decomposes the overall problem into two inherently connected stages: (A) feature extraction (FE), which maps the raw features of real-world data into a latent space, and (B) identification of representative prototypes and decision making based on similarity and association between the query and the prototypes. This addresses the issue of interpretability (stage B) while retaining the benefits of pretrained deep learning (DL) models. On a range of datasets (CIFAR-10, CIFAR-100, CalTech101, STL-10, Oxford-IIIT Pet, EuroSAT), we demonstrate, through an extensive set of experiments, how the choice of the latent space, prototype selection, and finetuning of the latent space affect accuracy and generalisation of the models on transfer learning scenarios for different backbones. Building upon this knowledge, we demonstrate that the proposed framework helps achieve an advantage over state-of-the-art baselines in class-incremental learning. The key findings can be summarised as follows: (1) the setting allows interpretability through prototypes, (2) lack of finetuning helps circumvent the issue of catastrophic forgetting, allowing efficient class-incremental transfer learning, while mitigating the issue of confounding bias, and (3) ViT architectures narrow the gap between finetuned and non-finetuned models allowing for transfer learning in a fraction of time without finetuning of the feature space on a target dataset with iterative supervised methods.

Machine learning, particularly through advanced imaging techniques such as three-dimensional Magnetic Resonance Imaging (MRI), has significantly improved medical diagnostics. This is especially critical for diagnosing complex conditions like Alzheimer's disease. Our study introduces Triamese-ViT, an innovative Tri-structure of Vision Transformers (ViTs) that incorporates a built-in interpretability function, it has structure-aware explainability that allows for the identification and visualization of key features or regions contributing to the prediction, integrates information from three perspectives to enhance brain age estimation. This method not only increases accuracy but also improves interoperability with existing techniques. When evaluated, Triamese-ViT demonstrated superior performance and produced insightful attention maps. We applied these attention maps to the analysis of natural aging and the diagnosis of Autism Spectrum Disorder (ASD). The results aligned with those from occlusion analysis, identifying the Cingulum, Rolandic Operculum, Thalamus, and Vermis as important regions in normal aging, and highlighting the Thalamus and Caudate Nucleus as key regions for ASD diagnosis.

This paper presents an unsupervised and model-independent concept drift detector based on quadtree spatial analysis (QTS).We used a d-dimensional quadtree to map the feature space and tracked a univariate curve that mimics the spatial behavior of the data stream.This curve serves as a helpful visual tool for analyzing concept drifts.Drifts are identified when there is a significant change in the current spatial mapping.Experimental results show that the proposed outperformed well-known drift detectors in terms of average precision and F1-score.

Adversarial attacks can cause misclassification in machine learning pipelines, posing a significant safety risk in critical applications such as autonomous systems or medical applications. Supervised learning-based methods for adversarial sample recovery rely heavily on large volumes of labeled data, which often results in substantial performance degradation when applying the trained model to new domains. In this paper, differing from conventional self-supervised learning techniques such as data augmentation, we present a novel two-stage self-supervised representation learning framework for the task of adversarial sample recovery, aimed at overcoming these limitations. In the first stage, we employ a clean image autoencoder (CAE) to learn representations of clean images. Subsequently, the second stage utilizes an adversarial image autoencoder (AAE) to learn a shared latent space that captures the relationships between the representations acquired by CAE and AAE. It is noteworthy that the input clean images in the first stage and adversarial images in the second stage are cross-domain and not paired. To the best of our knowledge, this marks the first instance of self-supervised adversarial sample recovery work that operates without the need for labeled data. Our experimental evaluations, spanning a diverse range of images, consistently demonstrate the superior performance of the proposed method compared to conventional adversarial sample recovery methods.

As the use of Deep Neural Networks (DNNs) becomes pervasive, their vulnerability to adversarial attacks and limitations in handling unseen classes poses significant challenges. The state-of-the-art offers discrete solutions aimed to tackle individual issues covering specific adversarial attack scenarios, classification or evolving learning. However, real-world systems need to be able to detect and recover from a wide range of adversarial attacks without sacrificing classification accuracy and to flexibly act in unseen scenarios. In this paper, UNICAD, is proposed as a novel framework that integrates a variety of techniques to provide an adaptive solution.For the targeted image classification, UNICAD achieves accurate image classification, detects unseen classes, and recovers from adversarial attacks using Prototype and Similarity-based DNNs with denoising autoencoders. Our experiments performed on the CIFAR-10 dataset highlight UNICAD’s effectiveness in adversarial mitigation and unseen class classification, outperforming traditional models.

Foundation Models (FM) developed on very large generic data sets transformed the landscape of machine learning (ML). Vision transformers (ViT) closed the gap in performance between fine-tuned and unsupervised transfer learning. This opens the possibility to abandon the widely used until recently end-to-end approach. Instead, we consider a two-stage ML pipeline, where the first stage constitutes extracting features by pre-training large, multi-layer model with billions of parameters, and the second stage is a computationally lightweight learning of an entirely new, simpler model architecture based on prototypes within this feature space. In this paper we consider such two-stage approach to ML. We further analyse the use of several alternative light-weight methods in the second stage, including strategies for semi-supervised learning and a variety of strategies for linear fine-tuning. We demonstrate on the basis of nine well known benchmark data sets that the ultra-light-weight ML alternatives for the second stage (such as clustering, PCA, LDA and combinations of these) offer for the price of negligible drop in accuracy a significant (several orders of magnitude) drop of computational costs (time, energy and related CO2emissions) as well as the ability to use no labels (fully unsupervised approach) or limited amount of labels (one per cluster labels) and the ability to address interpretability.

Fuzzy systems offer a formal and practically popular methodology for modeling nonlinear problems with inherent uncertainties, entailing strong performance and model interpretability. Particularly, semisupervised boosting is widely recognized as a powerful approach for creating stronger ensemble classification models in the absence of sufficient labeled data without introducing any modification to the employed base classifiers. However, the potential of fuzzy systems in semisupervised boosting has not been systematically explored yet. In this study, a novel semisupervised boosting algorithm devised for zero-order evolving fuzzy systems is proposed. It ensures both the consistence among predictions made by individual base classifiers at successive boosting iterations and the respective levels of confidence toward their predictions throughout the process of sample weight updating and ensemble output generation. In so doing, the base classifiers are empowered to gradually focus more on challenging samples that are otherwise hard to generalize, enabling the development of more precise integrated classification boundaries. Numerical evaluations on a range of benchmark problems are carried out, demonstrating the efficacy of the proposed semisupervised boosting algorithm for constructing ensemble fuzzy classifiers with high accuracy.

Image processing using neural networks act as a tool to speed up predictions for users, specifically on large-scale image samples. To guarantee the clean data for training accuracy, various deep learning-based adversarial attack detection techniques have been proposed. These crisp set-based detection methods directly determine whether an image is clean or attacked, while, calculating the loss is nondifferentiable and hinders training through normal back-propagation. Motivated by the recent success in fuzzy systems, in this work, we present an attack detection method to further improve detection performance, which is suitable for any pretrained neural network classifier. Subsequently, the fuzzification network is used to obtain feature maps to produce fuzzy sets of difference degree between clean and attacked images. The fuzzy rules control the intelligence that determines the detection boundaries. Different from previous fuzzy systems, we propose a fuzzy mean-intelligence mechanism with new support and confidence functions to improve fuzzy rule's quality. In the defuzzification layer, the fuzzy prediction from the intelligence is mapped back into the crisp model predictions for images. The loss between the prediction and label controls the rules to train the fuzzy detector. We show that the fuzzy rule-based network learns rich feature information than binary outputs and offer to obtain an overall performance gain. Experiment results show that compared to various benchmark fuzzy systems and adversarial attack detection methods, our fuzzy detector achieves better detection performance over a wide range of images.

Deep learning-based imperceptible adversarial attack detection methods have recently seen significant progress. However, the accuracy, latency, and computational cost of previous methods remain insufficient. Particularly, trained attack detection models can potentially be applied in previously unseen conditions, such as new datasets or attacks for real-world applications. Therefore, to improve domain generalization performance, we propose a new method for cross-domain imperceptible adversarial attack detection by leveraging domain generalization, where we train the model's feature extractor or detector with a partner well-tuned for different domains. Different from conventional domain generalization methods, we use the global loss and local loss to train each feature extractor or detector. Moreover, to efficiently re-use high-resolution feature maps from the feature extractor, we propose a feature fusion network, which exploits feature maps from images that are attacked with different error rates and helps extract rich features to further improve the attack detection accuracy. Extensive experiments on four public datasets are used to demonstrate the efficacy of the proposed method. The source code of the proposed method is available at https://github.com/Yukino-3/DTAD.

It is widely recognized that learning systems have to go deeper to exchange for more powerful representational learning capabilities in order to precisely approximate nonlinear complex problems. However, the best-known computational intelligence approaches with such characteristics, namely, deep neural networks, are often criticized for lacking transparency. In this article, a novel multilayer evolving fuzzy neural network (MEFNN) with a transparent system structure is proposed. The proposed MEFNN is a metalevel stacking ensemble learning system composed of multiple cascading evolving neuro-fuzzy inference systems (ENFISs), processing input data layer-by-layer to automatically learn multilevel nonlinear distributed representations from data. Each ENFIS is an evolving fuzzy system capable of learning from new data sample by sample to self-organize a set of human-interpretable IF– THEN fuzzy rules that facilitate approximate reasoning. Adopting ENFIS as its ensemble component, the multilayer system structure of the MEFNN is flexible and transparent, and its internal reasoning and decision-making mechanism can be explained and interpreted to/by humans. To facilitate information exchange between different layers and attain stronger representation learning capability, the MEFNN utilizes error backpropagation to self-update the consequent parameters of the IF–THEN rules of each ensemble component based on the approximation error propagated backward. To enhance the capability of the MEFNN to handle complex problems, a nonlinear activation function is introduced to modeling the consequent parts of the IF–THEN rules of ENFISs, thereby empowering both the representation and the reflection of nonlinearity in the resulting fuzzy outputs. Numerical examples on a wide variety of challenging (benchmark and real-world) classification and regression problems demonstrate the superior practical performance of the MEFNN, revealing the effectiveness and validity of the proposed approach.

Deep learning for graph matching has received growing interest and developed rapidly in the past decade. Although recent deep graph matching methods have shown excellent performance on matching between graphs of equal size in the computer vision area, the size-varied graph matching problem, where the number of keypoints in the images of the same category may vary due to occlusion, is still an open and challenging problem. To tackle this, we firstly propose to formulate the combinatorial problem of graph matching as an Integer Linear Programming (ILP) problem, which is more flexible and efficient to facilitate comparing graphs of varied sizes. A novel Graph-context Attention Network (GCAN), which jointly capture intrinsic graph structure and cross-graph information for improving the discrimination of node features, is then proposed and trained to resolve this ILP problem with node correspondence supervision. We further show that the proposed GCAN model is efficient to resolve the graph-level matching problem and is able to automatically learn node-to-node similarity via graph-level matching. The proposed approach is evaluated on three public keypoint-matching datasets and one graph-matching dataset for blood vessel patterns, with experimental results showing its superior performance over existing state-of-the-art algorithms for keypoint and graph-level matching.

In this paper, we propose a novel hand-based person recognition method for the purpose of criminal investigations since the hand image is often the only available information in cases of serious crime such as sexual abuse. Our proposed method, Multi-Branch with Attention Network (MBA-Net), incorporates both channel and spatial attention modules in branches in addition to a global (without attention) branch to capture global structural information for discriminative feature learning. The attention modules focus on the relevant features of the hand image while suppressing the irrelevant backgrounds. In order to overcome the weakness of the attention mechanisms, equivariant to pixel shuffling, we integrate relative positional encodings into the spatial attention module to capture the spatial positions of pixels. Extensive evaluations on two large multi-ethnic and publicly available hand datasets demonstrate that our proposed method achieves state-of-the-art performance, surpassing the existing hand-based identification methods. The source code is available at https://github.com/nathanlem1/MBA-Net.

Earth observation is fundamental for a range of human activities including flood response as it offers vital information to decision makers. Semantic segmentation plays a key role in mapping the raw hyper-spectral data coming from the satellites into a human understandable form assigning class labels to each pixel. Traditionally, water index based methods have been used for detecting water pixels. More recently, deep learning techniques such as U-Net started to gain attention offering significantly higher accuracy. However, the latter are hard to interpret by humans and use dozens of millions of abstract parameters that are not directly related to the physical nature of the problem being modelled. They are also labelled data and computational power hungry. At the same time, data transmission capability on small nanosatellites is limited in terms of power and bandwidth yet constellations of such small, nanosatellites are preferable, because they reduce the revisit time in disaster areas from days to hours. Therefore, being able to achieve as highly accurate models as deep learning (e.g. U-Net) or even more, to surpass them in terms of accuracy, but without the need to rely on huge amounts of labelled training data, computational power, abstract coefficients offers potentially game-changing capabilities for EO (Earth observation) and flood detection, in particular. In this paper, we introduce a prototype-based interpretable deep semantic segmentation (IDSS) method, which is highly accurate as well as interpretable. Its parameters are in orders of magnitude less than the number of parameters used by deep networks such as U-Net and are clearly interpretable by humans. The proposed here IDSS offers a transparent structure that allows users to inspect and audit the algorithm’s decision. Results have demonstrated that IDSS could surpass other algorithms, including U-Net, in terms of IoU (Intersection over Union) total water and Recall total water. We used WorldFloods data set for our experiments and plan to use the semantic segmentation results combined with masks for permanent water to detect flood events.

With the advancement in Machine Learning (ML) techniques, a wide range of applications that leverage ML have emerged across research, industry, and society to improve application performance. However, existing ML schemes used within such applications struggle to attain high model accuracy due to the heterogeneous and distributed nature of their generated data, resulting in reduced model performance. In this paper we address this challenge by proposing PPFM: an adaptive and hierarchical Peer-to-Peer Federated Meta-learning framework. Instead of leveraging a conventional static ML scheme, PPFM uses multiple learning loops to dynamically self-adapt its own architecture to improve its training effectiveness for different generated data characteristics. Such an approach also allows for PPFM to remove reliance on a fixed centralized server in a distributed environment by utilizing peer-to-peer Federated Learning (FL) framework. Our results demonstrate PPFM provides significant improvement to model accuracy across multiple datasets when compared to contemporary ML approaches.

Abstract It is essential for agents to work together with others to accomplish common objectives, without pre-programmed coordination rules or previous knowledge of the current teammates, a challenge known as ad-hoc teamwork. In these systems, an agent estimates the algorithm of others in an on-line manner in order to decide its own actions for effective teamwork. A common approach is to assume a set of possible types and parameters for teammates, reducing the problem into estimating parameters and calculating distributions over types. Meanwhile, agents often must coordinate in a decentralised fashion to complete tasks that are displaced in an environment (e.g., in foraging, de-mining, rescue or fire control), where each member autonomously chooses which task to perform. By harnessing this knowledge, better estimation techniques can be developed. Hence, we present On-line Estimators for Ad-hoc Task Execution (OEATE), a novel algorithm for teammates’ type and parameter estimation in decentralised task execution. We show theoretically that our algorithm can converge to perfect estimations, under some assumptions, as the number of tasks increases. Additionally, we run experiments for a diverse configuration set in the level-based foraging domain over full and partial observability, and in a “capture the prey” game. We obtain a lower error in parameter and type estimation than previous approaches and better performance in the number of completed tasks for some cases. In fact, we evaluate a variety of scenarios via the increasing number of agents, scenario sizes, number of items, and number of types, showing that we can overcome previous works in most cases considering the estimation process, besides robustness to an increasing number of types and even to an erroneous set of potential types.

In this paper, an approach that makes use of knuckle creases and fingernails for person identification is presented. It introduces a framework for automatic person identification that includes localisation of the region of interest (ROI) of many components within hand images, recognition and segmentation of the detected components using bounding boxes, and similarity matching between two different sets of segmented images. The following hand components are considered: i) the metacarpophalangeal (MCP) joint, commonly known as the base knuckle; ii) the proximal interphalangeal (PIP) joint, commonly known as the major knuckle; iii) the distal interphalangeal (DIP) joint, commonly known as the minor knuckle; iv) the interphalangeal (IP) joint, commonly known as the thumb knuckle, and v) the fingernails. Crucial elements of the proposed framework are the feature extraction and similarity matching. This paper exploits different deep learning neural networks (DLNNs), which are essential in extracting discriminative high-level abstract features. We further use various similarity measures for the matching process. We validate the proposed approach on well-known benchmarks, including the 11k Hands dataset and the Hong Kong Polytechnic University Contactless Hand Dorsal Images known as PolyU. The results indicate that knuckle patterns and fingernails play a significant role in the person identification framework. The 11K Hands dataset results indicate that the left-hand results are better than the right-hand results and the fingernails produce consistently higher identification results than other hand components, with a rank-1 score of 100%. In addition, the PolyU dataset attains 100% in the fingernail of the thumb finger.

Large-scale (large-area), fine spatial resolution satellite sensor images are valuable data sources for Earth observation while not yet fully exploited by research communities for practical applications. Often, such images exhibit highly complex geometrical structures and spatial patterns, and distinctive characteristics of multiple land-use categories may appear at the same region. Autonomous information extraction from these images is essential in the field of pattern recognition within remote sensing, but this task is extremely challenging due to the spectral and spatial complexity captured in satellite sensor imagery. In this research, a semi-supervised deep rule-based approach for satellite sensor image analysis (SeRBIA) is proposed, where large-scale satellite sensor images are analysed autonomously and classified into detailed land-use categories. Using an ensemble feature descriptor derived from pre-trained AlexNet and VGG-VD-16 models, SeRBIA is capable of learning continuously from both labelled and unlabelled images through self-adaptation without human involvement or intervention. Extensive numerical experiments were conducted on both benchmark datasets and real-world satellite sensor images to comprehensively test the validity and effectiveness of the proposed method. The novel information mining technique developed here can be applied to analyse large-scale satellite sensor images with high accuracy and interpretability, across a wide range of real-world applications.

Adaptive boosting (AdaBoost) is a widely used technique to construct a stronger ensemble classifier by combining a set of weaker ones. Zero-order fuzzy inference systems (FISs) are very powerful prototype-based predictive models for classification, offering both great prediction precision and high user interpretability. However, the use of zero-order FISs as base classifiers in AdaBoost has not been explored yet. To bridge the gap, in this article, a novel multiclass fuzzily weighted AdaBoost (FWAdaBoost)-based ensemble system with a self-organizing fuzzy inference system (SOFIS) as the ensemble component is proposed. To better incorporate the SOFIS, FWAdaBoost utilizes the confidence scores produced by the SOFIS in both sample weight updating and ensemble output generation, resulting in more accurate classification boundaries and greater prediction precision. Numerical examples on a wide range of benchmark classification problems demonstrate the efficacy of the proposed approach.

Evolving fuzzy systems (EFSs) are widely known as a powerful tool for streaming data prediction. In this article, a novel zero-order EFS with a unique belief structure is proposed for data stream classification. Thanks to this new belief structure, the proposed model can handle the interclass overlaps in a natural way and better capture the underlying multimodel structure of data streams in the form of prototypes. Utilizing data-driven soft thresholds, the proposed model self-organizes a set of prototype-basedif–thenfuzzy belief rules from data streams for classification, and its learning outcomes are practically meaningful. With no requirement of prior knowledge in the problem domain, the proposed model is capable of self-determining the appropriate level of granularity for rule-based construction, while enabling users to specify their preferences on the degree of fineness of its knowledge base. Numerical examples demonstrate the superior performance of the proposed model on a wide range of stationary and nonstationary classification benchmark problems.

Non-Gaussian noises always exist in the nonlinear system, which usually lead to inconsistency and divergence of the regression and identification applications. The conventional evolving fuzzy systems (EFSs) in common sense have succeeded to conquer the uncertainties and external disturbance employing the specific variable structure characteristic. However, non-Gaussian noises would trigger the frequent changes of structure under the transient criteria, which severely degrades performance. Statistical criterion provides an informed choice of the strategies of the structure evolution, utilizing the approximation uncertainty as the observation of model sufficiency. The approximation uncertainty can be always decomposed into model uncertainty term and noise term, and is suitable for the non-Gaussian noise condition, especially relaxing the traditional Gaussian assumption. In this article, a novel incremental statistical evolving fuzzy inference system (SEFIS) is proposed, which has the capacity of updating the system parameters, and evolving the structure components to integrate new knowledge in the new process characteristic, system behavior, and operating conditions with non-Gaussian noises. The system generates a new rule based on the statistical model sufficiency which gives so insight into whether models are reliable and their approximations can be trusted. The nearest rule presents the inactive rule under the current data stream and further would be deleted without losing any information and accuracy of the subsequent trained models when the model sufficiency is satisfied. In our article, an adaptive maximum correntropy extend Kalman filter is derived to update the parameters of the evolving rules to cope with the non-Gaussian noises problems to further improve the robustness of parameter updating process. The parameter updating process shares an estimate of the uncertainty with the criteria of the structure evolving process to make the computation less of a burden dramatically. The simulation studies show that the proposed SEFIS has faster learning speed and is more accurate than the existing evolving fuzzy systems (EFSs) in the case of noise free and noisy conditions.

In many cases of serious crime, images of a hand can be the only evidence available for the forensic identification of the offender. As well as placing them at the scene, such images and video evidence offer proof of the offender committing the crime. The knuckle creases of the human hand have emerged as an effective biometric trait and been used to identify the perpetrators of child abuse in forensic investigations. However, manual utilization of knuckle creases for identification is highly time consuming and can be subjective, requiring the expertise of experienced forensic anthropologists whose availability is very limited. Hence, there arises a need for an automated approach for localization and comparison of knuckle patterns. In this paper, we present a fully automatic end-to-end approach which localizes the minor, major and base knuckles in images of the hand, and effectively uses them for identification achieving state-of-the-art results. This work improves on existing approaches and allows us to strengthen cases further by objectively combining multiple knuckles and knuckle types to obtain a holistic matching result for comparing two hands. This yields a stronger and more robust multi-unit biometric and facilitates the large-scale examination of the potential of knuckle-based identification. Evaluated on two large landmark datasets, the proposed framework achieves equal error rates (EER) of 1.0-1.9%, rank-1 accuracies of 99.3-100% and decidability indices of 5.04-5.83. We make the full results available via a novel online GUI to raise awareness with the general public and forensic investigators about the identifiability of various knuckle regions. These strong results demonstrate the value of our holistic approach to hand identification from knuckle patterns and their utility in forensic investigations.

As COVID-19 hounds the world, the common cause of finding a swift solution to manage the pandemic has brought together researchers, institutions, governments, and society at large. The Internet of Things (IoT), artificial intelligence (AI)-including machine learning (ML) and Big Data analytics-as well as Robotics and Blockchain, are the four decisive areas of technological innovation that have been ingenuity harnessed to fight this pandemic and future ones. While these highly interrelated smart and connected health technologies cannot resolve the pandemic overnight and may not be the only answer to the crisis, they can provide greater insight into the disease and support frontline efforts to prevent and control the pandemic. This article provides a blend of discussions on the contribution of these digital technologies, propose several complementary and multidisciplinary techniques to combat COVID-19, offer opportunities for more holistic studies, and accelerate knowledge acquisition and scientific discoveries in pandemic research. First, four areas, where IoT can contribute are discussed, namely: 1) tracking and tracing; 2) remote patient monitoring (RPM) by wearable IoT (WIoT); 3) personal digital twins (PDTs); and 4) real-life use case: ICT/IoT solution in South Korea. Second, the role and novel applications of AI are explained, namely: 1) diagnosis and prognosis; 2) risk prediction; 3) vaccine and drug development; 4) research data set; 5) early warnings and alerts; 6) social control and fake news detection; and 7) communication and chatbot. Third, the main uses of robotics and drone technology are analyzed, including: 1) crowd surveillance; 2) public announcements; 3) screening and diagnosis; and 4) essential supply delivery. Finally, we discuss how distributed ledger technologies (DLTs), of which blockchain is a common example, can be combined with other technologies for tackling COVID-19.

Abstract In this paper, we break with the traditional approach to classification, which is regarded as a form of supervised learning. We offer a method and algorithm, which make possible fully autonomous (unsupervised) detection of new classes, and learning following a very parsimonious training priming (few labeled data samples only). Moreover, new unknown classes may appear at a later stage and the proposed xClass method and algorithm are able to successfully discover this and learn from the data autonomously. Furthermore, the features (inputs to the classifier) are automatically sub-selected by the algorithm based on the accumulated data density per feature per class. In addition, the automatically generated model is easy to interpret and is locally generative and based on prototypes which define the modes of the data distribution. As a result, a highly efficient, lean, human-understandable, autonomously self-learning model (which only needs an extremely parsimonious priming) emerges from the data. To validate our proposal, we approbated it on four challenging problems, including imbalanced Faces-1999 data base, Caltech-101 dataset, vehicles dataset, and iRoads dataset, which is a dataset of images of autonomous driving scenarios. Not only we achieved higher precision (in one of the problems outperforming by 25% all other methods), but, more significantly, we only used a single class beforehand, while other methods used all the available classes and we generated interpretable models with smaller number of features used, through extremely weak and weak supervision. We demonstrated the ability to detect and learn new classes for both images and numerical examples.

The antecedent and consequent parts of a first-order evolving intelligent system (EIS) determine the validity of the learning results and overall system performance. Nonetheless, the state-of-the-art techniques mostly stress on the novelty from the system identification point of view but pay less attention to the optimality of the learned parameters. Using the recently introduced autonomous learning multiple model (ALMMo) system as the implementation basis, this article introduces a particle swarm-based approach for the EIS optimization. The proposed approach is able to simultaneously optimize the antecedent and consequent parameters of ALMMo and effectively enhance the system performance by iteratively searching for optimal solutions in the problem spaces. In addition, the proposed optimization approach does not adversely influence the "one pass" learning ability of ALMMo. Once the optimization process is complete, ALMMo can continue to learn from new data to incorporate unseen data patterns recursively without full retraining. The experimental studies with a number of real-world benchmark problems validate the proposed concept and general principles. It is also verified that the proposed optimization approach can be applied to other types of EISs with similar operating mechanisms.

This article describes a novel approach to the problem of developing explainable machine learning models. We consider a deep reinforcement learning (DRL) model representing a highway path planning policy for autonomous highway driving [1]. The model constitutes a mapping from the continuous multidimensional state space characterizing vehicle positions and velocities to a discrete set of actions in longitudinal and lateral direction. It is obtained by applying a customized version of the double deep Q-network learning algorithm [2]. The main idea is to approximate the DRL model with a set of IF-THEN rules that provide an alternative interpretable model, which is further enhanced by visualizing the rules. This concept is rationalized by the universal approximation properties of the rule-based models with fuzzy predicates. The proposed approach includes a learning engine composed of zero-order fuzzy rules, which generalize locally around the prototypes by using multivariate function models. The adjacent (in the data space) prototypes, which correspond to the same action, are further grouped and merged into the so-called MegaClouds reducing significantly the number of fuzzy rules. The input selection method is based on ranking the density of the individual inputs. Experimental results show that the specific DRL agent can be interpreted by approximating with families of rules of different granularity. The method is computationally efficient and can be potentially extended to addressing the explainability of the broader set of fully connected deep neural network models.

A novel self-organizing fuzzy proportional-integral-derivative (SOF-PID) control system is proposed in this paper. The proposed system consists of a pair of control and reference models, both of which are implemented by a first-order autonomous learning multiple model (ALMMo) neuro-fuzzy system. The SOF-PID controller self-organizes and self-updates the structures and meta-parameters of both the control and reference models during the control process "on the fly". This gives the SOF-PID control system the capability of quickly adapting to entirely new operating environments without a full re-training. Moreover, the SOF-PID control system is free from user- and problem-specific parameters and is entirely data-driven. Simulations and real-world experiments with mobile robots demonstrate the effectiveness and validity of the proposed SOF-PID control system.

In this paper we introduce the DMR - a prototype-based method and network architecture for deep learning which is using a decision tree (DT)- based inference and synthetic data to balance the classes. It builds upon the recently introduced xDNN method addressing more complex multi-class problems, specifically when classes are highly imbalanced. DMR moves away from a direct decision based on all classes towards a layered DT of pair-wise class comparisons. In addition, it forces the prototypes to be balanced between classes regardless of possible class imbalances of the training data. It has two novel mechanisms, namely i) using a DT to determine the winning class label, and ii) balancing the classes by synthesizing data around the prototypes determined from the available training data. As a result, we improved significantly the performance of the resulting fully explainable DNN as evidenced on the well know benchmark problem Caltech-101. Furthermore, we also achieved high results in terms of accuracy for the well known Caltech-256 dataset, as well as surpassed the results of other approaches on Faces-1999 problem. In summary, we propose a new approach specifically advantageous for imbalanced multi-class problems on well known hard benchmark datasets. Moreover, DMR offers full explainability, does not require GPUs and can continue to learn from new data by adding new prototypes preserving the previous ones but not requiring full retraining.

Traditionally, in supervised machine learning, (a significant) part of the available data (usually 50%-80%) is used for training and the rest—for validation. In many problems, however, the data are highly imbalanced in regard to different classes or does not have good coverage of the feasible data space which, in turn, creates problems in validation and usage phase. In this paper, we propose a technique for synthesizing feasible and likely data to help balance the classes as well as to boost the performance in terms of confusion matrix as well as overall. The idea, in a nutshell, is to synthesize data samples in close vicinity to the actual data samples specifically for the less represented (minority) classes. This has also implications to the so-called fairness of machine learning. In this paper, we propose a specific method for synthesizing data in a way to balance the classes and boost the performance, especially of the minority classes. It is generic and can be applied to different base algorithms, for example, support vector machines, k-nearest neighbour classifiers deep neural, rule-based classifiers, decision trees, and so forth. The results demonstrated that (a) a significantly more balanced (and fair) classification results can be achieved and (b) that the overall performance as well as the performance per class measured by confusion matrix can be boosted. In addition, this approach can be very valuable for the cases when the number of actual available labelled data is small which itself is one of the problems of the contemporary machine learning.

This paper describes a new self-organizing neuro-fuzzy approach to autonomously learn interpretable models by self-driving cars. A new explainable self-organizing architecture and a new density-based feature selection method are proposed. These new approaches are used to classify different action states occurring from different self-driving conditions. The proposed approach is able to provide human understandable IF ... THEN rules representation due to its learning engine which is composed of a massively parallel set of 0-order fuzzy rules. The proposed density-based feature selection method is based on the ranking of the densities of each feature in the data space, and takes advantage of the parallel characteristic of the proposed explainable self-organizing approach to create individualized subsets of features per class. The main goal of both proposed methods is to provide highly accurate models with high transparency, interpretability, and explainability for self-driving vehicles. In order to validate our proposal, experiments were realized using a real dataset provided by Ford Motor Company. The dataset contains different driving states occurring during self-driving performances. Results demonstrate that the proposed approach could surpass its state-of-the-art competitors in terms of accuracy for this challenge multiclass classification problem.

In this paper, a new deep rule-based approach using high-level ensemble feature descriptor is proposed for aerial scene classification. By creating an ensemble of three pre-trained deep convolutional neural networks for feature extraction, the proposed approach is able to extract more discriminative representations from the local regions of aerial images. With a set of massively parallel IF...THEN rules built upon the prototypes identified through a self-organizing, nonparametric, transparent and highly human-interpretable learning process, the proposed approach is able to produce the state-of-the-art classification results on the unlabeled images outperforming the alternatives. Numerical examples on benchmark datasets demonstrate the strong performance of the proposed approach.

This paper presents an actively semi-supervised multi-layer neuro-fuzzy modeling method, ASSDRB, to classify different lighting conditions for driving scenes. ASSDRB is composed of a massively parallel ensemble of AnYa type 0-order fuzzy rules. It uses a recursive learning algorithm to update its structure when new data items are provided and, therefore, is able to cope with nonstationarities. Different lighting conditions for driving situations are considered in the analysis, which is used by self-driving cars as a safety mechanism. Differently from mainstream Deep Neural Networks approaches, the ASSDRB is able to learn from unseen data. Experiments on different lighting conditions for driving scenes, demonstrated that the deep neuro-fuzzy modeling is an efficient framework for these challenging classification tasks. Classification accuracy is higher than those produced by alternative machine learning methods. The number of algebraic calculations for the present method are significantly smaller and, therefore, the method is significantly faster than common Deep Neural Networks approaches. Moreover, DRB produced transparent AnYa fuzzy rules, which are human interpretable.

Satellite scene images contain multiple sub-regions of different land use categories; however, traditional approaches usually classify them into a particular category only. In this paper, a new approach is proposed for automatically analyzing the semantic content of sub-regions of satellite images. At the core of the proposed approach is the recently introduced deep rule-based image classification method. The proposed approach includes a self-organizing set of transparent zero order fuzzy IF-THEN rules with human-interpretable prototypes identified from the training images and a pre-trained deep convolutional neural network as the feature descriptor. It requires a very short, nonparametric, highly parallelizable training process and can perform a highly accurate analysis on the semantic features of local areas of the image with the generated IF-THEN rules in a fully automatic way. Examples based on benchmark datasets demonstrate the validity and effectiveness of the proposed approach.

In this paper, we introduce a new form of describing fuzzy sets (FSs) and a new form of fuzzy rule-based (FRB) systems, namely, empirical fuzzy sets (εFSs) and empirical fuzzy rule-based (εFRB) systems. Traditionally, the membership functions (MFs), which are the key mathematical representation of FSs, are designed subjectively or extracted from the data by clustering projections or defined subjectively. εFSs, on the contrary, are described by the empirically derived membership functions (εMFs). The new proposal made in this paper is based on the recently introduced Empirical Data Analytics (EDA) computational framework and is closely linked with the density of the data. This allows to keep and improve the link between the objective data and the subjective labels, linguistic terms, and classes definition. Furthermore, εFSs can deal with heterogeneous data combining categorical with continuous and/or discrete data in a natural way. εFRB systems can be extracted from data including data streams and can have dynamically evolving structure. However, they can also be used as a tool to represent expert knowledge. The main difference from the traditional FSs and FRB systems is that the expert does not need to define the MF per variable; instead, possibly multimodal, densities will be extracted automatically from the data and used as εMFs in a vector form for all numerical variables. This is done in a seamless way whereby the human involvement is only required to label the classes and linguistic terms. Moreover, even this intervention is optional. Thus, the proposed new approach to define and design the FSs and FRB systems puts the human “in the driving seat.” Instead of asking experts to define features and MFs correspondingly, to parameterize them, to define algorithm parameters, to choose types of MFs, or to label each individual item, it only requires (optionally) to select prototypes from data and (again, optionally) to label them. Numerical examples as well as a naïve empirical fuzzy (εF) classifier are presented with an illustrative purpose. Due to the very fundamental nature of the proposal, it can have a very wide area of applications resulting in a series of new algorithms such as εF classifiers, εF predictors, εF controllers, and so on. This is left for the future research.

In this letter, we propose a new approach for remote sensing scene classification by creating an ensemble of the recently introduced massively parallel deep (fuzzy) rule-based (DRB) classifiers trained with different levels of spatial information separately. Each DRB classifier consists of a massively parallel set of human-interpretable, transparent zero-order fuzzy IF...THEN... rules with a prototype-based nature. The DRB classifier can self-organize “from scratch” and self-evolve its structure. By employing the pretrained deep convolution neural network as the feature descriptor, the proposed DRB ensemble is able to exhibit human-level performance through a transparent and parallelizable training process. Numerical examples using benchmark data set demonstrate the superior accuracy of the proposed approach together with human-interpretable fuzzy rules autonomously generated by the DRB classifier.

In this paper, we propose a new approach to identify anomalous behaviour based on heterogeneous data and a data fusion technique. There are four types of datasets applied in this study including credit card, loyalty card, GPS, and image data. The first step of the complete framework in this proposed study is to identify the best features for every dataset. Then, the new anomaly detection technique which is recently introduced and known as empirical data analytics (EDA) is applied to detect the abnormal behaviour based on the datasets. Standardised eccentricity (a newly introduced within EDA measure offering a new simplified form of the well-known Chebyshev inequality) can be applied to any data distribution. Image data are processed using pre-trained deep learning network, and classification is done by using support vector machine. Most of the other data used in our previous work are of type “signal”/real number (e.g. credit card, loyalty card and GPS data). However, a clear conclusion that a misuse was made very often cannot be reached based on them only. When gender or age is different from the expected, it is obvious misuse. At the final stage of the proposed method is combining anomaly result and image recognition using data fusion technique. From the experiment results, this proposed technique may simplify the tedious job in the real complex cases of forensic investigation. The proposed technique is using heterogeneous data which combine all the data from the VAST Challenge as well as image data using an introduced data fusion technique. These can assist the human expert in processing huge amount of heterogeneous data to detect anomalies. In future research, text data can also be used as a part of heterogeneous data mixture, and the data fusion technique may be applied to other datasets.

Based on a critical analysis of data analytics and its foundations, we propose a functional approach to estimate data ensemble properties, which is based entirely on the empirical observations of discrete data samples and the relative proximity of these points in the data space and hence named empirical data analysis (EDA). The ensemble functions include the nonparametric square centrality (a measure of closeness used in graph theory) and typicality (an empirically derived quantity which resembles probability). A distinctive feature of the proposed new functional approach to data analysis is that it does not assume randomness or determinism of the empirically observed data, nor independence. The typicality is derived from the discrete data directly in contrast to the traditional approach, where a continuous probability density function is assumed a priori. The typicality is expressed in a closed analytical form that can be calculated recursively and, thus, is computationally very efficient. The proposed nonparametric estimators of the ensemble properties of the data can also be interpreted as a discrete form of the information potential (known from the information theoretic learning theory as well as the Parzen windows). Therefore, EDA is very suitable for the current move to a data-rich environment, where the understanding of the underlying phenomena behind the available vast amounts of data is often not clear. We also present an extension of EDA for inference. The areas of applications of the new methodology of the EDA are wide because it concerns the very foundation of data analysis. Preliminary tests show its good performance in comparison to traditional techniques.

In this paper, an approach to autonomous learning of a multimodel system from streaming data, named ALMMo, is proposed. The proposed approach is generic and can easily be applied also to probabilistic or other types of local models forming multimodel systems. It is fully data driven and its structure is decided by the nonparametric data clouds extracted from the empirically observed data without making any prior assumptions concerning data distribution and other data properties. All metaparameters of the proposed system are obtained directly from the data and can be updated recursively, which improves memory and calculation efficiencies of the proposed algorithm. The structural evolution mechanism and online data cloud quality monitoring mechanism of the ALMMo system largely enhance the ability of handling shifts and/or drifts in the streaming data pattern. Numerical examples of the use of ALMMo system for streaming data analytics, classification, and prediction are presented as a proof of the proposed concept.

In this paper, a correntropy-based evolving fuzzy neural system (CEFNS) is proposed for approximation of nonlinear systems. Different from the commonly used mean-square error criterion, correntropy has a strong outliers rejection ability through capturing the higher moments of the error distribution. Considering the merits of correntropy, this paper brings contributions to build evolving fuzzy neural system (EFNS) based on the correntropy concept to achieve a more stable evolution of the rule base and update of the rule parameters instead of the commonly used mean-square error criterion. The correntropy-EFNS (CEFNS) begins with an empty rule base, and all rules are evolved online based on the correntropy criterion. The consequent part parameters are tuned based on the maximum correntropy criterion, where the correntropy is used as the cost function so as to improve the non-Gaussian noise rejection ability. The steady-state convergence performance of the CEFNS is studied through the calculation of the steady-state excess mean square error (EMSE) in two cases: Gaussian noise; and non-Gaussian noise. Finally, the CEFNS is validated through a benchmark system identification problem, a Mackey-Glass time series prediction problem as well as five other real-world benchmark regression problems under both noise-free and noisy conditions. Compared with other EFNSs, the simulation results show that the proposed CEFNS produces better approximation accuracy using the least number of rules and training time and also owns superior non-Gaussian noise handling capability.

Evolving fuzzy systems (EFSs) are now well developed and widely used, thanks to their ability to self-adapt both their structures and parameters online. Since the concept was first introduced two decades ago, many different types of EFSs have been successfully implemented. However, there are only very few works considering the stability of the EFSs, and these studies were limited to certain types of membership functions with specifically predefined parameters, which largely increases the complexity of the learning process. At the same time, stability analysis is of paramount importance for control applications and provides the theoretical guarantees for the convergence of the learning algorithms. In this paper, we introduce the stability proof of a class of EFSs based on data clouds, which are grounded at the AnYa type fuzzy systems and the recently introduced empirical data analytics (EDA) methodological framework. By employing data clouds, the class of EFSs of AnYa type considered in this paper avoids the traditional way of defining membership functions for each input variable in an explicit manner and its learning process is entirely data driven. The stability of the considered EFS of AnYa type is proven through the Lyapunov theory, and the proof of stability shows that the average identification error converges to a small neighborhood of zero. Although, the stability proof presented in this paper is specially elaborated for the considered EFS, it is also applicable to general EFSs. The proposed method is illustrated with Box-Jenkins gas furnace problem, one nonlinear system identification problem, Mackey-Glass time series prediction problem, eight real-world benchmark regression problems as well as a high-frequency trading prediction problem. Compared with other EFSs, the numerical examples show that the considered EFS in this paper provides guaranteed stability as well as a better approximation accuracy.

In this paper, a new type of feedforward non-parametric deep learning network with automatic feature extraction is proposed. The proposed network is based on human-understandable local aggregations extracted directly from the images. There is no need for any feature selection and parameter tuning. The proposed network involves nonlinear transformation, segmentation operations to select the most distinctive features from the training images and builds RBF neurons based on them to perform classification with no weights to train. The design of the proposed network is very efficient (computation and time wise) and produces highly accurate classification results. Moreover, the training process is parallelizable, and the time consumption can be further reduced with more processors involved. Numerical examples demonstrate the high performance and very short training process of the proposed network for different applications.

Randomized neural network (RNN) is a highly feasible solution in the era of big data because it offers a simple and fast working principle in processing dynamic and evolving data streams. This paper proposes a novel RNN, namely recurrent type-2 random vector functional link network (RT2McRVFLN), which provides a highly scalable solution for data streams in a strictly online and integrated framework. It is built upon the psychologically inspired concept of metacognitive learning, which covers three basic components of human learning: what-to-learn, how-to-learn, and when-to-learn. The what-to-learn selects important samples on the fly with the use of online active learning scenario, which renders our algorithm an online semi-supervised algorithm. The how-to-learn process combines an open structure of evolving concept and a randomized learning algorithm of random vector functional link network (RVFLN). The efficacy of the RT2McRVFLN has been numerically validated through two real-world case studies and comparisons with its counterparts, which arrive at a conclusive finding that our algorithm delivers a tradeoff between accuracy and simplicity.

Human action recognition is an imperative research area in the field of computer vision due to its numerous applications. Recently, with the emergence and successful deployment of deep learning techniques for image classification, object recognition, and speech recognition, more research is directed from traditional handcrafted to deep learning techniques. This paper presents a novel method for human action recognition based on a pre-trained deep CNN model for feature extraction & representation followed by a hybrid Support Vector Machine (SVM) and K-Nearest Neighbor (KNN) classifier for action recognition. It has been observed that already learnt CNN based representations on large-scale annotated dataset could be transferred to action recognition task with limited training dataset. The proposed method is evaluated on two well-known action datasets, i.e., UCF sports and KTH. The comparative analysis confirms that the proposed method achieves superior performance over state-of-the-art methods in terms of accuracy.

In this paper, a fast, transparent, self-evolving, deep learning fuzzy rule-based (DLFRB) image classifier is proposed. This new classifier is a cascade of the recently introduced DLFRB classifier called MICE and an auxiliary SVM. The DLFRB classifier serves as the main engine and can identify a number of human interpretable fuzzy rules through a very short, transparent, highly parallelizable training process. The SVM based auxiliary plays the role of a conflict resolver when the DLFRB classifier produces two highly confident labels for a single image. Only the fundamental image transformation techniques (rotation, scaling and segmentation) and feature descriptors (GIST and HOG) are used for pre-processing and feature extraction, but the proposed approach significantly outperforms the state-of-art methods in terms of both time and precision. Numerical experiments based on a handwritten digits recognition problem are used to demonstrate the highly accurate and repeatable performance of the proposed approach.

In this paper, we propose an approach to data analysis, which is based entirely on the empirical observations of discrete data samples and the relative proximity of these points in the data space. At the core of the proposed new approach is the typicality—an empirically derived quantity that resembles probability. This nonparametric measure is a normalized form of the square centrality (centrality is a measure of closeness used in graph theory). It is also closely linked to the cumulative proximity and eccentricity (a measure of the tail of the distributions that is very useful for anomaly detection and analysis of extreme values). In this paper, we introduce and study two types of typicality, namely its local and global versions. The local typicality resembles the well-known probability density function (pdf), probability mass function, and fuzzy set membership but differs from all of them. The global typicality, on the other hand, resembles well-known histograms but also differs from them. A distinctive feature of the proposed new approach, empirical data analysis (EDA), is that it is not limited by restrictive impractical prior assumptions about the data generation model as the traditional probability theory and statistical learning approaches are. Moreover, it does not require an explicit and binary assumption of either randomness or determinism of the empirically observed data, their independence, or even their number (it can be as low as a couple of data samples). The typicality is considered as a fundamental quantity in the pattern analysis, which is derived directly from data and is stated in a discrete form in contrast to the traditional approach where a continuous pdf is assumed a priori and estimated from data afterward. The typicality introduced in this paper is free from the paradoxes of the pdf. Typicality is objectivist while the fuzzy sets and the belief-based branch of the probability theory are subjectivist. The local typicality is expressed in a closed analytical form and can be calculated recursively, thus, computationally very efficiently. The other nonparametric ensemble properties of the data introduced and studied in this paper, namely, the square centrality, cumulative proximity, and eccentricity, can also be updated recursively for various types of distance metrics. Finally, a new type of classifier called naïve typicality-based EDA class is introduced, which is based on the newly introduced global typicality. This is only one of the wide range of possible applications of EDA including but not limited for anomaly detection, clustering, classification, control, prediction, control, rare events analysis, etc., which will be the subject of further research.

The need to find related images from big data streams is shared by many professionals, such as architects, engineers, designers, journalist, and ordinary people. Users need to quickly find the relevant images from data streams generated from a variety of domains. The challenges in image retrieval are widely recognized, and the research aiming to address them led to the area of content-based image retrieval becoming a “hot” area. In this paper, we propose a novel computationally efficient approach, which provides a high visual quality result based on the use of local recursive density estimation between a given query image of interest and data clouds/clusters which have hierarchical dynamically nested evolving structure. The proposed approach makes use of a combination of multiple features. The results on a data set of 65,000 images organized in two layers of a hierarchy demonstrate its computational efficiency. Moreover, the proposed Look-a-like approach is self-evolving and updating adding new images by crawling and from the queries made.

In recent times there has been an increase in data availability in continuous data streams and clustering of this data has many advantages in data analysis. It is often the case that these data streams are not stationary, but evolve over time, and also that the clusters are not regular shapes but form arbitrary shapes in the data space. Previous techniques for clustering such data streams are either hybrid online / offline methods, windowed offline methods, or find only hyper-elliptical clusters. In this paper we present a fully online technique for clustering evolving data streams into arbitrary shaped clusters. It is a two stage technique that is accurate, robust to noise, computationally and memory efficient, with a low time penalty as the number of data dimensions increases. The first stage of the technique produces micro-clusters and the second stage combines these micro-clusters into macro-clusters. Dimensional stability and high speed is achieved through keeping the calculations both simple and minimal using hyper-spherical micro-clusters. By maintaining a graph structure, where the micro-clusters are the nodes and the edges are its pairs with intersecting micro-clusters, we minimise the calculations required for macro-cluster maintenance. The micro-clusters themselves are described in such a way that there is no calculation required for the core and shell regions and no separate definition of outer micro-clusters necessary. We demonstrate the ability of the proposed technique to join and separate macro-clusters as they evolve in a fully online manner. There are no other fully online techniques that the authors are aware of and so we compare the technique with popular online / offline hybrid alternatives for accuracy, purity and speed. The technique is then applied to real atmospheric science data streams and used to discover short term, long term and seasonal drift and their effects on anomaly detection. As well as having favourable computational characteristics, the technique can add analytic value over hyper-elliptical methods by characterising the cluster hyper-shape using Euclidean or fractal shape factors. Because the technique records macro-clusters as graphs, further analytic value accrues from characterising the order, degree, and completeness of the cluster-graphs as they evolve over time.

In this paper, we propose a novel approach to unsupervised and online data classification. The algorithm is based on the statistical analysis of selected features and development of a self-evolving fuzzy-rule-basis. It starts learning from an empty rule basis and, instead of offline training, it learns “on-the-fly”. It is free of parameters and, thus, fuzzy rules, number, size or radius of the classes do not need to be pre-defined. It is very suitable for the classification of online data streams with real-time constraints. The past data do not need to be stored in memory, since that the algorithm is recursive, which makes it memory and computational power efficient. It is able to handle concept-drift and concept-evolution due to its evolving nature, which means that, not only rules/classes can be updated, but new classes can be created as new concepts emerge from the data. It can perform fuzzy classification/soft-labeling, which is preferred over traditional crisp classification in many areas of application. The algorithm was validated with an industrial pilot plant, where online calculated period and amplitude of control signal were used as input to a fault diagnosis application. The approach, however, is generic and can be applied to different problems and with much higher dimensional inputs. The results obtained from the real data are very significant.

It is well known that clustering is an unsupervised machine learning technique. However, most of the clustering methods need setting several parameters such as number of clusters, shape of clusters, or other user- or problem-specific parameters and thresholds. In this paper, we propose a new clustering approach which is fully autonomous, in the sense that it does not require parameters to be pre-defined. This approach is based on data density automatically derived from their mutual distribution in the data space. It is called ADD clustering (Autonomous Data Density based clustering). It is entirely based on the experimentally observable data and is free from restrictive prior assumptions. This new method exhibits highly accurate clustering performance. Its performance is compared on benchmarked data sets with other competitive alternative approaches. Experimental results demonstrate that ADD clustering significantly outperforms other clustering methods yet does not require restrictive user- or problem-specific parameters or assumptions. The new clustering method is a solid basis for further applications in the field of data analytics.

In this paper the design, implementation, and evaluation of a general purpose smartphone based intelligent surveillance system is presented. It has two main elements; i) a detection module, and ii) a classification module. The detection module is based on the recently introduced approach that combines the well-known background subtraction method with the optical flow and recursively estimated density. The classification module is based on a neural network using Deep Learning methodology. Firstly, the architecture design of the convolutional neural network is presented and analyzed in the context of the four selected architectures (two of them recent successful types) and two custom modifications specifically made for the problem at hand. The results are carefully evaluated, and the best one is selected to be used within the proposed system. In addition, the system is implemented on both a PC (using Linux type OS) and on a smartphone (using Android). In addition to the compatibility with all modern Android-based devices, most GPU-powered platforms such as Raspberry Pi, Nvidia Tegra X1 and Jetson run on Linux. The proposed system can easily be installed on any such device benefiting from the advantage of parallelisation for faster execution. The proposed system achieved a performance which surpasses that of a human (classification accuracy of the top 1 class >95.9% for automatic recognition of a detected object into one of the seven selected categories. For the top-2 classes, the accuracy is even higher (99.85%). That means, at least, one of the two top classes suggested by the system is correct. Finally, a number of visual examples are showcased of the system in use in both PC and Android devices.

In this paper, a novel empirical data analysis approach (abbreviated as EDA) is introduced which is entirely data-driven and free from restricting assumptions and pre-defined problem- or user-specific parameters and thresholds. It is well known that the traditional probability theory is restricted by strong prior assumptions which are often impractical and do not hold in real problems. Machine learning methods, on the other hand, are closer to the real problems but they usually rely on problem- or user-specific parameters or thresholds making it rather art than science. In this paper we introduce a theoretically sound yet practically unrestricted and widely applicable approach that is based on the density in the data space. Since the data may have exactly the same value multiple times we distinguish between the data points and unique locations in the data space. The number of data points, k is larger or equal to the number of unique locations, l and at least one data point occupies each unique location. The number of different data points that have exactly the same location in the data space (equal value), f can be seen as frequency. Through the combination of the spatial density and the frequency of occurrence of discrete data points, a new concept called multimodal typicality, τMMis proposed in this paper. It offers a closed analytical form that represents ensemble properties derived entirely from the empirical observations of data. Moreover, it is very close (yet different) from the histograms, from the probability density function (pdf) as well as from fuzzy set membership functions. Remarkably, there is no need to perform complicated pre-processing like clustering to get the multimodal representation. Moreover, the closed form for the case of Euclidean, Mahalanobis type of distance as well as some other forms (e.g. cosine-based dissimilarity) can be expressed recursively making it applicable to data streams and online algorithms. Inference/estimation of the typicality of data points that were not present in the data so far can be made. This new concept allows to rethink the very foundations of statistical and machine learning as well as to develop a series of anomaly detection, clustering, classification, prediction, control and other algorithms.

In this paper, a novel autonomous data-driven clustering approach, called AD_clustering, is presented for live data streams processing. This newly proposed algorithm is a fully unsupervised approach and entirely based on the data samples and their ensemble properties, in the sense that there is no need for user-predefined or problem-specific assumptions and parameters, which is a problem most of the current clustering approaches suffer from. Moreover, the proposed approach automatically evolves its structure according to the experimentally observable streaming data and is able to recursively update its self-defined parameters using only the current data sample; meanwhile, it discards all the previously processed data samples. Experimental results based on benchmark datasets exhibit the higher performance of the proposed fully autonomous approach compared with the comparative approaches requiring user- and problem-specific parameters to be predefined. This new clustering algorithm is a promising tool for further applications in the field of real-time streaming data analytics.

This paper presents an improved version and a modification of Robust Evolving Cloud-based Controller (RECCo). The first modification is normalization of data space in RECCo. As a consequence, some of the evolving and adaptation parameters become independent of the range of the process output signal. Thus the controller tuning is simplified which makes the approach more appealing for the use in practical applications. The data space normalization is general and is used with Euclidean norm, but other distance metrics could also be used. Beside the normalization new adaptation scheme of the controller gain is proposed which improves the control performance in the case of a negative initial error in starting phase of the evolving process. At the end, different simulation scenarios are tested and analyzed for further practical implementation of the Cloud-based controller into real environments. For that reason a detail simulation study of a plate heat exchanger is performed and different scenarios were analyzed.

Outlier detection is a problem that has been largely studied in the past few years due to its great applicability in real world problems (e.g. financial, social, climate, security). Fault detection in industrial processes is one of these problems. In that context, several methods have been proposed in literature to address fault detection. In this paper we propose a comparative analysis of three recently introduced outlier detection methods: RDE, RDE with Forgetting and TEDA. Such methods were applied to the data set provided in DAMADICS benchmark, a very well-known real data tool for fault detection applications. The results, however, can be extended to similar problems of the area. Therewith, in this work we compare the main features of each method as well as the results obtained with them.

In this paper a new density-based, non-frequentistic data analytics tool, called typicality distribution function (TDF) is proposed. It is a further development of the recently introduced typicality- and eccentricity-based data analytics (TEDA) framework. The newly introduced TDF and its standardized form offer an effective alternative to the widely used probability distribution function (pdf), however, remaining free from the restrictive assumptions made and required by the latter. In particular, it offers an exact solution for any (except a single point) amount of non-coinciding data samples. For a comparison, that the well developed and widely used traditional probability theory and related statistical learning approaches require (theoretically) an infinitely large amount of data samples/ observations, although, in practice this requirement is often ignored. Furthermore, TDF does not require the user to pre-select or assume a particular distribution (e.g. Gaussian or other) or a mixture of such distributions or to pre-define the number of such distributions in a mixture. In addition, it does not require the individual data items to be independent. At the same time, the link with the traditional statistical approaches such as the well-known “nσ” analysis, Chebyshev inequality, etc. offers the interesting conclusion that without the restrictive prior assumptions listed above to which these traditional approaches are tied up the same type of analysis can be made using TDF automatically. TDF can provide valuable information for analysis of extreme processes, fault detection and identification were the amount of observations of extreme events or faults is usually disproportionally small. The newly proposed TDF offers a non-parametric, closed form analytical (quadratic) description extracted from the real data realizations exactly in contrast to the usual practice where such distributions are being pre-assumed or approximated. For example, so called particle filters are also a non-parametric approximation of the traditional statistics; however, they suffer from computational complexity and introduce a large number of dummy data. In addition to that, for several types of proximity/similarity measures (such as Euclidean, Mahalonobis, cosine) it can be calculated recursively, thus, computationally very efficiently and is suitable for real time and online algorithms. Moreover, with a very simple example, it has been illustrated that while traditional probability theory and related statistical approaches can lead in some cases to paradoxically incorrect results and/or to the need for hard prior assumptions to be made. In contrast, the newly proposed TDF can offer a logically meaningful result and an intuitive interpretation automatically and exactly without any prior assumptions. Finally, few simple univariate examples are provided and the process of inference is discussed and the future steps of the development of TDF and TEDA are outlined. Since it is a new fundamental theoretical innovation the areas of applications of TDF and TEDA can span from anomaly detection, clustering, classification, prediction, control, regression to (Kalman-like) filters. Practical applications can be even wider and, therefore, it is difficult to list all of them.

Fault detection is a task of major importance in industry nowadays, since that it can considerably reduce the risk of accidents involving human lives, in addition to production and, consequently, financial losses. Therefore, fault detection systems have been largely studied in the past few years, resulting in many different methods and approaches to solve such problem. This paper presents a detailed study on fault detection on industrial processes based on the recently introduced eccentricity and typicality data analytics (TEDA) approach. TEDA is a recursive and non-parametric method, firstly proposed to the general problem of anomaly detection on data streams. It is based on the measures of data density and proximity from each read data point to the analyzed data set. TEDA is an online autonomous learning algorithm that does not require a priori knowledge about the process, is completely free of user- and problem-defined parameters, requires very low computational effort and, thus, is very suitable for real-time applications. The results further presented were generated by the application of TEDA to a pilot plant for industrial process.

In this article the novel clustering and regression methods TEDACluster and TEDAPredict methods are described additionally to recently proposed evolving classifier TEDAClass. The algorithms for classification, clustering and regression are based on the recently proposed AnYa type fuzzy rule based system. The novel methods use the recently proposed TEDA framework capable of recursive processing of large amounts of data. The framework is capable of computationally cheap exact update of data per sample, and can be used for training `from scratch'. All three algorithms are evolving that is they are capable of changing its own structure during the update stage, which allows to follow the changes within the model pattern.

In this paper we introduce a new data driven method to novelty detection and object definition in dynamic video streams that indiscriminately detects both static and moving objects in the scene. A sliding window density estimation is introduced in order to reliably detect texture edges. A Sobel filtering process is used to extract gradient of edges. Using this new approach, the detection of object textures1 can be done accurately and in real-time. In this paper we demonstrate the capabilities of the algorithm on video scenarios, and show that object textures in the scene are reliably detected. We are able to show clearly the capability of the algorithm to be robust in occlusion scenarios, working in real-time, and defining clear objects where other techniques attribute such small detections to noise.

In this paper, a novel evolving fuzzy rule-based classifier is presented. The proposed classifier addresses the three fundamental issues of data stream learning, viz., computational efficiency in terms of processing time and memory requirements, adaptive to changes, and robustness to noise. Though, there are several online classifiers available, most of them do not take into account all the three issues simultaneously. The newly proposed classifier is inherently adaptive and can attend to any minute changes as it learns the rules in online manner by considering each incoming example. However, it should be emphasized that it can easily distinguish noise from new concepts and automatically handles noise. The performance of the classifier is evaluated using real-life data with evolving characteristic and compared with state-of-the-art adaptive classifiers. The experimental results show that the classifier attains a simple model in terms of number of rules. Further, the memory requirements and processing time per sample does not increase linearly with the progress of the stream. Thus, the classifier is capable of performing both prediction and model update in real-time in a streaming environment.

In this paper, a new fully unsupervised approach to fault detection and identification is proposed. It is based on a two-stage algorithm and starts with the recursive density estimation (RDE) in the feature space. The choice of the features is important and in the real world process that we consider these are control and error related variables. The basis of the proposed approach is the fully unsupervised evolving classifier AutoClass which can be seen as an extension of the earlier one, but is using data clouds and data density information. It has to be stressed that the density in the data space is not the same as the well-known and widely used in statistics probability density function (pdf) although it looks similar. The density in the data space, D is pivotal and instrumental for anomaly detection. It can be calculated recursively, which makes it very efficient in terms of memory, computational power and, thus, applicable to on-line applications. Importantly, the proposed method not only can detect anomalies, but also can identify and diagnose the fault during the second stage of the process. While the first stage is centred around RDE, the second stage is based on the evolving fuzzy rule-based (FRB) classifier AutoClass. A key advantage of AutoClass is that it is fully unsupervised (there is no need to pre-specify the fuzzy rules, number of classes) and can start learning "from scratch". AutoClass can be initialised with some prior knowledge (assuming that it does exists) and evolve/develop it further, but that is not mandatory. This new approach is generic, but in this paper without limiting the concept it is validated on a lab based control kit. In this particular example, the features are the control and error signals. The results significantly outperform alternative methods, which is in addition to the advantages that the approach is autonomous.

Autonomously detecting novelties using background subtraction has quickly become a very important area of image analysis with many different approaches to novelty detection and the output therein. The ultimate goal of the approaches is to be robust to false detections and noise whilst using as little computational power as possible. This review focuses on some of the most prominent pixel-wise background subtraction techniques currently in use, and compares and contrasts their attributes and capabilities. The purpose of this review is to practically summarize the pixel-wise approaches and suggest a way forward from these techniques.

Identification of models from input-output data essentially requires estimation of appropriate cluster centers. In this paper, a new online evolving clustering approach for streaming data is proposed. Unlike other approaches that consider either the data density or distance from existing cluster centers, this approach uses cluster weight and distance before generating new clusters. To capture the dynamics of the data stream, the cluster weight is defined in both data and time space in such a way that it decays exponentially with time. It also applies concepts from computational geometry to determine the neighborhood information while forming clusters. A distinction is made between core and noncore clusters to effectively identify the real outliers. The approach efficiently estimates cluster centers upon which evolving Takagi-Sugeno models are developed. The experimental results with developed models show that the proposed approach attains results at par or better than existing approaches and significantly reduces the computational overhead, which makes it suitable for real-time applications.

Most of the dynamics in real-world systems are compiled by shifts and drifts, which are uneasy to be overcome by omnipresent neuro-fuzzy systems. Nonetheless, learning in nonstationary environment entails a system owning high degree of flexibility capable of assembling its rule base autonomously according to the degree of nonlinearity contained in the system. In practice, the rule growing and pruning are carried out merely benefiting from a small snapshot of the complete training data to truncate the computational load and memory demand to the low level. An exposure of a novel algorithm, namely parsimonious network based on fuzzy inference system (PANFIS), is to this end presented herein. PANFIS can commence its learning process from scratch with an empty rule base. The fuzzy rules can be stitched up and expelled by virtue of statistical contributions of the fuzzy rules and injected datum afterward. Identical fuzzy sets may be alluded and blended to be one fuzzy set as a pursuit of a transparent rule base escalating human's interpretability. The learning and modeling performances of the proposed PANFIS are numerically validated using several benchmark problems from real-world or synthetic datasets. The validation includes comparisons with state-of-the-art evolving neuro-fuzzy methods and showcases that our new method can compete and in some cases even outperform these approaches in terms of predictive fidelity and model complexity.

Learning and prediction in a data streaming environment is challenging due to continuous arrival of enormous data in high speed that often evolves with time. In this paper we present a dynamically evolving fuzzy rule-based model that predicts and learns from each instance in the stream, taking into account the principal issues of streaming environment viz., limited memory, real time, and dynamic nature. The fuzzy model essentially uses a newly proposed dynamically evolving clustering method for learning the structure. Unlike other approaches that consider either the data density or distance from existing cluster centres, this approach considers both density and distance to decide if a new cluster is to be generated. To capture the dynamics of the data stream, the density is defined in both data and time space in such a way that it decays exponentially with time. A distinction is made between core and non-core clusters to effectively identify the real outliers. The experimental results using benchmark and real datasets show that the proposed approach attains results at par or better than existing approaches and significantly reduces the computational overhead.

In our work we used the capability of one-class support vector machine (SVM) method to develop a novel one-class classification approach. Algorithm is designed and tested within the project SVETLANA aimed for fault detection in complex technological systems, such as aircraft. The main objective of this project was to create an algorithm responsible for collecting and analyzing the data since the launch of an aircraft engine. Data can be transferred from a variety of sensors that are responsible for the speed, oxygen level etc. In order to apply real time (in flight) application a recursive learning algorithm is proposed. The proposed method analyzes both “positive”/”normal” and “negative”/ “abnormal” examples The overall model structure is the same as an outlier-detection approach. The most important benefits of the new algorithm based on our algorithm are verified in comparison with several classifiers, including the traditional one-class SVM. This algorithm has been tested on real flight data from the USA, Western European as well as Russia. The test results are presented in the final part of the article.

SUMMARY: IRootLab is a free and open-source MATLAB toolbox for vibrational biospectroscopy (VBS) data analysis. It offers an object-oriented programming class library, graphical user interfaces (GUIs) and automatic MATLAB code generation. The class library contains a large number of methods, concepts and visualizations for VBS data analysis, some of which are introduced in the toolbox. The GUIs provide an interface to the class library, including a module to merge several spectral files into a dataset. Automatic code allows developers to quickly write VBS data analysis scripts and is a unique resource among tools for VBS. Documentation includes a manual, tutorials, Doxygen-generated reference and a demonstration showcase. IRootLab can handle some of the most popular file formats used in VBS. License: GNU-LGPL. AVAILABILITY: Official website: http://irootlab.googlecode.com/. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

A new on-line evolving clustering approach for streaming data is proposed in this paper. The approach is based on the concept that local mean of samples within a region has the highest density and the gradient of the density points towards the local mean. The algorithm merely requires recursive calculation of local mean and variance, due to which it easily meets the memory and time constraints for data stream processing. The experimental results using synthetic and benchmark datasets show that the proposed approach attains results at par with offline approach and is comparable to popular density-based mean-shift clustering yet it is significantly more efficient being one-pass and non-iterative.

In this paper, a new approach for Self-evolving PArameter-free fuzzy Rule-based Controller (SPARC) is proposed. Two illustrative examples are provided aiming a proof of concept. The proposed controller can start with no pre-defined fuzzy rules, and does not need to pre-define the range of the output or control variables. This SPARC learns autonomously from its own actions while performing the control of the plant. It does not use any parameters, explicit membership functions, any off-line pre-training nor the explicit model (e.g. in a form of differential equations) of the plant. It combines the relative older concept of indirect adaptive control with the newer concepts of (self-)evolving fuzzy rule-based systems (and controllers, in particular) and with the very recent concept of parameter-free, data cloud and data density based fuzzy rule based systems (and controllers in particular). It has been demonstrated that a fully autonomously and in an unsupervised manner (based only on the data density and selecting representative prototypes/focal points from the control hyper-surface acting as a data space) it is possible generate a parameter-free control structure and evolve it in on-line mode. Moreover, the results demonstrate that this autonomous controller is effective (has comparative error and performance characteristics) to other known controllers, including self-learning ones, but surpasses them with its flexibility and extremely lean structure (small number of prototypes/focal points which serve as seeds to form parameter-free and membership function-free fuzzy rules based on them). The illustrative examples aim primarily proof of concept.

Knowledge about computer users is very beneficial for assisting them, predicting their future actions or detecting masqueraders. In this paper, a new approach for creating and recognizing automatically the behavior profile of a computer user is presented. In this case, a computer user behavior is represented as the sequence of the commands she/he types during her/his work. This sequence is transformed into a distribution of relevant subsequences of commands in order to find out a profile that defines its behavior. Also, because a user profile is not necessarily fixed but rather it evolves/changes, we propose an evolving method to keep up to date the created profiles using an Evolving Systems approach. In this paper, we combine the evolving classifier with a trie-based user profiling to obtain a powerful self-learning online scheme. We also develop further the recursive formula of the potential of a data point to become a cluster center using cosine distance, which is provided in the Appendix. The novel approach proposed in this paper can be applicable to any problem of dynamic/evolving user behavior modeling where it can be represented as a sequence of actions or events. It has been evaluated on several real data streams.

In this paper an original approach is proposed which makes possible autonomous scenes recognition performed on-line by an evolving self-learning classifier. Existing approaches for scene recognition are off-line and used in intelligent albums for picture categorization/selection. The emergence of powerful mobile platforms with camera on board and sensor-based autonomous (robotic) systems is pushing forward the requirement for efficient self-learning and adaptive/evolving algorithms. Fast real-time and online algorithms for categorisation of the real world environment based on live video stream are essential for understanding and situation awareness as well as for localization and context awareness. In scene analysis the critical problem is feature extraction mechanism for a quick description of the scene. In this paper we apply a well known technique called spatial envelop or GIST. Visual scenes can be quite different but very often they can be grouped in similar types/categories. For example, pictures from different cities across the Globe, e.g. Tokyo, Vancouver, New York Moscow, Dusseldorf, etc. bear the similar pattern of an urban scene high rise buildings, despite the differences in the architectural style. Same applies for the beaches of Miami, Maldives, Varna, Costa del Sol, etc. One assumption based on which such automatic video classifiers can be build is to pre-train them using a large number of such images from different groups. Variety of possible scenes suggests the limitations of such an approach. Therefore, we use in this paper the recently propose evolving fuzzy rule-based classifier, simpleClass, which is self learning and thus updates its rules and categories descriptions with each new image. In addition, it is fully recursive, computationally efficient and yet linguistically transparent.

A new approach to real-time human activity recognition (HAR) using evolving self-learning fuzzy rule-based classifier (eClass) will be described in this paper. A recursive version of the principle component analysis (PCA) and linear discriminant analysis (LDA) pre-processing methods is coupled with the eClass leading to a new approach for HAR which does not require computation and time consuming pre-training and data from many subjects. The proposed new method for evolving HAR (eHAR) takes into account the specifics of each user and possible evolution in time of her/his habits. Data streams from several wearable devices which make possible to develop a pervasive intelligence enabling them to personalize/tune to the specific user were used for the experimental part of the paper.

This paper presents the sequel of evolving fuzzy rule-based classifier eClass, called here as simplified evolving classifier, simpl_eClass. Similarly to eClass, simpl_eClass comprises of two different classifiers, namely zero and first order (simpl_eClass0 and simpl_eClass1). The two classifiers differ from each other in terms of the consequent part of the fuzzy rules, and the classification strategy used. The design of simpl_eClass is based on the density increment principle introduced recently in so called simpl_eTS+ approach. The rule learning in simpl_eClass does not involve computation of potential values that allows it to attain computationally much less expensive model update phase compared to eClass. As compared to other FRB classifiers, it retains all the advantages of eClass, such as being on-line and evolving, having zero and first order. In comparison with other non-fuzzy classifiers it has the advantage of interpretability and transparency (especially zero order type). The goals of this paper are to demonstrate the applicability of simpl_eTS+ to classification task, and to empirically show that the simplification of eClass to simpl_eClass by using potential-free approach does not compromise the accuracy of the classifiers. In order to attain the goals, the classifiers are tested by performing several experiments using benchmark data sets. The simpl_eClass1 classifier is also applied to the real-life problem of on-line scene categorization for low-resource devices benefiting from its low computational cost. The results obtained from the experiments endorse that simpl_eClass achieves the accuracy of eClass while simplifying rule learning process.

Recently, surveillance, security, patrol, search, and rescue applications increasingly require algorithms and methods that can work automatically in real time. This paper reports a new real-time approach based on three novel techniques for automatic detection, object identification, and tracking in video streams, respectively. The novelty detection and object identification are based on the newly proposed recursive density estimation (RDE) method. RDE is using a Cauchy-type of kernel, which is calculated recursively as opposed to the widely used (in particular in the kernel density estimation (KDE) approach) Gaussian one. The key difference is that the proposed approach works on a per frame basis and does not require a window (usually of size of several dozen) of frames to be stored in the memory and processed. It should be noted that the new RDE approach is free from user- or problem-specific thresholds by differ from the other state-of-the-art approaches. Finally, an evolving Takagi–Sugeno (eTS)-type fuzzy system is proposed for tracking. The proposed approach has been compared with KDE and Kalman filter (KF) and has proven to be significantly (in an order of magnitude) faster and computationally more efficient than RDE and more precise than KF. © 2010 Wiley Periodicals, Inc.

Neural networks (NNs) have numerous applications to online processes, but the problem of stability is rarely discussed. This is an extremely important issue because, if the stability of a solution is not guaranteed, the equipment that is being used can be damaged, which can also cause serious accidents. It is true that in some research papers this problem has been considered, but this concerns continuous-time NN only. At the same time, there are many systems that are better described in the discrete time domain such as population of animals, the annual expenses in an industry, the interest earned by a bank, or the prediction of the distribution of loads stored every hour in a warehouse. Therefore, it is of paramount importance to consider the stability of the discrete-time NN. This paper makes several important contributions. 1) A theorem is stated and proven which guarantees uniform stability of a general discrete-time system. 2) It is proven that the backpropagation (BP) algorithm with a new time-varying rate is uniformly stable for online identification and the identification error converges to a small zone bounded by the uncertainty. 3) It is proven that the weights' error is bounded by the initial weights' error, i.e., overfitting is eliminated in the proposed algorithm. 4) The BP algorithm is applied to predict the distribution of loads that a transelevator receives from a trailer and places in the deposits in a warehouse every hour, so that the deposits in the warehouse are reserved in advance using the prediction results. 5) The BP algorithm is compared with the recursive least square (RLS) algorithm and with the Takagi-Sugeno type fuzzy inference system in the problem of predicting the distribution of loads in a warehouse, giving that the first and the second are stable and the third is unstable. 6) The BP algorithm is compared with the RLS algorithm and with the Kalman filter algorithm in a synthetic example.

A general framework and a holistic concept are proposed in this paper that combine computationally light machine learning from streaming data with the online identification and adaptation of dynamic systems in regard to their structure and parameters. According to this concept, the system is assumed to be decomposable into a set of fuzzily connected simple local models. The main thrust of this paper is in the development of an original approach for the self-design, self-monitoring, self-management, and self-learning of such systems in a dynamic manner from data streams which automatically detect and react to the shift in the data distribution by evolving the system structure. Novelties of this contribution lie in the following: 1) the computationally simple approach (simpl_e_Clustering-simplified evolving Clustering) to data space partitioning by recursive evolving clustering based on the relative position of the new data sample to the mean of the overall data, 2) the learning technique for online structure evolution as a reaction to the shift in the data distribution, 3) the method for online system structure simplification based on utility and inputs/feature selection, and 4) the novel graphical illustration of the spatiotemporal evolution of the data stream. The application domain for this computationally efficient technique ranges from clustering, modeling, prognostics, classification, and time-series prediction to pattern recognition, image segmentation, vector quantization, etc., to more general problems in various application areas, e.g., intelligent sensors, mobile robotics, advanced manufacturing processes, etc.

In this paper, we propose an automated approach to track and recognize daily activities. Any activity is represented in this research as a sequence of raw sensors data. These sequences are treated using statistical methods in order to discover activity patterns. However, as the way to perform an activity is usually not fixed but it changes and evolves, we propose an activity recognition method based on Evolving Systems.

A new approach to real-time knowledge extraction from streaming data generated by wearable wireless accelerometers based on self-learning evolving fuzzy rule-based classifier is proposed and evaluated in this paper. Based on experiments with real subjects we collected data from 18 different classifieds activities. After preprocessing and classifying data depending on the sequence of activities regarding time, we achieved up to 99.81% of accuracy in recognizing a sequence of activities. This technique allows re-training the system as long as the application is running on the wearable intelligent/smart sensor, getting a better classification rate throughout the time without an increase of the delay in performance.

In this paper we present results and algorithm used to predict 14 days horizon from a number of time series provided by the NN GC1 concerning transportation datasets [1]. Our approach is based on applying the well known Evolving Takagi-Sugeno (eTS) Fuzzy Systems [2-6] to self-learn from the time series. ETS are characterized by the fact that they self-learn and evolve the fuzzy rule-based system which, in fact, represents their structure from the data stream on-line and in real-time mode. That means we used all the data samples from the time series only once, at any instant in time we only used one single input vector (which consist of few data samples as described below) and we do not iterate or memorize the whole sequence. It should be emphasized that this is a huge practical advantage which, unfortunately cannot be compared directly to the other competitors in NN GC1 if only precision/error is taken as a criteria. It is also worth to require time for calculations and memory usage as well as iterations and computational complexity to be provided and compared to build a fuller picture of the advantages the proposed technique offers. Nevertheless, we offer a computationally light and easy to use approach which in addition does not require any user-or problem-specific thresholds or parameters to be specified. Additionally, this approach is flexible in terms not only of its structure (fuzzy rule based and automatic self-development), but also in terms of automatic input selection as will be described below.

Knowledge about computer users is very beneficial for assisting them, predicting their future actions or detecting masqueraders. In this paper, an approach for creating and recognizing automatically the behavior profile of a computer user is combined with an evolving method to keep up to date the created profiles. The behavior of a computer is represented in this research as the sequence of commands s/he types during a period of time. This sequence is treated using statistical methods in order to create the corresponding user profile. However, as a user profile is usually not fixed but rather it changes and evolves, we propose a user profile classifier based on Evolving Systems. This paper describes briefly the model creation method and the evolving classifier, which are compared with well established off-line and on-line classifiers.

Environments equipped with intelligent sensors can be of much help if they can recognize the actions or activities of their users. If this activity recognition is done automatically, it can be very useful for different tasks such as future action prediction, remote health monitoring, or interventions. Although there are several approaches for recognizing activities, most of them do not consider the changes in how a human performs a specific activity. We present an automated approach to recognize daily activities from the sensor readings of an intelligent home environment. However, as the way to perform an activity is usually not fixed but it changes and evolves, we propose an activity recognition method based on Evolving Fuzzy Systems.

A new technique to the design and use of inferential sensors in the process industry is proposed in this paper, which is based on the recently introduced concept of evolving fuzzy models (EFMs). They address the challenge that the modern process industry faces today, namely, to develop such adaptive and self-calibrating online inferential sensors that reduce the maintenance costs while keeping the high precision and interpretability/transparency. The proposed new methodology makes possible inferential sensors to recalibrate automatically, which reduces significantly the life-cycle efforts for their maintenance. This is achieved by the adaptive and flexible open-structure EFM used. The novelty of this paper lies in the following: (1) the overall concept of inferential sensors with evolving and self-developing structure from the data streams; (2) the new methodology for online automatic selection of input variables that are most relevant for the prediction; (3) the technique to detect automatically a shift in the data pattern using the age of the clusters (and fuzzy rules); (4) the online standardization technique used by the learning procedure of the evolving model; and (5) the application of this innovative approach to several real-life industrial processes from the chemical industry (evolving inferential sensors, namely, eSensors, were used for predicting the chemical properties of different products in The Dow Chemical Company, Freeport, TX). It should be noted, however, that the methodology and conclusions of this paper are valid for the broader area of chemical and process industries in general. The results demonstrate that well-interpretable and with-simple-structure inferential sensors can automatically be designed from the data stream in real time, which predict various process variables of interest. The proposed approach can be used as a basis for the development of a new generation of adaptive and evolving inferential sensors that can address the challenges of the modern advanced process industry.

A new approach to fuzzy rule-based systems structure identification in on-line (possibly real-time) mode is described in this paper. It expands the so called evolving Takagi-Sugeno (eTS) approach by introducing self-learning aspects not only to the number of fuzzy rules and system parameters but also to the number of antecedent part variables (inputs). The approach can be seen as on-line sensitivity analysis or on-line feature extraction (if in a classification application, e.g. in eClass which is the classification version of eTS). This adds to the flexibility and self-learning capabilities of the proposed system. In this paper the mechanism of formation of new fuzzy sets as well as of new fuzzy rules is analyzed from the point of view of on-line (recursive) data density estimation. Fuzzy system structure simplification is also analyzed in on-line context. Utility- and age-based mechanisms to address this problem are proposed. The rule-base structure evolves based on a gradual update driven by; i) information coming from the new data samples; ii) on-line monitoring and analysis of the existing rules in terms of their utility, age, and variables that form them. The theoretical theses are supported by experimental results from a range of real industrial data from chemical, petro-chemical and car industries. The proposed methodology is applicable to a wide range of fault detection, prediction, and control problems when the input or feature channels are too many.

Autonomous systems for surveillance, security, patrol, search and rescue are the focal point of extensive research and interest from defense and the security related industry, traffic control and other institutions. A range of sensors can be used to detect and track objects, but optical cameras or camcorders are often considered due to their convenience and passive nature. Tracking based on color intensity information is often preferred than the motion cues due to being more robust. The technique presented in this paper can also be used in conjunction with infra-red cameras, 3D lasers which result in a grey scale image. Novelty detection and tracking are two of the key elements of such systems. Most of the currently reported techniques are characterized by high computational, memory storage costs and are not autonomous because they usually require a human operator in the loop. This paper presents new approaches to both the problem of novelty detection and object tracking in video streams. These approaches are rooted in the recursive techniques that are computationally efficient and therefore potentially applicable in real-time. A novel approach for recursive density estimation (RDE) using a Cauchy type of kernel (as opposed to the usually used Gaussian one) is proposed for visual novelty detection and the use of the recently introduced evolving Takagi-Sugeno (eTS) neuro-fuzzy system for tracking the object detected by the RDE approach is proposed as opposed to the usually used Kalman filter (KF). In fact, eTS can be seen as a fuzzily weighted mixture of KF. The proposed technique is significantly faster than the well known kernel density estimation (KDE) approach for background subtraction for novelty detection and is more precise than the usually used KF. Additionally the overall approach removes the need of manually selecting the object to be tracked which makes possible a fully autonomous system for novelty detection and tracking to be developed.

It is a well-recognized fact that the theory of fuzzy sets and systems, for the last four decades after the seminal paper by Professor Zadeh [1], has demonstrated its remarkable ability to go beyond conventional information representation. It resulted in a wide range of new formulations of practical problems, such as fuzzy control, fuzzy clustering and classification, fuzzy modeling, and fuzzy optimization [2]. Historically, the design of the fuzzy systems has been initially assumed to be centered on expert knowledge [3]. During the 1990s, a new trend emerged [4], [5] that offered techniques to make use of the experimental data. This data-centered approach can be used to enhance and validate the existing expert knowledge or can also be used to substitute its lack (as is the case with autonomous systems, for example). Neurofuzzy and hybrid learning systems were introduced, where fuzzy representation was integrated into a neural learning architecture to bring linguistic meaning of the learned information [5]. (c) IEEE Press

A new approach to the online classification of streaming data is introduced in this paper. It is based on a self-developing (evolving) fuzzy-rule-based (FRB) classifier system ofTakagi-Sugeno (eTS) type. The proposed approach, calledeClass(evolvingclassifier), includes different architectures and online learning methods. The family of alternative architectures includes: 1)eClass0, with the classifier consequents representing class label and 2) the newly proposed method for regression over the features using a first-ordereTSfuzzy classifier,eClass1. An important property ofeClassis that it can start learning ldquofrom scratch.rdquo Not only do the fuzzy rules not need to be prespecified, but neither do the number of classes foreClass(the number may grow, with new class labels being added by the online learning process). In the event that an initial FRB exists,eClasscan evolve/develop it further based on the newly arrived data. The proposed approach addresses the practical problems of the classification of streaming data (video, speech, sensory data generated from robotic, advanced industrial applications, financial and retail chain transactions, intruder detection, etc.). It has been successfully tested on a number of benchmark problems as well as on data from an intrusion detection data stream to produce a comparison with the established approaches. The results demonstrate that a flexible (with evolving structure) FRB classifier can be generated online from streaming data achieving high classification rates and using limited computational resources.

A novel approach to visual self-localization in completely unknown environment with a fully unsupervised and computationally efficient algorithm is proposed in this paper. It is based on the recently developed evolving fuzzy classifier (eClass). The problem of self localization and landmark recognition is of extreme importance for designing efficient and flexible land-based autonomous uninhabited vehicles (AUV). The availability of global coordinates, a GPS link, and unrestricted communication is often compromised by a number of factors, such as interference, weather, and mission objectives. The ability to self-localize and recognize landmarks is vital in such cases for an AUV to survive and function effectively. The self-organizing classifier (eClass) is designed by automatic labeling and grouping the landmarks that are detected in real-time based on the image data (video stream grabbed by the camera mounted on the mobile robot, AUV). The proposed approach makes possible autonomous joint landmark detection and recognition without the use of absolute coordinates, any communication link or any pre-training. The proposed algorithm is recursive, non-iterative, one pass and thus computationally inexpensive and suitable for real-time applications. A set of new formulae for on-line data normalization of the data are introduced in the paper. Real-life tests has been carried out in outdoor environment at the Lancaster University campus using Pioneer3 DX mobile robots equipped with a pan-tilt zoom camera and an on-board PC. The results illustrate the viability and flexibility of the proposed approach. Further investigations will be directed towards teams of mobile robots (AUV) performing a task in completely unknown environment

In this paper a new method for training single-model and multi-model fuzzy classifiers incrementally and adaptively is proposed, which is called FLEXFIS-Class. The evolving scheme for the single-model case exploits a conventional zero-order fuzzy classification model architecture with Gaussian fuzzy sets in the rules antecedents, crisp class labels in the rule consequents and rule weights standing for confidence values in the class labels. In the multi-model case FLEXFIS-Class exploits the idea of regression by an indicator matrix to evolve a Takagi-Sugeno fuzzy model for each separate class and combines the single models' predictions to a final classification statement. The paper includes a technique for increasing the prediction quality, whenever a drift in a data stream occurs. An empirical analysis will be given based on an online, adaptive image classification framework, where images showing production items should be classified into good or bad ones. This analysis will include the comparison of evolving single-and multi-model fuzzy classifiers with conventional batch modelling approaches with respect to achieved prediction accuracy on new online data. It will also be shown that multi-model architecture can outperform conventional single-model architecture ('classical' fuzzy classification models) for all data sets with respect to prediction accuracy.

In this paper the recently introduced evolving fuzzy classifier method called eClass is studied in respect to its architecture and evolution of the fuzzy rule-base. The proposed classifier has an open/evolving structure and can start 'from scratch', learning and adapting to the new data samples. Alternatively, if an initial fuzzy rule-based classifier, generated beforehand in off-line mode or provided by the operator, exists then eClass can evolve this initial classifier in on-line mode. In other words, the fuzzy rule base will evolve incorporating new rules, modifying and/or, possibly, removing some of the previously existing ones. Additionally, the parameters of both, the antecedent and the consequent parts are adapted. Note that eClass can start with an empty rule-base, which is a unique feature of this approach. The proposed approach is free from user-specified parameters and the mechanism of forming new rules is very robust. In this paper, four different modelling architectures are described and compared. The architectures are based on (i) unsupervised cluster partitions, eClassC; (ii) Sugeno fuzzy models with singleton consequents, eClassA; (iii) Takagi-Sugeno fuzzy models with linear consequent functions, eClassB; and (iv) a multi-model classification architecture, where separate TS regression models are combined to form an overall classification output of the system, eClassM. A thorough comparison of the results when applying each of these architectures and the results using previously existing classifiers has been made using an online interactive self-adaptive image classification framework.

Prediction of the properties of the crude oil distillation side streams based on statistical methods and laboratory-based analysis has been around for decades. However, there are still many problems with the existing estimators that require a development of new techniques especially for an on-line analysis of the quality of the distillation process. The nature of non-linear characteristics of the refinery process, the variety of properties to measure and control and the narrow window that normally refinery processes operates in are only some of the problems that a prediction technique should deal with in order to be useful for a practical application. There are many successful application cases that refinery units use real plant data to calibrate models. They can be used to predict quality properties of the gas oil, naphtha, kerosene and other products of a crude oil distillation tower. Some of these are distillation end points and cold properties (freeze, cloud). However, it is difficult to identify, control or compensate the dynamic process behaviour and the errors from instrumentation for an online model prediction. The objective of this work is to report an application an a study of a novel technique for real-time modelling, namely eXtended Evolving Fuzzy Takagi-Sugeno models (xTS) for prediction and online monitoring of these properties of the refinery distillation process. The results presented here include the online prediction of Soft Sensors for distillation of Naptha and Gasoil Side Streams. The application takes data in an automatic fashion and predicts the quality of the side stream evolving its fuzzy structure and cluster parameters to represent a better behaviour of the plant. These preliminary results show the performance of this technique as an online estimator.

This paper deals with a simplified version of the evolving Takagi-Sugeno (eTS) learning algorithm - a computationally efficient procedure for on-line learning TS type fuzzy models. It combines the concept of the scatter as a measure of data density and summarization ability of the TS rules, the use of Cauchy type antecedent membership functions, an aging indicator characterizing the stationarity of the rules, and a recursive least square algorithm to dynamically learn the structure and parameters of the eTS model

Evolving Takagi-Sugeno (eTS) fuzzy models and the method for their on-line identification has been recently introduced as an effective tool for design of flexible system models with minimum a priori information. Their structure develops on-line during the process of model identification itself. In this paper, this approach has been extended for the case of multi-input multi-output (MIMO) system model. Both parts of the identification algorithm, namely the unsupervised fuzzy rule-base antecedents learning by a recursive, noniterative clustering, and the supervised linear sub-model parameters learning by Kalman-filtering-based procedure, are extended for the MIMO case. The radius of influence of each fuzzy rule is considered a vector instead of a scalar as in the original eTS approach, allowing different areas of the data space to be covered by each input variable. As in the eTS, in MIMO eTS, the rule-base and parameters of the fuzzy model continually evolve by adding new rules with more summarization power and by modifying existing rules and parameters. Simulation results using a well-known benchmark are considered in this paper. Further investigation concern the application of MIMO eTS to predictive modeling of the speech spectrum magnitude, classification of multi-channel source modulation etc.

A flexible model in the form of an artificial neural network (NN) with evolving structure (eNN) is represented in the paper in the form of the evolving fuzzy Takagi-Sugeno model. It falls into the same category of models as the recently introduced evolving rule-based (eR) models. The learning algorithm is incremental, unsupervised and is based on the on-line identification of Takagi-Sugeno type quasilinear models. Both eR and eNN differ from the other model schemes by their gradually evolving structure as opposed to the fixed structure models, in which only parameters are subject to optimization or adaptation. Essentially, it represents a Takagi-Sugeno model with gradually evolving set of rules, determined on-line. This approach has potential in both modeling and control using indirect learning mechanisms. Its computational efficiency is based on the non-iterative and recursive procedure, which combines a Kalman filter with proper initializations, and online unsupervised clustering. eNN has been tested with data from a real air-conditioning installation. Applications to real-time adaptive non-linear control, fault detection and diagnostics, performance analysis, time-series forecasting, knowledge extraction and accumulation, etc. are possible directions of their use in the future research.

An approach to the online learning of Takagi-Sugeno (TS) type models is proposed in the paper. It is based on a novel learning algorithm that recursively updates TS model structure and parameters by combining supervised and unsupervised learning. The rule-base and parameters of the TS model continually evolve by adding new rules with more summarization power and by modifying existing rules and parameters. In this way, the rule-base structure is inherited and up-dated when new data become available. By applying this learning concept to the TS model we arrive at a new type adaptive model called the Evolving Takagi-Sugeno model (ETS). The adaptive nature of these evolving TS models in combination with the highly transparent and compact form of fuzzy rules makes them a promising candidate for online modeling and control of complex processes, competitive to neural networks. The approach has been tested on data from an air-conditioning installation serving a real building. The results illustrate the viability and efficiency of the approach. The proposed concept, however, has significantly wider implications in a number of fields, including adaptive nonlinear control, fault detection and diagnostics, performance analysis, forecasting, knowledge extraction, robotics, behavior modeling.

An approach to identification of evolving fuzzy rule-based (eR) models is proposed. eR models implement a method for the noniterative update of both the rule-base structure and parameters by incremental unsupervised learning. The rule-base evolves by adding more informative rules than those that previously formed the model. In addition, existing rules can be replaced with new rules based on ranking using the informative potential of the data. In this way, the rule-base structure is inherited and updated when new informative data become available, rather than being completely retrained. The adaptive nature of these evolving rule-based models, in combination with the highly transparent and compact form of fuzzy rules, makes them a promising candidate for modeling and control of complex processes, competitive to neural networks. The approach has been tested on a benchmark problem and on an air-conditioning component modeling application using data from an installation serving a real building. The results illustrate the viability and efficiency of the approach.

A new approach to fuzzy optimization based on the generalization of Bellman-Zadeh's (BZ) concept is proposed in this article. It consists of a parametric generalization of intersection of fuzzy sets and a generalized defuzzification method. This approach allows the solving of a fuzzy mathematical programming (FMP) problem without transformation to a crisp one. It takes into account all possible fuzzy decisions and allows the degree of conjunction of criteria and constraints to vary. BZ method can be considered a special case of the approach proposed here. A simple algorithm for noniterative solving FMP problem is proposed whereas well-known Zimmermann's approach uses numerical methods. an illustrative example is presented. © 1994 John Wiley & Sons, Inc.