Deciding where to go is one of the primary challenges in designing an agent that can explore an unknown environment. Grid-worlds provide a flexible framework for representing different variations of this problem, allowing for various types of goals and constraints. Typically, agents move one cell at a time, gathering new information at each time step. However, recomputing a new action after each step can lead to unintended behaviors, such as indecision and forgetting about previous goals. To mitigate this, we define a set of persistent feature layers that can be used by either a linear weighted policy or a neural network approach to identify potential destination locations. The outputs of these policies are processed using knowledge of the environment to ensure that objectives are met in a timely and effective manner. We demonstrate how to train and evaluate a U-Net model in a custom grid-world environment and provide guidance and suggestions for how to use this approach to build complex agent behaviors.

The spatial configuration of objects in a scene is important to many applications. In particular, 3D environments constructed from point cloud observations are often used for navigation and planning with real-time requirements. In these settings, the ability to recognize and distinguish one set of objects from another may depend largely on how they are positioned with respect to each other. In this article, we explore two different approaches for describing the relative spatial relationship between two objects represented as 3D points: the histogram of forces, and a method using bounding boxes and fuzzy numbers. We use 2D axis-aligned projections of the objects to facilitate the computation of force histograms, and compare this approach to the bounding box and fuzzy number method. Our experiments are performed on the NPM3D dataset, consisting of hand-labeled point cloud objects in an outdoor street-level environment. The results highlight the strengths and weaknesses of each approach and we discuss the most appropriate applications for both.

Modern machine learning (ML) is based to a great extent on supervised deep learning models that require large amounts of labeled training data. While image data sets with annotations exist, the annotations are produced manually and possess relatively simple descriptions. To date, none of the freely available labeled image data sets incorporate spatial reasoning, one of Gardner's nine human intelligences. This article presents a new process with open source tools provided to label imagery based on spatial interactions between image objects and auto-mated reasoning under uncertainty. The resulting annotated data can be used to train new ML/AI algorithms and/or help us better understand existing methodologies.

State-of-the-art machine learning, for computer vision applications, is based on data-driven feature learning. While these extraction paradigms often yield impressive results that outperform human hand-crafted solutions, they unfortunately suffer from a lack of explainability. In response to this, both the neural network and evolutionary communities have provided techniques tailored to visually explain how machines process new observations. Examples range from gradient-weighted class activation mapping to guided backpropagation, and from convolutional matrix transpose to improved evolutionary constructed computing. Previously, we put forth a framework called the Adversarial Modifier Set (AMS). In AMS, adversarial imagery is generated based on evolutionary feature identified regions. In this article, we seek to improve both AMS and general adversarial systems by performing a comparative analysis between neural attention modeling techniques and our previously used evolutionary strategy. Preliminary results on a computer vision dataset show that while the neural techniques are faster, evolutionary algorithms yield diverse and higher fidelity attention maps that give rise to improved features for adversarial learning.

Three dimensional point cloud data sets are easy to acquire and manipulate, but are often too large to process directly for embedded real-time applications. The spatial information in a point cloud can be represented in a variety of reduced forms, such as voxel grids, Gaussian mixture models, or spatial semantic structures. In this article, we show how a segmented point cloud can be represented as a spatial relationship graph using bounding boxes and triangular fuzzy numbers. This model is a lightweight encoding of the relative distance and direction between objects, and can be used to describe and query for particular spatial configurations using linguistic terms in a multicriteria framework. We show how this approach can be applied on a hand-segmented subset of the NPM3D data set with several illustrative examples. The work herein has useful applications in many applied domains, such as human-robot interaction with unmanned aerial systems.

This paper focuses on a powerful nonlinear aggregation function, the Choquet integral (ChI). Specifically, we focus on situations where the parameters of the ChI are learned from data. For N inputs, the ChI breaks down into N! underlying linear convex sums (LCSs) with 2Nshared variables. Typically, these LCSs are reducible into a drastically smaller number of linear order statistics (LOSs). In the spirit of explainable AI (XAI), our goal is to discover the minimal underlying operator structure of a learned ChI to be conveyed to its users. The challenge is, there does not appear to be widespread research or agreement regarding how to compute similarity within and between measures or integrals. In this paper, we explore the earth mover's distance (EMD), a parametric cross-bin measure, to capture semantic relatedness between LOSs. EMD is used to measure dissimilarity between integrals. In the case of a single ChI, underlying aggregation operator structure is discovered via EMD and clustering. A combination of synthetic and real-world experiments are provided to demonstrate interpretability and reduction of complexity.

The Choquet integral (ChI) is an aggregation operator defined with respect to a fuzzy measure (FM). The FM encodes the worth of all subsets of the sources of information that are being aggregated. The ChI is capable of representing many aggregation functions and has found its application in a wide range of decision fusion problems. In our prior work, we introduced a data support-based approach for learning the FM for decision fusion problems. This approach applies a quadratic programming (QP)-based method to train the FM. However, since the FM of ChI scales as$2^{N}$, where$N$is the number of input sources, the space complexity for learning the FM grows exponentially with$N$. This has limited the practical application of ChI-based decision fusion methods to small numbers of dimenstions—$N$≲ 6 is practical in most cases. In this work, we propose an iterative gradient descent-based approach to train the FM for ChI with an efficient method for handling the FM constraints. This method processes the training data, one observation at a time, and thereby significantly reduces the space complexity of the training process. We tested our online method on synthetic and real-world data sets, and compared the performance and convergence behaviour with our previously proposed QP-based method (i.e., batch method). On 10 out of 12 data sets, the online learning method has either matched or outperformed the batch method. We also show that we are able to use larger numbers of inputs with the online learning approach, extending the practical application of the ChI.

The adoption of artificial intelligence (AI) into domains that impact human life (healthcare, agriculture, security and defense, etc.) has led to an increased demand for explainable AI (XAI). Herein, we focus on an under represented piece of the XAI puzzle, information fusion. To date, a number of low-level XAI explanation methods have been proposed for the fuzzy integral (FI). However, these explanations are tailored to experts and its not always clear what to do with the information they return. In this article we review and categorize existing FI work according to recent XAI nomenclature. Second, we identify a set of initial actions that a user can take in response to these low-level statistical, graphical, local, and linguistic XAI explanations. Third, we investigate the design of an interactive user friendly XAI report. Two case studies, one synthetic and one real, show the results of following recommended actions to understand and improve tasks involving classification.

Mathematical morphology has been explored in deep learning architectures, as a substitute to convolution, for problems like pattern recognition and object detection. One major advantage of using morphology in deep learning is the utility of morphological erosion and dilation. Specifically, these operations naturally embody interpretability due to their underlying connections to the analysis of geometric structures. While the use of these operations results in explainable learned filters, morphological deep learning lacks attribution modeling, i.e., a paradigm to specify what areas of the original observed image are important. Furthermore, convolution-based deep learning has achieved attribution modeling through a variety of neural eXplainable Artificial Intelligence (XAI) paradigms (e.g., saliency maps, integrated gradients, guided backpropagation, and gradient class activation mapping). Thus, a problem for morphology-based deep learning is that these XAI methods do not have a morphological interpretation due to the differences in the underlying mathematics. Herein, we extend the neural XAI paradigm of saliency maps to morphological deep learning, and by doing, so provide an example of morphological attribution modeling. Furthermore, our qualitative results highlight some advantages of using morphological attribution modeling.

Zadeh's extension principle (ZEP) is a fundamental concept in fuzzy set (FS) theory that enables crisp mathematical operation on FSs. A well-known shortcoming of ZEP is that the height of the output FS is determined by the lowest height of the input FSs. In this article, we introduce a generalized extension principle (GEP) that eliminates this weakness and provides flexibility and control over how membership values are mapped from input to output. Furthermore, we provide a computationally efficient point-based FS representation. In light of our new definition, we discuss two approaches to perform aggregation of FSs using the Choquet integral. The resultant integrals generalize prior work and lay a foundation for future extensions. Last, we demonstrate the extended integrals via a combination of synthetic and real-world examples.

Fuzzy integrals (FIs) are powerful aggregation operators that fuse information from multiple sources. The aggregation is parameterized using a fuzzy measure (FM), which encodes the worths of all subsets of sources. Since the FI is defined with respect to an FM, much consideration must be given to defining the FM. However, in practice this is a difficult task—the number of values in an FM scales as$2^n$, where$n$is the number of input sources, thus manually specifying an FM quickly becomes tedious. In this article, we review an automatic, data-supported method of learning the FM by minimizing a sum-of-squared error objective function in the context of decision-level fusion of classifiers using the Choquet FI. While this solves the specification problem, we illuminate an issue encountered with many real-world data sets; i.e., if the training data do not contain a significant number of all possible sort orders, many of the FM values are not supported by the data. We propose various regularization strategies to alleviate this issue by pushing the learned FM toward a predefined structure; these regularizers allow the user to encode knowledge of the underlying FM to the learning problem. Furthermore, we propose another regularization strategy that constrains the learned FM's structure to be a linear order statistic. Finally, we perform several experiments using synthetic and real-world data sets and show that our proposed extensions can improve the learned FM behavior and classification accuracy. A previously proposed visualization technique is employed to simultaneously quantitatively illustrate the FM as well as the FI.

While most deep learning architectures are built on convolution, alternative foundations such as morphology are being explored for purposes such as interpretability and its connection to the analysis and processing of geometric structures. The morphological hit-or-miss operation has the advantage that it considers both foreground information and background information when evaluating the target shape in an image. In this article, we identify limitations in the existing hit-or-miss neural definitions and formulate an optimization problem to learn the transform relative to deeper architectures. To this end, we model the semantically important condition that the intersection of the hit and miss structuring elements (SEs) should be empty and present a way to express Don't Care (DNC), which is important for denoting regions of an SE that are not relevant to detecting a target pattern. Our analysis shows that convolution, in fact, acts like a hit-to-miss transform through semantic interpretation of its filter differences. On these premises, we introduce an extension that outperforms conventional convolution on benchmark data. Quantitative experiments are provided on synthetic and benchmark data, showing that the direct encoding hit-or-miss transform provides better interpretability on learned shapes consistent with objects, whereas our morphologically inspired generalized convolution yields higher classification accuracy. Finally, qualitative hit and miss filter visualizations are provided relative to single morphological layer.

Fuzzy integrals are useful general purpose aggregation operators, but they can be difficult to understand and visualize in practice. The interaction between an exponentially increasing number of variables-2nfuzzy measure variables for n inputs-makes it hard to understand what exactly is going on in a high dimensional space. We propose a new visualization scheme based on a weighted indicator matrix to better understand the inner workings of an arbitrary fuzzy measure. We provide ways of viewing the Shapley and interaction indices, as well as an optional data coverage histogram. This approach can give insight into which sources are the most relevant in the overall aggregation and decision making process, and it provides a way to visually compare fuzzy measures and subsequently integrals.

The ordered weighted average (OWA) operator is a well-known aggregation tool that is primarily used for decisionlevel fusion. However, the OWA is a convex sum, i.e., its learned coefficients are constrained to sum to one, and thus the output is restricted to lie between the maximum and minimum values of the inputs. Relaxing this constraint on the sum of weights transforms the OWA into a linear order statistic (LOS), which allows the aggregation operation to map the input to any value on the set of reals, thus behaving more like a regression operator. The LOS parameterizes the regression operation of d-features using just d parameters, which helps with the model's interpretability. However, learning just d parameters limits the amount of nonlinear space explored for an optimal solution, and thus reduces the expressibility of the LOS algorithm. We propose a novel aggregation method called the extended linear order statistic (ELOS), where for each position in the sorted input vector we have d parameters, one for each input feature, thus learning a total of d2weights for the aggregation of d features. The increased number of parameters helps the algorithm improve its expressibility while maintaining interpretability. In our experiments on real-world benchmark data sets, ELOS has outperformed both linear regression and LOS in 8 out of 10 experiments.

The Choquet integral (ChI) is an aggregation function that is defined with respect to a fuzzy measure (FM). Many ChI-based decision aggregation methods have been proposed to learn the underlying FM. However, FM's boundary and monotonicity constraints have limited the applicability of such methods to decision-level fusion. In a recent work, we removed the constraints on FM to develop a regression model based on the ChI. Our model has a generalized bias that enables capability beyond previously proposed ChI regression approaches. We also developed an approach for learning the parameters of the ChI regression from training data. In this paper, we develop a method to apply ℓ2-regularization on our training algorithm. In our experiments on real-world benchmark data sets, ridge regularized-ChI regression has outperformed the unregularized version in 22 out of 30 (73%) experiments. Also, when compared with several competing regression methods, results show that our approach has superior performance.

Artificial neural networks are a dominant force in our modern era of data-driven artificial intelligence. The adaptive neuro fuzzy inference system (ANFIS) is a neural network based on fuzzy logic versus a more traditional premise like convolution. Advantages of ANFIS include the ability to encode and potentially understand machine learned neural information in the pursuit of explainable, interpretable, and ultimately trustworthy artificial intelligence. However, real-world data is almost always imperfect, e.g., incomplete or noisy, and ANFIS is not naturally robust. Specifically, ANFIS is susceptible to over inflated uncertainty, poor antecedent (fuzzy set) data alignment, degenerate optimization conditions, and hard to interpret logic, to name a few factors. Herein, we explore the use of possibilistic clustering to identify outliers, specifically typicality degrees, to increase the robustness of ANFIS; or any fuzzy logic neuron/network. Experiments are presented that demonstrate the need and quality of the proposed solutions in the pursuit of robust interpretable machine learned neuro fuzzy logic solutions.

Deep convolutional neural networks (DCNN) have been the dominant methodology in the field of computer vision over the last decade, using various architectural organizations of successive convolutional layers to extract and assemble low level image features into visual component detectors. One of the tradeoffs that have been made as the community has migrated to deep neural models is the loss of explainability and understanding of which salient visual components are being recognized by a model for a particular task. However, there exists a significant heritage in the remote sensing community that has developed advanced algorithms to analyze the signal and structural characteristics of anthropogenic features. One such approach is the use of morphological image processing techniques to extract objects from imagery and aid in the structural analysis of shapes. In particular, the differential morphological profile (DMP) has had great success extracting object shapes, while naturally grouping the extracted shapes into scale ranges. In this research, we present a novel architecture that integrates an explicit (definable and explainable) scaled object extraction into the network architecture, allowing shallower convolutional layers and lower complexity neural models. The architecture is evaluated on a challenging remote sensing dataset of object classes, providing insights to this approach and illuminating future directions of integrating morphology into neural architectures for enhanced explainability.

The Choquet Integral (ChI) is an aggregation operator defined with respect to a Fuzzy Measure (FM). The FM encodes the worth of all subsets of the sources of information that are being aggregated. The monotonicity and the boundary conditions of the FM have limited its applicability to decision-level fusion. But in a recent work, we removed the boundary and monotonicity constraints of the FM, which we then called a bounded capacity (BC), to propose a Choquet Integral regression (CIR) approach that enables capability beyond previously proposed ChI regression methods. In the same work, we also presented a quadratic programming (QP)-based method, batch-CIR, to learn the BC parameters of the CIR from training data. However, the QP used for learning the BC scales exponentially with the dimensionality of the training data and thus it becomes impractical on data sets with 7 or more dimensions. In this paper we propose an iterative gradient descent approach, online-CIR, to learn the BC. This method iteratively processes the training data, one data point at a time, and therefore requires significantly less computation and space at any time during the training. The application of batch-CIR required the dimensionality reduction of high-dimensional data sets to enable computation in a reasonable time. The proposed online-CIR approach has enabled us to extend CIR to data sets with larger dimensionality. In our experimental evaluation using benchmark regression data sets, online-CIR has outperformed batch-CIR on high-dimensional data sets while also matching the batch-CIR performance on low-dimensional data sets.

Information fusion is an essential part of numerous engineering systems and biological functions, e.g., human cognition. Fusion occurs at many levels, ranging from the low-level combination of signals to the high-level aggregation of heterogeneous decision-making processes. While the last decade has witnessed an explosion of research in deep learning, fusion in neural networks has not observed the same revolution. Specifically, most neural fusion approaches are ad hoc, are not understood, are distributed versus localized, and/or explainability is low (if present at all). Herein, we prove that the fuzzy Choquet integral (ChI), a powerful nonlinear aggregation function, can be represented as a multilayer network, referred to hereafter as ChIMP. We also put forth an improved ChIMP (iChIMP) that leads to a stochastic-gradient-descent-based optimization in light of the exponential number of ChI inequality constraints. An additional benefit of ChIMP/iChIMP is that it enables explainable artificial intelligence (XAI). Synthetic validation experiments are provided, and iChIMP is applied to the fusion of a set of heterogeneous architecture deep models in remote sensing. We show an improvement in model accuracy, and our previously established XAI indices shed light on the quality of our data, model, and its decisions.

With a growing number of areas leveraging interval-valued data-including in the context of modeling human uncertainty (e.g., in cybersecurity), the capacity to accurately and systematically compare intervals for reasoning and computation is increasingly important. In practice, well established set-theoretic similarity measures, such as the Jaccard and Sørensen-Dice measures, are commonly used, whereas axiomatically, a wide breadth of possible measures have been theoretically explored. This article identifies, articulates, and addresses an inherent and so far not discussed limitation of popular measures-their tendency to be subject to aliasing-where they return the same similarity value for very different sets of intervals. The latter risks counter-intuitive results and poor-automated reasoning in real-world applications dependent on systematically comparing interval-valued system variables or states. Given this, we introduce new axioms establishing desirable properties for robust similarity measures, followed by putting forward a novel set-theoretic similarity measure based on the concept of bidirectional subsethood, which satisfies both traditional and new axioms. The proposed measure is designed to be sensitive to the variation in the size of intervals, thus avoiding aliasing. This article provides a detailed theoretical exploration of the new proposed measure, and systematically demonstrates its behavior using an extensive set of synthetic and real-world data. Specifically, the measure is shown to return robust outputs that follow intuition-essential for real-world applications. For example, we show that it is bounded above and below by the Jaccard and Sørensen-Dice similarity measures (when the minimum t-norm is used). Finally, we show that a dissimilarity or distance measure, which satisfies the properties of a metric, can easily be derived from the proposed similarity measure.

Feature extraction is widely considered one of the most critical components to classification performance in computer vision. In the past, human-designed features, such as the histogram of oriented gradients, were used for extracting statistically rich features. Recently, there has been a movement away from human-designed features to machine-learned features. Herein, we propose a novel genetic programming (GP) approach, coined GOOFeD, to automatically generate discriminative-rich features for image classification. This is achieved by greatly advancing GP in three ways: (1) promoting population diversity and redundancy removal, (2) introducing a unique adaptive mutation approach, and (3) controlling tree bloat through a new crossover technique. These improvements also lead to a population size required for learning that is smaller than that commonly used in the literature. To assess performance, GOOFeD is tested on the MIT urban and nature scene data set and a real-world buried explosive hazard data set. Experiments verify that, in terms of classification accuracy, GOOFeD outperforms many of the state-of-the-art human-designed features and feature learning techniques.

Regression is the process of learning the relationship between sets of variables, enabling predictions of continuous output variables. Many approaches have been proposed to learn parameterized models with respect to numerous error metrics. In this paper, we propose a regression model based on the Choquet integral with respect to a bounded capacity (of which fuzzy measures are a subset). Our model has a generalized bias that enables capability beyond previously proposed Choquet integral regression approaches. We also develop an approach for learning the parameters of the Choquet integral regression from training data. Simulated and real-world benchmark data are used to demonstrate the performance of our regression approach compared with several competing regression methods. Results show that our approach has superior performance and is computationally very fast. An additional benefit of our Choquet integral regression is that it enables interpretability of the learned model, which we will explore in later works.

Discontinuous intervals (DIs) arise in a wide range of contexts, from real world data capture of human opinion to α-cuts of non-convex fuzzy sets. Commonly, for assessing the similarity of DIs, the latter are converted into their continuous form, followed by the application of a continuous interval (CI) compatible similarity measure. While this conversion is efficient, it involves the loss of discontinuity information and thus limits the accuracy of similarity results. Further, most similarity measures including the most popular ones, such as Jaccard and Dice, suffer from aliasing, that is, they are liable to return the same similarity for very different pairs of CIs. To address both of these challenges, this paper proposes a generalized approach for calculating the similarity of DIs which leverages the recently introduced bidirectional subsethood based similarity measure (which avoids aliasing) while accounting for all pairs of the continuous subintervals within the DIs to be compared. We provide detail of the proposed approach and demonstrate its behaviour when applying bidirectional subsethood, Jaccard and Dice as similarity measures, using different pairs of synthetic DIs. The experimental results show that the similarity outputs of the new generalized approach follow intuition for all three similarity measures; however, it is only the proposed integration with the bidirectional subsethood similarity measure which also avoids aliasing for DIs.

The Choquet integral (ChI) is a proven tool for information aggregation. In prior work, we showed that learning a ChI from data results in missing variables. Herein, we explore two ways to transfer a known ChI from a source domain to a new under sampled target domain. The first method is based on regularization and it listens to the full source domain ChI. The second method optimizes what we can observe (target domain supported variables) and missing variables are the only thing migrated from the source domain. Synthetic experiments, aka we know the truth, are used to show the behavior of these methods with respect to transfering between ChIs.

Many state-of-the-art technologies developed in recent years have been influenced by machine learning to some extent. Most popular at the time of this writing are artificial intelligence methodologies that fall under the umbrella of deep learning. Deep learning has been shown across many applications to be extremely powerful and capable of handling problems that possess great complexity and difficulty. In this work, we introduce a new layer to deep learning: the fuzzy layer. Traditionally, the network architecture of neural networks is composed of an input layer, some combination of hidden layers, and an output layer. We propose the introduction of fuzzy layers into the deep learning architecture to exploit the powerful aggregation properties expressed through fuzzy methodologies, such as the Choquet and Sugueno fuzzy integrals. To date, fuzzy approaches taken to deep learning have been through the application of various fusion strategies at the decision level to aggregate outputs from state-of-the-art pre-trained models, e.g., AlexNet, VGG16, GoogLeNet, Inception-v3, ResNet-18, etc. While these strategies have been shown to improve accuracy performance for image classification tasks, none have explored the use of fuzzified intermediate, or hidden, layers. Herein, we present a new deep learning strategy that incorporates fuzzy strategies into the deep learning architecture focused on the application of semantic segmentation using per-pixel classification. Experiments are conducted on a benchmark data set as well as a data set collected via an unmanned aerial system at a U.S. Army test site for the task of automatic road segmentation, and preliminary results are promising.

Herein, a generalization of the ordered weighted average (OWA) is put forth relative to pattern recognition. The resultant linear order statistic neuron (LOSN) is unique in that it bridges fuzzy sets, specifically fuzzy data/information aggregation, with neural networks. This article discusses the gradient descent-based optimization and geometric interpretation of the LOSN. An advantage is that the LOSN is an efficient shared weight encoding of N! perceptrons, relative to N inputs. Open source codes are provided to facilitate reproducible research. Experiments are conducted to both validate the method and show its non-linear geometric expression.

Information fusion is an essential part of nearly all systems whose goal is to derive decisions from multiple sources. Often, a fusion solution has parameters and the goal is to learn them from data. Herein, we propose efficient evolutionary algorithm (EA) operators to facilitate learning the Choquet integral (ChI). Whereas many EAs provide a way to solve complex, unconstrained optimization tasks, most tend to perform relatively poor in light of constraints. Recently, a few EA-based approaches to optimizing the ChI have appeared. Namely, these methods focus on fixing the values of variables so conditions are met or feasible candidate pairs are identified for steps such as crossover. Herein, we introduce a new set of transparent operators that are guaranteed to naturally preserve constraints, thus eliminating the need to resort to costly evaluations and fixing of constraint violations. In particular, our method scales well to large numbers of inequality constraints, something that prior work does not. The proposed algorithm, coined efficient ChI genetic algorithm (ECGA), is evaluated on several synthetic data sets and it is compared with state-of-the-art algorithms. In particular, we show benefits in terms of solutions found and the time it takes to find such an answer.

Information or data aggregation is an important part of nearly all analysis problems as summarizing inputs from multiple sources is a ubiquitous goal. In this paper we propose a method for non-linear aggregation of data inputs that take the form of non-normal fuzzy sets. The proposed shape-preserving fuzzy integral (SPFI) is designed to overcome a well-known weakness of the previously-proposed sub-normal fuzzy integral (SuFI). The weakness of SuFI is that the output is constrained to have maximum membership equal to the minimum of the maximum memberships of the inputs; hence, if one input has a small height, then the output is constrained to that height. The proposed SPFI does not suffer from this weakness and, furthermore, preserves in the output the shape of the input sets. That is, the output looks like the inputs. The SPFI method is based on the well-known Choquet fuzzy integral with respect to a capacity measure, i.e., fuzzy measure. We demonstrate SPFI on synthetic and real-world data, comparing it to the SuFI and non-direct fuzzy integral (NDFI).

Similarity measures are useful for reasoning about fuzzy sets. Hence, many classical set-theoretic similarity measures have been extended for comparing fuzzy sets. In previous work, a set-theoretic similarity measure considering the bidirectional subsethood for intervals was introduced. The measure addressed specific concerns of many common similarity measures, and it was shown to be bounded above and below by Jaccard and Dice measures respectively. Herein, we extend our prior measure from similarity on intervals to fuzzy sets. Specifically, we propose a vertical-slice extension where two fuzzy sets are compared based on their membership values. We show that the proposed extension maintains all common properties (i.e., reflexivity, symmetry, transitivity, and overlapping) of the original fuzzy similarity measure. We demonstrate and contrast its behaviour along with common fuzzy set-theoretic measures using different types of fuzzy sets (i.e., normal, non-normal, convex, and non-convex) in respect to different discretization levels.

To date, numerous ways have been created to learn a fusion solution from data. However, a gap exists in terms of understanding the quality of what was learned and how trustworthy the fusion is for future-i.e., new-data. In part, the current paper is driven by the demand for so-called explainable AI (XAI). Herein, we discuss methods for XAI of the Choquet integral (ChI), a parametric nonlinear aggregation function. Specifically, we review existing indices, and we introduce new data-centric XAI tools. These various XAI-ChI methods are explored in the context of fusing a set of heterogeneous deep convolutional neural networks for remote sensing.

Deep learning usually requires big data, with respect to both volume and variety. However, most remote sensing applications only have limited training data, of which a small subset is labeled. Herein, we review three state-of-the-art approaches in deep learning to combat this challenge. The first topic is transfer learning, in which some aspects of one domain, e.g., features, are transferred to another domain. The next is unsupervised learning, e.g., autoencoders, which operate on unlabeled data. The last is generative adversarial networks, which can generate realistic looking data that can fool the likes of both a deep learning network and human. The aim of this article is to raise awareness of this dilemma, to direct the reader to existing work and to highlight current gaps that need solving.

Here we present techniques and algorithms to apply trained deep convolutional neural networks (DCNN) to high-resolution remote sensing imagery datasets covering large areas of the Earth. First, trained DCNN are used to process broad swaths of imagery in an area of interest (AOI) to produce a classification vector response field (CVRF). The CVRF is then aggregated using mode-seeking algorithms to detect potential objects of interest within the AOI. Our research explores the challenges and opportunities of transitioning DCNN out of the training-validation laboratory setting and into the real-world application domain. We show a scalable approach to leverage state-of-the-art DCNN for broad area automated search, detection, and annotation of objects such as tennis courts, storage tanks, runways, and airplanes.

Accurate land cover classification and detection of objects in high-resolution electro-optical remote sensing imagery (RSI) have long been a challenging task. Recently, important new benchmark data sets have been released which are suitable for land cover classification and object detection research. Here, we present state-of-the-art results for four benchmark data sets using a variety of deep convolutional neural networks (DCNN) and multiple network fusion techniques. We achieve 99.70%, 99.66%, 97.74%, and 97.30% classification accuracies on the PatternNet, RSI-CB256, aerial image, and RESISC-45 data sets, respectively, using the Choquet integral with a novel data-driven optimization method presented in this letter. The relative reduction in classification errors achieved by this data driven optimization is 25%-45% compared with the single best DCNN results.

The Choquet integral (ChI) is a parametric nonlinear aggregation function defined with respect to the fuzzy measure (FM). To date, application of the ChI has sadly been restricted to problems with relatively few numbers of inputs; primarily as the FM has 2Nvariables for N inputs and N(2N-1- 1) monotonicity constraints. In return, the community has turned to density-based imputation (e.g., Sugeno λ-FM) or the number of interactions (FM variables) are restricted (e.g., k-additivity). Herein, we propose a new scalable data-driven way to represent and learn the ChI, making learning computationally manageable for larger N. First, data supported variables are identified and used in optimization. Identification of these variables also allows us recognize future ill-posed fusion scenarios; ChIs involving variable subsets not supported by data. Second, we outline an imputation function framework to address data unsupported variables. Third, we present a lossless way to compress redundant variables and associated monotonicity constraints. Finally, we outline a lossy approximation method to further compress the ChI (if/when desired). Computational complexity analysis and experiments conducted on synthetic datasets with known FMs demonstrate the effectiveness and efficiency of the proposed theory.

It is widely accepted that feature extraction is quite possibly the most critical step in computer vision. Typically, feature extraction is performed using a method such as the histogram of oriented gradients. In recent years, a shift has occurred from human to machine learned features, e.g., convolutional neural networks (CNNs) and Evolution-Constructed (ECO) features. An advantage of our improved ECO (iECO) framework is it optimizes features on a per-descriptor basis. Herein, iECO is extended in order to represent a richer class of features, namely arithmetic combinations and compositions of iECOs. This extension, called Genetic programming Optimal Feature Descriptor (GOOFeD) is based on genetic programming (GP). Three experiments are performed on data from a U.S. Army test site that contains multiple target and clutter types, burial depths, and times of day for automatic buried explosive hazard detection. The first two experiments focus on GOOFeD initialization and parameter selection. The last experiment demonstrates that GOOFeD is superior to iECO in terms of the fitness of evolved individuals.

Recently, there has been much research into effective representation and analysis of uncertainty in human responses, with applications in cyber-security, forest and wildlife management, and product development, to name a few. Most of this research has focused on representing the response uncertainty as intervals, e.g., “I give the movie between 2 and 4 stars.” In this paper, we extend upon the model-based interval agreement approach (lAA) for combining interval data into fuzzy sets and propose the efficient IAA (eIAA) algorithm, which enables efficient representation of and operation on the fuzzy sets produced by IAA (and other interval-based approaches, for that matter). We develop methods for efficiently modeling, representing, and aggregating both crisp and uncertain interval data (where the interval endpoints are intervals themselves). These intervals are assumed to be collected from individual or multiple survey respondents over single or repeated surveys; although, without loss of generality, the approaches put forth in this paper could be used for any interval-based data where representation and analysis is desired. The proposed method is designed to minimize loss of information when transferring the interval-based data into fuzzy set models and then when projecting onto a compressed set of basis functions. We provide full details of eIAA and demonstrate it on real-world and synthetic data.

Numerous applications in engineering are plagued by incomplete data. The subject explored in this article is how to extend the fuzzy integral (FI), a parametric nonlinear aggregation function, to missing data. We show there is no universally correct solution. Depending on context, different types of uncertainty are present and assumptions are applicable. Two major approaches exist, use just observed data or model/impute missing data. Three extensions are put forth with respect to just use observed data and a two step process, modeling/imputation and FI extension, is proposed for using missing data. In addition, an algorithm is proposed for learning the FI relative to missing data. The impact of using and not using modeled/imputed data relative to different aggregation operators-selections of underlying fuzzy measure (capacity)-are also discussed. Last, a case study and data-driven learning experiment are provided to demonstrate the behavior and range of the proposed concepts.

In computing the similarity of intervals, current similarity measures such as the commonly used Jaccard and Dice measures are at times not sensitive to changes in the width of intervals, producing equal similarities for substantially different pairs of intervals. To address this, we propose a new similarity measure that uses a bi-directional approach to determine interval similarity. For each direction, the overlapping ratio of the given interval in a pair with the other interval is used as a measure of uni-directional similarity. We show that the proposed measure satisfies all common properties of a similarity measure, while also being invariant in respect to multiplication of the interval endpoints and exhibiting linear growth in respect to linearly increasing overlap. Further, we compare the behavior of the proposed measure with the highly popular Jaccard and Dice similarity measures, highlighting that the proposed approach is more sensitive to changes in interval widths. Finally, we show that the proposed similarity is bounded by the Jaccard and the Dice similarity, thus providing a reliable alternative.

The fuzzy integral (FI) is a nonlinear aggregation operator whose behavior is defined by the fuzzy measure (FM). As an aggregation operator, the FI is commonly used for evidence fusion where it combines sources of information based on the worth of each subset of sources. One drawback to FI-based methods, however, is the specification of the FM. Defining the FM manually quickly becomes too tedious since the number of FM terms scales as 2n, where n is the number of sources; thus, an automatic method of defining the FM is necessary. In this paper, we review a data-driven method of learning the FM via minimizing the sum-of-squared error (SSE) in the context of decision-level fusion and propose an extension allowing knowledge of the underlying FM to be encoded in the algorithm. The algorithm is applied to real-world and toy datasets and results show that the extension can improve classification accuracy. Furthermore, we introduce a visualization strategy to simultaneously show the quantitative information in the FM as well as the FI.

While the Choquet integral (Chi) is a powerful parametric nonlinear aggregation function, it has limited scope and is not a universal function generator. Herein, we focus on a class of problems that are outside the scope of a single Chi. Namely, we are interested in tasks where different subsets of inputs require different Chls. Herein, a genetic program (GF) is used to extend the Chi, referred to as GpChI hereafter, specifically in terms of compositions of Chls and/or arithmetic combinations of Chls. An algorithm is put forth to leam the different GP Chls via genetic algorithm (GA) optimization. Synthetic experiments demonstrate GpChI in a controlled fashion, i.e., we know the answer and can compare what is learned to the truth. Real-world experiments are also provided for the mult-sensor fusion of electromagnetic induction (EMI) and ground penetrating radar (GPR) for explosive hazard detection. Our mutli-sensor fusion experiments show that there is utility in changing aggregation strategy per different subsets of inputs (sensors or algorithms) and fusing those results.

The aggregation of multiple information sources has a long history and ranges from sensor fusion to the aggregation of individual algorithm outputs and human knowledge. A popular approach to achieve such aggregation is the fuzzy integral (FI) which is defined with respect to a fuzzy measure (FM) (i.e. a normal, monotone capacity). In practice, the discrete FI aggregates information contributed by a discrete number of sources through a weighted aggregation (post-sorting), where the weights are captured by a FM that models the typically subjective `worth' of subsets of the overall set of sources. While the combination of FI and FM has been very successful, challenges remain both in regards to the behavior of the resulting aggregation operators - which for example do not produce symmetrically mirrored outputs for symmetrically mirrored inputs - and also in a manifest difference between the intuitive interpretation of a stand-alone FM and its actual role and impact when used as part of information fusion with a FI. This paper elucidates these challenges and introduces a novel family of recursive average (RAV) operators as an alternative to the FI in aggregation with respect to a FM; focusing specifically on the arithmetic recursive average. The RAV is designed to address the above challenges, while also facilitating fine-grained analysis of the resulting aggregation of different combinations of sources. We provide the mathematical foundations of the RAV and include initial experiments and comparisons to the FI for both numeric and interval-valued data.

Kernel methods for classification is a well-studied area in which data are implicitly mapped from a lower-dimensional space to a higher dimensional space to improve classification accuracy. However, for most kernel methods, one must still choose a kernel to use for the problem. Since there is, in general, no way of knowing which kernel is the best, multiple kernel learning (MKL) is a technique used to learn the aggregation of a set of valid kernels into a single (ideally) superior kernel. The aggregation can be done using weighted sums of the precomputed kernels, but determining the summation weights is not a trivial task. Furthermore, MKL does not work well with large datasets because of limited storage space and prediction speed. In this paper, we address all three of these multiple kernel challenges. First, we introduce a new linear feature level fusion technique and learning algorithm, GAMKLp. Second, we put forth three new algorithms, DeFIMKL, DeGAMKL, and DeLSMKL, for nonlinear fusion of kernels at the decision level. To address MKL's storage and speed drawbacks, we apply the Nystrom approximation to the kernel matrices. We compare our methods to a successful and state-of-the-art technique called MKL-group lasso (MKLGL), and experiments on several benchmark datasets show that some of our proposed algorithms outperform MKLGL when applied to support vector machine (SVM)-based classification. However, to no surprise, there does not seem to be a global winner but instead different strategies that a user can employ. Experiments with our kernel approximation method show that we can routinely discard most of the training data and at least double prediction speed without sacrificing classification accuracy. These results suggest that MKL-based classification techniques can be applied to big data efficiently, which is confirmed by an experiment using a large dataset.

The Multiple Instance Choquet integral (MICI) for classifier fusion and an evolutionary algorithm for parameter estimation is presented. The Choquet integral has a long history of providing an effective framework for non-linear fusion. However, previous methods to learn an appropriate measure for the Choquet integral required accurate and precise training labels. In many applications, data-point specific labels are unavailable and infeasible to obtain. The proposed MICI algorithm allows for training with uncertain labels in which class labels are provided for sets of data points (i.e., “bags”) instead of individual data points (i.e., “instances”). The proposed algorithm is able to fuse multiple two-class classifier outputs by learning a monotonic and normalized fuzzy measure from uncertain training labels using an evolutionary algorithm. It produces enhanced classification performance by computing Choquet integral with the learned fuzzy measure. Results on both simulated and real hyperspectral data are presented in the paper.

Herein, we explore both a new supervised and unsupervised technique for dimensionality reduction or multispectral sensor design via band group selection in hyperspectral imaging. Specifically, we investigate two algorithms, one based on the improved visual assessment of clustering tendency (iVAT) and the other based on the automatic extraction of “blocklike” structure in a dissimilarity matrix (CLODD algorithm). In particular, the iVAT algorithm allows for identification of non-contiguous band groups. Experiments are conducted on a benchmark data set and results are compared to existing algorithms based on hierarchical and c-means clustering. Our results demonstrate the effectiveness of the proposed method.

Ontologies have been widely used as a knowledge representation framework, and numerous methods have been put forth to match ontologies. It is well known that ontology matchers behave differently in various domains, and it is a challenge to predict or characterize their behavior. Herein, a hybrid expertise-agreement aggregation strategy is proposed. Although others rely on the existence of a reference ontology, this typically does not exist in the real world. In this article, the fuzzy integral (FI) is used to aggregate multiple ontology matchers in lieu of a reference ontology. Specifically, we present a measure of expertise and fuse it with our previous agreement measure that is motivated by crowd sourcing to improve recall. This way, any available domain knowledge, in terms of partial ordering of a subset of inputs, can be included in the decision-making process. By adding the domain knowledge to the agreement model, we are able to reach the upmost performance. Preliminary results demonstrate the robustness of our approach across domains. Sensitivity analysis is also provided, which shows the limits to which extreme destructive expertise affects system performance.

The aim of this paper is the simultaneous minimization of model error and model complexity for the Choquet integral. The Choquet integral is a generator function, that is, a parametric function that yields a wealth of aggregation operators based on the specifics of the underlying fuzzy measure (aka normal and monotonic capacity). It is often the case that we desire to learn an aggregation operator from data and the goal is to have the smallest possible sum of squared error (SSE) between the trained model and a set of labels or function values. However, we also desire to learn the “simplest” solution possible, viz., the model with the fewest number of inputs. Previous works focused on the use of l1-norm regularization of a lexicographically encoded capacity vector relative to the Choquet integral, describing how to carry out the procedure and demonstrating encouraging results. However, no characterization or insights into the capacity and integral were provided. Herein, we investigate the impact of l1-norm regularization of a lexicographically encoded capacity vector in terms of what capacities and aggregation operators it strives to induce in different scenarios. Ultimately, this provides insight into what the regularization is really doing and when to apply such a method. Synthetic experiments are performed to illustrate the remarks, propositions, and concepts put forth.

Kernel methods for classification is a well-studied area in which data are implicitly mapped from a lower-dimensional space to a higher-dimensional space to improve classification accuracy. However, for most kernel methods, one must still choose a kernel to use for the problem. Since there is, in general, no way of knowing which kernel is the best, multiple kernel learning (MKL) is a technique used to learn the aggregation of a set of valid kernels into a single (ideally) superior kernel. The aggregation can be done using weighted sums of the pre-computed kernels, but determining the summation weights is not a trivial task. A popular and successful approach to this problem is MKL-group lasso (MKLGL), where the weights and classification surface are simultaneously solved by iteratively optimizing a min-max optimization until convergence. In this work, we propose an ℓp-normed genetic algorithm MKL (GAMKLp), which uses a genetic algorithm to learn the weights of a set of pre-computed kernel matrices for use with MKL classification. We prove that this approach is equivalent to a previously proposed fuzzy integral aggregation of multiple kernels called fuzzy integral: genetic algorithm (FIGA). A second algorithm, which we call decision-level fuzzy integral MKL (DeFIMKL), is also proposed, where a fuzzy measure with respect to the fuzzy Choquet integral is learned via quadratic programming, and the decision value-viz., the class label-is computed using the fuzzy Choquet integral aggregation. Experiments on several benchmark data sets show that our proposed algorithms can outperform MKLGL when applied to support vector machine (SVM)-based classification.

This paper introduces the concept of using a performance measure to select band groups when using a single classifier. Typically, band groups are selected using a proximity measure to determine the similarity or dissimilarity of hyperspectral bands. The problem with that approach is the similarity or dissimilarity of the hyperspectral bands may not be important for some problems. The novelty of using a performance measure is it can be used to select band groups that result in the best performance regardless of the similarity between the band groups. The results demonstrate better overall accuracy using our approach when compared to uniform partitioning. The improved performance is realized when using a Support Vector Machine or Bayes Maximum Likelihood classifier. Often these improvements are achieved using fewer band groups than uniform partitioning.

Object recognition from remote sensing systems is a task of immense interest. With the vast deployment of aerial vehicles and space borne sensors for a wide variety of purposes, it is critical to have robust image processing techniques to analyze massive streams of collected data. Herein, we explore the utility of a feature descriptor learning framework, called improved Evolution-COnstructed (iECO) features. Additionally, an investigation into the combination of iECO features with soft features is conducted. Soft features are a deterministic approach to highlighting pertinent information for improving the quality of features extracted specific to the object of interest while iECO is a way to learn from data the relevant information. Experiments are conducted using four-fold (scene based) cross-validation and are reported in terms of target recognition rates and false alarm rates. Results indicate that iECO features are individually best overall and the combination of iECO and soft features can lead to improved results.

The fuzzy integral (FI) with respect to a fuzzy measure (FM) is a powerful means of aggregating information. The most popular FIs are the Choquet and Sugeno, and most research focuses on these two variants. The arena of the FM is much more populated, including numerically derived FMs such as the Sugeno λ-measure and decomposable measure, expert-defined FMs, and data-informed FMs. The drawback of numerically derived and expert-defined FMs is that one must know something about the relative values of the input sources. However, there are many problems where this information is unavailable, such as crowdsourcing. This paper focuses on data-informed FMs, or those FMs that are computed by an algorithm that analyzes some property of the input data itself, gleaning the importance of each input source by the data they provide. The original instantiation of a data-informed FM is the agreement FM, which assigns high confidence to combinations of sources that numerically agree with one another. This paper extends upon our previous work in datainformed FMs by proposing the uniqueness measure and additive measure of agreement for interval-valued evidence. We then extend data-informed FMs to fuzzy number (FN)-valued inputs. We demonstrate the proposed FMs by aggregating interval and FN evidence with the Choquet and Sugeno FIs for both synthetic and real-world data.

In this paper, a new approach is presented to model interval-based data using fuzzy sets (FSs). Specifically, we show how both crisp and uncertain intervals (where there is uncertainty about the endpoints of intervals) collected from individual or multiple survey participants over single or repeated surveys can be modeled using type-1, interval type-2, or general type-2 FSs based on zSlices. The proposed approach is designed to minimize any loss of information when transferring the interval-based data into FS models, and to avoid, as much as possible, assumptions about the distribution of the data. Furthermore, our approach does not rely on data preprocessing or outlier removal, which can lead to the elimination of important information. Different types of uncertainty contained within the data, namely intra- and inter-source uncertainty, are identified and modeled using the different degrees of freedom of type-2 FSs, thus providing a clear representation and separation of these individual types of uncertainty present in the data. We provide full details of the proposed approach, as well as a series of detailed examples based on both real-world and synthetic data. We perform comparisons with analogue techniques to derive FSs from intervals, namely the interval approach and the enhanced interval approach, and highlight the practical applicability of the proposed approach.

In image processing and computer vision, significant progress has been made in feature learning for exploiting important cues in data that elude non-learned features. While the field of deep learning has demonstrated state-of-the-art performance, the Evolution-COnstructed (ECO) work of Lillywhite et. al has the advantage of interpretability, and it does not predispose the solution to one of convolution. This paper presents a novel approach for extending the ECO framework. We achieve this through two overarching ideas. First, we address a potential major shortcoming of ECO features - the “features” themselves. The so-called ECO features are simply a transformed image that has been unrolled into a large one dimensional vector. We propose employing feature descriptors to extract pertinent information from the ECO imagery. Furthermore, it is our hypothesis that there exists a unique set of transforms for each feature descriptor used on a given problem domain that leads to the descriptors extracting maximal discriminative information. Second, we introduce constraints on each individual's chromosome to promote population diversity and prevent infeasible solutions. We show through experiments that our proposed iECO framework results in, and benefits from, a unique series of transforms for each descriptor being learned and maintaining population diversity.

A number of data-driven fuzzy measure (FM) learning techniques have been put forth for the fuzzy integral (FI). Examples include quadratic programming, Gibbs sampling, gradient descent, reward and punishment and evolutionary optimization. However, most approaches focus solely on the minimization of the sum of squared error (SSE). Limited attention has been placed on characterizing and subsequently minimizing model (i.e., FM) complexity. Furthermore, the vast majority of learning techniques are highly susceptible to over-fitting and noise. Herein, we explore a regularization approach to learning the FM for the Choquet FI. We investigate the mathematical motivation for such an approach, its applicability and impact on different types of FMs, and its desirable properties for quadratic programming (QP) based optimization. We show that L\ regularization has a distinct meaning for measure learning and aggregation operators. Experiments are performed and validated with respect to the Shapley index. Specifically, we show that it is possible to reduce the effect of overfitting, we can identify higher quality measures and, if desired, force the learning of fewer numbers of sources.

Numerous fields require large-scale pattern matching to achieve a variety of computational goals. Herein, we present novel graphics processing unit (GPU) extensions that facilitate high-throughput pattern matching in a PostgreSQL database. We have developed an extension framework to perform data block processing of large pattern data sets, using a stream processing design that results in global k-nearest neighbor matches. This framework was specifically designed to support pattern matching on GPU from within the database environment. This approach avoids the necessity of storing an entire data set onto GPU hardware, which facilitates significant scale-up of pattern databases. This provides enormous potential to incorporate or exploit auxiliary (meta)data as part of the pattern matching process, as well as pipelining the results into traditional relational algebra expressions. By pipelining pattern matching results into a relational expression, the power of the database can be leveraged to build result sets based on various parameterized correlations between the query pattern(s) and the results. In this preliminary work, we have integrated GPU-based high-throughput p-norm metric functions into the database server. This allows one to design heterogeneous data processing techniques that combine large-scale content-based image retrieval (CBIR) with traditional data processing capabilities of the database such as relational, spatial, or text search. We present timing characteristics for various pattern sizes and metric combinations, as well as address the balancing of database and GPU parameterization. Our feature vector datasets range from 18 to 85 GB in database table storage size, reaching 100 million 128 dimensional vectors. We are able to efficiently execute global top k searches from within the database.

Geospatial data exists in a variety of formats, including rasters, vector data, and large-scale geospatial databases. There exists an ever-growing number of sensors that are collecting this data, resulting in the explosive growth and scale of high-resolution remote sensing geospatial data collections. A particularly challenging domain of geospatial data processing involves mining information from high resolution remote sensing imagery. The prevalence of high-resolution raster geospatial data collections represents a significant data challenge, as a single remote sensing image is composed of hundreds of millions of pixels. We have developed a robust application framework which exploits graphics processing unit (GPU) clusters to perform high-throughput geospatial data processing. We process geospatial raster data concurrently across tiles of large geospatial data rasters, utilizing GPU co-processors driven by CPU threads to extract refined geospatial information. The framework can produce output rasters or perform image information mining to write data into a geospatial database.

The fuzzy integral (FI) is an extremely flexible aggregation operator. It is used in numerous applications, such as image processing, multicriteria decision making, skeletal age-at-death estimation, and multisource (e.g., feature, algorithm, sensor, and confidence) fusion. To date, a few works have appeared on the topic of generalizing Sugeno's original real-valued integrand and fuzzy measure (FM) for the case of higher order uncertain information (both integrand and measure). For the most part, these extensions are motivated by, and are consistent with, Zadeh's extension principle (EP). Namely, existing extensions focus on fuzzy number (FN), i.e., convex and normal fuzzy set- (FS) valued integrands. Herein, we put forth a new definition, called the generalized FI (gFI), and efficient algorithm for calculation for FS-valued integrands. In addition, we compare the gFI, numerically and theoretically, with our non-EP-based FI extension called the nondirect FI (NDFI). Examples are investigated in the areas of skeletal age-at-death estimation in forensic anthropology and multisource fusion. These applications help demonstrate the need and benefit of the proposed work. In particular, we show there is not one supreme technique. Instead, multiple extensions are of benefit in different contexts and applications.

Fuzzy integrals are non-linear combinations of a hypothesis support function and the (possibly subjective) worth of subsets of sources of information, realized by a fuzzy measure. They are used in many applications, with data fusion being the most well-known. In most applications, the fuzzy measure is built by some external knowledge about the worth of subsets of the information sources, whether by a subjective expert or objective sensor property, such as signal-to-noise ratio. In this paper, we investigate fuzzy measures for interval-valued evidence that have no intrinsic known worth; hence, the fuzzy measure cannot or should not be built in the conventional ways. Instead, the fuzzy measure is built directly from the data. We examine the previously proposed fuzzy measure of agreement, which builds the fuzzy measure by a computation of the agreement of combinations of sources (sources from which the contributed evidence has a high degree of agreement with evidence from other sources have a high worth). We also propose a new fuzzy measure of generalized accord that addresses a theoretical weakness in the agreement measure. We compare the two fuzzy measures by performing aggregation experiments with the fuzzy Choquet integral. Tests on both synthetic and real data are performed. We also compare the two measures against the aggregation results obtained by a survey of several example data sets.

The so-called kernel-trick is a well-known method for mapping data in a lower dimensional space into a higher dimensional space to measure the similarity (inner product) of the data elements without ever explicitly performing the mapping. The hope is to induce an improved feature space in which to carry out pattern analysis. However, important questions remain, such as i) what is the best kernel, and ii) do some features or sensors require different kernels? One elegant way to address these problems is multiple kernel (MK) aggregation. To date, the research on MKs has predominately studied linear aggregation of kernels, namely weighted sums, e.g., conic and convex sums. In this paper, we propose a new method for kernel aggregation, fuzzy integral aggregation of MKs (FI-MK). We study different FI formulations to determine which ensures production of an aggregated kernel that is a valid Mercer kernel. We show that the Choquet integral (CI) achieves this goal for matrix-wise aggregation. We leverage our theoretical results to propose a genetic algorithm-based classification scheme called FIGA. Experiments on publicly available data sets are provided that demonstrate our FIGA algorithm produces superior results in the context of support vector machine (SVM)-based classification.

The Fuzzy Integral (FI) is a powerful approach for non-linear data aggregation. It has been used in many settings to combine evidence (typically objective) with the known “worth” (typically subjective) of each data source, where the latter is encoded in a Fuzzy Measure (FM). While initially developed for the case of numeric evidence (integrand) and numeric FM, Grabisch et al. extended the FI to the cases of continuous intervals and normal, convex fuzzy sets (i.e., fuzzy numbers). However, in many real-world applications, e.g., explosive hazard detection based on multi-sensor and/or multi-feature fusion, agreement based modeling of survey data, anthropology and forensic science, or computing with respect to linguistic descriptions of spatial relations from sensor data, discontinuous interval and/or non-convex fuzzy set data may arise. The problem is no theory and algorithm currently exists for calculating the FI for such a case. Herein, we propose an extension of the FI to discontinuous interval- and convex normal Interval Fuzzy Set (IFS)-valued integrands (with a numeric FM). Our approach arises naturally from analysis of the Extension Principle. Further, we provide a computationally efficient approach to computing the proposed extension based on the union of the FIs on the combinations of continuous sub-intervals and we demonstrate the approach using examples for both the Choquet FI (CFI) and Sugeno FI (SFI).

A number of noteworthy techniques have been put forth recently in different research fields for comparing clusterings. Herein, we introduce a new method for comparing soft (fuzzy, probabilistic, and possibilistic) partitions based on the earth mover's distance (EMD) and the ordered weighted average (OWA). The proposed method is a metric, depending on the ground distance, for all but possibilistic partitions. It is extremely flexible due to its EMD formulation, OWA aggregation, and abstract concept of ground distance. In theory, our method is agnostic to the type (uncertainty) of soft partition, clustering algorithm, and distance measure used in the clustering algorithm(s), and it is applicable to the clustering of both object and relational data. Validation is performed theoretically, experimentally, as well as in terms of computational complexity. Emphasis is placed on the set of possibilistic partitions, specifically noise and coincident clusters, which are important cases that have received little to no attention to date in the comparing clustering literature. Improvements are reported in terms of metric properties and computational complexity over existing extended concordance/discordance (e.g., soft Rand and Jaccard) approaches and improved design and robustness in comparison with existing transportation problem-based approaches.

Rapid detection of landmines and explosive hazards is a critical issue for modern military operations. Due to the varied nature of the objects of interest and the complexity of the surroundings, one approach is to utilize the superior recognition capabilities of the human brain in the detection process. We are developing frameworks and algorithms to process image video data from an RGB camera, mounted on a moving vehicle, and to provide cueing capability for a human-in-the-loop detection system. Feedback from the human operator is embedded into the system memory to aid future detection processes (causal). Due to the inherent variation of different objects of interest terms of color and texture appearance, and the inherent variation and complexity of the surroundings, we introduce a classification algorithm that operates on local decision boundaries. Each decision boundary is learned using the support vector machine (SVM) technique. Given test data, the K-nearest neighbor (KNN) method is used to pre-select the nearest training set to localize the scope of SVM training, giving us the local decision boundary.

In prior work, Grabisch put forth a direct (i.e., result of the Extension Principle) generalization of the Sugeno fuzzy integral (FI) for fuzzy set (FS)-valued normal (height equal to one) integrands and number-based fuzzy measures (FMs). Grabisch's proof is based in large on Dubois and Prade's analysis of functions on intervals, fuzzy numbers (thus normal FSs) and fuzzy arithmetic. However, a case not studied is the extension of the FI for sub-normal FS integrands. In prior work, we described a real-world forensic application in anthropology that requires fusion and has sub-normal FS inputs. We put forth an alternative non-direct approach for calculating FS results from sub-normal FS inputs based on the use of the number-valued integrand and number-valued FM Sugeno FI. In this article, we discuss a direct generalization of the Sugeno FI for sub-normal FS integrands and numeric FMs, called the Sub-normal Fuzzy Integral (SuFI). To no great surprise, it turns out that the SuFI algorithm is a special case of Grabisch's generalization. An algorithm for calculating SuFI and its mathematical properties are compared to our prior method, the Non-Direct Fuzzy Integral (NDFI). It turns out that SuFI and NDFI fuse in very different ways. We assert that in some settings, e.g., skeletal age-at-death estimation, NDFI is preferred to SuFI. Numeric examples are provided to stress important inner workings and differences between the FI generalizations.

Fuzzy measures (FMs) have been used to model the (typically subjective) "worth" of subsets of information sources relative to a decision making problem. The fuzzy integral (FI) is a way to fuse the information encoded in a FM with the (typically objective) confidences in the strength of a hypothesis arising from the information sources. In prior work, Yager discussed a set of aggregation functions for general FMs. However, that work is primarily focused on theoretical exploration versus application. Herein, we investigate the direct extraction of different FMs from data, one for specificity and another for agreement, in the context of crowd sourcing. In crowd sourcing, one often has a lack of a ground truth or information regarding the reliability of sources. That is, all sources must be assumed equal (in terms of knowledge, experience level, etc.). Our goal is the intelligent fusion of this data taking into account as much information as possible from the data itself. Once a set of FMs are extracted from the data, we aggregate the FMs (resulting in what we herein refer to as a meta-measure) and use it in fuzzy integration. The novel aspect of this work is the extraction of multiple FMs directly from the original pool of data and the use of the resultant meta-measure and a FI to fuse the data from which the FMs were extracted. Herein, our data is interval-valued, thus we focus on fusion with respect to the generalized interval FI.

We present a sliding window-based, per-pixel importance-weighted, multi-scale, cell-structured feature descriptor and demonstrate its performance for recognizing different aircraft from remotely sensed imagery. Opening and closing differential morphological profiles are constructed, then fused with the Choquet integral to create a soft segmentation. A per-pixel importance map is derived from the soft segmentation and used in the calculation of histogram of oriented gradients, local binary patterns, invariant object moments, and Haar-like features. Superiority is demonstrated in comparison to flat single-scale and non-importance weighted representations with encouraging results for both cross-validation and blind testing. Results show that the pyramid, cell-structured, importance-weighting performs better than traditional approaches in the difficult problem space of recognizing objects in remote sensing imagery.

Herein, we put forth a new similarity measure for anomaly detection and for comparing human behaviors based on the theories of learning automata, comparison of soft partitions, and temporal probabilistic order relations. In particular, focus is placed on monitoring individuals in a home setting for their own well-being. This work is a high-level investigation focused on the structure of human behavior. Examples demonstrate the utility of this approach for (1) understanding the similarity of pairs of behaviors for an individual (or alternatively between individuals) and (2) detecting significant change between changing behavior and a baseline model. In the context of eldercare, significant change in behavior can be a precursor to cognitive and/or functional health related problems. Simulated resident behavior is used to show different scenarios and the response of the proposed measure. © 2012 Wiley Periodicals, Inc.

Previously, we introduced a novel method to estimate adult skeletal age-at-death using the Sugeno fuzzy integral (FI). Specifically, we took a multi-hypothesis testing approach to make the classical FI yield a fuzzy set (FS)-valued result, which is not guaranteed to be normal or convex, based on interval-valued sources of information (aging methods). We showed quantitative results for summarizing the FS and comparing the single decision to a known age-at-death. In this article, we extend our prior work and present formulas to measure the uncertainty in the resultant FSs. We generate linguistic descriptions to establish domain standardization for the goal of assisting forensic and biological anthropologists. Specifically, we introduce fuzzy class definitions for age-at-death FSs and we present an OWA contrast approach to measure the degree of specificity in age-at-death FSs.

Previously, we presented a method for comparing soft partitions (i.e. crisp, probabilistic, fuzzy and possibilistic) to a known crisp reference partition. Many of the classical indices that have been used with outputs of crisp clustering algorithms were generalized so that they are applicable for candidate partitions of any type. In particular, focus was placed on generalizations of the Rand index. In this article, we extend our prior work by (1) investigating the behavior of the soft Rand for comparing non-crisp, specifically possibilistic, partitions and (2) we demonstrate how the possibilistic Rand and visual assessment of (cluster) tendency (VAT) algorithm can be used to discover the number of actual clusters and coincident clusters for outputs from the possibilistic c-means (PCM) algorithm.

In this paper, we propose a linguistic summarization procedure for describing long-term trends of change in human behavior. Our objective consists of defining methods that provide information to elders, caregivers, social workers or even family in an understandable language. We adapt a measure that we defined in previous work on soft cluster partition similarity for comparing behaviors that are adapted over time. From that measure, we are able to produce a time series that numerically describes change in behavior over time. In this article, the resulting time series is partitioned and linguistically summarized depending on a user's (caregiver, social worker, etc.) desired time resolution. Simulated resident behavior is used in order to explore a range of different scenarios and the response of the proposed linguistic summarization process is investigated.

For reasons such as computational complexity, spatial and temporal information reduction, and human understandability, it is important that computer vision systems be equipped with the means to summarize their content in a natural language. Such rich descriptions are of use by both humans and computers for describing, recognizing, and tracking objects, activity, and their interactions at a desired level of abstraction. A genetic algorithm is introduced here for segmenting non-human objects deemed relevant to human activity analysis in stereo vision acquired voxel environments. This approach is of use in an Eldercare setting as it relates to monitoring the “well-being” of residents through acquiring and detecting deviations in patterns of typical behavior as well as recognizing abnormal events, such as fall detection.

Fuzzy integrals have been used to fuse the evidence or opinions from a variety of sources. These integrals are nonlinear combinations of the support functions and the (possibly subjective) worth of subsets of the sources of information, realized by a fuzzy measure. There have been many applications and extensions of fuzzy integrals and this paper proposes a fuzzy Choquet integral, where the integrand takes an interval type-2 fuzzy number and the fuzzy measure is real number-valued. Interval type-2 fuzzy numbers encode the second-order uncertainty in a fuzzy number. Type-2 fuzzy numbers have been been shown to be useful in many applications, including computing with words and control systems. We illustrate our method on several numerical examples as well as on a bioinformatics application.

When clustering produces more than one candidate to partition a finite set of objectsO, there are two approaches to validation (i.e., selection of a “best” partition, and implicitly, a best value for c , which is the number of clusters inO). First, we may use an internal index, which evaluates each partition separately. Second, we may compare pairs of candidates with each other, or with a reference partition that purports to represent the “true” cluster structure in the objects. This paper generalizes many of the classical indices that have been used with outputs of crisp clustering algorithms so that they are applicable for candidate partitions of any type (i.e., crisp or soft, with soft comprising the fuzzy, probabilistic, and possibilistic cases). Space prevents inclusion of all of the possible generalizations that can be realized this way. Here, we concentrate on the Rand index and its modifications. We compare our fuzzy-Rand index with those of Campello, Hullermeier and Rifqi, and Brouwer, and show that our extension of the Rand index is O(n), while the other three are all O(n2). Numerical examples are given to illustrate various facets of the new indices. In particular, we show that our indices can be used, even when the partitions are probabilistic or possibilistic, and that our method of generalization is valid for any index that depends only on the entries of the classical (i.e., four-pair types) contingency table for this problem.

As part of an interdisciplinary collaboration on elder-care monitoring, a sensor suite for the home has been augmented with video cameras. Multiple cameras are used to view the same environment and the world is quantized into nonoverlapping volume elements (voxels). Through the use of silhouettes, a privacy protected image representation of the human acquired from multiple cameras, a 3-D representation of the human is built in real time, called voxel person. Features are extracted from voxel person and fuzzy logic is used to reason about the membership degree of a predetermined number of states at each frame. Fuzzy logic enables human activity, which is inherently fuzzy and case-based, to be reliably modeled. Membership values provide the foundation for rejecting unknown activities, something that nearly all current approaches are insufficient in doing. We discuss temporal fuzzy confidence curves for the common elderly abnormal activity of falling. The automated system is also compared to a ground truth acquired by a human. The proposed soft computing activity analysis framework is extremely flexible. Rules can be modified, added, or removed, allowing per-resident customization based on knowledge about their cognitive and functionality ability. To the best of our knowledge, this is a new application of fuzzy logic in a novel approach to modeling and monitoring human activity, in particular, the well-being of an elderly resident, from video.

As the size and operator complexity of a fuzzy logic system increases, computational tractability becomes a problem. There is a significant amount of parallelism in both the creation of the fuzzy rule base and in fuzzy inference. Traditional processors (CPUs) cannot take full advantage of this natural parallelism graphics processing units (GPUs) speed up rule construction and inference by utilizing up to 128 processing units operating in parallel. Normally, these processors are used to perform high speed graphics calculations for video games, movies, and other areas of intense graphical work. In this paper, a method is discussed for speeding up fuzzy logic by structuring it into a format such that it resembles the standard rendering procedure for a graphics pipeline based on rasterization.

Video cameras are a relatively low-cost, rich source of information that can be used for ldquowell-beingrdquo assessment and abnormal event detection for the goal of allowing elders to live longer and healthier independent lives. We previously reported a soft-computing fall detection system, based on two levels from a hierarchy of fuzzy inference using linguistic summarizations of activity acquired temporally from a three dimensional voxel representation of the human found by back projecting silhouettes acquired from multiple cameras. This framework is extremely flexible and rules can be modified, added, or removed, allowing for per-resident customization based on knowledge about their cognitive and physical ability. In this paper, we show the flexibility of our activity analysis framework by extending it to additional common elderly activities and contextual awareness is added for reasoning based on location or static objects in the apartment.

As the number of data points, feature dimensionality, and number of centers for clustering algorithms increase, computational tractability becomes a problem. The fuzzy c-means has a large degree of inherent algorithmic parallelism that modern CPU architectures do not exploit. Many pattern recognition algorithms can be sped up on a graphics processing unit (GPU) as long as the majority of computation at various stages and the components are not dependent on each other. We present a generalized method for offloading fuzzy clustering to a GPU, while maintaining control over the number of data points, feature dimensionality, and the number of cluster centers. GPU-based clustering is a high-performance low-cost solution that frees up the CPU. Our results show a speed increase of over two orders of magnitude for particular clustering configurations and platforms.

Heterogeneous genetic and epigenetic alterations are commonly found in human non-Hodgkin's lymphomas (NHL). One such epigenetic alteration is aberrant methylation of gene promoter-related CpG islands, where hypermethylation frequently results in transcriptional inactivation of target genes, while a decrease or loss of promoter methylation (hypomethylation) is frequently associated with transcriptional activation. Discovering genes with these relationships in NHL or other types of cancers could lead to a better understanding of the pathobiology of these diseases. The simultaneous analysis of promoter methylation using Differential Methylation Hybridization (DMH) and its associated gene expression using Expressed CpG Island Sequence Tag (ECIST) microarrays generates a large volume of methylation-expression relational data. To analyze this data, we propose a set of algorithms based on fuzzy sets theory, in particular Possibilistic c-Means (PCM) and cluster fuzzy density. For each gene, these algorithms calculate measures of confidence of various methylation-expression relationships in each NHL subclass. Thus, these tools can be used as a means of high volume data exploration to better guide biological confirmation using independent molecular biology methods.

Extracting a human silhouette from an image is the enabling step for many high-level vision processing tasks, such as human tracking and activity analysis. Although there are a number of silhouette extraction algorithms proposed in the literature, most approaches work efficiently only in constrained environments where the background is relatively simple and static. In a previous paper, we addressed some of the challenges in silhouette extraction and human tracking in a real-world unconstrained environment where the background is complex and dynamic. We extracted features from image regions, accumulated the feature information over time, fused high-level knowledge with low-level features, and built a time-varying background model. A problem with our system is that by adapting the background model, objects moved by a human are difficult to handle. In order to reinsert them into the background, we run the risk of cutting off part of the human silhouette, such as in a quick arm movement. In this paper, we develop a fuzzy logic inference system to detach the silhouette of a moving object from the human body. Our experimental results demonstrate that the fuzzy inference system is very efficient and robust.

Extracting a human silhouette from an image is the enabling step for many high-level vision processing tasks, such as human tracking and activity analysis. Although there are a number of silhouette extraction and human tracking algorithms proposed in the literature, most approaches work efficiently only in constrained environments where the background is relatively simple and static. In this work, we propose to address the challenges in silhouette extraction and human tracking in a real-world unconstrained environment where the background is complex and dynamic. We extract features from image regions, accumulate the feature information over time, fuse the high-level knowledge with low-level features, and build a time-varying background model. We develop a fuzzy decision process to detach foreground moving objects from the human body. Our experimental results demonstrate that the algorithm is very efficient and robust.

In this paper, we describe a prototype interface that facilitates the control of a mobile robot team by a single operator, using a sketch interface on a tablet PC. The user sketches a qualitative map of the scene and includes the robots in approximate starting positions. Both path and target position commands are supported as well as editing capabilities. Sensor feedback from the robots is included in the display such that the sketch interface acts as a two-way communication device between the user and the robots. The paper also includes results of a usability study, in which users were asked to perform a series of tasks