This paper focuses on a powerful nonlinear aggregation function, the Choquet integral (ChI). Specifically, we focus on situations where the parameters of the ChI are learned from data. For N inputs, the ChI breaks down into N! underlying linear convex sums (LCSs) with 2Nshared variables. Typically, these LCSs are reducible into a drastically smaller number of linear order statistics (LOSs). In the spirit of explainable AI (XAI), our goal is to discover the minimal underlying operator structure of a learned ChI to be conveyed to its users. The challenge is, there does not appear to be widespread research or agreement regarding how to compute similarity within and between measures or integrals. In this paper, we explore the earth mover's distance (EMD), a parametric cross-bin measure, to capture semantic relatedness between LOSs. EMD is used to measure dissimilarity between integrals. In the case of a single ChI, underlying aggregation operator structure is discovered via EMD and clustering. A combination of synthetic and real-world experiments are provided to demonstrate interpretability and reduction of complexity.

The Choquet integral (ChI) is an aggregation operator defined with respect to a fuzzy measure (FM). The FM encodes the worth of all subsets of the sources of information that are being aggregated. The ChI is capable of representing many aggregation functions and has found its application in a wide range of decision fusion problems. In our prior work, we introduced a data support-based approach for learning the FM for decision fusion problems. This approach applies a quadratic programming (QP)-based method to train the FM. However, since the FM of ChI scales as$2^{N}$, where$N$is the number of input sources, the space complexity for learning the FM grows exponentially with$N$. This has limited the practical application of ChI-based decision fusion methods to small numbers of dimenstions—$N$≲ 6 is practical in most cases. In this work, we propose an iterative gradient descent-based approach to train the FM for ChI with an efficient method for handling the FM constraints. This method processes the training data, one observation at a time, and thereby significantly reduces the space complexity of the training process. We tested our online method on synthetic and real-world data sets, and compared the performance and convergence behaviour with our previously proposed QP-based method (i.e., batch method). On 10 out of 12 data sets, the online learning method has either matched or outperformed the batch method. We also show that we are able to use larger numbers of inputs with the online learning approach, extending the practical application of the ChI.

The adoption of artificial intelligence (AI) into domains that impact human life (healthcare, agriculture, security and defense, etc.) has led to an increased demand for explainable AI (XAI). Herein, we focus on an under represented piece of the XAI puzzle, information fusion. To date, a number of low-level XAI explanation methods have been proposed for the fuzzy integral (FI). However, these explanations are tailored to experts and its not always clear what to do with the information they return. In this article we review and categorize existing FI work according to recent XAI nomenclature. Second, we identify a set of initial actions that a user can take in response to these low-level statistical, graphical, local, and linguistic XAI explanations. Third, we investigate the design of an interactive user friendly XAI report. Two case studies, one synthetic and one real, show the results of following recommended actions to understand and improve tasks involving classification.

Light field (LF) imaging has gained significant attention due to its recent success in 3-dimensional (3D) displaying and rendering as well as augmented and virtual reality usage. Because of the two extra dimensions, LFs are much larger than conventional images. We develop a JPEG-assisted learning-based technique to reconstruct an LF from a JPEG bitstream with a bit per pixel ratio of 0.0047 on average. For compression, we keep the LF's center view and use JPEG compression with 50% quality. Our reconstruction pipeline consists of a small JPEG enhancement network (JPEG-Hance), a depth estimation network (Depth-Net), followed by view synthesizing by warping the enhanced center view. Our pipeline is significantly faster than using video compression on pseudo-sequences extracted from an LF, both in compression and decompression, while maintaining effective performance. We show that with a 1% compression time cost and 18x speedup for decompression, our methods reconstructed LFs have better structural similarity index metric (SSIM) and comparable peak signal-to-noise ratio (PSNR) compared to the state-of-the-art video compression techniques used to compress LFs.

Highly-reverberate underwater environments pose challenges for conventional localization techniques due to the highly non-linear nature of reflective surfaces, multi-path, and scattering fields. In this paper, we compare different machine learning methods for passive localization and tracking of single, non-stationary, underwater acoustic sources using multiple underwater acoustic vector sensors. We incorporate ordinal classification for localization in a novel approach to acoustic localization and compare the results with other standard methods. Realworld experiments demonstrate that both categorical and ordinal classification using deep LSTM networks significantly reduce localization error.

Community detection is an important problem in complex network analysis. Among numerous approaches for community detection, label propagation (LP) has attracted a lot of attention. LP selects the optimum community (i.e., label) of a network vertex by optimizing an objective function (e.g., Newman’s modularity) subject to the available labels in the vicinity of the vertex. In this article, a novel analysis of Newman’s modularity gain with respect to label transitions in graphs is presented. Here, we propose a new form of Newman’s modularity gain calculation that quantifies available label transitions for any LP based community detection. The proposed approach is called Modularity Gain Acceleration (MGA) and is simplified and divided into two components, the local and global sum-weights. The Local Sum-Weight (LSW) is the component with lower complexity and is calculated for each candidate label transition. The General Sum-Weight (GSW) is more computationally complex, and is calculated only once per each label. GSW is updated by leveraging a simple process for each node-label transition, instead of for all available labels. The proposed technique is applied to selected state-of-the-art LP-based community detection methods and the resulting network modularity and execution time are compared with traditional methods over small to big real world data sets. By applying MGA to LP-based methods, the run-time is significantly reduced–sometimes finishing before the traditional approach even finishes one iteration–achieving the same modularity performance and number of communities, i.e., community detection result. The MGA approach leads to significant efficiency improvements by reducing time consumption up to 85 percent relative to the original algorithms with the exact same quality in terms of modularity value which is highly valuable in analyses of big data sets.

Zadeh's extension principle (ZEP) is a fundamental concept in fuzzy set (FS) theory that enables crisp mathematical operation on FSs. A well-known shortcoming of ZEP is that the height of the output FS is determined by the lowest height of the input FSs. In this article, we introduce a generalized extension principle (GEP) that eliminates this weakness and provides flexibility and control over how membership values are mapped from input to output. Furthermore, we provide a computationally efficient point-based FS representation. In light of our new definition, we discuss two approaches to perform aggregation of FSs using the Choquet integral. The resultant integrals generalize prior work and lay a foundation for future extensions. Last, we demonstrate the extended integrals via a combination of synthetic and real-world examples.

Fuzzy integrals (FIs) are powerful aggregation operators that fuse information from multiple sources. The aggregation is parameterized using a fuzzy measure (FM), which encodes the worths of all subsets of sources. Since the FI is defined with respect to an FM, much consideration must be given to defining the FM. However, in practice this is a difficult task—the number of values in an FM scales as$2^n$, where$n$is the number of input sources, thus manually specifying an FM quickly becomes tedious. In this article, we review an automatic, data-supported method of learning the FM by minimizing a sum-of-squared error objective function in the context of decision-level fusion of classifiers using the Choquet FI. While this solves the specification problem, we illuminate an issue encountered with many real-world data sets; i.e., if the training data do not contain a significant number of all possible sort orders, many of the FM values are not supported by the data. We propose various regularization strategies to alleviate this issue by pushing the learned FM toward a predefined structure; these regularizers allow the user to encode knowledge of the underlying FM to the learning problem. Furthermore, we propose another regularization strategy that constrains the learned FM's structure to be a linear order statistic. Finally, we perform several experiments using synthetic and real-world data sets and show that our proposed extensions can improve the learned FM behavior and classification accuracy. A previously proposed visualization technique is employed to simultaneously quantitatively illustrate the FM as well as the FI.

Soft overlapping clustering is one of the notable problems of community detection. Extensive research has been conducted to develop efficient methods for nonoverlapping and crisp-overlapping community detection in large-scale networks. In this article, fast fuzzy modularity maximization (FFMM) for soft overlapping community detection is proposed. FFMM exploits novel iterative equations to calculate the modularity gain associated with changing the fuzzy membership values of network vertices. The simplicity of the proposed scheme enables efficient modifications, reducing computational complexity to a linear function of the network size, and the number of communities. Moreover, to further reduce the complexity of FFMM for very large networks, multicycle FFMM (McFFMM) is proposed. The proposed McFFMM reduces complexity by breaking networks into multiple subnetworks and applying FFMM to detect their communities. Performance of the proposed techniques is demonstrated with real-world data and the Lancichinetti-Fortunato-Radicchi benchmark networks. Moreover, the performance of the proposed techniques is evaluated versus some state-of-the-art soft overlapping community detection approaches. Results show that the McFFMM produces a remarkable performance in terms of overlapping modularity with fuzzy memberships, computational time, number of detected overlapping nodes, and overlapping normalized mutual information.

The ordered weighted average (OWA) operator is a well-known aggregation tool that is primarily used for decisionlevel fusion. However, the OWA is a convex sum, i.e., its learned coefficients are constrained to sum to one, and thus the output is restricted to lie between the maximum and minimum values of the inputs. Relaxing this constraint on the sum of weights transforms the OWA into a linear order statistic (LOS), which allows the aggregation operation to map the input to any value on the set of reals, thus behaving more like a regression operator. The LOS parameterizes the regression operation of d-features using just d parameters, which helps with the model's interpretability. However, learning just d parameters limits the amount of nonlinear space explored for an optimal solution, and thus reduces the expressibility of the LOS algorithm. We propose a novel aggregation method called the extended linear order statistic (ELOS), where for each position in the sorted input vector we have d parameters, one for each input feature, thus learning a total of d2weights for the aggregation of d features. The increased number of parameters helps the algorithm improve its expressibility while maintaining interpretability. In our experiments on real-world benchmark data sets, ELOS has outperformed both linear regression and LOS in 8 out of 10 experiments.

The Choquet integral (ChI) is an aggregation function that is defined with respect to a fuzzy measure (FM). Many ChI-based decision aggregation methods have been proposed to learn the underlying FM. However, FM's boundary and monotonicity constraints have limited the applicability of such methods to decision-level fusion. In a recent work, we removed the constraints on FM to develop a regression model based on the ChI. Our model has a generalized bias that enables capability beyond previously proposed ChI regression approaches. We also developed an approach for learning the parameters of the ChI regression from training data. In this paper, we develop a method to apply ℓ2-regularization on our training algorithm. In our experiments on real-world benchmark data sets, ridge regularized-ChI regression has outperformed the unregularized version in 22 out of 30 (73%) experiments. Also, when compared with several competing regression methods, results show that our approach has superior performance.

Continued developments in information technologies allows for increasingly more data to be collected for decision making purposes. While statistical summaries and aggregations are commonly applied to such data, linguistic summaries capture essential features and relationships in the data and better support human users to understand complex data sets. The basic quality measure of linguistic summaries is the truth value, describing the validity of the sentence. Several methods for calculating the truth value have been proposed. In this paper we analyze several popular methods and show a strange, contradictory behavior in case of extended protoforms, which can result in misleading or non-intuitive results to the user. These results highlight the need for further research into linguistic summarization and the computation of truth values for real data sets.

The Choquet Integral (ChI) is an aggregation operator defined with respect to a Fuzzy Measure (FM). The FM encodes the worth of all subsets of the sources of information that are being aggregated. The monotonicity and the boundary conditions of the FM have limited its applicability to decision-level fusion. But in a recent work, we removed the boundary and monotonicity constraints of the FM, which we then called a bounded capacity (BC), to propose a Choquet Integral regression (CIR) approach that enables capability beyond previously proposed ChI regression methods. In the same work, we also presented a quadratic programming (QP)-based method, batch-CIR, to learn the BC parameters of the CIR from training data. However, the QP used for learning the BC scales exponentially with the dimensionality of the training data and thus it becomes impractical on data sets with 7 or more dimensions. In this paper we propose an iterative gradient descent approach, online-CIR, to learn the BC. This method iteratively processes the training data, one data point at a time, and therefore requires significantly less computation and space at any time during the training. The application of batch-CIR required the dimensionality reduction of high-dimensional data sets to enable computation in a reasonable time. The proposed online-CIR approach has enabled us to extend CIR to data sets with larger dimensionality. In our experimental evaluation using benchmark regression data sets, online-CIR has outperformed batch-CIR on high-dimensional data sets while also matching the batch-CIR performance on low-dimensional data sets.

Information fusion is an essential part of numerous engineering systems and biological functions, e.g., human cognition. Fusion occurs at many levels, ranging from the low-level combination of signals to the high-level aggregation of heterogeneous decision-making processes. While the last decade has witnessed an explosion of research in deep learning, fusion in neural networks has not observed the same revolution. Specifically, most neural fusion approaches are ad hoc, are not understood, are distributed versus localized, and/or explainability is low (if present at all). Herein, we prove that the fuzzy Choquet integral (ChI), a powerful nonlinear aggregation function, can be represented as a multilayer network, referred to hereafter as ChIMP. We also put forth an improved ChIMP (iChIMP) that leads to a stochastic-gradient-descent-based optimization in light of the exponential number of ChI inequality constraints. An additional benefit of ChIMP/iChIMP is that it enables explainable artificial intelligence (XAI). Synthetic validation experiments are provided, and iChIMP is applied to the fusion of a set of heterogeneous architecture deep models in remote sensing. We show an improvement in model accuracy, and our previously established XAI indices shed light on the quality of our data, model, and its decisions.

With a growing number of areas leveraging interval-valued data-including in the context of modeling human uncertainty (e.g., in cybersecurity), the capacity to accurately and systematically compare intervals for reasoning and computation is increasingly important. In practice, well established set-theoretic similarity measures, such as the Jaccard and Sørensen-Dice measures, are commonly used, whereas axiomatically, a wide breadth of possible measures have been theoretically explored. This article identifies, articulates, and addresses an inherent and so far not discussed limitation of popular measures-their tendency to be subject to aliasing-where they return the same similarity value for very different sets of intervals. The latter risks counter-intuitive results and poor-automated reasoning in real-world applications dependent on systematically comparing interval-valued system variables or states. Given this, we introduce new axioms establishing desirable properties for robust similarity measures, followed by putting forward a novel set-theoretic similarity measure based on the concept of bidirectional subsethood, which satisfies both traditional and new axioms. The proposed measure is designed to be sensitive to the variation in the size of intervals, thus avoiding aliasing. This article provides a detailed theoretical exploration of the new proposed measure, and systematically demonstrates its behavior using an extensive set of synthetic and real-world data. Specifically, the measure is shown to return robust outputs that follow intuition-essential for real-world applications. For example, we show that it is bounded above and below by the Jaccard and Sørensen-Dice similarity measures (when the minimum t-norm is used). Finally, we show that a dissimilarity or distance measure, which satisfies the properties of a metric, can easily be derived from the proposed similarity measure.

Regression is the process of learning the relationship between sets of variables, enabling predictions of continuous output variables. Many approaches have been proposed to learn parameterized models with respect to numerous error metrics. In this paper, we propose a regression model based on the Choquet integral with respect to a bounded capacity (of which fuzzy measures are a subset). Our model has a generalized bias that enables capability beyond previously proposed Choquet integral regression approaches. We also develop an approach for learning the parameters of the Choquet integral regression from training data. Simulated and real-world benchmark data are used to demonstrate the performance of our regression approach compared with several competing regression methods. Results show that our approach has superior performance and is computationally very fast. An additional benefit of our Choquet integral regression is that it enables interpretability of the learned model, which we will explore in later works.

Discontinuous intervals (DIs) arise in a wide range of contexts, from real world data capture of human opinion to α-cuts of non-convex fuzzy sets. Commonly, for assessing the similarity of DIs, the latter are converted into their continuous form, followed by the application of a continuous interval (CI) compatible similarity measure. While this conversion is efficient, it involves the loss of discontinuity information and thus limits the accuracy of similarity results. Further, most similarity measures including the most popular ones, such as Jaccard and Dice, suffer from aliasing, that is, they are liable to return the same similarity for very different pairs of CIs. To address both of these challenges, this paper proposes a generalized approach for calculating the similarity of DIs which leverages the recently introduced bidirectional subsethood based similarity measure (which avoids aliasing) while accounting for all pairs of the continuous subintervals within the DIs to be compared. We provide detail of the proposed approach and demonstrate its behaviour when applying bidirectional subsethood, Jaccard and Dice as similarity measures, using different pairs of synthetic DIs. The experimental results show that the similarity outputs of the new generalized approach follow intuition for all three similarity measures; however, it is only the proposed integration with the bidirectional subsethood similarity measure which also avoids aliasing for DIs.

The Choquet integral (ChI) is a proven tool for information aggregation. In prior work, we showed that learning a ChI from data results in missing variables. Herein, we explore two ways to transfer a known ChI from a source domain to a new under sampled target domain. The first method is based on regularization and it listens to the full source domain ChI. The second method optimizes what we can observe (target domain supported variables) and missing variables are the only thing migrated from the source domain. Synthetic experiments, aka we know the truth, are used to show the behavior of these methods with respect to transfering between ChIs.

Herein, a generalization of the ordered weighted average (OWA) is put forth relative to pattern recognition. The resultant linear order statistic neuron (LOSN) is unique in that it bridges fuzzy sets, specifically fuzzy data/information aggregation, with neural networks. This article discusses the gradient descent-based optimization and geometric interpretation of the LOSN. An advantage is that the LOSN is an efficient shared weight encoding of N! perceptrons, relative to N inputs. Open source codes are provided to facilitate reproducible research. Experiments are conducted to both validate the method and show its non-linear geometric expression.

Information or data aggregation is an important part of nearly all analysis problems as summarizing inputs from multiple sources is a ubiquitous goal. In this paper we propose a method for non-linear aggregation of data inputs that take the form of non-normal fuzzy sets. The proposed shape-preserving fuzzy integral (SPFI) is designed to overcome a well-known weakness of the previously-proposed sub-normal fuzzy integral (SuFI). The weakness of SuFI is that the output is constrained to have maximum membership equal to the minimum of the maximum memberships of the inputs; hence, if one input has a small height, then the output is constrained to that height. The proposed SPFI does not suffer from this weakness and, furthermore, preserves in the output the shape of the input sets. That is, the output looks like the inputs. The SPFI method is based on the well-known Choquet fuzzy integral with respect to a capacity measure, i.e., fuzzy measure. We demonstrate SPFI on synthetic and real-world data, comparing it to the SuFI and non-direct fuzzy integral (NDFI).

Similarity measures are useful for reasoning about fuzzy sets. Hence, many classical set-theoretic similarity measures have been extended for comparing fuzzy sets. In previous work, a set-theoretic similarity measure considering the bidirectional subsethood for intervals was introduced. The measure addressed specific concerns of many common similarity measures, and it was shown to be bounded above and below by Jaccard and Dice measures respectively. Herein, we extend our prior measure from similarity on intervals to fuzzy sets. Specifically, we propose a vertical-slice extension where two fuzzy sets are compared based on their membership values. We show that the proposed extension maintains all common properties (i.e., reflexivity, symmetry, transitivity, and overlapping) of the original fuzzy similarity measure. We demonstrate and contrast its behaviour along with common fuzzy set-theoretic measures using different types of fuzzy sets (i.e., normal, non-normal, convex, and non-convex) in respect to different discretization levels.

To date, numerous ways have been created to learn a fusion solution from data. However, a gap exists in terms of understanding the quality of what was learned and how trustworthy the fusion is for future-i.e., new-data. In part, the current paper is driven by the demand for so-called explainable AI (XAI). Herein, we discuss methods for XAI of the Choquet integral (ChI), a parametric nonlinear aggregation function. Specifically, we review existing indices, and we introduce new data-centric XAI tools. These various XAI-ChI methods are explored in the context of fusing a set of heterogeneous deep convolutional neural networks for remote sensing.

The Choquet integral (ChI) is a parametric nonlinear aggregation function defined with respect to the fuzzy measure (FM). To date, application of the ChI has sadly been restricted to problems with relatively few numbers of inputs; primarily as the FM has 2Nvariables for N inputs and N(2N-1- 1) monotonicity constraints. In return, the community has turned to density-based imputation (e.g., Sugeno λ-FM) or the number of interactions (FM variables) are restricted (e.g., k-additivity). Herein, we propose a new scalable data-driven way to represent and learn the ChI, making learning computationally manageable for larger N. First, data supported variables are identified and used in optimization. Identification of these variables also allows us recognize future ill-posed fusion scenarios; ChIs involving variable subsets not supported by data. Second, we outline an imputation function framework to address data unsupported variables. Third, we present a lossless way to compress redundant variables and associated monotonicity constraints. Finally, we outline a lossy approximation method to further compress the ChI (if/when desired). Computational complexity analysis and experiments conducted on synthetic datasets with known FMs demonstrate the effectiveness and efficiency of the proposed theory.

Recently, there has been much research into effective representation and analysis of uncertainty in human responses, with applications in cyber-security, forest and wildlife management, and product development, to name a few. Most of this research has focused on representing the response uncertainty as intervals, e.g., “I give the movie between 2 and 4 stars.” In this paper, we extend upon the model-based interval agreement approach (lAA) for combining interval data into fuzzy sets and propose the efficient IAA (eIAA) algorithm, which enables efficient representation of and operation on the fuzzy sets produced by IAA (and other interval-based approaches, for that matter). We develop methods for efficiently modeling, representing, and aggregating both crisp and uncertain interval data (where the interval endpoints are intervals themselves). These intervals are assumed to be collected from individual or multiple survey respondents over single or repeated surveys; although, without loss of generality, the approaches put forth in this paper could be used for any interval-based data where representation and analysis is desired. The proposed method is designed to minimize loss of information when transferring the interval-based data into fuzzy set models and then when projecting onto a compressed set of basis functions. We provide full details of eIAA and demonstrate it on real-world and synthetic data.

In computing the similarity of intervals, current similarity measures such as the commonly used Jaccard and Dice measures are at times not sensitive to changes in the width of intervals, producing equal similarities for substantially different pairs of intervals. To address this, we propose a new similarity measure that uses a bi-directional approach to determine interval similarity. For each direction, the overlapping ratio of the given interval in a pair with the other interval is used as a measure of uni-directional similarity. We show that the proposed measure satisfies all common properties of a similarity measure, while also being invariant in respect to multiplication of the interval endpoints and exhibiting linear growth in respect to linearly increasing overlap. Further, we compare the behavior of the proposed measure with the highly popular Jaccard and Dice similarity measures, highlighting that the proposed approach is more sensitive to changes in interval widths. Finally, we show that the proposed similarity is bounded by the Jaccard and the Dice similarity, thus providing a reliable alternative.

The fuzzy integral (FI) is a nonlinear aggregation operator whose behavior is defined by the fuzzy measure (FM). As an aggregation operator, the FI is commonly used for evidence fusion where it combines sources of information based on the worth of each subset of sources. One drawback to FI-based methods, however, is the specification of the FM. Defining the FM manually quickly becomes too tedious since the number of FM terms scales as 2n, where n is the number of sources; thus, an automatic method of defining the FM is necessary. In this paper, we review a data-driven method of learning the FM via minimizing the sum-of-squared error (SSE) in the context of decision-level fusion and propose an extension allowing knowledge of the underlying FM to be encoded in the algorithm. The algorithm is applied to real-world and toy datasets and results show that the extension can improve classification accuracy. Furthermore, we introduce a visualization strategy to simultaneously show the quantitative information in the FM as well as the FI.

The aggregation of multiple information sources has a long history and ranges from sensor fusion to the aggregation of individual algorithm outputs and human knowledge. A popular approach to achieve such aggregation is the fuzzy integral (FI) which is defined with respect to a fuzzy measure (FM) (i.e. a normal, monotone capacity). In practice, the discrete FI aggregates information contributed by a discrete number of sources through a weighted aggregation (post-sorting), where the weights are captured by a FM that models the typically subjective `worth' of subsets of the overall set of sources. While the combination of FI and FM has been very successful, challenges remain both in regards to the behavior of the resulting aggregation operators - which for example do not produce symmetrically mirrored outputs for symmetrically mirrored inputs - and also in a manifest difference between the intuitive interpretation of a stand-alone FM and its actual role and impact when used as part of information fusion with a FI. This paper elucidates these challenges and introduces a novel family of recursive average (RAV) operators as an alternative to the FI in aggregation with respect to a FM; focusing specifically on the arithmetic recursive average. The RAV is designed to address the above challenges, while also facilitating fine-grained analysis of the resulting aggregation of different combinations of sources. We provide the mathematical foundations of the RAV and include initial experiments and comparisons to the FI for both numeric and interval-valued data.

Kernel methods for classification is a well-studied area in which data are implicitly mapped from a lower-dimensional space to a higher dimensional space to improve classification accuracy. However, for most kernel methods, one must still choose a kernel to use for the problem. Since there is, in general, no way of knowing which kernel is the best, multiple kernel learning (MKL) is a technique used to learn the aggregation of a set of valid kernels into a single (ideally) superior kernel. The aggregation can be done using weighted sums of the precomputed kernels, but determining the summation weights is not a trivial task. Furthermore, MKL does not work well with large datasets because of limited storage space and prediction speed. In this paper, we address all three of these multiple kernel challenges. First, we introduce a new linear feature level fusion technique and learning algorithm, GAMKLp. Second, we put forth three new algorithms, DeFIMKL, DeGAMKL, and DeLSMKL, for nonlinear fusion of kernels at the decision level. To address MKL's storage and speed drawbacks, we apply the Nystrom approximation to the kernel matrices. We compare our methods to a successful and state-of-the-art technique called MKL-group lasso (MKLGL), and experiments on several benchmark datasets show that some of our proposed algorithms outperform MKLGL when applied to support vector machine (SVM)-based classification. However, to no surprise, there does not seem to be a global winner but instead different strategies that a user can employ. Experiments with our kernel approximation method show that we can routinely discard most of the training data and at least double prediction speed without sacrificing classification accuracy. These results suggest that MKL-based classification techniques can be applied to big data efficiently, which is confirmed by an experiment using a large dataset.

Particle swarm optimization (PSO) is a biologically-inspired optimization algorithm, which has evolved over time and has been extensively studied and meta-optimized. To solve an optimization problem, the algorithm sends a plethora of particles into the search space, each particle searching for, ideally, the global optima. The velocity of these particles are determined by three components: a social component; a personal component, and a momentum component. The social and personal components of the standard PSO rely on a uniform-random distribution. There has been work on changing this distribution from being uniform-random to Gaussian; this paper aims at generalizing this distribution. Through use of another PSO, the distributions are learned over a set of objective functions, resulting in the proposed algorithm HistPSO. A novel approach is used to represent the distributions as piecewise uniform distributions on sub-intervals of the [0, c] interval where c is either c\ or c2depending whether the distribution represents the personal or social component of the PSO velocity equation. HistPSO is then tested on two other objective functions on which it had not been trained, as well as the three objective functions from which it learned. The results are then compared with a meta-optimized version of the standard PSO algorithm and the two results are compared to find that HistPSO performs comparably, and in some cases outperforms the meta-optimized standard PSO.

In hostile environments, random deployment of a Wireless ¡Sensor Network (WSN) may be the only viable approach. However, this leads to coverage holes in the Region of Interest (ROI) of the network, which degrades the WSN's quality of service. Hence, there is a need for an algorithm that relocates the sensing nodes to maximize the coverage while minimizing the mobility cost. The cost of mobility is directly related to the traveled distance and the severity of the terrain. Since this problem is NP-complete, this work examines several evolutionary computation techniques in search for an optimal solution. Three algorithms are used to examine this problem: the Artificial Immune System (AIS) algorithm, the Normalized Genetic Algorithm (NGA) and the Particle Swarm Optimization (PSO) algorithm. Multiple experiments are carried out to assess the performance of the utilized algorithms, where depending on the scenario adopted for simulations, some algorithms perform better than the others. In the case where the execution time is not a critical issue, the AIS and NGA algorithms outperform the PSO algorithm in terms of coverage rate and mobility cost, especially for a lower count of sensors.

Clustering of big data has received much attention recently. In this paper, we present a new clusiVAT algorithm and compare it with four other popular data clustering algorithms. Three of the four comparison methods are based on the well known, classical batch k -means model. Specifically, we use k -means, single pass k -means, online k -means, and clustering using representatives (CURE) for numerical comparisons. clusiVAT is based on sampling the data, imaging the reordered distance matrix to estimate the number of clusters in the data visually, clustering the samples using a relative of single linkage (SL), and then noniteratively extending the labels to the rest of the data-set using the nearest prototype rule. Previous work has established that clusiVAT produces true SL clusters in compact-separated data. We have performed experiments to show that k -means and its modified algorithms suffer from initialization issues that cause many failures. On the other hand, clusiVAT needs no initialization, and almost always finds partitions that accurately match ground truth labels in labeled data. CURE also finds SL type partitions but is much slower than the other four algorithms. In our experiments, clusiVAT proves to be the fastest and most accurate of the five algorithms; e.g., it recovers 97% of the ground truth labels in the real world KDD-99 cup data (4 292 637 samples in 41 dimensions) in 76 s.

The aim of this paper is the simultaneous minimization of model error and model complexity for the Choquet integral. The Choquet integral is a generator function, that is, a parametric function that yields a wealth of aggregation operators based on the specifics of the underlying fuzzy measure (aka normal and monotonic capacity). It is often the case that we desire to learn an aggregation operator from data and the goal is to have the smallest possible sum of squared error (SSE) between the trained model and a set of labels or function values. However, we also desire to learn the “simplest” solution possible, viz., the model with the fewest number of inputs. Previous works focused on the use of l1-norm regularization of a lexicographically encoded capacity vector relative to the Choquet integral, describing how to carry out the procedure and demonstrating encouraging results. However, no characterization or insights into the capacity and integral were provided. Herein, we investigate the impact of l1-norm regularization of a lexicographically encoded capacity vector in terms of what capacities and aggregation operators it strives to induce in different scenarios. Ultimately, this provides insight into what the regularization is really doing and when to apply such a method. Synthetic experiments are performed to illustrate the remarks, propositions, and concepts put forth.

Kernel methods for classification is a well-studied area in which data are implicitly mapped from a lower-dimensional space to a higher-dimensional space to improve classification accuracy. However, for most kernel methods, one must still choose a kernel to use for the problem. Since there is, in general, no way of knowing which kernel is the best, multiple kernel learning (MKL) is a technique used to learn the aggregation of a set of valid kernels into a single (ideally) superior kernel. The aggregation can be done using weighted sums of the pre-computed kernels, but determining the summation weights is not a trivial task. A popular and successful approach to this problem is MKL-group lasso (MKLGL), where the weights and classification surface are simultaneously solved by iteratively optimizing a min-max optimization until convergence. In this work, we propose an ℓp-normed genetic algorithm MKL (GAMKLp), which uses a genetic algorithm to learn the weights of a set of pre-computed kernel matrices for use with MKL classification. We prove that this approach is equivalent to a previously proposed fuzzy integral aggregation of multiple kernels called fuzzy integral: genetic algorithm (FIGA). A second algorithm, which we call decision-level fuzzy integral MKL (DeFIMKL), is also proposed, where a fuzzy measure with respect to the fuzzy Choquet integral is learned via quadratic programming, and the decision value-viz., the class label-is computed using the fuzzy Choquet integral aggregation. Experiments on several benchmark data sets show that our proposed algorithms can outperform MKLGL when applied to support vector machine (SVM)-based classification.

The fuzzy integral (FI) with respect to a fuzzy measure (FM) is a powerful means of aggregating information. The most popular FIs are the Choquet and Sugeno, and most research focuses on these two variants. The arena of the FM is much more populated, including numerically derived FMs such as the Sugeno λ-measure and decomposable measure, expert-defined FMs, and data-informed FMs. The drawback of numerically derived and expert-defined FMs is that one must know something about the relative values of the input sources. However, there are many problems where this information is unavailable, such as crowdsourcing. This paper focuses on data-informed FMs, or those FMs that are computed by an algorithm that analyzes some property of the input data itself, gleaning the importance of each input source by the data they provide. The original instantiation of a data-informed FM is the agreement FM, which assigns high confidence to combinations of sources that numerically agree with one another. This paper extends upon our previous work in datainformed FMs by proposing the uniqueness measure and additive measure of agreement for interval-valued evidence. We then extend data-informed FMs to fuzzy number (FN)-valued inputs. We demonstrate the proposed FMs by aggregating interval and FN evidence with the Choquet and Sugeno FIs for both synthetic and real-world data.

One of the most important elements of social network analysis is community detection, i.e., finding groups of similar people based on their traits. In this paper, we present the fuzzy modularity maximization (FMM) approach for community detection, which finds overlapping - that is, fuzzy - communities (where appropriate) by maximizing a generalized form of Newman's modularity. The first proposed FMM solution uses a tree-based structure to find a globally optimal solution, while the second proposed solution uses alternating optimization to efficiently search for a locally optimal solution. Both of these approaches are based on a proposed algorithm called one-step modularity maximization (OSMM), which computes the optimal cluster memberships for one person in the social network. We prove that OSMM can be formulated as a simplified quadratic knapsack optimization problem, which is O(n) time complexity. We then propose a tree-based algorithm, called FMM/Find Best Leaf Node (FMM/FBLN), which represents sequences of OSMM steps in a tree-based structure. It is proved that FMM/FBLN finds globally optimal solutions for FMM; however, the time complexity of FMM/FBLN is O(nd), d ≥ 2; thus, it is impractical for most real-world networks. To combat this inefficiency, we propose five heuristic-based alternating optimization schemes, i.e., FMM/H1-H5, which are all shown to be O(n2) time complexity. We compare the results of the FMM/H solutions with those of state-of-the-art community detection algorithms, MULTICUT spectral FCM (MSFCM) and GALS, and with those of two fuzzy community detection algorithms called GA and vertex-similarity based gradient-descent method (VSGD) on ten real-world datasets. We conclude that one of the five heuristic algorithms (FMM/H2) is very competitive with GALS and much more effective than MSFCM, GA, and VSGD. Furthermore, all of the FMM/H schemes are at least two orders of magnitude faster than GALS in run time. Finally, FMM/H, unlike GALS (which only produces crisp partitions) and MSFCM (which always finds fuzzy partitions), is the only fuzzy community detection algorithm to date that can find the max-modularity partition, fuzzy or crisp.

In this paper, a new approach is presented to model interval-based data using fuzzy sets (FSs). Specifically, we show how both crisp and uncertain intervals (where there is uncertainty about the endpoints of intervals) collected from individual or multiple survey participants over single or repeated surveys can be modeled using type-1, interval type-2, or general type-2 FSs based on zSlices. The proposed approach is designed to minimize any loss of information when transferring the interval-based data into FS models, and to avoid, as much as possible, assumptions about the distribution of the data. Furthermore, our approach does not rely on data preprocessing or outlier removal, which can lead to the elimination of important information. Different types of uncertainty contained within the data, namely intra- and inter-source uncertainty, are identified and modeled using the different degrees of freedom of type-2 FSs, thus providing a clear representation and separation of these individual types of uncertainty present in the data. We provide full details of the proposed approach, as well as a series of detailed examples based on both real-world and synthetic data. We perform comparisons with analogue techniques to derive FSs from intervals, namely the interval approach and the enhanced interval approach, and highlight the practical applicability of the proposed approach.

A number of data-driven fuzzy measure (FM) learning techniques have been put forth for the fuzzy integral (FI). Examples include quadratic programming, Gibbs sampling, gradient descent, reward and punishment and evolutionary optimization. However, most approaches focus solely on the minimization of the sum of squared error (SSE). Limited attention has been placed on characterizing and subsequently minimizing model (i.e., FM) complexity. Furthermore, the vast majority of learning techniques are highly susceptible to over-fitting and noise. Herein, we explore a regularization approach to learning the FM for the Choquet FI. We investigate the mathematical motivation for such an approach, its applicability and impact on different types of FMs, and its desirable properties for quadratic programming (QP) based optimization. We show that L\ regularization has a distinct meaning for measure learning and aggregation operators. Experiments are performed and validated with respect to the Shapley index. Specifically, we show that it is possible to reduce the effect of overfitting, we can identify higher quality measures and, if desired, force the learning of fewer numbers of sources.

In this paper, we develop an intelligent neuro-fuzzy controller by using adaptive neuro fuzzy inference system (ANFIS) techniques. We begin by starting with a standard proportional-derivative (PD) controller and use the PD controller data to train the ANFIS system to develop a fuzzy controller. We then propose and validate a method to implement this control strategy on commercial off-the-shelf (COTS) hardware. Using model based design techniques, the models are implemented on an embedded system. This enables the deployment of fuzzy controllers on enthusiast-grade controllers. We evaluate the feasibility of the proposed control strategy in a model-in-the-loop simulation. We then propose a rapid prototyping strategy, allowing us to deploy these control algorithms on a system consisting of a combination of an ARM-based microcontroller and two Arduino-based controllers.

This paper investigates energy management strategies for a power split hybrid electric transmissions using a fuzzy design method. Hybrid Electric Vehicles (HEV) is one of the most promising research topics for developing efficient and environmentally-friendly transportation solutions. A power split hybrid combines the advantages of both series and parallel hybrid configurations by providing a higher degree of freedom to fulfill the demand of the driver while improving overall fuel efficiency at the cost of higher complexity and non-linearity in the control system design. There are two issues that should be addressed for energy management: torque distribution and battery charge sustenance. The fuzzy controller controls the torque request for the internal combustion engine by taking into consideration the vehicle speed, battery state-of-charge, and the normalized torque request from either the driver or an automated driving controller. The controller has several rules to control the internal combustion engine (ICE) torque request. We show that our rule base provides greater control over the ICE operation over a wide range of conditions for optimum fuel consumption. The controller was tested by integrating with a vehicle model and simulated by running it for multiple United States Environmental Protection Agency drive-cycles and then compared to a controller based on a commercially available HEV system.

Community structure in a network usually is an indicator of some important hidden pattern, and thus can deepen our understandings of some phenomenon and also enable useful applications. Even though the introduction of Newman's modularity stimulates a magnitude of modularity-based community detection methods, only a few of them are designed for uncovering fuzzy overlapping communities in a network, which in practice are really common in social networks. Considering that modularity maximization is NP-hard and that a genetic algorithm's ability to find fairly good solutions of an NP-hard problem, an O (n2) genetic algorithm for fuzzy community detection (GAFCD) is proposed based on the generalized modularity, a one-step extension of Newman's modularity. Crossover in GAFCD works as an outer-layer search framework that statistically determines sub search spaces with which a novel mutation operator finds the best partition in each sub search space. We compare our proposed GAFCD method with an existing fuzzy community detection algorithm, MULTICUT spectral FCM (MSFCM), and GALS, one of the most effective disjoint (or crisp) community detection, on 10 real world data sets; it is observed that GAFCD outperforms the other algorithms in terms of finding max-modularity communities. Furthermore, GAFCD is able to find both fuzzy partitions and crisp partitions while MSFCM can only find fuzzy partitions and GALS can only find crisp partitions. This unique feature makes GAFCD the first genetic algorithm for finding truly fuzzy (i.e., inclusive of both fuzzy and crisp communities) max-modularity community structure in a network.

The fuzzy integral (FI) is an extremely flexible aggregation operator. It is used in numerous applications, such as image processing, multicriteria decision making, skeletal age-at-death estimation, and multisource (e.g., feature, algorithm, sensor, and confidence) fusion. To date, a few works have appeared on the topic of generalizing Sugeno's original real-valued integrand and fuzzy measure (FM) for the case of higher order uncertain information (both integrand and measure). For the most part, these extensions are motivated by, and are consistent with, Zadeh's extension principle (EP). Namely, existing extensions focus on fuzzy number (FN), i.e., convex and normal fuzzy set- (FS) valued integrands. Herein, we put forth a new definition, called the generalized FI (gFI), and efficient algorithm for calculation for FS-valued integrands. In addition, we compare the gFI, numerically and theoretically, with our non-EP-based FI extension called the nondirect FI (NDFI). Examples are investigated in the areas of skeletal age-at-death estimation in forensic anthropology and multisource fusion. These applications help demonstrate the need and benefit of the proposed work. In particular, we show there is not one supreme technique. Instead, multiple extensions are of benefit in different contexts and applications.

Increased levels of particulate matter (PM) in the atmosphere have contributed to an increase in mortality and morbidity in communities and are the main contributing factor for respiratory health problems in the population. Currently, PM concentrations are sparsely monitored; for instance, a region of over 2200 square kilometers surrounding Melbourne in Victoria, Australia, is monitored using ten sensor stations. This paper proposes to improve the estimation of PM concentration by complementing the existing high-precision but expensive PM devices with low-cost lower precision PM sensor nodes. Our evaluation reveals that local PM estimation accuracies improve with higher densities of low-precision sensor nodes. Our analysis examines the impact of the precision of the lost-cost sensors on the overall estimation accuracy.

ABSTRACT Two new incremental models for online anomaly detection in data streams at nodes inwireless sensor networksare discussed. These models are incremental versions of a model that uses ellipsoids to detect first, second, and higher‐ordered anomaliesin arrears. The incremental versions can also be used this way but have additional capabilities offered by processing data incrementally as they arrive in time. Specifically, they can detect anomalies ‘on‐the‐fly’ in near real time. They can also be used to track temporal changes in near real‐time because of sensor drift, cyclic variation, or seasonal changes. One of the new models has a mechanism that enables graceful degradation of inputs in the distant past (fading memory). Three real datasets from single sensors in deployed environmental monitoring networks are used to illustrate various facets of the new models. Examples compare the incremental version with the previous batch and dynamic models and show that the incremental versions can detect various types of dynamic anomalies in near real time. Copyright © 2012 John Wiley & Sons, Ltd.

Recent algorithmic and computational improvements have reduced the time it takes to build a minimal spanning tree (MST) for big data sets. In this paper we compare single linkage clustering based on MSTs built with the Filter-Kruskal method to the proposed clusiVAT algorithm, which is based on sampling the data, imaging the sample to estimate the number of clusters, followed by non-iterative extension of the labels to the rest of the big data with the nearest prototype rule. Numerical experiments with both synthetic and real data confirm the theory that clusiVAT produces true single linkage clusters in compact, separated data. We also show that single linkage fails, while clusiVAT finds high quality partitions that match ground truth labels very well. And clusiVAT is fast: it recovers the preferred c = 3 Gaussian clusters in a mixture of 1 million two-dimensional data points with 100% accuracy in 3.1 seconds.

Virtually every sector of business and industry that use computing, including financial analysis, search engines, and electronic commerce, incorporate Big Data analysis into their business model. Sophisticated clustering algorithms are highly desired to deduce the nature of data by assigning labels to unlabeled data. We address two main challenges in Big Data. First, by definition, the volume of Big Data is too large to be loaded into a computer's memory (this volume changes based on the computer used or available). Second, in real-time applications, the velocity of new incoming data prevents historical data from being stored and future data from being accessed. Therefore, we propose our Streaming Kernel Fuzzy c-Means (stKFCM) algorithm, which reduces both computational complexity and space complexity significantly. The proposed stKFCM only requires O(n2) memory where n is the (predetermined) size of a data subset (or data chunk) at each time step, which makes this algorithm truly scalable (as n can be chosen based on the available memory). Furthermore, only 2n2elements of the full N × N (where N >> n) kernel matrix need to be calculated at each time-step, thus reducing both the computation time in producing the kernel elements and the complexity of the FCM algorithm. Empirical results show that stKFCM, even with very small n, can provide clustering performance as accurately as kernel fuzzy c-means run on the entire data set while achieving a significant speedup.

Fuzzy integrals are non-linear combinations of a hypothesis support function and the (possibly subjective) worth of subsets of sources of information, realized by a fuzzy measure. They are used in many applications, with data fusion being the most well-known. In most applications, the fuzzy measure is built by some external knowledge about the worth of subsets of the information sources, whether by a subjective expert or objective sensor property, such as signal-to-noise ratio. In this paper, we investigate fuzzy measures for interval-valued evidence that have no intrinsic known worth; hence, the fuzzy measure cannot or should not be built in the conventional ways. Instead, the fuzzy measure is built directly from the data. We examine the previously proposed fuzzy measure of agreement, which builds the fuzzy measure by a computation of the agreement of combinations of sources (sources from which the contributed evidence has a high degree of agreement with evidence from other sources have a high worth). We also propose a new fuzzy measure of generalized accord that addresses a theoretical weakness in the agreement measure. We compare the two fuzzy measures by performing aggregation experiments with the fuzzy Choquet integral. Tests on both synthetic and real data are performed. We also compare the two measures against the aggregation results obtained by a survey of several example data sets.

We discuss a new formulation of a fuzzy validity index that generalizes the Newman-Girvan (NG) modularity function. The NG function serves as a cluster validity functional in community detection studies. The input data is an undirected graph G = (V, E) that represents a social network. Clusters in V correspond to socially similar substructures in the network. We compare our fuzzy modularity to an existing modularity function using the well-studied Karate Club data set.

The iVAT algorithm reorders (symmetric) dissimilarity data so that an image of the data may reveal cluster substructure. This paper extends the method so that it can handle asymmetric dissimilarity data. The extension is based on replacing the asymmetric input data with its unique least-squared error approximation by a symmetric matrix. Examples are given to illustrate the new method, called asymmetric iVAT (asiVAT).

The so-called kernel-trick is a well-known method for mapping data in a lower dimensional space into a higher dimensional space to measure the similarity (inner product) of the data elements without ever explicitly performing the mapping. The hope is to induce an improved feature space in which to carry out pattern analysis. However, important questions remain, such as i) what is the best kernel, and ii) do some features or sensors require different kernels? One elegant way to address these problems is multiple kernel (MK) aggregation. To date, the research on MKs has predominately studied linear aggregation of kernels, namely weighted sums, e.g., conic and convex sums. In this paper, we propose a new method for kernel aggregation, fuzzy integral aggregation of MKs (FI-MK). We study different FI formulations to determine which ensures production of an aggregated kernel that is a valid Mercer kernel. We show that the Choquet integral (CI) achieves this goal for matrix-wise aggregation. We leverage our theoretical results to propose a genetic algorithm-based classification scheme called FIGA. Experiments on publicly available data sets are provided that demonstrate our FIGA algorithm produces superior results in the context of support vector machine (SVM)-based classification.

The Fuzzy Integral (FI) is a powerful approach for non-linear data aggregation. It has been used in many settings to combine evidence (typically objective) with the known “worth” (typically subjective) of each data source, where the latter is encoded in a Fuzzy Measure (FM). While initially developed for the case of numeric evidence (integrand) and numeric FM, Grabisch et al. extended the FI to the cases of continuous intervals and normal, convex fuzzy sets (i.e., fuzzy numbers). However, in many real-world applications, e.g., explosive hazard detection based on multi-sensor and/or multi-feature fusion, agreement based modeling of survey data, anthropology and forensic science, or computing with respect to linguistic descriptions of spatial relations from sensor data, discontinuous interval and/or non-convex fuzzy set data may arise. The problem is no theory and algorithm currently exists for calculating the FI for such a case. Herein, we propose an extension of the FI to discontinuous interval- and convex normal Interval Fuzzy Set (IFS)-valued integrands (with a numeric FM). Our approach arises naturally from analysis of the Extension Principle. Further, we provide a computationally efficient approach to computing the proposed extension based on the union of the FIs on the combinations of continuous sub-intervals and we demonstrate the approach using examples for both the Choquet FI (CFI) and Sugeno FI (SFI).

We describe a new cluster validity framework (CVF) that compares structure in the data (in dissimilarity form) to the structure of dissimilarity matrices induced by a matrix transformation of the partition being tested. As part of this framework, we show two possible cluster validation measures: one, visual cluster validity, that that uses visual comparison and another one, correlation cluster validity, based on correlation. Unlike many existing measures, the measures we propose can be applied to crisp or soft partitions obtained by any relational or object data clustering algorithm. We illustrate the new measures and compare them to several well-known existing measures using real and artificial data sets.

We discuss a new formulation of a fuzzy validity index that generalizes the Newman-Girvan (NG) modularity function. The NG function serves as a cluster validity functional in community detection studies. The input data is an undirected weighted graph that represents, e.g., a social network. Clusters correspond to socially similar substructures in the network. We compare our fuzzy modularity with two existing modularity functions using the well-studied Karate Club and American College Football datasets.

In prior work, Grabisch put forth a direct (i.e., result of the Extension Principle) generalization of the Sugeno fuzzy integral (FI) for fuzzy set (FS)-valued normal (height equal to one) integrands and number-based fuzzy measures (FMs). Grabisch's proof is based in large on Dubois and Prade's analysis of functions on intervals, fuzzy numbers (thus normal FSs) and fuzzy arithmetic. However, a case not studied is the extension of the FI for sub-normal FS integrands. In prior work, we described a real-world forensic application in anthropology that requires fusion and has sub-normal FS inputs. We put forth an alternative non-direct approach for calculating FS results from sub-normal FS inputs based on the use of the number-valued integrand and number-valued FM Sugeno FI. In this article, we discuss a direct generalization of the Sugeno FI for sub-normal FS integrands and numeric FMs, called the Sub-normal Fuzzy Integral (SuFI). To no great surprise, it turns out that the SuFI algorithm is a special case of Grabisch's generalization. An algorithm for calculating SuFI and its mathematical properties are compared to our prior method, the Non-Direct Fuzzy Integral (NDFI). It turns out that SuFI and NDFI fuse in very different ways. We assert that in some settings, e.g., skeletal age-at-death estimation, NDFI is preferred to SuFI. Numeric examples are provided to stress important inner workings and differences between the FI generalizations.

This paper presents cluster validity for kernel fuzzy clustering. First, we describe existing cluster validity indices that can be directly applied to partitions obtained by kernel fuzzy clustering algorithms. Second, we show how validity indices that take dissimilarity (or relational) data D as input can be applied to kernel fuzzy clustering. Third, we present four propositions that allow other existing cluster validity indices to be adapted to kernel fuzzy partitions. As an example of how these propositions are used, five well-known indices are formulated.We demonstrate several indices for kernel fuzzy c-means (kFCM) partitions of both synthetic and real data.

One of the applications that motivates this research is a system for detection of the anomalies in wireless sensor networks (WSNs). Individual sensor measurements are converted to ellipsoidal summaries; a data matrix D is built using a dissimilarity measure between pairs of ellipsoids; clusters of ellipsoids are suggested by dark blocks along the diagonal of an iVAT (improved Visual Assessment of Tendency) image of D; and finally, the single linkage algorithm extracts clusters from D, using the iVAT image as a guide to the selection of an optimal partition. We illustrate this model for higher dimensional data with synthetic, real and benchmark data sets. Our examples show that two of the four measures, viz, Focal distance and Bhattacharyya distance, provide very similar and reliable bases for estimating cluster structures in sets of higher dimensional ellipsoids, that single linkage can successfully extract the indicated clusters, and that our model can find both first and second order anomalies in WSN data.

Since 1998, a graphical representation used in visual clustering called the reordered dissimilarity image or cluster heat map has appeared in more than 4000 biological or biomedical publications. These images are typically used to visually estimate the number of clusters in a data set, which is the most important input to most clustering algorithms, including the popularly chosen fuzzy c-means and crisp k-means. This paper presents a new formulation of a matrix reordering algorithm, coVAT, which is the only known method for providing visual clustering information on all four types of cluster structure in rectangular relational data. Finite rectangular relational data are an m× n array R of relational values between m row objects Or and n column objects Oc. R presents four clustering problems: clusters in Or, Oc, Or∪c, and coclusters containing some objects from each of Or and Oc. coVAT1 is a clustering tendency algorithm that provides visual estimates of the number of clusters to seek in each of these problems by displaying reordered dissimilarity images. We provide several examples where coVAT1 fails to do its job. These examples justify the introduction of coVAT2, a modification of coVAT1 based on a different reordering scheme. We offer several examples to illustrate that coVAT2 may detect coclusters in R when coVAT1 does not. Furthermore, coVAT2 is not limited to just relational data R. The R matrix can also take the form of feature data, such as gene microarray data where each data element is a real number: Positive values indicate upregulation, and negative values indicate downregulation. We show examples of coVAT2 on microarray data that indicate coVAT2 shows cluster tendency in these data. © 2012 Wiley Periodicals, Inc.

Very large (VL) data or big data are any data that you cannot load into your computer's working memory. This is not an objective definition, but a definition that is easy to understand and one that is practical, because there is a dataset too big for any computer you might use; hence, this is VL data for you. Clustering is one of the primary tasks used in the pattern recognition and data mining communities to search VL databases (including VL images) in various applications, and so, clustering algorithms that scale well to VL data are important and useful. This paper compares the efficacy of three different implementations of techniques aimed to extend fuzzy c-means (FCM) clustering to VL data. Specifically, we compare methods that are based on 1) sampling followed by noniterative extension; 2) incremental techniques that make one sequential pass through subsets of the data; and 3) kernelized versions of FCM that provide approximations based on sampling, including three proposed algorithms. We use both loadable and VL datasets to conduct the numerical experiments that facilitate comparisons based on time and space complexity, speed, quality of approximations to batch FCM (for loadable data), and assessment of matches between partitions and ground truth. Empirical results show that random sampling plus extension FCM, bit-reduced FCM, and approximate kernel FCM are good choices to approximate FCM for VL data. We conclude by demonstrating the VL algorithms on a dataset with 5 billion objects and presenting a set of recommendations regarding the use of different VL FCM clustering schemes.

The VAT algorithm is a visual method for determining the possible number of clusters in, or the cluster tendency of a set of objects. The improved VAT (iVAT) algorithm uses a graph-theoretic distance transform to improve the effectiveness of the VAT algorithm for “tough” cases where VAT fails to accurately show the cluster tendency. In this paper, we present an efficient formulation of the iVAT algorithm which reduces the computational complexity of the iVAT algorithm from O(N^3) to O(N^2). We also prove a direct relationship between the VAT image and the iVAT image produced by our efficient formulation. We conclude with three examples displaying clustering tendencies in three of the Karypis data sets that illustrate the improvement offered by the iVAT transformation. We also provide a comparison of iVAT images to those produced by the Reverse Cuthill-Mckee (RCM) algorithm; our examples suggest that iVAT is superior to the RCM method of display.

The ubiquity of personal computing technology has produced an abundance of staggeringly large data sets-the Library of Congress has stored over 160 terabytes of web data and it is estimated that Facebook alone logs over 25 terabytes of data per day. There is a great need for systems by which one can elucidate the similarity and dissimilarity among and between groups in these data sets. Clustering is one way to find these groups. In this paper, we propose an approximation method for the fuzzy and possibilistic kernel c-means clustering algorithms. Our approximation constrains the cluster centers to be linear combinations of a size m randomly selected subset of the n input objects, where m ≪ n. The proposed algorithm only requires an m × n rectangular portion of the full n × n kernel matrix and the n diagonal values, resulting in significant memory savings. Furthermore, the computational complexity of the c-means algorithm is substantially reduced. We demonstrate that up to 3 orders of magnitude of speedup are possible while achieving almost the same performance as the original kernel c-means algorithm.

Digital data explosion mandates the development of scalable tools to organize the data in a meaningful and easily accessible form. Clustering is a commonly used tool for data organization. However, many clustering algorithms designed to handle large data sets assume linear separability of data and hence do not perform well on real world data sets. While kernel-based clustering algorithms can capture the non-linear structure in data, they do not scale well in terms of speed and memory requirements when the number of objects to be clustered exceeds tens of thousands. We propose an approximation scheme for kernel k-means, termed approximate kernel k-means, that reduces both the computational complexity and the memory requirements by employing a randomized approach. We show both analytically and empirically that the performance of approximate kernel k-means is similar to that of the kernel k-means algorithm, but with dramatically reduced run-time complexity and memory requirements.

A common question asked about unlabeled data sets is how many subsets (or clusters) of objects are represented in the data? The answer to this question is usually obtained by first clustering the data, and then employing a cluster validity measure to validate one or more candidate partitions of the objects. In this paper we describe an universal cluster validity measure that, unlike most existing measures, can be applied to partitions obtained by any relational or object data clustering algorithm. We illustrate the new measure, and compare it to several well known existing measures using a variety of artificial data sets.

We model anomalies in wireless sensor networks with ellipsoids that represent node measurements. Elliptical anomalies (EAs) are level sets of ellipsoids, and classify them as type 1, type 2 and higher order anomalies. Three measures of (dis)similarity between pairs of ellipsoids convert model ellipsoids into dissimilarity data. Clusters in the dissimilarity data may correspond to normal and anomalous measurements and nodes in the network. Assessment of (clustering) tendency is facilitated by visual inspection of (VAT/iVAT) images. Two examples illustrate the potential for anomaly detection.

Fuzzy integrals have been used to fuse the evidence or opinions from a variety of sources. These integrals are nonlinear combinations of the support functions and the (possibly subjective) worth of subsets of the sources of information, realized by a fuzzy measure. There have been many applications and extensions of fuzzy integrals and this paper proposes a fuzzy Choquet integral, where the integrand takes an interval type-2 fuzzy number and the fuzzy measure is real number-valued. Interval type-2 fuzzy numbers encode the second-order uncertainty in a fuzzy number. Type-2 fuzzy numbers have been been shown to be useful in many applications, including computing with words and control systems. We illustrate our method on several numerical examples as well as on a bioinformatics application.

The hard, fuzzy and possibilistic c-means clustering algorithms are widely used for partitioning a set of n objects into c groups. There are cases, however, when more than one type of partition is necessary to correctly describe the belongingness of an object to a group. Previously, Pal, Pal and Bezdek listed some of these cases and proposed a method to simultaneously produce both memberships and typicalities for a set of vectorial object data: the fuzzy possibilistic c-means (FPCM) clustering algorithm. However, FPCM is not directly applicable when the data are represented by object-object relationships. In this paper, we reformulate FPCM so that it can work with A-norm relational data. Extensions and properties of the relational clustering algorithm are also considered.

This paper addresses thecomputing-with-wordsparadigm by presenting an ontological self-organizing map (OSOM), which produces visualization and summarization information about datasets composed of words, namely, ontological data. The specific data that are used in this paper are the Gene Ontology (GO) annotations of genes and gene products. The OSOM is an extension of the SOM, which was initially developed by Kohonen. We adapt the SOM by integrating ontology-based similarity measures and relational-clustering distance measures. We also develop a novel prototype update. We present results on two datasets composed of GO annotations of genes and gene products. An OSOM-based summarization, which produces the term-based summarizations of the trained OSOM network, is also demonstrated. The results show that the OSOM-based visualization method correctly shows the cluster tendency of the genes and gene products and that the summarization provides useful information about the mapped groups of genes and gene products.

Numerous computational schemes have arisen over the years that attempt to learn information about objects based upon the similarity or dissimilarity of one object to another. One such scheme, clustering, looks for self-similar groups of objects. To use clustering algorithms, an investigator must often havea prioriknowledge of the number of clusters, i.e.,$c$, to search for in the data. Moreover, it is often convenient to have ways to rank the returned results, either for a single value of$c$, a range of$c$’s different clustering methods, or any combination thereof. However, the task of assessing the quality of the results, so that$c$may be determined objectively, is currently ill-defined for object–object relationships. To bridge this gap, we generalize three well-known validity indices: the modified Hubert’s Gamma, Xie–Beni, and the generalized Dunn’s indices, to relational data. In doing so, we develop a framework to convert many other validity indices to a relational form. Numerical examples on 12 datasets (samples from four normal mixtures, four real-world object datasets, and four real-world “pure relational” datasets) using the relational duals of the hard, fuzzy, and possibilistic$c$-means cluster algorithms are offered to illustrate and evaluate the new indices.

Clustering aims to identify groups of similar objects. To evaluate the results of cluster algorithms, an investigator uses cluster-validity indices. While the theory of cluster validity is well established for vector object data, little effort has been made to extend it to relationship-based data. As such, this paper proposes a theory of reformulation for object-data validity indices so that they can be used to rank the results produced by the relational -means clustering algorithms. More specifically, we create a class of relational validity indices, which is called dual-relational indices, that are guaranteed under certain, but easily met, constraints to produce the same results and, hence, the same cluster counts, as their object-data counterparts.

Video-based tracking of contours on the human body has been shown to be useful for many applications, including gait and gesture recognition, posture estimation, and activity analysis. We present a contour tracking method that incorporates a novel edge feature and fuzzy contour template. We apply our method in tracking the motions of older adults exercising in a gym environment. The output of our system is a dynamic fuzzy representation of the spine angle of the subject. We show that the method described in this paper is capable of tracking contours even in cases where human silhouette extraction is poor.

This paper presents a new technique for clustering either object or relational data. First, the data are represented as a matrix D of dissimilarity values. D is reordered to D* using a visual assessment of cluster tendency algorithm. If the data contain clusters, they are suggested by visually apparent dark squares arrayed along the main diagonal of an image I(D*) of D*. The suggested clusters in the object set underlying the reordered relational data are found by defining an objective function that recognizes this blocky structure in the reordered data. The objective function is optimized when the boundaries in I(D*) are matched by those in an aligned partition of the objects. The objective function combines measures of contrast and edginess and is optimized by particle swarm optimization. We prove that the set of aligned partitions is exponentially smaller than the set of partitions that needs to be searched if clusters are sought in D. Six numerical examples are given to illustrate various facets of the algorithm. © 2009 Wiley Periodicals, Inc.

This paper presents an ontological self-organizing map (OSOM), which is used to produce visualization and functional summarization information about gene products using gene ontology (GO) similarity measures. The OSOM is an extension of the self-organizing map as initially developed by Kohonen, which trains on data composed of sets of terms. Term-based similarity measures are used as a distance metric as well as in the update of the OSOM training procedure. We present an OSOM-based visualization method that shows the cluster tendency of the gene products. Also demonstrated is an OSOM-based functional summarization which produces the most representative term(s) (MRT) from the GO for each OSOM prototype and, subsequently, each gene product cluster. We validated the results of our method by applying the OSOM to a well-studied set of gene products.

Many important applications in biology have underlying datasets that are relational, that is, only the (dis)similarity between biological objects (amino acid sequences, gene expression profiles, etc.) is known and not their feature values in some feature space. Examples of such relational datasets are the gene similarity matrices obtained from BLAST, gene expression data, or gene ontology (GO) similarity measures. Once a relational dataset is obtained, a common question asked is how many groups of objects are represented in the original dataset. The answer to this question is usually obtained by employing a clustering algorithm and a cluster validity measure. In this article we describe a cluster validity measure for non-Euclidean relational fuzzy c-means that is based on the correlation between a relation induced on the data by the cluster memberships and the original relational data. This validity measure can be applied to partitions made by any fuzzy relational clustering algorithm. We illustrate our measure by validating clusters in several dissimilarity matrices for a set of 194 gene products obtained using BLAST and GO similarities.

This paper addresses the relationship between the visual assessment of cluster tendency (VAT) algorithm and Dunnpsilas cluster validity index. We present an analytical comparison in conjunction with numerical examples to demonstrate that the effectiveness of VAT in showing cluster tendency is directly related to Dunnpsilas index. This analysis is important to understanding the underlying theory of VAT and VAT-based algorithms and, more generally, other algorithms that are based on, or similar to, Primpsilas Algorithm.

There are many function optimization algorithms based on the collective behavior of natural systems -ParticleSwarmOptimization(PSO) andAntColonyOptimization(ACO) are two of the most popular. This paper presents a new adaptation of the PSO algorithm, entitled Roach Infestation Optimization (RIO), that is inspired by recent discoveries in the social behavior of cockroaches. We present the development of the simple behaviors of the individual agents, which emulate some of the discovered cockroach social behaviors. We also describe a ldquohungryrdquo version of the PSO and RIO, which we aptly call Hungry PSO and Hungry RIO. Comparisons with standard PSO show that Hungry PSO, RIO, and Hungry RIO are all more effective at finding the global optima of a suite of test functions.