We propose an embedded/integrated feature selection method based on neural networks with Group Lasso penalty. Group Lasso regularization is considered to produce sparsity on the inputs to the network, i.e., for selection of useful features. Lasso based feature selection using a multi-layer perceptron usually requires an additional set of weights, while our Group Lasso formulation does not require that. However, Group Lasso penalty is non-differentiable at the origin. This may lead to oscillations in numerical simulations and make it difficult to analyze theoretically. To address this issue, four smoothing Group Lasso penalties are introduced. A rigorous proof for the convergence of the proposed algorithm is presented under suitable assumptions. To verify the effectiveness, a three-step algorithmic architecture is adopted in implementation. Experimental results on several datasets validate the theoretical results and demonstrate the competitive performance of the proposed method.

The class imbalance problem in machine learning occurs when certain classes are underrepresented relative to the others, leading to a learning bias toward the majority classes. To cope with the skewed class distribution, many learning methods featuring minority oversampling have been proposed, which are proved to be effective. To reduce information loss during feature space projection, this study proposes a novel oversampling algorithm, named minority oversampling in kernel adaptive subspaces (MOKAS), which exploits the invariant feature extraction capability of a kernel version of the adaptive subspace self-organizing maps. The synthetic instances are generated from well-trained subspaces and then their pre-images are reconstructed in the input space. Additionally, these instances characterize nonlinear structures present in the minority class data distribution and help the learning algorithms to counterbalance the skewed class distribution in a desirable manner. Experimental results on both real and synthetic data show that the proposed MOKAS is capable of modeling complex data distribution and outperforms a set of state-of-the-art oversampling algorithms.

Features selected by a supervised/ unsupervised technique often include redundant or correlated features. While use of correlated features may result in an increase in the design and decision making cost, removing redundancy completely can make the system vulnerable to measurement errors. Most feature selection schemes do not account for redundancy at all, while a few supervised methods try to discard correlated features. We propose a novel unsupervised feature selection scheme (UFeSCoR), which not only discards irrelevant features, but also selects features with controlled redundancy. Here, the number of selected features can also be directed. Our algorithm optimizes an objective function, which tries to select a specified number of features, with a controlled level of redundancy, such that the topology of the original data set can be maintained in the reduced dimension. Here, we have used Sammon's error as a measure of preservation of topology. We demonstrate the effectiveness of the algorithm in terms of choosing relevant features, controlling redundancy, and selecting a given number of features using several data sets. We make a comparative study with five unsupervised feature selection methods. Our results reveal that the proposed method can select useful features with controlled redundancy.

Many attempts have been made to analyze gene expression data. Typical goals of such analysis include discovery of subclasses, designing predictors/classifiers for diseases, identifying marker genes, and trying to get a deeper understanding of underlying biological process. Success of each of these tasks strongly depends on the features used to solve the problem. The high dimensional nature of expression profiles makes the task very difficult. Consequently, many researchers have used some feature selection criteria to reduce the dimensionality of the problem. These approaches are off-line in nature, as feature selection is done in a separate phase from the system design phase. These approaches ignore the fact that utility of features depends on both the problem that is solved and the tool that is used to solve the problem. We here propose to use a novel neural scheme that picks up the necessary features on-line when the system learns the classification task. Because it considers all the features at one go, it does not miss any subtle combination of these features. We demonstrate the effectiveness of our on-line feature selection (OFS) scheme to distinguish between acute myeloid leukemia (AML) and acute lymphoblastic leukemia (ALL) cancer expression data set. Our scheme could identify only five genes that can produce results as good as or even better than what is reported in the literature on this data set. It identifies an important marker gene that alone has a very good discriminating power. This analysis method is quite general in nature and can be effectively used in other areas of bioinformatics. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 453–467, 2006.

The mountain clustering method and the subtractive clustering method are useful methods for finding cluster centers based on local density in object data. These methods have been extended to shell clustering. In this article, we propose a relational mountain clustering method (RMCM), which produces a set of (proto) typical objects as well as a crisp partition of the objects generating the relation, using a new concept that we call relational density. We exemplify RMCM by clustering several relational data sets that come from object data. Finally, RMCM is applied to web log analysis, where it produces useful user profiles from web log data. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 375–392, 2005.

A genetic algorithm (GA)-based scheme for learning fuzzy rules for controllers, called an optimized fuzzy logic controller (OFLC) was proposed by Chan, Xie and Rad (2000). In this article we first analyze their OFLC and discuss some of its limitations. We also propose some modifications on an OFLC to eliminate those limitations. Then, for systems with symmetrical rule base we propose a new method to reduce the number of rules, which reduces the search space as well as the design time. We define a fitness function that reduces the number of rules maintaining the performance of the rule set. The trade-off between the number of rules and the performance can be decided by changing the parameters of our fitness function. We theoretically analyzed the properties of “one-step change mutation” and compared that with our mutation scheme. The proposed scheme, for the inverted pendulum problem can find rule sets containing <5% of all possible fuzzy rules, having good integral time absolute error (ITAE) and it takes only a few steps to balance the system over the entire input space. To show the superiority of our scheme, we compare it with other methods. © 2003 Wiley Periodicals, Inc.

The mountain method of clustering and its relative, the subtractive clustering method, are studied here. A scheme to improve the accuracy of the prototypes obtained by the mountain method is proposed. Finally the mountain circular shell method to detect circular shells by using the mountain function is proposed. The proposed method is tested extensively on several synthetic data sets, and the results obtained are quite satisfactory. © 2000 John Wiley & Sons, Inc.

We justify the need for a connectionist implementation of compositional rule of inference (COI) and propose a network architecture for the same. We call it COIN—the compositional rule of inferencing. Given a relational representation of a set of rules, the proposed architecture can realize the COI. The outcome of COI depends on the choice of the implication function and also on choice of inferencing scheme. The problem of choosing an appropriate implication function is avoided through neural learning. The system automatically finds an “optimal” relation to represent a set of fuzzy rules. We suggest a suitable modeling of connection weights so as to ensure learned weights lie in [0, 1]. We demonstrate through numerical examples that the proposed neural realization can find a much better representation of the rules than that by usual implication and hence results in much better conclusions than the usual COI. Numerical examples exhibit that COIN outperforms not only usual COI but also some of the previous neural implementations of fuzzy logic. ©1999 John Wiley & Sons, Inc.

In this paper we analyze the neural network implementation of fuzzy logic proposed by Keller et al. [Fuzzy Sets Syst., 45, 1–12 (1992)], derive a learning algorithm for obtaining an optimal α for the net, and, for a special case, we show how one can directly (avoiding training) compute the optimal α. We address how training data can be generated for such a system. Effectiveness of the optimal α is then established through numerical examples. In this regard, several indices for performance evaluation are discussed. Finally, we propose a new architecture and demonstrate its effectiveness with numerical examples. © 1998 John Wiley & Sons, Inc.