Data reduction is an important step that helps ease the computational intractability for learning techniques when data are large. This is particularly true for the huge datasets that have become commonplace in recent times. The main problem facing both data preprocessors and learning techniques is that data are expanding both in terms of dimensionality and also in terms of the number of data instances. Approaches based on fuzzy-rough sets offer many advantages for both feature selection and classification, particularly for real-valued and noisy data; however, the majority of recent approaches tend to address the task of data reduction in terms of either dimensionality or training data size in isolation. This paper demonstrates how the notion of fuzzy-rough bireducts can be used for the simultaneous reduction of data size and dimensionality. It also shows how bireducts and, therefore, reduced subtables of data can be used not only as a preprocessing tool but also for the learning of compact and robust classifiers. Furthermore, the ideas can also be extended to the unsupervised domain when dealing with unlabeled data. Experimental evaluation of various techniques demonstrate that high levels of simultaneous reduction of both dimensionality and data size can be achieved whilst maintaining robust performance.

Fuzzy-rough set theory has been applied with much success to the problem of feature selection, where there is a clear link between the constructs (i.e., lower approximation, positive region, etc) and the problem (i.e., finding reducts). However, there has not been much development with regards to instance selection. Previous techniques have focused on preserving the positive region or the dependency, or have concentrated on prototype selection. This paper proposes a general instance selection approach that is efficient and effective in finding reductions that maintain the integrity of the underlying class structure. By utilizing the lower approximation information, instances can be removed that have a high similarity with more representative instances of a class. In addition to a threshold-based approach, a fully automated method that requires no user input is also presented. The experimentation demonstrates that the proposed method is indeed effective at reducing the number of instances with minimal information loss.

The k nearest-neighbour ( k NN) algorithm has enjoyed much attention since its inception as an intuitive and effective classification method. Many further developments of k NN have been reported such as those integrated with fuzzy sets, rough sets, and evolutionary computation. In particular, the fuzzy and rough modifications of k NN have shown significant enhancement in performance. This paper presents another significant improvement, leading to a multi-functional nearest-neighbour (MFNN) approach which is conceptually simple to understand. It employs an aggregation of fuzzy similarity relations and class memberships in playing the critical role of decision qualifier to perform the task of classification. The new method offers important adaptivity in dealing with different classification problems by nearest-neighbour classifiers, due to the large and variable choice of available aggregation methods and similarity metrics. This flexibility allows the proposed approach to be implemented in a variety of forms. Both theoretical analysis and empirical evaluation demonstrate that conventional k NN and fuzzy nearest neighbour, as well as two recently developed fuzzy-rough nearest-neighbour algorithms can be considered as special cases of MFNN. Experimental results also confirm that the proposed approach works effectively and generally outperforms many state-of-the-art techniques.

The application of fuzzy-rough sets for the task of feature selection and rule induction has been the topic of much interest recently. However, applications for data instance or object selection have attracted much less attention. In this paper a novel approach for dataset condensation based on ordered weighted aggregation (OWA) fuzzy-rough sets is proposed in the context of the KNN classifier. Initially, a rank is assigned to each data instance of the dataset, based upon a novel measure inspired by fuzzy-rough sets. High quality data instances which possess higher ranks are retained based on another new metric whilst others can then be removed. An additional novel innovation is the elimination of any subjective user-specified threshold in order to determine which particular data instances are candidates for removal. This is in keeping with the rough set ideology of data-driven approaches. A series of non-parametric statistical tests demonstrate that the technique is very effective and can produce useful condensations of the data.

Semi-supervised learning incorporates aspects of both supervised and unsupervised learning. In semi-supervised classification, only some data instances have associated class labels, while others are unlabelled. One particular group of semi-supervised classification approaches are those known as self-labelling techniques, which attempt to assign class labels to the unlabelled data instances. This is achieved by using the class predictions based upon the information of the labelled part of the data. In this paper, the applicability and suitability of fuzzy rough set theory for the task of self-labelling is investigated. An important preparatory experimental study is presented that evaluates how accurately different fuzzy rough set models can predict the classes of unlabelled data instances for semi-supervised classification. The predictions are made either by considering only the labelled data instances or by involving the unlabelled data instances as well. A stability analysis of the predictions also helps to provide further insight into the characteristics of the different fuzzy rough models. Our study shows that the ordered weighted average based fuzzy rough model performs best in terms of both accuracy and stability. Our conclusions offer a solid foundation and rationale that will allow the construction of a fuzzy rough self-labelling technique. They also provide an understanding of the applicability of fuzzy rough sets for the task of semi-supervised classification in general.

Fuzzy rule interpolation forms an important approach for performing inference with systems comprising sparse rule bases. Even when a given observation has no overlap with the antecedent values of any existing rules, fuzzy rule interpolation may still derive a useful conclusion. Unfortunately, very little of the existing work on fuzzy rule interpolation can conjunctively handle more than one form of uncertainty in the rules or observations. In particular, the difficulty in defining the required precise-valued membership functions for the fuzzy sets that are used in conventional fuzzy rule interpolation techniques significantly restricts their application. In this paper, a novel rough-fuzzy approach is proposed in an attempt to address such difficulties. The proposed approach allows the representation, handling and utilisation of different levels of uncertainty in knowledge. This allows transformation-based fuzzy rule interpolation techniques to model and harness additional uncertain information in order to implement an effective fuzzy interpolative reasoning system. Final conclusions are derived by performing rough-fuzzy interpolation over this representation. The effectiveness of the approach is illustrated by a practical application to the prediction of diarrhoeal disease rates in remote villages. It is further evaluated against a range of other benchmark case studies. The experimental results confirm the efficacy of the proposed work.

The task of handwritten Chinese character recognition is one of the most challenging areas of human handwriting classification. The main reason for this is related to the writing system itself which encompasses thousands of characters, coupled with high levels of diversity in personal writing styles and attributes. Much of the existing work for both online and off-line handwritten Chinese character recognition has focused on methods which employ feature extraction and segmentation steps. The preprocessed data from these steps form the basis for the subsequent classification and recognition phases. This paper proposes an approach for handwritten Chinese character recognition and classification using only an image alignment technique and does not require the aforementioned steps. Rather than extracting features from the image, which often means building models from very large training data, the proposed method instead uses the mean image transformations as a basis for model building. The use of an image-only model means that no subjective tuning of the feature extraction is required. In addition by employing a fuzzy-entropy-based metric, the work also entails improved ability to model different types of uncertainty. The classifier is a simple distance-based nearest neighbour classification system based on template matching. The approach is applied to a publicly available real-world database of handwritten Chinese characters and demonstrates that it can achieve high classification accuracy and is robust in the presence of noise.

Due to the explosive growth of stored information worldwide, feature selection (FS) is becoming an increasingly important step, particularly given the abundance of noisy, irrelevant or misleading features. The main aim of FS is to determine a minimal feature subset from a problem domain while retaining a suitably high accuracy in representing the original set of features. However, the problem of finding optimal reductions is challenging as there is always a trade-off between the extent of reduction and the resulting information loss. This topic has been of particular interest in rough and fuzzy-rough set theory, as these provide a mechanism for defining optimality using only the data itself. Evolutionary methods have been used to try to find rough and fuzzy-rough optimal reductions, but these approaches ignore the fact that not all equally-sized reducts have the same utility for classifiers. This paper presents a novel approach for fuzzy-rough feature selection that uses Estimation of Distribution Algorithms to maintain information about the quality of features, to then obtain a better quality reduct that is more useful in general.

Much use has been made of particle swarm optimisation as a tool to solve complex optimisation tasks, and many extensions and modifications to the original algorithm have been proposed. One such extension is related to the murmuration or flocking behaviour of starling birds and their flight trajectories in relation to flock cohesion giving rise to the so-called flock of starlings optimisation algorithm. This algorithm uses the topological model of starling bird flocks as a basis for modifying the original particle swarm optimisation approach. In this paper, two novel approaches for feature selection using fuzzy-rough sets and based upon two different interpretations of the flock of starlings algorithm are proposed. The results demonstrate that the approach can converge quickly and can discover subsets of smaller size and which are more stable than traditional PSO.

Rough set theory has proven to be a useful mathematical basis for developing automated computational approaches which are able to deal with and utilise imperfect knowledge. Fuzzy-rough set theory is an extension to rough set theory and enhances the ability to model uncertainty and vagueness more effectively. There have been many developments in this area which offer robust methods for feature selection or instance selection. However, these are often carried out in isolation rather than considering both types of selection simultaneously. For this purpose, the notion of a bireduct has been proposed recently but the task of finding bireducts of high quality remains a significant challenge. This paper presents a heuristic strategy for the identification of fuzzy-rough bireducts, which is based on a music-inspired global optimisation algorithm called harmony search. The concept of e-bireducts is employed in this approach for the evaluation and improvisation of the candidate solutions. The stochastically-selected bireducts are also utilised to construct classifier ensembles. The presented technique is experimentally evaluated using a number of real-valued benchmark data sets.

Data dimensionality has become a pervasive problem in many areas that require the learning of interpretable models. This has become particularly pronounced in recent years with the seemingly relentless growth in the size of datasets. Indeed, as the number of dimensions increases, the number of data instances required in order to generate accurate models increases exponentially. Feature selection has therefore become not only a useful step in the process of model learning, but rather an increasingly necessary one. Rough set and fuzzy-rough set theory have been used as such dataset pre-processors with much success, however the underlying time/space complexity of the subset evaluation metric is an obstacle to the processing of very large data. This paper proposes a general approach to this problem that employs a novel feature grouping step in order to alleviate the processing overhead for large datasets. The approach is framed within the context of (and applied to) fuzzy-rough sets, although it can be used with other subset evaluation techniques. The experimental evaluation demonstrates that considerable computational effort can be avoided, and as a result efficiency can be improved considerably for larger datasets.

Various strategies have been exploited for the task of feature selection, in an effort to identify more compact and better quality feature subsets. Most existing approaches focus on selecting from a static pool of training instances with a fixed number of original features. However, in practice, data may be gradually refined, and information regarding the problem domain may be actively added or removed. In this paper, a technique based on fuzzy-rough sets is extended to support dynamic feature selection. The proposed method is capable of carrying out on-line selection with incrementally added features or instances. Also, the cases of feature or instance removal are investigated. This brings a novel and beneficial addition to the current research in feature selection. Four possible dynamic selection scenarios are considered, with algorithms proposed in order to handle such individual situations. Simulated experimentation is carried out using real world benchmark data sets, in order to demonstrate the efficacy of the proposed work.

Rough set theory has proven to be a useful mathematical basis for developing automated computational approaches which are able to deal with and utilise imperfect knowledge. Ever since its inception, this theory has been successfully employed for developing computationally efficient techniques for addressing problems such as the discovery of hidden patterns in data, decision rule induction, and feature selection. As an extension to this theory, fuzzy-rough sets enhance the ability to model uncertainty and vagueness more effectively. The efficacy of fuzzy-rough set based approaches for the tasks of feature selection and rule induction is now well established in the literature. Although some work has been carried out using fuzzy-rough set theory for the tasks of feature selection and instance selection in isolation, the potential of this theory for its application to tasks for the simultaneous selection of both features and instances has not been investigated thus far. This paper proposes a novel method for simultaneous instance and feature selection based on fuzzy-rough sets. The initial experimentation demonstrates that the method can significantly reduce both the number of instances and features whilst maintaining high classification accuracies.

Group-wise image alignment or image congealing is an image processing technique which allows the joint alignment of a collection of images. Typically, information theoretic metrics have been employed as the objective function for the assessment of the process of alignment of the images for such methods. However, these objective functions rely on their probabilistic foundations and cannot model the underlying vagueness or uncertainty that is captured by approaches such as those based on fuzzy sets. In this paper a novel fuzzy-entropy based approach is presented for the task of image congealing. This approach allows for much flexibility in terms of employing different definitions for both similarity and fuzzy-entropy. Indeed, the existing approach for image congealing is subsumed as a special case of the approach proposed in this paper. The novel fuzzy-entropy congealing technique is applied to different benchmark problems and also to a medical imaging dataset with good results.

Much work has been carried out in the area of fuzzy-rough sets for supervised learning. However, very little has been accomplished for the unsupervised or semi-supervised tasks. For many real-word applications, it is often expensive, time-consuming and difficult to obtain labels for all data objects. This often results in large quantities of data which may only have very few labelled data objects. This paper proposes a novel fuzzy-rough based semi-supervised self-learning or self-training approach for the assignment of labels to unlabelled data. Unlike other semi-supervised approaches, the proposed technique requires no subjective thresholding or domain information. An experimental evaluation is performed on artificial data and also applied to a real-world mammographic risk assessment problem with encouraging results.

Fuzzy-rough sets play an important role in dealing with imprecision and uncertainty for discrete and real-valued or noisy data. However, there are some problems associated with the approach from both theoretical and practical viewpoints. These problems have motivated the hybridisation of fuzzy-rough sets with kernel methods. Existing work which hybridises fuzzy-rough sets and kernel methods employs a constraint that enforces the transitivity of the fuzzy T-norm operation. In this paper, such a constraint is relaxed and a new kernel-based fuzzy-rough set approach is introduced. Based on this, novel kernel-based fuzzy-rough nearest-neighbour algorithms are proposed. The work is supported by experimental evaluation, which shows that the new kernel-based methods offer improvements over the existing fuzzy-rough nearest neighbour classifiers. The abstract goes here.

The accuracy of methods for the assessment of mammographic risk analysis is heavily related to breast tissue characteristics. Previous work has demonstrated considerable success in developing an automatic breast tissue classification methodology which overcomes this difficulty. This paper proposes a unified approach for the application of a number of rough and fuzzy-rough set methods to the analysis of mammographic data. Indeed this is the first time that fuzzy-rough approaches have been applied to this particular problem domain. In the unified approach detailed here feature selection methods are employed for dimensionality reduction developed using rough sets and fuzzy-rough sets. A number of classifiers are then used to examine the data reduced by the feature selection approaches and assess the positive impact of these methods on classification accuracy. Additionally, this paper also employs a new fuzzy-rough classifier based on the nearest neighbour classification algorithm. The novel use of such an approach demonstrates its efficiency in improving classification accuracy for mammographic data, as well as considerably removing redundant, irrelevant, and noisy features. This is supported with experimental application to two well-known datasets. The overall result of employing the proposed unified approach is that feature selection can identify only those features which require extraction. This can have the positive effect of increasing the risk assessment accuracy rate whilst additionally reducing the time required for expert scrutiny, which in-turn means the risk analysis process is potentially quicker and involves less screening.

Feature Selection (FS) or Attribute Reduction techniques are employed for dimensionality reduction and aim to select a subset of the original features of a data set which are rich in the most useful information. The benefits of employing FS techniques include improved data visualization and transparency, a reduction in training and utilization times and potentially, improved prediction performance. Many approaches based on rough set theory up to now, have employed the dependency function, which is based on lower approximations as an evaluation step in the FS process. However, by examining only that information which is considered to be certain and ignoring the boundary region, or region of uncertainty, much useful information is lost. This paper examines a rough set FS technique which uses the information gathered from both the lower approximation dependency value and a distance metric which considers the number of objects in the boundary region and the distance of those objects from the lower approximation. The use of this measure in rough set feature selection can result in smaller subset sizes than those obtained using the dependency function alone. This demonstrates that there is much valuable information to be extracted from the boundary region. Experimental results are presented for both crisp and real-valued data and compared with two other FS techniques in terms of subset size, runtimes, and classification accuracy.

For supervised learning, feature selection algorithms attempt to maximise a given function of predictive accuracy. This function usually considers the ability of feature vectors to reflect decision class labels. It is therefore intuitive to retain only those features that are related to or lead to these decision classes. However, in unsupervised learning, decision class labels are not provided, which poses questions such as; which features should be retained? and, why not use all of the information? The problem is that not all features are important. Some of the features may be redundant, and others may be irrelevant and noisy. In this paper, some new fuzzy-rough set-based approaches to unsupervised feature selection are proposed. These approaches require no thresholding or domain information, and result in a significant reduction in dimensionality whilst retaining the semantics of the data.

Dataset dimensionality is undoubtedly the single most significant obstacle which exasperates any attempt to apply effective computational intelligence techniques to problem domains. In order to address this problem a technique which reduces dimensionality is employed prior to the application of any classification learning. Such feature selection (FS) techniques attempt to select a subset of the original features of a dataset which are rich in the most useful information. The benefits can include improved data visualisation and transparency, a reduction in training and utilisation times and potentially, improved prediction performance. Methods based on fuzzy-rough set theory have demonstrated this with much success. Such methods have employed the dependency function which is based on the information contained in the lower approximation as an evaluation step in the FS process. This paper presents three novel feature selection techniques employing fuzzy entropy to locate fuzzy-rough reducts. This approach is compared with two other fuzzy-rough feature selection approaches which utilise other measures for the selection of subsets.

Feature selection (FS) is a technique for dimensionality reduction. Its aims are to select a subset of the original features of a dataset which are rich in the most useful information. The benefits include improved data visualisation, transparency, a reduction in training and utilisation times and potentially, improved prediction performance. Many approaches based on rough set theory have employed the dependency function which is based on the information contained in the lower approximation as an evaluation step in the FS process with much success. This paper presents a novel rough set FS technique which uses the information of both the lower approximation dependency value and a distance metric for the consideration of objects in the boundary region. The use of this measure in rough set feature selection can result in smaller subset sizes than those obtained using the dependency function alone.

Feature selection (FS) is a dimensionality reduction technique that aims to select a subset of the original features of a dataset which offer the most useful information. The benefits of feature selection include improved data visualisation, transparency, reduction in training and utilisation times and improved prediction performance. Methods based on fuzzy-rough set theory (FRFS) have employed the dependency function to guide the process with much success. This paper presents a novel fuzzy-rough FS technique which is guided by fuzzy entropy. The use of this measure in fuzzy-rough feature selection can result in smaller subset sizes than those obtained through FRFS alone, with little loss or even an increase in overall classification accuracy.