Popular PEFT methods reduce trainable parameter count for fine-tuning by parameterizing new low-rank or sparse trainable weights in parallel to the frozen pre-trained weights W. However, these weights are trained from scratch, and there exists a performance gap between these methods and full fine-tuning, especially in low-budget settings. We introduce VectorFit, a new way of parameterization that efficiently utilizes the existing knowledge embedded in W by adaptively training their singular vectors and biases. We show that utilizing the structural and transformational properties of W in this way can lead to high-rank incremental weight matrices ΔW, comparable to that of full fine-tuning. VectorFit delivers superior results with 9× fewer trainable parameters than the leading PEFT methods. Through comprehensive experiments across 19 datasets covering a wide range of language and vision tasks such as natural language understanding and generation, question answering, image classification, and image generation, we demonstrate that VectorFit surpasses baselines in terms of performance as a function of parameter-efficiency.

Fuzzy clustering is a common way to divide data into groups. Even though it has been improved a lot, fuzzy clustering still has problems while clustering real high-dimensional Big Data with complicated latent distributions. To solve this problem, this study comes up with a way to represent the data in a feature space that was built from a scalable deep neural network using Apache Spark on HPC. In this paper, we proposed SDnnRSIO-FCM, a Scalable Deep Neural Network Random Sampling Iterative Optimization-FCM clustering algorithm, and the SDnnLFCM, a scalable version of the Deep Neural Network Literal Fuzzy c-Means algorithm. We focus on the design and implementation of the proposed SDnnRSIO-FCM and SDnnLFCM algorithms using the Apache Spark cluster in a High-Performance Computing (HPC) environment by representing the data in a feature space produced by the neural network to handle Big Data. First, data is mapped into new feature space to aid in the reconstruction of the original data by providing a good representation. Second, scalable fuzzy clustering is embedded with neural networks to propose deep fuzzy clustering methods. The experimental results conducted on two huge benchmark datasets show that the SDnnRSIO-FCM algorithm outperforms the SDnnLFCM algorithm in terms of Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F-score. Furthermore, the proposed SDnnRSIO-FCM applied to huge soybean protein sequences in comparison with SDnnLFCM shows a significant improvement in terms of Silhouette index (SI), Davies-Bouldin index (DBI), and Calinski-Harabasz index (CHI).

System Evolution Analytics on a system that evolves is a challenge because it makes a State Series SS = {S1, S2…SN} (i.e., a set of states ordered by time) with several inter-connected entities changing over time. We present stability characteristics of interesting evolution rules occurring in multiple states. We defined an evolution rule with its stability as the fraction of states in which the rule is interesting. Extensively, we defined stable rule as the evolution rule having stability that exceeds a given threshold minimum stability (minStab). We also defined persistence metric, a quantitative measure of persistent entity-connections. We explain this with an approach and algorithm for System Network Analytics (SysNet-Analytics), which uses minStab to retrieve Network Evolution Rules (NERs) and Stable NERs (SNERs). The retrieved information is used to calculate a proposed System Network Persistence (SNP) metric. This work is automated as a SysNet-Analytics Tool to demonstrate application on real world systems including: software system, natural-language system, retail market system, and IMDb system. We quantified stability and persistence of entity-connections in a system state series. This results in evolution information, which helps in system evolution analytics based on knowledge discovery and data mining.

Learning in non-stationary environments require modern tools and algorithms to quickly adapt to the new pattern because concept drift can change the underlying distribution. So, the existing assumption that the data is independent and identically distributed may be invalid in data stream scenarios. Given the massive volume of high-speed data streams and the concept drift, traditional machine learning algorithms must be self-adapting. One of the difficulties in handling regression tasks is the complexities of equations for the regression models when combined with drift handling techniques. The high dimensional protein data is a major challenge for bioinformatics researchers to analyse the dynamics of the sequences. This paper proposes a Scalable Fuzzy Clustering induced Regression (SFC-R) algorithm to predict the isoelectric point of the plant protein sequences using Apache Spark clusters. The SFC-R algorithm uses the input features extracted from the plant protein sequences and validates performance in terms of mean squared error (MAE) and root-mean-square error (RMSE). Experiments on plant protein datasets are carried out to validate the high accuracy and robustness of our approach.

Anomalous events occur rarely and are challenging to model. Therefore, automatic recognition of abnormal activities in surveillance videos is a non-trivial task. Though with the availability of video datasets of abnormal activities, there has been some progress, recognition of abnormal activities in real-time with high confidence remains unsolved. Existing video-based anomaly detection techniques using traditional machine learning and deep-learning are compute-intensive and give low recognition accuracy. This paper presents a robust and computationally efficient deep learning-based framework to recognize different real-world anomalies from the video. The proposed scheme uses a Fuzzy rule to summarize the video to scale the problem into fewer frames and the slow-fast neural network for classification. Intuitively, the designed pipeline aims to solve two significant problems that arise with video classification; one is to reduce the redundant frames and avoid the computation of optical flow for a video that has a substantial computational requirement. The proposed scheme tested on the UCF-crime dataset and has achieved recognition accuracy of 53%.

In the past few years, kernel-based one-class extreme learning machine (ELM) receives quite a lot of attention by researchers for offline/batch learning due to its noniterative and fast learning capability. This paper extends this concept for adaptive online learning with regularized kernel-based one-class ELM classifiers for detection of outliers, and are collectively referred to as ORK-OCELM. Two frameworks, viz., boundary and reconstruction, are presented to detect the target class in ORK-OCELM. The kernel hyperplane-based baseline one-class ELM model considers whole data in a single chunk, however, the proposed one-class classifiers are adapted in an online fashion from the stream of training samples. The performance of ORK-OCELM is evaluated on a standard benchmark as well as synthetic datasets for both types of environments, i.e., stationary and nonstationary. While evaluating on stationary datasets, these classifiers are compared against batch learning-based one-class classifiers. Similarly, while evaluating on nonstationary datasets, the comparison is done with incremental learning-based online one-class classifiers. The results indicate that the proposed classifiers yield better or similar outcomes for both. In the nonstationary dataset evaluation, adaptability of the proposed classifiers in a changing environment is also demonstrated. It is further shown that the proposed classifiers have large stream data handling capability even under limited system memory. Moreover, the proposed classifiers gain significant time improvement compared to traditional online one-class classifiers (in all aspects of training and testing). A faster learning ability of the proposed classifiers makes them more suitable for real-time anomaly detection.

Deep Neural Networks are one of the machine learning techniques which are increasingly used in a variety of applications. However, the significantly high memory and computation demands of deep neural networks often limit their deployment on embedded systems. Many recent works have considered this problem by proposing different types of data quantization schemes. However, most of these techniques either require post-quantization retraining of deep neural networks or bear a significant loss in output accuracy. In this paper, we propose a novel quantization technique for parameters of pre-trained deep neural networks. Our technique significantly maintains the accuracy of the parameters and does not require retraining of the networks. Compared to the single-precision floating-point numbers-based implementation, our proposed 8-bit quantization technique generates only ~1% and the ~0.4%, loss in top-1 and top-5 accuracies respectively for VGG16 network using ImageNet dataset.

In this paper, a kernelized version of the random vector functional link network is proposed for multi-label classification. This classifier uses pseudoinverse to find output weights of the network. As pseudoinverse is non-iterative in nature, it requires less fine-tuning to train the network. Kernelization of RVFL makes it robust and stable as no need to tune the number of neuron in the enhancement layer. A threshold function is used with a kernelized random vector functional link network to make it suitable for multi-label learning problems. Experiments performed on three benchmark multi-label datasets bibtex, emotions, and scene shows that proposed classifier outperforms various the existing multi-label classifiers.

Abstract Epilepsy, sometimes called seizure disorder, is a neurological condition that justifies itself as a susceptibility to seizures. A seizure is a sudden burst of rhythmic discharges of electrical activity in the brain that causes an alteration in behaviour, sensation, or consciousness. It is essential to have a method for automatic detection of seizures, as these seizures are arbitrary and unpredictable. A profound study of the electroencephalogram (EEG) recordings is required for the accurate detection of these epileptic seizures. In this study, an Innovative Genetic Programming framework is proposed for classification of EEG signals into seizure and nonseizure. An empirical mode decomposition technique is used for the feature extraction followed by genetic programming for the classification. Moreover, a method for intron deletion, hybrid crossover, and mutation operation is proposed, which are responsible for the increase in classification accuracy and a decrease in time complexity. This suggests that the Innovative Genetic Programming classifier has a potential for accurately predicting the seizures in an EEG signal and hints on the possibility of building a real‐time seizure detection system.

Entities (or components) in an evolving system keeps on evolving, which makes a state series SS = {S1, S2… SN}, where Si is the ith state of the system. There exist connections (or relationships) between entities, which also evolve over system state, and make a series of evolving networks EN = {EN1, EN2… ENN}. We can use these evolving networks to do learning over evolving system states for system evolution analysis. In this paper, we introduce a System Evolution Analytics model, which is based on proposed System Evolution Learning. The network pattern information is trained using graph structure learning. The evolution information is trained using evolution and change learning. We accomplish this by implementing a deep evolution learning. This technique uses an evolving matrix to generate evolving memory in the form of a proposed System Neural Network (SysNN). The SysNN is useful to predict and recommend based on system evolution learning. The technique is prototyped as a tool, which is used to do experiments on six evolving systems. We applied our tool to do system evolution analysis. The experiments are conducted to generate and report evolving memory as SysNN that helps to do recommendation about system.

There are many entities (or components) in a system that keeps on evolving over system states. The connection (or relationship) between entities also keep on evolving over system state, which makes series of evolving networks. Such networks can be studied over evolving state to provide system evolution information for analysis. This can be achieved with the help of hybrid mining approaches. The network rule information can be detected using network rule mining. The network subgraph information can be retrieved using network subgraph mining. The evolution information is detected using evolution mining. In this paper, we introduce a “System Evolution Analytics” model, which is explained using two pattern-mining techniques: network evolution rule mining and network evolution subgraph mining. The first technique retrieves network evolution rules (NERs), and the second technique retrieves network evolution subgraphs (NESs). The two techniques are prototyped as two System Evolution Analytics tools that are used to do experiments on six evolving systems. We demonstrated the application of the tools for the system evolution analysis.

Construction of multi-class classifiers using homogeneous combination of Extreme Learning Machine (ELM) based one-class classifiers have been proposed in this paper. Each class has been trained using individual one-class classifier and any new sample will belong to that class, which will yield maximum value. Proposed methods can be used to detect unknown outliers using multi-class classifiers. Two recently proposed one-class classifiers viz., kernel and random feature mapping based one-class ELM, is extended for multi-class construction in this paper. Further, we construct one-class classifier based multi-class classifier in two ways: with rejection and without rejection of few samples during training. We also perform consistency based model selection for optimal parameters selection in one-class classifier. We have tested the generalization capability of the proposed classifiers on 6 synthetic datasets and two benchmark datasets.

A huge amount of digital data containing useful information, called Big Data, is generated everyday. To mine such useful information, clustering is widely used data analysis technique. A large number of Big Data analytics frameworks have been developed to scale the clustering algorithms for big data analysis. One such framework called Apache Spark works really well for iterative algorithms by supporting in-memory computations, scalability etc. We focus on the design and implementation of partitional based clustering algorithms on Apache Spark, which are suited for clustering large datasets due to their low computational requirements. In this paper, we propose Scalable Random Sampling with Iterative Optimization Fuzzy c-Means algorithm (SRSIO-FCM) implemented on an Apache Spark Cluster to handle the challenges associated with big data clustering. Experimental studies on various big datasets have been conducted. The performance of SRSIO-FCM is judged in comparison with the proposed scalable version of the Literal Fuzzy c-Means (LFCM) and Random Sampling plus Extension Fuzzy c-Means (rseFCM) implemented on the Apache Spark cluster. The comparative results are reported in terms of time and space complexity, run time and measure of clustering quality, showing that SRSIO-FCM is able to run in much less time without compromising the clustering quality.

In this paper, a Quantum-Inspired Evolutionary Fuzzy C-Means (QIE-FCM) algorithm is proposed. The proposed approach find the true number of clusters and the appropriate value of weighted exponent (m) which is required to be known in advance to perform clustering using Fuzzy C-Means (FCM) algorithm. However, the selection of inappropriate value of m and C may lead the algorithm to converge to the local optima. To address the issue of selecting the appropriate value of m and corresponding value of C. In QIE-FCM, the quantum concept is used in classical computer where m is represented in terms of quantum bits (qubits). The QIE-FCM is based on generations. At each generation (g), quantum gates are used to generate a new value of m. For each generated value of m, FCM algorithm is executed by varying values of C. Then, corresponding to m value appropriate value of C is identified by evaluating local fitness function for generation g. To achieve the global best value of m and C, the global fitness function is evaluated by comparing the local best fitness value in current generation with the best fitness value obtained among all the previous generations. To judge the efficacy of QIE-FCM algorithm, it is compared with two well-known indices and three evolutionary fuzzy based clustering algorithm and their performance is evaluated on four benchmark datasets. Furthermore, the sensitivity of QIE-FCM is also experimentally investigated in this paper.

A common problem in the diagnosis of epilepsy is the volatile and unpredictable nature of the epileptic seizures. Hence, it is essential to develop Automatic seizure detection methods. Genetic programming (GP) has a potential for accurately predicting a seizure in an EEG signal. However, the destructive nature of crossover operator in GP decreases the accuracy of predicting the onset of a seizure. Designing constructive crossover and mutation operators (CCM) and integrating local hill climbing search technique with the GP have been put forward as solutions. In this paper, we proposed a hybrid crossover and mutation operator, which uses both the standard GP and CCM-GP, to choose high performing individuals in the least possible time. To demonstrate our approach, we tested it on a benchmark EEG signal dataset. We also compared and analyzed the proposed hybrid crossover and mutation operation with the other state of art GP methods in terms of accuracy and training time. Our method has shown remarkable classification results. These results affirm the potential use of our method for accurately predicting epileptic seizures in an EEG signal and hint on the possibility of building a real time automatic seizure detection system.

In this paper a quantum based binary neural network algorithm is proposed, named as Advance Quantum based Binary Neural Network Learning Algorithm (AQ-BNN). It forms neural network structure constructively by adding neurons at hidden layer. The connection weights and separability parameter are decided using quantum computing concept. Constructive way of deciding network not only eliminates over-fitting and underfitting problem but also saves time. The connection weights have been decided by quantum way, it gives large space to select optimal weights. A new parameter that is quantum separability is introduced here which find optimal separability plane to classify input sample in quantum way. For each connection weights it searches for optimal separability plane. Thus the best separability plane is found out with respect to connection weights. This algorithm is tested with three benchmark data set and produces improved results than existing quantum inspired and other classification approaches.

Cluster validity index is a measure to determine the optimal number of clusters denoted by (C) and an optimal fuzzy partition for clustering algorithms. In this paper, we proposed a new cluster validity index to determine an optimal number of hyper-ellipsoid or hyper-spherical shape clusters generated by Fuzzy C-Means (FCM) algorithm called as VIDSOindex. The proposed validity index jointly exploits all the three measures named as intra-cluster compactness, an inter-cluster separation and overlap between the clusters. The proposed intra-cluster compactness is based on relative variability concept which is a statistical measure of relative dispersion or scattering of data in various dimensions within the clusters. The proposed inter-cluster separation measure indicates the isolation or distance between the fuzzy clusters. The proposed inter-cluster overlap measure determines the degree of overlap between the fuzzy clusters. The best fuzzy partition produced by the VIDSOindex is expected to have low degree of intra-cluster compactness, higher degree of inter-cluster separation and low degree of inter-cluster overlap. The efficacy of VIDSOindex is evaluated on six benchmark data sets and compared with a number of known validity indices. The experimental results and the comparative study demonstrate that, the proposed index is highly effective and reliable in estimating the optimal value of C and an optimal fuzzy partition for each data set because, it is insensitive with change in values of fuzzification parameter denoted by m. In contrast, the other indices [2], [3], [6], [7] fails to achieve the optimal value of C due to it is susceptibility with change in m.

This paper presents a genetic programming (GP) based approach for designing classifiers with feature selection using a modified crossover operator. The proposed GP methodology simultaneously selects a good subset of features and constructs a classifier using the selected features. For a c-class problem, it provides a classifier having c trees. To overcome the difficulties with standard crossover operator, we have used a crossover operator which discovers the best possible crossover site for a subtree and attains higher fitness values while processing fewer individuals. We have tested our method on several datasets having large number of features. We have compared the performance of our method with results available in the literature and found that the proposed method generates good results.

In this paper, a Binary Neural Network Classifier (BNNC) is proposed in which hidden layer training is done in parallel. Learning Algorithm for the BNNC is described, which is based on the principle of Fast Covering Learning Algorithm (FCLA) proposed by Wang and Chaudhari. The BNNC offers high degree of parallelism in hidden layer formation. Each module in the hidden layer of BNNC is exposed to the patterns of only one class. For achieving better accuracy, issue of overlapped classes are also handled. The method is tested on few benchmark datasets, accuracies are within the acceptable range. Due to parallelism at hidden layer level, training time is decreased, therefore, it can be used for voluminous realistic database. An analytical formulation is developed to evaluate the number of hidden layer neurons, it is in the O(log(N)), where N represents the number of inputs.

In this paper we propose a novel Support Vector Machine(SVM) based approach for noisy data removal from datasets. It is observed that the instability present in the dataset greatly affects the overall performance of the any classifier. Hence, we propose a methodology for removal of such instabilities. In the proposed approach, we proceed by determining the clusters formed using support equilibrium points. Then analyzing, each cluster and remove the noisy data using the accuracy factor. Our approach, provide an important feature for reducing the training time and reducing the misclassification test error. The methodology if adopted for classifiers before the training phase will enhance the efficiency of the system. The approach is being tested on benchmark dataset, and it is observed that the efficiency of classifier increased by 15-20%.

This paper presents an approach for designing classifiers for a multiclass problem using Genetic Programming (GP). The proposed approach takes an integrated view of all classes when GP evolves. An individual of the population will be represented using multiple trees. The GP is trained with a set of N training samples in steps. A concept of unfitness of a tree is used in order to improve genetic evolution. Weak trees having poor performance are given more chance to participate in the genetic operations, and thus improve themselves. In this context, a new mutation operation called nondestructive directed point mutation is used, which reduces the destructive nature of mutation operation. The approach is being demonstrated by experimenting on some datasets.

To construct decision boundaries for two-class classification, SVM approach is attractive due to its efficiency. However, this approach is useful for 2-class classification and when the classes (labels) for the data are known. In practice, we have collection of labeled as well as unlabelled data, and it gives rise to semi-supervised classification problem. In this paper, we give a semi-supervised classification algorithm based on support vector machine (SVM). Novel feature of our approach is the formulation of spherical decision boundaries and the exploitation of the dynamical system associated with support function to obtain the number of clusters. The experimental results on a few well-known datasets, namely, Iris dataset, Shuttle landing control dataset, Wisconsin Breast cancer dataset, glass dataset, and balance scale dataset, indicate that our approach results in satisfactory classification as well as generalization accuracy.