Although spiking neural network (SNN) has the advantages of strong brain-likeness and low energy consumption due to the use of discrete spikes for information representation and transmission, its performance still needs to be improved. This article improves SNN in terms of the coding process and the spike response function by invoking fuzzy sets. In terms of coding, a new fuzzy C-means coding (FCMC) method is proposed, which breaks the limitation of uniformly distributed receptive fields of existing coding methods and automatically determines suitable receptive fields that reflect the density distribution of the input data for encoding through the fuzzy C-means clustering. In terms of spike response function, triangular fuzzy numbers instead of the commonly used alpha-type function are used as the spike response function. Different from other functions of fixed shape, width parameters of the proposed function are learnt in the iterative way like weights of synapses do. Experimental results obtained on seven benchmark datasets and two real-world datasets with eleven approaches demonstrate that SNN with triangular spike response functions (abbreviated as T-SNN) combining FCMC can achieve improved performance in terms of accuracy, F-measure, AUC, required epochs, running time, and stability.

This article presents a novel network, contribution-degree-based spiking neural network (CDSNN), which combines ideas of spiking neural network (SNN) and fuzzy set theory. In this framework, two types of information, interval and instantaneous information conveyed by the membrane potential are described by two concepts such as area under membrane potential (AUM) and firing strength. Given that the neuron with large AUM or strong firing strength would enhance the frequency of action potentials of its postsynaptic neurons, the connection between the neuron and its postsynaptic neurons should be strengthened. Combined with an idea of membership function, three contribution degrees ($\boldsymbol{\mu}_E$,$\boldsymbol{\mu}_S$, and$\boldsymbol{\mu}_{ES}$) are defined to quantify the ability of a neuron to provide information for postsynaptic neurons. According to these three degrees, the corresponding SpikeProp learning algorithms, referred to as SPE, SPS, and SPES, are developed. Experimental results obtained on ten benchmark datasets, one high-dimensional feature dataset, one big dataset, and one time series dataset with some commonly used algorithms, networks and CDSNN demonstrate that CDSNN can achieve improved performance in terms of accuracy, generalization, precision, recall and F-measure. The article demonstrates that the mechanism by which interval-instantaneous information is simultaneously learned in a SNN is feasible.

Dropout and DropConnect are two techniques to facilitate the regularization of neural network models, having achieved the state-of-the-art results in several benchmarks. In this paper, to improve the generalization capability of spiking neural networks (SNNs), the two drop techniques are first applied to the state-of-the-art SpikeProp learning algorithm resulting in two improved learning algorithms called SPDO (SpikeProp with Dropout) and SPDC (SpikeProp with DropConnect). In view that a higher membrane potential of a biological neuron implies a higher probability of neural activation, three adaptive drop algorithms, SpikeProp with Adaptive Dropout (SPADO), SpikeProp with Adaptive DropConnect (SPADC), and SpikeProp with Group Adaptive Drop (SPGAD), are proposed by adaptively adjusting the keep probability for training SNNs. A convergence theorem for SPDC is proven under the assumptions of the bounded norm of connection weights and a finite number of equilibria. In addition, the five proposed algorithms are carried out in a collaborative neurodynamic optimization framework to improve the learning performance of SNNs. The experimental results on the four benchmark data sets demonstrate that the three adaptive algorithms converge faster than SpikeProp, SPDO, and SPDC, and the generalization errors of the five proposed algorithms are significantly smaller than that of SpikeProp. Furthermore, the experimental results also show that the five algorithms based on collaborative neurodynamic optimization can be improved in terms of several measures.

Exploitability assessment of vulnerabilities is important for both defenders and attackers. The ultimate way to assess the exploitability is crafting a working exploit. However, it usually takes tremendous hours and significant manual efforts. To address this issue, automated techniques can be adopted. Existing solutions usually explore in depth the crashing paths , i.e., paths taken by proof-of-concept (PoC) inputs triggering vulnerabilities, and assess exploitability by finding exploitable states along the paths. However, exploitable states do not always exist in crashing paths. Moreover, existing solutions heavily rely on symbolic execution and are not scalable in path exploration and exploit generation. In this paper, we propose a novel solution to generate exploit for userspace programs or facilitate the process of crafting a kernel UAF exploit. Technically, we utilize oriented fuzzing to explore diverging paths from vulnerability point. For userspace programs, we adopt a control-flow stitching solution to stitch crashing paths and diverging paths together to generate exploit. For kernel UAF, we leverage a lightweight symbolic execution to identify, analyze and evaluate the system calls valuable and useful for exploiting vulnerabilities. We have developed a prototype system and evaluated it on a set of 19 CTF (capture the flag) programs and 15 realworld Linux kernel UAF vulnerabilities. Experiment results showed it could generate exploit for most of the userspace test set, and it could also facilitate security mitigation bypassing and exploitability evaluation for kernel test set.

In this paper, we study the subspace clustering problem under complex noise. A wide class of reconstruction-based methods model the subspace clustering problem by combining a quadratic data-fidelity term and a regularization term. In a statistical framework, the data-fidelity term assumes to be contaminated by a unimodal Gaussian noise, which is a popular setting in most current subspace clustering models. However, the realistic noise is much more complex than our assumptions. Besides, the coarse representation of the data-fidelity term may depress the clustering accuracy, which is often used to evaluate the models. To address this issue, we propose the mixture of Gaussian regression (MoG Regression) for subspace clustering. The MoG Regression seeks a valid way to model the unknown noise distribution, which approaches the real one as far as possible, so that the desired affinity matrix is better at characterizing the structure of data in the real world, and furthermore, improving the performance. Theoretically, the proposed model enjoys the grouping effect, which encourages the coefficients of highly correlated points are nearly equal. Drawing upon the ideal of the minimum message length, a model selection strategy is proposed to estimate the numbers of the Gaussian components that shows a way how to seek the number of Gaussian components besides determining it by empirical value. In addition, the asymptotic property of our model is investigated. The proposed model is evaluated on the challenging datasets. The experimental results show that the proposed MoG Regression model significantly outperforms several state-of-the-art subspace clustering methods.

The k nearest-neighbour ( k NN) algorithm has enjoyed much attention since its inception as an intuitive and effective classification method. Many further developments of k NN have been reported such as those integrated with fuzzy sets, rough sets, and evolutionary computation. In particular, the fuzzy and rough modifications of k NN have shown significant enhancement in performance. This paper presents another significant improvement, leading to a multi-functional nearest-neighbour (MFNN) approach which is conceptually simple to understand. It employs an aggregation of fuzzy similarity relations and class memberships in playing the critical role of decision qualifier to perform the task of classification. The new method offers important adaptivity in dealing with different classification problems by nearest-neighbour classifiers, due to the large and variable choice of available aggregation methods and similarity metrics. This flexibility allows the proposed approach to be implemented in a variety of forms. Both theoretical analysis and empirical evaluation demonstrate that conventional k NN and fuzzy nearest neighbour, as well as two recently developed fuzzy-rough nearest-neighbour algorithms can be considered as special cases of MFNN. Experimental results also confirm that the proposed approach works effectively and generally outperforms many state-of-the-art techniques.

Subspace clustering aims to cluster a collection of data points lying in a union of subspaces. Based on the assumption that each point can be approximately represented as a linear combination of other points, extensive efforts have been made to compute an affinity matrix in a self-expressive framework for describing the similarity between points. However, the existing clustering methods consider the average feature solutions, which would not be powerful enough to capture the intrinsic relationship between points. In this paper, we present the k-support norm subspace clustering (KSC) method by utilizing k-support norm regularization. The k-support norm trades off the sparsity of ℓ1norm and the uniform shrinkage of ℓ2norm to yield better predictive performance on the data connection. The theoretical analysis of KSC makes up a large proportion of paper. In the noise-free case, we provide the kEBD condition, which ensures the coefficient matrix to be block diagonal. If the data are corrupted, we prove the incompletion-grouping effect for KSC. Moreover, we provide the statistical recovery guarantee for both noise-free and noise cases. The theory analyses show the validity and feasibility of KSC, and the experimental results on multiple challenging databases demonstrate the effectiveness of the proposed algorithm.

BACKGROUND: Essential proteins are indispensable to the survival and development process of living organisms. To understand the functional mechanisms of essential proteins, which can be applied to the analysis of disease and design of drugs, it is important to identify essential proteins from a set of proteins first. As traditional experimental methods designed to test out essential proteins are usually expensive and laborious, computational methods, which utilize biological and topological features of proteins, have attracted more attention in recent years. Protein-protein interaction networks, together with other biological data, have been explored to improve the performance of essential protein prediction. RESULTS: The proposed method SCP is evaluated on Saccharomyces cerevisiae datasets and compared with five other methods. The results show that our method SCP outperforms the other five methods in terms of accuracy of essential protein prediction. CONCLUSIONS: In this paper, we propose a novel algorithm named SCP, which combines the ranking by a modified PageRank algorithm based on subcellular compartments information, with the ranking by Pearson correlation coefficient (PCC) calculated from gene expression data. Experiments show that subcellular localization information is promising in boosting essential protein prediction.

Essential proteins play a crucial role in the survival and development process of life, as they provide all available nutrients to maintain life. Therefore, many researchers pay attention to the identification of essential proteins. As experiments methods are usually costly and time-consuming, more and more computational algorithms have been developed to discover essential proteins based on biological and topological features. Given that the subcellular localization is very important in understanding protein-protein interaction, in this paper, a novel method is proposed to predict essential proteins, which integrates the subcellular compartments information with Pearson correlation coefficient (PCC) of gene expression data. We name this method SCP in this paper. In order to evaluate the prediction performance of our method, several experiments are carried out to compare SCP with other methods. The results demonstrate that SCP has a better prediction performance of essential proteins than other methods.

Visualization is an important method in microbiome data analysis, and dimensionality reduction is a necessary procedure to achieve it. Multidimensional Scaling (MDS) is a popular method, which is necessary to compute the distance matrix. The Unifrac distance is very reasonable and biologically meaningful in the analysis of microbiome data. Due to the complexity of the phylogenetic tree and the high dimensionality of data, MDS needs a large amount of calculations to determine all the distances between pairs. In this paper, we proposed a novel dimensionality reduction algorithm based on Laplace matrix (DRLM) for the analysis of microbiome data. The experimental results indicate that both on synthesized and microbiome data, our algorithm DRLM can not only cluster the data more clearly, but also can significantly reduce the computational cost.

Analysis of DNA sequence motifs is becoming increasingly important in the study of gene regulation, and the identification of motif in DNA sequences is a complex problem in computational biology. Motif discovery has attracted the attention of more and more researchers, and varieties of algorithms have been proposed. Most existing motif discovery algorithms fix the motif's length as one of the input parameters. In this paper, a novel method is proposed to identify the optimal length of the motif and the optimal motif with that length, through an iteration process on increasing length numbers. For each fixed length, a modified genetic algorithm (GA) is used for finding the optimal motif with that length. Three operators are used in the modified GA: Mutation that is similar to the one used in usual GA but is modified to avoid local optimum in our case, and Addition and Deletion that are proposed by us for the problem. A criterion is given for singling out the optimal length in the increasing motif's lengths. We call this method AMDILM (an algorithm for motif discovery with iteration on lengths of motifs). The experiments on simulated data and real biological data show that AMDILM can accurately identify the optimal motif length. Meanwhile, the optimal motifs discovered by AMDILM are consistent with the real ones and are similar with the motifs obtained by the three well-known methods: Gibbs Sampler, MEME and Weeder.

Compared with traditional learning methods such as the back propagation (BP) method, extreme learning machine provides much faster learning speed and needs less human intervention, and thus has been widely used. In this paper we combine the L 1/2 regularization method with extreme learning machine to prune extreme learning machine. A variable learning coefficient is employed to prevent too large a learning increment. A numerical experiment demonstrates that a network pruned L 1/2 regularization has fewer hidden nodes but provides better performance than both the original network and the network pruned by L 2 regularization.

In the training of feedforward neural networks, it is usually suggested that the initial weights should be small in magnitude in order to prevent premature saturation. The aim of this paper is to point out the other side of the story: In some cases, the gradient of the error functions is zero not only for infinitely large weights but also for zero weights. Slow convergence in the beginning of the training procedure is often the result of sufficiently small initial weights. Therefore, we suggest that, in these cases, the initial values of the weights should be neither too large, nor too small. For instance, a typical range of choices of the initial weights might be something like (−0.4,−0.1) ∪ (0.1, 0.4), rather than (−0.1, 0.1) as suggested by the usual strategy. Our theory that medium size weights should be used has also been extended to a few commonly used transfer functions and error functions. Numerical experiments are carried out to support our theoretical findings.

Fuzzy-rough sets play an important role in dealing with imprecision and uncertainty for discrete and real-valued or noisy data. However, there are some problems associated with the approach from both theoretical and practical viewpoints. These problems have motivated the hybridisation of fuzzy-rough sets with kernel methods. Existing work which hybridises fuzzy-rough sets and kernel methods employs a constraint that enforces the transitivity of the fuzzy T-norm operation. In this paper, such a constraint is relaxed and a new kernel-based fuzzy-rough set approach is introduced. Based on this, novel kernel-based fuzzy-rough nearest-neighbour algorithms are proposed. The work is supported by experimental evaluation, which shows that the new kernel-based methods offer improvements over the existing fuzzy-rough nearest neighbour classifiers. The abstract goes here.

Weight-decay method as one of classical complexity regularizations is simple and appears to work well in some applications for multi-layer perceptron network (MPN). This paper shows results for the weak and strong convergence for cyclic and almost cyclic learning MPN with penalty term (weight-decay). The convergence is guaranteed under some relaxed conditions such as the activation functions, learning rate and the assumption for the stationary set of error function. Furthermore, the boundedness of the weights in the training procedure is obtained in a simple and clear way.

Two backpropagation algorithms with momentum for feedforward neural networks with a single hidden layer are considered. It is assumed that the training samples are supplied to the network in a cyclic or an almost-cyclic fashion in the learning procedure, i.e., in each training cycle, each sample of the training set is supplied in a fixed or a stochastic order respectively to the network exactly once. A restart strategy for the momentum is adopted such that the momentum coefficient is set to zero at the beginning of each training cycle. Corresponding weak and strong convergence results are then proved, indicating that the gradient of the error function goes to zero and the weight sequence goes to a fixed point, respectively. The convergence conditions on the learning rate, the momentum coefficient, and the activation functions are much relaxed compared with those of the existing results.

In order to more efficiently realize Boolean functions by using neural networks, we propose a binary product-unit neural network (BPUNN) and a binary π-ς neural network (BPSNN). The network weights can be determined by one-step training. It is shown that the addition " σ," the multiplication " π," and two kinds of special weighting operations in BPUNN and BPSNN can implement the logical operators " ∨," " ∧," and " ¬" on Boolean algebra 〈Z(2),∨,∧,¬,0,1〉 (Z(2)={0,1}), respectively. The proposed two neural networks enjoy the following advantages over the existing networks: 1) for a complete truth table of N variables with both truth and false assignments, the corresponding Boolean function can be realized by accordingly choosing a BPUNN or a BPSNN such that at most 2(N-1) hidden nodes are needed, while O(2(N)), precisely 2(N) or at most 2(N), hidden nodes are needed by existing networks; 2) a new network BPUPS based on a collaboration of BPUNN and BPSNN can be defined to deal with incomplete truth tables, while the existing networks can only deal with complete truth tables; and 3) the values of the weights are all simply -1 or 1, while the weights of all the existing networks are real numbers. Supporting numerical experiments are provided as well. Finally, we present the risk bounds of BPUNN, BPSNN, and BPUPS, and then analyze their probably approximately correct learnability.

From a probabilistic point of view, this paper deduces an optimal initial value of the bias for max-min fuzzy neural network with n input neurons, which converges to 1 as n increases. Supporting numerical experiments are provided.

In this brief, we consider an online gradient method with penalty for training feedforward neural networks. Specifically, the penalty is a term proportional to the norm of the weights. Its roles in the method are to control the magnitude of the weights and to improve the generalization performance of the network. By proving that the weights are automatically bounded in the network training with penalty, we simplify the conditions that are required for convergence of online gradient method in literature. A numerical example is given to support the theoretical analysis.

This paper is concerned with the approximation capability of feedforward neural networks to a compact set of functions. We follow a general approach that covers all the existing results and gives some new results in this respect. To elaborate, we have proved the following: If a family of feedforward neural networks is dense inH, a complete linear metric space of functions, then given a compact setV ⊂ Hand an error boundε, one can fix the quantity of the hidden neurons and the weights between the input and hidden layers, such that in order to approximate any functionf ϵ Vwith accuracyε, one only has to further choose suitable weights between the hidden and output layers.

This paper studies the L(p) approximation capabilities of sum-of-product (SOPNN) and sigma-pi-sigma (SPSNN) neural networks. It is proved that the set of functions that are generated by the SOPNN with its activation function in $L_{loc};p(\mathcal{R})$ is dense in $L;p(\mathcal{K})$ for any compact set $\mathcal{K}\subset \mathcal{R};N$, if and only if the activation function is not a polynomial almost everywhere. It is also shown that if the activation function of the SPSNN is in ${L_{loc};\infty(\mathcal{R})}$, then the functions generated by the SPSNN are dense in $L;p(\mathcal{K})$ if and only if the activation function is not a constant (a.e.).

A pi-sigma network is a class of feedforward neural networks with product units in the output layer. An online gradient algorithm is the simplest and most often used training method for feedforward neural networks. But there arises a problem when the online gradient algorithm is used for pi-sigma networks in that the update increment of the weights may become very small, especially early in training, resulting in a very slow convergence. To overcome this difficulty, we introduce an adaptive penalty term into the error function, so as to increase the magnitude of the update increment of the weights when it is too small. This strategy brings about faster convergence as shown by the numerical experiments carried out in this letter.

A gradient method with momentum for two-layer feedforward neural networks is considered. The learning rate is set to be a constant and the momentum factor an adaptive variable. Both the weak and strong convergence results are proved, as well as the convergence rates for the error function and for the weight. Compared to the existing convergence results, our results are more general since we do not require the error function to be quadratic.

Online gradient methods are widely used for training feedforward neural networks. We prove in this paper a convergence theorem for an online gradient method with variable step size for backward propagation (BP) neural networks with a hidden layer. Unlike most of the convergence results that are of probabilistic and nonmonotone nature, the convergence result that we establish here has a deterministic and monotone nature.