Utilizing generative adversarial networks (GANs) for oversampling imbalanced data has demonstrated its effectiveness. However, many GAN-based oversampling methods are confronted with a significant challenge, namely, mode collapse, especially when dealing with tabular imbalanced data. In this paper, two unique penalty terms are respectively incorporated into the loss functions of the discriminator and the generator of GAN to promote the generated samples to exhibit not just statistical but also spatial information consistency with the minority samples, thereby alleviating the issue of mode collapse. In contrast to other studies that fix the coefficient of the penalty terms, the optimal coefficients of the penalty terms are adaptively searched using a meta-learning approach, where Bayesian optimization is firstly employed to effectively handle situations involving small size of minority samples in the imbalanced data. We call the proposed model as META_GAN. Experimental results demonstrate that META_GAN outperforms alternative oversampling methods on general tabular and image imbalanced datasets and long-tailed datasets in terms of different metrics.

Kleinberg (2002) introduced three axioms to formalize the behavior of clustering algorithms and presented that no clustering algorithm satisfies them. However, this result usually is inconsistent with the practical experience of clustering algorithms. In this article, we reformulate these axioms to fill the gap and verify the existence of clustering algorithms satisfying the modified axioms when the point set is well-clusterable. In particular, the concept of $(\delta ,\varepsilon)$ -K segmentation is proposed to characterize the set that has the potential to be clustered well. Then, we verify the existence and the uniqueness of $(\delta ,\varepsilon)$ -K segmentation of a set, respectively. Next, we demonstrate that the $(\delta ,\varepsilon)$ -K segmentation is compatible with K-means, Min-Cut, DBSCAN, and several clustering internal evaluation (CIE) indexes. In addition, the ratio $\delta / \varepsilon $ can be treated as the measure of not only characterizing well-clusterable sets but also of evaluating the performance of clustering results, respectively.

Coping with imbalanced data is a challenging task in practical classification problems. One of effective methods to solve imbalanced problems is to oversample the minority class. Gls SMOTE is a classical oversampling method. However, it exhibits two disadvantages, namely, a linear generation and overgeneralization. In this article, an improved synthetic minority oversampling technique (SMOTE) method, FE- SMOTE, is proposed based on the idea of the method of finite elements. FE- SMOTE not only overcomes the above two disadvantages of SMOTE but also can generate samples that are more in line with the density distribution of the original minority class than those generated by the existing SMOTE variants. The originality of the proposed method stems from constructing a simplex for every minority sample and then triangulating it to expand the region of synthetic samples from lines to space. A new definition of the relative size for triangular elements not only helps determine the number of synthetic samples but also weakens the adverse impact of outliers. Generated samples by FE- SMOTE can effectively reflect the local potential distribution structure arising around every minority sample. Compared with 16 commonly studied oversampling methods, FE- SMOTE produces promising results quantified in terms of$G$-mean, AUC,$F$-measure, and accuracy on 22 benchmark imbalanced datasets and the big dataset MNIST.

Data-imbalanced problems are present in many applications. A big gap in the number of samples in different classes induces classifiers to skew to the majority class and thus diminish the performance of learning and quality of obtained results. Most data level imbalanced learning approaches generate new samples only using the information associated with the minority samples through linearly generating or data distribution fitting. Different from these algorithms, we propose a novel oversampling method based on generative adversarial networks (GANs), named OS-GAN. In this method, GAN is assigned to learn the distribution characteristics of the minority class from some selected majority samples but not random noise. As a result, samples released by the trained generator carry information of both majority and minority classes. Furthermore, the central regularization makes the distribution of all synthetic samples not restricted to the domain of the minority class, which can improve the generalization of learning models or algorithms. Experimental results reported on 14 datasets and one high-dimensional dataset show that OS-GAN outperforms 14 commonly used resampling techniques in terms of G-mean, accuracy and F1-score.

Although spiking neural network (SNN) has the advantages of strong brain-likeness and low energy consumption due to the use of discrete spikes for information representation and transmission, its performance still needs to be improved. This article improves SNN in terms of the coding process and the spike response function by invoking fuzzy sets. In terms of coding, a new fuzzy C-means coding (FCMC) method is proposed, which breaks the limitation of uniformly distributed receptive fields of existing coding methods and automatically determines suitable receptive fields that reflect the density distribution of the input data for encoding through the fuzzy C-means clustering. In terms of spike response function, triangular fuzzy numbers instead of the commonly used alpha-type function are used as the spike response function. Different from other functions of fixed shape, width parameters of the proposed function are learnt in the iterative way like weights of synapses do. Experimental results obtained on seven benchmark datasets and two real-world datasets with eleven approaches demonstrate that SNN with triangular spike response functions (abbreviated as T-SNN) combining FCMC can achieve improved performance in terms of accuracy, F-measure, AUC, required epochs, running time, and stability.

This article presents a novel network, contribution-degree-based spiking neural network (CDSNN), which combines ideas of spiking neural network (SNN) and fuzzy set theory. In this framework, two types of information, interval and instantaneous information conveyed by the membrane potential are described by two concepts such as area under membrane potential (AUM) and firing strength. Given that the neuron with large AUM or strong firing strength would enhance the frequency of action potentials of its postsynaptic neurons, the connection between the neuron and its postsynaptic neurons should be strengthened. Combined with an idea of membership function, three contribution degrees ($\boldsymbol{\mu}_E$,$\boldsymbol{\mu}_S$, and$\boldsymbol{\mu}_{ES}$) are defined to quantify the ability of a neuron to provide information for postsynaptic neurons. According to these three degrees, the corresponding SpikeProp learning algorithms, referred to as SPE, SPS, and SPES, are developed. Experimental results obtained on ten benchmark datasets, one high-dimensional feature dataset, one big dataset, and one time series dataset with some commonly used algorithms, networks and CDSNN demonstrate that CDSNN can achieve improved performance in terms of accuracy, generalization, precision, recall and F-measure. The article demonstrates that the mechanism by which interval-instantaneous information is simultaneously learned in a SNN is feasible.

Dropout and DropConnect are two techniques to facilitate the regularization of neural network models, having achieved the state-of-the-art results in several benchmarks. In this paper, to improve the generalization capability of spiking neural networks (SNNs), the two drop techniques are first applied to the state-of-the-art SpikeProp learning algorithm resulting in two improved learning algorithms called SPDO (SpikeProp with Dropout) and SPDC (SpikeProp with DropConnect). In view that a higher membrane potential of a biological neuron implies a higher probability of neural activation, three adaptive drop algorithms, SpikeProp with Adaptive Dropout (SPADO), SpikeProp with Adaptive DropConnect (SPADC), and SpikeProp with Group Adaptive Drop (SPGAD), are proposed by adaptively adjusting the keep probability for training SNNs. A convergence theorem for SPDC is proven under the assumptions of the bounded norm of connection weights and a finite number of equilibria. In addition, the five proposed algorithms are carried out in a collaborative neurodynamic optimization framework to improve the learning performance of SNNs. The experimental results on the four benchmark data sets demonstrate that the three adaptive algorithms converge faster than SpikeProp, SPDO, and SPDC, and the generalization errors of the five proposed algorithms are significantly smaller than that of SpikeProp. Furthermore, the experimental results also show that the five algorithms based on collaborative neurodynamic optimization can be improved in terms of several measures.

Analysis of DNA sequence motifs is becoming increasingly important in the study of gene regulation, and the identification of motif in DNA sequences is a complex problem in computational biology. Motif discovery has attracted the attention of more and more researchers, and varieties of algorithms have been proposed. Most existing motif discovery algorithms fix the motif's length as one of the input parameters. In this paper, a novel method is proposed to identify the optimal length of the motif and the optimal motif with that length, through an iteration process on increasing length numbers. For each fixed length, a modified genetic algorithm (GA) is used for finding the optimal motif with that length. Three operators are used in the modified GA: Mutation that is similar to the one used in usual GA but is modified to avoid local optimum in our case, and Addition and Deletion that are proposed by us for the problem. A criterion is given for singling out the optimal length in the increasing motif's lengths. We call this method AMDILM (an algorithm for motif discovery with iteration on lengths of motifs). The experiments on simulated data and real biological data show that AMDILM can accurately identify the optimal motif length. Meanwhile, the optimal motifs discovered by AMDILM are consistent with the real ones and are similar with the motifs obtained by the three well-known methods: Gibbs Sampler, MEME and Weeder.

In the training of feedforward neural networks, it is usually suggested that the initial weights should be small in magnitude in order to prevent premature saturation. The aim of this paper is to point out the other side of the story: In some cases, the gradient of the error functions is zero not only for infinitely large weights but also for zero weights. Slow convergence in the beginning of the training procedure is often the result of sufficiently small initial weights. Therefore, we suggest that, in these cases, the initial values of the weights should be neither too large, nor too small. For instance, a typical range of choices of the initial weights might be something like (−0.4,−0.1) ∪ (0.1, 0.4), rather than (−0.1, 0.1) as suggested by the usual strategy. Our theory that medium size weights should be used has also been extended to a few commonly used transfer functions and error functions. Numerical experiments are carried out to support our theoretical findings.

Two backpropagation algorithms with momentum for feedforward neural networks with a single hidden layer are considered. It is assumed that the training samples are supplied to the network in a cyclic or an almost-cyclic fashion in the learning procedure, i.e., in each training cycle, each sample of the training set is supplied in a fixed or a stochastic order respectively to the network exactly once. A restart strategy for the momentum is adopted such that the momentum coefficient is set to zero at the beginning of each training cycle. Corresponding weak and strong convergence results are then proved, indicating that the gradient of the error function goes to zero and the weight sequence goes to a fixed point, respectively. The convergence conditions on the learning rate, the momentum coefficient, and the activation functions are much relaxed compared with those of the existing results.

In order to more efficiently realize Boolean functions by using neural networks, we propose a binary product-unit neural network (BPUNN) and a binary π-ς neural network (BPSNN). The network weights can be determined by one-step training. It is shown that the addition " σ," the multiplication " π," and two kinds of special weighting operations in BPUNN and BPSNN can implement the logical operators " ∨," " ∧," and " ¬" on Boolean algebra 〈Z(2),∨,∧,¬,0,1〉 (Z(2)={0,1}), respectively. The proposed two neural networks enjoy the following advantages over the existing networks: 1) for a complete truth table of N variables with both truth and false assignments, the corresponding Boolean function can be realized by accordingly choosing a BPUNN or a BPSNN such that at most 2(N-1) hidden nodes are needed, while O(2(N)), precisely 2(N) or at most 2(N), hidden nodes are needed by existing networks; 2) a new network BPUPS based on a collaboration of BPUNN and BPSNN can be defined to deal with incomplete truth tables, while the existing networks can only deal with complete truth tables; and 3) the values of the weights are all simply -1 or 1, while the weights of all the existing networks are real numbers. Supporting numerical experiments are provided as well. Finally, we present the risk bounds of BPUNN, BPSNN, and BPUPS, and then analyze their probably approximately correct learnability.

From a probabilistic point of view, this paper deduces an optimal initial value of the bias for max-min fuzzy neural network with n input neurons, which converges to 1 as n increases. Supporting numerical experiments are provided.