Understanding the effect of hyperparameters of the network structure on the performance of Convolutional Neural Networks (CNNs) remains the most fundamental and urgent issue in deep learning, and we attempt to address this issue based on the piecewise linear (PWL) function nature of CNNs in this paper. Firstly, the operations of convolutions, ReLUs and Max pooling in a CNN are represented as the multiplication of multiple matrices for a fixed sample in order to obtain an algebraic expression of CNNs, this expression clearly suggests that CNNs are PWL functions. Although such representation has high time complexity, it provides a more convenient and intuitive way to study the mathematical properties of CNNs. Secondly, we develop a tight bound of the number of linear regions and the upper bounds of generalization error for CNNs, both taking into account factors such as the number of layers, dimension of pooling, and the width in the network. The above research results provide a possible guidance for designing and training CNNs.

In multilabel learning, the precise mining and appropriate application of label correlation can improve the effectiveness and generalization of prediction models. In order to characterize label correlation more carefully, the concept of operator-valued kernel is introduced. The value of operator-valued kernel is an operator on Hilbert space, and when applied to a practical problem, the function-valued operator degenerates into a positive-definite matrix, which aims to describe the label correlation. However, existing works focus on the basic theory of operator-valued kernel, and lack way to learn specific kernel from specific datasets, thus the application of operator-valued kernel in practical problem is greatly hindered. In this article, we focus on learning operator-valued kernels with fuzzy rough sets from multilabel datasets and designing learning algorithm for multilabel classification. First, the importance distribution of feature set to different labels at each sample is measured by using kernelized fuzzy rough sets. For a single sample, label correlation matrix is constructed based on the consistency of the importance distribution of features to labels, so as to characterize the correlation information between different labels. By considering the interaction information between two label correlation matrices, the label incidence matrix between two samples is obtained. Therefore, a new operator-valued kernel is defined by using label incidence matrices as elements. This operator-valued kernel is further proved to be an entangled and transformable kernel. On the basis, the proposed operator-valued kernel is applied to develop an efficient learning algorithm for multilabel classification. The generalization error bound of the prediction function is measured by Rademacher complexity. In order to illustrate the effectiveness of our algorithm, the classification experiments and statistical analysis results on twelve multilabel datasets are provided, which are compared with seven high performance algorithms.

In multilabel learning, research on label correlation provides an effective solution to compress the hypothesis space of classifiers. However, this article focus on the label correlation adapted to overall data, while ignoring the locally targeted information presented by some instances. The lack exploration on the distribution of local label correlation in multilabel instance space undoubtedly limits the in-depth application of label correlation in multilabel learning. Based on the fuzzy rough set theory, the instance-level label correlation distribution is first proposed in this article and applied to design a novel multilabel learner. For each multilabel instance, the local importance of features to label is quantitatively analyzed, by considering the decisive influence of input information on decision making. According to coincidence degree between local feature weight distribution for different labels, the instance-level label correlation is constructed. In order to reflect the internal relationship between label variables objectively, the instance-level label correlation distribution is integrated into the empirical label relevance. On the basis, the label relevance matrix is used to define the constraints of the optimization function in a new form. The relative position of subseparating hyperplanes in input space is quantitatively characterized to reduce the complexity of the multilabel classifier and improve the learning performance. The experiment results on 18 multilabel datasets illustrate the effectiveness of our algorithm. The impact of core parameters on the performance is also dissected.

Multioutput regression aims to predict multiple continuous outputs simultaneously using the common set of input variables. The significant challenge arises from modeling relevance between inputs and outputs. Moreover, the shortage of labeled multioutput data and the divergence of data are other factors that impede the development of multioutput regression problems. The recent emergence of transfer learning techniques, which have the ability of leveraging previously acquired knowledge from a similar domain, provide a solution to the above issues. In this article, a novel fuzzy transfer learning method is proposed to tackle the multioutput regression problems in homogeneous and heterogeneous scenarios. By considering output–input dependencies and inter-output correlations, fuzzy rules are extracted to reflect the shared characteristics of different outputs and capture their uniqueness. For a homogeneous scenario, fuzzy rules are first accumulated in a related domain (called the source domain), which has a sufficient amount of training data. Based on different transform strategies, the fuzzy rules are then transferred to improve the new but similar regression tasks in the current domain (called the target domain), where only a few data have multiple responses. On this basis, we handle a more complex heterogeneous scenario by learning a latent input space to reduce the disagreement of variables between domains. The experiment results on thirteen real-world datasets with multiple outputs illustrate the effectiveness of our method. The impact of core coefficients on performance is also analyzed.