Clinical named entity recognition aims to identify and classify clinical terms such as diseases, symptoms, treatments, exams, and body parts in electronic health records, which is a fundamental and crucial task for clinical and translational research. In recent years, deep neural networks have achieved significant success in named entity recognition and many other natural language processing tasks. Most of these algorithms are trained end to end, and can automatically learn features from large scale labeled datasets. However, these data-driven methods typically lack the capability of processing rare or unseen entities. Previous statistical methods and feature engineering practice have demonstrated that human knowledge can provide valuable information for handling rare and unseen cases. In this paper, we propose a new model which combines data-driven deep learning approaches and knowledge-driven dictionary approaches. Specifically, we incorporate dictionaries into deep neural networks. In addition, two different architectures that extend the bi-directional long short-term memory neural network and five different feature representation schemes are also proposed to handle the task. Computational results on the CCKS-2017 Task 2 benchmark dataset show that the proposed method achieves the highly competitive performance compared with the state-of-the-art deep learning methods.

Existing learning models for classification of imbalanced data sets can be grouped as either boundary-based or nonboundary-based depending on whether a decision hyperplane is used in the learning process. The focus of this paper is a new approach that leverages the advantage of both approaches. Specifically, our new model partitions the input space into three parts by creating two additional boundaries in the training process, and then makes the final decision based on a heuristic measurement between the test sample and a subset of selected training samples. Since the original hyperplane used by the underlying original classifier will be eliminated, the proposed model is named the boundary-eliminated (BE) model. Additionally, the pseudoinverse linear discriminant (PILD) is adopted for the BE model so as to obtain a novel classifier abbreviated as BEPILD. Experiments validate both the effectiveness and the efficiency of BEPILD, compared with 13 state-of-the-art classification methods, based on 31 imbalanced and 7 standard data sets.

There is very little practicable significance to prove the equivalency between a pseudo-inverse linear discriminant (PILD) with the desired outputs in reverse proportion to the number of within-class samples and a Fisher linear discriminant (FLD) with the totally projected mean thresholds which are disadvantageous to improve the overall classification accuracy. Even if so, several examples have borne out that a PILD is not wholly equivalent to an FLD. Consequently, the most often used total-projected-mean thresholds usually behave poor. Starting from the customarily desired targets {1, -1}, a simple practicable threshold is gotten, which is only related to sample sizes. By substituting the desired targets with the actually algebraic distances of all training samples, a new threshold is obtained. When the desired targets are different from each other, the weight vector and the threshold given by a PILD are not equal to the ones given by an FLD anymore. At the moment, a PILD is wholly different from an FLD.

A pseudo-inverse linear discriminants has nothing in common with a Fisher linear discriminant (FLD) if the desired outputs of each sample are changeable. With the customarily desired outputs {1, -1}, a simple and size-related threshold is acquired, which. Multiple thresholds related to sample sizes and distribution regions are thus developed, and the optimal ones may be singled out from among by means of the OCA criterions. Enormous experimental results for the benchmark datasets have verified that the PILDs with optimal thresholds have good learning and generalization performances, and even reach the top OCAs for some datasets among the existing classifiers.

Multi-view learning aims to effectively learn from data represented by multiple independent sets of attributes, where each set is taken as one view of the original data. In real-world application, each view should be acquired in unequal cost. Taking web-page classification for example, it is cheaper to get the words on itself (view one) than to get the words contained in anchor texts of inbound hyper-links (view two). However, almost all the existing multi-view learning does not consider the cost of acquiring the views or the cost of evaluating them. In this paper, we support that different views should adopt different representations and lead to different acquisition cost. Thus we develop a new view-dependent cost different from the existing both class-dependent cost and example-dependent cost. To this end, we generalize the framework of multi-view learning with the cost-sensitive technique and further propose a Cost-sensitive Multi-View Learning Machine named CMVLM for short. In implementation, we take into account and measure both the acquisition cost and the discriminant scatter of each view. Then through eliminating the useless views with a predefined threshold, we use the reserved views to train the final classifier. The experimental results on a broad range of data sets including the benchmark UCI, image, and bioinformatics data sets validate that the proposed algorithm can effectively reduce the total cost and have a competitive even better classification performance. The contributions of this paper are that: (1) first proposing a view-dependent cost; (2) establishing a cost-sensitive multi-view learning framework; (3) developing a wrapper technique that is universal to most multiple kernel based classifier.

The classical back-propagation learning algorithms of neural networks suffer from a major disadvantage that of excessive computational burden encountered by processing all the data. Relatively speaking, the samples near the separating boundary have a more important influent on the final weights than those far. This paper presents a dynamic back-propagation algorithm which is just based on those decision boundary samples. The dynamic back-propagation algorithm using those boundary samples to update weights can not only greatly improve the learning speed, but also can improve the classification correction. The experimental results for the Letter data set verified that the proposed method is effective. It is far faster than classical learning algorithm and gets 91.1% classification correction.

This paper studies the iterative learning of Fisher linear discriminants (FLDs) for handwritten digit recognition. We present an epoch-limited iterative learning strategy to update the weight vectors and thresholds on condition that the error rates for the current training subsets come down. The within-class scatter matrices being or approximately singular should be moderately reduced in dimensionality but not added with tiny perturbations. We suggest that the thresholds be given by the mean-projected midpoints but not by the least-mean-squared points. Combining the ideas together, this paper proposes a type of integrated FLDs. The experimental results over the MNIST and USPS handwritten digits have demonstrated that the integrated FLDs have obvious advantages over conventional FLDs in the aspects of learning and generalization performances.

SUMMARY: ChemMapper is an online platform to predict polypharmacology effect and mode of action for small molecules based on 3D similarity computation. ChemMapper collects >350 000 chemical structures with bioactivities and associated target annotations (as well as >3 000 000 non-annotated compounds for virtual screening). Taking the user-provided chemical structure as the query, the top most similar compounds in terms of 3D similarity are returned with associated pharmacology annotations. ChemMapper is designed to provide versatile services in a variety of chemogenomics, drug repurposing, polypharmacology, novel bioactive compounds identification and scaffold hopping studies. AVAILABILITY: http://lilab.ecust.edu.cn/chemmapper/. CONTACT: [email protected] or [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

SUMMARY: Although in silico drug discovery approaches are crucial for the development of pharmaceuticals, their potential advantages in agrochemical industry have not been realized. The challenge for computer-aided methods in agrochemical arena is a lack of sufficient information for both pesticides and their targets. Therefore, it is important to establish such knowledge repertoire that contains comprehensive pesticides' profiles, which include physicochemical properties, environmental fates, toxicities and mode of actions. Here, we present an integrated platform called Pesticide-Target interaction database (PTID), which comprises a total of 1347 pesticides with rich annotation of ecotoxicological and toxicological data as well as 13 738 interactions of pesticide-target and 4245 protein terms via text mining. Additionally, through the integration of ChemMapper, an in-house computational approach to polypharmacology, PTID can be used as a computational platform to identify pesticides targets and design novel agrochemical products. AVAILABILITY: http://lilab.ecust.edu.cn/ptid/. CONTACT: [email protected]; [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

The concentration estimation for multiple kinds of odors is regarded first as multiple two-class classification and then as multiple approximation problems, and solved by multiple single-output multi-layer perceptrons (MLPs) lined up in two parallel rows. A pair of MLPs in cascade is on behalf of a specified odor. n pairs of MLPs represent n kinds of odors, one for one. An MLP in the first row separates its represented odor from the others. Because the two-class training subsets are often unbalanced, the samples from the minority sides are virtually reinforced. The generalization of an MLP is limited in local regions with respect to the distribution of the represented odor. An MLP in the second row approximates the relationship between the responses of the sensor array and the concentrations of the represented odor. A sample is assigned to a kind of odor by the MLP with the maximum output in the first row, and then its concentration is estimated by another MLP in the corresponding pair. The effectiveness of the proposed MLP models is verified by the experiments for 4 kinds of fragrant materials as well as their extended dataset.

This paper focuses on support vector machines (SVMs) with radial basis function (RBF) kernels to solve the large-scale classification problems. We decompose a large-scale learning problem into multiple two-class problems with the one-verse-all decomposition technique, and then propose an adoptively clustering method. An initial support vector (SV) coincides with a certain clustering center, and its width is equal to the max Euclid distance in the clustering region. Therefore, the initial number of SVs is equal to that of the clustering centers, and different RBF kernels are with different widths. The optimization of SVMs is only to determine the Lagrange multipliers. The resulting kernel space for optimization becomes relatively lower in dimensionality, and the final SVs are from a part of the clustering centers. The experimental results for the letter and the handwritten digit recognitions show that the proposed methods are effective.

This paper focuses on how to use class-modular single-hidden-layer perceptrons (MLPs) with sigmoid activation functions (SAFs) to solve the multi-class learning problems, and pays special attention to the unbalanced data sets. Our solutions are as follows. (A) An n-class learning problem first decomposes into n two-class problems (B) A single-output MLP is responsible for solving a two-class problem, separating its represented class with all the other classes, and trained only by the samples from the represented class and some neighboring ones. (C) The samples from the minority classes or in the thin regions are virtually reinforced (D)The generalization region of an MLP is localized. The proposed method is verified effective by the experimental result of letter recognition.

This paper uses multiple approximation model ensembles to solve a multi-input multi-output learning task. An ensemble is on behalf of a specified class, and composed of several multi-input single-output (MISO) approximation models. An MISO model may be either a multivariable cubic polynomial, or a multi-variable quartic polynomial, or a single-hidden-layer perceptron. The number of ensembles is equal to that of the existing classes, and all the members in an ensemble are trained only by the samples from the represented category. The ensemble in which all the members have the most identical viewpoint finally determines the label and position of one sample. The "most identical viewpoint" can be scaled by the corrected relative standard deviation. The proposed method is verified to be effective by a synthetic dataset.

This paper first decomposes an n-class problem into n two-class problems, and then uses n single-output perceptrons to solve them one by one. A single-output perceptron is responsive for forming the decision boundaries of its represented class, and trained only by the samples from the represented class and some neighboring ones. The perceptrons thus have to face with such unfavorable situations as unequal number of samples between two classes, locally sparse and weak distributions, and a tiny part of strange samples. One of solutions is that the samples from the smaller sides or located in the thin regions are virtually reinforced by enlargement factors. And next, the signs of a tiny part of the mislabeled samples are simply changed The experimental results for the IRIS and handwritten digit recognitions show that the proposed methods are effective.

One of solutions for kernel Fisher discriminants (KFDs) to solve the large-scale learning problems is that all the training samples in a class are covered by multiple hyperspheres one by one, and each sphere should include as many samples from the class as possible. In that way, the KFDs can be carried out in some relatively lower dimensional kernel space. This paper clarifies the fact that a nonlinearly separable dataset in the input space does not certainly become linearly separable in the kernel space. We thus propose a kernel nearest neighbor classification method, i.e., a sample is labeled according to the minimum distance between it and the surfaces of the existing kernels. The experimental results for the letter and the handwritten digit recognitions show that the presented method is quite effective for solving the large-scale learning problems.

This paper decomposes a large-scale learning problem into multiple limited-scale pairs of training subsets and cross validation (CV) subsets. One training subset only consists of its own class and some most neighboring samples from the other categories. Naturally, modular multilayer perceptrons (MLPs) come into being. If the final decision region of an MLP is open, its real outputs must be amended. According to the fuzzy set theory, each output of MLPs is added a correction coefficient, which is related to the class mean and covariance. In addition, weight increment correction factors are added to solve the sample disequilibrium problems in training subsets. The result for letter recognition shows that the above methods are quite effective

This paper presents a type of combinative and modular single-layer radial basis function (RBF) neural network classifiers for solving the large-scale learning problems. We pay attention simultaneously to large samples, high dimensionality and multiple categories, not to only one or two terms among them. Above all, we divide a large-scale learning problem into multiple limited-scale simple problems. Each learning subset only includes a small part of samples from the original learning set. And furthermore, we propose a kind of modular single-layer RBF classifiers, in which each module is made up of multiple RBF kernels. The optimization method for determining the number, locations, widths and target values of RBF kernels is gone into details. The real output of one module is the sum of its components, and the class label of a certain sample is finally determined by the module with the maximum output. An application example, i.e., letter recognition, shows that the proposed RBF network is quite effective for solving the large-scale learning problems

In order to use combinative classifiers to effectively solve large-scale learning problems, this paper focuses on the following aspects. (A) Decomposition of large-scale learning problems. (B) Selection of units of combinative classifiers. (C) Transformation of outputs of single classifiers into the grades of membership. We select improved kernel Fisher, Mahalanobis distance, and 10-nearest-neighbor classifier, as the combinative units, only let the most relative part of the original datasets to take part in training a single classifier, and then transform the outputs of each classifier into the same grades of membership. The experiment for letter recognition shows that the proposed method is effective

This paper focuses on combinative and modular approximation models to simultaneously estimate odor classes and strengths. We first decompose a many-to-many approximation task into multiple many-to-one tasks, and then realize them using multiple many-to-one approximation models. A single model is regarded as an expert, and a panel or ensemble is made up of multiple such experts. Each expert is either a multivariate logarithmic regression model, or a multilayer perceptron (MLP), or a support vector machine (SVM). A panel is on behalf of a kind of odor. The most similar panel gives the class label and strength of an odor. The experiment for estimating 4 kinds of fragrant materials shows that the proposed model is effective.

This paper focuses on using radial basis function (RBF) network classifiers to solve the large-scale learning problems. Above all, a large-scale dataset is divided into multiple limited-scale subsets, and each subset only includes a small part of samples from the original dataset. Naturally, modular single-layer RBF classifiers come into being, in which each module is made up of multiple RBF kernels. The number, locations, widths of kernels may adoptively be determined, and the module with the max output gives the class label of a certain sample. This paper clarifies that a nonlinearly separable problem may still keep so in the kernel space. Two-spirals and letter recognition results show that the proposed method is quite effective.

In order to use kernel Fisher discriminant (KFD) classifiers to solve large-scale learning problems, this paper decomposes an n-class dataset into n two-class subsets, and use a subset only composed of a small part of the original dataset in determining the structure of a single KFD classifier. The large number of samples in a class can be further represented by only a small number of prototypes with changeable widths, which are on behalf of kernels. Training samples are not certainly linearly separable in the kernel space, so additional expansive and contractive transformation is needed. Sigmoid functions can be use to implement such tasks. The results of two-spirals and letter recognition show that the proposed method is quite effective.

In order to use combinative classifiers to effectively solve the large-scale learning problems, this paper focuses on the following aspects: (A) decomposition of large-scale learning problems; (B) selection of units of combinative classifiers; and (C) transformation of outputs of single classifiers into the grades of memberships. We select Gaussian kernel, 10-nearest-neighbor, and quadratic polynomial, as the combinative units, only let the most relative part of the original datasets to take part in training a single classifier, and then transform the outputs of each classifier into the same grades of memberships. The experiment for letter recognition shows that the proposed method is effective.

Time-domain analysis and frequency-domain analysis are two angles of view for us to study and survey a continuous function. We observe the function approximation problems from the frequency domain. We consider that a single-frequency sine function can be approximated by two Gaussian kernels in one period. According to that, we present that the first maximum amplitudes as well as their frequencies and initial phases can be used to determine the initial number, centers and widths of radial basis function (RBF) kernels. After the initial structure of an RBF network is determined like that, a small number of RBF kernels can be added in order to further improve the local approximation accuracy. The above viewpoint is verified by two approximation examples.

This paper proposes a task decomposition method, which divides a large-scale learning problem into multiple limited-scale pairs of training subsets and cross validation (CV) subsets. Correspondingly, modular multilayer perceptrons are set up. At first, one training subset only consists of its own class and several most neighboring categories, and then some classes in the CV subset are moved into it according to the generalization error of the module. This work presents an empirical formula for selecting the initial number of hidden nodes, and a method for determining the optimal number of hidden units with the help of singular value decomposition. The result for letter recognition shows that the above methods are quite effective.

This paper sets up an electronic nose, and presents a kind of combinative and modular single-hidden-layer perceptrons. Every module is made up of multiple single-input single-output multilayer perceptrons (MLPs). One MLP is regarded as an expert, and one module consists of several such experts. In electronic noses, one module is behalf of a kind of odor, and determines its similar degrees, namely its strengths. The most similar module gives the class and strength of the odor. By means of enlarging the input components to the range of [0, 6.0] and transforming the standard sigmoid activation function to be f(x)=3(1+exp(-x/3))/sup -1/, the learning speeds of MLPs are sped up. The experiment for simultaneously estimating the classes and concentrations of 4 kinds of fragrant materials, namely ethanol, ethyl acetate, ethyl caproate and ethyl lactate in different concentrations, shows that the proposed method is quite effective.

This paper compares the similarities and differences between multilayer perceptrons (MLPs) and radial basis function (RBF) neural networks, proposes the method of how to decompose a large-sample and multiple-category training set into many small-sample and two-class training subsets. Further-more, we take single-hidden-layer perceptrons and RBF networks as the basis units to construct combinative neural-network-based classifiers. This kind of combinative classifiers has higher classification accuracy and better generalization performances than their component parts. The results for recognizing the handwritten numerals and English letters show that the presented combinative classifiers are quite effective for solving the large-sample, high-dimensional and multiple-category classification problems.

This paper studies the transformation mechanisms of multilayer perceptrons with sigmoid activation functions for classifications. The viewpoint is presented that in the input spaces the hyperplanes determined by the hidden basis functions with values of 0 do not play the role of separate hyperplanes, and furthermore such "hyperplanes" do not certainly go through the marginal regions between different classes. The number of hidden units is only related to the number of categories and the sample distribution shapes. The rank of output matrix of hidden units should be taken as the basis for pruning or growing the hidden nodes. As a result, an empirical formula for optimally determining the number of hidden neurons is proposed. Finally, two examples are given to verify it.