Collaborative filtering (CF) has been extensively studied in recommendation, spawning various solutions. While graph convolution networks (GCNs) are effective at representation learning, their efficiency is lacking. Comparatively, item similarity model efficiently establishes direct relationships between items. In spite of this, the modeling problem grows quadratically as the number of items increases. This poses critical scalability issues. In this paper, through an investigation of the latest GCN model, we reveal the feasibility of optimizing the process of similarity modeling using the underlying group structure in the item set. Based on these findings, we propose a novel model which introduces graph partitioning to reduce the scale of similarity modeling problem, dubbed FPSR+. Specifically, we divide similarity modeling of items into sub-problems within each partition, and incorporate global and local prior knowledge to alleviate information loss. Following an analysis of the properties of different items in partitioning, we propose a new hub set selection strategy that improves the robustness of FPSR+ in the small partition case. Extensive experiments on four real-world datasets demonstrate the superior performance of FPSR+ compared with state-of-the-art GCN models and item similarity models, as well as several-fold speedups and reductions in parameter storage.

Pathology image are essential for accurately interpreting lesion cells in cytopathology screening, but acquiring high-resolution digital slides requires specialized equipment and long scanning times. Though super-resolution (SR) techniques can alleviate this problem, existing deep learning models recover pathology image in a black-box manner, which can lead to untruthful biological details and misdiagnosis. Additionally, current methods allocate the same computational resources to recover each pixel of pathology image, leading to the sub-optimal recovery issue due to the large variation of pathology image. In this paper, we propose the first hierarchical reinforcement learning framework named Spatial-Temporal hierARchical Reinforcement Learning (STAR-RL), mainly for addressing the aforementioned issues in pathology image super-resolution problem. We reformulate the SR problem as a Markov decision process of interpretable operations and adopt the hierarchical recovery mechanism in patch level, to avoid sub-optimal recovery. Specifically, the higher-level spatial manager is proposed to pick out the most corrupted patch for the lower-level patch worker. Moreover, the higher-level temporal manager is advanced to evaluate the selected patch and determine whether the optimization should be stopped earlier, thereby avoiding the over-processed problem. Under the guidance of spatial-temporal managers, the lower-level patch worker processes the selected patch with pixel-wise interpretable actions at each time step. Experimental results on medical images degraded by different kernels show the effectiveness of STAR-RL. Furthermore, STAR-RL validates the promotion in tumor diagnosis with a large margin and shows generalizability under various degradations. The source code is available at https://github.com/CUHK-AIM-Group/STAR-RL.

Existing partial sequence labeling models mainly focus on a max-margin framework that fails to provide an uncertainty estimation of the prediction. Furthermore, the unique ground-truth disambiguation strategy employed by these models may include wrong label information for parameter learning. In this article, we propose structured Gaussian processes for partial sequence labeling (SGPPSL), which encodes uncertainty in the prediction and does not need extra effort for model selection and hyperparameter learning. The model employs factor-as-piece approximation that divides the linear-chain graph structure into the set of pieces, which preserves the basic Markov random field structure and effectively avoids handling a large number of candidate output sequences generated by partially annotated data. Then, confidence measure is introduced in the model to address different contributions of candidate labels, which enables the ground-truth label information to be utilized in parameter learning. Based on the derived lower bound of the variational lower bound of the proposed model, variational parameters and confidence measures are estimated in the framework of alternating optimization. Moreover, a weighted Viterbi algorithm is proposed to incorporate confidence measures to sequence prediction, which considers label ambiguity arose from multiple annotations in the training data and thus helps improve the performance. SGPPSL is evaluated on several sequence labeling tasks and the experimental results show the effectiveness of the proposed model.

In collaborative filtering, distance metric learning has been applied to matrix factorization techniques with promising results. However, matrix factorization lacks the ability of capturing collaborative information, which has been remarked by recent works and improved by interpreting user interactions as signals. This paper aims to find out how metric learning connect to these signal-based models. By adopting a generalized distance metric, we discovered that in signal-based models, it is easier to estimate the residual of distances, which refers to the difference between the distances from a user to a target item and another item, rather than estimating the distances themselves. Further analysis also uncovers a link between the normalization strength of interaction signals and the novelty of recommendation, which has been overlooked by existing studies. Based on the above findings, we propose a novel model to learn a generalized distance user-item distance metric to capture user preference in interaction signals by modeling the residuals of distance. The proposed CoRML model is then further improved in training efficiency by a newly introduced approximated ranking weight. Extensive experiments conducted on 4 public datasets demonstrate the superior performance of CoRML compared to the state-of-the-art baselines in collaborative filtering, along with high efficiency and the ability of providing novelty-promoted recommendations, shedding new light on the study of metric learning-based recommender systems.

Collaborative filtering (CF) is widely searched in recommendation with various types of solutions. Recent success of Graph Convolution Networks (GCN) in CF demonstrates the effectiveness of modeling high-order relationships through graphs, while repetitive graph convolution and iterative batch optimization limit their efficiency. Instead, item similarity models attempt to construct direct relationships through efficient interaction encoding. Despite their great performance, the growing item numbers result in quadratic growth in similarity modeling process, posing critical scalability problems. In this paper, we investigate the graph sampling strategy adopted in latest GCN model for efficiency improving, and identify the potential item group structure in the sampled graph. Based on this, we propose a novel item similarity model which introduces graph partitioning to restrict the item similarity modeling within each partition. Specifically, we show that the spectral information of the original graph is well in preserving global-level information. Then, it is added to fine-tune local item similarities with a new data augmentation strategy acted as partition-aware prior knowledge, jointly to cope with the information loss brought by partitioning. Experiments carried out on 4 datasets show that the proposed model outperforms state-of-the-art GCN models with 10x speed-up and item similarity models with 95% parameter storage savings.

This paper focuses on opinion summarization for constructing subjective and concise summaries representing essential opinions of online text reviews. As previous works rarely focus on the relationship between opinions, topics, and sentences, we propose a set of new requirements for Opinion-Topic-Sentence, which are essential for performing opinion summarization. We prove that Opinion-Topic-Sentence can be theoretically analyzed by submodular information measures. Thus, our proposed method can reduce redundant information, strengthen the relevance to given topics, and informatively represent the underlying emotional variations. While conventional methods require human-labeled topics for extractive summarization, we use unsupervised topic modeling methods to generate topic features. We propose four submodular functions and two optimization algorithms with proven performance bounds that can maximize opinion summarization's utility. An automatic evaluation metric, Topic-based Opinion Variance, is also derived to compensate for ROUGE-based metrics of opinion summarization evaluation. Four large, diversified, and representative corpora, OPOSUM, Opinosis, Yelp, and Amazon reviews, are used in our study. The results on these online review texts corroborate the efficacy of our proposed metric and framework.

The rapid development of Internet services and social platforms encourages users to share their opinions. To help users give valuable comments, content providers expect the recommender system to offer appropriate suggestions, including specific features of the item described in texts and emojis, which are all considered aspects of the user reviews. Hence, the review aspect recommendation task has become significant, where the key lies in handling personal preferences and semantic correlations between suggested items. This article proposes a correlation-aware review aspect recommender (CARAR) system model by constructing self-representation correlations between different views of review aspects, including textual aspects and emojis to make a personalized recommendation. The dependencies between different textual aspects and emojis can be identified and utilized to facilitate the factorization process to learn user and item latent factors. The cross-view correlation mapping between textual aspects and emojis can be built to enhance the recommendation performance. Moreover, the additional information in the real-world environment is also applied to our model to adjust the recommendation results. We constructed experiments on five self-collected and public datasets and compared with six existing models. The results show that our model can outperform the existing models on review aspects recommendation tasks, validating the effectiveness of our approach.

Multioutput learning targeting predicting multiple outputs for each input has attracted increasing attention due to its ability to handle diverse data types in outputs. Most feature extraction-based methods for tackling multioutput problems neglect the local correlation between different outputs, while most sample extraction-based methods overlook the gap between inputs and outputs. To overcome these two major limitations, in this article, we propose a topic-aware method where we assume inputs and outputs can be jointly embedded in a topic space. Both feature and sample extraction are performed in a latent topic space that encodes output correlation at a topic level and aligns inputs and outputs in an explainable manner. Different from current topic models, we extract independent and interdependent topics based on informative topics. These two components are verified as useful in performance improvement in an ablation study. The proposed method can also be easily extended and applied in multiview learning. The experimental results of the proposed topic-aware model on multiple benchmarks for different multioutput tasks in single-/multi-views with$p$-values less than 0.05 indicate that the improvement achieved by the proposed method over compared baselines is statistically significant.

Existing disambiguation strategies for partial structured output learning just cannot generalize well to solve the problem that there are some candidates that can be false positive or similar to the ground-truth label. In this article, we propose a novel weak disambiguation for partial structured output learning (WD-PSL). First, a piecewise large margin formulation is generalized to partial structured output learning, which effectively avoids handling a large number of candidate-structured outputs for complex structures. Second, in the proposed weak disambiguation strategy, each candidate label is assigned with a confidence value indicating how likely it is the true label, which aims to reduce the negative effects of wrong ground-truth label assignment in the learning process. Then, two large margins are formulated to combine two types of constraints which are the disambiguation between candidates and noncandidates, and the weak disambiguation for candidates. In the framework of alternating optimization, a new 2n -slack variables cutting plane algorithm is developed to accelerate each iteration of optimization. The experimental results on several sequence labeling tasks of natural language processing show the effectiveness of the proposed model.

Multilabel learning, which handles instances associated with multiple labels, has attracted much attention in recent years. Many extant multilabel feature selection methods target global feature selection, which means feature selection weights for each label are shared by all instances. Also, many extant multilabel classification methods exploit global label selection, which means labels correlations are shared by all instances. In real-world objects, however, different subsets of instances may share different feature selection weights and different label correlations. In this article, we propose a novel framework with local feature selection and local label correlation, where we assume instances can be clustered into different groups, and the feature selection weights and label correlations can only be shared by instances in the same group. The proposed framework includes a group-specific feature selection process and a label-specific group selection process. The former process projects instances into different groups by extracting the instance-group correlation. The latter process selects labels for each instance based on its related groups by extracting the group-label correlation. In addition, we also exploit the intergroup correlation. These three kinds of group-based correlations are combined to perform effective multilabel classification. The experimental results on various datasets validate the effectiveness of our approach.

Multilabel learning focuses on assigning instances with different labels. In essence, the multilabel learning aims at learning a predictive function from feature space to a label space. The predictive function learning procedure can be regarded as a feature selection procedure and as a classifier construction procedure. For feature selection, we extract features for each label based on the learned positive and negative feature-label correlations. The positive and negative relationships can illustrate which labels can and cannot be well presented by the corresponding features, respectively, due to the semantic gap. For classifier construction, we perform sample-specific and label-specific classifications. The interlabel and interinstance correlations are combined in these two kinds of classifications. These two correlations are learned from both input features and output labels when the output labels are too sparse to reveal the informative correlation. However, there exists the semantic gap when combining input and output spaces to mine the labelwise relationship. The semantic gap can be bridged by the learned feature-label correlation. Finally, extensive experimental results on several benchmarks under four domains are presented to show the effectiveness of the proposed framework.

Missing data widely exist in industrial processes and lead to difficulties in modeling, monitoring, fault diagnosis, and control. In this article, we propose a nonlinear method to handle the missing data problem in the offline modeling stage or/and the online monitoring stage of statistical process monitoring. We provide a fast incremental nonlinear matrix completion (FINLMC) method for missing data imputation, which enables us to use kernel methods such as kernel principal component analysis to monitor nonlinear multivariate processes even when there are missing data. We also provide theoretical analysis for the effectiveness of the proposed method. Experiments show that the proposed method can reduce the false alarm rate and improve the fault detection rate in nonlinear processing monitoring with missing data. The proposed FINLMC method can also be used to solve missing data in other problems such as classification and process control.

Recently, domain adaptation has received extensive attention for solving intelligent fault diagnosis problems. It aims to reduce the distribution discrepancy between the source domain and target domain through learning domain-invariant features. However, most existing domain adaptation methods mainly focus on global domain adaptation and overlook subdomain adaptation, which results in the loss of fine-grained information and discriminative features. To address this problem, in this article, a deep adversarial subdomain adaptation network is proposed. This network aligns the relevant distributions of subdomains by minimizing the local maximum mean discrepancy loss of the same categories in the source domain and target domain. Under the constraints of global domain adaptation and subdomain adaptation, the distribution discrepancy is reduced from the domain and category levels. Four transfer tasks under different machine rotating speeds and six transfer tasks on different but related machines were used to evaluate the effectiveness of the proposed method. The results demonstrated the robustness and superiority of the proposed method over five other methods.

Multilabel learning has been extensively studied in the past years, as it has many applications in different domains. It aims at annotating the labels for unseen data according to training data, which are often high dimensional in both instance and feature levels. The training data often have noisy and redundant information on these two levels. As an effective data preprocessing step, instance and feature selection should both be performed to find relevant training instances for each testing instance and relevant features for each label, respectively. However, most of the existing methods overlook the input-output correlation in each kind of selection. It will lead to the performance degradation. This article presents a formulation for multilabel learning from a topic view that exploits the dependence between features and labels in a topic space. We can perform effective instance and feature selection in the latent topic space, as the relationship between the input and output spaces is well captured in this space. The results from intensive experiments on various benchmarks demonstrate the effectiveness of the proposed framework.

Multilabel classification deals with instances assigned with multiple labels simultaneously. It focuses on learning a mapping from feature space to label a space for out-of-sample extrapolation. The mapping can be seen as a feature selection process in the feature domain or as a classifier training process in the classifier domain. The existing methods do not effectively learn the mapping when combining these two domains together. In this article, we derive a mechanism to extract label-specific features in local and global levels. We also derive a mechanism to train label-specific classifiers in individual and joint levels. Extracting features globally and training classifiers jointly can be seen as a dual process of learning the mapping function on two domains in a coarse-tuned way, while extracting features locally and training classifiers individually can be seen as a dual process of learning the mapping function on two domains in a fine-tuned way. The two-level feature selection and the two-level classifier training are derived to make the entire mapping learning process robust. Finally, extensive experimental results on several benchmarks under four domains are presented to demonstrate the effectiveness of the proposed approach.

Active failure prediction of hard disk drives (HDDs) is critical to prevent data loss and spare parts replacement decisions. Existing methods for failure predictions of HDDs always used a binary classifier to distinguish the healthy or failed HDDs and cannot address the problem of variable degradation states. In this article, an adaptive error tracking method is proposed for the HDD failure prediction. This method regards the extracted degradation feature as time serials and uses a state filter to estimate the real-time HDD's health status. Then, the HDD failure online prediction is achieved according to the alarm threshold determined by the adaptive error tracking. The degradation of an HDD is described by a first-order Markov hybrid jump degradation model, and the advanced Rao-Blackwellized particle filter algorithm, together with the expectation-maximization (EM) algorithm, is derived to estimate the model parameters adaptively. Finally, to verify the effectiveness of the proposed method, an accelerated degradation test (ADT) based on the vibration was carried out. And the data from ADT and real data center show that the proposed method performs much better than the previous methods, such as Kalman filter, SVM, MD, and recurrent neural network (RNN) based methods, with respect to failure prediction and the alarm distance, which helps to backup data and optimize maintenance decision costs for users.

In recent years, the estimation of the remaining useful life (RUL) has become an increasingly important topic. Existing RUL estimation studies mainly focus on linear degradation cases or degradation processes that can be linearized. A few nonlinear degradation models often rely on a training process based on a batch of samples obtained from the same population. Consequently, large bias or uncertainties may often occur under varying stress conditions. To address this problem, this article proposes a prognostic approach based on an adaptive particle filter (PF) to predict the RUL of the dynamic degradation systems using system degradation records. First, a nonlinear degradation model based on the fusion of an exponential item and a power law wear model were derived to capture the wear process under varying stress conditions. Second, the PF method was used to update the model parameters by treating the parameters as hidden state variables. Third, an adaptive strategy was derived based on the expectation-maximization algorithm and particle smoother algorithm to recursively update the hidden variables. Finally, an actual magnetic head wear dataset obtained from an actual manufacturing plant is used to verify the effectiveness of the proposed approach. The results reveal that the proposed approach significantly improves the prediction accuracy compared with the PF-based approach and extends Kalman-filter-based approach.

Existing methods for keyphrase extraction need preprocessing to generate candidate phrase or post-processing to transform keyword into keyphrase. In this paper, we propose a novel approach called duration modeling with semi-Markov Conditional Random Fields (DM-SMCRFs) for keyphrase extraction. First of all, based on the property of semi-Markov chain, DM-SMCRFs can encode segment-level features and sequentially classify the phrase in the sentence as keyphrase or non-keyphrase. Second, by assuming the independence between state transition and state duration, DM-SMCRFs model the distribution of duration (length) of keyphrases to further explore state duration information, which can help identify the size of keyphrase. Based on the convexity of parametric duration feature derived from duration distribution, a constrained Viterbi algorithm is derived to improve the performance of decoding in DM-SMCRFs. We thoroughly evaluate the performance of DM-SMCRFs on the datasets from various domains. The experimental results demonstrate the effectiveness of proposed model.

Classical matrix completion methods are not effective in recovering missing entries of data drawn from multiple subspaces because the matrices are often of high-rank. Recently a few advanced matrix completion methods were proposed to solve the problem but they are not scalable to large matrices and big data problems. This paper proposes a sparse factorization method for matrix completion on multiple-subspace data. The method factorizes the given incomplete matrix into a dense matrix and a sparse matrix, while the factorization errors of the observed entries are minimized. To solve the optimization problem, an accelerated proximal alternating linearized minimization (APALM) algorithm is proposed. As a non-trivial task owing to the alternation, linearization, nonconvexity, and extrapolation, the convergence of APALM is proved. APALM can solve a large class of optimization problems such as matrix factorization with nonsmooth regularizations. In addition, we show that, to recover an m × n matrix consisting of data drawn from k subspaces of dimension r0, the number of observed entries required in our matrix completion method is O(nr0logklog n) while that in conventional methods is O(nr0klog n), which theoretically proves the superiority of our method on multiple-subspace data and high-rank matrices. The proposed matrix completion method is compared with state-of-the-art on synthetic data and real collaborative filtering problems. The experimental results corroborate that the proposed method can handle large matrices efficiently and provide high recovery accuracy.

Recommender systems are currently utilized widely in e-commerce for product recommendations and within content delivery platforms. Previous studies usually use independent features to represent item content. As a result, the relationship hidden among the content features is overlooked. In fact, the reason that an item attracts a user may be attributed to only a few set of features. In addition, these features are often semantically coupled. In this paper, we present an optimization model for extracting the relationship hidden in content features by considering user preferences. The learned feature relationship matrix is then applied to address the cold-start recommendations and content-based recommendations. It could also easily be employed for the visualization of feature relation graphs. Our proposed method was examined on three public datasets: 1) hetrec-movielens-2k-v2; 2) book-crossing; and 3) Netflix. The experimental results demonstrated the effectiveness of our method in comparison to the state-of-the-art recommendation methods.

This paper presents a new framework for matching clothes by considering item in-between compatibility. In contrast to the use of visual features of clothing items, we only utilized their textual descriptions, i.e., title sentences, to constitute the basic features. Specifically, a longshort-term memory (LSTM) network was used for feature embeddings of title sentences. Given item pairs of queries and candidates, their feature embeddings achieved by Siamese LSTMs were integrated into style-compatible space characterized by a compatibility matrix. Our framework is examined on three large-scaled clothing item sets collected from Amazon, Taobao, and Polyvore, respectively. Experiments confirm the efficacy of our approach compared with several baseline methods.

Change detection has received extensive attention because of its realistic significance and broad application fields. However, none of the existing change detection algorithms can handle all scenarios and tasks so far. Different from the most of contributions from the research community in recent years, this paper does not work on designing new change detection algorithms. We, instead, solve the problem from another perspective by enhancing the raw detection results after change detection. As a result, the proposed method is applicable to various kinds of change detection methods, and regardless of how the results are detected. In this paper, we propose Fast Spatiotemporal Tree Filter (FSTF), a purely unsupervised detection method, to enhance coarse binary detection masks obtained by different kinds of change detection methods. In detail, the proposed FSTF has adopted a volumetric structure to effectively synthesize spatiotemporal information of the same target from the current time and history frames to enhance detection. The computational complexity analyzed in the view of graph theory also show that the fast realization of FSTF is a linear time algorithm, which is capable of handling efficient on-line detection tasks. Finally, comprehensive experiments based on qualitative and quantitative analysis verify that FSTF-based change detection enhancement is superior to several other state-of-the-art methods including fully connected Conditional Random Field (CRF), joint bilateral filter, and guided filter. It is illustrated that FSTF is versatile enough to also improve saliency detection as well as semantic image segmentation.

While the majority of methods for community detection produce disjoint communities of nodes, most real-world networks naturally involve overlapping communities. In this paper, a scalable method for the detection of overlapping communities in large networks is proposed. The method is based on an extension of the notion of normalized cut to cope with overlapping communities. A spectral clustering algorithm is formulated to solve the related cut minimization problem. When available, the algorithm may take into account prior information about the likelihood for each node to belong to several communities. This information can either be extracted from the available metadata or from node centrality measures. We also introduce a hierarchical version of the algorithm to automatically detect the number of communities. In addition, a new benchmark model extending the stochastic blockmodel for graphs with overlapping communities is formulated. Our experiments show that the proposed spectral method outperforms the state-of-the-art algorithms in terms of computational complexity and accuracy on our benchmark graph model and on five real-world networks, including a lexical network and large-scale social networks. The scalability of the proposed algorithm is also demonstrated on large synthetic graphs with millions of nodes and edges.

Robust principal component analysis (RPCA) can recover low-rank matrices when they are corrupted by sparse noises. In practice, many matrices are, however, of high rank and, hence, cannot be recovered by RPCA. We propose a novel method called robust kernel principal component analysis (RKPCA) to decompose a partially corrupted matrix as a sparse matrix plus a high- or full-rank matrix with low latent dimensionality. RKPCA can be applied to many problems such as noise removal and subspace clustering and is still the only unsupervised nonlinear method robust to sparse noises. Our theoretical analysis shows that, with high probability, RKPCA can provide high recovery accuracy. The optimization of RKPCA involves nonconvex and indifferentiable problems. We propose two nonconvex optimization algorithms for RKPCA. They are alternating direction method of multipliers with backtracking line search and proximal linearized minimization with adaptive step size (AdSS). Comparative studies in noise removal and robust subspace clustering corroborate the effectiveness and the superiority of RKPCA.

Hard disk drives (HDDs) are the core components of data center in IT companies. A breakdown of HDD may cause horrible data loss and great economic loss. Therefore, failure prediction for HDDs is significant to avoid loss and make a data backup plan in advance. Existing prediction methods always focus on a fixed threshold to distinguish whether a HDD is healthy or not, and these methods neglect the problem of multi-stage degradation phenomenon of HDDs. To solve these problems, this paper proposes a dynamic tracking method for HDD failure prediction based on a switchable state stochastic process model. By utilizing Rao-Blackwellized particle filter, the model estimates and parameters are updated by newly available data. To improve model ability, a sensitive health indicator is constructed from SMART attributes based on multiple regression analysis. Then, based on the statistical property of the tracking residuals, the dynamic failure threshold is designed to realize the online prediction of HDD failure. Furthermore, experiments of proposed method are carried out in a real-life data set. The results show the validity of the proposed method.

Supervised and semi-supervised sequence labeling methods require large amounts of fully annotated training sequences or exact annotations of structured outputs. The problem of learning from partially annotated sequences arises in many applications, for example, Natural Language Processing and Computational Biology. In this paper, we propose Piecewise Convex Learning from Partial Labels (PW-CLPL) which is an effective discriminative structured learning method for sequence labeling as global training is intractable for partially annotated sequences. A small number of constraints is reformulated for the optimization, which improve the efficiency of parameters learning. Experimental results on the reconstructed dataset CoNLL-2000 show the effectiveness of the proposed model in the setting of partial annotations.

Current Remaining useful life studies mainly focus on the linear degradation case. A few nonlinear degradation models often rely on parameters learning process from a batch of samples obtained from the same population. This result in large estimation biases and uncertainties in term of RUL prediction under varying stress conditions. To address this problem, this paper proposed an adaptive RUL prediction method only using the observed head wear data. First, an exponential item is incorporated into a power law wear model to build a nonlinear wear model under varying stress conditions. Second, PF method was used to update the model parameters dynamically. Third, an adaptive strategy was developed based on expectation-maximization algorithm to update these initial values of parameters of degradation model recursively. Finally, a real-life data set was used to verify the effectiveness of our proposed approach, and the results show that the proposed approach can improve the prediction accuracy significantly.

In multilabel learning (MLL), each instance can be assigned by several concepts simultaneously from a class dictionary. Usually, labels in the class dictionary have semantic correlations and semantic hierarchy. Instances can be categorized into different topics. Each topic has its own label candidates, and some topics have overlapped label candidates. In this paper, we propose a novel MLL method to deal with missing labels. The proposed algorithm can recover the label matrix according to local, topic-wise, and global semantic properties. Specifically, in the global level, label consistency, label-wise semantic correlations, and semantic hierarchy are exploited; in the local level, label importance and instance-wise semantic correlations in each topic are extracted; and in the topic level, label importance similarities and instance-wise semantic similarities between topics are mined. The experimental results on five image data sets in different applications demonstrate the effectiveness of the proposed approach.

The tree structure is one of the most powerful structures for data organization. An efficient learning framework for transforming tree-structured data into vectorial representations is presented. First, in attempting to uncover the global discriminative information of child nodes hidden at the same level of all of the trees, a clustering technique can be adopted for allocating children into different clusters, which are used to formulate the components of a vector. Moreover, a locality-sensitive reconstruction method is introduced to model a process, in which each parent node is assumed to be reconstructed by its children. The resulting reconstruction coefficients are reversely transformed into complementary coefficients, which are utilized for locally weighting the components of the vector. A new vector is formulated by concatenating the original parent node vector and the learned vector from its children. This new vector for each parent node is inputted into the learning process of formulating vectorial representation at the upper level of the tree. This recursive process concludes when a vectorial representation is achieved for the entire tree. Our method is examined in two applications: book author recommendations and content-based image retrieval. Extensive experimental results demonstrate the effectiveness of the proposed method for transforming tree-structured data into vectors.

The ever-growing amount of textual data available online creates the need for automatic text summarization tools. Probabilistic topic models are able to infer semantic relationships between sentences which is a key step of extractive summarization methods. However, they strongly rely on word co-occurrence patterns and fail to capture the actual semantic relationships between words such as synonymy, antonymy, etc. We propose a novel algorithm which incorporates pre-trained word embeddings in the probabilistic topic model in order to capture semantic similarities between sentences. These similarities provide the basis for a sentence ranking algorithm for query-oriented summarization. The summary is then produced by extracting highly ranked sentences from the original corpus. Our method is shown to outperform state-of-the-art algorithms on a benchmark dataset.

Social networks have become one of the most important research platforms in the big data era. Modelling social networks enables researchers and engineers to understand and analyze their intrinsic properties thereby implementing their real applications. A number of studies on social network modelling focus on a few characteristics, such as the number of edges (i.e., two-star motifs), scale-free degree distribution, and assortative property. This paper proposes an exponential triangle model for a typical social network, namely the Facebook network, established based on big data, and further analyzes its primary attributes of common interest on topological features. This new model has a power-law node-degree distribution with a flat top and an exponential cut-off tail, in remarkable agreement with one large-scale Facebook dataset. It can be used to predict future links of the Facebook network and help improve the friend-recommendation system. Furthermore, this work provides a useful graph-theoretic tool for Facebook network studies and enhances potential applications of social networks in general.

In this paper, we present new models and algorithms for object-level video advertising. A framework that aims to embed content-relevant ads within a video stream is investigated in this context. First, a comprehensive optimization model is designed to minimize intrusiveness to viewers when ads are inserted in a video. For human clothing advertising, we design a deep convolutional neural network using face features to recognize human genders in a video stream. Human parts alignment is then implemented to extract human part features that are used for clothing retrieval. Second, we develop a heuristic algorithm to solve the proposed optimization problem. For comparison, we also employ the genetic algorithm to find solutions approaching the global optimum. Our novel framework is examined in various types of videos. Experimental results demonstrate the effectiveness of the proposed method for object-level video advertising.

This paper focuses on an important and fundamental problem of connecting two points by a cable, subject to a tradeoff between cost and earthquake survivability. In particular, we address the problem of selecting a route for laying a cable under arbitrary topography, based on earthquake data. First, we derive a semi-supervised probability density estimation model for the likelihood of earthquake disaster. Based on this probabilistic model, we generate a nearest neighbor graph. The graph represents each data point with a four-dimensional space formed by the three-dimensional undersea coordinates and the one-dimensional data of earthquake disaster level. It then forms the weight on graph between any positions. The data used in this study are all real data of undersea topography and earthquake information of the Taiwan Strait. As a result, both the undersea topology and the earthquake level can be transferred into a distance for shortest route finding. Finally, Dijkstra's algorithm is used for finding the optimal shortest route for cabling between the two given points on the graph. Extensive simulations based on a synthetic dataset and the Taiwan Strait real-world dataset corroborate the effectiveness of the proposed method.

Logistics network optimization plays a critical role in contemporary logistics planning and supply chain network designs, and the vehicle route planning is essential for logistics network optimizations. In this paper, we present an efficient and effective approach for vehicle route planning. The new approach has utilized multiple spanning trees as a criterion to categorize the customers into several sub areas. In addition, by iterative choose some customers in the boundary of sub areas and resign them to different spanning trees, we can get a more compact ones with smaller distances. As a result, since the spanning tree represent the lower bound of vehicle route in all sub areas, we can also conduct the vehicular dispatching in each sub areas. Extensive simulation has verified the effectiveness of the proposed methods.

This paper formulates wireless sensor networks (WSNs) fault diagnosis problem as a pattern-classification problem and introduces a newly developed algorithm, neighborhood hidden conditional random field (NHCRF), for determining hidden states between sensors. The health conditions of WSN are determined by using the NHCRF model to estimate the posterior probability of different faulty scenarios. The NHCRF model can improve the WSN fault diagnosis, because it has relaxed the independence assumption of the hidden Markov model. To enhance the robustness and antinoise ability of the NHCRF, the concept of nearest neighbors is used when estimating dependencies. In this paper, a 200-sensor-node WSN is used to show that the proposed NHCRF method can deliver excellent and effective results for WSN-health diagnosis. Our study also presents thorough results on different types of WSN traffic, the free traffic, light traffic, and heavy traffic. Comparative results indicate that our method can deliver superior classification performance compared with other methods.

This paper introduces a new framework for the organization of electronic books (e-books) and their corresponding authors using a multilayer self-organizing map (MLSOM). An author is modeled by a rich tree-structured representation, and an MLSOM-based system is used as an efficient solution to the organizational problem of structured data. The tree-structured representation formulates author features in a hierarchy of author biography, books, pages, and paragraphs. To efficiently tackle the tree-structured representation, we used an MLSOM algorithm that serves as a clustering technique to handle e-books and their corresponding authors. A book and author recommender system is then implemented using the proposed framework. The effectiveness of our approach was examined in a large-scale data set containing 3868 authors along with the 10500 e-books that they wrote. We also provided visualization results of MLSOM for revealing the relevance patterns hidden from presented author clusters. The experimental results corroborate that the proposed method outperforms other content-based models (e.g., rate adapting poisson, latent Dirichlet allocation, probabilistic latent semantic indexing, and so on) and offers a promising solution to book recommendation, author recommendation, and visualization.

Wireless sensor networks are widely deployed in different industrial applications. Fault diagnosis of sensors is essential for maintaining a robust WSN operation. In this paper, we show faulty sensors diagnosis can be transformed into a pattern classification problem. We also introduce an efficient algorithm, called, Neighborhood Hidden Conditional Random Field, to recognize sensor states and the faulty scene of an WSN. Compared to conventional methods, the proposed diagnosis method incorporating hidden states can estimate the posterior probability of different faulty scenes. In addition, nearest neighbors are selected for estimating the dependencies among sensors, and the dependencies are subsequently used for diagnosing faulty scenes. Simulations based on an WSN are presented to show the effectiveness of the proposed methods. The results demonstrate that our proposed algorithm can achieve better classification performance compared with other state-of-art methods.

A new graph based constrained semi-supervised learning (G-CSSL) framework is proposed. Pairwise constraints (PC) are used to specify the types (intra- or inter-class) of points with labels. Since the number of labeled data is typically small in SSL setting, the core idea of this framework is to create and enrich the PC sets using the propagated soft labels from both labeled and unlabeled data by special label propagation (SLP), and hence obtaining more supervised information for delivering enhanced performance. We also propose a Two-stage Sparse Coding, termed TSC, for achieving adaptive neighborhood for SLP. The first stage aims at correcting the possible corruptions in data and training an informative dictionary, and the second stage focuses on sparse coding. To deliver enhanced inter-class separation and intra-class compactness, we also present a mixed soft-similarity measure to evaluate the similarity/dissimilarity of constrained pairs using the sparse codes and outputted probabilistic values by SLP. Simulations on the synthetic and real datasets demonstrated the validity of our algorithms for data representation and image recognition, compared with other related state-of-the-art graph based semi-supervised techniques.

Multilabel learning aims to predict labels of unseen instances by learning from training samples that are associated with a set of known labels. In this paper, we propose to use a hierarchical tree model for multilabel learning, and to develop the ML-Tree algorithm for finding the tree structure. ML-Tree considers a tree as a hierarchy of data and constructs the tree using the induction of one-against-all SVM classifiers at each node to recursively partition the data into child nodes. For each node, we define a predictive label vector to represent the predictive label transmission in the tree model for multilabel prediction and automatic discovery of the label relationships. If two labels co-occur frequently as predictive labels at leaf nodes, these labels are supposed to be relevant. The amount of predictive label co-occurrence provides an estimation of the label relationships. We examine the ML-Tree method on 11 real data sets of different domains and compare it with six well-established multilabel learning algorithms. The performances of these approaches are evaluated by 16 commonly used measures. We also conduct Friedman and Nemenyi tests to assess the statistical significance of the differences in performance. Experimental results demonstrate the effectiveness of our method.

Cluster analysis of unlabeled data sets has been recognized as a key research topic in varieties of fields. In many practical cases, no a priori knowledge is specified, for example, the number of clusters is unknown. In this paper, grid clustering based on the polar self-organizing map (PolSOM) is developed to automatically identify the optimal number of partitions. The data topology consisting of both the distance and density is exploited in the grid clustering. The proposed clustering method also provides a visual representation as PolSOM allows the characteristics of clusters to be presented as a 2-D polar map in terms of the data feature and value. Experimental studies on synthetic and real data sets demonstrate that the proposed algorithm provides higher clustering accuracy and lower computational cost compared with six conventional methods.

This paper discusses a new setting of graph based semi-supervised learning (SSL) guided using pairwise constraints (PCs). Technically, we propose a novel Graph based Constrained Semi-Supervised Learning (G-CSSL) framework. In this setting, PCs are used to specify the types (intra- or inter-class) of points with labels. Because the number of labeled data is typically small in SSL setting, the core idea of this framework is to create and enrich the PCs sets using the propagated soft labels from both labeled and unlabeled data via special label propagation (SLP), and hence obtaining more supervised information for delivering enhanced learning performance. To obtain the predicted labels of unlabeled data, we calculate the sparse codes of all data vectors jointly to assign weights for SLP. To deliver enhanced inter-class separation and intra-class compactness, we also present a mixed soft-similarity measure to evaluate the similarity/dissimilarity of constrained sample pairs by using the sparse codes and outputted probabilistic values by SLP. Extensive simulations demonstrated the effectiveness of our G-CSSL for image representation and recognition, compared with other related SSL techniques.

Dementia is one of the most common neurological disorders among the elderly. Identifying those who are of high risk suffering dementia is important for early diagnosis in order to slow down the disease progression and help preserve some cognitive functions of the brain. To achieve accurate classification, significant amount of subject feature information are involved. Hence identification of demented subjects can be transformed into a pattern classification problem. In this letter, we introduce a graph based semi-supervised learning algorithm for Medical Diagnosis by using partly labeled samples and large amount of unlabeled samples. The new method is derived by a compact graph that can well grasp the manifold structure of medical data. Simulation results show that the proposed method can achieve better sensitivities and specificities compared with other state-of-art graph based semi-supervised learning methods.

Normal and visually-impaired zebrafish larvae have differentiable light-induced locomotor response (LLR), which is composed of visual and non-visual components. It is recently demonstrated that differences in the acute phase of the LLR, also known as the visual motor response (VMR), can be utilized to evaluate new eye drugs. However, most of the previous studies focused on the average LLR activity of a particular genotype, which left information that could address differences in individual zebrafish development unattended. In this study, machine learning techniques were employed to distinguish not only zebrafish larvae of different genotypes, but also different batches, based on their response to light stimuli. This approach allows us to perform efficient high-throughput zebrafish screening with relatively simple preparations. Following the general machine learning framework, some discriminative features were first extracted from the behavioral data. Both unsupervised and supervised learning algorithms were implemented for the classification of zebrafish of different genotypes and batches. The accuracy of the classification in genotype was over 80 percent and could achieve up to 95 percent in some cases. The results obtained shed light on the potential of using machine learning techniques for analyzing behavioral data of zebrafish, which may enhance the reliability of high-throughput drug screening.

Predicting the impending failure of hard disk drives (HDDs) is crucial for preventing essential data from losing. In this paper, a two-step parametric method was developed to predict the impending failure of HDDs using the aggregate of statistical models. This method deals with the problem of failure prediction in two steps: anomaly detection and failure prediction. First, Mahalanobis distance was used for aggregating all the monitored variables into one index, which was then transformed into Gaussian variables by Box–Cox transformation. By defining an appropriate threshold, anomalies in HDDs were detected as a result. Second, a sliding-window-based generalized likelihood ratio test was proposed to track the anomaly progression in an HDD. When the occurrence of anomalies in a time interval is found to be statistically significant, indicating the HDD is approaching failure. In this work, we also derived a new cost function to adjust the prediction rate. This is important in a way to balance the failure detection rate and false alarm rate as well as to provide an advanced warning of HDD failures to the users, whereby the users can back up their data in time. Then the developed method was applied on a synthetic data set showing its effectiveness on predicting failures. To demonstrate the practical usefulness, this method was also applied on a real-life HDD data set. The result shows that our method could achieve 68% failure detection rate with 0% false alarm rate. This is much better than the results achieved by the state-of-the-art methods, such as support vector machine and hidden Markov models.

The problem of learning from both labeled and unlabeled data is considered. In this paper, we present a novel semisupervised multimodal dimensionality reduction (SSMDR) algorithm for feature reduction and extraction. SSMDR can preserve the local and multimodal structures of labeled and unlabeled samples. As a result, data pairs in the close vicinity of the original space are projected in the nearby of the embedding space. Due to overfitting, supervised dimensionality reduction methods tend to perform inefficiently when only few labeled samples are available. In such cases, unlabeled samples play a significant role in boosting the learning performance. The proposed discriminant technique has an analytical form of the embedding transformations that can be effectively obtained by applying the eigen decomposition, or finding two close optimal sets of transforming basis vectors. By employing the standard kernel trick, SSMDR can be extended to the nonlinear dimensionality reduction scenarios. We verify the feasibility and effectiveness of SSMDR through conducting extensive simulations including data visualization and classification on the synthetic and real‐world datasets. Our obtained results reveal that SSMDR offers significant advantages over some widely used techniques. Compared with other methods, the proposed SSMDR exhibits superior performance on multimodal cases.

Dementia is one of the most common neurological disorders among the elderly. Identifying those who are of high risk suffering dementia is important to the administration of early treatment in order to slow down the progression of dementia symptoms. However, to achieve accurate classification, significant amount of subject feature information are involved. Hence identification of demented subjects can be transformed into a pattern recognition problem with high-dimensional nonlinear datasets. In this paper, we introduce trace ratio linear discriminant analysis (TR-LDA) for dementia diagnosis. An improved ITR algorithm (iITR) is developed to solve the TR-LDA problem. This novel method can be integrated with advanced missing value imputation method and utilized for the analysis of the nonlinear datasets in many real-world medical diagnosis problems. Finally, extensive simulations are conducted to show the effectiveness of the proposed method. The results demonstrate that our method can achieve higher accuracies for identifying the demented patients than other state-of-art algorithms.

Isomap is a well-known nonlinear dimensionality reduction (DR) method, aiming at preserving geodesic distances of all similarity pairs for delivering highly nonlinear manifolds. Isomap is efficient in visualizing synthetic data sets, but it usually delivers unsatisfactory results in benchmark cases. This paper incorporates the pairwise constraints into Isomap and proposes a marginal Isomap (M-Isomap) for manifold learning. The pairwise Cannot-Link and Must-Link constraints are used to specify the types of neighborhoods. M-Isomap computes the shortest path distances over constrained neighborhood graphs and guides the nonlinear DR through separating the interclass neighbors. As a result, large margins between both interand intraclass clusters are delivered and enhanced compactness of intracluster points is achieved at the same time. The validity of M-Isomap is examined by extensive simulations over synthetic, University of California, Irvine, and benchmark real Olivetti Research Library, YALE, and CMU Pose, Illumination, and Expression databases. The data visualization and clustering power of M-Isomap are compared with those of six related DR methods. The visualization results show that M-Isomap is able to deliver more separate clusters. Clustering evaluations also demonstrate that M-Isomap delivers comparable or even better results than some state-of-the-art DR algorithms.

Visualizing similarity data of different objects by exhibiting more separate organizations with local and multimodal characteristics preserved is important in multivariate data analysis. Laplacian Eigenmaps (LAE) and Locally Linear Embedding (LLE) aim at preserving the embeddings of all similarity pairs in the close vicinity of the reduced output space, but they are unable to identify and separate interclass neighbors. This paper considers the semi-supervised manifold learning problems. We apply the pairwise Cannot-Link and Must-Link constraints induced by the neighborhood graph to specify the types of neighboring pairs. More flexible regulation on supervised information is provided. Two novel multimodal nonlinear techniques, which we call trace ratio (TR) criterion-based semi-supervised LAE (S2LAE) and LLE (S2LLE), are then proposed for marginal manifold visualization. We also present the kernelized S2LAE and S2LLE. We verify the feasibility of S2LAE and S2LLE through extensive simulations over benchmark real-world MIT CBCL, CMU PIE, MNIST, and USPS data sets. Manifold visualizations show that S2LAE and S2LLE are able to deliver large margins between different clusters or classes with multimodal distributions preserved. Clustering evaluations show they can achieve comparable to or even better results than some widely used methods.

This paper incorporates the group sparse representation into the well-known canonical correlation analysis (CCA) framework and proposes a novel discriminant feature extraction technique named group sparse canonical correlation analysis (GSCCA). GSCCA uses two sets of variables and aims at preserving the group sparse (GS) characteristics of data within each set in addition to maximize the global interset covariance. With GS weights computed prior to feature extraction, the locality, sparsity and discriminant information of data can be adaptively determined. The GS weights are obtained from an NP-hard group-sparsity promoting problem that considers all highly correlated data within a group. By defining one of the two variable sets as the class label matrix, GSCCA is effectively extended to multiclass scenarios. Then GSCCA is theoretically formulated as a least-squares problem as CCA does. Comparative analysis between this work and the related studies demonstrate that our algorithm is more general exhibiting attractive properties. The projection matrix of GSCCA is analytically solved by applying eigen-decomposition and trace ratio (TR) optimization. Extensive benchmark simulations are conducted to examine GSCCA. Results show that our approach delivers promising results, compared with other related algorithms.

The cooperation behaviors existing in the animal and human being societies, have been modeled for the numerical optimization, but the local cooperation has not been modeled separately in optimization problems. In this paper the local cooperation is newly modeled as Neighborhood Field Model (NFM). Based on NFM, a new optimization technique called Neighborhood Field Optimization algorithm (NFO) is firstly proposed to deliver global optimization. In NFO, each individual is attracted by its superior neighbor and repulsed by its inferior neighbor to search a better solution. In this paper, NFO is compared with certain algorithms under twelve different benchmark functions. The results show that NFO can outperform them on multimodal functions in the respect of accuracy, effectiveness and robustness. It also can be noted that the cooperation behavior can play a dominant role in the optimization algorithm separately.

Semi-supervised dimensionality reduction is an important research topic in many pattern recognition and machine learning applications. Among all the methods for semi-supervised dimensionality reduction, SDA and Lap-LDA are two popular ones. Both SDA and Lap-LDA can perform dimensionality reduction by preserving the discriminative structure embedding in the labeled samples as well as the manifold structure embedded both in labeled and unlabeled samples. But they apply different schemes for semi-supervised dimensionality reduction. SDA has added the manifold term to the objective function of LDA while Lap-LDA has added such term to the objective function of Least Square with certain class indicator. In this paper, we further analyze the schemes of two methods and build the equivalence between them by giving a certain condition. We then show their difference when the certain condition cannot be satisfied. Extensive simulations have been conducted based several datasets. Both theoretical analysis and simulation results confirm the analysis. Finally, motivated by the equivalence and differences between two methods, we then propose an improved approach for semi-supervised dimensionality reduction. The proposed approach is actually a two-stage approach and can obtain the optimal solution equivalent to Lap-LDA (in this first stage) and SDA (in the second stage) with less computational cost.

This paper discusses the semi-supervised marginal projection problems learning from partial constrained data. Two effective multimodal dimensionality reduction (DR) algorithms, which we call semi-supervised marginal projections (SSMP) and orthogonal SSMP (OSSMP), are proposed. By specifying the types of similarity pairs with the pairwise constraints (PC), our techniques can preserve the global structures of all points as well as local geometrical and discriminant structures embedded in the PC. SSMP in singular case is also discussed. Because in all the PC guided methods, extracting the informative constraints is difficult and random constraints greatly affect the learning performance of techniques, this work also presents an effective and efficient methodology of optimally selecting the informative constraints for learning. The analytic form of the marginal projections can be effectively obtained by eigen-decomposition. The connections between this present work and the related semi-supervised algorithms are also detailed. The effectiveness of our proposed informative constraint selection method and algorithms are evaluated by benchmark problems. Results show our methods are capable of delivering competitive results with some widely used state-of-the-art semi-supervised algorithms.

Linear Discriminant Analysis (LDA) is a well-known dimensionality reduction algorithm for pattern recognition and machine learning. And Trace Ratio LDA (TR-LDA) and Null-space LDA (NLDA) are two popular variants of LDA. Both NLDA and TR-LDA can result in orthogonal transformations. However, they applied different schemes in deriving the optimal transformation. NLDA computes an orthogonal transformation in the null space of the within-class scatter matrix, while TRLDA computes an orthogonal transformation by an iterative procedure. In this paper, by using the trace difference problem as a bridge, we show that the above two algorithms can be equivalent when confronts with singularity problem. In addition, extensive simulations were conducted based on several datasets. Both theoretical analysis and simulation results confirm the equivalent relationship.

This paper presents a model for optimal asset maintenance inspection services. The model is designed to support through-life service in the form of multiple nested inspections and maintenance to meet defined asset availability and capability requirements, as well as achieving successful through-life technology insertions. The inspections and maintenance activities are assumed to be performed at more than one level, but nested and aimed at different types of defects or subsystems over a fixed period of time (the designed asset life). This practice is common in many industries, particularly in the defense industry. The impact of technological insertions is reflected through changes in the failure behavior of the asset. We use the delay time concept to model the failure mechanism of the asset, and the arrivals of defects are assumed to follow Poisson processes. The decision variables are the inspection intervals, while the objective function can be expressed in terms of cost, downtime, or reliability. The model is demonstrated through a numerical example. The model can be used for optimizing two-level inspection intervals with technological insertions.

Dimensionality reduction has been a fundamental tool when dealing with high-dimensional dataset. And trace ration optimization has been widely used in dimensionality reduction because Trace ratio can directly reflect the similarity (Euclidean distance) of data points. Conventionally, there is no close-form solution to the original trace ratio problem. Prior works have indicated that trace ratio problem can be solved by an iterative way. In this paper, we propose an efficient algorithm to find the optimal solutions. The proposed algorithm can be easily extended to its corresponding kernel version for handling the nonlinear problems. Finally, we evaluate our proposed algorithm based on extensive simulations of real world datasets. The results show our proposed method is able to deliver marked improvements over other supervised and unsupervised algorithms.

This letter presents a Tensor Locally Linear Discriminative Analysis (TLLDA) method for image presentation. TLLDA is originated from the Local Fisher Discriminant Analysis (LFDA), but TLLDA offers some advantages over LFDA. 1) TLLDA can preserve the local discriminative information of image data as LFDA. 2) TLLDA represents images as matrices or 2-order tensors rather than vectors, so TLLDA keeps the spatial locality of pixels in the images. 3) TLLDA avoids the singularity that may be suffered by LFDA. 4) TLLDA is faster than LFDA. Simulations on two real databases verified the validity of TLLDA. Results show that TLLDA is highly competitive with some widely used techniques.

A novel framework using a Bayesian approach for content-based phishing web page detection is presented. Our model takes into account textual and visual contents to measure the similarity between the protected web page and suspicious web pages. A text classifier, an image classifier, and an algorithm fusing the results from classifiers are introduced. An outstanding feature of this paper is the exploration of a Bayesian model to estimate the matching threshold. This is required in the classifier for determining the class of the web page and identifying whether the web page is phishing or not. In the text classifier, the naive Bayes rule is used to calculate the probability that a web page is phishing. In the image classifier, the earth mover's distance is employed to measure the visual similarity, and our Bayesian model is designed to determine the threshold. In the data fusion algorithm, the Bayes theory is used to synthesize the classification results from textual and visual content. The effectiveness of our proposed approach was examined in a large-scale dataset collected from real phishing cases. Experimental results demonstrated that the text classifier and the image classifier we designed deliver promising results, the fusion algorithm outperforms either of the individual classifiers, and our model can be adapted to different phishing cases.

Conventional shape from shading (SFS) algorithms are unable to deal with multi-color image satisfactory. This is because the assumption of constant surface albedo in the algorithms is not applicable to multi-color images. This paper proposes a new SFS approach for multi-color images through a segmentation-based shading recovery technique. With this technique a gray image is firstly extracted from the multi-color image containing better shading information compared with other color-to-gray conversion methods. The shading is recovered in the gray image as if the objects were made of single color. Shape of the multi-color object can then be recovered by classical gray-scaled SFS methods. Experimental results with synthetic and real multi-color images are presented. The obtained results corroborate that the proposed scheme is able to deliver better performance compared with other color SFS methods.

This paper presents a novel optimization algorithm called self-organizing potential field network (SOPFN). The SOPFN algorithm is derived from the idea of the vector potential field. In the proposed network, the neuron with the best weight is considered as the target with the attractive force, while the neuron with the worst weight is considered as the obstacle with the repulsive force. The competitive and cooperative behaviors of SOPFN provide a remarkable ability to escape from the local optimum. Simulations were performed, compared, and analyzed on eight benchmark functions. The results presented illustrate that the SOPFN algorithm achieves a significant performance improvement on multimodal problems compared with other evolutionary optimization algorithms.

This paper proposes a new document retrieval (DR) and plagiarism detection (PD) system using multilayer self-organizing map (MLSOM). A document is modeled by a rich tree-structured representation, and a SOM-based system is used as a computationally effective solution. Instead of relying on keywords/lines, the proposed scheme compares a full document as a query for performing retrieval and PD. The tree-structured representation hierarchically includes document features as document, pages, and paragraphs. Thus, it can reflect underlying context that is difficult to acquire from the currently used word-frequency information. We show that the tree-structured data is effective for DR and PD. To handle tree-structured representation in an efficient way, we use an MLSOM algorithm, which was previously developed by the authors for the application of image retrieval. In this study, it serves as an effective clustering algorithm. Using the MLSOM, local matching techniques are developed for comparing text documents. Two novel MLSOM-based PD methods are proposed. Detailed simulations are conducted and the experimental results corroborate that the proposed approach is computationally efficient and accurate for DR and PD.

A new efficient unsupervised feature selection method is proposed to handle nominal data without data transformation. The proposed feature selection method introduces a new data distribution factor to select appropriate clusters. The proposed method combines the compactness and separation together with a newly introduced concept of singleton item. This new feature selection method considers all features globally. It is computationally inexpensive and able to deliver very promising results. Eight datasets from the University of California Irvine (UCI) machine learning repository and a high-dimensional cDNA dataset are used in this paper. The obtained results show that the proposed method is very efficient and able to deliver very reliable results.

MOTIVATION: Most gene-expression based studies aim to identify genes with the capability of distinguishing different phenotypes. Although analysis at the genomic level is important, results of the molecular/cellular level are essential for understanding biological mechanisms. To deliver molecular/cellular-level results, a two-stage scheme is widely employed. This scheme just evaluates biological processes/molecular activities individually, totally overlooking the relationship between processes/activities. This treatment conflicts with the fact that most biological processes/molecular activities do not work alone. In order to deliver improved results, this shortcoming should be addressed. RESULTS: We design a selection model from a novel perspective to directly detect important gene functional categories (each category represents a cellular process or a molecular activity). More importantly, the correlations between gene categories are considered. Contributed by this capability, the proposed method shows its advantages over others. AVAILABILITY: the source code in Matlab is accessible via http://www.ee.cityu.edu.hk/~twschow/category_selection/category_selection.htm

Microarray gene expression data usually consist of a large amount of genes. Among these genes, only a small fraction is informative for performing cancer diagnostic test. This paper focuses on effective identification of informative genes. We analyze gene selection models from the perspective of optimization theory. As a result, a new strategy is designed to modify conventional search engines. Also, as overfitting is likely to occur in microarray data because of their small sample set, a point injection technique is developed to address the problem of overfitting. The proposed strategies have been evaluated on three kinds of cancer diagnosis. Our results show that the proposed strategies can improve the performance of gene selection substantially. The experimental results also indicate that the proposed methods are very robust under all the investigated cases.

A self-organizing and self-evolving agents (SOSENs) neural network is proposed. Each neuron of the SOSENs evolves itself with a simulated annealing (SA) algorithm. The self-evolving behavior of each neuron is a local improvement that results in speeding up the convergence. The chance of reaching the global optimum is increased because multiple SAs are run in a searching space. Optimum results obtained by the SOSENs are better in average than those obtained by a single SA. Experimental results show that the SOSENs have less temperature changes than the SA to reach the global minimum. Every neuron exhibits a self-organizing behavior, which is similar to those of the self-organizing map (SOM), particle swarm optimization (PSO), and self-organizing migrating algorithm (SOMA). At last, the computational time of parallel SOSENs can be less than the SA.

Data reduction algorithms determine a small data subset from a given large data set. In this article, new types of data reduction criteria, based on the concept of entropy, are first presented. These criteria can evaluate the data reduction performance in a sophisticated and comprehensive way. As a result, new data reduction procedures are developed. Using the newly introduced criteria, the proposed data reduction scheme is shown to be efficient and effective. In addition, an outlier-filtering strategy, which is computationally insignificant, is developed. In some instances, this strategy can substantially improve the performance of supervised data analysis. The proposed procedures are compared with related techniques in two types of application: density estimation and classification. Extensive comparative results are included to corroborate the contributions of the proposed algorithms.

A novel feature selection method using the concept of mutual information (MI) is proposed in this paper. In all MI based feature selection methods, effective and efficient estimation of high-dimensional MI is crucial. In this paper, a pruned Parzen window estimator and the quadratic mutual information (QMI) are combined to address this problem. The results show that the proposed approach can estimate the MI in an effective and efficient way. With this contribution, a novel feature selection method is developed to identify the salient features one by one. Also, the appropriate feature subsets for classification can be reliably estimated. The proposed methodology is thoroughly tested in four different classification applications in which the number of features ranged from less than 10 to over 15,000. The presented results are very promising and corroborate the contribution of the proposed feature selection methodology.

Self-organizing map (SOM) is an approach of nonlinear dimension reduction and can be used for visualization. It only preserves topological structures of input data on the projected output space. The interneuron distances of SOM are not preserved from input space into output space such that the visualization of SOM can be degraded. Visualization-induced SOM (ViSOM) has been proposed to overcome this problem. However, ViSOM is derived from heuristic and no cost function is assigned to it. In this paper, a probabilistic regularized SOM (PRSOM) is proposed to give a better visualization effect. It is associated with a cost function and gives a principled rule for weight-updating. The advantages of both multidimensional scaling (MDS) and SOM are incorporated in PRSOM. Like MDS, The interneuron distances of PRSOM in input space resemble those in output space, which are predefined before training. Instead of the hard assignment by ViSOM, the soft assignment by PRSOM can be further utilized to enhance the visualization effect. Experimental results demonstrate the effectiveness of the proposed PRSOM method compared with other dimension reduction methods.

A radial-basis-function (RBF) neural network based fault detection system is developed for performing induction machine fault detection and analysis. The optimal network architecture of the RBF network is determined automatically by our proposed cell-splitting, grid (CSG) algorithm. This facilitates the conventional laborious trial-and-error procedure in establishing an optimal architecture. The proposed RBF machine fault diagnostic system has been intensively tested with unbalanced electrical faults and mechanical faults operating at different rotating speeds. The proposed system is not only able to detect electrical and mechanical faults, but the system is also able to estimate the extent of faults.

In this paper, a new algorithm of support vector visualization and clustering (SVVC) based on self-organizing map (SOM) and support vector one-class classification (SVOCC) is presented. Original SVOCC is to identify the support domain of input data. When it is used for clustering, the high computational complexity for identifying cluster gaps between any pair points makes it less likely to be used in large data sets. In addition, the identified clusters cannot be visually displayed in high dimensions larger than three. Self-organizing map (SOM) is a neural network approach, which can project high-dimensional data into usually 2-D grid while preserving topology of input data. By using the proposed SVVC algorithm, resulting map can visually display high-dimensional cluster shapes and corresponding clusters can be found. Outliers and cluster borders can be clearly identified on the map, which is better than other visualization and clustering methods on SOM. The computational complexity of SVVC is less than the method of directly clustering by SVOCC.

In this article, a new methodology for color shape from shading (SFS) problem is proposed. The problem of color SFS refers to the well-known fact that most real objects usually contain mixtures of diffuse and specular color reflections and are affected by the multicolored interreflection under unknown reflectivity. In this article, these limitations are addressed, and a new color SFS methodology is proposed. The proposed approach focuses on two main parts. First, a generalized neural-based color reflectance model is developed. Second, an iterative recursive method is developed to reconstruct a multicolor 3D surface. Experimental results on synthetic-colored objects and real-colored objects were performed to demonstrate the performance of the proposed methodology.

It is known that most real surfaces usually are neither perfectly Lambertian model nor ideally specular model; rather, they are formed by the hybrid structure of these two models. This hybrid reflectance model still suffers from the noise, strong specular, and unknown reflectivity conditions. In this article, these limitations are addressed, and a new neural-based hybrid reflectance model is proposed. The goal of this method is to optimize a proper reflectance model by learning the weight and parameters of the hybrid structure of feedforward neural networks and radial basis function networks and to recover the 3D object shape by the shape from shading technique with this resulting model. Experimental results, including synthetic and real images, were performed to demonstrate the performance of the proposed reflectance model in the case of different specular effects and noise environments.

The shape from shading problem refers to the well-known fact that most real images usually contain specular components and are affected by unknown reflectivity. In this paper, these limitations are addressed and a new neural-based 3D shape reconstruction model is proposed. The idea behind this approach is to optimize a proper reflectance model by learning the parameters of the proposed neural reflectance model. In order to do this, new neural-based reflectance models are presented. The feedforward neural network (FNN) model is able to generalize the diffuse term, while the RBF model is able to generalize the specular term. A hybrid structure of FNN-based and RBF-based models is also presented because most real surfaces are usually neither Lambertian models nor ideally specular models. Experimental results, including synthetic and real images, are presented to demonstrate the performance of our approach given different specular effects, unknown illuminate conditions, and different noise environments.

This letter aims at determining the optimal bias and magnitude of initial weight vectors based on multidimensional geometry. This method ensures the outputs of neurons are in the active region and the range of the activation function is fully utilized. In this letter, very thorough simulations and comparative study were performed to validate the performance of the proposed method. The obtained results on five well-known benchmark problems demonstrate that the proposed method deliver consistent good results compared with other weight initialization methods.

For the shape from shading problem, it is known that most real images usually contain specular components and are affected by unknown reflectivity. In this paper, these limitations are addressed and a new neural-based specular reflectance model is proposed. The idea of this method is to optimize a proper specular model by learning the parameters of a radial basis function network and to recover the object shape by the variational approach with this resulting model. The obtained results are very encouraging and the performance is demonstrated by using the synthetic and real images in the case of different specular effects and noisy environments.

In this paper, we present a neural-based reflectance model of which the physical parameters of the reflectivity under different lighting conditions are interpreted by the network weights. The idea of our method is to optimize a proper reflectance model by an effective learning algorithm and to recover the object surface by a simple shape from shading recursive algorithm with this resulting model. Experimental results, including synthetic and real images, were performed to demonstrate the performance of the proposed method for practical applications.

In this paper, a new neural approach is introduced for the problem of blind equalization in digital communications. Necessary and sufficient conditions for blind equalization are proposed, which can be implemented by a two-layer linear neural network. In the hidden layer, the received signals are whitened, while the network outputs provide directly an estimation of the source symbols. We consider a stochastic approximate learning algorithm for each layer according to the property of the correlation matrices of the transmitted symbols. The proposed class of networks yield good results in simulation examples for the blind equalization of a three-ray multipath channel.

A neural-based crowd estimation system for surveillance in complex scenes at underground station platform is presented. Estimation is carried out by extracting a set of significant features from sequences of images. Those feature indexes are modeled by a neural network to estimate the crowd density. The learning phase is based on our proposed hybrid of the least-squares and global search algorithms which are capable of providing the global search characteristic and fast convergence speed. Promising experimental results are obtained in terms of accuracy and real-time response capability to alert operators automatically.

This paper develops a new sliding mode neural network control scheme for a class of nonlinear discrete systems, which can eliminate a chattering effect. The control system is designed on the basis of the discrete Lyapunov theory which assures the system reaches sliding mode manifolds. The equivalent control is used directly as the control input after reaching sliding mod manifolds. A part of equivalent control is estimated by an online estimator which is realized by a recurrent neural network (RNN). The real-time recurrent learning algorithm is improved and used to train the RNN. Due to its real-time learning ability, the stability of the control system are guaranteed. The proposed control scheme eliminates chattering and provides sliding mode motion on the selected manifolds in the state space. The detailed control procedure is given and numerical examples are used to validate the proposed control scheme.

This paper includes some comments and amendments of the above-mentioned paper by Igelnik et al. (1995). Subsequently, Theorem 1 in the above-mentioned paper has been revised. The significant change of the original theorem is the space of the thresholds in the hidden layer. The revised theorem says that the thresholds of hidden b/sub 0/, should be -w/sub 0//spl middot/y/sub 0/-u/sub 0/, where w/sub 0/=/spl alpha/w/spl circ//sub 0/; w/spl circ//sub 0/=(w/spl circ//sub 01/, /spl middot//spl middot//spl middot/, y/sub 0d/), and u/sub 0/ be independent and uniformly distributed in V/sup d/=[0; /spl Omega/]/spl times/[-/spl Omega/; /spl Omega/]/sup d-1/, I/sup d/, and [-2d/spl Omega/, 2d/spl Omega/], respectively. In reply, Igelnik et al. acknowledge that a factor of two was omitted in the statement of a trigonometric identity. However, the validity of the essential point of Theorem 1 is unaltered.

An extended least squares-based algorithm for feedforward networks is proposed. The weights connecting the last hidden and output layers are first evaluated by least squares algorithm. The weights between input and hidden layers are then evaluated using the modified gradient descent algorithms. This arrangement eliminates the stalling problem experienced by the pure least squares type algorithms; however, still maintains the characteristic of fast convergence. In the investigated problems, the total number of FLOPS required for the networks to converge using the proposed training algorithm are only 0.221%-16.0% of that using the Levenberg-Marquardt algorithm. The number of floating point operations per iteration of the proposed algorithm are only 1.517-3.521 times of that of the standard backpropagation algorithm.

Describes a nonlinear preprocessing method to enhance the output performance of a network. The introduction of the nonlinear preprocessing method redistributes the distributions of input and output vectors, and makes the input and output variables more "orthogonal" that results in facilitating the network optimization. In some of the examples, this nonlinear preprocessing technique enables test set error to be reduced by a magnitude of 98%. Three applications of time-series predictions applied to evaluate the performance of the proposed method are presented.