Most recent research on multiagent reinforcement learning (MARL) has explored how to deploy cooperative policies for homogeneous agents. However, realistic multiagent environments may contain heterogeneous agents that have different attributes or tasks. The heterogeneity of the agents and the diversity of relationships cause the learning of policy excessively tough. To tackle this difficulty, we present a novel method that employs a heterogeneous graph attention network to model the relationships between heterogeneous agents. The proposed method can generate an integrated feature representation for each agent by hierarchically aggregating latent feature information of neighbor agents, with the importance of the agent level and the relationship level being entirely considered. The method is agnostic to specific MARL methods and can be flexibly integrated with diverse value decomposition methods. We conduct experiments in predator-prey and StarCraft Multiagent Challenge (SMAC) environments, and the empirical results demonstrate that the performance of our method is superior to existing methods in several heterogeneous scenarios.

Most existing image captioning methods use only the visual information of the image to guide the generation of captions, lack the guidance of effective scene semantic information, and the current visual attention mechanism cannot adjust the focus intensity on the image. In this article, we first propose an improved visual attention model. At each timestep, we calculated the focus intensity coefficient of the attention mechanism through the context information of the model, then automatically adjusted the focus intensity of the attention mechanism through the coefficient to extract more accurate visual information. In addition, we represented the scene semantic knowledge of the image through topic words related to the image scene, then added them to the language model. We used the attention mechanism to determine the visual information and scene semantic information that the model pays attention to at each timestep and combined them to enable the model to generate more accurate and scene-specific captions. Finally, we evaluated our model on Microsoft COCO (MSCOCO) and Flickr30k standard datasets. The experimental results show that our approach generates more accurate captions and outperforms many recent advanced models in various evaluation metrics.

Recommending valuable contents for microblog users is an important way to improve users' experiences. As high quality descriptors of user semantics, tags have always been used to represent users' interests or attributes. In this work, we propose a microblog recommendation approach via hypergraph random walk tag expansion and user social relation. More specifically, microblogs are considered as hyperedges and terms are taken as hypervertexs for each user, and the weighting strategies for both hyperedges and hypervertexs are established. Random walk is performed on the weighted hypergraph to obtain a number of terms as tags for users. And then the tag similarity matrix and the user-tag matrix can be constructed based on tag probability correlations and weight of each tag. Besides, the significance of user social relation is also considered for recommendation. Moreover, an iterative updating scheme is developed to get the final user-tag matrix for computing the similarities between microblogs and users. Experimental results show that the algorithm is effective in microblog recommendation.

Supervised dimensionality reduction has shown great advantages in finding predictive subspaces. Previous methods rarely consider the popular maximum margin principle and are prone to overfitting to usually small training data, especially for those under the maximum likelihood framework. In this paper, we present a posterior-regularized Bayesian approach to combine Principal Component Analysis (PCA) with the max-margin learning. Based on the data augmentation idea for max-margin learning and the probabilistic interpretation of PCA, our method can automatically infer the weight and penalty parameter of max-margin learning machine, while finding the most appropriate PCA subspace simultaneously under the Bayesian framework. We develop a fast mean-field variational inference algorithm to approximate the posterior. Experimental results on various classification tasks show that our method outperforms a number of competitors.

ABGP is a special cognitive model, which consists of awareness, beliefs, goals and plans. As most agent architectures, ABGP agents obtain knowledge from the natural scenes only through single preestablished rules as well, don't directly capture the natural scenes information like human visual. Inspired by the biological visual cortex (V1) and the higher brain areas perceiving visual features, we propose a novel deep network model convolutional generative stochastic model (CGSM) used to visual feature representation, and firstly introduce it into the awareness module of the cognitive model ABGP to construct a state-of-the-art cognitive model ABGP-CGSM. For the novel cognitive model ABGP-CGSM, we construct a rat-robot maze search simulation platform to show the validity recognizing natural scenes. According to the simulation results on the noise and noiseless natural scenes, the rat-robot implemented by ABGP-CGSM has an excellent success rate when passing through the maze. The simulation shows that the ABGP-CGSM model proposed in our work can directly enhance the capability of communication between agent and natural scenes, improve the ability to cognize the real world as human being and conduct the agent to plan independently its path in terms of the visual information from the natural scenes.

Summary Bilevel decision addresses compromises between two interacting decision entities within a given hierarchical complex system under distributed environments. Bilevel programming typically solves bilevel decision problems. However, formulation of objectives and constraints in mathematical functions is required, which are difficult, and sometimes impossible, in real‐world situations because of various uncertainties. Our study develops a rule‐set based bilevel decision approach, which models a bilevel decision problem by creating, transforming and reducing related rule sets. This study develops a new rule‐sets based solution algorithm to obtain an optimal solution from the bilevel decision problem described by rule sets. A case study and a set of experiments illustrate both functions and the effectiveness of the developed algorithm in solving a bilevel decision problem. Copyright © 2012 John Wiley & Sons, Ltd.

Most existing rat able aspect generating methods for aspect mining focus on identifying and rating aspects of reviews with overall ratings, while huge amount of unrated reviews are beyond their ability. This drawback motivates the research problem in this paper: predicting aspect ratings and overall ratings for unrated reviews. To solve this problem, we novelly propose a topic model based on Latent Dirichlet Allocation with indirect supervision. Compared with the previous bag-of-words representation of review documents, we utilize the quad-tuples of (head, modifier, rating, entity) to explicitly model the associations between modifiers and ratings. Specifically, our solution for aspect mining in unrated reviews is decomposed into three steps. Firstly, rat able aspects are generated over sentiments from training reviews with overall ratings. Afterwards, inference of aspect identification and rating for unrated reviews are provided. Finally, overall ratings are predicted for unrated reviews. Under this framework, aspect and sentiment associations are captured in the form of joint probabilities through a generative process. The effectiveness of our approach is testified on a real-world dataset crawled from Trip Advisor http://www.tripadvisor.com/, and extensive experiments show that our method significantly outperforms state-of-the-art methods.

Memory is a fundamental component in human brain and plays very important roles for all mental processes. The analysis of memory systems through cognitive architectures can be performed at the computational, or functional level, on the basis of empirical data. In this paper we discuss memory systems in the extended Consciousness and Memory Model (CAM) The knowledge representations used in CAM for working memory, semantic memory, episodic memory and procedural memory are introduced. It will be explained how, in CAM, all of these knowledge types are represented in dynamic decription logic (DDL), a formal logic with the capability for description and reasoning regarding dynamic application domains characterized by actions.

The bootstrap provides a simple and powerful means of improving the accuracy of clustering. However, for today's increasingly large datasets, the computation of bootstrap-based quantities can be prohibitively demanding. In this paper we introduce the Bag of Little Bootstraps Clustering (BLBC), a new procedure which utilizes the Bag of Little Bootstraps technique to obtain a robust, computationally efficient means of clustering for massive data. Moreover, BLBC is suited to implementation on modern parallel and distributed computing architectures which are often used to process large datasets. We investigate empirically the performance characteristics of BLBC and compare to the performances of existing methods via experiments on simulated data and real data. The results show that BLBC has a significantly more favorable computational profile than the bootstrap based clustering while maintaining good statistical correctness.

Transfer learning focuses on the learning scenarios when the test data from target domains and the training data from source domains are drawn from similar but different data distributions with respect to the raw features. Along this line, some recent studies revealed that the high-level concepts, such as word clusters, could help model the differences of data distributions, and thus are more appropriate for classification. In other words, these methods assume that all the data domains have the same set of shared concepts, which are used as the bridge for knowledge transfer. However, in addition to these shared concepts, each domain may have its own distinct concepts. In light of this, we systemically analyze the high-level concepts, and propose a general transfer learning framework based on nonnegative matrix trifactorization, which allows to explore both shared and distinct concepts among all the domains simultaneously. Since this model provides more flexibility in fitting the data, it can lead to better classification accuracy. Moreover, we propose to regularize the manifold structure in the target domains to improve the prediction performances. To solve the proposed optimization problem, we also develop an iterative algorithm and theoretically analyze its convergence properties. Finally, extensive experiments show that the proposed model can outperform the baseline methods with a significant margin. In particular, we show that our method works much better for the more challenging tasks when there are distinct concepts in the data.

Many classification techniques work well only under a common assumption that the training and test data are drawn from the same feature space and the same distribution. However, big velocity data usually show disobedience of this assumption. For example, in the field of web-document classification, new document is continuously emerging every day. Transfer learning aims at leveraging the knowledge in labeled source domains to predict the unlabeled data in a target domain, where the distributions are different in domains. As one of the important research directions of transfer learning, one kind of approaches focus on the correspondence between pivot features and all the other specific features from different domains, to extract some relevant features that may reduce the difference between the domains, have attracted wide attention and study. However, the limitation caused by the vague meanings in different domains prevents these algorithms from further improvement. To tackle this problem, we propose a cross-domain canonical correlation analysis algorithm called CD-CCA by applying Canonical Correlation Analysis (CCA) to transfer learning. CD-CCA can learn a semantic space of multi-view correspondences from different domains respectively and transfer the knowledge by dimensionality reduction in a multi-view way. Experimental results on the 144×6 classification problems in 20Newsgroups, show that CD-CCA can significantly improve the prediction accuracy.

Sparse coding is an active research topic in machine learning and signal processing community. In this paper, we propose a novel local sparse model for multi-label image annotation. Existing feature descriptors and extraction algorithms pay less attention to semantic information and extracted feature dimension usually is high, which leads to heavy computation. Noise and redundant information often reduce the performance of sparse model. To address these issues, we combine label and visual information for feature selection while most previous work only utilizes labels and ignores visual information itself. First of all, we make use of label sets to seek images neighbor relations and generate the Gaussian kernel matrix over these neighbor images, then use LLP(Local Learning Projection) algorithm to get minimal local estimation error. After that, for each query image, we find its K nearest neighbors in the transformed space and use these neighbors to reconstruct it via sparse coding. Moreover, during coding, we penalize the corresponding reconstruction coefficients to implicitly reflect the neighbor relations. Finally, propagating tags from training data to test data. Image annotation experiments on the Corel5k dataset show the performance of our approach is comparable to several state-of-the-art algorithms.

We present a new method for refining image annotation by fusing probabilistic latent semantic analysis (PLSA) with max-bisection (MB). We first construct a PLSA model with asymmetric modalities to estimate the posterior probabilities of each annotating keyword for an image, and then a label similarity graph is built by a weighted linear combination of label similarity and visual similarity. Followed by the rank-two relaxation heuristics over the constructed label graph is employed to further mine the correlation of the keywords so as to capture the refining annotation, which plays a critical role in semantic based image retrieval. The novelty of our method mainly lies in two aspects: exploiting PLSA to accomplish the initial semantic annotation task and implementing max-bisection based on the rank-two relaxation algorithm over the weighted label graph to refine the candidate annotations generated by the PLSA. We evaluate our method on the standard Corel dataset and the experimental results are competitive to several state-of-the-art approaches.

Collaborative Filtering (CF) is one of the most popular techniques for information filtering in recommendation systems. Currently, there are many linear and nonlinear regression algorithms for CF. However, to our knowledge, these regression algorithms may not give satisfactory results in some practical applications. In this paper, Extreme Learning Machine (ELM), which is famous with its fast speed and good performance in generalization, is firstly employed to build a nonlinear regression model for CF, namely ELM for CF (ELMCF) algorithm. Then by combining ELM and Weighted Nonnegative Matrix Tri-Factorization (WNMTF), which can alleviate the data sparsity problem of the user-item matrix, a new nonlinear regression model is proposed, namely Extreme Learning Machine Combining Matrix Factorization for Collaborative Filtering (CELMCF) algorithm, to construct regression based CF algorithms and improve the performance of recommendation systems. Experiments are conducted on several benchmark datasets from different application domains. Experimental results show that the proposed CELMCF algorithm outperforms some state-of-the-art regression based CF algorithms (including ELMCF algorithm, Linear Regression for CF (LRCF) algorithm and Memory based CF (MemCF) algorithm) more efficiently with the competitive effectiveness.

Transfer learning focuses on the learning scenarios when the test data from target domains and the training data from source domains are drawn from similar but different data distributions with respect to the raw features. Along this line, some recent studies revealed that the high-level concepts, such as word clusters, could help model the differences of data distributions, and thus are more appropriate for classification. In other words, these methods assume that all the data domains have the same set of shared concepts, which are used as the bridge for knowledge transfer. However, in addition to these shared concepts, each domain may have its own distinct concepts. In light of this, we systemically analyze the high-level concepts, and propose a general transfer learning framework based on nonnegative matrix trifactorization, which allows to explore both shared and distinct concepts among all the domains simultaneously. Since this model provides more flexibility in fitting the data, it can lead to better classification accuracy. Moreover, we propose to regularize the manifold structure in the target domains to improve the prediction performances. To solve the proposed optimization problem, we also develop an iterative algorithm and theoretically analyze its convergence properties. Finally, extensive experiments show that the proposed model can outperform the baseline methods with a significant margin. In particular, we show that our method works much better for the more challenging tasks when there are distinct concepts in the data.

The training algorithm of classical twin support vector regression (TSVR) can be attributed to the solution of a pair of quadratic programming problems (QPPs) with inequality constraints in the dual space. However, this solution is affected by time and memory constraints when dealing with large datasets. In this paper, we present a least squares version for TSVR in the primal space, termed primal least squares TSVR (PLSTSVR). By introducing the least squares method, the inequality constraints of TSVR are transformed into equality constraints. Furthermore, we attempt to directly solve the two QPPs with equality constraints in the primal space instead of the dual space; thus, we need only to solve two systems of linear equations instead of two QPPs. Experimental results on artificial and benchmark datasets show that PLSTSVR has comparable accuracy to TSVR but with considerably less computational time. We further investigate its validity in predicting the opening price of stock.

Multi-task learning has proven to be useful to boost the learning of multiple related but different tasks. Meanwhile, latent semantic models such as LSA and LDA are popular and effective methods to extract discriminative semantic features of high dimensional dyadic data. In this paper, we present a method to combine these two techniques together by introducing a new matrix tri-factorization based formulation for semi-supervised latent semantic learning, which can incorporate labeled information into traditional unsupervised learning of latent semantics. Our inspiration for multi-task semantic feature learning comes from two facts, i.e., 1) multiple tasks generally share a set of common latent semantics, and 2) a semantic usually has a stable indication of categories no matter which task it is from. Thus to make multiple tasks learn from each other we wish to share the associations between categories and those common semantics among tasks. Along this line, we propose a novel joint Nonnegative matrix tri-factorization framework with the aforesaid associations shared among tasks in the form of a semantic-category relation matrix. Our new formulation for multi-task learning can simultaneously learn (1) discriminative semantic features of each task, (2) predictive structure and categories of unlabeled data in each task, (3) common semantics shared among tasks and specific semantics exclusive to each task. We give alternating iterative algorithm to optimize our objective and theoretically show its convergence. Finally extensive experiments on text data along with the comparison with various baselines and three state-of-the-art multi-task learning algorithms demonstrate the effectiveness of our method.

Time series shapelets are small and local time series subsequences which are in some sense maximally representative of a class. E.Keogh uses distance of the shapelet to classify objects. Even though shapelet classification can be interpretable and more accurate than many state-of-the-art classifiers, there is one main limitation of shapelets, i.e. shapelet classification training process is offline, and uses subsequence early abandon and admissible entropy pruning strategies, the time to compute is still significant. In this work, we address the later problem by introducing a novel algorithm that finds time series shapelet in significantly less time than the current methods by extracting infrequent time series shapelet candidates. Subsequences that are distinguishable are usually infrequent compared to other subsequences. The algorithm called ISDT (Infrequent Shapelet Decision Tree) uses infrequent shapelet candidates extracting to find shapelet. Experiments demonstrate the efficiency of ISDT algorithm on several benchmark time series datasets. The result shows that ISDT significantly outperforms the current shapelet algorithm.

In this talk I introduce cloud computing based cross-media knowledge discovery. We propose a framework for cross-media semantic understanding which contains discriminative modeling, generative modeling and cognitive modeling. In cognitive modeling a new model entitled CAM is proposed which is suitable for cross-media semantic understanding. We develop an agent-aid model for load balance in cloud computing environment. For quality of service we present a utility function to evaluate the cloud performance. A Cross-Media Intelligent Retrieval System (CMIRS), which is managed by ontology-based knowledge system KMSphere, will be illustrated. Finally, the directions for further researches on cloud computing based cross-media knowledge discovery will be pointed out and discussed.

Searching frequent patterns in transactional databases is considered as one of the most important data mining problems and Apriori is one of the typical algorithms for this task. Developing fast and efficient algorithms that can handle large volumes of data becomes a challenging task due to the large databases. In this paper, we implement a parallel Apriori algorithm based on MapReduce, which is a framework for processing huge datasets on certain kinds of distributable problems using a large number of computers (nodes). The experimental results demonstrate that the proposed algorithm can scale well and efficiently process large datasets on commodity hardware.

A novel method based on rough sets (RS) and the affinity propagation (AP) clustering algorithm is developed to optimize a radial basis function neural network (RBFNN). First, attribute reduction (AR) based on RS theory, as a preprocessor of RBFNN, is presented to eliminate noise and redundant attributes of datasets while determining the number of neurons in the input layer of RBFNN. Second, an AP clustering algorithm is proposed to search for the centers and their widths without a priori knowledge about the number of clusters. These parameters are transferred to the RBF units of RBFNN as the centers and widths of the RBF function. Then the weights connecting the hidden layer and output layer are evaluated and adjusted using the least square method (LSM) according to the output of the RBF units and desired output. Experimental results show that the proposed method has a more powerful generalization capability than conventional methods for an RBFNN.

The distribution difference among multiple domains has been exploited for cross-domain text categorization in recent years. Along this line, we show two new observations in this study. First, the data distribution difference is often due to the fact that different domains use different index words to express the same concept. Second, the association between the conceptual feature and the document class can be stable across domains. These two observations actually indicate the distinction and commonality across domains. Inspired by the above observations, we propose a generative statistical model, named Collaborative Dual-PLSA (CD-PLSA), to simultaneously capture both the domain distinction and commonality among multiple domains. Different from Probabilistic Latent Semantic Analysis (PLSA) with only one latent variable, the proposed model has two latent factors y and z, corresponding to word concept and document class, respectively. The shared commonality intertwines with the distinctions over multiple domains, and is also used as the bridge for knowledge transformation. An Expectation Maximization (EM) algorithm is developed to solve the CD-PLSA model, and further its distributed version is exploited to avoid uploading all the raw data to a centralized location and help to mitigate privacy concerns. After the training phase with all the data from multiple domains we propose to refine the immediate outputs using only the corresponding local data. In summary, we propose a two-phase method for cross-domain text classification, the first phase for collaborative training with all the data, and the second step for local refinement. Finally, we conduct extensive experiments over hundreds of classification tasks with multiple source domains and multiple target domains to validate the superiority of the proposed method over existing state-of-the-art methods of supervised and transfer learning. It is noted to mention that as shown by the experimental results CD-PLSA for the collaborative training is more tolerant of distribution differences, and the local refinement also gains significant improvement in terms of classification accuracy.

The distribution difference among multiple domains has been exploited for cross-domain text categorization in recent years. Along this line, we show two new observations in this study. First, the data distribution difference is often due to the fact that different domains use different index words to express the same concept. Second, the association between the conceptual feature and the document class can be stable across domains. These two observations actually indicate the distinction and commonality across domains. Inspired by the above observations, we propose a generative statistical model, named Collaborative Dual-PLSA (CD-PLSA), to simultaneously capture both the domain distinction and commonality among multiple domains. Different from Probabilistic Latent Semantic Analysis (PLSA) with only one latent variable, the proposed model has two latent factors y and z, corresponding to word concept and document class, respectively. The shared commonality intertwines with the distinctions over multiple domains, and is also used as the bridge for knowledge transformation. An Expectation Maximization (EM) algorithm is developed to solve the CD-PLSA model, and further its distributed version is exploited to avoid uploading all the raw data to a centralized location and help to mitigate privacy concerns. After the training phase with all the data from multiple domains we propose to refine the immediate outputs using only the corresponding local data. In summary, we propose a two-phase method for cross-domain text classification, the first phase for collaborative training with all the data, and the second step for local refinement. Finally, we conduct extensive experiments over hundreds of classification tasks with multiple source domains and multiple target domains to validate the superiority of the proposed method over existing state-of-the-art methods of supervised and transfer learning. It is noted to mention that as shown by the experimental results CD-PLSA for the collaborative training is more tolerant of distribution differences, and the local refinement also gains significant improvement in terms of classification accuracy.

Distributed and Parallel algorithms have attracted a vast amount of interest and research in recent decades, to handle large-scale data set in real-world applications. In this paper, we focus on a parallel implementation of KD-Tree based outlier detection method to deal with large-scale data set. As one of the state-of-the-art outlier detection methods, KD-Tree based has been approved to be an effective algorithm. However, it still cannot process large-scale data set efficiently due to its serial implementation. Based on the current and powerful parallel programming framework -- MapReduce, we propose to implement the parallel KD-Tree based outlier detection algorithm (e.g., PKDTree for short). Experimental results demonstrate the efficiency of PKDTree according to the evaluation criterions of scale up, speedup and size up.

Curve extraction is an important and basic technique in image processing and computer vision. Due to the complexity of the images and the limitation of segmentation algorithms, there are always a large number of noisy pixels in the segmented binary images. In this paper, we present an approach based on ant colony system (ACS) to detect nonparametric curves from a binary image containing discontinuous curves and noisy points. Compared with the well-known Hough transform (HT) method, the ACS-based curve extraction approach can deal with both regular and irregular curves without knowing their shapes in advance. The proposed approach has many characteristics such as faster convergence, implicit parallelism and strong ability to deal with highly-noised images. Moreover, our approach can extract multiple curves from an image, which is impossible for the previous genetic algorithm based approach. Experimental results show that the proposed ACS-based approach is effective and efficient.

The content and structure of linked information such as sets of web pages or research paper archives are dynamic and keep on changing. Even though different methods are proposed to exploit both the link structure and the content information, no existing approach can effectively deal with this evolution. We propose a novel joint model, called Link-IPLSI, to combine texts and links in a topic modeling framework incrementally. The model takes advantage of a novel link updating technique that can cope with dynamic changes of online document streams in a faster and scalable way. Furthermore, an adaptive asymmetric learning method is adopted to freely control the assignment of weights to terms and citations. Experimental results on two different sources of online information demonstrate the time saving strength of our method and indicate that our model leads to systematic improvements in the quality of classification and link prediction.

The distribution difference among multiple data domains has been considered for the cross-domain text classification problem. In this study, we show two new observations along this line. First, the data distribution difference may come from the fact that different domains use different key words to express the same concept. Second, the association between this conceptual feature and the document class may be stable across domains. These two issues are actually the distinction and commonality across data domains. Inspired by the above observations, we propose a generative statistical model, named Collaborative Dual-PLSA (CD-PLSA), to simultaneously capture both the domain distinction and commonality among multiple domains. Different from Probabilistic Latent Semantic Analysis (PLSA) with only one latent variable, the proposed model has two latent factors y and z, corresponding to word concept and document class respectively. The shared commonality intertwines with the distinctions over multiple domains, and is also used as the bridge for knowledge transformation. We exploit an Expectation Maximization (EM) algorithm to learn this model, and also propose its distributed version to handle the situation where the data domains are geographically separated from each other. Finally, we conduct extensive experiments over hundreds of classification tasks with multiple source domains and multiple target domains to validate the superiority of the proposed CD-PLSA model over existing state-of-the-art methods of supervised and transfer learning. In particular, we show that CD-PLSA is more tolerant of distribution differences.

Automatic image annotation has become an important and challenging problem due to the existence of semantic gap. In this paper, we firstly extend probabilistic latent semantic analysis (PLSA) to model continuous quantity. In addition, corresponding Expectation-Maximization (EM) algorithm is derived to determine the model parameters. Furthermore, in order to deal with the data of different modalities in terms of their characteristics, we present a semantic annotation model which employs continuous PLSA and standard PLSA to model visual features and textual words respectively. The model learns the correlation between these two modalities by an asymmetric learning approach and then it can predict semantic annotation for unseen images. We compare our approach with several state-of-the-art approaches on a standard Corel dataset. The experiment results show that our approach performs more effectively and accurately.

Understanding and modeling the function of the neurons and neural systems are primary goal of systems neuroscience. Sparse coding theory demonstrates that the neurons in primary visual cortex form a sparse representation of natural scenes in the viewpoint of statistics. In this paper, we propose a novel sparse coding model based on structural similarity (SS_SC) for natural image feature extraction. The advantage for our model is to be able to preserve structural information from a scene, which human visual perception is highly adapted for. Using the proposed sparse coding model, the validity of image feature extraction is testified. Furthermore, compared with standard sparse coding (SC) model, the experimental results show that the quality of reconstructed images obtained by our method outperforms the SC method.

We study what we call semi-defined classification, which deals with the categorization tasks where the taxonomy of the data is not well defined in advance. It is motivated by the real-world applications, where the unlabeled data may also come from some other unknown classes besides the known classes for the labeled data. Given the unlabeled data, our goal is to not only identify the instances belonging to the known classes, but also cluster the remaining data into other meaningful groups. It differs from traditional semi-supervised clustering in the sense that in semi-supervised clustering the supervision knowledge is too far from being representative of a target classification, while in semi-defined classification the labeled data may be enough to supervise the learning on the known classes. In this paper we propose the model of Double-latent-layered LDA (D-LDA for short) for this problem. Compared with LDA with only one latent variable y for word topics, D-LDA contains another latent variable z for (known and unknown) document classes. With this double latent layers consisting of y and z and the dependency between them, D-LDA directly injects the class labels into z to supervise the exploiting of word topics in y. Thus, the semi-supervised learning in D-LDA does not need the generation of pair wise constraints, which is required in most of the previous semi-supervised clustering approaches. We present the experimental results on ten different data sets for semi-defined classification. Our results are either comparable to (on one data sets), or significantly better (on the other nine data set) than the six compared methods, including the state-of-the-art semi-supervised clustering methods.

In the real world, images always have several visual objects instead of only one, which makes it difficult for conventional object recognition methods to deal with them. In this paper, we present a topologically sorted classifier chain method for learning images with multi-label. We first provide a means of generating a topo-logically sorted label chain ordering by employing a topological sort algorithm and then apply the chain ordering to the classifier chain model proposed by [1] to classify multi-label images. Our method can capture the correlations between labels very effectively due to the sorted label chain ordering and the advantages brought by classifier chain method. We evaluate the proposed method on Corel dataset and demonstrate the micro and macro F1 measures superior to the state-of-the-art methods.

Cross-domain text categorization targets on adapting the knowledge learnt from a labeled source-domain to an unlabeled target-domain, where the documents from the source and target domains are drawn from different distributions. However, in spite of the different distributions in raw word features, the associations between word clusters (conceptual features) and document classes may remain stable across different domains. In this paper, we exploit these unchanged associations as the bridge of knowledge transformation from the source domain to the target domain by the nonnegative matrix tri-factorization. Specifically, we formulate a joint optimization framework of the two matrix tri-factorizations for the source and target domain data respectively, in which the associations between word clusters and document classes are shared between them. Then, we give an iterative algorithm for this optimization and theoretically show its convergence. The comprehensive experiments show the effectiveness of this method. In particular, we show that the proposed method can deal with some difficult scenarios where baseline methods usually do not perform well.

With the rapid development of XML language which has good flexibility and interoperability, more and more log files of software running information are represented in XML format, especially for Web services. Fault diagnosis by analyzing semi-structured and XML like log files is becoming an important issue in this area. For most related learning methods, there is a basic assumption that training data should be in identical structure, which does not hold in many situations in practice. In order to learn from training data in different structures, we propose a similarity-based Bayesian learning approach for fault diagnosis in this paper. Our method is to first estimate similarity degrees of structural elements from different log files. Then the basic structure of combined Bayesian network (CBN) is constructed, and the similarity-based learning algorithm is used to compute probabilities in CBN. Finally, test log data can be classified into possible fault categories based on the generated CBN. Experimental results show our approach outperforms other learning approaches on those training datasets which have different structures.

Classification across different domains studies how to adapt a learning model from one domain to another domain which shares similar data characteristics. While there are a number of existing works along this line, many of them are only focused on learning from a single source domain to a target domain. In particular, a remaining challenge is how to apply the knowledge learned from multiple source domains to a target domain. Indeed, data from multiple source domains can be semantically related, but have different data distributions. It is not clear how to exploit the distribution differences among multiple source domains to boost the learning performance in a target domain. To that end, in this paper, we propose a consensus regularization framework for learning from multiple source domains to a target domain. In this framework, a local classifier is trained by considering both local data available in one source domain and the prediction consensus with the classifiers learned from other source domains. Moreover, we provide a theoretical analysis as well as an empirical study of the proposed consensus regularization framework. The experimental results on text categorization and image classification problems show the effectiveness of this consensus regularization learning method. Finally, to deal with the situation that the multiple source domains are geographically distributed, we also develop the distributed version of the proposed algorithm, which avoids the need to upload all the data to a centralized location and helps to mitigate privacy concerns.

Since the scalability of logic circuits is becoming larger and more complex, the auto-design is becoming more and more difficult. In order to improve automatic design and performance evaluation of logic circuits in efficiency and capability of optimization, multiobjective simulated annealing (MSA) based increasable evolution approach is designed to evolve logic circuits automatically with an extended matrix encoding method, which can be able to reflect the potential performance of a circuit and reduce the risk of deleting a circuit with a good developing potential during evolution is devised. In the process of evolution, each individual is renewedly associated to a corresponding objective in terms of a novel adaptive evaluation method at each generation. In experiments, complicated arithmetic circuits are designed to assess the performance of MSA against other algorithms. Results indicate that the proposed method could design logic circuits efficiently.

In this paper, we present an approach based on probabilistic latent semantic analysis (PLSA) to accomplish the tasks of automatic image annotation. In order to model training data precisely, we represent an image as a bag of visual words and employ two PLSA models to capture semantic information from visual and textual modalities respectively. Furthermore, an adaptive learning approach is proposed to combine the aspects learned from both modalities. For each image document, distribution over aspects is fused by different weight in terms of the entropy of its feature distribution. Consequently, the two models are linked with the same distribution over aspects. This structure can predict semantic annotation for an unseen image because it associates visual and textual modalities properly. We compare our approach with several previous approaches on a standard Corel dataset. The experiment results show that our approach performs more effectively and accurately.

We describe a method for filtering object category from a large number of noisy images. This problem is particularly difficult due to the greater variation within object categories and lack of labeled object images. Our method deals with it by combining a co-training algorithm CoBoost with two features - 1stand 2ndorder features, which define bag of words representation and spatial relationship between local features respectively. We iteratively train two boosting classifiers based on the 1stand 2ndorder features, during which each classifier provides labeled data for the other classifier. It is effective because the 1stand 2ndorder features make up an independent and redundant feature split. We evaluate our method on Berg dataset and demonstrate the precision comparative to the state-of-the-art.

Automatic image annotation has become an important and challenging problem due to the existence of semantic gap. In this paper, we present an approach based on probabilistic latent semantic analysis (PLSA) to accomplish the tasks of semantic image annotation and retrieval. In order to model training images precisely, we employ two PLSA models to capture semantic information from visual and textual modalities respectively. Then an adaptive asymmetric learning approach is proposed to fuse aspects which are learned from both modalities. For each image document, the weight of each modality is determined by its contribution to the content of the image. Consequently, the two models are linked with the same distribution over aspects. This structure can predict semantic annotation for an unseen image because it associates visual and textual modalities properly. Finally, we compare our approach with several previous approaches on a standard Corel dataset. The experiment results show that our approach performs more effective and accurate.

We describe a method for filtering object category from a large number of noisy images. This problem is particularly difficult due to the greater variation within object categories and only a few labeled object images available. Our method deals with it by using visual consistency and semi-supervised approach. The images of one category often share some visual consistency so that the most irrelevant images can be first removed. Among the left images, a voting method is used to obtain more object exemplars with the initial object exemplars manually selected by users. Finally with all the obtained exemplars and those unlabeled images, we create a semi-supervised classifier to rank all the images. We evaluate our method on Berg dataset and demonstrate the precision comparative to the state-of-the-art. Besides, we collect five more categories from Google images to show the effectiveness of the method.

Web content analysis often has two sequential and separate steps: Web Classification to identify the target Web pages, and Web Information Extraction to extract the metadata contained in the target Web pages. This decoupled strategy is highly ineffective since the errors in Web classification will be propagated to Web information extraction and eventually accumulate to a high level. In this paper we study the mutual dependencies between these two steps and propose to combine them by using a model of Conditional Random Fields (CRFs). This model can be used to simultaneously recognize the target Web pages and extract the corresponding metadata. Systematic experiments in our project OfCourse for online course search show that this model significantly improves the F1 value for both of the two steps. We believe that our model can be easily generalized to many Web applications.

Conventional object recognition techniques rely heavily on manually annotated image datasets to achieve good performances. However, collecting high quality datasets is really laborious. In this paper, we propose a semi-supervised framework for learning visual categories from Google Images. The 1st and 2nd order features, which define bag of words representation and spatial relationship between local features respectively, make up an independent and redundant feature split. We then integrate a cotraining algorithm CoBoost with these two features. We create two boosting classifiers based on the 1st and 2nd order features respectively in the training, during which one classifier provides labels for the other. Besides, the 2nd order features are generated dynamically rather than extracted exhaustively to avoid high computation. An active learning technique is also introduced to further improve the performance. We evaluate our method on the benchmark datasets, showing results competitive with the state-of-the-art unsupervised approaches and some supervised techniques.

This paper presents a formalism for representing and reasoning about Web services with local closed world assumption (LCWA) on the basis of $\mathcal{ALCO@K}$. In our formalism, the knowledge about the states of the world is encoded in $\mathcal {ALCO@}$-ABoxes; atomic services are represented in terms of their preconditions (epistemic queries to the knowledge base) and effects (possibly negated $\mathcal{ALCO@}$-assertions involving only atomic concepts); and composite services are built up with action constructors in dynamic logics. We also summarize some reasoning tasks and develop a calculus for them. Our formalism also enjoys \emph{introspection}. The main features of our proposal (i.e., dynamic reasoning, local closed world assumption and introspection) make it more philosophically satisfying and much closer towards a practical formalism for agents with incomplete knowledge in the Web full of static information and dynamic processing.

This paper presents a framework that actively selects informative documents pairs for semi-supervised document clustering. The semi-supervised document clustering algorithm is a Constrained DBSCAN (Cons-DBSCAN), which incorporates instance-level constraints to guide the clustering process in DBSCAN. By obtaining user feedbacks, our proposed active learning algorithm can get informative instance level constraints to aid clustering process. Experimental results show that Cons-DBSCAN with the proposed active learning approach can provide an appealing clustering performance.

This paper studies the generalization and normalization issues of information-theoretic distance measures for clustering validation. Along this line, we first introduce a uniform representation of distance measures, defined as quasi-distance, which is induced based on a general form of conditional entropy. The quasi-distance possesses three properties: symmetry, the triangle law, and the minimum reachable. These properties ensure that the quasi-distance naturally lends itself as the external measure for clustering validation. In addition, we observe that the ranges of the distance measures are different when they apply for clustering validation on different data sets. Therefore, when comparing the performances of clustering algorithms on different data sets, distance normalization is required to equalize ranges of the distance measures. A critical challenge for distance normalization is to obtain the ranges of a distance measure when a data set is provided. To that end, we theoretically analyze the computation of the maximum value of a distance measure for a data set. Finally, we compare the performances of the partition clustering algorithm K-means on various real-world data sets. The experiments show that the normalized distance measures have better performance than the original distance measures when comparing clusterings of different data sets. Also, the normalized Shannon distance has the best performance among four distance measures under study.

Exploring mechanism of cognitive process is a valuable means of investigating artificial intelligence. But human beingpsilas perception of cognitive process is very limited so far, and related researches are required to be further deepened. This paper focuses on modeling cognitive process based on theory of granular computing(GrC). First, fundamental principle of GrC is used to analyze cognitive process and establish its model. Secondly, a tolerance relation on vector space(ldquorv_spacerdquo for short) is defined by using a distance fucntion dis, and a tolerance granular space model is established, in which the problem of approximate representation of unknown concept(abstraction of perceptive information) is discussed and analyzed. The analyses show that a given concept, sometimes, is difficult to be learned(i.e., difficult to be accurately represented), so appropriate granular world is required. The solution to this problem involves relationship and transformation between granular worlds. At last, such relationship and transformation are investigated, and many related properties and theorems are deduced, solving the facing problem to some extent. The theory and approach of representation of a concept in a granular world and transformation between different granular worlds are made of support cognitive processpsila granular model.

Dynamic description logic (DDL) is among the few emerging service composition solutions through logical reasoning. To overcome low efficiency and lacking context-aware support of DDL reasoning, we propose a new DDL-based service composition model, which supports context-based service pre-filtering over DDL reasoning space. The pre-filtering runs under the BPEL workflow and a distributed reasoning algorithm need to reasoning different contexts after pre-filtering.

Focused crawlers selectively retrieve Web documents that are relevant to a predefined set of topics. To intelligently make predictions and decisions about relevant URLs and web pages, different topic models have been introduced to represent topic-specific knowledge. Yet it is difficult to support semantic interoperability among different models. Moreover, some manually specified additional semantic information, such as semantic markups and social annotations, could not be effectively used to improve crawling. This paper proposes to boost focused crawling with four kinds of semantic models and semantic information, including thesauruses, categories, ontologies, and folksonomies. A statistical semantic association model is proposed to integrate different semantic models, represent heterogeneous semantic information, and support semantic relevance computation. A focused crawling framework is developed which adopts both keyword based contents and different kinds of additional information for relevance prediction and ranking. Experiments show that the proposed model and framework effectively integrates heterogeneous semantic information for focused crawling.

Hyper Surface Classification (HSC), which is based on Jordan Curve Theorem in Topology, has proven to be a simple and effective method for classifying a larger database in our previous work. To select a representative subset from the original sample set, the Minimal Consistent Subset (MCS) of HSC is studied in this paper. For HSC method, one of the most important features of MCS is that it has the same classification model as the entire sample dataset, and can totally reflect its classification ability. From this point of view, MCS is the best way of sampling from the original dataset for HSC. Furthermore, because of the minimum property of MCS, every single deletion or multiple deletions from it will lead to a reduction in generalization ability, which can be exactly predicted by the proposed formula in this paper.

This work studies the problem of distributed classification in peer-to-peer(P2P) networks. While there has been a significant amount of work in distributed classification, most of existing algorithms are not designed for P2P networks. Indeed, as server-less and router-less systems, P2P networks impose several challenges for distributed classification: (1) it is not practical to have global synchronization in large-scale P2P networks; (2)there are frequent topology changes caused by frequent failure and recovery of peers; and (3) there are frequent on-the-fly data updates on each peer.

Business intelligence (BI) is the process of gathering enough of the right information in the right manner at the right time, and delivering the right results to the right people for decision-making purposes so that it can continue to yield real business benefits, or have a positive impact on business strategy, tactics, and operations in the enterprises. This paper was intended as a short introduction to the study of business intelligence in enterprise computing environment. In addition, the conclusions point out the challenges to broad and deep deployment of business intelligence systems, and provide the proposals of making business intelligence more effective.

In information systems in particular business Intelligence systems, for data produced at physically distributed locations most traditional data mining approaches require data to be transmitted to a single location for centralized processing and mining. However, the continual transmission of a large number of data to a central location must be impractical and expensive. Thus, distributed and parallel data mining algorithms and applications were rapidly developed. The paper surveys the-state-of-the art in approaches and applications of distributed computing environment. The goal is to summarize a brief introduction to this field with pointers for further exploration.

Partition entropyis the numerical metric of uncertainty within a partition of a finite set, whileconditional entropymeasures the degree of difficulty in predicting a decision partition when a condition partition is provided. Since two direct methods exist for defining conditional entropy based on its partition entropy, the inequality postulates of monotonicity, which conditional entropy satisfies, are actually additional constraints on its entropy. Thus, in this paper partition entropy is defined as a function of probability distribution, satisfying all the inequalities of not only partition entropy itself but also its conditional counterpart. These inequality postulates formalize the intuitive understandings of uncertainty contained in partitions of finite sets. We study the relationships between these inequalities, and reduce the redundancies among them. According to two different definitions of conditional entropy from its partition entropy, the convenient and unified checking conditions for any partition entropy are presented, respectively. These properties generalize and illuminate the common nature of all partition entropies.

Web services composition techniques are gaining momentum as the opportunity to establish reusable and versatile inter-operability applications. The purpose of semantic Web services is to use semantic specification to automate the discovery, invocation, and composition Web services. Description logics is the formalized foundation of semantic Web services and provides well-defined semantics. And many researchers propose their composition approach based on planning techniques. We propose our service composition methods based on description logics and AI planning technologies. Our algorithm for services composition uses backward-chaining search method to find potential candidate services. And we propose a DAG-based method to generate the planning process and filtering the inappropriate services during the DAG generation process. We test our approach on a simple, yet realistic example, and the preliminary results demonstrate that our implementation provides a practical solution

Business Intelligence (BI) has been viewed as sets of powerful tools and approaches to improving business executive decision-making, business operations, and increasing the value of the enterprise. The technology categories of BI mainly encompass Data Warehousing, OLAP, and Data Mining. This article reviews the concept of Business Intelligence and provides a survey, from a comprehensive point of view, on the BI technical framework, process, and enterprise solutions. In addition, the conclusions point out the possible reasons for the difficulties of broad deployment of enterprise BI, and the proposals of constructing a better BI system.

Abstract:The computing‐intensive data mining (DM) process calls for the support of a heterogeneous computing system, which consists of multiple computers with different configurations connected by a high‐speed large‐area network for increased computational power and resources. The DM process can be described as a multi‐phase pipeline process, and in each phase there could be many optional methods. This makes the workflow for DM very complex and it can be modeled only by a directed acyclic graph (DAG). A heterogeneous computing system needs an effective and efficient scheduling framework, which orchestrates all the computing hardware to perform multiple competitive DM workflows. Motivated by the need for a practical solution of the scheduling problem for the DM workflow, this paper proposes a dynamic DAG scheduling algorithm according to the characteristics of an execution time estimation model for DM jobs. Based on an approximate estimation of job execution time, this algorithm first maps DM jobs to machines in adecentralizedanddiligent(defined in this paper) manner. Then the performance of this initial mapping can be improved throughjob migrationswhen necessary. The scheduling heuristic used considers the factors of both theminimal completion timecriterion and thecritical pathin a DAG. We implement this system in an established multi‐agent system environment, in which the reuse of existing DM algorithms is achieved by encapsulating them into agents. The system evaluation and its usage in oil well logging analysis are also discussed.

This paper deals with the possible benefits of perceptual learning in artificial intelligence. On the one hand, perceptual learning is more and more studied in neurobiology and is now considered as an essential part of any living system. In fact, perceptual learning and cognitive learning are both necessary for learning and often depend on each other. On the other hand, many works in machine learning are concerned with "abstraction" in order to reduce the amount of complexity related to some learning tasks. In the abstraction framework, perceptual learning can be seen as a specific process that learns how to transform the data before the traditional learning task itself takes place. In this paper, we argue that biologically inspired perceptual learning mechanisms could be used to build efficient low-level abstraction operators that deal with real-world data. To illustrate this, we present an application where perceptual-learning-inspired metaoperators are used to perform an abstraction on an autonomous robot visual perception. The goal of this work is to enable the robot to learn how to identify objects it encounters in its environment.

The combination of large data set size, geographic distribution of resources and users, and sophisticated applications on data and information needs robust infrastructures. In this paper, knowledge management sphere (KMSphere) is proposed and developed to explore important aspects of service-oriented and ontology-driven knowledge management on the grid. The main idea of KMSphere is to integrate ontologies with a service-oriented grid, build a knowledge space on top of databases, and then organize, utilize, and manage the knowledge resources in that space.

We put forward a hypothesis that there exist common meaningful structures among ontologies whose domains are analogous to each other The initial motivation of our hypothesis is to make full use of the structural information in existing ontologies, in order to benefit the domain of ontology. To verify the hypothesis we give a precise definition of the candidate of the common meaningful structure called MICISO (maximum isomorphic common induced sub-ontology). Based on the hypothesis and the definition we present a novel data mining problem called MICISO mining, whose aim is learning from ontologies to find out MICISOs and further recommend the common meaningful structures. We also provide an algorithm for MICISO mining, based on which we have developed a practical tool for mining and checking such structures. With the tool, the algorithm is implemented with quite a few pairs of existing ontologies, and the interesting meaningful results support our hypothesis. Thus we consider that the hypothesis is preliminarily verified. We suppose that our work sparks a novel promising thinking for the domain of ontology -to study existing ontologies for useful things.

This paper studies the relations of individuals in evolutionary populations, and then investigates some features of the relations. The goal is to find efficient methods to construct the non-dominated set. It is proved that the individuals can be sorted by quick sort. To demonstrate the efficiency of our new method, we propose a multi-objective genetic algorithm (MOGA) based on quick sort, which is called QKMOGA. We apply QKMOGA on automatic negotiation for agents. A simple negotiation model between two agents is described, and the negotiation protocols are constructed with QKMOGA. Two experimental results show that the performance is satisfactory on the diversity and efficiency of the solutions.

Rough set theory is a relatively new mathematical tool for use in computer applications in circumstances which are characterized by vagueness and uncertainty. In this paper, we introduce the concepts of information entropy, rough entropy and knowledge granulation in rough set theory, and establish the relationships among those concepts. These results will be very helpful for understanding the essence of concept approximation and establishing granular computing in rough set theory.

This paper presents a document clustering algorithm based on swarm intelligence and k-means: CSIM. First, a document clustering algorithm based on swarm intelligence is employed. It is derived from a basic model interpreting ant colony organization of cemeteries. Swarm intelligence for flexibility, self-organization and robustness has been applied in a variety of areas. Taking advantage of these traits, good initial clusters are obtained in the first phase in CSIM. We then combine it with the classical k-means clustering method by using the clusters as initial centers. CSIM inherits the prominent properties of both swarm intelligence and k-means. It also offsets the weakness of those two techniques. Experimental results show the good performance of the hybrid document clustering algorithm.

Decision models play an important role in decision-making, and supporting model-building is one of the most important functions of model management in decision support systems. As concepts at different abstract levels have to be used in the process of model-building, representing these concepts in a coherent way has been recognized as a key research topic. In this paper, a model-building framework is proposed which integrates object-oriented analysis with action logic as the representation tool. This model-building framework can provide representations for concepts at different abstract levels and can describe the process of abstracting decision models from decision situations or problems represented in lower abstract level concepts.

The form has become an important abstraction for data management in an office application environment.Structured office forms present data to users in an easily understood and easily manipulated manner.In this paper we classify forms systems in terms of three dimensions: data structuring, user interfaces, and programming interfaces.Current forms systems are analyzed under these dimensions.We have designed a comprehensive forms management system, FORMANAGER, that includes facilities for form specification, form processing, and form control.The system transforms data from a relational database into a hierarchical data structure which defines the form.The design and algorithms for implementation of the system are described, and future extensions to enhance the capabilities of forms systems are proposed.