Training deep neural networks typically relies on gradient descent learning schemes, which is usually time-consuming, and the design of complex network architectures is often intractable. In this article, we explore the building of multilayer neural networks based on an efficient gradient-free learning scheme offering a potential solution to the architectural design. The proposed learning scheme encompasses both forward and backward training (BT) processes. In the forward process, the pseudoinverse learning (PIL) algorithm is employed to train a multilayer neural network, in which the network is dynamically constructed leveraging a layer-by-layer greedy strategy, enabling the automatic determination of the architecture across different hierarchies in a data-driven manner. The network architecture and connection weights determined in the forward training (FT) process are shared with the backward process which also conducts gradient-free learning to update the connection weights. After the bidirectional learning, a neural network comprising two twin subnetworks is obtained, and the fused features of subnetworks are used as inputs for downstream tasks. Comprehensive experiments and detailed analyses demonstrate the effectiveness and superiority of the proposed learning scheme.

For completely unknown affine nonlinear systems, in this article, a synergetic learning algorithm (SLA) is developed to learn an optimal control. Unlike the conventional Hamilton-Jacobi-Bellman equation (HJBE) with system dynamics, a model-free HJBE (MF-HJBE) is deduced by means of off-policy reinforcement learning (RL). Specifically, the equivalence between HJBE and MF-HJBE is first bridged from the perspective of the uniqueness of the solution of the HJBE. Furthermore, it is proven that once the solution of MF-HJBE exists, its corresponding control input renders the system asymptotically stable and optimizes the cost function. To solve the MF-HJBE, the two agents composing the synergetic learning (SL) system, the critic agent and the actor agent, can evolve in real-time using only the system state data. By building an experience reply (ER)-based learning rule, it is proven that when the critic agent evolves toward the optimal cost function, the actor agent not only evolves toward the optimal control, but also guarantees the asymptotic stability of the system. Finally, simulations of the F16 aircraft system and the Van der Pol oscillator are conducted and the results support the feasibility of the developed SLA.

Large Language Models (LLMs) have achieved impressive results in Machine Translation by simply following instructions, even without training on parallel data. However, LLMs still face challenges on low-resource languages due to the lack of pre-training data. In real-world situations, humans can become proficient in their native languages through abundant and meaningful social interactions and can also learn foreign languages effectively using well-organized textbooks. Drawing inspiration from human learning patterns, we introduce the Translate After LEarNing Textbook (TALENT) approach, which aims to enhance LLMs’ ability to translate low-resource languages by learning from a textbook. TALENT follows a step-by-step process: (1) Creating a Textbook for low-resource languages. (2) Guiding LLMs to absorb the Textbook’s content for Syntax Patterns. (3) Enhancing translation by utilizing the Textbook and Syntax Patterns. We thoroughly assess TALENT’s performance using 112 low-resource languages from FLORES-200 with two LLMs: ChatGPT and BLOOMZ. Evaluation across three different metrics reveals that TALENT consistently enhances translation performance by 14.8% compared to zero-shot baselines. Further analysis demonstrates that TALENT not only improves LLMs’ comprehension of low-resource languages but also equips them with the knowledge needed to generate accurate and fluent sentences in these languages.

A text corpus centered on events is foundational to research concerning the detection, representation, reasoning, and harnessing of online events. The majority of current event-based datasets mainly target sentence-level tasks, thus to advance event-related research spanning from sentence to document level, this paper introduces DEIE, a unified large-scale document-level event information extraction dataset with over 56,000+ events and 242,000+ arguments. Three key features stand out: large-scale manual annotation (20,000 documents), comprehensive unified annotation (encompassing event trigger/argument, summary, and relation at once), and emergency events annotation (covering 19 emergency types). Notably, our experiments reveal that current event-related models struggle with DEIE, signaling a pressing need for more advanced event-related research in the future.

Driven by deep learning, the field of content generation has witnessed remarkable progress. However, it still faces several challenges, such as low efficiency and high difficulty in training. To confront these obstacles, we propose an efficient and effective lightweight generative model, termed the pseudoinverse learning based variational autoencoder, within the framework of the synergetic learning system. The proposed learning system in this study comprises a reductive subsystem and a generative subsystem, incorporates a non-gradient learning scheme. The reductive subsystem employs variants of the pseudoinverse learning algorithm and probabilistic principal component analysis to embed the inputs into the latent space which is constrained to follow a standard normal distribution. The generative subsystem performs the inverse reconstruction process of the reductive subsystem, which can be used for content generation after training. The experimental results show that the proposed model significantly speeds up the training while achieving comparable generation quality to the baselines.

In today's digital age, accessing information across language barriers poses a significant challenge, with conventional search systems often struggling to interpret and retrieve multilingual content accurately. Addressing this issue, our study introduces a novel integration of applying Large Language Models (LLMs) as Cross-lingual Readers in information retrieval systems, specifically targeting the complexities of cross-lingual information retrieval (CLIR). We present an innovative approach: Activation Steered Multilingual Retrieval (ASMR) that employs "steering activations''-a method to adjust and direct the LLM's focus-enhancing its ability to understand user queries and generate accurate, language-coherent responses. ASMR adeptly combines a Multilingual Dense Passage Retrieval (mDPR) system with an LLM, overcoming the limitations of traditional search engines in handling diverse linguistic inputs. This approach is particularly effective in managing the nuances and intricacies inherent in various languages. Rigorous testing on established benchmarks such as XOR-TyDi QA, and MKQA demonstrates that ASMR not only meets but surpasses existing standards in CLIR, achieving state-of-the-art performance. The results of our research hold significant implications for understanding the inherent features of how LLMs understand and generate natural languages, offering an attempt towards more inclusive, effective, and linguistically diverse information access on a global scale.

In the contemporary digital landscape, search engines play an invaluable role in information access, yet they often face challenges in Cross-Lingual Information Retrieval (CLIR). Though attempts are made to improve CLIR, current methods still leave users grappling with issues such as misplaced named entities and lost cultural context when querying in non-native languages. While some advances have been made using Neural Machine Translation models and cross-lingual representation, these are not without limitations. Enter the paradigm shift brought about by Large Language Models (LLMs), which have transformed search engines from simple retrievers to generators of contextually relevant information. This paper introduces the Multilingual Information Model for Intelligent Retrieval (MIMIR). Built on the power of LLMs, MIMIR directly responds in the language of the user's query, reducing the need for post-search translations. Our model's architecture encompasses a dual-module system: a retriever for searching multilingual documents and a responder for crafting answers in the user's desired language. Through a unique unified training framework, with the retriever serving as a reward model supervising the responder, and in turn, the responder producing synthetic data to refine the retriever's proficiency, MIMIR's retriever and responder iteratively enhance each other. Performance evaluations via CLEF and MKQA benchmarks reveal MIMIR's superiority over existing models, effectively addressing traditional CLIR challenges.

While exogenous variables have a major impact on performance improvement in time series analysis, interseries correlation and time dependence among them are rarely considered in the present continuous methods. The dynamical systems of multivariate time series could be modeled with complex unknown partial differential equations (PDEs) which play a prominent role in many disciplines of science and engineering. In this article, we propose a continuous-time model for arbitrary-step prediction to learn an unknown PDE system in multivariate time series whose governing equations are parameterized by self-attention and gated recurrent neural networks. The proposed model, exogenous-guided PDE network (EgPDE-Net), takes account of the relationships among the exogenous variables and their effects on the target series. Importantly, the model can be reduced into a regularized ordinary differential equation (ODE) problem with specially designed regularization guidance, which makes the PDE problem tractable to obtain numerical solutions and feasible to predict multiple future values of the target series at arbitrary time points. Extensive experiments demonstrate that our proposed model could achieve competitive accuracy over strong baselines: on average, it outperforms the best baseline by reducing 9.85% on RMSE and 13.98% on MAE for arbitrary-step prediction.

Stacking pseudoinverse learner (SP) is an ensemble learning technology, and its generalization performance greatly affects the effect of image classification. Currently, most SPs randomly initialize the input weight matrix in a random subspace without limiting the random initial values, resulting in unstable training results and a decrease in generalization performance; in addition, training all samples at once may cause the classifier redundant and also affect the generalization performance of the model. To efficiently address the above issues, we propose a new framework called progressive stacking pseudoinverse learner (PSP), which aims to enhance the generalization performance of SP via active learning (AL) in random subspaces. Specifically, on the one hand, a random feature SP (RFSP) model is proposed, which constrains the random subspace by initializing the input weight matrix into different random specific distributions to improve the generalization performance of SP. On the other hand, an AL progressive (ALP) model based on RFSP is proposed. By iteratively selecting useful samples to optimize the classification results, the training sample information is effectively used to progressively enhance the generalization performance of the model. Experimental results on three public datasets show that our proposed PSP algorithm achieves better performance in accuracy, precision, recall, and$F1$score, and the results are competitive with state-of-the-art methods.

In this article, a decentralized optimal fault tolerant tracking control (DOFTTC) approach is proposed for unknown nonlinear interconnected systems (NISs) with partial loss of actuator effectiveness (PLOAE) through a two-level local observer structure. To begin with, the upper boundedness assumption of the interconnected term in traditional decentralized control methods is relaxed by replacing the actual states of the coupled subsystems with their desired ones. At the first level, a local neural network observer (LNNO) is established for the fault-free subsystem to get the local control coefficient matrix. At the second level, another LNNO is designed based on the LNNO developed at the first level to estimate the effectiveness factor (EF) for the faulty subsystem with PLOAE fault, thereby the safety and reliability of the subsystem are ensured. To achieve the trajectory tracking control, an augmented system is established by combining the tracking error dynamics and reference trajectory dynamics. To obtain the decentralized optimal tracking control (DOTC), an improved local cost function is designed for each subsystem, and then, a local adaptive critic mechanism with cooperative adaptive tuning laws is constructed to solve the local Hamilton–Jacobi–Bellman equation. Hereafter, the DOFTTC can be derived with the assistance of DOTC and the EF estimation. Besides, the closed-loop NIS with asymptotic stability is analyzed via the Lyapunov stability theorem. Simulation results further demonstrate the effectiveness and reliability of the developed DOFTTC approach.

In this article, a real-time online off-policy reinforcement learning (RL) method is developed for the optimal control problem of unknown continuous-time nonlinear systems. First, by applying the temporal difference technique to the iterative procedure of off-policy RL, the iterative value function and the iterative policy input can be learned in real-time online. It is proven that the fitting error of neural network (NN) weights is exponentially convergent in each iteration. Second, a model-free Hamilton–Jacobi–Bellman equation (MF-HJBE) is deduced by taking the limit of the iterative procedure of off-policy RL. In this manner, it not only eliminates system dynamics in the classical HJBE, but also vanishes the iteration index. By applying temporal difference to the MF-HJBE, a real-time online tuning rule is designed to learn the optimal value function and the optimal policy input. It is proven that the fitting error of NN weights caused by the real-time online tuning rule is exponentially convergent. Note that the two online tuning rules, the iterative one and the real-time one, use only current and previous state data extracted from system trajectories. Meanwhile, it is proven using the Lyapunov’s direct method that the system solution is uniformly ultimately bounded. Finally, simulation results demonstrate the validity of the proffered method.

A critical challenge for open-domain dialogue agents is to generate persona-relevant and consistent responses. Due to the nature of persona sparsity in conversation scenarios, previous persona-based dialogue agents trained with Maximum Likelihood Estimation tend to overlook the given personas and generate responses irrelevant or inconsistent with personas. To address this problem, we propose a two-stage coarse-to-fine persona-aware training framework to improve the persona consistency of a dialogue agent progressively. Specifically, our framework first trains the dialogue agent to answer the constructed persona-aware questions, making it highly sensitive to the personas to generate persona-relevant responses. Then the dialogue agent is further trained with a contrastive learning paradigm by explicitly perceiving the difference between the consistent and the generated inconsistent responses, forcing it to pay more attention to the key persona information to generate consistent responses. By applying our proposed training framework to several representative baseline models, experimental results show significant boosts on both automatic and human evaluation metrics, especially the consistency of generated responses.

Recent studies have shown the effectiveness of retrieval augmentation in many generative NLP tasks.These retrieval-augmented methods allow models to explicitly acquire prior external knowledge in a non-parametric manner and regard the retrieved reference instances as cues to augment text generation.These methods use similarity-based retrieval, which is based on a simple hypothesis: the more the retrieved demonstration resembles the original input, the more likely the demonstration label resembles the input label.However, due to the complexity of event labels and sparsity of event arguments, this hypothesis does not always hold in document-level EAE.This raises an interesting question: How do we design the retrieval strategy for document-level EAE?We investigate various retrieval settings from the input and label distribution views in this paper.We further augment document-level EAE with pseudo demonstrations sampled from event semantic regions that can cover adequate alternatives in the same context and event schema.Through extensive experiments on RAMS and WikiEvents, we demonstrate the validity of our newly introduced retrieval-augmented methods and analyze why they work.

Large Language Models have shown great capability to comprehend natural language and provide reasonable responses. However, previous researches have shown weak performance of these models on low-resource (long-tail) languages. It remains to be a problem to mitigate the performance gap between long-tail languages and rich-resource ones, which is referred to as long-tail language representation collapsing. Though some previous works can generate pseudo-parallel corpora with the auto-regressive generation, this generation progress is time-consuming and remains low quality, particularly for long-tail languages. In this paper, we propose a (X) Cross-lingual Bootstrapped Unsupervised Fine-tuning Framework (X-BUFF) to mitigate long-tail language representation collapsing. X-BUFF iteratively updates cross-lingual PLMs in a curriculum way. In each iteration of X-BUFF, we (1) select sentences with complementary semantics from monolingual corpora in long-tail languages. (2) match these selected sentences with semantic equivalent sentences in many other languages to create parallel sentence pairs, which we then merge with previous sentence pairs to build a larger and more difficult bootstrapped parallel queue. (3) fine-tune the PLMs with the bootstrapped parallel queue. Extensive experiments show that X-BUFF can mitigate the long-tail language representation collapsing problem in cross-lingual PLMs and achieve significant improvements over the previous baselines on several cross-lingual evaluation benchmarks.

Federated Learning (FL), as an emerging distributed learning paradigm to protect data privacy, facilitates model training collaborations among entities. However, performance of FL model in such a promising paradigm would drop dramatically without adequate participating entities' data. To maintain sustainability and efficiency of FL system, it is inevitable to design a reasonable and fair contribution estimation method. Despite of success, existing studies of contribution estimation do not adequately balance efficiency and fairness. To overcome such a limitation, we propose two plug-and-play methods, named as vector projection method (VPM) and income exclusion method (IEM), to estimate the contribution of each entity in FL. As standalone contribution measurement modules, they can be deployed to almost any FL server flexibly. We conduct extensive experiments on widely-used Fashion-MNIST and CIFAR-10 datasets. The experimental results demonstrate the promising effectiveness of contribution estimation, and show the convergence acceleration under contribution-weighted aggregation mechanism.

Expressing universal semantics common to all languages is helpful to understand the meanings of complex and culture-specific sentences. The research theme underlying this scenario focuses on learning universal representations across languages with the usage of massive parallel corpora. However, due to the sparsity and scarcity of parallel data, there is still a big challenge in learning authentic ``universals'' for any two languages. In this paper, we propose Emma-X: an EM-like Multilingual pre-training Algorithm, to learn Cross-lingual universals with the aid of excessive multilingual non-parallel data. Emma-X unifies the cross-lingual representation learning task and an extra semantic relation prediction task within an EM framework. Both the extra semantic classifier and the cross-lingual sentence encoder approximate the semantic relation of two sentences, and supervise each other until convergence. To evaluate Emma-X, we conduct experiments on xrete, a newly introduced benchmark containing 12 widely studied cross-lingual tasks that fully depend on sentence-level representations. Results reveal that Emma-X achieves state-of-the-art performance. Further geometric analysis of the built representation space with three requirements demonstrates the superiority of Emma-X over advanced models.

For event-triggered optimal control problems of general nonlinear systems, it is very difficult to obtain the optimal analytical solution. In this paper, an adaptive near optimal output feedback control method is presented for discrete-time (DT) nonlinear systems. The event-triggered mechanism is introduced to significantly reduce the execution costs through aperiodic control updating intervals without affecting system responses. Furthermore, an integral term is presented in the performance index function to achieve the optimal constrained control. Consequently, iterative dual heuristic dynamic programming algorithm (DHP) is adopted to learn the optimal control law and the costate function. Finally, two examples are provided to illustrate the effectiveness of the proposed approach.

Transforming the large amounts of unstructured text on the Internet into structured event knowledge is a critical, yet unsolved goal of NLP, especially when addressing document-level text. Existing methods struggle in Document-level Event Extraction (DEE) due to its two intrinsic challenges: (a) Nested arguments, which means one argument is the sub-string of another one. (b) Multiple events, which indicates we should identify multiple events and assemble the arguments for them. In this paper, we propose a role-interactive multi-event head attention network (CLIO) to solve these two challenges jointly. The key idea is to map different events to multiple subspaces (i.e. multi-event head). In each event subspace, we draw the semantic representation of each role closer to its corresponding arguments, then we determine whether the current event exists. To further optimize event representation, we propose an event representation enhancing strategy to regularize pre-trained embedding space to be more isotropic. Our experiments on two widely used DEE datasets show that CLIO achieves consistent improvements over previous methods.

Story Ending Generation (SEG) is a challenging task in natural language generation. Recently, methods based on Pre-trained Language Models (PLM) have achieved great prosperity, which can produce fluent and coherent story endings. However, the pre-training objective of PLM-based methods is unable to model the consistency between story context and ending. The goal of this paper is to adopt contrastive learning to generate endings more consistent with story context, while there are two main challenges in contrastive learning of SEG. First is the negative sampling of wrong endings inconsistent with story contexts. The second challenge is the adaptation of contrastive learning for SEG. To address these two issues, we propose a novel Contrastive Learning framework for Story Ending Generation (CLseg)†, which has two steps: multi-aspect sampling and story-specific contrastive learning. Particularly, for the first issue, we utilize novel multi-aspect sampling mechanisms to obtain wrong endings considering the consistency of order, causality, and sentiment. To solve the second issue, we well-design a story-specific contrastive training strategy that is adapted for SEG. Experiments show that CLseg outperforms baselines and can produce story endings with stronger consistency and rationality.

Distant supervision (DS) is widely used in relation extraction which can automatically generate large-scale training data by aligning a knowledge base with an unlabeled corpus. However, it suffers from the label noise problem. In this paper, we propose a novel explicit training loss based DS relation extraction de-noising method to generate a cleansed dataset. Specifically, we firstly design a noise detector to select noisy bags using average loss during cyclical training, which is based on the idea that the noisy samples will have different loss variation process during training compared to the clean samples. Then we propose a label corrector to generate right labels for the selected samples, which can keep more useful information to a great extent. Finally, the experimental results show that our de-noising method is robust that can detect the label noise quite well on both large scale and small scale dataset and the generated cleansed dataset significantly improves the performance of previous distant supervision models.

The approximate optimal control design for continuous-time nonlinear systems with partially unknown dynamics is studied in this paper. Based on the state-dependent coefficient parameterization, the dynamics of nonlinear systems are represented in a resemble linear manner. In this case, the Hamilton-Jacobi-Bellman equation can be recast in the form of the state-dependent Riccati equation (SDRE). Based on the integral reinforcement learning, an online policy iteration algorithm is developed to iteratively solve the SDRE using status and control data. In addition, it is proved that the proposed algorithm is equivalent to the traditional iterative solution of SDRE. A suboptimal control policy can be attained under proper conditions. The iterative feedback control policy has the ability to stabilize closed-loop system. The effectiveness of the presented algorithm is validated by simulation results.

By training different models and averaging their predictions, the performance of the machine-learning algorithm can be improved. The performance optimization of multiple models is supposed to generalize further data well. This requires the knowledge transfer of generalization information between models. In this article, a multiple kernel mutual learning method based on transfer learning of combined mid-level features is proposed for hyperspectral classification. Three-layer homogenous superpixels are computed on the image formed by PCA, which is used for computing mid-level features. The three mid-level features include: 1) the sparse reconstructed feature; 2) combined mean feature; and 3) uniqueness. The sparse reconstruction feature is obtained by a joint sparse representation model under the constraint of three-scale superpixels' boundaries and regions. The combined mean features are computed with average values of spectra in multilayer superpixels, and the uniqueness is obtained by the superposed manifold ranking values of multilayer superpixels. Next, three kernels of samples in different feature spaces are computed for mutual learning by minimizing the divergence. Then, a combined kernel is constructed to optimize the sample distance measurement and applied by employing SVM training to build classifiers. Experiments are performed on real hyperspectral datasets, and the corresponding results demonstrated that the proposed method can perform significantly better than several state-of-the-art competitive algorithms based on MKL and deep learning.

Pseudo-inverse learners (PILs) are a kind of feedforward neural network trained with the pseudoinverse learning algorithm, which can be traced back to 1995 originally. PIL is an approach for nongradient descent learning, and its main advantage is the lower computational cost and fast learning procedure, which is especially relevant in the edge computing research field. However, PIL is mostly applied to a deterministic learning problem, while in the real world, the greatest case that is of concern is the uncertainty learning problem. In this work, under the framework of the synergetic learning system (SLS), we introduce an approximated synergetic learning scheme, which can transform uncertainty learning into deterministic learning. We call this new learning framework the Bayesian PIL, and the advantages are also demonstrated in this work.

This paper presents an ensemble classification model with unsupervised feature learning for driving stress recognition under real-world driving conditions. The driving stress is detected using drivers’ different physiological signals, specifically the electromyogram, electrocardiogram, galvanic skin response, heart rate and respiration. The proposed model consists of two modules: 1) a multilayer representation learning module using autoencoder as its building block. The autoencoders are trained with a quasi-automated, non-gradient descent based unsupervised learning algorithm; 2) an ensemble classification module under the AdaBoost framework. The proposed model is completely data driven, does not require additional feature extraction and feature selection process, and can perform in an end-to-end way in which it takes the physiological signal as the input instead of the handcrafted features. Experimental results show that our proposed model can effectively recognize the driving stress with fewer physiological sensors compared with most of state of the art methods. Experiments also demonstrate that the proposed model can simplify the model structure tuning and improve the learning efficiency compared with the baseline deep learning model.

The traditional gradient descent based optimization algorithms for neural network are subjected too many vulnerabilities, such as slow convergent rate, gradient vanishing and falling into local minima. Therefore, the alternative non-gradient descent learning algorithm was proposed and prevalently applied in kinds of domains, such as pseudoinverse learning algorithm (PIL). However, when a special variant of the PIL, taking the random configuration of weight parameters, is adopted, the generalization ability needs further improvement although it has excellent training efficiency. Thus, on consideration of integrating the idea of ensemble learning, we proposes two methods to enhance basic PIL. One method is equivalent to an additive model, which can raise the network's performance by introducing boosting mechanism, and the other is to adopt a recursive way to rectify the hidden layer output of the neural network, then the relative better model is used in the subsequent prediction. Comprehensive evaluating experiments are conducted on several datasets, and the experimental results illustrate that the our proposed methods are effective on the classification accuracy.

The embedding generated by pre-trained models has attracted the attention of many scholars in the past few years. Most of the context-sensitive embeddings have confirmed the positive impact on some basic tasks of classification, which have only a few types. In this paper, we make an empirical comparison of different pre-trained embeddings on the task of ultra-fine entity typing which has more than 10k types. We apply 7 kinds of pre-trained embedding to the typing model to prove whether the pre-trained embedding has a positive effect. The results indicate that almost all context-sensitive pre-trained embeddings improve the performance of models using Glove. The pre-trained embedding generated by BERT achieves the best performance in the Ultra-Fine dataset and OntoNotes dataset, which shows BERT has better capability to extract finer-grained information than other pre-trained models.

The aircraft arrival scheduling (AAS) problem is an important issue in airport management that requires well-designed scheduling approaches to help improve the operating efficiency of airports. However, most of the existing approaches are based on a single runway while the multi-runway situation is more common and more complex in reality. In order to solve the multi-runway AAS (MRAAS) problem, this paper proposes a receding horizon control (RHC) based two-level ant colony system (ACS) approach. The RHC divides the problem with receding time windows. Therefore, the number of aircraft considered in each RHC stage is reduced, so that the ACS algorithm can endure less computational burden and search for good scheduling sufficiently. Moreover, in each RHC stage to schedule the aircraft, a two-level scheduling strategy is proposed to assist the ACS to deal with the multi-runway difficulty. During the first level scheduling, the ACS algorithm schedules aircraft based on a single runway. Then in the second level scheduling, we assign the aircraft to real multi-runways based on both the obtained sequence of the first level scheduling and the real occupancy condition of each runway. We have conducted experiments based on case study and compared the results with first come first serve (FCFS) approach, showing the feasibility as well as the higher performance of the proposed RHC-ACS-MRAAS approach.

In Bayesian theory, the maximum posterior estimator uses prior information to estimate the noise in the machine learning model by adding the regularization term. The regularization terms L1and L2correspond to Laplacian prior and Guassian prior, respectively. In existing deep learning models, in order to use the gradient descent optimization algorithm and achieve good results, most models take L2regularization as the regularization term of the network model to fit the complex Guassian noise. However in practice, the Laplace noise and the Guassian noise are both considered as data noise. For multi-layer perceptrons, the difficulty caused by adding L1and L2into the optimization function of the network is solved by proposing an ensemble model for error modeling through adopting the divide and conquer strategy. First, several base learners are trained to fit different noise distributions of data, then the final results can be obtained by taking the results of each base leaner as new data to train a meta leaner, and get the final results. Among them, coordinate regression method is used to solve L1loss, while the pseudo-inverse learning algorithm is employed to solve L2loss. Both methods are nongradient optimization algorithms. The comparison results of the model on several data sets show that the proposed ensemble model achieves better performance.

Pulsars search has always been one of the most concerned problem in the field of astronomy. Nowadays, with the development of astronomical instruments and observation technology, the amount of data is getting bigger and bigger. Radio pulsar surveys have generated and will generate vast amounts of data. To handle big data, developing new technologies and frameworks to efficiently and accurately analyze these data become increasing urgent. The number of positive and negative samples in pulsar candidate data set is very unbalanced, if we only use these a few positive samples to train a deep neural network (DNN), the trained DNN is prone because of the problem of overfitting and will affect the generalization ability. Motivated by the mixtures of experts network architecture, we proposed a hierarchical model for pulsar candidate selection which assembles a set of trained base classifiers. Moreover, training a neural network always takes a lot of time because of using gradient descent (GD) based algorithm. In this work, we utilize the pseudoinverse learning algorithm instead of GD based algorithm to train proposed model. With the designed network architecture and adopted training algorithm, our model has the advantages not only with high steady-state precision but also good generalization performance.

Sparse autoencoder is one approach to automatically learn features from unlabeled data and received significant attention during the development of deep neural networks. However, the learning algorithm of sparse autoencoder suffers from slow learning speed because of gradient descent based algorithms have many drawbacks. In this paper, a fast learning algorithm for sparse autoenceder is proposed which based on pseudoinverse learning algorithm (PIL). The proposed method calculates encoder weight matrix by truncating the pseudoinverse matrix of input data. The pseudoinverse truncation matrix is used as the weights of encoder, and then the input data is mapped to the hidden layer space through the biased ReLU activation function. The decoder weights are also can computed by the PIL. Unlike the gradient descent based algorithm, the proposed method does not require a time-consuming iterative optimization process and select many user-dependent parameters such as learning rate or momentum constant too. The experimental results indicate the superiority of proposed method which is very efficient and also can learned the sparsity of samples.

This paper proposes a novel vision-based object detection method in autonomous driving, which introduces the temporal information into the deep learning-based detection method for moving object detection. Vision-based object detection is a critical technology for autonomous driving. The objects in the real world such as driving cars, don't have great changes in their positions and velocities. So the position change of objects between two consecutive frames is not large. This is usually ignored by traditional works, which usually use object detection methods on still-images to detect moving objects. Considering the relationship among consecutive frames (temporal information), we present a robust and real-time tracking method following image detection to refine the object detection results. Based on the three key attributes (distances, sizes and positions), the tracking method aims to build the association between the detected objects on the current frame and those in previous frames. The proposed object detection with temporal information dramatically improves the performance of existing object detection algorithms based on stillimage. With the proposed method, we won the champion in the preceding vehicle detection task in 2017 intelligent vehicle future challenge(2017 IVFC)1.

Predicting the click-through rates (CTR) of items for a specific user, is of great importance to recommender systems. In most cases, recommendation items are presented by an ordered list. Most previous studies, however, focus on the CTR prediction for a user-item pair, and ignore the interactions among the items in the list. Through our study, with different preceding items, the CTRs of the same active item become significantly different. It implies if this kind of dependency is not considered, the CTR prediction will not be accurate enough in practice. To solve this limitation, we investigate the listwise user-specific CTR prediction for items in the whole recommendation list, rather than for a single item. Specifically, a joint prediction framework based on matrix factorization (MF) and recurrent neural network (RNN) is proposed, which exploits the features of item-item interactions for a user, as well as the features of a single user-item pair. In the process of list generation, our model can improve the CTR prediction for the trailing items, with considering the determined preceding items. We evaluate the proposed approach in a real news recommendation scenario from industry. Compared with a competitive baseline, the performance has been improved by 0.64% in the AUC metric from the magnitude of 0.7357, and by 4.29% in the RIG metric from the magnitude of 0.1095. The improvement on specific position is up to 1.21% and 9.44% in the metric of AUC and RIG, respectively.

Autoencoder is one approach to automatically learn features from unlabeled data and received significant attention during the development of deep neural networks. However, the learning algorithm of autoencoder suffers from slow learning speed because of gradient descent based algorithms have many drawbacks. Pseudoinverse learning algorithm is a fast and fully automated method to train autoencoders. While when the dimension of data is far less than the number of data, the pseudoinverse learning can only obtain the optimal initial value of the autoencoder network and need further learning to achieve satisfactory results. In order to overcome the shortcomings mentioned above, we present a broad learning strategy to transform the input space to the high dimensional space through receptive function in this paper. The transformed data can be more suitable to pseudoinverse learning algorithm which can be obtained the accurate results of autoencoder efficiently. The experimental results show that the proposed method can achieve a comprehensively better performance in terms of training autoencoder efficiency and accuracy.

On-line news reading has become the most popular way for user to obtain real-time information. With the millions of news, it is a key challenge to help user find the articles that are interesting to read. Although great achievements have been made, there is little work to focus on combing news language with external knowledge graphs and expanding news text from a word-level. Taking this issue into consideration, we introduce a novel self-attention based mechanism in news recommendation. The key component of our model is multiple self-attention modules: the word-level attention, which takes tags of news, entities in external knowledge graph and entities' contexts as the input to calculate the semantic-level and knowledge-level representation of the news; the item-level attention module, which used to fuse the two-level representation into the same low-dimension and get a overall embedding of user history behavior sequence. Specially, in order to deal with the diversity of user preferences, we use another self-attention module dynamically aggregate user click history and select candidate news. And finally, a multi-head attention module is used to connect history and candidate news and then calculate the click-through-rate(CTR) via a fully connected layer. Through amount of experiments on a real-world online news website, we demonstrate that our model outperforms better results than previous start-of-art recommendation models.

Most color images obtained from the real world usually contain complex areas, such as nature scene images, remote sensing images, and medical images. All these type of images are very difficult to be separated accurately and automatically for complex color and structures included. In this paper, we focus on detecting hybrid cues of color image to segment complex scene in a bottom-up framework. The main idea of the proposed segmentation method is based on a two-step procedure: 1) a reasonable superpixels computing method is conducted and 2) a Mumford-Shah (M-S) optimal merging model is proposed for presegment suerpixels. First, a set of seed pixels is positioned at the lowest texture energy map computed from structure tensor diffusion features. Next, we implement a growing procedure to extract superpixels from selected seed pixels with color and texture cues. After that, a color-texture histograms feature is defined to measure similarity between regions, and an M-S optimal merging process is executed by comparing the similarity of adjacent regions with standard deviation constraints to get final segmentation. Extensive experiments are conducted on the Berkeley segmentation database, some remote sensing images, and medical images. The results of experiments have verified that the segmentation effectiveness of the proposed method in segmenting complex scenes and indicated that it is more robust and accurate than conventional methods.

In this paper, we present a novel method on image calibration, utilizing Total Least Square (TLS) method and Feedforward Neural Network, to solve the aberration problem of LAMOST two-dimensional astronomical spectral images. In our method, training sample set is generated with domain knowledge, from which a number of discrete points are are extracted from spectral images with fiber tracing method, and output vectors are formed by the corresponding calibrated points, obtained by utilizing the TLS method. The Feed-forward Neural Network is trained to obtain the transformation matrix, casting about for the matching relationship between the input and output sets. We also perform comparative experiments on fiber tracing and spectrum extraction results between calibrated spectral images and uncalibrated spectral images, the results show an advantage of higher accuracy and precision by our proposed method.

Accurately predicting driving service orders in different regions is an essential task for service companies, in order to improve the service quality. In this paper, a specific ensemble multi-view prediction framework is proposed to address this task. It ensembles several different multi-view-based models with a weighted linear combination. Specifically, we have designed three specific multi-view-based models, which of them contains two types of views. In first view, a spatio-temporal prediction model (ST-Model) is employed to construct the features on historical orders. In second view, a specific CRF is designed to joint adjacent regions to collaborate predict future orders in these regions. The two views are learned and inferred simultaneously. Extensive evaluations on real order data in Beijing show that the proposed framework outperforms all baselines and participated multi-view-based models significantly in terms of mean absolute error (MAE) and root mean square error (RMSE).

Deep multi-layer neural networks are generally trained using variants of the gradient descent based algorithm. However, this kind of algorithms usually encounter a series of shortcomings, such as low training efficiency, local minimum, difficult control parameter tuning, and gradient vanishing or exploding. Besides, for a specific application, how to design the structure of the network, that is, how many neurons in each hidden layer and how many hidden layers is needed, is also a very tricky problem and is usually solved by trial and error in practice. To overcome the shortcomings mentioned above, we present a fast and fully automated method to train stacked autoencoders based deep neural networks in this paper. The proposed method trains the stacked autoencoders adopting the pseudoinverse learning algorithm with the low rank approximation. The entire training process neither need to set the learning control parameters, nor specify the number of hidden layers and the number of neurons in each hidden layer. The experimental results show that the proposed method can achieve a comprehensively better performance in terms of training efficiency and accuracy.

Most news recommendation methods focus on using textual information of news to solve data sparseness problem of collaborative filtering. While if the text is not informative enough, these methods can't work well. A collaborative model combining matrix factorization, topic analysis and knowledge graph representation is proposed by introducing the knowledge from external knowledge base to alleviate the deficiency of the text. The experiment conducted on real life news dataset shows that the joint model outperforms the state-of-the-art method by 14% in Recall@200 metric, and improves the recommendation performance on sparse items by 20%.

Image annotation is a hard multi-label learning problem which aims at automatically tagging each input image with relevant keywords reflecting its semantic concepts. Recently, several late fusion methods were proposed to improve the accuracy of image annotation. But these late fusion methods need normalization of confidence score vectors of independent models corresponding to distinct representations. Choosing a good normalization function is tricky and difficult. In this paper, we propose a new method of late fusion for image annotation based on rank minimization. The proposed method avoids normalization by transforming confidence score vectors into pairwise relationship matrices. And an optimal matrix is obtained by solving a minimization optimization problem. With the optimal matrix, a fused confidence score vector can be recovered, which gives the final prediction of tags. Experiments on standard Corel5K and our Campus-Indoor dataset confirm the effectiveness of our late fusion method for image annotation.

Deep learning scheme has received significant attention during these years, particularly as a way of building hierarchical representations from unlabeled data for a variety of signal and information processing tasks. However, deep neural networks suffer from slow learning speed since most used training algorithms are based on variations of the gradient descent algorithms which require iterative optimization and thus are time-consuming. In addition, a series of control parameters need to be specified empirically which lacks of the theoretical guidance, and current learning algorithms for deep networks are not very suitable to incremental learning scenario. To address these issues, we propose a fast learning scheme in this paper. The basic idea of our approach is to pre-train basic units such as auto-encoders of the deep architecture in an analytical way without any iterative optimization procedure. This scheme is also extended to an incremental learning version. The experimental result shows the superiority of our approach over the state-of-the-art gradient descent based algorithms. To demonstrate the impact of our algorithm on complicated real world applications, we give an example of its performance in astronomical spectra pattern recognition.

Kernel independent component analysis (KICA) has an important application in blind source separation, in which how to select the optimal kernel, including the kernel functional form and its parameters, is the key issue for obtaining the optimal performance. In practices, a single kernel is usually chosen as the kernel model of KICA in light of experience. However, selecting a suitable kernel model is a more difficult problem if one has not sufficient experience. To deal with this problem, an evolution based method to select the kernel model of KICA is proposed in this paper. There are two main features of the proposed method: one is that using a multiple kernel model, a convex combination of several single kernels, replaces the single kernel model; another is that particle swarm optimization (PSO) algorithm is utilized to find the combination weights of the composite kernel. Experiments conducted on separating one-dimensional mixed signals, nature images, and spectroscopic CCD images showed that using multiple kernels model with PSO kernel selection algorithm can enhance the performance of KICA.

In this paper, a sub-dictionary based sparse coding method is proposed for image representation. The novel sparse coding method substitutes a new regularization item for L1-norm in the sparse representation model. The proposed sparse coding method involves a series of sub-dictionaries. Each sub-dictionary contains all the training samples except for those from one particular category. For the test sample to be represented, all the sub-dictionaries should linearly represent it apart from the one that does not contain samples from that label, and this sub-dictionary is called irrelevant sub-dictionary. This new regularization item restricts the sparsity of each sub-dictionary's residual, and this restriction is helpful for classification. The experimental results demonstrate that the proposed method is superior to the previous related sparse representation based classification.

In this paper, a novel image stitching method is proposed, which utilizes scale-invariant feature transform (SIFT) feature and single-hidden layer feedforward neural network (SLFN) to get higher precision of parameter estimation. In this method, features are extracted from the image sets by the SIFT descriptor and form into the input vector of the SLFN. The output of the SLFN is those translation, rotation and scaling parameters with respect to reference and registered image sets. We also apply a fast learning scheme, called pseudoinverse learning, to train SLFN to get higher training efficiency. Comparative experiments are performed between our proposed method and the traditional random sample consensus (RANSAC) based method. The results show that our method has the advantage not only at accuracy but also remarkably at fast speed.

The success of deep learning proves that deep models are able to achieve much better performance than shallow models in representation learning. However, deep neural networks with auto-encoder stacked structure suffer from low learning efficiency since common used training algorithms are variations of iterative algorithms based on the time-consuming gradient descent, especially when the network structure is complicated. To deal with this complicated network structure problem, we employ a “divide and conquer” strategy to design a locally connected network structure to decrease the network complexity. The basic idea of our approach is to force the basic units of the deep architecture, e.g., auto-encoders, to extract local features in an analytical way without iterative optimization and assemble these local features into a unified feature. We apply this method to process astronomical spectral data to illustrate the superiority of our approach over other baseline algorithms. Furthermore, we investigate visual interpretations of high level features and the model to demonstrate what exactly the model learn from the data.

Software aging, also called smooth degradation or chronics, has been observed in a long running software application, accompanied by performance degradation, hang/crash failures or both. The key for software aging problem is how to fast and accurately detect software aging occurrence, which is a hard work due to the long delay before aging appearance. In this paper, two problems about software aging prediction are solved, which are how to accurately find proper running software system variables to represent system state and how to predict software aging state in a running software system with a minor error rate. Firstly, the authors use proposed stepwise forward selection algorithm and stepwise backward selection algorithm to find a proper subset of variables set. Secondly, a classification algorithm is used to model software aging process. Lastly, t-test with k-fold cross validation is used to compare performance of two classification algorithms. In the experiments, the authors find that their proposed method is an efficient way to forecast software aging problems in advance.

Affinity Propagation (AP) algorithm is a useful clustering technique with a lot of noteworthy advantages. It has been successfully applied in many applications. However, this algorithm does not scale for large scale data sets because it requires quadratic computational time and memory usage in the problem size. In this paper, we concentrate on the needs of big data analytics and propose an effective and efficient scheme to decrease the computational complexity and memory usage of AP algorithm. The basic idea of our approach is embedding data points in distance-preserving binary codes and then decomposing the original big data set into a series of small subsets by aggregating similar data points according to their binary codes. The experimental results and the real world astronomical spectral data application demonstrate the effectiveness of our approach quantitatively and visually.

This paper focuses on the research issue of supervised topic models. Traditionally, an unsupervised topic model is typically supervised by incorporating a supervised linear operator. Although this kind of methods has successfully achieved supervised topic representations, as well as tractable computational complexity, the main limitation lies in that it assumes the topic data is linearly distributed. Therefore, when the practical data does not follow the linear property, the model cannot perform well. To solve this problem, a non-linear supervised topic model is proposed in this paper. Specifically, the mixture of experts (ME), as a kind of "divide and conquer", is utilized to deal with the regression of non-linear of topic data. We integrate the mixture of experts and unsupervised latent Dirichlet allocation (LDA) in a Bayesian manner. The proposed model can also reduce overfitting problem of ME with inputting high-dimensional data. An elegant learning algorithm is derived based on variational expectation maximization algorithm. Experimental results show that the proposed model has better predictive performances compared with usupervised topic models, and some state-of-the-art supervised topic regression models (including sLDA model and MedLDA model), on two textual datasets and one image dataset.

Deconvolution is known as an ill-posed problem. In order to solve such a problem, a regularization method is needed to constrain the solution space and find a plausible and stable solution. In practice, it is very computation intensive when using cross-validation method to select the regularization parameter. In this paper, we present an adaptive regularization method to find the optimal regularization parameter value and represent the trade-off between model fitness of the data and the smoothness of the extracted signal. Spectral signal extraction experimental results demonstrate that the time complexity the proposed method is much lower than the one without adaptive regularization and is convenient for users also. And quantitative performance analysis show that the proposed intelligent approach performs better than that of current deconvolution extraction method and other extraction method used in the Large Area Multi-Objects Fiber Spectroscopy Telescope spectral signal processing pipeline.

In this paper, we propose the new type of fuzzy number, intuitionistic trapezoidal fuzzy number(ITFN) and interval-valued intuitionistic trapezoidal fuzzy number(IVITFN), study some properties of them, and investigate some algebraic operations between two IVITFNs. We also present the sorting criteria for IVITFN based on its definition. Finally, we use our proposed IVITFN and algebraic operation to assess production line, the experiment results show that our proposed method is effective and available.

In this paper, we investigate the efficiency and scalability of Gaussian mixture model based learning algorithm for the detection of Near-Earth objects in large scale astronomy image data. We propose an effective scheme to reduce the computational complexity of current learning algorithm, this is achieved by adopting the perceptual image hashing method. Our proposed scheme is validated on raw astronomy image data. The experiment results illustrate that both efficiency and scalability are improved significantly in astronomical scenario and other scenario.

A frequent behavior of internet users is to compare among various comparable entities for decision making. As an instance, a user may compare among iPhone 5, Lumia 920 etc. products before deciding which cellphone to buy. However, it is a challenging problem to know what entities are generally comparable from the users' viewpoints in the open domain Web. In this paper, we propose a novel solution, which is known as Comparable Entity Graph Mining (CEGM), to learn an open-domain comparable entity graph from the user search queries. CEGM firstly mine seed comparable entity pairs from user search queries automatically using predefined query patterns. Next, it discovers more entity pairs with a confidence classifier in a bootstrapping fashion. Newly discovered entity pairs are organized into an open-domain comparable entity graph. Based on our empirical study over 1 billion queries of a commercial search engine, we build a comparable entity graph which covers 73.4% queries in the top 50 million unique queries of a commercial search engine. Through manual labeling in sampled sub-graphs, the average precision of comparable entities is 89.4%. As applications of the learned entity graph, the entity recommendation in Web search is empirically studied.

Texture feature extraction plays an important role in texture image classification. In this paper, we have proposed a texture feature extraction method by utilizing the Short-time Fourier Transform to provide local image information, and for the global geometric correspondence we have proposed to use Spatial Pyramid Matching in frequency domain named as Short-time Fourier Transform with Spatial Pyramid Matching (STFT-SPM). The experiments are conducted on standard benchmark datasets for texture classification like Brodatz and KTH-TIPS2-a, shows that STFT-SPM can achieve significant improvement compared to the Local Phase Quantization, Weber local Descriptor and local Binary Pattern methods.

Recently sparse coding with spatial pyramid matching method has shown its excellent performance in image classification. Inspired by this technique, we present an image classification approach by learning the optimal Multiple Pooling Combination strategy based on Non-Negative Sparse Coding (MPC-NNSC) in this paper. First, non-negative sparse coding with three different pooling methods as well as spatial pyramid matching method are utilized to encode local descriptors for image representation, respectively. Then a promising weight learning approach is employed to find a set of optimal weights for best fusing all these pooling methods in different scales. Lastly, support vector machine classifier with linear and histogram intersection kernel is employed for the final classification task. Experiments on two popular benchmark datasets are presented and they demonstrate the better performance of the proposed scheme compared to the state-of-the-art methods.

At present, multiple scattering problems in participating media is still very challenging for real time rendering. Some methods have proposed to describe multiple scattering phenomena, however, there are some restriction conditions such as requiring the medium is static, etc., and rendering speed is not satisfied real-time requirement. In order to speed up the multiple scattering rendering, we propose a GPU based algorithm in this paper. First of all, the media is initialized with a particle system and the property of each particle is defined; secondly, according to the properties of each particle, a method of tracing the particle path, which is generated by uniformly sampling the surrounding particles of one particle, is proposed and this method is used to compute in-scattering radiance for each particle; finally, the total radiance is calculated by summing up contributions of particles along ray paths and the final image is rendered. The experimental results show that the proposed algorithm can achieve the real-time rendering effect.

In Epilepsy EEG signal classification, the main time-frequency features can be extracted by using sparse representation with marching pursuit (MP) algorithm. However, the computational burden is so heavy that it is almost impossible to apply MP to real time signal processing. To reduce complexity of sparse representation, we propose to adopt harmony search method in searching the best atoms. Because harmony search method can find the best atoms in continuous time-frequency dictionary, the performance of epilepsy EEG signal classification is enhanced. The validity of this method is proved by experimental results.

In order to get better semantic annotation performance, block-global features are extracted as low-level visual features for image semantic annotation. Specifically, wellknown global feature extraction method, namely two-dimensional principal component analysis (2DPCA) is applied to extract the image block-global features. Unlike typical image annotation methods which use local features or global features separately, we propose to extract global features from image local regions (block) with the expectation of: a) combining the advantages of local and global features; b) discovering multiple semantic meanings in one image. In the experiment, comparative studies have been done for the performance of block-global feature extraction methods with widely used local feature extraction method such as scale invariant feature transform. The results show that 2DPCA has a significantly better performance than the performance of other methods.

Self-Organizing Maps (SOM) have presented excellent effect in color image segmentation; the scale of SOM will directly affect the accuracy of segmentation results. In this paper, we proposed a novel scale estimated of self-organizing map (SE-SOM) for color image segmentation based on SOM clustering. Different from conventional SOM model, it determines the number of nodes of competition layer by 3-D spatial distribution of pixels in HSV (Hue-Saturation-value) color space. Then sample pixels to train the map topology of the image and segment pixels by computing similarity between their feature vectors with weights of each node. Finally, design a connectivity filter to update labels of image to decrease noise. Statistical information are used to design map scale, which adapted the final SOM scale to the distribution feature of pixels, clustering results more accurate and stable, Experiments results show that the algorithm can produce ideal results with manual segmentation and suitable PNSR values.

In order to improve the classifier performance in semantic image annotation, we propose a novel method which adopts learning vector quantization (LVQ) technique to optimize low level feature data extracted from given image. Some representative vectors are selected with LVQ to train support vector machine (SVM) classifier instead of using all feature data. Performance is compared between the methods with and without feature data optimization when SVM is applied to semantic image annotation. Experiment results show that the proposed method has a better performance than that without using LVQ technique.

In automatic image annotation, it is often extracting low-level visual features from original image for the purpose of mapping to high level image semantic information. In this paper, we propose a novel method which integrates kernel independent component analysis (KICA) and support vector machine (SVM) for analyzing the semantic information of natural images. KICA, which contains a nonlinear kernel mapping component, is adopted to extract low-level features from the original image data. Then these feature vectors are mapped to high-level semantic words using SVM to annotate images with labels in a given semantic label set. Comparative studies have done for the performance of KICA with traditional color histogram and discrete cosine transform features. The experimental results show that the proposed method is capable of extracting the components of images as key features, and with these features to map into semantic categories, higher accuracy is achieved.

In classification of multi-source remote sensing image, it is usually difficult to obtain higher classification accuracy. In the previous work, the modeling technique for the remote sensing image classification based on the minimum description length (MDL) principle with mixture model is analyzed theoretically. In this work, experimental studies are performed for investigating the modeling technique. With intensive experiments and sophisticated analysis, it is found that the developed modeling technique can build a robust classification system, which can avoid classifier over-fitting training data and make the learning process trade-off between bias and variance. Meanwhile, designed mixture model is more efficient to represent real multi-source remote sensing images compared to single model.

The purpose of fusion the multispectral (MS) and panchromatic (PAN) remote sensing images is to obtain high spatial resolution and quality of the PAN image as well as to preserve spectral information of the MS image. The parameter selection of fusion rule will directly affect the fusion result. In this paper, a new fusion method is presented based on multi-objective evolutionary algorithm (called SMS-EMOA). First, the MS image is converted from the RGB color space into the HSI (Hue-Saturation-Intensity) color space. Then, by applying Contourlet transform to the PAN image and the Intensity component of the MS image, the weighted model is used to fuse the sub-images, and the SMS-EMOA is adopted for optimal parameter selection. Finally, a fusion image is obtained by the inverse Contourlet and HSI transform. The experimental results show that the proposed fusion rule optimization method not only can gain the spatial resolution, but also can preserve the spectral information of the original MS image very well.

Shadow detection in high spatial resolution remote sensing image is very critical for locating geographical targets. In this paper, we proposed a new shadow detection method using Affinity Propagation (AP) algorithm in the Hue-Saturation-Intensity (HSI) color space. Because the pixel matrix is a large-scale matrix, if we apply AP algorithm directly on the raw pixel space, it will be computation intensive to calculate the similarity matrix. To solve this problem, we propose to divide the matrix into several blocks and then applying AP to detect shadows in H, S and I components respectively. Then, three detected images are fused to obtain a final shadow detection result. Comparative experiments are performed for K-means and threshold segmentation methods. The experimental results show that higher detection accuracy of the proposed approach is obtained, and it can solve the problems of false dismissals of K-means and threshold segmentation method.

In order to investigate the performance of visual feature extraction method for automatic image annotation, three visual feature extraction methods, namely discrete cosine transform, Gabor transform and discrete wavelet transform, are studied in this paper. These three methods are used to extract low-level visual feature vectors from images in a given database separately, then these feature vectors are mapped to high-level semantic words to annotate images with labels in a given semantic label set. As it is more efficient to depict the visual features of an image by the feature distribution than to resort to image segmentation technology for semantic image blocks, this paper is going to find out which of the three feature extraction methods performs better in image annotation based on the distribution of feature vectors from the image. The performance of three different kinds of feature extraction method is fully analyzed, and it is found that discrete cosine transform method is more suitable for Gaussian mixture model in automatic image annotation.

DNA Microarray technique has produced large amount of gene expression data. To analyze these data, many excellent machine learning techniques have been proposed in recent related work. In this paper, we try to perform the clustering of microarray data by combining the recently proposed Locality Preserving Projection (LPP) method with PCA, i.e. PCA-LPP. The comparison between PCA and PCA-LPP is performed based on two clustering algorithms, K-means and agglomerative hierarchical clustering. As we already known, clustering with the components extracted by PCA instead of the original variables does improve cluster quality. Moreover, our empirical study shows that by using LPP to perform further process the dimensions of components extracted by PCA can be further reduced and the quality of the clusters can be improved greatly meanwhile. Particularly, the first few components obtained by PCA-LPP capture more information of the cluster structure than those of PCA.

In classification of a multispectral remote sensing image, it is usually difficult to obtain higher classification accuracy if we only consider the image's spectral feature or texture feature alone. In this paper, we present a new approach by applying the Ant Colony Optimization (ACO) algorithm to find a multi-feature vector composed of spectral and texture features in order to get a better result in the classification. The experimental results show that ACO algorithm is helpful in subset searching of the features used to classify the multispectral remote sense image. Using the combination of the spectral and texture features obtained by ACO in classification always produces a better accuracy.

Software metrics are collected at various phases of the software development process. These metrics contain the information of the software and can be used to predict software quality in the early stage of software life cycle. Intelligent computing techniques such as data mining can be applied in the study of software quality by analyzing software metrics. Clustering analysis, which can be considered as one of the data mining techniques, is adopted to build the software quality prediction models in the early period of software testing. In this paper, a new clustering method called Affinity Propagation is investigated for the analysis of two software metric datasets extracted from real-world software projects. Meanwhile, K-Means clustering method is also applied for comparison. The numerical experiment results show that the Affinity Propagation algorithm can be applied well in software quality prediction in the very early stage, and it is more effective on reducing Type II error.

We present a new feature extraction method, which called the complete two-dimensional principal component analysis (Complete 2DPCA), for image registration. Complete 2DPCA is based on 2D image matrices. Two image covariance matrices are constructed directly using the original image matrix and their eigenvectors are derived for image feature extraction. In the 2D image registration scheme, we propose complete 2DPCA to extract features from the image sets, and these features are input vectors of feedforward neural networks (FNN). Neural network outputs are registration parameters with respect to reference and observed image sets. Comparative experiments are performed between complete 2DPCA based method and other feature based methods. The results show that the proposed method has an encouraging performance.

In face recognition, the dimensionality of raw data is very high, dimension reduction (Feature Extraction) should be applied before classification. There exist several feature extraction methods, commonly used are Principle Component Analysis (PCA) and Linear Discriminant Analysis (LDA) techniques. In this paper, we present a comparative study of some feature extraction methods for face recognition in the same conditions. The methods evaluated here include Eigenfaces, Kernel Principal Component Analysis (KPCA), Fisherfaces, Direct Linear Discriminant Analysis (D-LDA), Regularized Linear Discriminant Analysis (R-LDA), and Kernel Direct Discriminant Analysis (KDDA). For the purpose of comparison on feature extraction methods, we adopt Nearest Neighbor (NN) algorithm from existed classifiers of face recognition, since this classifier is common and simpleness. Empirical studies are conducted to evaluate these feature extraction methods with images from ORL Face Database, and it is found that in most cases LDA-based methods are efficient than PCA-based ones.

We present a comprehensive system for weather data visualization. Weather data are multivariate and contain vector fields formed by wind speed and direction. Several well-established visualization techniques such as parallel coordinates and polar systems are integrated into our system. We also develop various novel methods, including circular pixel bar charts embedded into polar systems, enhanced parallel coordinates with S-shape axis, and weighted complete graphs. Our system was used to analyze the air pollution problem in Hong Kong and some interesting patterns have been found.

In this paper, a novel image registration method is proposed. In the proposed method, kernel independent component analysis (KICA) is applied to extract features from the image sets, and these features are input vectors of feedforward neural networks (FNN). Neural network outputs are those translation, rotation and scaling parameters with respect to reference and observed image sets. Comparative experiments are performed between KICA based method and other six feature extraction based method: principal component analysis (PCA), independent component analysis (ICA), kernel principal component analysis (KPCA), the discrete cosine transform (DCT), Zernike moment and the complete isometric mapping (Isomap). The results show that the proposed method is much improved not only at accuracy but also remarkably at robust to noise.

In the literature of astronomical spectral classification, linear principle component analysis (PCA) was frequently employed to extract features of spectra data. However, the spectral data are too complicated to be well described by a linear model. In this paper, kernel independent component analysis (KICA), which contains a nonlinear kernel mapping component, is adopted to extract features from the spectra of galaxies. Then, a radial basis function neural network is adopted as a classifier to implement the classification. Experiments with real-world spectral data set show that KICA is a very appropriate technique to describe the important features of celestial objects, and the correct classification rate is improved compared with PCA method.

Mean-shift analysis is a general nonparametric clustering technique based on density estimation for the analysis of complex feature spaces. It has been successfully applied to many applications such as segmentation and tracking. However, despite its promising performance, there are applications for which the algorithm converges too slowly and is not practical. In this paper, an improved version of mean shift algorithm is proposed and implemented. The fast mean shift procedure uses a new iteration strategy and re-sampling. The new iteration strategy is based on updating cluster centers according to dynamically updated sample set. And the original data set is simplified by re-sampling, which accelerates the algorithm more significantly. Experimental results demonstrate the efficiency of the fast mean shift procedure in clustering problems.

The software development process imposes major impacts on the quality of software at every development stage; therefore, a common goal of each software development phase concerns how to improve software quality. Software quality prediction thus aims to evaluate software quality level periodically and to indicate software quality problems early. In this paper, we propose a novel technique to predict software quality by adopting support vector machine (SVM) in the classification of software modules based on complexity metrics. Because only limited information of software complexity metrics is available in early software life cycle, ordinary software quality models cannot make good predictions generally. It is well known that SVM generalizes well even in high dimensional spaces under small training sample conditions. We consequently propose a SVM-based software classification model, whose characteristic is appropriate for early software quality predictions when only a small number of sample data are available. Experimental results with a medical imaging system software metrics data show that our SVM prediction model achieves better software quality prediction than some commonly used software quality prediction models.

A novel defocused image restoration technique is proposed, which is based on radial basis function (RBF) neural network and Kalman filter. In this technique, firstly a RBF neural network is trained in wavelet domain to estimate defocus parameter. After obtaining the point spread function (PSF) parameter, Kalman filter is adopted to complete the restoration. We experimentally illustrate its performance on simulated data and compare it with other methods. Results show that the proposed PSF parameter estimation technique is more robust to noise.

Feature extraction of multispectral remote sensing image is an important task before classifying the image. When land areas are clustered into groups of similar land cover, one of the most important things is to extract the key features of a given image. Usually multispectral remote sensing images have many bands, and there may have been much redundancy information and it becomes difficult to extract the key features of the image. Therefore, it is necessary to study methods regarding how to extract the main features of the image effectively. In this paper, five methods are comparatively studied to reduce the multi-bands into lower dimensions in order to extract the most available features. These methods include the Euclid distance measurement (EDM), the discrete measurement criteria function (DMCF), the minimum differentiated entropy (MDE), the probability distance criterion (PDC), and the principle component analysis (PCA) method. The advantage and disadvantage of each method are evaluated by the classification results.

Under the framework of the Kullback-Leibler (KL) distance, we show that a particular case of Gaussian probability function for feedforward neural networks (NNs) reduces into the first-order Tikhonov regularizer. The smooth parameter in kernel density estimation plays the role of regularization parameter. Under some approximations, an estimation formula is derived for estimating regularization parameters based on training data sets. The similarity and difference of the obtained results are compared with other work. Experimental results show that the estimation formula works well in sparse and small training sample cases.

One major problem in cluster analysis is the determination of the number of clusters. In this paper, we describe both theoretical and experimental results in determining the cluster number for a small set of samples using the Bayesian-Kullback Ying-Yang (BYY) model selection criterion. Under the second-order approximation, we derive a new equation for estimating the smoothing parameter in the cost function. Finally, we propose a gradient descent smoothing parameter estimation approach that avoids complicated integration procedure and gives the same optimal result.