To facilitate the large-scale deployment of autonomous driving in real-world scenarios, developing low-cost and high-performance 3D object detection systems has become a critical technical challenge. Although high-beam LiDARs provide denser point cloud data, their prohibitive hardware cost and high power consumption limit their practicality. In contrast, low-beam LiDARs offer advantages in terms of affordability and energy efficiency, but often suffer from inadequate perception accuracy due to their sparser point cloud data. This paper focuses on the task of multimodal 3D object detection with low-beam LiDARs, and proposes a novel approach that integrates temporal and spatial representation learning to enhance detection accuracy under sparser sensor conditions. Specifically, our approach comprises: (1) a Temporal Feature Prediction Learning (TFPL) module, which predicts the current BEV representation based on a sequence of historical BEV features; (2) a Spatial Feature Observation Learning (SFOL) module, which aligns BEV (Bird's-Eye-View) features from high-beam and low-beam LiDAR to enforce the low-beam features to approximate high-beam representations; (3) an Uncertainty-Aware Fusion (UAF) strategy, which performs feature-wise weighting between the predicted and observed BEV features by leveraging channel-wise variances, effectively mitigating perturbations in the learned BEV representations. Extensive experiments on the KITTI and nuScenes 3D object detection datasets demonstrate that the proposed approach significantly improves detection performance under low-beam LiDAR configurations.

Although multiview learning methods have been widely studied, they mostly focus on improving accuracy while ignoring decision uncertainty. In the real world, multiview data often encounters misalignment issues, resulting in conflictive instances and further limiting the application of these methods in safety-critical domains. Recently, some efforts have been made to improve the reliability of multiview learning methods by estimating decision uncertainty, but most methods often experience performance degradation due to their inability to handle conflictive instances. To address this issue, we propose a Robust Trusted Conflictive Multiview Collaborative Contrastive Learning (RCMCL) method, which enhances the model's robustness and generalization ability in conflictive multiview scenarios. Specifically, RCMCL first uses an evidential deep neural network to construct view-specific opinions, and then employs dissonance-based evidence contrastive learning to enhance the consistency of these opinions across different views. Subsequently, RCMCL performs collaborative learning of consistent evidence and complementary evidence. It first introduces the vacuity degree into the complementary evidence to extract more useful information, and then employs category-level contrastive learning to separate consistent and complementary evidence. In addition, consistent and complementary evidence is combined to make a joint decision. Finally, experimental results on eight benchmark datasets verify the superiority of RCMCL over state-of-the-art methods.

Inducing-point-based sparse variational approximation scales Gaussian process models to large datasets but tends to overestimate observation noise and underestimate posterior variance. Parametric predictive Gaussian process regressor (PPGPR) improve on point-wise uncertainty estimations, especially for heteroskedastic data, by repairing an mismatch between the training loss and the predictive metric for sparse variational Gaussian process (SVGP). In this paper, we approach uncertainty estimation with Gaussian process models from the perspective of information theory. We show both SVGP and PPGPR use the information bottleneck (IB) principle but in sub-optimal ways, which make the former fail to fully use the input-dependent latent function variance to model uncertainty, while the latter tend to underestimate observation noise and provide ill-conditioned predictive covariance. We further propose a fix for these problems through a decomposition of mutual information and designing two coupled decoders, resulting in a method named sparse variational information bottleneck Gaussian process (SVIBGP). Experiments on both synthetic and real-world datasets demonstrate that SVIBGP has the ability to account for heteroskedastic noise as well as provide improved uncertainty estimation.

In the 3D object detection task of autonomous driving systems, LiDAR and camera are the most crucial sensors, and current methods primarily focus on fusion strategies for these two complementary modalities. However, in real-world driving scenarios, potential sensor failures pose critical risks, potentially undermining the effectiveness of fusion-based detection approaches. This work presents RoLiC, a robust LiDAR-camera fusion framework designed to handle three challenging deployment scenarios within a unified model: LiDAR failure, camera failure, and simultaneous LiDAR-camera failure. To mitigate cross-modal dependency and recover missing information, RoLiC introduces two cross-modality feature transformers (L2C and C2L) that bidirectionally complete features between modalities when partial data is available. To further enhance feature reliability, we design a Sparse Similarity Loss (SSL) that constrains feature learning within high-probability object regions. Moreover, RoLiC integrates a task-aware two-stage feature knowledge distillation strategy (MS1 and MS2), where MS1 captures cross-modality complementarities, and MS2 distills knowledge between the fused modality and complete modality settings. Extensive experiments on the nuScenes and KITTI benchmarks demonstrate that RoLiC consistently outperforms state-of-the-art methods across all sensor-failure conditions. Code is available at https://github.com/JLIN77/RoLiC.

Humans are capable of inferring dynamic context from a still image and, with the provision of additional commonsense knowledge, can accurately complete visual commonsense reasoning tasks. Nevertheless, this remains a highly challenging cognitive-level task for current vision-language models. Previous work has primarily focused on utilizing models fine-tuned for specific downstream tasks and introduces external world knowledge to tackle these challenging tasks, while neglecting the importance of accurate context and the key role of commonsense knowledge in reasoning. In this paper, we propose a novel framework to enhance visual commonsense reasoning by incorporating context and commonsense knowledge. We decompose the visual commonsense reasoning problem into four distinct but interrelated sub-problems and combine visual language models with a large language model to enable zero-shot reasoning. The uniqueness of this work lies in the proposed commonsense knowledge filtering module, which filters out relevant commonsense knowledge through the causal strength of visual context. This process constructs Visual Context and Commonsense-guided Causal Chain-of-Thought ($\mathrm{VC^{3}}$-CoT) reasoning paths, thereby providing double robustness to visual commonsense reasoning by incorporating weighted majority voting strategy. Extensive experiments on several downstream tasks demonstrate that the proposed method significantly improves performance compared to baseline models and the state-of-the-art method, and confirm the effectiveness of the proposed components.

Although significant progress has been made in multi-view learning over the past few decades, it remains challenging, especially in the context of incomplete multi-view clustering, where modeling complex correlations among different views and handling missing data are key difficulties. In this paper, we propose a novel incomplete multi-view clustering network to address the aforementioned issue, named Incomplete Multi-view Clustering via Multi-level Contrastive Learning (IMC-MCL). Specifically, the proposed model aims to minimize the conditional entropy between views to recover missing data by dual prediction strategy. Moreover, the approach learns multi-level features, including latent, high-level and semantic features, with the goal of satisfying both reconstruction and consistency objectives in distinct feature spaces. Specifically, latent features are utilized to accomplish the reconstruction objective, while high-level features and semantic labels are employed to achieve the two consistency goals through contrastive learning. This framework enables the exploration of shared semantics within high-level features and achieves clustering assignment using semantic features. Extensive experiments have shown that the proposed approach outperforms other state-of-the-art incomplete multi-view clustering methods on seven challenging datasets.

Few-shot knowledge graph completion (FKGC) addresses the long-tail problem of relations by leveraging a few observed support entity pairs to infer unknown facts for tail-located relations. Learning the relation representation of entity pairs and evaluating the match of query and support entity pairs are the two key steps of FKGC. Existing methods learn the representation of entity pairs by either aggregating neighbors of entities or integrating relation representations in the connected paths from head to tail. However, in few-shot scenarios, the limited number of support entity pairs and insufficient structural information with a single neighborhood topology will lead to matching failure. To this end, we consider the star and ring topological information for a given entity pair: (1) Entity neighborhood, which captures multi-hop neighbors of entities; (2) Relational path, which characterizes compound relation forms. Furthermore, to effectively fuse the two kinds of heterogeneous topological information, we design the multi-aggregator and the fine-grained path correlation matching algorithm to obtain more delicate and balanced matching. Based on the proposed relational path correlation matching module, we propose the relation adaptive network to solve the few-shot temporal knowledge graph completion problem. The experimental results show that our method continuously outperforms the state-of-the-art methods.

Offline reinforcement learning (RL) has been widely used in practice due to its efficient data utilization, but it still faces the challenge of training vulnerability caused by policy deviation. Existing offline RL methods that add policy constraints or perform conservative Q-value estimation are pessimistic, making the learned policy suboptimal. In this article, we address the pessimism problem by focusing on accurate Q-value estimation. We propose the de-pessimism (DEP) operator to estimate Q values using the optimal Bellman operator or the compensation operator according to whether the actions are in the behavior support set. The compensation operator qualitatively determines the positive or negative nature of out-of-distribution (OOD) actions based on their performance compared with the behavior actions. It leverages differences in state values to compensate for the Q value of positive OOD actions, thereby alleviating pessimism. We theoretically demonstrate the convergence of DEP and its effectiveness in policy improvement. To further advance the practical application, we integrate DEP into the soft actor-critic (SAC) algorithm, yielding the value-compensated de-pessimism offline RL (DoRL-VC). Experimentally, DoRL-VC achieves state-of-the-art (SOTA) performance across mujoco locomotion, Maze 2-D, and challenging Adroit tasks, illustrating the efficacy of DEP in mitigating pessimism.

To leverage the abundant speech data available for pretraining, current models excel in generalization across diverse tasks. Nevertheless, real-world challenges emerge when addressing unfamiliar speech scenarios far from the pretrained speech, owing to the domain shift between pretraining (source) and fine-tuning (target) data. To overcome this barrier, we propose a variational Bayesian adapter (VB-Adapter) for cross-domain speech representation learning during fine-tuning. First, we establish a latent variable model to construct a desired posterior distribution after incorporating domain-specific knowledge to bridge the gap between the source and target domains. Then, an adaptive objective is presented to maximize the mutual information of the latent variables with and without domain-specific knowledge to facilitate model adaptation. Finally, we introduce contrastive learning on samples to optimize the lower bound of the above adaptive objective. Our experiments apply the VB-Adapter on transformers for dysarthric speech recognition (DSR) and the integration of Whisper-encoder and Llama for Mandarin speech recognition (MSR). The results reveal the effectiveness of VB-Adapter in modeling the uncertainties arising from domain shift and enhancing the robustness of speech representations.

Multi-view K-means clustering successfully generalizes K-means from single-view to multi-view, and obtains excellent clustering performance. In every view, it makes each data point close to the center of the corresponding cluster. However, multi-view K-means only considers the compactness of each cluster, but ignores the separability of different clusters, which is of great importance to producing a good clustering result. In this paper, we propose Discriminatively Fuzzy Multi-view K-means clustering with Local Structure Preserving (DFMKLS). On the basis of minimizing the distance between each data point and the center of the corresponding cluster, DFMKLS separates clusters by maximizing the distance between the centers of pairwise clusters. DFMKLS also relaxes its objective by introducing the idea of fuzzy clustering, which calculates the probability that a data point belongs to each cluster. Considering multi-view K-means mainly focuses on the global information of the data, to efficiently use the local information, we integrate the local structure preserving into the framework of DFMKLS. The effectiveness of DFMKLS is evaluated on benchmark multi-view datasets. It obtains superior performances than state-of-the-art multi-view clustering methods, including multi-view K-means.

Whole slide image (WSI) classification is a crucial component in automated pathology analysis. Due to the inherent challenges of high-resolution WSIs and the absence of patch-level labels, most of the proposed methods follow the multiple instance learning (MIL) formulation. While MIL has been equipped with excellent instance feature extractors and aggregators, it is prone to learn spurious associations that undermine the performance of the model. For example, relying solely on color features may lead to erroneous diagnoses due to spurious associations between the disease and the color of patches. To address this issue, we develop a causal MIL framework for WSI classification, effectively distinguishing between causal and spurious associations. Specifically, we use the expectation of the intervention P(Y | do(X)) for bag prediction rather than the traditional likelihood P(Y | X). By applying the front-door adjustment, the spurious association is effectively blocked, where the intervened mediator is aggregated from patch-level features. We evaluate our proposed method on two publicly available WSI datasets, Camelyon16 and TCGA-NSCLC. Our causal MIL framework shows outstanding performance and is plug-and-play, seamlessly integrating with various feature extractors and aggregators.

Unsupervised domain adaptation of multivariate time series aims to train a model to adapt its classification ability from a labeled source domain to an unlabeled target domain, where there are differences in the distribution between domains. Existing methods extract domain-invariant features directly via a shared feature extractor, neglecting the exploration of the underlying causal patterns, which undermines their reliability, especially in complex multivariate dynamic systems. To address this problem, we propose CauDiTS, an innovative framework for unsupervised domain adaptation of multivariate time series. CauDiTS adopts an adaptive rationale disentangler to disentangle domain-common causal rationales and domain-specific correlations from variable interrelationships. The stability of causal rationales across domains is vital for filtering domainspecific perturbations and facilitating the extraction of domain-invariant representations. Moreover, we promote the cross-domain consistency of intra-class causal rationales employing the learning strategies of causal prototype consistency and domain-intervention causality invariance. CauDiTS is evaluated on four benchmark datasets, demonstrating its effectiveness and outperforming state-of-the-art methods.

Multi-grained cross-modal image-text retrieval models have demonstrated promising outcomes through the alignment of local and global features. However, this advancement often results in larger model sizes and higher computational requirements, which raises concerns regarding the balance between performance and efficiency. To address this challenge, we introduce a novel lightweight multi-grained (LMG) image-text retrieval paradigm aimed at enhancing model efficiency. Specifically, in our approach, we first re-frame the retrieval problem as a cascaded representation learning task. This involves leveraging only fine-grained features to capture coarse-grained constraints, thereby reducing computational burden while maintaining accuracy. Furthermore, we replace computationally expensive parametric feature aggregation methods with three efficient parameter-free alternatives: auto-correlation matrix, discrete linear convolution, and discrete Fourier transform. The proposed LMG model is extensively compared with state-of-the-art approaches on two benchmark datasets, i.e., Flickr30K and MSCOCO, and the experimental results highlight the superior performance of LMG. Additionally, we explore the impact of different feature aggregation methods on LMG and conduct a sensitivity analysis on the coarse and fine-grained constraints ratio hyper-parameter.

Continual zero-shot learning (CZSL) aims to develop a model that accumulates historical knowledge to recognize unseen tasks, while eliminating catastrophic forgetting for seen tasks when learning new tasks. However, existing CZSL methods, while mitigating catastrophic forgetting for old tasks, often lead to negative transfer problem for new tasks by over-focusing on accumulating old knowledge and neglecting the plasticity of the model for learning new tasks. To tackle these problems, we propose PAMK, a prototype augmented multi-teacher knowledge transfer network that strikes a trade-off between recognition stability for old tasks and generalization plasticity for new tasks. PAMK consists of a prototype augmented contrastive generation (PACG) module and a multi-teacher knowledge transfer (MKT) module. To reduce the cumulative semantic decay of the class representation embedding and mitigate catastrophic forgetting, we propose a continual prototype augmentation strategy based on relevance scores in PACG. Furthermore, by introducing the prototype augmented semantic-visual contrastive loss, PACG promotes intra-class compactness for all classes across all tasks. MKT effectively accumulates semantic knowledge learned from old tasks to recognize new tasks via the proposed multi-teacher knowledge transfer, eliminating the negative transfer problem. Extensive experiments on various CZSL settings demonstrate the superior performance of PAMK compared to state-of-the-art methods. In particular, in the practical task-free CZSL setting, PAMK achieves impressive gains of 3.28%, 3.09% and 3.71% in mean harmonic accuracy on the CUB, AWA1, and AWA2 datasets, respectively.

Lane detection is one of the most fundamental tasks in autonomous driving perception, but it still faces many challenges in some special driving scenarios. For example, in dazzling light, crowded roads, etc., lane detection is very dependent on surrounding visual cues. Previous segmentation-based lane detection methods have not paid enough attention to the surrounding visual range, resulting in poor performance. In this paper, we design a novel lane detection network namely HW-Transformer, which is based on row and column multi-head self-attention. It restricts the attention only to their respective rows and columns, and transfers information across rows and columns by intersection features. In this way, the attention to the visual range around the lane is greatly expanded, and the communication of global information can be achieved through intersecting features. In addition, we further propose a self-attention knowledge distillation (SAKD) method for the Transformer model, where higher-level attention guides lower-level attention to learn. SAKD not only helps to improve the performance of lane detection, but also has universality in better learning semantic features from general images. Extensive experiments on BDD100K, TuSimple, CULane, and VIL100 datasets demonstrate that our method outperforms the state-of-the-art segmentation-based lane detection methods. We also apply the proposed SAKD to DeiT-tiny, and it achieves 1.51 Top-1 accuracy improvement on ImageNet-1K dataset. Our code will be available at https://github.com/Cuibaby/HWLane.

Graph-based multiview clustering methods have attracted much attention because of their ability to mine nonlinear structural information among instances. Although they perform well in many scenarios, they consume a lot of computational resources when dealing with large-scale multiview scenarios. To address this issue, we present a new insight into the anchor graph mechanism and propose a novel Nonnegative Anchor Graph Reconstruction (NAGR) model. NAGR introduces the sparse similarity graph into the symmetric matrix factorization and gets the nonnegative representation that retains the graph structural information. Thereafter, we develop a novel Efficient Multiview nonnegative Representation learning framework with Correntropy and Anchor graph (EMR-CA), which integrates multiview anchor graph reconstruction and consensus nonnegative representation learning into a unified framework. EMR-CA uses multiview anchor graph reconstruction to learn consensus nonnegative representation, where correntropy rather than F-norm is used as the approximation measurement criterion. Specifically, normalized anchor graphs of different views are decomposed into a consensus nonnegative representation and multiple view-specific representations, where the consensus representation retains the neighbor graph information between multiview instances and representative anchors on different views. Finally, the effectiveness of the proposed EMR-CA framework is verified by theoretical analysis and experimental results on large-scale realistic multiview scenarios.

LiDAR and camera are the most common used sensors to percept the road scenes in autonomous driving. Current methods tried to fuse the two complementary information to boost 3D object detection. However, there are still two burning problems for multi-modality 3D object detection. One is the detection problem for the objects with sparse point clouds. The other is the misalignment of different sensors caused by the fixed physical locations. Therefore, this paper argues that explicitly fusing information from the two modalities with the physical misalignment is suboptimal for multi-modality 3D object detection. This paper presents a novel virtual point generation network, VirPNet, to overcome the multi-modality fusion challenges. On one hand, it completes sparse point cloud objects from image source and improves the final detection accuracy. On the other hand, it directly detects 3D targets from raw point clouds to avoid the physical misalignment between LiDAR and camera sensors. Different from previous point cloud completion methods, VirPNet fully utilizes the geometric information of pixels and point clouds and simplifies 3D point cloud regression into a 2D distance regression problem through a virtual plane. Experimental results on KITTI 3D object detection dataset and nuScenes dataset demonstrate that VirPNet improves the detection accuracy with the help of the generated virtual points.

Multiview learning has made significant progress in recent years. However, an implicit assumption is that multiview data are complete, which is often contrary to practical applications. Due to human or data acquisition equipment errors, what we actually get is partial multiview data, which existing multiview algorithms are limited to processing. Modeling complex dependencies between views in terms of consistency and complementarity remains challenging, especially in partial multiview data scenarios. To address the above issues, this article proposes a deep Gaussian cross-view generation model (named PMvCG), which aims to model views according to the principles of consistency and complementarity and eventually learn the comprehensive representation of partial multiview data. PMvCG can discover cross-view associations by learning view-sharing and view-specific features of different views in the representation space. The missing views can be reconstructed and are applied in turn to further optimize the model. The estimated uncertainty in the model is also considered and integrated into the representation to improve the performance. We design a variational inference and iterative optimization algorithm to solve PMvCG effectively. We conduct comprehensive experiments on multiple real-world datasets to validate the performance of PMvCG. We compare the PMvCG with various methods by applying the learned representation to clustering and classification. We also provide more insightful analysis to explore the PMvCG, such as convergence analysis, parameter sensitivity analysis, and the effect of uncertainty in the representation. The experimental results indicate that PMvCG obtains promising results and surpasses other comparative methods under different experimental settings.

Despite incomplete multiview clustering (IMC) being widely studied in the past decade, it is still difficult to model the correlation among multiple views due to the absence of partial views. Most existing works for IMC only mine the correlation among multiple views from available views and ignore the importance of missing views. To address this issue, we propose a novel Incomplete Multiview Nonnegative representation learning model with Graph completion and Adaptive neighbors (IMNGA), which performs common graph learning, missing graph completion, and consensus nonnegative representation learning simultaneously. In IMNGA, the common graph on all views and the incomplete graph of each view are used to reconstruct the completed graph of the corresponding view, where the common graph satisfies the neighbor constraints of incomplete multiview data and consensus representation. IMNGA gets consensus representation by factorizing completed and incomplete graphs, where consensus representation satisfies the common graph constraint. IMNGA shows its effectiveness by outperforming other state-of-the-art methods.

Human-object interaction (HOI) detection involves identifying interactions represented as , requiring the localization of human-object pairs and interaction classification within an image. This work focuses on the challenge of detecting HOIs with unseen objects using the prevalent Transformer architecture. Our empirical analysis reveals that the performance degradation of novel HOI instances primarily arises from misclassifying unseen objects as confusable seen objects. To address this issue, we propose a similarity propagation (SP) scheme that leverages cosine similarity distance to regulate the prediction margin between seen and unseen objects. In addition, we introduce pseudo-supervision for unseen objects based on class semantic similarities during training. Furthermore, we incorporate semantic-aware instance-level and interaction-level contrastive losses with Transformer to enhance intraclass compactness and interclass separability, resulting in improved visual representations. Extensive experiments on two challenging benchmarks, V-COCO and HICO-DET, demonstrate the effectiveness of our model, outperforming current state-of-the-art methods under various zero-shot settings.

The conversational recommender system (CRS) aims to provide high-quality recommendations through interactive dialogues. However, previous CRS models have no effective mechanisms for task planning and topic elaboration, and thus they hardly maintain coherence in multi-task recommendation dialogues. Inspired by recent advances in prompt-based learning, we propose a novel contextual prompting framework for dialogue management, which optimizes prompts based on context, topics, and user profiles. Specifically, we develop a topic controller to sequentially plan the subtasks, and a prompt search module to construct context-aware prompts. We further adopt external knowledge to enrich user profiles and make knowledge-aware recommendations. Incorporating these techniques, we propose a conversational recommender system with contextual prompting, namely CP-Rec. Experimental results demonstrate that it achieves state-of-the-art recommendation accuracy and generates more coherent and informative conversations.

Doubly stochastic matrix plays an essential role in several areas such as statistics and machine learning. In this paper we consider the optimal approximation of a square matrix in the set of doubly stochastic matrices. A structured BFGS method is proposed to solve the dual of the primal problem. The resulting algorithm builds curvature information into the diagonal components of the true Hessian, so that it takes only additional linear cost to obtain the descent direction based on the gradient information without having to explicitly store the inverse Hessian approximation. The cost is substantially fewer than quadratic complexity of the classical BFGS algorithm. Meanwhile, a Newton-based line search method is presented for finding a suitable step size, which in practice uses the existing knowledge and takes only one iteration. The global convergence of our algorithm is established. We verify the advantages of our approach on both synthetic data and real data sets. The experimental results demonstrate that our algorithm outperforms the state-of-the-art solvers and enjoys outstanding scalability.

Gaussian process dynamical systems (GPDSs) have shown their effectiveness in many tasks of machine learning. However, when they address multi-view data, current GPDSs do not explicitly model the dependence between private and shared latent variables. Instead, they introduce structurally and intrinsically discrete segmentation in the latent space. In this paper, we propose the multi-view collaborative Gaussian process dynamical systems (McGPDSs) model, which assumes that the private latent variable for each view is controlled by its dynamical prior and the shared latent variable. The relevance between private and shared latent variables can be automatically learned by optimization in the Bayesian framework. The model is capable of learning an effective latent representation and generating novel data of one view given data of the other view. We evaluate our model on two-view data sets, and our model obtains better performance compared with the state-of-the-art multi-view GPDSs.

Multi-view learning is a widely studied topic in machine learning, which considers learning with multiple views of samples to improve the prediction performance. Even though some approaches have sprung up recently, it is still challenging to jointly explore information contained in different views. Multi-view deep Gaussian processes have shown strong advantages in unsupervised representation learning. However, they are limited when dealing with labeled multi-view data for supervised learning, and ignore the application potential of uncertainty estimation. In this paper, we propose a supervised multi-view deep Gaussian process model (named SupMvDGP), which uses the label of the views to further improve the performance, and takes the quantitative uncertainty estimation as a supplement to assist humans to make better use of prediction. According to the diversity of views, the SupMvDGP can establish asymmetric depth structure to better model different views, so as to make full use of the property of each view. We provide a variational inference method to effectively solve the complex model. Finally, we conduct comprehensive comparative experiments on multiple real world datasets to evaluate the performance of SupMvDGP. The experimental results show that the SupMvDGP achieves the state-of-the-art results in multiple tasks, which verifies the effectiveness and superiority of the proposed approach. Meanwhile, we provide a case study to show that the SupMvDGP has the ability to provide uncertainty estimation than alternative deep models, which can alert people to better treat the prediction results in high-risk applications.

Multivariate time series clustering has become an important research topic in the time series learning task, which aims to discover the correlation among multiple sequences and partition multivariate time series data into several subsets. Although there are currently some methods that can handle this task, most of them fail to discover informative subsequences from multivariate time series instances. In this paper, we first propose a novel unsupervised shapelet learning with adaptive neighbors (USLA) model for learning salient multivariate subsequences (i.e., multivariate shapelets), where the importance of each variate can be auto-determined when given a candidate multivariate shapelet. USLA performs multivariate shapelet-transformed representation learning and local structure learning simultaneously, but the performance of USLA with multivariate shapelets of different lengths is comparable to that of isometric multivariate shapelets. In fact, the shapelet-transformed representations learned from multivariate shapelets of different lengths can all represent multivariate time series instances separately and often contain complementary information to each other. Therefore, we develop a novel multiview USLA (MUSLA) model which treats shapelet-transformed representations learned from shapelets of different lengths as different views. In this way, MUSLA learns the importance of each view and the neighbor graph matrix among multiview representations when candidate multivariate shapelets of different lengths are determined. Experimental results show that MUSLA outperforms other state-of-the-art multivariate time series algorithms on real-world multivariate time series datasets.

As one category of important incomplete multi-view clustering methods, subspace based methods seek the common latent representation of incomplete multi-view data by matrix factorization and then partition the latent representation to get clustering results. However, these methods ignore missing views in the process of matrix factorization, which makes the connection of different views be exploited inadequately. This paper proposes Incomplete Multi-view Clustering with Reconstructed Views (IMCRV), which utilizes the incomplete examples sufficiently. In IMCRV, the missing views of incomplete examples are reconstructed and the reconstructed views are also used to seek the common latent representation. IMCRV also involves the Laplacian regularization to preserve the global property of the latent representation. A novel gradient descent method with the multiplicative update rule is designed to solve the objective function of IMCRV. The corresponding iterative algorithm is developed and the convergence of the algorithm is proved. IMCRV is compared with many state-of-the-art incomplete multi-view clustering methods under different Incomplete Example Rates (IER) on public multi-view datasets. The experimental results demonstrate the superior effectiveness of IMCRV.

Extreme multi-label text classification (XMTC) aims to tag a text instance with the most relevant subset of labels from an extremely large label set. XMTC has attracted much recent attention due to massive label sets yielded by modern applications, such as news annotation and product recommendation. The main challenges of XMTC are the datascalabilityandsparsity, thereby leading to two issues: i) the intractability to scale to the extreme label setting, ii) the presence of long-tailed label distribution, implying that a large fraction of labels have few positive training instances. To overcome these problems, we propose BGNN-XML, a scalable graph neural network framework tailored for XMTC problems. Specifically, we exploit label correlations via excavating their co-occurrence patterns and build a label graph based on the correlation matrix. We then conduct the attributed graph clustering by performing graph convolution with a low-pass graph filter to jointly model label dependencies and label features, which induces semantic label clusters. We further propose a bilateral-branch graph isomorphism network to decouple representation learning and classifier learning for better modelingtail labels. Experimental results on multiple benchmark datasets demonstrate that BGNN-XML significantly outperforms state-of-the-art baselines while maintaining comparable prediction efficiency and model size.

Black-box attacks in deep reinforcement learning usually retrain substitute policies to mimic behaviors of target policies as well as craft adversarial examples, and attack the target policies with these transferable adversarial examples. However, the transferability of adversarial examples is not always guaranteed. Moreover, current methods of crafting adversarial examples only utilize simple pixel space metrics which neglect semantics in the whole images, and thus generate unnatural adversarial examples. To address these problems, we propose an advRL-GAN framework to directly generate semantically natural adversarial examples in the black-box setting, bypassing the transferability requirement of adversarial examples. It formalizes the black-box attack as a reinforcement learning (RL) agent, which explores natural and aggressive adversarial examples with generative adversarial networks and the feedback of target agents. To the best of our knowledge, it is the first RL-based adversarial attack on a deep RL agent. Experimental results on multiple environments demonstrate the effectiveness of advRL-GAN in terms of reward reductions and magnitudes of perturbations, and validate the sparse and targeted property of adversarial perturbations through visualization.

This work proposes a novel unsupervised cross-modality adaptive segmentation method for medical images to tackle the performance degradation caused by the severe domain shift when neural networks are being deployed to unseen modalities. The proposed method is an end-2-end framework, which conducts appearance transformation via a domain-shared shallow content encoder and two domain-specific decoders. The feature extracted from the encoder is enhanced to be more domain-invariant by a similarity learning task using the proposed Semantic Similarity Mining (SSM) module which has a strong help of domain adaptation. The domain-invariant latent feature is then fused into the target domain segmentation sub-network, trained using the original target domain images and the translated target images from the source domain in the framework of adversarial training. The adversarial training is effective to narrow the remaining gap between domains in semantic space after appearance alignment. Experimental results on two challenging datasets demonstrate that our method outperforms the state-of-the-art approaches.

Temporal knowledge graphs (TKGs) have been widely used in various fields that model the dynamics of facts along the timeline. In the extrapolation setting of TKG reasoning, since facts happening in the future are entirely unknowable, insight into history is the key to predicting future facts. However, it is still a great challenge for existing models as they hardly learn the characteristics of historical events adequately. From the perspective of historical development laws, comprehensively considering the sequential, repetitive, and cyclical patterns of historical facts is conducive to predicting future facts. To this end, we propose a novel representation learning model for TKG reasoning, namely TiRGN, a time-guided recurrent graph network with local-global historical patterns. Specifically, TiRGN uses a local recurrent graph encoder network to model the historical dependency of events at adjacent timestamps and uses the global history encoder network to collect repeated historical facts. After the trade-off between the two encoders, the final inference is performed by a decoder with periodicity. We use six benchmark datasets to evaluate the proposed method. The experimental results show that TiRGN outperforms the state-of-the-art TKG reasoning methods in most cases.

Although deep neural networks have shown great potential for many tasks, they are vulnerable to adversarial examples, which are generated by adding small perturbations to natural examples. Recently, many studies have proved that making full use of different modalities can effectively enhance the representational ability of deep neural networks. We propose a multi-modal deep fusion Transformer, termed MDFT. First, the audio feature and the rich semantic text features are extracted by audio encoders and text encoders, respectively. Then, multi-modal attention mechanisms are established to capture the high-level interactions between the audio and linguistic domains to obtain joint multi-modal representation. Finally, the representation is propagated to a dense layer to generate the detection result. The accuracy of this model compared with its unimodal variant on WiAd dataset and BlAd dataset are improved by 0.12 % and 0.19 %, respectively. Experimental results on the two datasets show that MDFT outperforms its unimodal variant model.

Cross-modal retrieval is usually implemented based on cross-modal representation learning, which is used to extract semantic information from cross-modal data. Recent work shows that cross-modal representation learning is vulnerable to adversarial attacks, even using large-scale pre-trained networks. By attacking the representation, it can be simple to attack the downstream tasks, especially for cross-modal retrieval tasks. Adversarial attacks on any modality will easily lead to obvious retrieval errors, which brings the challenge to improve the adversarial robustness of cross-modal retrieval. In this paper, we propose a robust cross-modal retrieval method (RoCMR), which generates adversarial examples for both the query modality and candidate modality and performs adversarial training for cross-modal retrieval. Specifically, we generate adversarial examples for both image and text modalities and train the model with benign and adversarial examples in the framework of contrastive learning. We evaluate the proposed RoCMR on two datasets and show its effectiveness in defending against gradient-based attacks.

The challenge of diverse machine translation is to ensure both diversity and quality, in which diversity desires the distinction between multiple hypotheses in the syntactic or word level, while quality requires hypotheses to be consistent with certain references. Existing work on diverse machine translation, most of which boosts translation diversity but compromises trans-lation quality, either resorts to special search strategies to realize word-level diversity or trains several decoders simultaneously to enable possible syntactic multiplicity. In this work, we propose to improve the translation diversity without a severe drop in quality through constructing a novel translation memory based NMT and designing a global diverse beam search strategy. Specifically, the translation memory retrieved from target-size corpus act as experts to interact with standard NMT, which not only generates various hypotheses, but also enhances the quality to a great degree. Moreover, we exploit a novel diverse beam search to further avoid token reuse across different hypotheses and improve local diversity. Experiments on two benchmarks (JRC-Acquis and WMT) demonstrate that our approaches achieve a compelling promotion both of translation quality and diversity compared with other diverse approaches.

Recently, the restricted Boltzmann machine (RBM) has aroused considerable interest in the multiview learning field. Although effectiveness is observed, like many existing multiview learning models, multiview RBM ignores the local manifold structure of multiview data. In this article, we first propose a novel graph RBM model, which preserves the data manifold structure and is amenable to Gibbs sampling. Then, we develop a multiview graph RBM model on the basis of the graph RBM, which performs local structural learning and multiview representation learning simultaneously. The proposed multiview model has the following merits: 1) it preserves the data manifold structure for multiview classification and 2) it performs view-consistent representation learning and view-specific representation learning simultaneously. The experimental results show that the proposed multiview model outperforms other state-of-the-art multiview classification algorithms.

Lane detection differs from general object detection in that lane lines are usually long and narrow in the road image, and more attention to image features at different scales is required to reason about lane lines under occlusion, degradation, and bad weather. However, most existing semantic segmentation-based lane detection methods focus on solving the convolutional receptive field through aggregating information vertically and horizontally in the same feature map, which may ignore important information contained in multi-scale features. Besides, the high-level semantic information of whether the lane exists is not fully utilized, as they often add a module at the final stage of the network output to determine whether the lane exists, which is a dispensable for their network. Based on the above analysis, we design a novel lane detection network based on semantic segmentation which consists of a Multi-scale Feature Information Aggregator (MFIA) module and a Channel Attention (CA) module. Many experiments on the TRLane dataset, the generated Lane dataset, BDD100K dataset, TuSimple dataset, VIL-100 dataset and CULane dataset show that our approach can achieve the state-of-the-art performance (our code will be available athttps://github.com/Cuibaby/MFIALane). In addition, considering that different perceptual tasks in autonomous driving are able to share the feature extraction network, we also conduct the experiment for drivable area segmentation on BDD100K dataset. Our approach also achieves good results compared to many existing methods, showing that our proposed model is capable of simultaneously handling multiple perceptual tasks in autonomous driving scenarios.

Recently, multiview learning has been increasingly focused on machine learning. However, most existing multiview learning methods cannot directly deal with multiview sequential data, in which the inherent dynamical structure is often ignored. Especially, most traditional multiview machine learning methods assume that the items at different time slices within a sequence are independent of each other. In order to solve this problem, we propose a new multiview discriminant model based on conditional random fields (CRFs) to model multiview sequential data, called multiview CRF. It inherits the advantages of CRFs that build a relationship between items in each sequence. Moreover, by introducing specific features designed on the CRFs for multiview data, the multiview CRF not only considers the relationship among different views but also captures the correlation between the features from the same view. Particularly, some features can be reused or divided into different views to build an appropriate size of feature space. This helps to avoid underfitting problems caused by too small feature space or overfitting problems caused by too large feature space. In order to handle large-scale data, we use the stochastic gradient method to speed up our model. The experimental results on the text and video data illustrate the superiority of the proposed model.

Estimating the complete 3D point cloud from an incomplete one lies at the core of many vision and robotics applications. Existing methods typically predict the complete point cloud based on the global shape representation extracted from the incomplete input. Although they could predict the overall shape of 3D objects, they are incapable of generating structure details of objects. Moreover, the partial input point sets obtained from range scans are often sparse, noisy and non-uniform, which largely hinder shape completion. In this paper, we propose an adaptive sampling and hierarchical folding network (ASHF-Net) for robust 3D point cloud completion. Our main contributions are two-fold. First, we propose a denoising auto-encoder with an adaptive sampling module, aiming at learning robust local region features that are insensitive to noise. Second, we propose a hierarchical folding decoder with the gated skip-attention and multi-resolution completion goal to effectively exploit the local structure details of partial inputs. We also design a KL regularization term to evenly distribute the generated points. Extensive experiments demonstrate that our method outperforms existing state-of-the-art methods on multiple 3D point cloud completion benchmarks.

Humans learn language speaking and translation with the help of common knowledge of the external world, while standard machine translation depends only on parallel corpora. There-fore, zero-resource translation has been proposed to explore a way to make models learn to translate without any parallel corpora but with external knowledge. Current models in the field usually combine an image encoder with a textual decoder, which leads to extra efforts to make use of the original model in machine translation. On the other hand, Transformers have achieved great progress in neural language processing while they are rarely utilized in the field of zero-resource translation with image pivot. In this paper, we investigate how to use visual information as an auxiliary hint for a Transformer-based system in a zero-resource translation scenario. Our model achieves state-of-the-art BLEU scores in the field of zero-resource machine translation with image pivot.

Dysarthria is a motor speech disorder caused by damage to the part of the nervous system that controls the physical production of speech. It poses great challenges in building robust dysarthric speech recognition (DSR) due to the high inter- and intra-speaker variability. To this end, we propose a multi-task Transformer with input feature reconstruction as an auxiliary task, where the main task of DSR and the auxiliary reconstruction task share the same encoder network. The auxiliary task aims to reconstruct clear speech features from corrupted speech of healthy speakers (intra-domain) or dysarthric speakers (cross-domain). Further, to alleviate the imbalanced distribution of dysarthria data sets, we devise an adaptive rebalance sampling scheme to improve the utterance sampling frequency of dysarthric speech. Experimental results show that the proposed model considerably outperforms other baselines across speakers with varying severity of dysarthria.

Dysarthria is a manifestation of disruption in the neuromuscular physiology resulting in uneven, slow, slurred, harsh, or quiet speech. Despite the remarkable progress of automatic speech recognition (ASR), it poses great challenges in developing stable ASR for dysarthric individuals due to the high intra- and inter-speaker variations and data deficiency. In this paper, we propose a contrastive learning framework for robust dysarthric speech recognition (DSR) by capturing the dysarthric speech variability. Several speech data augmentation strategies are explored to form two branches of the framework, meanwhile alleviating the scarcity of dysarthria data. We also develop an efficient projection head acting on a sequence of learned hidden representations for defining contrastive loss. Experiment results on DSR demonstrate that the model is better than or comparable to the supervised baseline.

The scarcity of available samples and the high annotation cost of medical data cause a bottleneck in many digital diagnosis tasks based on deep learning. This problem is especially severe in pediatric tumor tasks, due to the small population base of children and high sample diversity caused by the high metastasis rate of related tumors. Targeted research on pediatric tumors is urgently needed but lacks sufficient attention. In this work, we propose a novel model to solve the diagnosis task of small round blue cell tumors (SRBCTs). To solve the problem of high noise and high diversity in the small sample scenario, the model is constrained to pay attention to the valid areas in the pathological image with a masking mechanism, and a length-aware loss is proposed to improve the tolerance to feature diversity. We evaluate this framework on a challenging small sample SRBCTs dataset, whose classification is difficult even for professional pathologists. The proposed model shows the best performance compared with state-of-the-art deep models and generalization on another pathological dataset, which illustrates the potentiality of deep learning applications in difficult small sample medical tasks.

Recently, abstractive summarization models are preferred over extractive summarization models as they can generate words that do not exist in the original text, whose summary descriptions are more flexible and natural. Neural network-based models learn the pattern of summary generation from the training data by modeling the relationship between the original text and the reference summary, which is very dependent on the reference summary. Although we intuitively feel that summary with higher Abstraction Degree (quantified by the number of words in the summary which do not appear in the original text) will be more general, manually generated summaries with high Abstraction Degree are most likely subliminally written with additional knowledge. It's difficult to learn the generation pattern of such reference summary using limited training data. What's more, such reference summaries can even harm the model performance. To this end, we design a learning method that can adaptively weighted difference training samples based on their Abstraction Degree, so that the model will pay less attention to the samples with higher Abstraction Degree. Experiments of LCSTS and CNN-DM dataset show that our method greatly improves the performance of the summarization model and is resilient in the face of training data containing low quality reference summaries.

In view of the vulnerability of multi-view deep models to adversarial perturbation, this paper designs two kinds of multi-view adversarial autoencoders (MAAEs). We first propose the MAAE1 defense, in which each view is used to train the corresponding single-view adversarial autoencoders separately, and then the reconstructed output is fed into the target model for classification. Based on the consistency and complementary principle, we further integrate the scheme of adjusting the weights of different views adaptively into the design of multi-view defense, referred to it as MAAE2. Experimental results verify that the proposed multi-view adversarial defense effectively enhances the adversarial robustness of the deep multi-view model without negatively affecting the classification accuracy in the original examples. Besides, MAAEs show superior adversarial robustness to unseen perturbations.

Multi-view representation learning is a promising and challenging research topic, which aims to integrate multiple data information from different views to improve the learning performance. The recent deep Gaussian processes (DGPs) have the advantages of good uncertainty estimates, powerful non-linear mapping ability and great generalization capability, which can be used as an excellent data representation learning method. However, DGPs only focus on single view data and are rarely applied to the multi-view scenario. In this paper, we propose a multi-view representation learning algorithm with deep Gaussian processes (named MvDGPs), which inherits the advantages of deep Gaussian processes and multi-view representation learning, and can learn more effective representation of multi-view data. The MvDGPs consist of two stages. The first stage is multi-view data representation learning, which is mainly used to learn more comprehensive representations of multi-view data. The second stage is classifier design, which aims to select an appropriate classifier to better employ the representations obtained in the first stage. In contrast with DGPs, MvDGPs support asymmetrical modeling depths for different views of data, resulting in better characterizations of the discrepancies among different views. Experimental results on real-world multi-view data sets verify the effectiveness of the proposed algorithm, which indicates that MvDGPs can integrate the complementary information in multiple views to discover a good representation of the data.

Multi-label classification is an important research topic in machine learning, for which exploiting label dependencies is an effective modeling principle. Recently, probabilistic models have shown great potential in discovering dependencies among labels. In this paper, motivated by the recent success of multi-view learning to improve the generalization performance, we propose a novel multi-view probabilistic model named latent conditional Bernoulli mixture (LCBM) for multi-label classification. LCBM is a generative model taking features from different views as inputs, and conditional on the latent subspace shared by the views a Bernoulli mixture model is adopted to build label dependencies. Inside each component of the mixture, the labels have a weak correlation which facilitates computational convenience. The mean field variational inference framework is used to carry out approximate posterior inference in the probabilistic model, where we propose a Gaussian mixture variational autoencoder (GMVAE) for effective posterior approximation. We further develop a scalable stochastic training algorithm for efficiently optimizing the model parameters and variational parameters, and derive an efficient prediction procedure based on greedy search. Experimental results on multiple benchmark datasets show that our approach outperforms other state-of-the-art methods under various metrics.

The most frequent extracranial solid tumors of childhood, named peripheral neuroblastic tumors (pNTs), are very challenging to diagnose due to their diversified categories and varying forms. Auxiliary diagnosis methods of such pediatric malignant cancers are highly needed to provide pathologists assistance and reduce the risk of misdiagnosis before treatments. In this paper, inspired by the particularity of microscopic pathology images, we integrate neural networks with the texture energy measure (TEM) and propose a novel network architecture named DetexNet (deep texture network). This method enforces the low-level representation pattern clearer via embedding the expert knowledge as prior, so that the network can seize the key information of a relatively small pathological dataset more smoothly. By applying and finetuning TEM filters in the bottom layer of a network, we greatly improve the performance of the baseline. We further pre-train the model on unlabeled data with an auto-encoder architecture and implement a color space conversion on input images. Two kinds of experiments under different assumptions in the condition of limited training data are performed, and in both of them, the proposed method achieves the best performance compared with other state-of-the-art models and doctor diagnosis.

Gaussian process (GP) models are flexible nonparametric models widely used in a variety of tasks. Variational sparse GP (VSGP) scales GP models to large data sets by summarizing the posterior process with a set of inducing points. In this article, we extend VSGP to handle multiview data. We model each view with a VSGP and augment it with an additional set of inducing points. These VSGPs are coupled together by enforcing the means of their posteriors to agree at the locations of these inducing points. To learn these shared inducing points, we introduce an additional GP model that is defined in the concatenated feature space. Experiments on real-world data sets show that our multiview VSGP (MVSGP) model outperforms single-view VSGP consistently and is superior to state-of-the-art kernel-based multiview baselines for classification tasks.

Path tracking control of intelligent vehicles has to deal with the difficulties of model uncertainties and nonlinearities. As a class of adaptive optimal control methods, reinforcement learning (RL) has received increasing attention in solving difficult control problems. However, feature representation and online learning ability are two major problems to be solved for learning control of uncertain dynamic systems. In this article, we propose a multi-kernel online RL approach for path tracking control of intelligent vehicles. In the proposed approach, a multiple kernel feature learning framework is designed for online learning control based on dual heuristic programming (DHP) and the new online learning control algorithm is called multi-kernel DHP (MKDHP). In MKDHP, instead of the expert knowledge for selecting and fine-tuning of a suitable kernel function, only a set of basic kernel functions is required to be predefined and the multi-kernel features can be learned for value function approximation in the critic. The simulation studies on path tracking control for intelligent vehicles have been conducted under$S$-curve and urban road conditions. The results demonstrated that compared with other typical path tracking controllers for intelligent vehicles, such as the linear quadratic regulator (LQR), the pure pursuit controller and the ribbon-based controller, the proposed multi-kernel learning controller can achieve better performance in terms of tracking precision and smoothness.

The representation approximated by a single deep network is usually limited for reinforcement learning agents. We propose a novel multi-view deep attention network (MvDAN), which introduces multi-view representation learning into the reinforcement learning task for the first time. The proposed model approximates a set of strategies from multiple representations and combines these strategies based on attention mechanisms to provide a comprehensive strategy for a single-agent. Experimental results on eight Atari video games show that the MvDAN has effective competitive performance than single-view reinforcement learning methods.

Adversarial attack on graph neural network (GNN) is distinctive as it often jointly trains the available nodes to generate a graph as an adversarial example. Existing attacking approaches usually consider the case that all the training set is available which may be impractical. In this paper, we propose a novel Bayesian adversarial attack approach based on projected gradient descent optimization, called Bayesian PGD attack, which gets more general attack examples than deterministic attack approaches. The generated adversarial examples by our approach using the same partial dataset as deterministic attack approaches would make the GNN have higher misclassification rate on graph node classification. Specifically, in our approach, the edge perturbation Z is used for generating adversarial examples, which is viewed as a random variable with scale constraint, and the optimization target of the edge perturbation is to maximize the KL divergence between its true posterior distribution p(Z|D) and its approximate variational distribution qθ(Z). We experimentally find that the attack performance will decrease with the reduction of available nodes, and the effect of attack using different nodes varies greatly especially when the number of nodes is small. Through experimental comparison with the state-of-the-art attack approaches on GNNs, our approach is demonstrated to have better and robust attack performance.

We propose a unified multilingual model for humor detection which can be trained under a transfer learning framework. 1) The model is built based on pre-trained multilingual BERT, thereby is able to make predictions on Chinese, Russian and Spanish corpora. 2) We step out from single sentence classification and propose sequence-pair prediction which considers the inter-sentence relationship. 3) We propose the Sentence Discrepancy Prediction (SDP) loss, aiming to measure the semantic discrepancy of the sequence-pair, which often appears in the setup and punchline of a joke. Our method achieves two SoTA and a second-place on three humor detection corpora in three languages (Russian, Spanish and Chinese), and also improves F1-score by 4%-6%, which demonstrates the effectiveness of it in humor detection tasks.

Machine learning develops rapidly, which has made many theoretical breakthroughs and is widely applied in various fields. Optimization, as an important part of machine learning, has attracted much attention of researchers. With the exponential growth of data amount and the increase of model complexity, optimization methods in machine learning face more and more challenges. A lot of work on solving optimization problems or improving optimization methods in machine learning has been proposed successively. The systematic retrospect and summary of the optimization methods from the perspective of machine learning are of great significance, which can offer guidance for both developments of optimization and machine learning research. In this article, we first describe the optimization problems in machine learning. Then, we introduce the principles and progresses of commonly used optimization methods. Finally, we explore and give some challenges and open problems for the optimization in machine learning.

Multi-view learning (MVL) is an active direction in machine learning that aims at exploiting the consensus and complementarity information among multiple distinct feature sets to boost the generalization performance of the counterpart algorithm. So far, two classical SVM-based MVL methods are SVM-2K and multi-view twin support vector machine (MvTSVM). They are designed only for two-view classification and cannot tackle the general multi-view classification problem. They also cannot effectively leverage the complementarity information among different feature views. In this paper, we propose two novel multi-view support vector machines with the consensus and complementarity information for MVL that not only can deal with the two-view classification problem but also the general multi-view classification problem by jointly learning multiple different views in a non-pairwise way. The disagreement among different views is regarded as a constraint or a regularization term in the objective function which plays an important role in exploring the consensus information. Combination weights for the reconstruction of each view in regularization terms are learned to explore complementarity information among different views. Finally, an efficient iteration algorithm with the classical convex quadratic programming is developed for optimization. Experimental results validate the effectiveness of our proposed methods.

Canonical Correlation Analysis (CCA) is a popular multiview dimension reduction method, which aims to maximize the correlation between two views to find the common subspace shared by these two views. However, it can only deal with two-view data, while the number of views frequently exceeds two in many real applications. To handle data with more than two views, in the previous studies, either the pairwise correlation or the high-order correlation was employed. These two types of correlation define the relation of multiview data from different viewpoints, and both have special effects for view consistency. To obtain flexible view consistency, in this article, we propose multiview uncorrelated locality preserving projection (MULPP), which considers two types of correlation simultaneously. The MULPP also considers the complementary property of different views by preserving the local structures of all the views. To obtain multiple projections and minimize the redundancy of low-dimensional features, for each view, the MULPP makes the features extracted by different projections uncorrelated. The MULPP is solved by an iteration algorithm, and the convergence of the algorithm is proven. The experiments on Multiple Feature, Coil-100, 3Sources, and NUS-WIDE data sets demonstrate the effectiveness of MULPP.

We develop a new paradigm for the task of joint entity relation extraction.It first identifies entity spans, then performs a joint inference on entity types and relation types.To tackle the joint type inference task, we propose a novel graph convolutional network (GCN) running on an entity-relation bipartite graph.By introducing a binary relation classification task, we are able to utilize the structure of entity-relation bipartite graph in a more efficient and interpretable way.Experiments on ACE05 show that our model outperforms existing joint models in entity performance and is competitive with the state-of-the-art in relation performance.

Generalized eigenvalue proximal support vector machines (GEPSVMs) are a simple and effective binary classification method in which each hyperplane is closest to one of the two classes and as far as possible from the other class. They solve a pair of generalized eigenvalue problems to obtain two nonparallel hyperplanes. Multiview learning considers learning with multiple feature sets to improve the learning performance. In this paper, we propose multiview GEPSVMs (MvGSVMs) which effectively combine two views by introducing a multiview co-regularization term to maximize the consensus on distinct views, and skillfully transform a complicated optimization problem to a simple generalized eigenvalue problem. We also propose multiview improved GEPSVMs (MvIGSVMs), which use the minus instead of ratio in MvGSVMs to measure the differences of the distances between the two classes and the hyperplane and lead to a simpler eigenvalue problem. Linear MvGSVMs and MvIGSVMs are generalized to the nonlinear case by the kernel trick. Experimental results on multiple data sets show the effectiveness of our proposed approaches.

The limited memory version of the Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm is the most popular quasi-Newton algorithm in machine learning and optimization. Recently, it was shown that the stochastic L-BFGS (sL-BFGS) algorithm with the variance-reduced stochastic gradient converges linearly. In this paper, we propose a new sL-BFGS algorithm by importing a proper momentum. We prove an accelerated linear convergence rate under mild conditions. The experimental results on different data sets also verify this acceleration advantage.

We investigate the task of joint entity relation extraction.Unlike prior efforts, we propose a new lightweight joint learning paradigm based on minimum risk training (MRT).Specifically, our algorithm optimizes a global loss function which is flexible and effective to explore interactions between the entity model and the relation model.We implement a strong and simple neural network where the MRT is executed.Experiment results on the benchmark ACE05 and NYT datasets show that our model is able to achieve state-of-the-art joint extraction performances.

PAC-Bayes bounds have been proposed to get risk estimates based on a training sample. In this paper the PAC-Bayes approach is combined with stability of the hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting is used with a Gaussian prior centered at the expected output. Thus a novelty of our paper is using priors defined in terms of the data-generating distribution. Our main result estimates the risk of the randomized algorithm in terms of the hypothesis stability coefficients. We also provide a new bound for the SVM classifier, which is compared to other known bounds experimentally. Ours appears to be the first uniform hypothesis stability-based bound that evaluates to non-trivial values.

We investigate the task of open domain opinion relation extraction. Different from works on manually labeled corpus, we propose an efficient distantly supervised framework based on pattern matching and neural network classifiers. The patterns are designed to automatically generate training data, and the deep learning model is design to capture various lexical and syntactic features. The result algorithm is fast and scalable on large-scale corpus. We test the system on the Amazon online review dataset. The result shows that our model is able to achieve promising performances without any human annotations.

Gaussian Processes (GPs) are powerful tools for machine learning which have been applied to both classification and regression. The mixture models of GPs were later proposed to further improve GPs for data modeling. However, these models are formulated for regression problems. In this work, we propose a new Mixture of Gaussian Processes for Classification (MGPC). Instead of the Gaussian likelihood for regression, MGPC employs the logistic function as likelihood to obtain the class probabilities, which is suitable for classification problems. The posterior distribution of latent variables is approximated through variational inference. The hyperparameters are optimized through the variational EM method and a greedy algorithm. Experiments are performed on multiple real-world datasets which show improvements over five widely used methods on predictive performance. The results also indicate that for classification MGPC is significantly better than the regression model with mixtures of GPs, different from the existing consensus that their single model counterparts are comparable.

In this work, we try to understand the differences between exact and approximate inference algorithms in structured prediction. We compare the estimation and approximation error of both underestimate and overestimate models. The result shows that, from the perspective of learning errors, performances of approximate inference could be as good as exact inference. The error analyses also suggest a new margin for existing learning algorithms. Empirical evaluations on text classification, sequential labelling and dependency parsing witness the success of approximate inference and the benefit of the proposed margin.

We propose a novel algorithm for supervised dimensionality reduction named manifold partition discriminant analysis (MPDA). It aims to find a linear embedding space where the within-class similarity is achieved along the direction that is consistent with the local variation of the data manifold, while nearby data belonging to different classes are well separated. By partitioning the data manifold into a number of linear subspaces and utilizing the first-order Taylor expansion, MPDA explicitly parameterizes the connections of tangent spaces and represents the data manifold in a piecewise manner. While graph Laplacian methods capture only the pairwise interaction between data points, our method captures both pairwise and higher order interactions (using regional consistency) between data points. This manifold representation can help to improve the measure of within-class similarity, which further leads to improved performance of dimensionality reduction. Experimental results on multiple real-world data sets demonstrate the effectiveness of the proposed method.

Cross-view data are collected from two different views or sources about the same subjects. As the information from these views often consolidate and/or complement each other, cross-view data analysis can gain more insights for decision making. A main challenge of cross-view data analysis is how to effectively explore the inherently correlated and high-dimensional data. Dimension reduction offers an effective solution for this problem. However, how to choose right models and parameters involved for dimension reduction is still an open problem. In this paper, we propose an effective sparse learning algorithm for cross-view dimensionality reduction. A distinguished character of our model selection is that it is nonparametric and automatic. Specifically, we represent the correlation of cross-view data using a covariance matrix. Then, we decompose the matrix into a sequence of low-rank ones by solving an optimization problem in an alternating least squares manner. More importantly, a new and nonparametric sparsity-inducing function is developed to derive a parsimonious model. Extensive experiments are conducted on real-world data sets to evaluate the effectiveness of the proposed algorithm. The results show that our method is competitive with the state-of-the-art sparse learning algorithms.

This issue marks the first anniversary issue since I was honored to serve as the Editor-in-Chief (EiC) of the IEEE Transactions on Neural Networks and Learning Systems (TNNLS). I am happy to report that we had a very successful year and here are a few highlights that I would like to share with the community.•The latest impact factor of TNNLS is 4.854 according to the Journal Citation Reports. This marks a record high impact factor for our journal and places TNNLS as the number one scholarly publication in Computer Science (Hardware & Architecture), number three in Computer Science (Theory & Methods), and number ten in Electrical and Electronic Engineering journals.

Maximum entropy discrimination (MED) is a general framework for discriminative estimation which integrates the principles of maximum entropy and maximum margin. In this paper, we propose a novel approach named multi-kernel MED (MKMED) for multi-view learning (MVL), which takes advantage of the comp lementary principle for MVL. Multiple kernels encode the similarities in different views. We obtain a kernel matrix by multiple kernel combination to make use of the complementary information in different views. Based on the kernel matrix obtained by multiple kernel combination, we can proceed MVL within the MED framework. The experimental results on multiple datasets demonstrate the effectiveness of the proposed MKMED. MKMED outperforms the single-view MEDs and a competing MVL mothod named SVM-2K, and is competitive with the state-of-the-art multi-view MED (MVMED) and even sometimes exceeds it.

This paper presents a dependent multi-output Gaussian process (GP) for modeling complex dynamical systems. The outputs are dependent in this model, which is largely different from previous GP dynamical systems. We adopt convolved multi-output GPs to model the outputs, which are provided with a flexible multi-output covariance function. We adapt the variational inference method with inducing points for learning the model. Conjugate gradient based optimization is used to solve parameters involved by maximizing the variational lower bound of the marginal likelihood. The proposed model has superiority on modeling dynamical systems under the more reasonable assumption and the fully Bayesian learning framework. Further, it can be flexibly extended to handle regression problems. We evaluate the model on both synthetic and real-world data including motion capture data, traffic flow data and robot inverse dynamics data. Various evaluation methods are taken on the experiments to demonstrate the effectiveness of our model, and encouraging results are observed.

Multiview learning is more robust than single-view learning in many real applications. Canonical correlation analysis (CCA) is a popular technique to utilize information stemming from multiple feature sets. However, it does not exploit label information effectively. Later multiview linear discriminant analysis (MLDA) was proposed through combining CCA and linear discriminant analysis (LDA). Due to the successful application of uncorrelated LDA (ULDA), which seeks optimal discriminant features with minimum redundancy, we propose a new supervised learning method called multiview ULDA (MULDA) in this paper. This method combines the theory of ULDA with CCA. Then we adapt discriminant CCA (DCCA) instead of the CCA in MLDA and MULDA, and discuss about the effect of this modification. Furthermore, we generalize these methods to the nonlinear case by kernel-based learning techniques. The new method is called kernel multiview uncorrelated discriminant analysis (KMUDA). Then we modify kernel multiview discriminant analysis and KMUDA by replacing Kernel CCA with Kernel DCCA. Our methods are tested on different real datasets and compared with other state-of-the-art methods. Experimental results validate the effectiveness of our methods.

Traffic flow prediction, which predicts the future flow using historic flows, is an important task in intelligent transportation systems (ITS). Efficient and accurate models for traffic flow prediction greatly contribute to the development of ITS. In this paper, we adopt the Gaussian process dynamical model (GPDM) to a fourth-order GPDM, which is more suitable for modeling traffic flow data. Specifically, the latent variables in the fourth-order GPDM is a fourth-order Markov Gaussian process, and the weighted k-NN is incorporated in the model to predict latent variables for efficient prediction. After training the model, the future flow is estimated by the average of the results predicted by the fourth-order GPDM and k-NN. Compared with other popular methods, the proposed method performs best and yields significant improvements of prediction performance.

Maximum entropy discrimination (MED) is a general framework for discriminative estimation based on maximum entropy and maximum margin principles, and can produce hard-margin support vector machines under some assumptions. Recently, the multiview version of MED multiview MED (MVMED) was proposed. In this paper, we try to explore a more natural MVMED framework by assuming two separate distributions p1( Θ1) over the first-view classifier parameter Θ1 and p2( Θ2) over the second-view classifier parameter Θ2 . We name the new MVMED framework as alternative MVMED (AMVMED), which enforces the posteriors of two view margins to be equal. The proposed AMVMED is more flexible than the existing MVMED, because compared with MVMED, which optimizes one relative entropy, AMVMED assigns one relative entropy term to each of the two views, thus incorporating a tradeoff between the two views. We give the detailed solving procedure, which can be divided into two steps. The first step is solving our optimization problem without considering the equal margin posteriors from two views, and then, in the second step, we consider the equal posteriors. Experimental results on multiple real-world data sets verify the effectiveness of the AMVMED, and comparisons with MVMED are also reported.

Semisupervised learning has been an active research topic in machine learning and data mining. One main reason is that labeling examples is expensive and time-consuming, while there are large numbers of unlabeled examples available in many practical problems. So far, Laplacian regularization has been widely used in semisupervised learning. In this paper, we propose a new regularization method called tangent space intrinsic manifold regularization. It is intrinsic to data manifold and favors linear functions on the manifold. Fundamental elements involved in the formulation of the regularization are local tangent space representations, which are estimated by local principal component analysis, and the connections that relate adjacent tangent spaces. Simultaneously, we explore its application to semisupervised classification and propose two new learning algorithms called tangent space intrinsic manifold regularized support vector machines (TiSVMs) and tangent space intrinsic manifold regularized twin SVMs (TiTSVMs). They effectively integrate the tangent space intrinsic manifold regularization consideration. The optimization of TiSVMs can be solved by a standard quadratic programming, while the optimization of TiTSVMs can be solved by a pair of standard quadratic programmings. The experimental results of semisupervised classification problems show the effectiveness of the proposed semisupervised learning algorithms.

We present a text detection method in natural scene images based on two-stage nontext filtering. Firstly, we detect multi-channel maximally stable extremal regions (MSERs) as character candidates. To reduce the amount of repeating components, we merge the MSERs by choosing the most character-like ones when overlap happens. Then nontext components are filtered out by a two-stage labeling procedure, wherein we combine random forests with CRF. Finally, components labeled as text are grouped into words by an edge-cut strategy, and false positives are eliminated by a HOG-based classifier. The experimental results on the ICDAR2013 database show the effectiveness of the proposed method.

We investigate the bilinear model, which is a matrix form linear model with the rank 1 constraint. A new online learning algorithm is proposed to train the model parameters. Our algorithm runs in the manner of online mirror descent, and gradients are computed by the power iteration. To analyze it, we give a new second order approximation of the squared spectral norm, which helps us to get a regret bound. Experiments on two sequential labelling tasks give positive results.

In this paper, we present a novel dimensionality reduction method, called sparse uncorrelated cross-domain feature extraction (SUFE), for signal classification in brain-computer interfaces (BCIs). Considering the differences between the source and target distributions of signals from different subjects, we construct an optimization objective which aims to find a projection matrix to transform the original data in a high-dimensional space into a low-dimensional latent space. In the low-dimensional space, both the discrimination of different classes and transferability between the source and target domains are preserved. To make sure the minimum information redundancy, the extracted features are designed to be statistically uncorrelated. Then, by adding the l1-norm penalty, we incorporate sparsity into the uncorrelated transformation. In the experiments, we evaluate the method with multiple datasets, and compare with the state-of-the-art methods. The results show that the proposed approach has better performance and is suitable for cross-domain signal classification.

This paper presents a novel dimensionality reduction method, called uncorrelated transferable feature extraction (UTFE), for signal classification in brain-computer interfaces (BCIs). Considering the difference between the source and target distributions of signals from different subjects, we construct an optimization objective that finds a projection matrix to transform the original data in a high-dimensional space into a low-dimensional latent space and that guarantees both the discrimination of different classes and transferability between the source and target domains. In the low-dimensional latent space, the model constructed in the source domain can generalize well to the target domain. Additionally, the extracted features are statistically uncorrelated, which ensure the minimum informative redundancy in the latent space. In the experiments, we evaluate the method with data from nine BCI subjects, and compare with the state-of-the-art methods. The results demonstrate that our method has better performance and is suitable for signal classification in BCIs.

Twin support vector machines are a powerful learning method for binary classification. Compared to standard support vector machines, they learn two hyperplanes rather than one as in standard support vector machines, and work faster and sometimes perform better than support vector machines. However, relatively little is known about their theoretical performance. As recent tightest bounds for practical applications, PAC-Bayes bounds are based on a prior and posterior over the distribution of classifiers. In this paper, we study twin support vector machines from a theoretical perspective and use the PAC-Bayes bound to measure the generalization error bound of twin support vector machines. Experimental results on real-world datasets show better predictive capabilities of the PAC-Bayes bound for twin support vector machines compared to the PAC-Bayes bound for support vector machines.

Twin support vector machines are a recently proposed learning method for binary classification. They learn two hyperplanes rather than one as in conventional support vector machines and often bring performance improvements. Multi-view learning is concerned about learning from multiple distinct f eature sets, which aims to exploit distinct views to improve generalization performance. In this paper, we propose multi-view twin support vector machines by solving a pair of quadratic programming problems. This paper gives a detailed derivation of the Lagrange dual optimization formulation. The linear multi-view twin support vector machines are further generalized to the nonlinear case by the kernel trick. Experimental results demonstrate that our proposed methods are effective.

A brain-computer interface (BCI) is a system that allows its users to control external devices which are independent of peripheral nerves and muscles with brain activities. Electroencephalogram (EEG) signals are electrical signals collected from the scalp. They are frequently used in brain-computer interaction. However, EEG signals which change over time are highly non-stationary. One major challenge in current BCI research is how to extract features of time-varying EEG signals and classify the signals as accurately as possible. An effective BCI should be robust against and adaptive to the dynamic variations of brain activities. Adaptive learning in a BCI system, a rapidly developing application of machine learning, would be an effective approach to conquer the challenge. This paper reviews representative adaptive feature extraction and classification methods for EEG-based BCIs and further discusses some important open problems which can hopefully be useful to promote the research of the BCIs.

In this paper, we propose a supervised Bayesian sparse coding (SBSC) model for classification. The sparse coding with Laplacian scale mixture prior is formulated as a weighted l1minimization problem. Category-specific discriminative dictionaries and regularization parameters are learned using variational EM algorithm from the training samples of each category. Instability of previous sparse coding methods is alleviated through the regularizer design. Classification of a test sample is done using the MAP estimate of the sparse codes. We have tested the model on different recognition tasks and demonstrated the effectiveness of the model.

Learning from multiple feature sets, which is also called multi-view learning, is more robust than single view learning in many real applications. Canonical correlation analysis (CCA) is a popular technique to utilize information from multiple views. However, as an unsupervised method, it does not exploit the label information. In this paper, we propose an algorithm which combines uncorrelated linear discriminant analysis (ULDA) with CCA, named multi-view uncorrelated linear discriminant analysis (MULDA). Due to the successful application of ULDA, which seeks optimal discriminant features with minimum redundancy in the single view situation, it could be expected that the recognition performance would be enhanced. Experiments on handwritten digit data verify this expectation with results outperform other related methods.

In transfer learning scenarios, previous discriminative dimensionality reduction methods tend to perform poorly owing to the difference between source and target distributions. In such cases, it is unsuitable to only consider discrimination in the low-dimensional source latent space since this would generalize badly to target domains. In this paper, we propose a new dimensionality reduction method for transfer learning scenarios, which is called transferable discriminative dimensionality reduction (TDDR). By resolving an objective function that encourages the separation of the domain-merged data and penalizes the distance between source and target distributions, we can find a low-dimensional latent space which guarantees not only the discrimination of projected samples, but also the transferability to enable later classification or regression models constructed in the source domain to generalize well to the target domain. In the experiments, we firstly analyze the perspective of transfer learning in brain-computer interface (BCI) research and then test TDDR on two real datasets from BCI applications. The experimental results show that the TDDR method can learn a low-dimensional latent feature space where the source models can perform well in the target domain.

Transfer learning, serving as one of the most important research directions in machine learning, has been studied in various fields in recent years. In this paper, we integrate the theory of multi-view learning into transfer learning and propose a new algorithm named Multi-View Transfer Learning with Adaboost (MV-TL Adaboost). Different from many previous works on transfer learning, we not only focus on using the labeled data from one task to help to learn another task, but also consider how to transfer them in different views synchronously. We regard both the source and target task as a collection of several constituent views and each of these two tasks can be learned from every views at the same time. Moreover, this kind of multi-view transfer learning is implemented with adaboost algorithm. Furthermore, we analyze the effectiveness and feasibility of MV-TL Adaboost. Experimental results also validate the effectiveness of our proposed approach.

Extreme energy ratio (EER) is a recently proposed feature extractor to learn spatial filters for electroencephalogram (EEG) signal classification. It is theoretically equivalent and computationally superior to the common spatial patterns (CSP) method which is a widely used technique in brain-computer interfaces (BCIs). However, EER may seriously overfit on small training sets due to the presence of large noise. Moreover, it is a totally supervised method that cannot take advantage of unlabeled data. To overcome these limitations, we propose a regularization constraint utilizing local temporal information of unlabeled trails. It can encourage the temporal smoothness of source signals discovered, and thus alleviate their tendency to overfit. By combining this regularization trick with the EER method, we present a semi-supervised feature extractor termed semi-supervised extreme energy ratio (SEER). After solving two eigenvalue decomposition problems, SEER recovers latent source signals that not only have discriminative energy features but also preserve the local temporal structure of test trails. Compared to the features found by EER, the energy features of these source signals have a stronger generalization ability, as shown by the experimental results. As a nonlinear extension of SEER, we further present the kernel SEER and provide the derivation of its solutions.

Distance metric learning is a powerful tool to improve performance in classification, clustering and regression tasks. Many techniques have been proposed for distance metric learning based on convex programming, kernel learning, dimension reduction and large margin. The recently proposed large margin nearest neighbor classification (LMNN) improves the performance of k-nearest neighbors classification (k-nn) by a learned global distance metric. However, it does not consider the locality of data distributions. We demonstrate a novel local distance metric learning method called hierarchical distance metric learning (HDM) which first builds a hierarchical structure by grouping data points according to the overlapping ratios defined by us and then learns distance metrics sequentially. In this paper, we combine HDM with LMNN and further propose a new method named hierarchical distance metric learning for large margin nearest neighbor classification (HLMNN). Experiments are performed on many artificial and real-world data sets. Comparisons with the traditional k-nn and the state-of-the-art LMNN show the effectiveness of the proposed HLMNN.

Co-training is a multiview semi-supervised learning algorithm to learn from both labeled and unlabeled data, which iteratively adopts a classifier trained on one view to teach the other view using some confident predictions given on unlabeled examples. However, as it does not examine the reliability of the labels provided by classifiers on either view, co-training might be problematic. Even very few inaccurately labeled examples can deteriorate the performance of learned classifiers to a large extent. In this paper, a new method named robust co-training is proposed, which integrates canonical correlation analysis (CCA) to inspect the predictions of co-training on those unlabeled training examples. CCA is applied to obtain a low-dimensional and closely correlated representation of the original multiview data. Based on this representation the similarities between an unlabeled example and the original labeled examples are determined. Only those examples whose predicted labels are consistent with the outcome of CCA examination are eligible to augment the original labeled data. The performance of robust co-training is evaluated on several different classification problems where encouraging experimental results are observed.

This paper proposes a new variational approximation for infinite mixtures of Gaussian processes. As an extension of the single Gaussian process regression model, mixtures of Gaussian processes can characterize varying covariances or multimodal data and reduce the deficiency of the computationally cubic complexity of the single Gaussian process model. The infinite mixture of Gaussian processes further integrates a Dirichlet process prior to allowing the number of mixture components to automatically be determined from data. We use variational inference and a truncated stick-breaking representation of the Dirichlet process to approximate the posterior of hidden variables involved in the model. To fix the hyperparameters of the model, the variational EM algorithm and a greedy algorithm are employed. In addition to presenting the variational infinite-mixture model, we apply it to the problem of traffic flow prediction. Experiments with comparisons to other approaches show the effectiveness of the proposed model.

Distance metric learning has exhibited its great power to enhance performance in metric related pattern recognition tasks. The recent large margin nearest neighbor classification (LMNN) improves the performance of k-nearest neighbor classification by learning a global distance metric. However, it does not consider the locality of data distributions, which is crucial in determining a proper metric. In this paper, we propose a novel local distance metric learning method called hierarchical LMNN (HLMNN) which first builds a hierarchical structure by grouping data points according to the overlapping ratios defined by us and then learns distance metrics sequentially. Experimental results on real-world data sets including comparisons with the traditional k-nearest neighbor and the state-of-the-art LMNN show the effectiveness of the proposed HLMNN.

Support vector machines (SVMs) are effective kernel methods to solve pattern recognition problems. Traditionally, they adopt a single kernel chosen beforehand, which makes them lack flexibility. The recent multiple kernel learning (MKL) overcomes this issue by optimizing over a linear combination of kernels. Despite its success, MKL neglects useful information generated from the nonlinear interaction of different kernels. In this paper, we propose SVMs based on the nonlinear combination of multiple kernels (NCMK) which surmounts the drawback of previous MKL by the potential to exploit more information. We show that our method can be formulated as a semi-definite programming (SDP) problem then solved by interior-point algorithms. Empirical studies on several data sets indicate that the presented approach is very effective.

A visual attention based approach is proposed to extract texts from complicated background in camera-based images. First, it applies the simplified visual attention model to highlight the region of interest (ROI) in an input image and to yield a map, named the VA map, consisting of the ROIs. Second, an edge map of image containing the edge information of four directions is obtained by Sobel operators. Character areas are detected by connected component analysis and merged into candidate text regions. Finally, the VA map is employed to confirm the candidate text regions. The experimental results demonstrate that the proposed method can effectively extract text information and locate text regions contained in camera-based images. It is robust not only for font, size, color, language, space, alignment and complexity of background, but also for perspective distortion and skewed texts embedded in images.

Ensemble learning plays an important role in pattern recognition. It combines multiple generated models to solve learning problems, such as classification, regression and feature selection. Existing ensemble methods combine classifiers which are generated and run in parallel. In this paper, a novel ensemble learning approach for semi-supervised learning is proposed. Different from existing ensemble methods, classifiers used for prediction are not generated in parallel. Instead, they are generated evolutionarily in the process of iterative semi-supervised learning. As we know, in the iterative semi-supervised learning the labeled set is increased gradually, based on which multiple classifiers would be generated. This evolutionary ensemble tries to combine the classifiers generated evolutionarily in the process of semi-supervised learning to tackle the learning problems. Compared to traditional semi-supervised learning, the proposed method can generate multiple classifiers with no added training time. Some classifier combining rules are also evaluated in this paper. Moreover, the single-view scenario is generalized to the scenario of multiple-view for a wide application. In order to assess the proposed method, four experiments including single-view and multiple-view problems are implemented. The empirical results indicate that our proposed method can significantly improve the accuracy and reliability of semi-supervised learning.

Most semi-supervised learning methods assume there are a number of labeled data available in order to learn a classifier which then exploits a large set of unlabeled data. However, for some applications, there are only extremely spare labeled examples attainable (say, one example per category). In this case, these semi-supervised learning methods can not work. In this paper, a new method for seeking more examples with high reliable labels based on the limited labeled data is proposed. By investigating the correlation between different views through canonical correlation analysis, our method can launch semi-supervised learning using only one labeled example from each class. Experiments on text classification show the effectiveness of the proposed method.

This paper introduces an ensemble approach for electroencephalogram (EEG) signal classification, which aims to overcome the instability of the Fisher discriminant feature extractor for brain-computer interface (BCI) applications. Through the random selection of electrodes from candidate electrodes, multiple individual classifiers are constructed. In a feature subspace determined by a couple of randomly selected electrodes, principal component analysis (PCA) is first used to implement dimensionality reduction. Successively Fisher discriminant is adopted for feature extraction, and a Bayesian classifier with a Gaussian mixture model (GMM) is trained to carry out classification. The outputs from all the individual classifiers are combined to give a final label. Experiments with real EEG signals taken from a BCI indicate the validity of the proposed random electrode selection (RES) approach.

Classification of time-varying electrophysiological signals is an important problem in the development of brain-computer interfaces (BCIs). Designing adaptive classifiers is a potential way to address this task. In this paper, Bayesian classifiers with Gaussian mixture models (GMMs) are adopted as the decision rule to classify electroencephalogram (EEG) signals. The stochastic approximation method (SAM) is used as the specific gradient descent method for updating the parameters of mean values and covariance matrices in the distribution of GMMs, where the parameters are simultaneously updated in a batch mode. Experimental results using data from a BCI show that the stochastic approximation method is effective for EEG classification tasks.

Biometrics based on electroencephalogram (EEG) signals is an emerging research topic. Several recent results have shown its feasibility and potential for personal identification. However, they all use a single task (e.g., signals recorded during imagination of repetitive left hand movements or during resting with eyes open) for classifier design and subsequent identification. In contrast with this, in this paper multiple related tasks are used simultaneously for classifier learning. This mechanism has the advantage of integrating information from extra tasks and thus hopefully can guide classifier learning in a hypothesis space more effectively. Experimental results on EEG-based personal identification show the effectiveness of the proposed multitask learning approach.

Traditional neural network approaches for traffic flow forecasting are usually single task learning (STL) models, which do not take advantage of the information provided by related tasks. In contrast to STL, multitask learning (MTL) has the potential to improve generalization by transferring information in training signals of extra tasks. In this paper, MTL based neural networks are used for traffic flow forecasting. For neural network MTL, a backpropagation (BP) network is constructed by incorporating traffic flows at several contiguous time instants into an output layer. Nodes in the output layer can be seen as outputs of different but closely related STL tasks. Comprehensive experiments on urban vehicular traffic flow data and comparisons with STL show that MTL in BP neural networks is a promising and effective approach for traffic flow forecasting.

Energy is very important in electroencephalogram (EEG) signal classification. In this paper, a criterion called extreme energy difference (EED) is devised, which is a discriminative objective function to guide the process of spatially filtering EEG signals. The energy of the filtered EEG signals has the optimal discriminative capability under the EED criterion, and therefore EED can be considered as a feature extractor. The solution which optimizes the EED criterion is presented in this paper and according to experimental results, EED is a promising method for extracting energy features in EEG signal classification.

Traffic flow forecasting is an important issue for the application of Intelligent Transportation Systems. Due to practical limitations, traffic flow data may be incomplete (partially missing or substantially contaminated by noises), which will aggravate the difficulties for traffic flow forecasting. In this paper, a new approach, termed the selective random subspace predictor (SRSP), is developed, which is capable of implementing traffic flow forecasting effectively whether incomplete data exist or not. It integrates the entire spatial and temporal traffic flow information in a transportation network to carry out traffic flow forecasting. To forecast the traffic flow at an object road link, the Pearson correlation coefficient is adopted to select some candidate input variables that compose the selective input space. Then, a number of subsets of the input variables in the selective input space are randomly selected to, respectively, serve as specific inputs for prediction. The multiple outputs are combined through a fusion methodology to make final decisions. Both theoretical analysis and experimental results demonstrate the effectiveness and robustness of the SRSP for traffic flow forecasting, whether for complete data or for incomplete data

A new approach based on Bayesian networks for traffic flow forecasting is proposed. In this paper, traffic flows among adjacent road links in a transportation network are modeled as a Bayesian network. The joint probability distribution between the cause nodes (data utilized for forecasting) and the effect node (data to be forecasted) in a constructed Bayesian network is described as a Gaussian mixture model (GMM) whose parameters are estimated via the competitive expectation maximization (CEM) algorithm. Finally, traffic flow forecasting is performed under the criterion of minimum mean square error (mmse). The approach departs from many existing traffic flow forecasting models in that it explicitly includes information from adjacent road links to analyze the trends of the current link statistically. Furthermore, it also encompasses the issue of traffic flow forecasting when incomplete data exist. Comprehensive experiments on urban vehicular traffic flow data of Beijing and comparisons with several other methods show that the Bayesian network is a very promising and effective approach for traffic flow modeling and forecasting, both for complete data and incomplete data.

The classification of electroencephalographic (EEG) signals is an important issue in the ongoing research of brain-computer interface (BCI) technology. One such BCI uses slow cortical potential measures to infer user intent from the original brain activity. Seven features based on the standard low-level signal properties are evaluated for their ability to classify brain activities, and thus make up for the scarcity of signal features for the current EEG signal categorization. In addition, a paradigm is proposed to select effective low-level features for EEG signal classification. Combining the features selected by the paradigm with the DC value of slow cortical potentials for categorization based on a Bayesian classifier, we obtained significant improvement on classification accuracy for data set Ia of the BCI competition 2003, which is a typical representative of one kind of BCI data.