Open Set Recognition (OSR) aims to accurately classify known classes and effectively reject unknown classes in open-world scenarios, which is essential for building safe and reliable intelligent systems. Existing studies have shown that auxiliary data-based methods—utilizing outlier exposure or prompt-based guidance to strengthen OSR models—have achieved significant improvements. However, these methods are highly sensitive to the selection of data. Considering the demand for domain expertise and the cost implications, synthesizing data from known samples becomes a preferable alternative. To achieve performance on par with methods that incorporate auxiliary data, it is essential to address the following key challenges: (1) obtaining meaningful pseudo-unknown samples and prompts without manual intervention, and (2) establishing a reliable recognition mechanism during the inference phase. In this paper, we propose a novel Dual-Space Prompt Learning with Context Bias Decoupling (DSPL) method to address the above issues. Specifically, we symmetrically model the same number of unknown classes as known classes to achieve a balanced class distribution in the embedding space. Learnable textual templates are designed for both types of data. Meanwhile, boundary samples are synthesized and treated as pseudo-unknown samples by decoupling discriminative and co-occurring features. Additionally, unknown sample detection is performed by integrating the maximum posterior over known and unknown classes, thereby extending the advantages of symmetric modeling from the training phase to inference. Extensive experiments demonstrate that the proposed method DSPL achieves state-of-the-art performance compared with the most advanced OSR methods.

Multi-label learning focuses on tasks where each instance is associated with multiple labels. Due to the high cost of obtaining accurate annotations, real-world multi-label datasets often contain noisy labels from crowdsourcing or automated annotation. Noisy multi-label learning has hence been studied to address the label noise problem. However, existing noisy multi-label methods struggle to handle instance-dependent noise (IDN), which is a complex and common type of label noise in practical applications, severely affecting the reliability of multi-label learning. In contrast, existing methods designed for single-label IDN cannot be directly applied to multi-label data. To address these challenges, we propose Robust multi-label learning with Instance-Dependent label noisE (RIDE), a framework for multi-label learning with IDN. Specifically, RIDE first decomposes the observed label matrix into clean and noisy components via a joint low-rank and sparse decomposition. Secondly, a linear sparse mapping from feature space to label space is introduced to explicitly model how instance features induce IDN. Thirdly, to further improve the denoising accuracy, RIDE estimates a noise suppression coefficient for each sample, which weights the sparse regularization term of the noise decomposition. In addition, theoretical analysis is provided to derive upper bounds on the noise estimation and generalization errors. Extensive experiments on benchmark multi-label datasets with varying noise rates show that RIDE outperforms state-of-the-art methods. The code is available at https://github.com/View5U/RIDE.

Knowledge Graphs (KGs) often suffer from link incompleteness, limiting their performance in downstream tasks. Knowledge graph completion (KGC) has emerged to complete the missing links between entities, so to integrate new knowledge into existing KGs. Unfortunately, traditional KGC models mainly focus on static KGs with a fixed set of entities and relations, thereby struggling in generalization to dynamic KGs with possible entity drift. In this paper, we propose a Link Completion model for dynamic KGs with Entity Drift (LCED) to solve the issue. Unlike traditional link completion models, LCED can generate accurate embeddings for drifted entities, without imposing any assumption on their time or rule characteristic. The drifted entities are incorporated into the KG where their links to existing entities are predicted in an incremental fashion, without retraining the entire KG for computational efficiency. Extensive experiments conducted on benchmark datasets demonstrate the effectiveness of our proposed LCED in link completion for dynamic KGs by outperforming its state-of-the-art competitors. Code and datasets are available at https://github.com/field658/LCED.

Cold-start recommendation is a well-known problem in practical application scenarios. Generating reliable recommendations can be challenging when interactions are typically sparse. To mitigate the cold-start problem, some methods incorporate auxiliary information about users and items, and others adopt meta-learning to improve recommendation accuracy. However, these approaches overlook the fact that items are interdependent and likely to be related or similar. Moreover, user preference distributions in the meta-training and meta-testing phases are different in the cold-start scenario. To address these problems, we present a novel strategy called Causal Meta-learning with Multi-view Graphs (CausalMMG). Specifically, we first construct multi-view item-item graphs to explore the correlations and similarities between items from multiple perspectives. A multi-view item representer is then used to learn item representations, exploiting graph convolution neural networks to capture the structure of these different item–item graphs. We then resort to the structural causal models of causal inference and further develop a causality-enhanced bi-level adaptive meta-learner to eliminate bias caused by the different distributions of user preferences. Moreover, the meta-learner learns the user preferences for items in different orders through hierarchical and task-level adaptations. Finally, we evaluate CausalMMG on several real-world datasets, demonstrating its effectiveness in various scenarios. The results show that the proposed CausalMMG is significantly superior to competitive baseline methods for cold-start recommendation on all datasets, highlighting the importance of incorporating the multiple relationships between items and modeling different user preference distributions in recommender systems.

Knowledge Graphs (KGs) often suffer from incompleteness and this issue motivates the task of Knowledge Graph Completion (KGC). Traditional KGC models mainly concentrate on static KGs with a fixed set of entities and relations, or dynamic KGs with temporal characteristics, faltering in their generalization to constantly evolving KGs with possible irregular entity drift. Thus, in this paper, we propose a novel link prediction model based on the embedding representation to handle the incompleteness of KGs with entity drift, termed as DCEL. Unlike traditional link prediction, DCEL could generate precise embeddings for drifted entity without imposing any regular temporal characteristic. The drifted entity is added into the KG with its links to the existing entity predicted in an incremental fashion with no requirement to retrain the whole KG for computational efficiency. In terms of DCEL model, it fully takes advantages of unstructured textual description, and is composed of four modules, namely MRC (Machine Reading Comprehension), RCAA (Relation Constraint Attentive Aggregator), RSA (Relation Specific Alignment) and RCEO (Relation Constraint Embedding Optimization). Specifically, the MRC module is first employed to extract short texts from long and redundant descriptions. Then, RCAA is used to aggregate the embeddings of textual description of drifted entity and the pre-trained word embeddings learned from corpus to a single text-based entity embedding while shielding the impact of noise and irrelevant information. After that, RSA is applied to align the text-based entity embedding to graph-based space to obtain the corresponding graph-based entity embedding, and then the learned embeddings are fed into the gate structure to be optimized based on the RCEO to improve the accuracy of representation learning. Finally, the graph-based model TransE is used to perform link prediction for drifted entity. Extensive experiments conducted on benchmark datasets in terms of evaluation protocols of MRR and Hits@ \(k\) reveal the superiority of DCEL model compared to its SOTAs.

Real-world applications have produced massive short text streams. Contrary to the traditional normal texts, they present the characteristics such as short length, only having few labeled data, high-velocity, high-volume and dynamic data distributions, which deteriorate the issues of data sparseness, label missing and concept drift. Obviously, it is a huge challenge for existing short text (stream) classification algorithms due to the poor effectiveness, where they always assume all short texts are completely labeled and little attention is paid on the concept drift issue hidden in short text streams. Therefore, we propose a novel semi-supervised short text steam classification method based on the drift-aware incremental deep learning ensemble model. Specifically, with the sliding window mechanism, we firstly fuse three types of statistical, semantic and structure information to solve the data sparseness issue. Secondly, a semi-supervised incremental deep learning ensemble model based on GCN and the refined LSTM is developed to adapt to the high-volume, high-velocity and label missing short text streams. Thirdly, a label-probability distribution based concept drift detector is introduced to distinguish concept drifts. Finally, as compared with eleven well-known classification methods, extensive experiments demonstrate the effectiveness of the proposed method in the handling of short text streams with limited labeled data.

Time series data are ubiquitous in a variety of disciplines. Early classification of time series, which aims to predict the class label of a time series as early and accurately as possible, is a significant but challenging task in many time-sensitive applications. Existing approaches mainly utilize heuristic stopping rules to capture stopping signals from the prediction results of time series classifiers. However, heuristic stopping rules can only capture obvious stopping signals, which makes these approaches give either correct but late predictions or early but incorrect predictions. To tackle the problem, we propose a novel second-order confidence network for early classification of time series, which can automatically learn to capture implicit stopping signals in early time series in a unified framework. The proposed model leverages deep neural models to capture temporal patterns and outputs second-order confidence to reflect the implicit stopping signals. Specifically, our model exploits the data not only from a time step but also from the probability sequence to capture stopping signals. By combining stopping signals from the classifier output and the second-order confidence, we design a more robust trigger to decide whether or not to request more observations from future time steps. Experimental results show that our approach can achieve superior results in early classification compared to state-of-the-art approaches.

Conversational recommender systems (CRS) have garnered significant attention in academia and industry because of their ability to capture user preferences via system questions and user responses. Typically, in a CRS, reinforcement learning (RL) is utilized to determine the optimal timing for requesting attribute information or suggesting items. However, existing methods consider user-preferred attributes independently and ignore that attributes may be of different importance to the same user, in the attribute and item selection phases, which limits the accuracy and interpretability of CRS. Inspired by this, we propose deep conversational path reasoning (DeepCPR), which involves constructing a reasoning path on a graph with a series of user-favored attributes. It utilizes the attention mechanism to thoroughly examine the connections between these attributes and provide improved explanations for which attributes to inquire about or which items to recommend. In DeepCPR, two deep-learning-based modules are proposed to realize attribute and item selection. In the first module, the sequence of attributes confirmed by the user in conversation is encoded with a gated graph neural network to obtain the user’s long-term preference using a self-attention mechanism for the selection of candidate attributes. In the second module, a self-attention approach with more appropriate strategies is developed to dynamically select candidate items. In addition, to achieve fine-grained user preference modeling, a recurrent neural network is employed to aggregate the sequence of attributes that interact with the users. Numerous experimental evaluations conducted on four real CRS datasets show that the proposed method significantly outperforms existing advanced methods in terms of conversational recommendations.

Local-to-global learning approach plays an essential role in Bayesian network (BN) structure learning. Existing local-to-global learning algorithms first construct the skeleton of a DAG (directed acyclic graph) by learning the MB (Markov blanket) or PC (parents and children) of each variable in a dataset, then orient edges in the skeleton. However, existing MB or PC learning methods are often computationally expensive especially with a large-sized BN, resulting in inefficient local-to-global learning algorithms. To tackle the problem, in this article, we link feature selection with local BN structure learning and develop an efficient local-to-global learning approach using filtering feature selection. Specifically, we first analyze the rationale of the well-known Minimum-Redundancy and Maximum-Relevance (MRMR) feature selection approach for learning a PC set of a variable. Based on the analysis, we propose an efficient F2SL (feature selection-based structure learning) approach to local-to-global BN structure learning. The F2SL approach first employs the MRMR approach to learn the skeleton of a DAG, then orients edges in the skeleton. Employing independence tests or score functions for orienting edges, we instantiate the F2SL approach into two new algorithms, F2SL-c (using independence tests) and F2SL-s (using score functions). Compared to the state-of-the-art local-to-global BN learning algorithms, the experiments validated that the proposed algorithms in this article are more efficient and provide competitive structure learning quality than the compared algorithms.

Directed acyclic graph (DAG) learning plays a key role in causal discovery and many machine learning tasks. Learning a DAG from high-dimensional data always faces scalability problems. A local-to-global DAG learning approach can be scaled to high-dimensional data, however, existing local-to-global DAG learning algorithms employ either the AND-rule or the OR-rule for constructing a DAG skeleton. Simply using either rule, existing local-to-global methods may learn an inaccurate DAG skeleton, leading to unsatisfactory DAG learning performance. To tackle this problem, in this paper, we propose anAdaptiveDAGLearning (ADL) algorithm. The novel contribution of ADL is that it can simultaneously and adaptively use the AND-rule and the OR-rule to construct an accurate global DAG skeleton. We conduct extensive experiments on both benchmark and real-world datasets, and the experimental results show that ADL is significantly better than some existing local-to-global and global DAG learning algorithms.

Multi-label data streams such as Web texts and images have been popular on the Web. These data present the characteristics of multiple label, high dimensionality, high volume, high velocity and especial concept drift etc. Thus, multi-label data stream classification is a very challenging and significant task especially in the handling of high-dimensional data with concept drifts. However, this challenge has received little attention from the research community. Therefore, we propose the max-relevance and min-redundancy based algorithm adaptation approach for the efficient and effective classification on multi-label data streams with high-dimensional attributes and concept drifts .11.Source codes and data sets are available at below. https://github.com/peipeilihfut/MLStreamClassificationIn order to reduce the impact from the high-dimensional data with noisy attributes, we first refine the minimal-redundancy-maximal-relevance criterion based on mutual information to select qualified features in multi-label data streams. Secondly, we propose the data distribution based concept drifting detection approach to distinguish concept drifts hidden in data streams. Finally, we build an incremental ensemble classification model for efficiently classifying multi-label data streams. Extensive studies show that our approach can get optimal subsets of features while maintaining a good performance in the multi-label classification, as compared to several state-of-the-art multi-label feature selection algorithms using two efficient multi-label classification methods as base classifiers. Meanwhile, our approach is superior to three well-known multi-label data stream classification approaches in the effectiveness and efficiency.

A huge amount of texts available on the World Wide Web presents an unprecedented opportunity for information extraction (IE). One important assumption in IE is that frequent extractions are more likely to be correct. Sparse IE is hence a challenging task because no matter how big a corpus is, there are extractions supported by only a small amount of evidence in the corpus. However, there is limited research on sparse IE, especially in the assessment of the validity of sparse IEs. Motivated by this, we introduce a lightweight, explicit semantic approach for assessing sparse IE. 1 We first use a large semantic network consisting of millions of concepts, entities, and attributes to explicitly model the context of any semantic relationship. Second, we learn from three semantic contexts using different base classifiers to select an optimal classification model for assessing sparse extractions. Finally, experiments show that as compared with several state-of-the-art approaches, our approach can significantly improve the F -score in the assessment of sparse extractions while maintaining the efficiency.

Short text stream classification is a challengingand significant task due to the characteristics of short length, weak signal, high velocity and especially topic drifting in short text stream. However, this challenge has received little attention from the research community. Motivated by this, we propose a new feature extension approach for short text stream classification using a large scale, general purpose semantic network obtained from a web corpus. Our approach is built on an incremental ensemble classification model. First, in terms of the open semantic network, we introduce more semantic contexts in short texts to make up of the data sparsity. Meanwhile, we disambiguate terms by their semantics to reduce the noise impact. Second, to effectively track hidden topic drifts, we propose a concept cluster based topic drifting detection method. Finally, extensive experiments demonstratethat our approach can detect topic drifts effectively compared to several well-known concept drifting detection methods in data streams. Meanwhile, our approach can perform best in the classification of text data streams compared to several stateof-the-art short text classification approaches.

Measuring semantic similarity between two terms is essential for a variety of text analytics and understanding applications. Currently, there are two main approaches for this task, namely the knowledge based and the corpus based approaches. However, existing approaches are more suitable for semantic similarity between words rather than the more general multi-word expressions (MWEs), and they do not scale very well. Contrary to these existing techniques, we propose an efficient and effective approach for semantic similarity using a large scale semantic network. This semantic network is automatically acquired from billions of web documents. It consists of millions of concepts, which explicitly model the context of semantic relationships. In this paper, we first show how to map two terms into the concept space, and compare their similarity there. Then, we introduce a clustering approach to orthogonalize the concept space in order to improve the accuracy of the similarity measure. Finally, we conduct extensive studies to demonstrate that our approach can accurately compute the semantic similarity between terms of MWEs and with ambiguity, and significantly outperforms 12 competing methods under Pearson Correlation Coefficient. Meanwhile, our approach is much more efficient than all competing algorithms, and can be used to compute semantic similarity in a large scale.

Online selection of dynamic features has attracted intensive interest in recent years. However, existing online feature selection methods evaluate features individually and ignore the underlying structure of a feature stream. For instance, in image analysis, features are generated in groups which represent color, texture, and other visual information. Simply breaking the group structure in feature selection may degrade performance. Motivated by this observation, we formulate the problem as an online group feature selection. The problem assumes that features are generated individually but there are group structures in the feature stream. To the best of our knowledge, this is the first time that the correlation among streaming features has been considered in the online feature selection process. To solve this problem, we develop a novel online group feature selection method named OGFS. Our proposed approach consists of two stages: online intra-group selection and online inter-group selection. In the intra-group selection, we design a criterion based on spectral analysis to select discriminative features in each group. In the inter-group selection, we utilize a linear regression model to select an optimal subset. This two-stage procedure continues until there are no more features arriving or some predefined stopping conditions are met. Finally, we apply our method to multiple tasks including image classification and face verification. Extensive empirical studies performed on real-world and benchmark data sets demonstrate that our method outperforms other state-of-the-art online feature selection methods.

One important assumption of information extraction is that extractions occurring more frequently are more likely to be correct. Sparse information extraction is challenging because no matter how big a corpus is, there are extractions supported by only a small amount of evidence in the corpus. A pioneering work known as REALM learns HMMs to model the context of a semantic relationship for assessing the extractions. This is quite costly and the semantics revealed for the context are not explicit. In this work, we introduce a lightweight, explicit semantic approach for sparse information extraction. We use a large semantic network consisting of millions of concepts, entities, and attributes to explicitly model the context of semantic relationships. Experiments show that our approach improves the F-score of extraction by at least 11.2% over state-of-the-art, HMM based approaches while maintaining more efficiency.

Computing semantic similarity between two terms is essential for a variety of text analytics and understanding applications. However, existing approaches are more suitable for semantic similarity between words rather than the more general multi-word expressions (MWEs), and they do not scale very well. Therefore, we propose a lightweight and effective approach for semantic similarity using a large scale semantic network automatically acquired from billions of web documents. Given two terms, we map them into the concept space, and compare their similarity there. Furthermore, we introduce a clustering approach to orthogonalize the concept space in order to improve the accuracy of the similarity measure. Extensive studies demonstrate that our approach can accurately compute the semantic similarity between terms with MWEs and ambiguity, and significantly outperforms 12 competing methods.

Tracking recurring concept drifts is a significant issue for machine learning and data mining that frequently appears in real-world stream classification problems. It is a challenge for many streaming classification algorithms to learn recurring concepts in a data stream environment with unlabeled data, and this challenge has received little attention from the research community. Motivated by this challenge, this article focuses on the problem of recurring contexts in streaming environments with limited labeled data. We propose a semi-supervised classification algorithm for data streams with REcurring concept Drifts and Limited LAbeled data, called REDLLA, in which a decision tree is adopted as the classification model. When growing a tree, a clustering algorithm based on k -means is installed to produce concept clusters and unlabeled data are labeled in the method of majority-class at leaves. In view of deviations between history and new concept clusters, potential concept drifts are distinguished and recurring concepts are maintained. Extensive studies on both synthetic and real-world data confirm the advantages of our REDLLA algorithm over three state-of-the-art online classification algorithms of CVFDT, DWCDS, and CDRDT and several known online semi-supervised algorithms, even in the case with more than 90% unlabeled data.