Traditional domain-specific causal discovery relies on expert knowledge to guide the data-based structure learning process, thereby improving the reliability of recovered causality. Recent studies have shown promise in using the Large Language Model (LLM) as causal experts to construct autonomous expert-guided causal discovery systems through causal reasoning between pairwise variables. However, their performance is hampered by inaccuracies in aligning LLM-derived causal knowledge with the actual causal structure. To address this issue, this paper proposes a novel LLM-driven causal discovery framework that limits LLM’s prior within a reliable range. Instead of pairwise causal reasoning that requires both precise and comprehensive output results, the LLM is directed to focus on each single aspect separately. By combining these distinct causal insights, a unified set of structural constraints is created, termed a harmonized prior, which draws on their respective strengths to ensure prior accuracy. On this basis, we introduce plug-and-play integrations of the harmonized prior into mainstream categories of structure learning methods, thereby enhancing their applicability in practical scenarios. Evaluations on real-world data demonstrate the effectiveness of our approach.

Time series classification (TSC) is crucial in many applications, yet accurately modeling complex time series patterns remains challenging. Model-based TSC strives to aptly model time series by capturing their intrinsic temporal dynamics, deriving effective dynamic representations for classification. Despite significant progress in this domain, existing works are still constrained by a singular and overly simplistic modeling paradigm, which proves inadequate to handle the multiscale hierarchies inherent in time series. Additionally, the prevailing reliance on manual model configuration fails to address the diverse dynamic characteristics across varying data scenarios. In this paper, we amalgamate multiple recurrent reservoirs to devise a model-based Multiscale Temporal Dynamic Learning (MsDL) approach. These reservoirs are endowed with varied recurrent connection skips, ensuring a comprehensive capture of temporal dynamics across different timescales. We also present a multi-objective optimization algorithm, which adaptively configures the memory length of each reservoir, allowing for more accurate time series modeling. This optimization further encourages time series from the same class to look closer, while separating those from different classes, thereby enhancing the category-discriminability. Extensive experiments on public datasets demonstrate that MsDL outperforms the state-of-the-art methods. Additionally, ablation studies confirm that our multiscale design and optimization algorithm effectively enhance classification accuracy.

Causal discovery faces significant challenges as the number of hypotheses grows exponentially with the number of variables. This complexity becomes particularly daunting when dealing with large sets of variables. We introduce a novel divide-and-conquer method that uniquely handles this challenge. The existing division strategies often rely on conditional independency (CI) tests or data-driven clustering to split variables, which can suffer from the typical data scarcity in large-scale settings, thus leading to inaccurate division results. The proposed method overcomes this by implementing a data-independent division strategy, which constructs a prior structure, informed by potential causal relationships identified using a Large Language Model (LLM), to guide recursively dividing variables into sub-sets. This approach avoids the impact of data insufficiency and is robust against potential incompleteness in the prior structure. In the merging phase, we adopt a score-based refinement strategy to address fake causal links caused by hidden variables in sub-sets, which eliminates edges in the intersected parts of sub-sets to optimize the score of local structures. While maintaining both correctness and completeness under the faithfulness assumption, this novel merging approach demonstrates enhanced performance than the conventional CI-test based merging strategy in practical scenarios. Empirical evaluations on various large-scale datasets demonstrate the proposed approach's superior accuracy and efficiency compared to existing causal discovery methods.

Multi-Behavior Recommendation (MBR) aims to model personalized user preferences by integrating diverse interaction behaviors (e.g., page view, favorite, add to cart, purchase). However, latent confounders such as contextual influences and social relationships can obscure the true causal effects in real-world scenarios, thereby confounding the model’s prediction. Although existing MBR research extensively explores behavioral dependencies and heterogeneity, it frequently overlooks the impact of latent confounders, thereby limiting its ability to capture users’ genuine preferences. To address the limitations of existing methods, we identify two key challenges in MBR: (1) how to infer latent confounders, and (2) how to mitigate their influence across multi-behavior interactions. To this end, we propose Causal Variational Inference for Deconfounded (CVID) MBR. CVID employs a variational graph autoencoder to model latent uncertainty in multi-behavior interactions and introduces a confounder inference module to generate behavior-specific latent confounders via variational inference. In the conditional diffusion module, noise is progressively injected during the forward process to simulate the dynamic evolution of user preferences, while the reverse process leverages the inferred latent confounders to guide denoising through back-door adjustment, thereby recovering the true causal effects between multi-behavior interactions and the model’s prediction. Extensive experiments on public multi-behavior datasets demonstrate that CVID consistently outperforms state-of-the-art baselines in mitigating confounding effects and improving recommendation accuracy, validating its effectiveness and superiority.

To generate novel views of artistic portraits from a source face image and a reference style exemplar in real-time, we propose a novel structure, namely DualStyle3D. In our DualStyle3D, we introduce a novel "encoder-decoder'' structure to reduce the generation time cost by directly taking face images and exemplars as inputs while eliminating the time-consuming 3D-GAN inversion process. Moreover, to manage the style of the generated portrait, we propose a StyleAdaption3D module to control the color tune and the structure of the synthesized artistic portrait based on three different granular features. Furthermore, we employ a progressive training strategy to distill knowledge from both pretrained 3D-GAN and 2D artistic GAN to achieve robust convergence. Our experiments demonstrate that the time required to synthesize an artistic portrait from an input portrait using our DualStyle3D is 0.08% of CIPS-3D, 0.16% of 3DAvatarGAN, and 30.56% of DeformToon3D on Nvidia RTX 3090, showcasing its potential for real-time, exemplar-based portrait style transfer.

With the personal partial rankings from agents over a subset of alternatives, the goal of preference completion is to infer the agent’s personalized preference over all alternatives including those the agent has not yet handled from uncertain preference of third parties. By combining the partial rankings of the target agent and the partial rankings from third parties to settle some disagreement with three-way preference completion, which includes a general strategy, an optimal strategy, and a pessimistic strategy, it forms the weighted preference graph. Technically, to settle the disagreement and obtain the completed preference of the target agent in the weighted preference graph, maximum likelihood estimation (MLE) under Mallows is proposed and validated theoretically by removing edges with the minimum weight in the weighted preference graph. However, it is not easy to locate the edges with the minimum weight efficiently in a big graph. Hence, an optimal MLE algorithm and three greedy MLE algorithms are proposed to process the MLE. Furthermore, these proposed algorithms are experimentally validated and compared with each other by both the synthetic dataset and the Flixter dataset.

Graph pattern matching (GPM) is widely used in social network analysis, such as expert finding, social group query, and social position detection. Technically, GPM is to find matched subgraphs that meet the requirements of pattern graphs in big social networks. In the application of expert community location, the nodes in the pattern graph and data graph represent expert entities, and the edges represent previous cooperations between them. However, the existing GPM methods focus on shortening the matching time and without considering the preference of the decision maker (DM), which makes it difficult for the DM to find ideal teams from numerous matches to complete the assigned task. In this article, as for the process of graph pattern matching and rematching, with a preferred expert set, i.e., the DM hopes that one or more experts in this set will appear in matched subgraphs, we propose a Dual Simulation-based Edge Sequencing-oriented Semi-Supervised GPM method (DsEs-ssGPM). In addition, considering a preferred expert set and a dispreferred expert set together, the DM hopes that experts in the dispreferred expert set will not appear in final matches, so we have the DsEs-ssGPM+ method. Technically, these DsEs-ssGPM methods conduct the matching process from the preferred expert set during dual simulation-based edge sequencing, and based on the edge sequence, these edges are searched recursively. Especially, as for the rematching process, when the preferred and/or the dispreferred expert sets change continuously, to process the GPM again is unnecessary and it is possible to revise the previous matched results partially with DsEs-ssGPM methods. Experiments on four large datasets demonstrate the effectiveness, efficiency and stability of our proposed DsEs-ssGPM methods, and the necessity of introducing an edge sequencing mechanism.

Incomplete instances with various missing values in real-world scenes have brought challenges to the classification tasks. Many existing methods impute the incomplete instances based on their neighbouring instances before classification. However, the construction of such neighbourhood relationships in the original data space may become unreliable since missing values could disturb the traditional distance measurement. Moreover, these methods decouple the imputation from the classification, which makes it difficult for the former to learn from the supervised information, resulting in sub-optimal performance. To this end, this paper proposes graph neural networks for missing values classification (GNN4MV), which directly classify the incomplete instances based on their neighbourhood relationships constructed in a novel task-driven metric space. Specifically, the supervised information is taken as additional guidance in a task-driven metric space to reduce the impact of the missing values for neighbourhood relationship construction. Furthermore, a novel neighbourhood graph convolutional network is proposed in GNN4MV, which enables direct classification of the incomplete instances without imputation by utilizing the graph topology in the constructed neighbourhood relationships. Experiments on real-world datasets demonstrate the robustness and effectiveness of the proposed algorithm.

Recent years have witnessed the proliferation of the Functional Causal Model (FCM) for causal learning due to its intuitive representation and accurate learning results. However, existing FCM-based algorithms suffer from the ubiquitous nonlinear relations in time-series data, mainly because these algorithms either assume linear relationships, or nonlinear relationships with additive noise, or do not introduce additional assumptions but can only identify nonlinear causality between two variables. This paper contributes in particular to a practical FCM-based causal learning approach, which can maintain effectiveness for real-world nonstationary data with general nonlinear relationships and unlimited variable scale.Specifically, the non-stationarity of time series data is first exploited with the nonlinear independent component analysis, to discover the underlying components or latent disturbances. Then, the conditional independence between variables and these components is studied to obtain a relation matrix, which guides the algorithm to recover the underlying causal graph. The correctness of the proposal is theoretically proved, and extensive experiments further verify its effectiveness. To the best of our knowledge, the proposal is the first so far that can fully identify causal relationships under general nonlinear conditions.

In the research area of time series classification, the ensemble shapelet transform algorithm is one of state-of-the-art algorithms for classification. However, its high time complexity is an issue to hinder its application since its base classifier shapelet transform includes a high time complexity of a distance calculation and shapelet selection. Therefore, in this paper we introduce a novel algorithm, i.e. short isometric shapelet transform, which contains two strategies to reduce the time complexity. The first strategy of SIST fixes the length of shapelet based on a simplified distance calculation, which largely reduces the number of shapelet candidates as well as speeds up the distance calculation in the ensemble shapelet transform algorithm. The second strategy is to train a single linear classifier in the feature space instead of an ensemble classifier. The theoretical evidences of these two strategies are presented to guarantee a near-lossless accuracy under some preconditions while reducing the time complexity. Furthermore, empirical experiments demonstrate the superior performance of the proposed algorithm.

Due to the interpretability and robustness, Markov boundary (MB) has received much attention and been widely applied to causal feature selection. However, enormous empirical studies show that, existing algorithms achieve outstanding performance only on the standard Bayesian network data. While on the real-world data, they could not identify some of the relevant features since the large conditioning set and the ignored multivariate dependence lead to performance degradation. In this paper, we propose a tolerant MB discovery algorithm (TLMB), which maps the feature space and target space to a reproducing kernel Hilbert space through the conditional covariance operator, to measure the causal information carried by a feature. Specifically, TLMB uses a score function to filter the redundant features first and then minimize the trace of the conditional covariance operator, where both of the score function and the optimization problem work in the reproducing kernel Hilbert space so that TLMB can select features with not only pairwise dependence but also multivariate dependence. Moreover, as a MB-based method, TLMB can automatically determine the number of selected features due to the property of MB.

Multi-task learning aims to learn multiple tasks jointly by sharing information among related tasks such that the generalization performance over different tasks could be improved. Although multi-task learning has been demonstrated to obtain performance gain in comparison with the single task learning, the main challenge that learning what to share with whom is still not fully resolved. In this paper, we propose a robust clustered multi-task learning approach that clusters tasks into several groups by learning the representative tasks. The main assumption behind our approach is that each task can be represented by a linear combination of some representative tasks that can characterize all tasks. The correlation between tasks can be indicated by the corresponding combination coefficient. By imposing a row-sparse constraint on the correlation matrix, our approach could select the representative tasks and encourage information sharing among the related tasks. In addition, the $l_1,2 $-norm is applied to the representation loss to enhance the robustness of our approach. To solve the resulting bi-convex optimization problem, we design an efficient optimization method based on the alternating direction method of multipliers and accelerated proximal gradient method. Finally, experimental results on synthetic and real-world data sets validate the effectiveness of the proposed approach.

In classification tasks, classifiers trained with finite examples might generalize poorly to new data with unknown variance. For this issue, data augmentation is a successful solution where numerous artificial examples are added to training sets. In this article, we focus on the data augmentation for improving the accuracy of action recognition, where action videos are modeled by linear dynamical systems and approximately represented as linear subspaces. These subspace representations lie in a non-Euclidean space, named Grassmann manifold, containing points as orthonormal matrixes. It is our concern that poor generalization may result from the variance of manifolds when data come from different sources or classes. Thus, we introduce infinitely many variant Grassmann manifolds (VGM) subject to a known distribution, then represent each action video as different Grassmann points leading to augmented representations. Furthermore, a prior based on the stability of subspace bases is introduced, so the manifold distribution can be adaptively determined, balancing discrimination and representation. Experimental results of multi-class and multi-source classification show that VGM softmax classifiers achieve lower test error rates compared to methods with a single manifold.

Sparse Bayesian learning is a state-of-the-art supervised learning algorithm that can choose a subset of relevant samples from the input data and make reliable probabilistic predictions. However, in the presence of high-dimensional data with irrelevant features, traditional sparse Bayesian classifiers suffer from performance degradation and low efficiency due to the incapability of eliminating irrelevant features. To tackle this problem, we propose a novel sparse Bayesian embedded feature selection algorithm that adopts truncated Gaussian distributions as both sample and feature priors. The proposed algorithm, called probabilistic feature selection and classification vector machine (PFCVM LP ) is able to simultaneously select relevant features and samples for classification tasks. In order to derive the analytical solutions, Laplace approximation is applied to compute approximate posteriors and marginal likelihoods. Finally, parameters and hyperparameters are optimized by the type-II maximum likelihood method. Experiments on three datasets validate the performance of PFCVM LP along two dimensions: classification performance and effectiveness for feature selection. Finally, we analyze the generalization performance and derive a generalization error bound for PFCVM LP . By tightening the bound, the importance of feature selection is demonstrated.

Buried pipes beneath our city are blood vessels that feed human civilization through the supply of water, gas, electricity, and so on, and mapping the buried pipes has long been addressed as an issue. In this article, a suitable coordinate of the detected area is established, the noisy Ground Penetrating Radar (GPR) and Global Positioning System (GPS) data are analyzed and normalized, and the pipeline is described mathematically. Based on these, the Probabilistic Mixture Model is proposed to map the buried pipes, which takes discrete noisy GPR and GPS data as the input and the accurate pipe locations and directions as the output. The proposed model consists of the Preprocessing, the Pipe Fitting algorithm, the Classification Fitting Expectation Maximization (CFEM) algorithm, and the Angle-limited Hough (Al-Hough) transform. The direction information of the detecting point is added into the measuring of the distance from the point to nearby pipelines, to handle some areas where the pipes are intersected or difficult to classify. The Expectation Maximization (EM) algorithm is upgraded to CFEM algorithm that is able to classify detecting points into different classes, and connect and fit multiple points in each class to get accurate pipeline locations and directions, and the Al-Hough transform provides reliable initializations for CFEM, to some extent, ensuring the convergence of the proposed model. The experimental results on the simulated and real-world datasets demonstrate the effectiveness of the proposed model.

Semi-supervised learning (SSL) concerns the problem of how to improve classifiers’ performance through making use of prior knowledge from unlabeled data. Many SSL methods have been developed to integrate unlabeled data into the classifiers based on either the manifold or cluster assumption in recent years. In particular, the graph-based approaches, following the manifold assumption, have achieved a promising performance in many real-world applications. However, most of them work well on small-scale data sets only and lack probabilistic outputs. In this paper, a scalable graph-based SSL framework through sparse Bayesian model is proposed by defining a graph-based sparse prior. Based on the traditional Bayesian inference technique, a sparse Bayesian SSL algorithm (SBS$^2$L) is obtained, which can remove the irrelevant unlabeled samples and make probabilistic prediction for out-of-sample data. Moreover, in order to scale SBS$^2$L to large-scale data sets, an incremental SBS$^2$L (ISBS$^2$L) is derived. The key idea of ISBS$^2$L is employing an incremental strategy and sequentially selecting parts of unlabeled samples that contribute to the learning instead of using all available unlabeled samples directly. ISBS$^2$L has lower time and space complexities than previous SSL algorithms with the use of all unlabeled samples. Extensive experiments on various data sets verify that our algorithms can achieve comparable classification effectiveness and efficiency with much better scalability. Finally, the generalization error bound is derived based on robustness analysis.

We present novel, efficient, model based kernels for time series data rooted in the reservoir computation framework. The kernels are implemented by fitting reservoir models sharing the same fixed deterministically constructed state transition part to individual time series. The proposed kernels can naturally handle time series of different length without the need to specify a parametric model class for the time series. Compared with most time series kernels, our kernels are computationally efficient. We show how the model distances used in the kernel can be calculated analytically or efficiently estimated. The experimental results on synthetic and benchmark time series classification tasks confirm the efficiency of the proposed kernel in terms of both generalization accuracy and computational speed. This paper also investigates on-line reservoir kernel construction for extremely long time series.

Negative Correlation Learning (NCL) [CHECK END OF SENTENCE], [CHECK END OF SENTENCE] is a neural network ensemble learning algorithm which introduces a correlation penalty term to the cost function of each individual network so that each neural network minimizes its mean-square-error (MSE) together with the correlation. This paper describes NCL in detail and observes that the NCL corresponds to training the entire ensemble as a single learning machine that only minimizes the MSE without regularization. This insight explains that NCL is prone to overfitting the noise in the training set. The paper analyzes this problem and proposes the multiobjective regularized negative correlation learning (MRNCL) algorithm which incorporates an additional regularization term for the ensemble and uses the evolutionary multiobjective algorithm to design ensembles. In MRNCL, we define the crossover and mutation operators and adopt nondominated sorting algorithm with fitness sharing and rank-based fitness assignment. The experiments on synthetic data as well as real-world data sets demonstrate that MRNCL achieves better performance than NCL, especially when the noise level is nontrivial in the data set. In the experimental discussion, we give three reasons why our algorithm outperforms others.

An ensemble is a group of learners that work together as a committee to solve a problem. The existing ensemble learning algorithms often generate unnecessarily large ensembles, which consume extra computational resource and may degrade the generalization performance. Ensemble pruning algorithms aim to find a good subset of ensemble members to constitute a small ensemble, which saves the computational resource and performs as well as, or better than, the unpruned ensemble. This paper introduces a probabilistic ensemble pruning algorithm by choosing a set of ldquosparserdquo combination weights, most of which are zeros, to prune the ensemble. In order to obtain the set of sparse combination weights and satisfy the nonnegative constraint of the combination weights, a left-truncated, nonnegative, Gaussian prior is adopted over every combination weight. Expectation propagation (EP) algorithm is employed to approximate the posterior estimation of the weight vector. The leave-one-out (LOO) error can be obtained as a by-product in the training of EP without extra computation and is a good indication for the generalization error. Therefore, the LOO error is used together with the Bayesian evidence for model selection in this algorithm. An empirical study on several regression and classification benchmark data sets shows that our algorithm utilizes far less component learners but performs as well as, or better than, the unpruned ensemble. Our results are very competitive compared with other ensemble pruning algorithms.