Federated causal discovery aims to uncover causal relationships while protecting data privacy, with significant real-world applications. Existing methods focus on horizontal federated settings where clients share the same variables but have different samples. However, in practice, clients may have different variables, leading to spurious causal relationships. To address this issue, we comprehensively consider causal structure learning methods under both horizontal and vertical federated settings. Interestingly, we find that, higher-order cumulants rely solely on the joint distribution of the relevant variables and are useful to solve the above problem in the linear non-Gaussian case. This motivates us to provide the identification theories for determining the causal order over observed variables, leveraging the difference in the product of the (cross) cumulants of the specific variables. Based on these theories, we develop a method for learning causal order in the horizontal and vertical federated scenarios. Specifically, we first obtain local (cross) cumulant matrices of observed variables from all participating clients to construct a global cumulant matrix. This global cumulant matrix is then used for recursive source variable identification, ultimately yielding a causal strength matrix of the union of variables from all clients. Our algorithm demonstrates superior performance in experiments on both synthetic and real-world data.

Data augmentation is crucial for addressing insufficient training data, especially for augmenting positive samples. However, existing methods mostly rely on neural network-based feedback for data augmentation and often overlook the optimization of feature distribution. In this study, we present a practical, distribution-preserving data augmentation pipeline that augments positive samples by optimizing a feature indicator (e.g., two-dimensional entropy), aiming to maintain alignment with the original data distribution. Inspired by the manifold hypothesis, we propose a Manifold Heuristic Optimization Algorithm (MHOA), which augments positive samples by exploring the low-dimensional Euclidean space around object contour pixels instead of the entire decision space. Guided by a "distribution-preservation-first" perspective, our approach explicitly optimizes fidelity to the original data manifold and only retains augmented samples whose feature statistics (e.g., mean, variance) align with the source class. It significantly improves image classification accuracy across neural networks, outperforming state-of-the-art data augmentation methods-especially when the dataset's feature indicator follows a Gaussian distribution. The algorithm's search space, focused on neighborhoods of key feature pixels, is the core driver of its superior performance.

ABSTRACT Objective The task of code generation aims to transform natural language descriptions into corresponding target code. Among the various approaches, syntax‐aware code generation has emerged as a significant approach that strives to generate code by directly modeling the underlying syntactic rules. However, existing works typically adopt an autoregressive approach to sequentially generate each abstract syntax rule, which inevitably neglects the rich structural semantic information inherent within the syntax rules. To address this issue, we propose an enhanced architecture of structure semantics based on Graph Neural Network for code generation. Methods Our approach explicitly models the internal structure of syntactic rules by treating them as graph data, thereby enabling the extraction of deeper structural semantics. Furthermore, we jointly model both the sequential semantics and structural semantics of syntactic rules, effectively addressing the limitations of solely sequence‐based approaches in capturing the inherent structural semantics of code. Results Experimental results on two widely used code generation datasets demonstrate that the proposed model consistently outperforms strong baselines, with gains of up to 2.14 BLEU points and 2.02 CodeBLEU points, highlighting the effectiveness of our structural‐semantic modeling approach for code generation.

Temporal recommendation is an important class of tasks in recommender systems, which focuses on modeling and capturing temporal patterns in user behavior to achieve finer-grained and higher-quality recommendations. In real-world scenario, users' temporal behaviors are not only characterized by sequential dependencies among consecutive items, but also by periodic correlations of different items and time-varying similarity of different users. In this paper, we propose an Adaptive Temporal Recommendation (AdaTR) algorithm to capture the inherent features of temporal behaviors and dynamic collaborative signals. Firstly, based on the periodic characteristics of user behaviors, the user-item interactions are counted and aggregated in different time segments across multiple periods, which forms the temporal user-item interaction matrix. Then, in order to capture the time-varying collaborative signals between different users, a deep spectral clustering (DSC) method is implemented on the temporal user-item interaction matrix, where the original representation of user-item interaction is projected into a latent space, and users' temporal behaviors are clustered into different groups. Furthermore, an Adaptive Deep Matrix Factorization (AdaDMF) module is designed to learn the time-varying representations of user preferences on each cluster of temporal user behaviors, which incoporate dynamic collaborative signals among different users. Finally, we combine users' short-term and long-term preferences to generate personalized temporal recommendations. Extensive experiments on four datasets demonstrate that AdaTR performs significantly better than the state-of-the-art baselines.

Causal discovery plays a pivotal role in scientific inquiry and subsequent applications in prediction or decision-making. While many methods have been proposed, many of them rely on independence tests. However, these tests are difficult to implement and computationally intensive. In this article, we aim to propose a direct and computationally efficient method to determine the causal relationship between two observed variables in the linear non-Gaussian case. Building on the insight that cumulants provide information about the shape of a probability distribution, we show that interestingly, the (in)dependence between two observed variables can be directly inferred from the difference in the product of certain joint cumulants of these variables. This concept is named the cause difference criterion. Based on this criterion, we introduce two practical methods, high-order cumulant (HC) and HC-linear non-Gaussian acyclic model (LiNGAM), for causal discovery in the high-dimensional case. Theoretical analyses ensure the identifiability of the proposed criteria and methods. Experimental results indicate that our methods outperform most existing methods.

Current methods for time series forecasting struggle in the online scenario, since it is difficult to preserve long-term dependency while adapting short-term changes when data are arriving sequentially. Although some recent methods solve this problem by controlling the updates of latent states, they cannot disentangle the long/short-term states, leading to the inability to effectively adapt to nonstationary. To tackle this challenge, we propose a general framework to disentangle long/short-term states for online time series forecasting. Our idea is inspired by the observations where short-term changes can be led by unknown interventions like abrupt policies in the stock market. Based on this insight, we formalize a data generation process with unknown interventions on short-term states. Under mild assumptions, we further leverage the independence of short-term states led by unknown interventions to establish the identification theory to achieve the disentanglement of long/short-term states. Built on this theory, we develop a Long Short-Term Disentanglement model (LSTD) to extract the long/short-term states with long/short term encoders, respectively. Furthermore, the LSTD model incorporates a smooth constraint to preserve the long-term dependencies and an interrupted dependency constraint to enforce the forgetting of short-term dependencies, together boosting the disentanglement of long/short-term states. Experimental results on several benchmark datasets show that our LSTD model outperforms existing methods for online time series forecasting, validating its efficacy in real-world applications.

Dialogues Aspect-based Sentiment Quadruple Extraction (DiaASQ) aims to extract all target-aspect-opinion-sentiment quadruples from a given multi-round, multi-participant dialogue. Existing methods typically learn word relations across entire dialogues, assuming a uniform distribution of sentiment elements. However, we find that dialogues often contain multiple semantically independent sub-dialogues without clear dependencies between them. Therefore, learning word relationships across the entire dialogue inevitably introduces additional noise into the extraction process. To address this, our method focuses on partitioning dialogues into semantically independent sub-dialogues. Achieving completeness while minimizing these sub-dialogues presents a significant challenge. Simply partitioning based on reply relationships is ineffective. Instead, we propose utilizing a structural entropy minimization algorithm to partition the dialogues. This approach aims to preserve relevant utterances while distinguishing irrelevant ones as much as possible. Furthermore, we introduce a two-step framework for quadruple extraction: first extracting individual sentiment elements at the utterance level, then matching quadruples at the sub-dialogue level. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in DiaASQ with much lower computational costs.

We study the problem of learning the causal structure of a latent tree model only from the observational data. Prior works often assume either a sufficient number of measured variables or the observed variables can only be the child of latent variables (known as measurement assumption). However, they may yield incorrect or uninformative results when some observed variables also cause the latent variable, or when the number of measured variables is less than two. In this paper, we focus on the linear non-Gaussian latent polytree model, where the observed and latent variables can exhibit arbitrary causal dependence and the number of child variables for each latent variable may be only one. By leveraging the non-Gaussianity within the causal model, we introduce rank constraints of high-order cumulants. These constraints align with trek separation within the causal graph and enable the identification of exogenous variables for the relative set. Such properties have intriguing possibilities for identifying the entire latent polytree structure, including not only the number of latent variables but also causal directions. Consequently, we develop an identification algorithm to learn latent polytree by only using the rank constraints of high-order cumulants, and we verify its effectiveness in simulation experiments.

Current Emotion Recognition in Conversation (ERC) research follows a closed-domain assumption.However, there is no clear consensus on emotion classification in psychology, which presents a challenge for models when it comes to recognizing previously unseen emotions in real-world applications.To bridge this gap, we introduce the Unseen Emotion Recognition in Conversation (UERC) task for the first time and propose ProEmoTrans, a solid prototype-based emotion transfer framework.This prototype-based approach shows promise but still faces key challenges: First, implicit expressions complicate emotion definition, which we address by proposing an LLM-enhanced description approach.Second, utterance encoding in long conversations is difficult, which we tackle with a proposed parameter-free mechanism for efficient encoding and overfitting prevention.Finally, the Markovian flow nature of emotions is hard to transfer, which we address with an improved Attention Viterbi Decoding (AVD) method to transfer seen emotion transitions to unseen emotions.Extensive experiments on three datasets show that our method serves as a strong baseline for preliminary exploration in this new area.

We study the problem of learning discrete latent variable causal structures from mixed-type observational data. Traditional methods, such as those based on the tensor rank condition, are designed to identify discrete latent structure models and provide robust identification bounds for discrete causal models. However, when observed variables—specifically, those representing the children of latent variables—are collected at various levels with continuous data types, the tensor rank condition is not applicable, limiting further causal structure learning for latent variables. In this paper, we consider a more general case where observed variables can be either continuous or discrete, and further allow for scenarios where multiple latent parents cause the same set of observed variables. We show that, under the completeness condition, it is possible to discretize the data in a way that satisfies the full-rank assumption required by the tensor rank condition. This enables the identifiability of discrete latent structure models within mixed-type observational data. Moreover, we introduce the two-sufficient measurement condition, a more general structural assumption under which the tensor rank condition holds and the underlying latent causal structure is identifiable by a proposed two-stage identification algorithm. Extensive experiments on both simulated and real-world data validate the effectiveness of our method.

Testing conditional independence is a critical task, particularly in causal discovery and learning in Bayesian networks. However, in many real-world scenarios, variables are often measured with errors, such as those introduced by insufficient measurement accuracy, complicating the testing process. This paper focuses on testing conditional independence in the linear non-Gaussian measurement error model, under the condition that measurement error noise follows a Gaussian distribution. By leveraging high-order cumulants, we derive rank constraints on the cumulant matrix and establish their role in effectively assessing conditional independence, even in the presence of measurement errors. Based on these theoretical results, we leverage the rank constraints of the cumulant matrix as a tool for conditional independence testing and incorporate it into the PC algorithm, resulting in the PC-ME algorithm — a method designed to learn causal structures from observed data while accounting for measurement errors. Experimental results demonstrate that the proposed method outperforms existing approaches, particularly in cases other methods encounter difficulties.

Time series imputation is one of the most challenging problems and has broad applications in various fields like health care and the Internet of Things. Existing methods mainly aim to model the temporally latent dependencies and the generation process from the observed time series data. In real-world scenarios, different types of missing mechanisms, like MAR (Missing At Random) and MNAR (Missing Not At Random), can occur in time series data. However, existing methods often overlook the difference among the aforementioned missing mechanisms and use a single model for time series imputation, which can easily lead to misleading results due to mechanism mismatching. In this paper, we propose a framework for the time series imputation problem by exploring Different Missing Mechanisms (DMM in short) and tailoring solutions accordingly. Specifically, we first analyze the data generation processes with temporal latent states and missing cause variables for different mechanisms. Sequentially, we model these generation processes via variational inference and estimate prior distributions of latent variables via a normalizing flow-based neural architecture. Furthermore, we establish identifiability results under the nonlinear independent component analysis framework to show that latent variables are identifiable. Experimental results show that our method surpasses existing time series imputation techniques across various datasets with different missing mechanisms, demonstrating its effectiveness in real-world applications.

With the development of deep neural networks, deep reinforcement learning (DRL) has shown great potential in solving complex decision-making problems in high-dimensional environments. Among various DRL methods, Q-value function approaches are well-grounded in theory and exhibit strong practical performance, but often suffer from overestimation of the value function. We find that this overestimation primarily stems from unobserved random factors (latent variables) within the environmental dynamics, while existing solutions fail to address this issue, relying mainly on low-noise and unbiased sampling environments. To address this problem, we introduce latent variables to model unobserved randomness behind the environmental dynamics and employ causal models to capture the underlying data generation mechanisms. Subsequently, we propose a causal Q-value function method based on latent causal models. Specifically, we first design a causal encoding-decoding framework for learning the latent causal models, then utilize the inferred latent variables to construct a causal Q-value function, effectively mitigating the Q-value function overestimation problem. Experimental results show that our method significantly improves the stability of policy learning and the speed of convergence, enhancing the robustness of reinforcement learning in complex dynamic environments.

Multi-modal time series data is common in web technologies like the Internet of Things (IoT). Existing methods for multi-modal time series representation learning aim to disentangle the modality-shared and modality-specific latent variables. Although achieving notable performances on downstream tasks, they usually assume an orthogonal latent space. However, the modality-specific and modality-shared latent variables might be dependent on real-world scenarios. Therefore, we propose a general generation process, where the modality-shared and modality-specific latent variables are dependent, and further develop a Multi-modAl TEmporal Disentanglement (MATE) model. Specifically, our MATE model is built on a temporally variational inference architecture with the modality-shared and modality-specific prior networks for the disentanglement of latent variables. Furthermore, we establish identifiability results to show that the extracted representation is disentangled. More specifically, we first achieve the subspace identifiability for modality-shared and modality-specific latent variables by leveraging the pairing of multi-modal data. Then we establish the component-wise identifiability of modality-specific latent variables by employing sufficient changes of historical latent variables. Extensive experimental studies on 12 datasets show a general improvement in different downstream tasks, highlighting the effectiveness of our method in real-world scenarios.

Contemporary datasets sourced from the web often adopt a multiview format, collecting data from diverse sources, domains, or modules.Existing methodologies employed to analyze such datasets frequently overlook or inaccurately allocate the view-weights, pivotal metrics reflecting each view's significance.This work introduces EVA-MVC, a simple yet effective algorithm designed for Equitable View-weight Allocation (EVA) seamlessly integrated with arbitrary Multi-view Clustering (MVC) methods.Within the EVA module, we establish theoretical connections between view supplementarity and Multi-view Subspace Learning (MSL), leading to the partition of views into View Communities (VCs) based on these foundational principles.These VCs exhibit internal supplementarity similarities, facilitating Equitable View-weights Allocation through VCspecific MSL.The proposed EVA process precedes and operates independently of traditional or SOTA MVC approaches, requiring no additional processing or specialized design, making it an ideal preprocessing step for MVC applications.Through comprehensive evaluations across diverse multi-view datasets, our findings reveal that our EVA significantly enhances the effectiveness of mainstream MVC frameworks, resulting in a notable performance improvement.

Learning Granger causality from event sequences has important applications in various scenarios. Many methods have been developed based on Hawkes process with a stationarity assumption. However, these methods often fail in real-world scenarios due to violating the stationarity assumption, as an event sequence can be generated under different states at varying times. Although some work tries to model non-stationarity by searching for best segmentation, they still suffer from the lack of robustness and identification guarantee. An intuitive solution is to model the non-stationary generation process in a unified probabilistic generative framework. This presents two significant challenges: how to model the generation process considering both the stationarity of each subsequence and the non-stationarity among the subsequences, and how to identify the Granger causality. To address these challenges, we devise State Hawkes Processes (StateHPs). For the first challenge, StateHPs formulates the state assignments of each subsequence as a Dirichlet distribution and each state as a Hawkes process. For the second challenge, StateHPs introduces a variational Expectation-Maximization algorithm to identify the Granger causal graph. We also develop the identification theories for StateHPs. On real-world data, StateHPs achieves 35.5%, 33.9%, and 36.7% improvement among F1, Precision, and Recall metrics compared to the SOTA baselines.

Although graph neural networks have achieved great success in the task of molecular property prediction in recent years, their generalization ability under out-of-distribution (OOD) settings is still under-explored. Most of the existing methods rely on learning discriminative representations for prediction, often assuming that the underlying semantic components are correctly identified. However, this assumption does not always hold, leading to potential misidentifications that affect model robustness. Different from these discriminative-based methods, we propose a generative model to ensure the Semantic-Components Identifiability, named SCI. We demonstrate that the latent variables in this generative model can be explicitly identified into semantic-relevant (SR) and semantic-irrelevant (SI) components, which contributes to better OOD generalization by involving minimal change properties of causal mechanisms. Specifically, we first formulate the data generation process from the atom level to the molecular level, where the latent space is split into SI substructures, SR substructures, and SR atom variables. Sequentially, to reduce misidentification, we restrict the minimal changes of the SR atom variables and add a semantic latent substructure regularization to mitigate the variance of the SR substructure under augmented domain changes. Under mild assumptions, we prove the block-wise identifiability of the SR substructure and the comment-wise identifiability of SR atom variables. Experimental studies achieve state-of-the-art performance and show general improvement on 21 datasets in 3 mainstream benchmarks. Moreover, the visualization results of the proposed SCI method provide insightful case studies and explanations for the prediction results.

Drug combinations play very important roles in cancer therapy, as they can enhance curative efficacy and overcome drug resistance. Due to the increasing size of combinatorial space, experimental screening for all the drug combinations becomes infeasible in practice. Therefore, there is a great need to develop accurate computational approaches that can predict potential drug combinations to direct the experimental screening. In this paper, we propose a novel method called GNNSynergy to learn drug embeddings for drug synergy prediction. Given a specific cancer cell line, we propose a multi-view graph neural network framework which considers the current cell line as main view while other cell lines from the same tissue as sub-views. In each view, we first construct different graphs to describe drug synergistic and antagonistic interactions, and adopt graph neural network as encoder to learn drug embeddings. We further combine both the main view and sub-views via an attention mechanism to derive the final drug embeddings for drug synergy prediction. We perform extensive experiments on DrugComb database and the experimental results demonstrate that our proposed GNNSynergy significantly outperforms state-of-the-art methods for novel synergistic drug combination prediction.

Flow visualization through motion estimation using time-sequenced images plays a significant role in analyzing and understanding complex flow phenomena, and it is widely used in meteorology, oceanography, medicine, astronomy, experimental fluid mechanics, etc. However, it is difficult for current motion estimators to adapt to illumination changes, remove instable perturbation, and capture diverse motion patterns. In this paper, a novel flow visualization tool is developed to address these issues by employing a structure-enhanced motion estimator composed of a data term and a regularization term. Specifically, a statistical correlation descriptor is designed for the data term to improve the accuracy of motion estimation by enhancing both illumination robustness and matching discrimination. Inspired by the strong distinguishability of a structure-texture distribution in a local window, a structure-enhanced regularizer that considers the physical mechanism of fluid diffusion is introduced to capture different motion patterns, enhance prominent flow structures, and remove unnecessary ripples or textures caused by instable perturbation or noise. The experimental results demonstrate that our approach significantly outperforms current motion estimators in handling illumination changes and predicting complex fluid flows, and it also achieves state-of-the-art evaluation results on the public fluid flow datasets. Furthermore, the designed flow visualization tool successfully captures diverse motion patterns in Jupiter’s White Ovals, which is crucial for understanding the physical mechanisms behind their formation and sustenance.

Considering the download traffic requirements of the terminals within a certain range, the two-sided fuzzy relation inequalities with addition-min have been recently introduced for modeling the P2P network system. In such a model, the solutions of the FRIs correspond the feasible flow schemes in the P2P network system. To characterize the stability of a given flow scheme, we define the amplitude of the corresponding solution, in this work. If a solution has a bigger amplitude, then it is able to bear a larger perturbation among its components. Accordingly, the corresponding flow scheme is considered to be more stable. For obtaining the amplitude of a given solution, we propose some effective algorithms to compute the upper and lower amplitudes. Following our proposed algorithms, the stability of any pair of solutions could be compared, based on the obtained amplitudes. Our provided results are demonstrated by some numerical examples

In the domain of Multi-view Subspace Clustering (MSC) in Latent Embedding Space (LES), existing methods aim to capture and leverage critical multi-view information by mapping it into a low-dimensional LES. However, several aspects can be further improved: (i) Fusion Strategy: Existing methods adopt either early fusion or late fusion to integrate multi-view information, limiting the effectiveness of the fusion. (ii) Diversity: Current methods often overlook the inherent diversity in the multi-view data by focusing on a single LES. (iii) Efficiency: LES-based methods exhibit high computational complexity, with cubic time and quadratic space requirements based on the number of samples. To address these issues, we propose a novel framework called MSC-DOLES (Multi-view Subspace Clustering in Diverse Orthogonal Latent Embedding Spaces), a novel framework designed to tackle these challenges. MSC-DOLES incorporates a two-stage fusion approach that generates and learns from multiple LES to maximize cross-view diversity. Orthogonality constraints on individual LES ensure view-internal diversity, resulting in a set of Diverse Orthogonal Latent Embedding Spaces (DOLES). The DOLES are then fused into a consensus anchor graph using learnable anchors. The final clustering is induced by partitioning the obtained graph without pre-processing. We develop an eight-step optimization algorithm for MSC-DOLES, which exhibits nearly linear time and space complexities relative to the number of samples. Extensive experiments demonstrate the superiority of MSC-DOLES over state-of-the-art methods.

With long-tailed data and complex label hierarchy, hierarchical text classification (HTC) is a challenging multi-label text classification task. Applying prompts to pre-trained language models (PLMs) has recently become a mainstream approach in HTC. However, existing prompt-based models experience a significant drop in classification performance on tail labels. Due to the imbalanced data, HTC models still face two challenges. First, text embeddings, learned for classification, often lack distinctiveness for tail categories. Second, label embeddings suffer from significant degeneration, especially for tail labels. To address these issues, in this paper, we propose a novel Hierarchical Text Classification Optimization method via Structural Entropy and SIngular Spectrum Smoothing, namely SIHTC. SIHTC contains two parts: text embedding optimization and label embedding optimization. First, based on the structural information theory, we design a tree aggregation network and construct encoding trees to minimize the structural entropy of texts under the hierarchical labels. In this manner, SIHTC injects label structural information into text embeddings, hierarchically optimizing the embedding space by enclosing the text embeddings within related ground truth labels while separating them from unrelated ground truth labels. Second, we propose a global and local singular spectrum smoothing regularization method to maximize the area under the singular value curve. In this way, SIHTC decreases representation degeneration and learns label embeddings with improved label generalization capability. Extensive experiments are conducted on three popular HTC datasets. The results show that SIHTC outperforms all baseline methods, especially with an advantage in handling tail labels, indicating the effectiveness of the above two optimizations

The objective of Multi-Source Domain Adaptation (MSDA) is to train a neural network on labeled data from multiple joint source distributions (source domains) and unlabeled data from a joint target distribution (target domain), and use the trained network to estimate the target data labels. The challenge in this MSDA problem is that the multiple joint source distributions are relevant but distinct from the joint target distribution. To address this challenge, we propose a Joint Distribution Weighted Alignment (JDWA) approach to align a weighted joint source distribution to the joint target distribution under the relative entropy. Specifically, the weighted joint source distribution is defined as the weighted sum of the multiple joint source distributions, and is parameterized by the relevance weights. Since the relative entropy is unknown in practice, we propose a Kernel Relative Entropy Estimation (KREE) method to estimate it from data. Our KREE method first reformulates relative entropy as the negative of the minimal value of a functional, then exploits a function from the Reproducing Kernel Hilbert Space (RKHS) as the functional's input, and finally solves the resultant convex problem with a global optimal solution. We also incorporate entropy regularization to enhance the network's performance. Together, we minimize cross entropy, relative entropy, and entropy to learn both the relevance weights and the neural network. Experimental results on benchmark image classification datasets demonstrate that our JDWA approach performs better than the comparison methods. Pytorch code of our approach will be released upon the paper's publication.

Multiview graph clustering (MGC) methods are increasingly being studied due to the explosion of multiview data with graph structural information. The critical point of MGC is to better utilize view-specific and view-common information in features and graphs of multiple views. However, existing works have an inherent limitation that they are unable to concurrently utilize the consensus graph information across multiple graphs and the view-specific feature information. To address this issue, we propose a variational graph generator for MGC (VGMGC). Specifically, a novel variational graph generator is proposed to extract common information among multiple graphs. This generator infers a reliable variational consensus graph based on a priori assumption over multiple graphs. Then, a simple yet effective graph encoder in conjunction with the multiview clustering objective is presented to learn the desired graph embeddings for clustering, which embeds the inferred view-common graph and view-specific graphs together with features. Finally, theoretical results illustrate the rationality of the VGMGC by analyzing the uncertainty of the inferred consensus graph with the information bottleneck (IB) principle. Extensive experiments demonstrate the superior performance of our VGMGC over state-of-the-art methods (SOTAs). The source code is publicly available at: https://github.com/cjpcool/VGMGC.

Conditional independence (CI) testing is an important problem, especially in causal discovery. Most testing methods assume that all variables are fully observable and then test the CI among the observed data. Such an assumption is often untenable beyond applications dealing with, e.g., psychological analysis about the mental health status and medical diagnosing (researchers need to consider the existence of latent variables in these scenarios); and typically adopted latent CI test schemes mainly suffer from robust or efficient issues. Accordingly, this article investigates the problem of testing CI between latent variables. To this end, we offer an auxiliary regression-based CI (AReCI) test by taking the measured variable as the surrogate variable of the latent variables to conduct the regression over the latent variables under the linear causal models, in which each latent variable has some certain measured variables. Specifically, given a pair of latent variables$L_X$and$L_Y$, and a corresponding latent variable set$\mathcal{L}_{O}$,$L_X \CI L_Y | \mathcal{L}_{O}$holds if and only if$A_{\{L_X\}}-\omega_1^\intercal A^{\prime}_{\{\mathcal{L}_{O}\}}$and$A_{\{L_Y\}}-\omega_2^\intercal A^{\prime\prime}_{\{\mathcal{L}_{O}\}}$are statistically independent, where$A^{\prime}$and$A^{\prime\prime}$are the two disjoint subset of the measured variable for the corresponding latent variables,$A^{\prime}_{\{\mathcal{L}_{O}\}} \cap A^{\prime\prime}_{\{\mathcal{L}_{O}\}} =\emptyset$, and$\omega_1$is a parameter vector characterized from the cross covariance between$A_{\{L_X\}}$and$A^{\prime}_{\{\mathcal{L}_{O}\}}$, and$\omega_{2}$is a parameter vector characterized from the cross covariance between$A_{\{L_Y\}}$and$A^{\prime\prime}_{\{\mathcal{L}_{O}\}}$. We theoretically show that the AReCI test is capable of addressing both Gaussian and non-Gaussian data. In addition, we find that the well-known partial correlation test can be seen as a special case of the AReCI test. Finally, we devise a causal discovery method by using the AReCI test as the CI test. The experimental results on synthetic and real-world data illustrate the effectiveness of our method.

Neural architecture search (NAS) is crucial for text representation in natural language processing (NLP); however, much less work on NAS for text classification has been proposed compared with NAS for computer vision. Similar to NAS for vision tasks, most existing work rely on a manually designed search space defined by a directed acyclic graph (DAG), resulting in limited generalization capability and high computational complexity. In text classification, the topological order of the NAS operators is essential for enhancing generalization, which cannot be accurately represented by a DAG. To address this issue, we propose a bipartite graph-based NAS (BGNAS) for text classification, which converts a DAG into a dual graph and then into a bipartite graph. This transformation makes it possible to accurately capture the topological order using multi-bigraph matching. In addition, we formulate NAS as a problem of identifying the lower bound of a submodular function, theoretically ensuring that optimal architectures in a bipartite graph-based search space can be identified using fewer search operators. Reduction of the search space is achieved by eliminating ineffective associated matching rules among search operators with a pruning strategy. As a result, the bipartite graph-based search space becomes more compact and less dependent on complex contextual semantics of text data. Experimental results on public benchmark problems demonstrate that BGNAS achieves better performance than the state-of-the-art NAS algorithms and is computationally more efficient. We also demonstrate that the bipartite graph search space can more effectively capture contextual semantics, thereby enhancing the generalization capability.

Multiview clustering (MVC) has been widely studied in machine learning and data mining for its capability of improving clustering performance by fusing the information from multiview data. In the past decade, a large number of MVC methods have made impressive progress, but most of them suffer from computational burdens, especially in large-scale tasks. Binary MVC (BMVC) is proposed to address this issue by representing the large-scale high-dimensional dataset as a group of consensus and low-dimensional binary codes. However, current BMVC-based approaches generate the clustering by executing binary k-means on the obtained binary codes, which fail to capture the embedded geometric information, leading to poor clustering performance. In addition, parameter selection is another "mission impossible" in unsupervised learning tasks including MVC. To tackle these challenges, a framework of multiview clustering via partitioning the signed prototype graph (SPGMVC) is proposed in this work. The SPGMVC framework offers several contributions. First, SPGMVC is designed as a unified framework for MVC. It combines effective technologies, such as consensus binary coding, code compression (CC), signed prototype graph (SPG) partitioning, and prototype-based cluster assignment. Second, SPGMVC partitions the signed graph (SG) based on the relationships between positive and negative edges. By capturing the underlying structure of the data, this partitioning strategy improves clustering accuracy (ACC). CC techniques are applied to reduce the graph's scale, enabling further partitioning and enhancing computational efficiency. Third, SPGMVC employs an alternate minimizing strategy to efficiently handle the optimization problem. This strategy has nearly linear time and space complexity with respect to the data volume, making it suitable for large-scale tasks. Fourth, SPGMVC proposes an automatic parameter selection strategy, eliminating the need for extensive parameter exploration. Comprehensive experiments illustrate the superiority of our model. The implementation of SPGMVC is available at: https://github.com/gepingyang/PSGMVC.

While reinforcement learning (RL) achieves tremendous success in sequential decision-making problems of many domains, it still faces key challenges of data inefficiency and the lack of interpretability. Interestingly, many researchers have leveraged insights from the causality literature recently, bringing forth flourishing works to unify the merits of causality and address well the challenges from RL. As such, it is of great necessity and significance to collate these causal RL (CRL) works, offer a review of CRL methods, and investigate the potential functionality from causality toward RL. In particular, we divide the existing CRL approaches into two categories according to whether their causality-based information is given in advance or not. We further analyze each category in terms of the formalization of different models, ranging from the Markov decision process (MDP), partially observed MDP (POMDP), multiarmed bandits (MABs), imitation learning (IL), and dynamic treatment regime (DTR). Each of them represents a distinct type of causal graphical illustration. Moreover, we summarize the evaluation matrices and open sources, while we discuss emerging applications, along with promising prospects for the future development of CRL.

Causal discovery with latent variables is a crucial but challenging task. Despite the emergence of numerous methods aimed at addressing this challenge, they are not fully identified to the structure that two observed variables are influenced by one latent variable and there might be a directed edge in between. Interestingly, we notice that this structure can be identified through the utilization of higher-order cumulants. By leveraging the higher-order cumulants of non-Gaussian data, we provide an analytical solution for estimating the causal coefficients or their ratios. With the estimated (ratios of) causal coefficients, we propose a novel approach to identify the existence of a causal edge between two observed variables subject to latent variable influence. In case when such a causal edge exits, we introduce an asymmetry criterion to determine the causal direction. The experimental results demonstrate the effectiveness of our proposed method.

Deep neural networks (DNNs) have been demonstrated to be vulnerable to well-crafted adversarial examples, which are generated through either well-conceived L_p-norm restricted or unrestricted attacks. Nevertheless, the majority of those approaches assume that adversaries can modify any features as they wish, and neglect the causal generating process of the data, which is unreasonable and unpractical. For instance, a modification in income would inevitably impact features like the debt-to-income ratio within a banking system. By considering the underappreciated causal generating process, first, we pinpoint the source of the vulnerability of DNNs via the lens of causality, then give theoretical results to answer where to attack. Second, considering the consequences of the attack interventions on the current state of the examples to generate more realistic adversarial examples, we propose CADE, a framework that can generate Counterfactual ADversarial Examples to answer how to attack. The empirical results demonstrate CADE's effectiveness, as evidenced by its competitive performance across diverse attack scenarios, including white-box, transfer-based, and random intervention attacks.

Learning Granger causality from event sequences is a challenging but essential task across various applications. Most existing methods rely on the assumption that event sequences are independent and identically distributed (i.i.d.). However, this i.i.d. assumption is often violated due to the inherent dependencies among the event sequences. Fortunately, in practice, we find these dependencies can be modeled by a topological network, suggesting a potential solution to the non-i.i.d. problem by introducing the prior topological network into Granger causal discovery. This observation prompts us to tackle two ensuing challenges: 1) how to model the event sequences while incorporating both the prior topological network and the latent Granger causal structure, and 2) how to learn the Granger causal structure. To this end, we devise a unified topological neural Poisson auto-regressive model with two processes. In the generation process, we employ a variant of the neural Poisson process to model the event sequences, considering influences from both the topological network and the Granger causal structure. In the inference process, we formulate an amortized inference algorithm to infer the latent Granger causal structure. We encapsulate these two processes within a unified likelihood function, providing an end-to-end framework for this task. Experiments on simulated and real-world data demonstrate the effectiveness of our approach.

Missing data are an unavoidable complication frequently encountered in many causal discovery tasks. When a missing process depends on the missing values themselves (known as self-masking missingness), the recovery of the joint distribution becomes unattainable, and detecting the presence of such self-masking missingness remains a perplexing challenge. Consequently, due to the inability to reconstruct the original distribution and to discern the underlying missingness mechanism, simply applying existing causal discovery methods would lead to wrong conclusions. In this work, we found that the recent advances additive noise model has the potential for learning causal structure under the existence of the self-masking missingness. With this observation, we aim to investigate the identification problem of learning causal structure from missing data under an additive noise model with different missingness mechanisms, where the `no self-masking missingness' assumption can be eliminated appropriately. Specifically, we first elegantly extend the scope of identifiability of causal skeleton to the case with weak self-masking missingness (i.e., no other variable could be the cause of self-masking indicators except itself). We further provide the sufficient and necessary identification conditions of the causal direction under additive noise model and show that the causal structure can be identified up to an IN-equivalent pattern. We finally propose a practical algorithm based on the above theoretical results on learning the causal skeleton and causal direction. Extensive experiments on synthetic and real data demonstrate the efficiency and effectiveness of the proposed algorithms.

Count data naturally arise in many fields, such as finance, neuroscience, and epidemiology, and discovering causal structure among count data is a crucial task in various scientific and industrial scenarios. One of the most common characteristics of count data is the inherent branching structure described by a binomial thinning operator and an independent Poisson distribution that captures both branching and noise. For instance, in a population count scenario, mortality and immigration contribute to the count, where survival follows a Bernoulli distribution, and immigration follows a Poisson distribution. However, causal discovery from such data is challenging due to the non-identifiability issue: a single causal pair is Markov equivalent, i.e., X->Y and Y->X are distributed equivalent. Fortunately, in this work, we found that the causal order from X to its child Y is identifiable if X is a root vertex and has at least two directed paths to Y, or the ancestor of X with the most directed path to X has a directed path to Y without passing X. Specifically, we propose a Poisson Branching Structure Causal Model (PB-SCM) and perform a path analysis on PB-SCM using high-order cumulants. Theoretical results establish the connection between the path and cumulant and demonstrate that the path information can be obtained from the cumulant. With the path information, causal order is identifiable under some graphical conditions. A practical algorithm for learning causal structure under PB-SCM is proposed and the experiments demonstrate and verify the effectiveness of the proposed method.

We investigate the problem of explainability for machine learning models, focusing on Feature Attribution Methods (FAMs) that evaluate feature importance through perturbation tests. Despite their utility, FAMs struggle to distinguish the contributions of different features, when their prediction changes are similar after perturbation. To enhance FAMs’ discriminative power, we introduce Feature Attribution with Necessity and Sufficiency (FANS), which find a neighborhood of the input such that perturbing samples within this neighborhood have a high Probability of being Necessity and Sufficiency (PNS) cause for the change in predictions, and use this PNS as the importance of the feature. Specifically, FANS compute this PNS via a heuristic strategy for estimating the neighborhood and a perturbation test involving two stages (factual and interventional) for counterfactual reasoning. To generate counterfactual samples, we use a resampling-based approach on the observed samples to approximate the required conditional distribution. We demonstrate that FANS outperforms existing attribution methods on six benchmarks. Please refer to the source code via https://github.com/DMIRLAB-Group/FANS.

Causal effect estimation under networked interference is an important but challenging problem. Available parametric methods are limited in their model space, while previous semiparametric methods, e.g., leveraging neural networks to fit only one single nuisance function, may still encounter misspecification problems under networked interference without appropriate assumptions on the data generation process. To mitigate bias stemming from misspecification, we propose a novel doubly robust causal effect estimator under networked interference, by adapting the targeted learning technique to the training of neural networks. Specifically, we generalize the targeted learning technique into the networked interference setting and establish the condition under which an estimator achieves double robustness. Based on the condition, we devise an end-to-end causal effect estimator by transforming the identified theoretical condition into a targeted loss. Moreover, we provide a theoretical analysis of our designed estimator, revealing a faster convergence rate compared to a single nuisance model. Extensive experimental results on two real-world networks with semisynthetic data demonstrate the effectiveness of our proposed estimators.

Unobserved discrete data are ubiquitous in many scientific disciplines, and how to learn the causal structure of these latent variables is crucial for uncovering data patterns. Most studies focus on the linear latent variable model or impose strict constraints on latent structures, which fail to address cases in discrete data involving non-linear relationships or complex latent structures. To achieve this, we explore a tensor rank condition on contingency tables for an observed variable set $\mathbf{X}_p$, showing that the rank is determined by the minimum support of a specific conditional set (not necessary in $\mathbf{X}_p$) that d-separates all variables in $\mathbf{X}_p$. By this, one can locate the latent variable through probing the rank on different observed variables set, and further identify the latent causal structure under some structure assumptions. We present the corresponding identification algorithm and conduct simulated experiments to verify the effectiveness of our method. In general, our results elegantly extend the identification boundary for causal discovery with discrete latent variables and expand the application scope of causal discovery with latent variables.

This paper focuses on the problem of semi-supervised domain adaptation for time-series forecasting, which is underexplored in literature, despite being often encountered in practice. Existing methods on time-series domain adaptation mainly follow the paradigm designed for static data, which cannot handle domain-specific complex conditional dependencies raised by data offset, time lags, and variant data distributions. In order to address these challenges, we analyze variational conditional dependencies in time-series data and find that the causal structures are usually stable among domains, and further raise the causal conditional shift assumption. Enlightened by this assumption, we consider the causal generation process for time-series data and propose an end-to-end model for the semi-supervised domain adaptation problem on time-series forecasting. Our method can not only discover the Granger-Causal structures among cross-domain data but also address the cross-domain time-series forecasting problem with accurate and interpretable predicted results. We further theoretically analyze the superiority of the proposed method, where the generalization error on the target domain is bounded by the empirical risks and by the discrepancy between the causal structures from different domains. Experimental results on both synthetic and real data demonstrate the effectiveness of our method for the semi-supervised domain adaptation method on time-series forecasting.

Maximum margin clustering (MMC) is a typical clustering method which aims to maximize the margin between different clusters. However, in practical applications, a data object may be represented by multiple feature sets (views), with each feature set representing different information of the underlying data. The traditional MMC methods can handle only the data from a single view and are unable to utilize the multi-view data to enhance the clustering model. In multi-view clustering, there are two basic principles: the consensus principle and complementarity principle. Most multi-view clustering methods implement mainly the consensus principle, while the complementarity principle has not been sufficiently taken into account. Distinguished from the existing methods,$\text {M}^{3}\text {CP}$introduces the idea of privileged information learning into multi-view clustering and implements both of the consensus principle and complementarity principle. Based on privileged information learning,$\text {M}^{3}\text {CP}$embodies the complementarity principle by considering one view as the main learning information and the other views as the privileged information, so that multiple views can provide information to complement each other. The derived learning problem is then solved by applying the constrained concave–convex procedure and cutting plane techniques. By employing these techniques, the computational time of$\text {M}^{3}\text {CP}$is able to scale linearly with respect to the dataset size. Numerical experiments on real-life multi-view datasets demonstrate that$\text {M}^{3}\text {CP}$is able to achieve better clustering accuracy and meanwhile needs less computational time, compared to state-of-the-art multi-view clustering methods.

Constrained optimization is an active research topic in evolutionary computation. It challenges evolutionary algorithms in allocating fitness evaluations to the minimization of constraint violations and the optimization of objectives. Most existing evolutionary algorithms implement fixed allocation schemes by using non-priority, priority, or priority-complete comparison criteria. This paper argues that different constrained optimization problems should be solved by algorithms with a dynamic allocation scheme, and the key to adjusting the allocation scheme is a judgement on whether the objective optimization or the constraint violation minimization is beneficial to finding the optimum. In this paper, correlations between objectives and constraints are measured for the judgement. Based on the correlations, a dynamic allocation scheme for fitness evaluations is proposed for constrained evolutionary algorithms. The objective priority criterion or the objective priority-complete criterion is dynamically selected to allocate more fitness evaluations to the objective optimization if the optimization of objectives is beneficial. Otherwise, the constraint priority criterion or the constraint priority-complete criterion is selected to allocate fitness evaluations. Experimental results show that algorithms with the selected criterion significantly outperform state-of-the-art algorithms in terms of attaining feasible solutions with better objective values. Furthermore, algorithms with the dynamic allocation scheme have the advantage of consistently finding feasible solutions with better objective values than algorithms based on non-beneficial, fixed, and randomly selected schemes. The results demonstrate the positive impact of the correlation-based dynamic allocation scheme on evolutionary algorithms for solving constrained optimization problems.

Many models and methods have been developed to determine numerical thresholds in three-way decision. However, there are challenges such as local optimum-based thresholds, initial point-related optimal solutions, and a limited model generalization. To address these challenges, we design a penalty mechanism-based approximate model and solving algorithm. First, a standard optimization model and its approximate model are established by means of a penalty mechanism. Moreover, three properties of the approximate model are explored, and the relationships between both types of models are analyzed. Second, a penalty mechanism-based particle swarm optimization (PMPSO) algorithm is designed to solve general frameworks with our established models, and comparative experiments are conducted to verify the effectiveness and advantages of the algorithm. Third, we generalize the established model by fuzzy loss representations, and take linguistic intuitionistic fuzzy numbers (LIFNs) as a representation to establish general threshold-determined models based on single and multiple ranking measure functions of LIFNs. Then, we prove the existence and uniqueness of the optimal solution and develop a three-way decision method with the PMPSO algorithm. Finally, an illustrative example and more comparative analyses are considered to demonstrate the effectiveness of our method.

Fuzzy relation system with max-min composition could be applied to describe the three-tier multimedia streaming architecture. In such an architecture, the regional servers supply their local multimedia streaming services to the clients. Random fault has not been considered in the relevant existing researches. However, the random fault of a regional server is possible to occur, or even unavoidable, due to some unpredictable reasons. In this article, we consider a random fault occurring at either one of the regional servers. With such a consideration, the corresponding architecture is characterized by the so-called variable-absent fuzzy relation inequalities system with max-min composition. Some properties of our proposed variable-absent fuzzy relation inequalities system are investigated. Based on such properties, we further propose a 2-D path approach for obtaining the complete solution of the variable-absent system. In addition, we also study a corresponding weighted minimax optimization problem motivated by the practical management objective. An effective algorithm is developed for searching the optimal solution, with an illustrative example.

Multi-view clustering can explore consensus information from multiple views and has attracted increasing attention in the past two decades. However, existing works face two major challenges: i) how to deal with the conflict between learning view-consensus information and reconstructing inconsistent viewprivate information, and ii) how to mitigate representation degeneration caused by implementing the consistency objective for multi-view data. To address these challenges, we propose a novel framework of self-weighted contrastive fusion for deep multi-view clustering (SCMVC). First, our method establishes a hierarchical feature fusion framework, effectively segregating the consistency objective from the reconstruction objective. Then, multi-view contrastive fusion is implemented via maximizing consistency expression between the view-consensus representation and global representation, fully exploring the view consistency and complementary. More importantly, we propose to measure the discrepancy between pairwise representations, and then introduce a self-weighting method, which adaptively strengthens useful views in feature fusion and weakens unreliable views, to mitigate representation degeneration. Extensive experiments on nine public datasets demonstrate that our proposed method achieves state-of-the-art clustering performance. The code is available athttps://github.com/SongwuJob/SCMVC.

Graphs can model complicated interactions between entities, which naturally emerge in many important applications. These applications can often be cast into standard graph learning tasks, in which a crucial step is to learn low-dimensional graph representations. Graph neural networks (GNNs) are currently the most popular model in graph embedding approaches. However, standard GNNs in the neighborhood aggregation paradigm suffer from limited discriminative power in distinguishing high-order graph structures as opposed to low-order structures. To capture high-order structures, researchers have resorted to motifs and developed motif-based GNNs. However, the existing motif-based GNNs still often suffer from less discriminative power on high-order structures. To overcome the above limitations, we propose motif GNN (MGNN), a novel framework to better capture high-order structures, hinging on our proposed motif redundancy minimization operator and injective motif combination. First, MGNN produces a set of node representations with respect to each motif. The next phase is our proposed redundancy minimization among motifs which compares the motifs with each other and distills the features unique to each motif. Finally, MGNN performs the updating of node representations by combining multiple representations from different motifs. In particular, to enhance the discriminative power, MGNN uses an injective function to combine the representations with respect to different motifs. We further show that our proposed architecture increases the expressive power of GNNs with a theoretical analysis. We demonstrate that MGNN outperforms state-of-the-art methods on seven public benchmarks on both the node classification and graph classification tasks.

Causal discovery with latent confounders is an important but challenging task in many scientific areas. Despite the success of some overcomplete independent component analysis (OICA) based methods in certain domains, they are computationally expensive and can easily get stuck into local optima. We notice that interestingly, by making use of higher-order cumulants, there exists a closed-form solution to OICA in specific cases, e.g., when the mixing procedure follows the One-Latent-Component structure. In light of the power of the closed-form solution to OICA corresponding to the One-Latent-Component structure, we formulate a way to estimate the mixing matrix using the higher-order cumulants, and further propose the testable One-Latent-Component condition to identify the latent variables and determine causal orders. By iteratively removing the share identified latent components, we successfully extend the results on the One-Latent-Component structure to the Multi-Latent-Component structure and finally provide a practical and asymptotically correct algorithm to learn the causal structure with latent variables. Experimental results illustrate the asymptotic correctness and effectiveness of the proposed method.

We study the problem of learning hierarchical causal structure among latent variables from measured variables. While some existing methods are able to recover the latent hierarchical causal structure, they mostly suffer from restricted assumptions, including the tree-structured graph constraint, no ``triangle" structure, and non-Gaussian assumptions. In this paper, we relax these restrictions above and consider a more general and challenging scenario where the beyond tree-structured graph, the ``triangle" structure, and the arbitrary noise distribution are allowed. We investigate the identifiability of the latent hierarchical causal structure and show that by using second-order statistics, the latent hierarchical structure can be identified up to the Markov equivalence classes over latent variables. Moreover, some directions in the Markov equivalence classes of latent variables can be further identified using partially non-Gaussian data. Based on the theoretical results above, we design an effective algorithm for learning the latent hierarchical causal structure. The experimental results on synthetic data verify the effectiveness of the proposed method.

Multi-source domain adaptation (MSDA) methods aim to transfer knowledge from multiple labeled source domains to an unlabeled target domain. Although current methods achieve target joint distribution identifiability by enforcing minimal changes across domains, they often necessitate stringent conditions, such as an adequate number of domains, monotonic transformation of latent variables, and invariant label distributions. These requirements are challenging to satisfy in real-world applications. To mitigate the need for these strict assumptions, we propose a subspace identification theory that guarantees the disentanglement of domain-invariant and domain-specific variables under less restrictive constraints regarding domain numbers and transformation properties and thereby facilitating domain adaptation by minimizing the impact of domain shifts on invariant variables. Based on this theory, we develop a Subspace Identification Guarantee (SIG) model that leverages variational inference. Furthermore, the SIG model incorporates class-aware conditional alignment to accommodate target shifts where label distributions change with the domain. Experimental results demonstrate that our SIG model outperforms existing MSDA techniques on various benchmark datasets, highlighting its effectiveness in real-world applications.

Automated tuning can significantly improve productivity and save the costs of manual operation in the microwave filter manufacturing industry. This article proposes a mathematical model of scattering data optimization to find the accurate coupling matrix for multiple-version microwave filters, a core step of automated microwave filter tuning. For the large-scale problem of coupling coefficient combination, we propose a decision set decomposition strategy that evenly divides the entire frequency interval into several subintervals according to the correlation between scattering data. With this strategy, we design a microscale (small-size subsets of the decomposed decision set) searching algorithm, which solves each suboptimization problem by searching the decision subset instead of the entire decision set. To verify the validity of the proposed algorithm for multiple-version microwave filters, experiments are conducted on three versions of microwave filters from a real-world production line, including the two-port eighth-order, ninth-order, and tenth-order microwave filters. Experimental results show that the proposed model is feasible within the industrial error for the multiversion microwave filter tuning problem. Besides, the proposed algorithm outperforms the state-of-the-art optimization algorithms in the coupling matrix optimization problem.

Scheduling is significant in improving the production efficiency and reducing delivery delays for manufacturing enterprises. Unlike the flexible job-shop scheduling problem, two special constraints are encountered in real-world power supply manufacturing systems: 1) periodic maintenance and 2) mandatory outsourcing. As the characteristics of these constraints are not considered in existing scheduling algorithms, schedules generated by most existing approaches are not optimal or even conflict with these constraints. In this article, a self-organizing neural scheduler (SoNS) is proposed to overcome this limitation. A long short-term memory encoder is developed to transform the variable-length structural information into fixed-length feature vectors. Moreover, the reinforcement learning model is proposed to automatically select policies for improving candidate schedules. To validate the effectiveness of the proposed algorithm, extensive experiments are conducted on over 300 problem instances. The nonparametric Kruskal-Wallis tests confirm that the proposed algorithm outperforms several state-of-the-art methods in terms of effectiveness and robustness within a limited computational budget. It demonstrates that the proposed SoNS can solve scheduling problems with the periodic maintenance and mandatory outsourcing constraints effectively.

Spectral Clustering (SC) is an effective clustering method for its excellent performance in partitioning non-linearly distributed data. On the other hand, Ensemble Clustering (EC), a different clustering technology, can promote cluster quality by ensembling the results of base clusterings. In this work, we concentrate on an EC framework that utilizes SC as the base method. Nevertheless, SC suffers from scalability due to its high computational complexity in constructing the Laplacian graph and computing the corresponding eigendecomposition. In the past decades, many efforts have been made to it. However, SC suffers from the scalability issue in processing extensive data, especially in web-scale scenarios. Additionally, EC requires multiple clustering results as the ensemble bases, which further aggravates resource consumption. To address this issue, LiteWSEC, a simple yet efficient Lightweight Framework for Web-scale Spectral Ensemble Clustering, is proposed to cluster web-scale data with limited resource requirements. It adopts the Web-scale Spectral Clustering (WSC) as the base method, which has minimal space overhead without computing overall embedding explicitly. LiteWSEC is highly flexible in the memory requirement, which is adaptive to the available resource. It can partition web-scale data (e.g.,$n $= 8,000 k) in an resource-limited host (e.g., memory is restricted to 1 GB). Experiments on real-world, large-scale, and web-scale datasets demonstrate both the efficiency and effectiveness of LiteWSEC over state-of-the-art SC and EC methods.

Nonlinear causal discovery with high-dimensional data where each variable is multidimensional plays a significant role in many scientific disciplines, such as social network analysis. Previous work majorly focuses on exploiting asymmetry in the causal and anticausal directions between two high-dimensional variables (a cause-effect pair). Although there exist some works that concentrate on the causal order identification between multiple variables, i.e., more than two high-dimensional variables, they do not validate the consistency of methods through theoretical analysis on multiple-variable data. In particular, based on the asymmetry for the cause-effect pair, if model assumptions for any pair of the data are violated, the asymmetry condition will not hold, resulting in the deduction of incorrect order identification. Thus, in this article, we propose a causal functional model, namely high-dimensional deterministic model (HDDM), to identify the causal orderings among multiple high-dimensional variables. We derive two candidates' selection rules to alleviate the inconvenient effects resulted from the violated-assumption pairs. The corresponding theoretical justification is provided as well. With these theoretical results, we develop a method to infer causal orderings for nonlinear multiple-variable data. Simulations on synthetic data and real-world data are conducted to verify the efficacy of our proposed method. Since we focus on deterministic relations in our method, we also verify the robustness of the noises in simulations.

An important problem across multiple disciplines is to infer and understand meaningful latent variables. One strategy commonly used is to model the measured variables in terms of the latent variables under suitable assumptions on the connectivity from the latents to the measured (known as measurement model). Furthermore, it might be even more interesting to discover the causal relations among the latent variables (known as structural model). Recently, some methods have been proposed to estimate the structural model by assuming that the noise terms in the measured and latent variables are non-Gaussian. However, they are not suitable when some of the noise terms become Gaussian. To bridge this gap, we investigate the problem of identification of the structural model with arbitrary noise distributions. We provide necessary and sufficient condition under which the structural model is identifiable: it is identifiable iff for each pair of adjacent latent variables Lx, Ly, (1) at least one of Lx and Ly has non-Gaussian noise, or (2) at least one of them has a non-Gaussian ancestor and is not d-separated from the non-Gaussian component of this ancestor by the common causes of Lx and Ly. This identifiability result relaxes the non-Gaussianity requirements to only a (hopefully small) subset of variables, and accordingly elegantly extends the application scope of the structural model. Based on the above identifiability result, we further propose a practical algorithm to learn the structural model. We verify the correctness of the identifiability result and the effectiveness of the proposed method through empirical studies.

Two-echelon vehicle routing problem (2E-VRP) is an NP-hard combinatorial optimization problem and a basic mathematical model of modern city logistics. While it is difficult to obtain the optimal solution of 2E-VRP, this study finds a breakthrough that the structure of the optimal route planning for 2E-VRP is usually an embedded Hamiltonian graph. In the graph, routes can be drawn in a planar graph as Hamiltonian circuits without intersections. Based on this finding, an embedded Hamiltonian graph-guided heuristic algorithm is proposed to solve 2E-VRP. As a crucial part of the algorithm, an initialization scheme is designed to search for the farthest vertices from each route and insert the rest of the vertices. In the satellite-adjustment process, a dynamic adjustment for satellites scheme is proposed to adjust the state of satellites. The two schemes aim to construct Hamiltonian circuits with few intersections. Experiments have been conducted on 207 instances to demonstrate the effect of the proposed algorithm on solving 2E-VRP. Experimental results show that the proposed algorithm can obtain more solutions of 2E-VRP with significantly smaller objective-function values. Furthermore, the number of intersections in routes generated by the proposed algorithm is much less than those obtained by the compared algorithms. With the use of the two schemes, the embedded Hamiltonian graph-guided heuristic algorithm significantly outperforms the compared algorithms for 2E-VRP.

Depression is one of the most common mental illnesses, and the symptoms shown by patients are different, making it difficult to diagnose in the process of clinical practice and pathological research. Although researchers hope that artificial intelligence can contribute to the diagnosis and treatment of depression, the traditional centralized machine learning methods need to aggregate patient data, and the data privacy of patients with mental illness needs to be strictly confidential, which hinders machine learning algorithms’ clinical application. To solve the problem of medical data privacy with depression, in this article, we implement a study of federated learning to analyze and diagnose depression. First, we propose a general multiview federated learning framework using multisource data, which can extend any traditional machine learning model to support federated learning across different institutions or parties. Second, we employ later fusion methods to solve the problem of inconsistent time series of multiview data. Finally, we compare the federated framework with other cooperative learning frameworks in performance and discuss the related results. The experimental results show that in the case of participating in federated learning with enough participants, the prediction accuracy of depression score can reach 85.13%, which is about 15% higher than local training. When the number of participants is small and the amount of data is sufficient, the prediction accuracy of depression score can also reach 84.32%, and the improvement rate is about 9%.

Causal discovery from observational data is a fundamental problem in science. Though the linear non-Gaussian acyclic model (LiNGAM) has shown promising results in various applications, it still faces the following challenges in the data with multiple latent confounders: 1) how to detect the latent confounders and 2) how to uncover the causal relations among observed and latent variables. To address these two challenges, we propose a hybrid causal discovery method for the LiNGAM with multiple latent confounders (MLCLiNGAM). First, we utilize the constraint-based method to learn the causal skeleton. Second, we identify the causal directions, by conducting regression and independence tests on the adjacent pairs in the causal skeleton. Third, we detect the latent confounders with the help of the maximal clique patterns raised by the latent confounders and reconstruct the causal structure with latent variables. Theoretical results show the correctness and efficiency of the algorithms. We conduct extensive experiments on synthetic and real data, which illustrates the efficiency and effectiveness of the proposed algorithms.

Cardiotocography (CTG) monitoring is a primary tool to assess the health of the fetus. It is widely used to identify the risk of fetal distress. With the outbreak of big data and artificial intelligence, the use of machine learning to assist obstetricians in CTG interpretation is important to improve diagnostic accuracy and save medical resources. However, imbalanced CTG data brings great challenges to machine learning in intelligent multi-classification. Therefore, we propose a stacked model based on Extreme-Gradient Boosting and Random Forest (XGB-RF) feature selection. Firstly, we use the XGB-RF feature selection method to extract the CTG features of significant influence. Then, we implement the fusion idea of stacking to construct the intelligent model that has a strong anti-interference ability under unbalanced CTG data with selected diverse classifiers. The experimental results showed that compared with existing CTG classification models in the public CTG dataset, the proposed model has further improved the performance and reduced the misjudgment between different classes. According to the results, the accuracy is 96.08%, the F1 score is 93.36%, and area under the ROC curve (AUC) is 0.9883. This indicates that the proposed techniques have a great clinical significance in fetal health monitoring.

The realization of intelligent fetal monitoring is helpful to timely detect fetal abnormality and save medical costs. However, the probability of misjudgment tends to be high when modeling the imbalanced clinical data directly. Moreover, reliable interpretation of cardiotocography (CTG) requires multiple obstetricians to annotate, which consumes excessive time and labor cost. In this paper, a K-means method based on adaptive Genetic Algorithm-Fl balanced diversity Weighted Query-by-Committee (QBC) algorithm (KGA-WQBC) was proposed to solve the above problems. The proposed active learning algorithm was improved from the following two aspects. Firstly, an adaptive KGA operator was designed to conduct a preliminary selection of a large number of unlabeled samples, which increased the accuracy of the committee. Then, the WQBC operator was designed to improve the measurement of the committee's disagreement degree after data exploration. Compared with other initial sample selection strategies and disagreement degree weighting means, the results showed that only 41% of the labeled instances were used to make the evaluation index as high as 98.02%, which had the best overall performance. In conclusion, the proposed KGA-WQBC algorithm is reasonable and feasible, and effectively solves the problem of imbalanced data and reduces the cost of medical personnel labeling for intelligent fetal monitoring.

Discovering causal structures among latent factors from observed data is a particularly challenging problem. Despite some efforts for this problem, existing methods focus on the single-domain data only. In this paper, we propose Multi-Domain Linear Non-Gaussian Acyclic Models for LAtent Factors (MD-LiNA), where the causal structure among latent factors of interest is shared for all domains, and we provide its identification results. The model enriches the causal representation for multi-domain data. We propose an integrated two-phase algorithm to estimate the model. In particular, we first locate the latent factors and estimate the factor loading matrix. Then to uncover the causal structure among shared latent factors of interest, we derive a score function based on the characterization of independence relations between external influences and the dependence relations between multi-domain latent factors and latent factors of interest. We show that the proposed method provides locally consistent estimators. Experimental results on both synthetic and real-world data demonstrate the efficacy and robustness of our approach.

Identification of causal direction between a causal-effect pair from observed data has recently attracted much attention. Various methods based on functional causal models have been proposed to solve this problem, by assuming the causal process satisfies some (structural) constraints and showing that the reverse direction violates such constraints. The nonlinear additive noise model has been demonstrated to be effective for this purpose, but the model class does not allow any confounding or intermediate variables between a cause pair–even if each direct causal relation follows this model. However, omitting the latent causal variables is frequently encountered in practice. After the omission, the model does not necessarily follow the model constraints. As a consequence, the nonlinear additive noise model may fail to correctly discover causal direction. In this work, we propose a confounding cascade nonlinear additive noise model to represent such causal influences–each direct causal relation follows the nonlinear additive noise model but we observe only the initial cause and final effect. We further propose a method to estimate the model, including the unmeasured confounding and intermediate variables, from data under the variational auto-encoder framework. Our theoretical results show that with our model, the causal direction is identifiable under suitable technical conditions on the data generation process. Simulation results illustrate the power of the proposed method in identifying indirect causal relations across various settings, and experimental results on real data suggest that the proposed model and method greatly extend the applicability of causal discovery based on functional causal models in nonlinear cases.

Synthetic lethality (SL) is a very important concept for the development of targeted anticancer drugs. However, experimental methods for SL detection often suffer from various issues like high cost and low consistency across cell lines. Hence, computational methods for predicting novel SLs have recently emerged as complements for wet-lab experiments. In addition, SL data can be represented as a graph where nodes are genes and edges are the SL interactions. It is thus motivated to design advanced graph-based machine learning algorithms for SL prediction. In this paper, we propose a novel SL prediction method using Multi-view Graph Auto-Encoder (SLMGAE). We consider the SL graph as the main view and the graphs from other data sources (e.g., PPI, GO, etc.) as support views. Multiple Graph Auto-Encoders (GAEs) are implemented to reconstruct the graphs for different views. We further design an attention mechanism, which assigns different weights for support views, to combine all the reconstructed graphs for SL prediction. The overall SLMGAE model is then trained by minimizing both the reconstruction error and prediction error. Experimental results on the SynLethDB dataset show that SLMGAE outperforms state-of-the-arts. The case studies on novel predicted SLs also illustrate the effectiveness of our SLMGAE method.

Antepartum cardiotocography (CTG) monitoring is a crucial screening tool widely utilized to evaluate fetal wellbeing. However, the complexity and non-linearity of CTG usually result in inter-observer and intra-observer variability in a visual CTG interpretation using clinical guidelines. In this paper, a fuzzy C-means clustering based adaptive neuro-fuzzy inference system (FCM-ANFIS) was proposed to automatically classify CTG for antenatal fetal monitoring. Data visualization and spearman correlation analysis were implemented to select CTG features. Then, the fuzzy space was partitioned by using fuzzy Cmeans clustering algorithm, and the adjustment parameters were adjusted through the self-learning mechanism of neural networks and least squares algorithm. The experimental results show that the fuzzy space partition based on FCM clustering could improve the performance of ANFIS, and the proposed FCM-ANFIS model outperforms the state-of-the-art automatic classification of CTG models. In conclusion, the proposed FCM-ANIFIS model has promising learning ability and adaptability for the complexity and uncertainty of antenatal CTG interpretation.

Causal discovery aims to recover causal structures or models underlying the observed data. Despite its success in certain domains, most existing methods focus on causal relations between observed variables, while in many scenarios the observed ones may not be the underlying causal variables (e.g., image pixels), but are generated by latent causal variables or confounders that are causally related. To this end, in this paper, we consider Linear, Non-Gaussian Latent variable Models (LiNGLaMs), in which latent confounders are also causally related, and propose a Generalized Independent Noise (GIN) condition to estimate such latent variable graphs. Specifically, for two observed random vectors $\mathbf{Y}$ and $\mathbf{Z}$, GIN holds if and only if $\omega^{\intercal}\mathbf{Y}$ and $\mathbf{Z}$ are statistically independent, where $\omega$ is a parameter vector characterized from the cross-covariance between $\mathbf{Y}$ and $\mathbf{Z}$. From the graphical view, roughly speaking, GIN implies that causally earlier latent common causes of variables in $\mathbf{Y}$ d-separate $\mathbf{Y}$ from $\mathbf{Z}$. Interestingly, we find that the independent noise condition, i.e., if there is no confounder, causes are independent from the error of regressing the effect on the causes, can be seen as a special case of GIN. Moreover, we show that GIN helps locate latent variables and identify their causal structure, including causal directions. We further develop a recursive learning algorithm to achieve these goals. Experimental results on synthetic and real-world data demonstrate the effectiveness of our method.

In this paper, supereigenvalue and constrained supereigenvalue problems of an addition-min fuzzy matrix are investigated. In a peer-to-peer file sharing system, the download requirement of the terminals could be described by a constant constraint system with addition-min fuzzy relation inequalities. Without considering the constant constraint system, we first study the supereigenvalue problem. While considering the constant constraint system, the so-called constrained supereigenvalue is further proposed and investigated. A constrained supereigenvalue represents the ratio of the data-download quality to the data-sending quality under a constant constraint. In order to arouse the enthusiasm of the terminals to share their local file data, we aim to maximize the constrained supereigenvalue. A nonlinear programming approach is developed to find the unique maximum constrained supereigenvalue, with some illustrative examples.

Domain adaptation is becoming increasingly important for learning systems in recent years, especially with the growing diversification of data domains in real-world applications, such as the genetic data from various sequencing platforms and video feeds from multiple surveillance cameras. Traditional domain adaptation approaches target to design transformations for each individual domain so that the twisted data from different domains follow an almost identical distribution. In many applications, however, the data from diversified domains are simply dumped to an archive even without clear domain labels. In this article, we discuss the possibility of learning domain adaptations even when the data does not contain domain labels. Our solution is based on our new model, named domain adaption using cross-domain homomorphism (DACH in short), to identify intrinsic homomorphism hidden in mixed data from all domains. DACH is generally compatible with existing deep learning frameworks, enabling the generation of nonlinear features from the original data domains. Our theoretical analysis not only shows the universality of the homomorphism, but also proves the convergence of DACH for significant homomorphism structures over the data domains is preserved. Empirical studies on real-world data sets validate the effectiveness of DACH on merging multiple data domains for joint machine learning tasks and the scalability of our algorithm to domain dimensionality.

The discovery of causal relationships from the observational data is an important task. To identify the unique causal structure belonging to a Markov equivalence class, a number of algorithms, such as the linear non-Gaussian acyclic model (LiNGAM), have been proposed. However, two challenges remain to be met: 1) these algorithms fail to work on the data which follow linear structural equation model with Gaussian noise and 2) they misjudge the causal direction when the data contain additional measurement errors. In this paper, we propose an entropy-based two-phase iterative algorithm for arbitrary distribution data with additional measurement errors under some mild assumptions. In the first phase of the algorithm, based on the property that entropy can measure the amount of information behind the data with arbitrary distribution, we design a general approach for the identification of exogenous variable on both Gaussian and non-Gaussian data, and we give the corresponding theoretical derivation. In the second phase, to eliminate the effects of measurement errors, we revise the value of the exogenous variable by removing its measurement error and further use the revised value to remove its effect on the remaining variables. Experimental results on real-world causal structures are presented to demonstrate the effectiveness and stability of our method. We also apply the proposed algorithm on the mobile-base-station data with measurement errors, and the results further prove the effectiveness of our algorithm.

Domain adaptation is an important but challenging task. Most of the existing domain adaptation methods struggle to extract the domain-invariant representation on the feature space with entangling domain information and semantic information. Different from previous efforts on the entangled feature space, we aim to extract the domain invariant semantic information in the latent disentangled semantic representation (DSR) of the data. In DSR, we assume the data generation process is controlled by two independent sets of variables, i.e., the semantic latent variables and the domain latent variables. Under the above assumption, we employ a variational auto-encoder to reconstruct the semantic latent variables and domain latent variables behind the data. We further devise a dual adversarial network to disentangle these two sets of reconstructed latent variables. The disentangled semantic latent variables are finally adapted across the domains. Experimental studies testify that our model yields state-of-the-art performance on several domain adaptation benchmark datasets.

Identification of causal direction between a causal-effect pair from observed data has recently attracted much attention. Various methods based on functional causal models have been proposed to solve this problem, by assuming the causal process satisfies some (structural) constraints and showing that the reverse direction violates such constraints. The nonlinear additive noise model has been demonstrated to be effective for this purpose, but the model class is not transitive--even if each direct causal relation follows this model, indirect causal influences, which result from omitted intermediate causal variables and are frequently encountered in practice, do not necessarily follow the model constraints; as a consequence, the nonlinear additive noise model may fail to correctly discover causal direction. In this work, we propose a cascade nonlinear additive noise model to represent such causal influences--each direct causal relation follows the nonlinear additive noise model but we observe only the initial cause and final effect. We further propose a method to estimate the model, including the unmeasured intermediate variables, from data, under the variational auto-encoder framework. Our theoretical results show that with our model, causal direction is identifiable under suitable technical conditions on the data generation process. Simulation results illustrate the power of the proposed method in identifying indirect causal relations across various settings, and experimental results on real data suggest that the proposed model and method greatly extend the applicability of causal discovery based on functional causal models in nonlinear cases.

Learning causal structure from observational data has attracted much attention, and it is notoriously challenging to find the underlying structure in the presence of confounders (hidden direct common causes of two variables). In this paper, by properly leveraging the non-Gaussianity of the data, we propose to estimate the structure over latent variables with the so-called Triad constraints: we design a form of "pseudo-residual" from three variables, and show that when causal relations are linear and noise terms are non-Gaussian, the causal direction between the latent variables for the three observed variables is identifiable by checking a certain kind of independence relationship. In other words, the Triad constraints help us to locate latent confounders and determine the causal direction between them. This goes far beyond the Tetrad constraints and reveals more information about the underlying structure from non-Gaussian data. Finally, based on the Triad constraints, we develop a two-step algorithm to learn the causal structure corresponding to measurement models. Experimental results on both synthetic and real data demonstrate the effectiveness and reliability of our method.

Image matting is evolving for a wide range of applications including image/video editing. Sampling-based image matting aims to estimate the opacity of foreground objects by properly selecting a pair of foreground and background pixels for every unknown pixel. Sampling-based image matting is essentially an uncertain multicriteria optimization problem (UMCOP). It shows unique advantages in parallelization and handling spatially disconnected regions. However, sampling-based approaches encounter difficulty in accurately evaluating pixel pairs and efficiently optimizing the large-scale UMCOP. To address these two problems, a fuzzy multicriteria evaluation (FMCE) and a multiobjective evolutionary algorithm based on multicriteria decomposition (MOEA-MCD) are proposed. We model three fuzzy membership functions for three selection criteria and aggregate them by Einstein and averaging operators providing FMCE for pixel pairs. MOEA-MCD uses the heuristic information for each criterion by multicriteria decomposition that divides the single objective into multiple objectives and optimizes them simultaneously using a multiobjective optimizer with neighborhood grouping strategy. Experimental results show that FMCE accurately evaluates pixel pairs even in uncertain cases with low satisfaction degree of some evaluation criteria, and the heuristic information for each criterion enhances the population diversity of MOEA-MCD. MOEA-MCD outperforms state-of-the-art large-scale optimization approaches and sampling-based image matting approaches.

Causal discovery from a set of observations is one of the fundamental problems across several disciplines. For continuous variables, recently a number of causal discovery methods have demonstrated their effectiveness in distinguishing the cause from effect by exploring certain properties of the conditional distribution, but causal discovery on categorical data still remains to be a challenging problem, because it is generally not easy to find a compact description of the causal mechanism for the true causal direction. In this paper we make an attempt to find a way to solve this problem by assuming a two-stage causal process: the first stage maps the cause to a hidden variable of a lower cardinality, and the second stage generates the effect from the hidden representation. In this way, the causal mechanism admits a simple yet compact representation. We show that under this model, the causal direction is identifiable under some weak conditions on the true causal mechanism. We also provide an effective solution to recover the above hidden compact representation within the likelihood framework. Empirical studies verify the effectiveness of the proposed approach on both synthetic and real-world data.

Scalable causal discovery is an essential technology to a wide spectrum of applications, including biomedical studies and social network evolution analysis. To tackle the difficulty of high dimensionality, a number of solutions are proposed in the literature, generally dividing the original variable domain into smaller subdomains by computation intensive partitioning strategies. These approaches usually suffer significant structural errors when the partitioning strategies fail to recognize true causal edges across the output subdomains. Such a structural error accumulates quickly with the growing depth of recursive partitioning, due to the lack of correction mechanism over causally connected variables when they are wrongly divided into two subdomains, finally jeopardizing the robustness of the integrated results. This paper proposes a completely different strategy to solve the problem, powered by a lightweight random partitioning scheme together with a carefully designed merging algorithm over results from the random partitions. Based on the randomness properties of the partitioning scheme, we design a suite of tricks for the merging algorithm, in order to support propagation-based significance enhancement, maximal acyclic subgraph causal ordering, and order-sensitive redundancy elimination. Theoretical studies as well as empirical evaluations verify the genericity, effectiveness, and scalability of our proposal on both simulated and real-world causal structures when the scheme is used in combination with a variety of causal solvers known effective on smaller domains.

Understanding the causality behind the observational data is of great importance to a lot of real world applications, e.g., the improvement of Quality of Service. Non-Gaussianity has been exploited in numerous causal discovery methods for observational linear acyclic data. Transforming non-Gaussianity into indirect metrics is a conventional solution employed by existing methods, although this usually results in unreliable estimations or locally optimal solutions. In this work, we employs the excess kurtosis, a direct measure of non-Gaussianity, to establish a causal discovery method for linear non-Gaussian acyclic data. Firstly, we theoretically prove that an exogenous variable has the largest excess kurtosis when disturbance variables follow independent and identically distributions. Secondly, based on this property of exogenous variables, we propose an efficient exogenous variable identification algorithm, and develop a causal discovery method. Extensive experiment results verify the effectiveness and efficiency of the proposed approach.

Social causality study on human action sequences is useful and important to improve our understandings to human behaviors on online social networks. The redundant indirect causalities and unobserved confounding factors, such as homophily and simultaneity phenomena, contribute to the huge challenges on accurate causal discovery on such human actions. A causal relationship exists between two persons, if the actions of one person are significantly affected by the actions of the other person, while fairly independent of her/his own prior actions. In this paper, we design a systematic approach based on conditional independence testing to detect such asymmetric relations, even when there are latent confounders underneath the observational action sequences. Technically, a group of asymmetric independence tests are conducted to infer the loose causal directions between action sequence pairs, followed by another group of tests to distinguish different types of relationships, e.g., homophily and simultaneity. Finally, a causal structure learning method is employed to output pairwise causalities with redundant indirect causalities eliminated. Empirical evaluations on simulated data verify the effectiveness and scalability of our proposals. We also present four interesting patterns of causal relations found by our algorithm, on real Sina Weibo feeds, including two new patterns never reported in previous studies.

This paper presents a human-computer cooperative brain storm optimization algorithm, which is based on an improved brain storm optimization algorithm with human intelligence in computer game. In our algorithm, the initial population is provided with some better ideas obtained by computer game. Moreover, converging operation and diverging operation also employ the solutions from different players to generate ideas during evolution process. With the help of human-machine cooperation, our algorithm, integrating strategy development capabilities of players with brain storm optimization algorithm, is applied to solve some complex optimized problems. We apply the proposed method to two-echelon vehicle routing problem to verify its effectiveness and usefulness.

Differential privacy enables organizations to collect accurate aggregates over sensitive data with strong, rigorous guarantees on individuals' privacy. Previous work has found that under differential privacy, computing multiple correlated aggregates as a batch, using an appropriate strategy, may yield higher accuracy than computing each of them independently. However, finding the best strategy that maximizes result accuracy is non-trivial, as it involves solving a complex constrained optimization program that appears to be non-convex. Hence, in the past much effort has been devoted in solving this non-convex optimization program. Existing approaches include various sophisticated heuristics and expensive numerical solutions. None of them, however, guarantees to find the optimal solution of this optimization problem.

MOTIVATION: Genome-wide association studies (GWASs) are commonly applied on human genomic data to understand the causal gene combinations statistically connected to certain diseases. Patients involved in these GWASs could be re-identified when the studies release statistical information on a large number of single-nucleotide polymorphisms. Subsequent work, however, found that such privacy attacks are theoretically possible but unsuccessful and unconvincing in real settings. RESULTS: We derive the first practical privacy attack that can successfully identify specific individuals from limited published associations from the Wellcome Trust Case Control Consortium (WTCCC) dataset. For GWAS results computed over 25 randomly selected loci, our algorithm always pinpoints at least one patient from the WTCCC dataset. Moreover, the number of re-identified patients grows rapidly with the number of published genotypes. Finally, we discuss prevention methods to disable the attack, thus providing a solution for enhancing patient privacy. AVAILABILITY AND IMPLEMENTATION: Proofs of the theorems and additional experimental results are available in the support online documents. The attack algorithm codes are publicly available at https://sites.google.com/site/zhangzhenjie/GWAS_attack.zip. The genomic dataset used in the experiments is available at http://www.wtccc.org.uk/ on request.

Differential privacy is a promising privacy-preserving paradigm for statistical query processing over sensitive data. It works by injecting random noise into each query result such that it is provably hard for the adversary to infer the presence or absence of any individual record from the published noisy results. The main objective in differentially private query processing is to maximize the accuracy of the query results while satisfying the privacy guarantees. Previous work, notably Li et al. [2010], has suggested that, with an appropriate strategy, processing a batch of correlated queries as a whole achieves considerably higher accuracy than answering them individually. However, to our knowledge there is currently no practical solution to find such a strategy for an arbitrary query batch; existing methods either return strategies of poor quality (often worse than naive methods) or require prohibitively expensive computations for even moderately large domains. Motivated by this, we propose a low-rank mechanism (LRM), the first practical differentially private technique for answering batch linear queries with high accuracy. LRM works for both exact (i.e., ϵ-) and approximate (i.e., (ϵ, δ)-) differential privacy definitions. We derive the utility guarantees of LRM and provide guidance on how to set the privacy parameters, given the user's utility expectation. Extensive experiments using real data demonstrate that our proposed method consistently outperforms state-of-the-art query processing solutions under differential privacy, by large margins.

Differential privacy is a promising privacy-preserving paradigm for statistical query processing over sensitive data. It works by injecting random noise into each query result, such that it is provably hard for the adversary to infer the presence or absence of any individual record from the published noisy results. The main objective in differentially private query processing is to maximize the accuracy of the query results, while satisfying the privacy guarantees. Previous work, notably the matrix mechanism [16], has suggested that processing a batch of correlated queries as a whole can potentially achieve considerable accuracy gains, compared to answering them individually. However, as we point out in this paper, the matrix mechanism is mainly of theoretical interest; in particular, several inherent problems in its design limit its accuracy in practice, which almost never exceeds that of naïve methods. In fact, we are not aware of any existing solution that can effectively optimize a query batch under differential privacy. Motivated by this, we propose the Low-Rank Mechanism (LRM), the first practical differentially private technique for answering batch queries with high accuracy, based on a low rank approximation of the workload matrix. We prove that the accuracy provided by LRM is close to the theoretical lower bound for any mechanism to answer a batch of queries under differential privacy. Extensive experiments using real data demonstrate that LRM consistently outperforms state-of-the-art query processing solutions under differential privacy, by large margins.