Graph invariant learning (GIL) seeks invariant relations between graphs and labels under distribution shifts. Recent works try to extract an invariant subgraph to improve out-of-distribution (OOD) generalization, yet existing approaches either lack explicit control over compactness or rely on hard top-$k$k selection that shrinks the solution space and is only partially differentiable. In this paper, we provide an in-depth analysis of the drawbacks of some existing works and propose a few general principles for invariant subgraph extraction: 1) separability, as encouraged by our sparsity-driven mechanism, to filter out the irrelevant common features; 2) softness, for a broader solution space; and 3) differentiability, for a soundly end-to-end optimization pipeline. Specifically, building on optimal transport, we propose Graph Sinkhorn Attention (GSINA), a fully differentiable, cardinality-constrained attention mechanism that assigns sparse-yet-soft edge weights via Sinkhorn iterations and induces node attention. GSINA provides explicit controls for separability and softness, and uses a Gumbel reparameterization to stabilize training. It convergence behavior is also theoretically studied. Extensive empirical experimental results on both synthetic and real-world datasets validate its superiority.

As deep learning develops, the major research methodologies of time series forecasting can be divided into two categories, i.e., iterative and direct methods. In the iterative methods, since a small amount of error is produced at each time step, the recursive structure can potentially lead to large error accumulations over longer forecasting horizons. Although the direct methods can avoid this puzzle involved in the iterative methods, they face abuse of conditional independence among time points. This impractical assumption can also lead to biased models. To solve these challenges, we propose a direct approach for multi-horizon probabilistic forecasting, which can effectively characterize the dependence across future horizons. Specifically, we consider the multi-horizon target as a random vector. The direction of the vector embodies the temporal dependence, and the length of the vector measures the overall scale across each horizon. Therefore, we respectively apply the von Mises-Fisher (VMF) distribution and the truncated normal distribution to characterize the target vector’s angle and magnitude in our model. Extensive results demonstrate the superiority of our framework over eight state-of-the-art methods.

Machine Learning Model Deployment as a Service (MLaaS) has surged in popularity, offering substantial business value. However, the significant resources and costs required to train models have raised concerns about Model Stealing Attacks (MSAs), where attackers create a clone model to replicate the knowledge of a victim model without access to its parameters. In data-free MSA, attackers also lack access to the training data for the victim model. In this setting, existing MSA methods rely on Generative Adversarial Networks (GANs) to generate images to query the victim model. However, GANs are known to suffer from model collapse, resulting in limited diversity in generated images. The lack of diversity in generated images will significantly impact the accuracy of the clone model, especially in stealing robust models trained with adversarial training. Recent studies have demonstrated that Denoising Diffusion Probabilistic Models (DDPMs) outperform GANs in generating images with greater diversity. In our data-free MSA framework, using DDPM as the generator to steal robust models significantly increases the effectiveness, improving the accuracy of the clone model from 21.34% to 60.23% compared to the GANs-based approach DFME, and requires fewer queries. We further use denoise diffusion GANs to address the problem of low sampling speed of DDPM, while retaining the advantage of its high sample diversity and obtaining better results.

Various probabilistic time series forecasting models have sprung up and shown remarkably good performance. However, the choice of model highly relies on the characteristics of the input time series and the fixed distribution that model is based on. Due to the fact that the probability distributions cannot be averaged over different models straightforwardly, the current time series model ensemble methods cannot be directly applied to improve the robustness and accuracy of forecasting. To address this issue, we propose pTSE, a multi-model distribution ensemble method for probabilistic forecasting based on Hidden Markov Model (HMM). pTSE only takes off-the-shelf outputs from member models without requiring further information about each model. Besides, we provide a complete theoretical analysis of pTSE to prove that the empirical distribution of time series subject to an HMM will converge to the stationary distribution almost surely. Experiments on benchmarks show the superiority of pTSE over all member models and competitive ensemble methods.