This paper addresses differential inference in time-varying parametric probabilistic models, like graphical models with changing structures. Instead of estimating a high-dimensional model at each time and estimating changes later, we directly learn the differential parameter, i.e., the time derivative of the parameter. The main idea is treating the time score function of an exponential family model as a linear model of the differential parameter for direct estimation. We use time score matching to estimate parameter derivatives. We prove the consistency of a regularized score matching objective and demonstrate the finite-sample normality of a debiased estimator in high-dimensional settings. Our methodology effectively infers differential structures in high-dimensional graphical models, verified on simulated and real-world datasets. The code reproducing our experiments can be found at: \url{https://github.com/Leyangw/tsm}.

This paper introduces a novel iterative method for missing data imputation that sequentially reduces the mutual information between data and the corresponding missingness mask. Inspired by GAN-based approaches that train generators to decrease the predictability of missingness patterns, our method explicitly targets this reduction in mutual information. Specifically, our algorithm iteratively minimizes the KL divergence between the joint distribution of the imputed data and missingness mask, and the product of their marginals from the previous iteration. We show that the optimal imputation under this framework can be achieved by solving an ODE whose velocity field minimizes a rectified flow training objective. We further illustrate that some existing imputation techniques can be interpreted as approximate special cases of our mutual-information-reducing framework. Comprehensive experiments on synthetic and real-world datasets validate the efficacy of our proposed approach, demonstrating its superior imputation performance. Our implementation is available at \url{https://github.com/yujhml/MIRI-Imputation}.

Optimising probabilistic models is a well-studied field in statistics. However, its connection with the training of generative models remains largely under-explored. In this paper, we show that the evolution of time-varying generative models can be projected onto an exponential family manifold, naturally creating a link between the parameters of a generative model and those of a probabilistic model. We then train the generative model by moving its projection on the manifold according to the natural gradient descent scheme. This approach also allows us to efficiently approximate the natural gradient of the KL divergence without relying on MCMC for intractable models. Furthermore, we propose particle versions of the algorithm, which feature closed-form update rules for any parametric model within the exponential family. Through toy and real-world experiments, we validate the effectiveness of the proposed algorithms. The code of the proposed algorithms can be found at \url{https://github.com/anewgithubname/iNGD}.

In recent years, diffusion-based methods have outperformed traditional models in many cloud removal tasks due to their strong generative capabilities. However, these methods face the challenges of long inference time and poor recovery effect in cloud regions. To address this issue, this paper proposes a Patch-Based Improved Denoising Diffusion Model with a high grey value attention for cloud removal in optical remote sensing images. We introduce an overlapping fixed-sized patches method in Improved Denoising Diffusion Model. The patch-based diffusion modeling approach enables size-agnostic image restoration by employing a guided denoising process with smoothed noise estimates across overlapping patches during inference. Additionally, we introduce a high grey value attention module, specifically designed to focus on thick cloud regions, enhancing attention on areas with relatively high grey values within the image. When compared with other existing cloud removal models on the RICE dataset, our model outperformed them in terms of both peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). Qualitative results demonstrate that the proposed method effectively removes clouds from images while preserving texture details. Ablation studies further confirm the effectiveness of the high grey value attention module. Overall, the proposed model delivers superior cloud removal performance compared to existing state-of-the-art methods.