Influence maximization (IM) algorithms aim to iteratively identify a seed set that could maximize the spreading range. In this paper, we concentrate on the hypergraph influence maximization (HyperIM) problem. The overlapping neighborhood caused by the higher-order interactions leads to an overestimation of the diffusion capability of candidate nodes. Moreover, selecting the candidate node with a high infected probability as a new seed node is low payoff with a small influence range gain. Thus, we develop adaptive metrics and propose two algorithms, i.e., high adaptive contact efficiency (HACE) algorithm and high contact with a low infected probability (HCLI) algorithm. First, we penalize the contribution of the neighborhood of the seed set to the evaluation of the influence gain to the seed set to correct the impact of overlapping influence. Additionally, the proposed HACE algorithm uses the being contacted capability to reveal the infected possibility of candidate nodes, while the proposed HCLI algorithm estimates the global infected probability of nodes. The experiments and analysis on eight real-world hypergraphs demonstrate the better balance of the HACE and HCLI algorithms than the state-of-the-art (SOTA) algorithms in selecting influential seed set and ensuring computational efficiency. Compared with the existing SOTA algorithms, HACE and HCLI run at least ten times faster than SOTA, and at most nearly 70 times faster. On large-scale hypergraphs, the HACE and HCLI algorithms still show great computational efficiency and significantly improved performances compared with other low-time complexity algorithms.

Learning unnormalized statistical models (e.g., energy-based models) is computationally challenging due to the complexity of handling the partition function. To eschew this complexity, noise-contrastive estimation (NCE) has been proposed by formulating the objective as the logistic loss between the real data and the artificial noise. However, previous research indicates that NCE may perform poorly in many tasks due to its flat loss landscape and slow convergence. In this paper, we study a direct approach for optimizing the negative log-likelihood of unnormalized models through the lens of compositional optimization. To tackle the partition function, a noise distribution is introduced such that the log partition function can be expressed as a compositional function whose inner function can be estimated using stochastic samples. Consequently, the objective can be optimized via stochastic compositional optimization algorithms. Despite being a simple method, we demonstrate it is more favorable than NCE by (1) establishing a fast convergence rate and quantifying its dependence on the noise distribution through the variance of stochastic estimators; (2) developing better results in Gaussian mean estimation by showing our method has a much favorable loss landscape and enjoys faster convergence; (3) demonstrating better performance on various applications, including density estimation, out-of-distribution detection, and real image generation.

Continual learning aims to learn multiple tasks sequentially while preserving prior knowledge, but faces the challenge of catastrophic forgetting when adapting to new tasks. Recently, approaches leveraging pre-trained models have gained increasing popularity in mitigating this issue, due to the strong generalization ability of foundation models. To adjust pre-trained models for new tasks, existing methods usually employ low-rank adaptation, which restricts parameter updates to a fixed low-rank subspace. However, constraining the optimization space inherently compromises the model's learning capacity, resulting in inferior performance. To address this limitation, we propose Continuous Subspace Optimization for Continual Learning (CoSO) to fine-tune the model in a series of subspaces rather than a single one. These sequential subspaces are dynamically determined through the singular value decomposition of the gradients. CoSO updates the model by projecting gradients onto these subspaces, ensuring memory-efficient optimization. To mitigate forgetting, the optimization subspace of each task is constrained to be orthogonal to the historical task subspace. During task learning, CoSO maintains a task-specific component that captures the critical update directions for the current task. Upon completing a task, this component is used to update the historical task subspace, laying the groundwork for subsequent learning. Extensive experiments on multiple datasets demonstrate that CoSO significantly outperforms state-of-the-art methods, especially in challenging scenarios with long task sequences.

Learning unnormalized statistical models (e.g., energy-based models) is computationally challenging due to the complexity of handling the partition function. To eschew this complexity, noise-contrastive estimation (NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise. However, as found in previous works, NCE may perform poorly in many tasks due to its flat loss landscape and slow convergence. In this paper, we study a direct approach for optimizing the negative log-likelihood of unnormalized models from the perspective of compositional optimization. To tackle the partition function, a noise distribution is introduced such that the log partition function can be written as a compositional function whose inner function can be estimated with stochastic samples. Hence, the objective can be optimized by stochastic compositional optimization algorithms. Despite being a simple method, we demonstrate that it is more favorable than NCE by (1) establishing a fast convergence rate and quantifying its dependence on the noise distribution through the variance of stochastic estimators; (2) developing better results for one-dimensional Gaussian mean estimation by showing our objective has a much favorable loss landscape and hence our method enjoys faster convergence; (3) demonstrating better performance on multiple applications, including density estimation, out-of-distribution detection, and real image generation.

The convolutional neural network (CNN) has emerged as a powerful tool for decoding electroencephalogram (EEG), which owns the potential use in the event-related potential-based brain-computer interface (ERP-BCI). However, the intra-individual difference of ERP makes the traditional learning models trained on static EEG data hard to decode when the EEG features vary along the time, which limits the long-time performance of the model. Addressing this problem, this study proposes a three-dimension CNN (3D-CNN)-based model to decode the ERPs dynamically. As input, the EEG is transformed into a brain topographic map stream along time. Then the 3D-CNN applies three-dimension kernels to capture the dynamical characteristic of spatial feature at several time points. Ten subjects participated in a cross-time task for 6 or 12[Formula: see text]h. The 3D-CNN shows higher accuracies and shorter computational cost than the baseline models of the 2D-CNN, the long short term memory (LSTM), the back propagation (BP), and the fisher linear discriminant analysis (FLDA) when detecting the ERPs. In addition, four schemes of the 3D-CNN are compared to explore the influence of the structure on the performance. This result demonstrates advanced robustness of the 3D-CNN kernel to the intra-individual EEG difference, helping to launch a more practical EEG decoding model for a long-time use.