Large-scale global optimization (LSGO) presents significant challenges due to the high dimensionality and complexity of the search space. We propose an IGKT (Inter-Generational Knowledge Transfer) optimization method incorporating a novel knowledge transfer mechanism to address these challenges. The proposed mechanism enables the algorithm to reduce reliance on stochastic exploration and enhance convergence efficiency by transferring information from the best-performing individuals across generations, guiding the population toward promising regions in the search space. Experimental results on the CEC2013 LSGO benchmark suite demonstrate that IGKT outperforms several state-of-the-art algorithms across various tested functions, achieving superior convergence speed and solution quality. Additionally, the IGKT framework handles both separable and non-separable functions and tasks, including those with overlapping and highly coupled variables. In summary, IGKT represents a powerful tool for addressing complex, high-dimensional optimization problems, providing a robust and adaptable solution for LSGO.

Competitive swarm optimizer (CSO) is an efficient and effective swarm intelligence approach, especially for large-scale optimization. This paper presents an enhanced version of CSO termed improved CSO with linear population reduction (L-ICSO). The novel triple-individuals competitive mechanism is introduced to strengthen the optimization performance of L-ICSO, and the linear population reduction mechanism from L-SHADE is integrated into L-ICSO to highlight the explorative search in the initial phase of optimization and emphasize the exploitative behavior in the late phase. We conduct comprehensive numerical experiments in 100-dimensional CEC2017 benchmark functions. Ten state-of-the-art optimizers such as L-SHADE, jSO, L-SHADE-cnEpSin, and the original CSO are employed as competitor algorithms. The Mann–Whitney U and Holm multiple comparison tests are used to measure the statistical significance between L-ICSO and competitor algorithms. The experimental results and statistical analysis confirm the efficiency and effectiveness of our proposed L-ICSO in addressing large-scale optimization problems. The source code of L-ICSO can be found at https://github.com/RuiZhong961230/L-ICSO.

This paper presents a novel approach to addressing the long-sequence problem in high-resolution medical images for Vision Transformers (ViTs). Using smaller patches as tokens can enhance ViT performance, but quadratically increases computation and memory requirements. Therefore, the common practice for applying ViTs to high-resolution images is either to: (a) employ complex sub-quadratic attention schemes or (b) use large to medium-sized patches and rely on additional mechanisms within the model to capture the spatial hierarchy of details. We propose Symmetrical Hierarchical Forest (SHF), a lightweight approach that adaptively patches the input image to increase token information density and encode hierarchical spatial structures into the input embedding. We then apply a reverse depatching scheme to the output embeddings of the transformer encoder, eliminating the need for convolution-based decoders. Unlike previous methods that modify attention mechanisms \wahib{or use a complex hierarchy of interacting models}, SHF can be retrofitted to any ViT model to allow it to learn the hierarchical structure of details in high-resolution images without requiring architectural changes. Experimental results demonstrate significant gains in computational efficiency and performance: on the PAIP WSI dataset, we achieved a 3$\sim$32$\times$ speedup or a 2.95\% to 7.03\% increase in accuracy (measured by Dice score) at a $64K^2$ resolution with the same computational budget, compared to state-of-the-art production models. On the 3D medical datasets BTCV and KiTS, training was 6$\times$ faster, with accuracy gains of 6.93\% and 5.9\%, respectively, compared to models without SHF.

In this study, we propose a novel deep neural model, CWTLNet, specifically designed to address the unique characteristics of cryptocurrency trading, including significant short-term volatility, which operates around the clock, and is prone to sudden price jumps and drops. The architecture consists of two branches. The CWT branch employs a continuous wavelet transform (CWT) to extract time-frequency feature maps from the input time series. These feature maps are then processed by a stack of CWT Blocks, which are composed of residual structures with two Conv1D layers, allowing the model to capture local, ultra-short-term patterns in the data. The linear branch first decomposes the time series into its trend and residual components. Each component is modeled separately using linear models and subsequently recombined. The linear branch is particularly effective at capturing the periodic patterns embedded in the time series. Finally, the outputs of the two branches are fused through a gated mechanism with a temperature coefficient, enabling adaptive weighting of the branch outputs. Experimental results demonstrate that CWTLNet outperforms commonly used models — including LSTM, Transformer, and DLinear — on minute-level trading data for seven different cryptocurrencies. For example, The performance of CWTLNet on the Bitcoin dataset, in terms of Mean Squared Error (MSE) and Mean Absolute Error (MAE), shows improvements of at least 2.7% and 1.7%, respectively, compared to the aforementioned three models.

In recent years, numerous studies have explored how to allocate limited defense resource among meters to increase difficulties for launching attacks in power systems. However, existing studies often simplify this problem by assuming the linearity of attack cost functions, which creates an inevitable gap between theoretical analysis and practical scenarios. In this article, general nonlinear attack cost functions are considered in the formulation of the optimal defense resource problem, resulting in a mixed-integer nonlinear programming problem. This poses a challenging task for numerical methods or popular commercial solvers. To address this issue, an advanced slime mould algorithm with specialized initialization and evolutionary schemes is proposed. Specifically, a customized initialization strategy is designed such that the population can be easily initialized within the nonconvex feasible region aligning with the nonlinear constraints. Besides, in the evolutionary process, the fitness function is well-designed to encourage the individuals violating nonlinear constraints to evolve toward feasible directions. Comprehensive case studies verify that, compared to the state-of-the-art solvers, the proposed approach can achieve satisfactory defense resource allocation results in terms of feasibility, optimality, and real-time performance.

Overfitting is a well-documented and studied issue in supervised learning. Human experts have been designing methods to reduce over-fitting by observing the validation knowledge, e.g., learning rate schedules, dropout, and adversarial training. We propose a validation-loss landscape exploration/exploitation method called VKI (Validation Knowledge Inheritance). We reformulate the traditional gradient optimization problem as a reinforcement learning task to explore the validation-loss landscape. In particular, by treating the gradient descent process as a Markov Decision Process (MDP), where the validation losses are treated as the costs, we train a Q-network as a controller to learn the future rewards and later use it to decrease the validation loss of workers. We conduct the experiments in two reduced gradient action spaces on the MNIST, CIFAR-10, and CIFAR-100 datasets with naive dense neural networks and ResNet-56. Our results show that VKI could rediscover hyperparameters schedule rules and improve the models’ training and generalization by only exploring/exploiting the validation loss landscape, e.g., increasing the learning rate accelerated training and penalty factor when overfitting. In comparison to other metaHPO (Hyper Parameter Optimization) methods, we empirically show that VKI can leverage its weights-loss regression of Q-Net to enable the transfer to the target dataset without heavy retraining but with light fine-tuning.

This paper proposes a novel decomposition method: Adjacent Intensity Matrix with Linkage Identification (AIM-LI) for large-scale optimization problems (LSOPs) in noisy environments. The most advanced differential grouping (DG)-based methods such as DG2, Recursive DG, and dual DG are environmentally sensitive, and the presence of noise may misguide these methods to identify the originally separable decision variables as non-separable. Although the noise may affect the absolute intensity of interactions, we can detect the intensity between separable and non-separable decision variables with potential relative differences. This difference can help us classify the separability. Based on this hypothesis, we define the interaction intensity of pairwise decision variables and save them to AIM. A significant intensity (SI) is selected from AIM to transform the AIM into the dependency structure matrix (DSM). To verify the feasibility of AIM-LI, we first run the pre-experiments and mathematically analyze the possibility of interaction identification in noisy environments. Furthermore, we implement a decomposition experiment on CEC2013 with different strengths of noise. Theoretical analysis and experimental results show that our proposal has great potential to detect and classify the separability of problems in noisy environments.

When training neural networks, the weights of the model are updated at each optimization step, and the older weights are discarded. In this paper, we propose a method called, Training Knowledge Inheritance (TKI), to use the knowledge about the progression of weight and loss data in reducing overfitting and improving the generalization in the later stages of training. We reformulate the traditional gradient optimization problem as a reinforcement learning task. In particular, by treating the trainable weight space as an environment, the learning rate as action, and the validation accuracies as the rewards, we train a Q-network (controller) to learn the discounted future validation accuracy and guide the later training of another network (worker). We conduct the experiments on the MNIST, CIFAR-10, and CIFAR-100 datasets with naive dense neural networks and ResNet-56. Our results show that TKI could rediscover learning rate schedule rules similar to previous works, including increasing, decaying, and cyclical repeating.