In this paper, we present an open-source pure-Python library called PyPop7 for black-box optimization (BBO). As population-based methods (e.g., evolutionary algorithms, swarm intelligence, and pattern search) become increasingly popular for BBO, the design goal of PyPop7 is to provide a unified API and elegant implementations for them, particularly in challenging high-dimensional scenarios. Since these population-based methods easily suffer from the notorious curse of dimensionality owing to random sampling as one of core operations for most of them, recently various improvements and enhancements have been proposed to alleviate this issue more or less mainly via exploiting possible problem structures: such as, decomposition of search distribution or space, low-memory approximation, low-rank metric learning, variance reduction, ensemble of random subspaces, model self-adaptation, and fitness smoothing. These novel sampling strategies could better exploit different problem structures in high-dimensional search space and therefore they often result in faster rates of convergence and/or better qualities of solution for large-scale BBO. Now PyPop7 has covered many of these important advances on a set of well-established BBO algorithm families and also provided an open-access interface to adding the latest or missed black-box optimizers for further functionality extensions. Its well-designed source code (under GPL-3.0 license) and full-fledged online documents (under CC-BY 4.0 license) have been freely available at https://github.com/Evolutionary-Intelligence/pypop and https://pypop.readthedocs.io, respectively.

In the post-Moore era, main performance gains of black-box optimizers are increasingly depending on parallelism, especially for large-scale optimization (LSO). Here we propose to parallelize the well-established covariance matrix adaptation evolution strategy (CMA-ES) and in particular its one latest LSO variant called limited-memory CMA-ES (LM-CMA). To achieve efficiency while approximating its powerful invariance property, we present a multilevel learning-based meta-framework for distributed LM-CMA. Owing to its hierarchically organized structure, Meta-ES is well-suited to implement our distributed meta-framework, wherein the outer-ES controls strategy parameters while all parallel inner-ESs run the serial LM-CMA with different settings. For the distribution mean update of the outer-ES, both the elitist and multi-recombination strategy are used in parallel to avoid stagnation and regression, respectively. To exploit spatiotemporal information, the global step-size adaptation combines Meta-ES with the parallel cumulative step-size adaptation. After each isolation time, our meta-framework employs both the structure and parameter learning strategy to combine aligned evolution paths for CMA reconstruction. Experiments on a set of large-scale benchmarking functions with memory-intensive evaluations, arguably reflecting many data-driven optimization problems, validate the benefits (e.g., effectiveness w.r.t. solution quality, and adaptability w.r.t. second-order learning) and costs of our meta-framework.

Decomposition-based optimizers have shown very promising computational and convergence performance on many large-scale real-parameter optimization problems. Among them, a class of recently proposed cooperative coevolutionary algorithms (CCEAs) and a type of conventional block coordinate descent algorithms (BCDAs) are arguably the two most representative frameworks applied to the minimization of non-differentiable and differentiable objective function, respectively. This paper explores the connections between CCEAs and BCDAs, which can help gain deeper understandings of CCEAs. First, we propose a unified analytical framework for both CCEAs and BCDAs to capture the common game-theoretic nature by combining their respective theoretical advances. Second, many real-world objective functions are non-additively separable, where all decision variables interact with each other in a direct or indirect fashion. However, most of the state-of-the-art decomposition strategies for CCEAs can only capture the simple additive separability and cannot recognize the non-additive separability, but which has been widely studied in the BCDAs context. The performance of CCEAs on such functions is yet to be fully understood since intuitively CCEAs seem to be not suitable for them. We use the proposed framework to confirm and extend CCEAs' applicability to a special class of non-additively separable functions. Finally, based on the proposed framework, we provide two practical suggestions as well as a suite of new test functions to help design practically better CCEAs for large-scale optimization.