Understanding the behavior of numerical metaheuristic optimization algorithms is critical for advancing their development and application. Traditional visualization techniques, such as convergence plots, trajectory mapping, and fitness landscape analysis, often fall short in illustrating the structural dynamics of the search process, especially in high-dimensional or complex solution spaces. To address this, we propose a novel representation and visualization methodology that clusters solution candidates explored by the algorithm and tracks the evolution of cluster memberships across iterations, offering a dynamic and interpretable view of the search process. Additionally, we introduce two metrics – algorithm stability and algorithm similarity – to quantify the consistency of search trajectories across runs of an individual algorithm and the similarity between different algorithms, respectively. We apply this methodology on a set of ten numerical metaheuristic algorithms, revealing insights into their stability and comparative behaviors, thereby providing a deeper understanding of their search dynamics.

Algorithm selection, aiming to identify the best algorithm for a given problem, plays a pivotal role in continuous black-box optimization. A common approach involves representing optimization functions using a set of features, which are then used to train a machine learning meta-model for selecting suitable algorithms. Various approaches have demonstrated the effectiveness of these algorithm selection meta-models. However, not all evaluation approaches are equally valid for assessing the performance of metamodels. We highlight methodological issues that frequently occur in the community and should be addressed when evaluating algorithm selection approaches. First, we identify flaws with the "leave-instance-out" evaluation technique. We show that non-informative features and meta-models can achieve high accuracy, which should not be the case with a well-designed evaluation framework. Second, we demonstrate that measuring the performance of optimization algorithms with metrics sensitive to the scale of the objective function requires careful consideration of how this impacts the construction of the meta-model, its predictions, and the model's error. Such metrics can falsely present overly optimistic performance assessments of the meta-models. This paper emphasizes the importance of careful evaluation, as loosely defined methodologies can mislead researchers, divert efforts, and introduce noise into the field.

Exploratory Landscape Analysis (ELA) features are the most common choice for representing single-objective con-tinuous optimization problem instances in Algorithm Selection (AS) methods. However, ELA features have been shown to have low generalization to unseen problems. Recently, it has also been shown that scaling objective function values before ELA feature calculation can be beneficial for AS methods evaluated on the Black-Box Optimization Benchmarking suite. In this paper, we aim to evaluate whether the same holds true for other benchmarks. In particular, we take into account four different benchmark suites and investigate the ability of an AS model trained on one benchmark to generalize to another benchmark. We also evaluate the impact of scaling objective function values before ELA feature calculation on AS performance. We observe a benefit of scaling objective function values in the case of the use of all ELA features together, however, conflicting outcomes are obtained when different feature groups are used individually. Our analysis shows no benefit of the joint use of the ELA features calculated on the original and scaled objective function values.

This study examines the generalization ability of algorithm performance prediction models across various bench-mark suites. Comparing the statistical similarity between the problem collections with the accuracy of performance prediction models that are based on exploratory landscape analysis features, we observe that there is a positive correlation between these two measures. Specifically, when the high-dimensional feature value distributions between training and testing suites lack statistical significance, the model tends to generalize well, in the sense that the testing errors are in the same range as the training errors. Two experiments validate these findings: one involving the standard benchmark suites, the BBOB and CEC collections, and another using five collections of affine combinations of BBOB problem instances.

The application of machine learning (ML) models to the analysis of optimization algorithms requires the representation of optimization problems using numerical features. These features can be used as input for ML models that are trained to select or to configure a suitable algorithm for the problem at hand. Since in pure black-box optimization information about the problem instance can only be obtained through function evaluation, a common approach is to dedicate some function evaluations for feature extraction, e.g., using random sampling. This approach has two key downsides: (1) It reduces the budget left for the actual optimization phase, and (2) it neglects valuable information that could be obtained from a problem-solver interaction.

Nowadays, it is really important and crucial to follow the new biomedical knowledge that is presented in scientific literature. To this end, Information Extraction pipelines can help to automatically extract meaningful relations from textual data that further require additional checks by domain experts. In the last two decades, a lot of work has been performed for extracting relations between phenotype and health concepts, however, the relations with food entities which are one of the most important environmental concepts have never been explored. In this study, we propose FooDis, a novel Information Extraction pipeline that employs state-of-the-art approaches in Natural Language Processing to mine abstracts of biomedical scientific papers and automatically suggests potential cause or treat relations between food and disease entities in different existing semantic resources. A comparison with already known relations indicates that the relations predicted by our pipeline match for 90% of the food-disease pairs that are common in our results and the NutriChem database, and 93% of the common pairs in the DietRx platform. The comparison also shows that the FooDis pipeline can suggest relations with high precision. The FooDis pipeline can be further used to dynamically discover new relations between food and diseases that should be checked by domain experts and further used to populate some of the existing resources used by NutriChem and DietRx.

Accurate and reliable forecasting is a crucial task in many different domains. The selection of a forecasting algorithm that is suitable for a specific time series can be a challenging task, since the algorithms’ performance depends on the time-series properties, as well as the properties of the forecasting algorithms. The methodology and analysis presented in this paper are contributing towards understanding the performance of time-series forecasting methods. Instead of using time-series meta-features only to obtain a good meta-model that can predict the performance of a forecasting algorithm, the methodology can link which features are important for which forecasting methods. We used time-series meta-features extracted using the tsfresh and catch22 libraries. We also found that the importance of the meta-features changes depending on the meta-model that is used. There are only a few meta-features that always appear important for a given forecasting method no matter which meta-model will be used for learning, which further provides opportunities to select a model-agnostic feature portfolio. In addition, different feature importance techniques can provide different results that are related to the methodology that is used by the meta-model. By using the feature importance obtained by a meta-model and a specified feature importance technique, we can define a representation of a forecasting method behavior, which can further provide an insight into which forecasting methods have similar behavior.

Named Entity Recognition (NER) and Named Entity Linking (NEL) are key tasks in Information Extraction, addressing the identification and normalization of entity mentions from raw text. In the domain of food and nutrition, there have been several NER methods already developed, however, when applied to scientific text, they fail to generalize and produce large performance degradation. This introduces the need for new food NER and NEL models, developed specifically for extracting food entities from scientific text. In this paper, we present a scientific food NER and NEL model, SciFoodNER, obtained by fine-tuning transformer models on a corpus of scientific abstracts annotated with food entities. The models can identify mentions of food entites from raw text, and link the food entities to the Hansard Taxonomy, the FoodOn ontology and the Systematised Nomenclature of Medicine Clinical Terms (SNOMEDCT). Out of the evaluated models, the BioBERT model achieves the best results, reaching a median macro-averaged F1 score of 0.90 for the NER task, 0.66 for the NEL task linking to the Hansard Taxonomy, 0.43 for the NEL task linking to the FoodOn ontology and 0.58 for the NEL task linking to the SNOMEDCT ontology.

Exploratory Landscape Analysis (ELA) enables the characterization of black-box optimization problem instances in the form of numerical features. Such features can be used to train a Machine Learning (ML) model to automatically predict the performance of an optimization algorithm on a specific problem instance. However, computing ELA features is a time consuming process and relatively expensive. In this paper, we aim to evaluate the usefulness of ELA features and identify features which are the most informative in automated algorithm performance prediction. The goal is to find a subset of features which are sufficient to train a reliable ML model for algorithm performance prediction, with reduced computational costs for calculating the ELA features. We focus on the performance prediction of the Covariance Matrix Adaptation Evolution Strat-egy (CMA-ES) algorithm on the COCO benchmark problems. The results showed that the number of ELA features that lead to a reliable algorithm performance prediction depends on the modular CMA-ES configuration under consideration. However, the set of features that are selected to be useful across different modular CMA-ES configurations are similar.

Fair algorithm evaluation is conditioned on the existence of high-quality benchmark datasets that are non-redundant and are representative of typical optimization scenarios. In this paper, we evaluate three heuristics for selecting diverse problem instances which should be involved in the comparison of optimization algorithms in order to ensure robust statistical algorithm performance analysis. The first approach employs clustering to identify similar groups of problem instances and subsequent sampling from each cluster to construct new benchmarks, while the other two approaches use graph algorithms for identifying dominating and maximal independent sets of nodes. We demonstrate the applicability of the proposed heuristics by performing a statistical performance analysis of five portfolios consisting of three optimization algorithms on five of the most commonly used optimization benchmarks.

In this paper, we have proposed a new pipeline for landscape analysis of time-series machine learning datasets that enables us to better understand a benchmarking problem landscape, allows us to select a diverse benchmark datasets portfolio, and reduces the presence of performance assessment bias via bootstrapping evaluation. Combining a large multi-domain representation corpus of time-series specific features and the results of a large empirical study of time-series classification (TSC) benchmark, we showcase the capability of the pipeline to point out issues with non-redundancy and representativeness in the benchmark. By observing discrepancy between the empirical results of the bootstrap evaluation and recently adopted practices in TSC literature when introducing novel methods, we warn on the potentially harmful effects of tuning the methods on certain parts of the landscape (unless this is an explicit and desired goal of the study). Finally, we propose a set of datasets uniformly distributed across the landscape space one should consider when benchmarking novel TSC methods.

To trace the skills of the employees and to link them to their responsibilities, organizations need support from methods that automatically support such kind of activities. Even more, the COVID-19 pandemic completely transformed organizations’ workplace and culture, so project managers should take care of the skills of their team members in order to successfully realize a project. For this purpose, we have introduced rule- based named-entity recognition methods that are able to extract soft and technical skills using textual data such as job postings, curricula vitae (CVs), shout-outs, performance reviews, etc. The best model using a fusion of several skills-related dictionaries provides an F1 score of 59%, which can be further used as a base for creating an annotated silver corpus consisting of textual data and all skills found there. Further, we demonstrated the application of the developed model to link the required skills for a certain job title. Being able to trace the skills in an automatic way, we can further link them to tasks and actions to fully understand their economic value.

In the modern era of big data, one of the biggest challenges is to find an efficient way of extracting information from unstructured data and structuring it in a form that can be interpreted and utilized by both humans and computers. In this paper, we focus on the domain of food and nutrition by introducing a Machine Learning (ML) based Named Entity Recognition (NER) method, which is a crucial step in extracting information from unstructured textual data. To the best of our knowledge, this is the first corpus-based food NER method that has been enabled by the recently published FoodBase corpus. The method is based on Bidirectional Long Short-Term Memory (BiLSTM) in conjunction with Conditional Random Fields (CRF) and Representation Learning (RL). Our experiments show that, despite the relatively small amount of annotated data, BuTTER is able to successfully identify food entities from raw text, with the best of the proposed models achieving an average macro F1 score of 0.946.