Optimization algorithms are traditionally represented by their descriptions, source code, and comparative results in benchmark environments. In recent years, however, it has become more popular to extract numerical features describing the behavior of algorithms during their operation. Such metrics can distinguish between algorithms, assess their novelty, or assist in selecting the best algorithm for a given problem. Unlike existing trajectory-based approaches that use partial trajectory information to derive hand-crafted behavioral features, we propose to represent complete execution trajectories as graphs and learn their embeddings using graph neural networks. In this paper, we explore two graph construction schemes and enrich graph vertices with evaluation and topological features. We compare the proposed approach with existing behavioral features and introduce additional experiments demonstrating the utility of behavioral analysis. Through extensive experiments, we show that the learned embeddings can effectively distinguish between algorithms even when evaluated across diverse benchmark functions. We introduce quantitative measures to assess how configuration influences behavioral expression and demonstrate that partial trajectory features can predict optimization success, enabling informed early stopping decisions. The resulting embedding space provides a foundation for constructing diverse algorithm portfolios, evaluating parameter significance, and enabling future analyses of relationships between problem characteristics and algorithm performance.

Decision trees are widely used in machine learning due to their simplicity and interpretability, but they often lack robustness to adversarial attacks and data perturbations. The paper proposes a novel island-based coevolutionary algorithm (ICoEvoRDF) for constructing robust decision tree ensembles. The algorithm operates on multiple islands, each containing populations of decision trees and adversarial perturbations. The populations on each island evolve independently, with periodic migration of top-performing decision trees between islands. This approach fosters diversity and enhances the exploration of the solution space, leading to more robust and accurate decision tree ensembles. ICoEvoRDF utilizes a popular game theory concept of mixed Nash equilibrium for ensemble weighting, which further leads to improvement in results. ICoEvoRDF is evaluated on 20 benchmark datasets, demonstrating its superior performance compared to state-of-the-art methods in optimizing both adversarial accuracy and minimax regret. The flexibility of ICoEvoRDF allows for the integration of decision trees from various existing methods, providing a unified framework for combining diverse solutions. Our approach offers a promising direction for developing robust and interpretable machine learning models.

This paper introduces the Augmented Decision Space Optimization (ADSO) method for sparsity-driven optimization of mixed-strategies in Stackelberg Security Games (SSGs). The proposed method enhances traditional strategy optimization by combining binary variables to represent the presence of pure strategies with real-valued variables to refine their selection probabilities. Specifically, instead of waiting for an evolutionary process to gradually discover sparse solutions, the binary variables in ADS allow the real-valued variables to be switched on or off, thereby directly enforcing sparsity. This dual codification scheme achieves targets such as sparsification and computational efficiency in large-scale games. We demonstrate that ADS outperforms existing heuristic methods, offering superior solution quality, scalability, and stability. Empirical results across three different benchmark games show that ADS generates compact strategies with minimal computational overhead, achieving performance close to the exact methods. Furthermore, state-of-the-art results are obtained for problems where exact methods fail to scale effectively. Our framework promises broad applicability beyond SSGs, encompassing a wide range of game-theoretic and combinatorial optimization problems.

The paper introduces a novel hybrid island-based framework in which diverse metaheuristics cooperate to effectively explore the search space. A core component of the framework is a diversity-driven migration mechanism, enabling adaptive management of the information flow between islands. Three fundamental aspects of migration - what to migrate, when to migrate, and where to migrate - are thoroughly analyzed, leading to the development of strategies that foster synergy between heterogeneous algorithms. These strategies balance exploration and exploitation, ensuring effective global and local search. The framework was evaluated on a set of diverse optimization benchmarks, both discrete (Traveling Salesman Problem instances) and continuous (BBOB functions). Experimental results demonstrate that the proposed approach surpasses traditional algorithms and their island-based variants in convergence speed, solution quality, and resilience to stagnation. Adaptive mechanisms dynamically adjust migration strategies during the optimization process, further enhancing the framework's effectiveness. The proposed method represents an advancement in hybrid metaheuristic systems, offering scalability and flexibility that are essential for solving complex optimization tasks.

Abstract visual reasoning (AVR) involves discovering shared concepts across images through analogy, akin to solving IQ test problems. Bongard Problems (BPs) remain a key challenge in AVR, requiring both visual reasoning and verbal description. We investigate whether multimodal large language models (MLLMs) can solve BPs by formulating a set of diverse MLLM-suited solution strategies and testing $4$ proprietary and $4$ open-access models on $3$ BP datasets featuring synthetic (classic BPs) and real-world (Bongard HOI and Bongard-OpenWorld) images. Despite some successes on real-world datasets, MLLMs struggle with synthetic BPs. To explore this gap, we introduce Bongard-RWR, a dataset representing synthetic BP concepts using real-world images. Our findings suggest that weak MLLM performance on classical BPs is not due to the domain specificity, but rather comes from their general AVR limitations. Code and dataset are available at: https://github.com/pavonism/bongard-rwr

We study generalization and knowledge reuse capabilities of deep neural networks in the domain of abstract visual reasoning (AVR), employing Raven's Progressive Matrices (RPMs), a recognized benchmark task for assessing AVR abilities. Two knowledge transfer scenarios referring to the I-RAVEN dataset are investigated. Firstly, inspired by generalization assessment capabilities of the PGM dataset and popularity of I-RAVEN, we introduce Attributeless-I-RAVEN (A-I-RAVEN), a benchmark with 10 generalization regimes that allow to systematically test generalization of abstract rules applied to held-out attributes at various levels of complexity (primary and extended regimes). In contrast to PGM, A-I-RAVEN features compositionality, a variety of figure configurations, and does not require substantial computational resources. Secondly, we construct I-RAVEN-Mesh, a dataset that enriches RPMs with a novel component structure comprising line-based patterns, facilitating assessment of progressive knowledge acquisition in transfer learning setting. We evaluate 13 strong models from the AVR literature on the introduced datasets, revealing their specific shortcomings in generalization and knowledge transfer.

The abstract visual reasoning (AVR) domain presents a diverse suite of analogy-based tasks devoted to studying model generalization. Recent years have brought dynamic progress in the field, particularly in i.i.d. scenarios, in which models are trained and evaluated on the same data distributions. Nevertheless, o.o.d. setups that assess model generalization to new test distributions remain challenging even for the most recent models. To advance generalization in AVR tasks, we present the Pathways of Normalized Group Convolution model (PoNG), a novel neural architecture that features group convolution, normalization, and a parallel design. We consider a wide set of AVR benchmarks, including Raven's Progressive Matrices and visual analogy problems with both synthetic and real-world images. The experiments demonstrate strong generalization capabilities of the proposed model, which in several settings outperforms the existing literature methods.

Parallel Algorithm Portfolio (PAP) is a promising approach to solving computationally hard problems efficiently. A portfolio is a set of problem solvers, each optimized for different problem types. However, the portfolio configuration process is computationally expensive, requiring numerous solver evaluations. This paper explores two methods to accelerate PAP construction. The first method leverages the vast amount of data generated during portfolio configuration to train a surrogate model, reducing the need for costly evaluations. The second method investigates training portfolios on smaller problem instances before applying them to more complex target instances. A comprehensive experimental study on the Traveling Salesman Problem instances demonstrates that both methods significantly reduce computational time, albeit at the cost of slightly reducing portfolio performance.

The optimization of complex problems remains a significant challenge across various domains of science and engineering. This paper introduces a novel approach to island-based optimization that dynamically adapts metaheuristic selection during runtime, extending the Diversity-driven Cooperating Portfolio of Metaheuristics (DdCPM) framework. Our method integrates additional metaheuristics beyond the original implementation and proposes adaptation strategies that dynamically reconfigure the algorithm portfolio based on performance indicators and population characteristics. Experimental results across both discrete and continuous optimization benchmarks demonstrate that adaptive metaheuristic selection enhances solution quality and convergence rates compared to static approaches. The proposed framework represents an advancement in hybrid optimization systems, offering improved performance through intelligent adaptation mechanisms that correspond to the evolving state of the search process.

Abstract Music has always been an essential aspect of human culture, and the methods for its creation and analysis have evolved alongside the advancement of computational capabilities. With the emergence of artificial intelligence (AI) and one of its major goals referring to mimicking human creativity, the interest in music-related research has increased significantly. This review examines current literature from renowned journals and top-tier conferences, published between 2017 and 2023, regarding the application of AI to music-related topics. The study proposes a division of AI-in-music research into three major categories: music classification, music generation and music recommendation. Each category is segmented into smaller thematic areas, with detailed analysis of their inter- and intra-similarities and differences. The second part of the study is devoted to the presentation of the AI methods employed, with specific attention given to deep neural networks—the prevailing approach in this domain, nowadays. In addition, real-life applications and copyright aspects of generated music are outlined. We believe that a detailed presentation of the field along with pointing out possible future challenges in the area will be of some value for both the established AI-in-music researchers, as well as the new scholars entering this fascinating field.

Game tree search algorithms play a vital role in application of computational and artificial intelligence methods in games and sequential combinatorial optimization problems. This paper addresses the need for robust and comprehensive benchmarking of these algorithms through the procedural generation of benchmarks. We introduce a novel game tree generator AutoGT built upon the idea of top-down propagation of various game parameters. Thanks to its extensive parameterization, AutoGT enables researchers to evaluate their methods on virtually unlimited number of unique benchmarks. To demonstrate the efficacy and usefulness of the generator, we present comparative results of several standard game-playing algorithms: a game theory optimal player (GTO), a player making moves according to a uniform random distribution, and a few variants of Monte Carlo Tree Search (MCTS) based players. Our findings indicate that for a given set of benchmark parameters, the results are repeatable with low deviation. However, when testing across various parameter settings, the performance becomes more diverse, allowing for a broader and more detailed assessment of the algorithms. The paper includes a link to the full code of AutoGT benchmark generation method.

The majority of the unsupervised feature selection methods usually explore the first-order similarity of the data while ignoring the high-order similarity of the instances, which makes it easy to construct a suboptimal similarity graph. Furthermore, such methods, often are not suitable for performing feature selection due to their high complexity, especially when the dimensionality of the data is high. To address the above issues, a novel method, termed as unsupervised feature selection for high-order embedding learning and sparse learning (UFSHS), is proposed to select useful features. More concretely, UFSHS first takes advantage of the high-order similarity of the original input to construct an optimal similarity graph that accurately reveals the essential geometric structure of high-dimensional data. Furthermore, it constructs a unified framework, integrating high-order embedding learning and sparse learning, to learn an appropriate projection matrix with row sparsity, which helps to select an optimal subset of features. Moreover, we design a novel alternative optimization method that provides different optimization strategies according to the relationship between the number of instances and the dimensionality, respectively, which significantly reduces the computational complexity of the model. Even more amazingly, the proposed optimization strategy is shown to be applicable to ridge regression, broad learning systems and fuzzy systems. Extensive experiments are conducted on nine public datasets to illustrate the superiority and efficiency of our UFSHS.

Abstract Visual Reasoning (AVR) comprises a wide selection of various problems similar to those used in human IQ tests. Recent years have brought dynamic progress in solving particular AVR tasks, however, in the contemporary literature AVR problems are largely dealt with in isolation, leading to highly specialized task-specific methods. With the aim of developing universal learning systems in the AVR domain, we propose the unified model for solving Single-Choice Abstract visual Reasoning tasks (SCAR), capable of solving various single-choice AVR tasks, without making any a priori assumptions about the task structure, in particular the number and location of panels. The proposed model relies on a novel Structure-Aware dynamic Layer (SAL), which adapts its weights to the structure of the considered AVR problem. Experiments conducted on Raven's Progressive Matrices, Visual Analogy Problems, and Odd One Out problems show that SCAR (SAL-based models, in general) effectively solves diverse AVR tasks, and its performance is on par with the state-of-the-art task-specific baselines. What is more, SCAR demonstrates effective knowledge reuse in multi-task and transfer learning settings. To our knowledge, this work is the first successful attempt to construct a general single-choice AVR solver relying on self-configurable architecture and unified solving method. With this work we aim to stimulate and foster progress on task-independent research paths in the AVR domain, with the long-term goal of development of a general AVR solver.

In recent years, there has been growing interest in developing robust machine learning (ML) models that can withstand adversarial attacks, including one of the most widely adopted, efficient, and interpretable ML algorithms—decision trees (DTs). This paper proposes a novel coevolutionary algorithm (CoEvoRDT) designed to create robust DTs capable of handling noisy high-dimensional data in adversarial contexts. Motivated by the limitations of traditional DT algorithms, we leverage adaptive coevolution to allow DTs to evolve and learn from interactions with perturbed input data. CoEvoRDT alternately evolves competing populations of DTs and perturbed features, enabling construction of DTs with desired properties. CoEvoRDT is easily adaptable to various target metrics, allowing the use of tailored robustness criteria such as minimax regret. Furthermore, CoEvoRDT has potential to improve the results of other state-of-the-art methods by incorporating their outcomes (DTs they produce) into the initial population and optimize them in the process of coevolution. Inspired by the game theory, CoEvoRDT utilizes mixed Nash equilibrium to enhance convergence. The method is tested on 20 popular datasets and shows superior performance compared to 4 state-of-the-art algorithms. It outperformed all competing methods on 13 datasets with adversarial accuracy metrics, and on all 20 considered datasets with minimax regret. Strong experimental results and flexibility in choosing the error measure make CoEvoRDT a promising approach for constructing robust DTs in real-world applications.

Games have long served as an effective method for developing and testing artificial intelligence (AI) models designed to function in a complex and possibly adversarial environment. While computers have already outperformed humans in practically all traditional board games, such as checkers, chess or Go, as well as various games with imperfect information, e.g. Poker or Bridge, certain imperfect information games, especially those devised for the purpose of testing particular capabilities of AI agents, remain a challenge. An example of such a game is Reconnaissance Blind Chess (RBC) - a variant of chess in which players do not have full access to the information defining the current board state. In this paper, we present Zubat, an AI RBC playing agent. The agent harnesses the strength of the Stockfish chess engine and enriches it with auxiliary modules. Firstly, we propose to estimate the oppnnent's uncertainty with a recurrent neural network in order to prefer positions that maximize the opponent's uncertainty, thus impeding their selection of optimal moves. Secondly, we develop the risk-taker module that identifies high-reward moves that could potentially exploit the opponent's uncertainty. The mediator module is constructed to select a move to be played by combining all three pieces of available information (the Stockfish assessment of moves, the expected opponent's uncertainty level, potential gains from risk-taking moves). Experiments conducted in the publicly available RBC match-making system show high competitiveness of the proposed agent, which, at the time of writing, was ranked 2nd out of 117 bots and human players, on the RBC leaderboard.

In the paper, we present the use of various models based on standard algorithms, ensemble methods, and neural networks for the fast prediction of the optical properties of nanostructured fibers. Such fibers are fabricated from several thousand elements, the spatial distribution of which determines the optical properties of the fiber. We show how to build a training set for a given class of nanostructured fibers and how different machine learning algorithms handle the estimation of a specific optical parameter. As a predicted parameter, we chose the zero dispersion wavelength, which is non-trivially dependent on the refractive index distribution in the fiber core. This approach allows for skipping time-consuming physical simulations and allows rapid verification of the properties of new fibers.

This article investigates the application of spiking neural networks (SNNs) to the problem of topic modeling (TM): the identification of significant groups of words that represent human-understandable topics in large sets of documents. Our research is based on the hypothesis that an SNN that implements the Hebbian learning paradigm is capable of becoming specialized in the detection of statistically significant word patterns in the presence of adequately tailored sequential input. To support this hypothesis, we propose a novel spiking topic model (STM) that transforms text into a sequence of spikes and uses that sequence to train single-layer SNNs. In STM, each SNN neuron represents one topic, and each of the neuron's weights corresponds to one word. STM synaptic connections are modified according to spike-timing-dependent plasticity; after training, the neurons' strongest weights are interpreted as the words that represent topics. We compare the performance of STM with four other TM methods Latent Dirichlet Allocation (LDA), Biterm Topic Model (BTM), Embedding Topic Model (ETM) and BERTopic on three datasets: 20Newsgroups, BBC news, and AG news. The results demonstrate that STM can discover high-quality topics and successfully compete with comparative classical methods. This sheds new light on the possibility of the adaptation of SNN models in unsupervised natural language processing.

For a long time, the ability to solve abstract reasoning tasks was considered one of the hallmarks of human intelligence. Recent advances in the application of deep learning (DL) methods led to surpassing human abstract reasoning performance, specifically in the most popular type of such problems-Raven's progressive matrices (RPMs). While the efficacy of DL systems is indeed impressive, the way they approach the RPMs is very different from that of humans. State-of-the-art systems solving RPMs rely on massive pattern-based training and sometimes on exploiting biases in the dataset, whereas humans concentrate on the identification of the rules/concepts underlying the RPM to be solved. Motivated by this cognitive difference, this work aims at combining DL with the human way of solving RPMs. Specifically, we cast the problem of solving RPMs into a multilabel classification framework where each RPM is viewed as a multilabel data point, with labels determined by the set of abstract rules underlying the RPM. For efficient training of the system, we present a generalization of the noise contrastive estimation algorithm to the case of multilabel samples and a new sparse rule encoding scheme for RPMs. The proposed approach is evaluated on the two most popular benchmark datasets [I-RAVEN and procedurally generated matrices (PGM)] and on both of them demonstrate an advantage over the state-of-the-art results.

Counterfactual Regret Minimization algorithms are the most popular way of estimating the Nash Equilibrium in imperfect-information zero-sum games. In particular, DeepStack -- the state-of-the-art Poker bot -- employs the so-called Deep Counterfactual Value Network (DCVN) to learn the Counterfactual Values (CFVs) associated with various states in the game. Each CFV is a multiplication of two factors: (1) the probability that the opponent would reach a given state in a game, which can be explicitly calculated from the input data, and (2) the expected value (EV) of a payoff in that state, which is a complex function of the input data, hard to calculate. In this paper, we propose a simple yet powerful modification to the CFVs estimation process, which consists in utilizing a deep neural network to estimate only the EV factor of CFV. This new target setting significantly simplifies the learning problem and leads to much more accurate CFVs estimation. A direct comparison, in terms of CFVs prediction losses, shows a significant prediction accuracy improvement of the proposed approach (DEVN) over the original DCVN formulation (relatively by 9.18-15.70% when using card abstraction, and by 3.37-8.39% without card abstraction, depending on a particular setting). Furthermore, the application of DEVN improves the theoretical lower bound of the error by 29.05-31.83% compared to the DCVN pipeline when card abstraction is applied. Additionally, DEVN is able to achieve the goal using significantly smaller, and faster to infer, networks. While the proposed modification may seem to be of a rather technical nature, it, in fact, presents a fundamentally different approach to the overall process of learning and estimating CFVs, since the distributions of the training signals differ significantly between DCVN and DEVN. The former estimates CFVs, which are biased by the probability of reaching a given game state, while training the latter relies on a direct EV estimation, regardless of the state probability. In effect, the learning signal of DEVN presents a better estimation of the true value of a given state, thus allowing more accurate CFVs estimation.

Genetic programming (GP), as a predictive data analytic tool, has difficulties dealing with high-dimensional problems. Therefore, some GP variants have been proposed for this type of problem, such as multi-stage GP (MSGP). Filter-based feature selection is commonly used in the literature for various machine learning purposes. However, its application for GP is overlooked due to GP's capability to operate as a wrapper-based feature selection while trying to find an optimal expression of the target variable via a functional combination of predictors. The effectiveness of wrapper- and filer-based feature selection approaches in machine learning has been the subject of a long-standing debate in the literature. This study aims to introduce an efficient feature selection approach and couple it with MSGP in order to handle high-dimensional problems. In addition, the stages of the GP are systematically ordered based on the variables' information. The proposed approach is tested against five real high-dimensional datasets. The results show that GP's inherent wrapper feature selection ability can be advanced further by using a filter-based feature selection approach to shrink the search space, which results in improving computational costs, expression complexity and the accuracy of MSGP.

One of the relevant aspects of Artificial General Intelligence is the ability of machines to demonstrate abstract reasoning skills, for instance, through solving (human) IQ tests. This work presents a new approach to machine IQ tests solving formulated as Raven’s Progressive Matrices (RPMs), called Duel-IQ. The proposed solution incorporates the concept of a tournament in which the best answer is chosen based on a set of duels between candidate RPM answers. The three relevant aspects are: (1) low computational and design complexity, (2) proposition of two schemes of pairing up candidate answers for the duels and (3) evaluation of the system on a dataset of shapes other than those used for training. Depending on a particular variant, the system reaches up to $82.8%$ accuracy on average in RPM tasks with 5 candidate answers and is on par with human performance and superior to other literature approaches of comparable complexity when training and test sets are from the same distribution.

Evolutionary algorithms have proven to be highly effective in continuous optimization, especially when numerous fitness function evaluations (FFEs) are possible. In certain cases, however, an expensive optimization approach (i.e. with relatively low number of FFEs) must be taken, and such a setting is considered in this work. The paper introduces an extension to the well-known LSHADE algorithm in the form of a pre-screening mechanism (psLSHADE). The proposed pre-screening relies on the three following components: a specific initial sampling procedure, an archive of samples, and a global linear meta-model of a fitness function that consists of 6 independent transformations of variables. The pre-screening mechanism preliminary assesses the trial vectors and designates the best one of them for further evaluation with the fitness function. The performance of psLSHADE is evaluated using the CEC2021 benchmark in an expensive scenario with an optimization budget of 102 - 104 FFEs per dimension. We compare psLSHADE with the baseline LSHADE method and the MadDE algorithm. The results indicate that with restricted optimization budgets psLSHADE visibly outperforms both competitive algorithms. In addition, the use of the pre-screening mechanism results in faster population convergence of psLSHADE compared to LSHADE.

Green Security Games have become a popular way to model scenarios involving the protection of natural resources, such as wildlife. Sensors (e.g. drones equipped with cameras) have also begun to play a role in these scenarios by providing real-time information. Incorporating both human and sensor defender resources strategically is the subject of recent work on Security Games with Signaling (SGS). However, current methods to solve SGS do not scale well in terms of time or memory. We therefore propose a novel approach to SGS, which, for the first time in this domain, employs an Evolutionary Computation paradigm: EASGS. EASGS effectively searches the huge SGS solution space via suitable solution encoding in a chromosome and a specially-designed set of operators. The operators include three types of mutations, each focusing on a particular aspect of the SGS solution, optimized crossover and a local coverage improvement scheme (a memetic aspect of EASGS). We also introduce a new set of benchmark games, based on dense or locally-dense graphs that reflect real-world SGS settings. In the majority of 342 test game instances, EASGS outperforms state-of-the-art methods, including a reinforcement learning method, in terms of time scalability, nearly constant memory utilization, and quality of the returned defender's strategies (expected payoffs).

Cross-modal representation learning allows to integrate information from different modalities into one representation. At the same time, research on generative models tends to focus on the visual domain with less emphasis on other domains, such as audio or text, potentially missing the benefits of shared representations. Studies successfully linking more than one modality in the generative setting are rare. In this context, we verify the possibility to train variational autoencoders (VAEs) to reconstruct image archetypes from audio data. Specifically, we consider VAEs in an adversarial training framework in order to ensure more variability in the generated data and find that there is a trade-off between the consistency and diversity of the generated images - this trade-off can be governed by scaling the reconstruction loss up or down, respectively. Our results further suggest that even in the case when the generated images are relatively inconsistent (diverse), features that are critical for proper image classification are preserved.

Spike-timing-dependent plasticity (STDP) is one of the most popular and deeply biologically motivated forms of unsupervised Hebbian-type learning. In this article, we propose a variant of STDP extended by an additional activation-dependent scale factor. The consequent learning rule is an efficient algorithm, which is simple to implement and applicable to spiking neural networks (SNNs). It is demonstrated that the proposed plasticity mechanism combined with competitive learning can serve as an effective mechanism for the unsupervised development of receptive fields (RFs). Furthermore, the relationship between synaptic scaling and lateral inhibition is explored in the context of the successful development of RFs. Specifically, we demonstrate that maintaining a high level of synaptic scaling followed by its rapid increase is crucial for the development of neuronal mechanisms of selectivity. The strength of the proposed solution is assessed in classification tasks performed on the Modified National Institute of Standards and Technology (MNIST) data set with an accuracy level of 94.65% (a single network) and 95.17% (a network committee)-comparable to the state-of-the-art results of single-layer SNN architectures trained in an unsupervised manner. Furthermore, the training process leads to sparse data representation and the developed RFs have the potential to serve as local feature detectors in multilayered spiking networks. We also prove theoretically that when applied to linear Poisson neurons, our rule conserves total synaptic strength, guaranteeing the convergence of the learning process.

In this article, Polar Bear Optimization Algorithm (PBO) is parallelized to solve the problem of computed tomography (CT) with incomplete data. It is vary hard to model correctly 2D and 3D objects by using CT scanners when information is incomplete. Our approach is to use PBO to reduce recovery time and simplify specificity of the phenomenon. Results from our research show that proposed approach is enabling fast and accurate reconstruction of objects modeled in projection space.

The huge popularity of heuristics contributes not only to the improvement and modeling of new solutions but also to their adaptation to selected goals. Recent years have shown the popularity of their use also in machine learning as a training algorithm or allowing for the selection of optimal architecture or hyper-parameters. In this paper, we propose an adaptation of a nature-inspired algorithm for preprocessing images in a parallel way for obtaining higher classification results. The proposed idea is based on analyzing images by heuristic representative which is Red Fox Optimization Algorithm and returning a specific value. These values are used in deciding to classify the entire image or trim it to eliminate unnecessary objects. We modeled this solution and evaluated using the learning transfer method for VOC 2007 dataset. The obtained results were compared on selected classes to show the advantages of a proposal.

Facial dysmorphology or malocclusion is frequently associated with abnormal growth of the face. The ability to predict facial growth (FG) direction would allow clinicians to prepare individualized therapy to increase the chance for successful treatment. Prediction of FG direction is a novel problem in the machine learning (ML) domain. In this paper, we perform feature selection and point the attribute that plays a central role in the abovementioned problem. Then we successfully apply data augmentation (DA) methods and improve the previously reported classification accuracy by 2.81%. Finally, we present the results of two experienced clinicians that were asked to solve a similar task to ours and show how tough is solving this problem for human experts.

The paper proposes a novel neuroevolutionary method (NESG) for calculating leader's payoff in Stackelberg Security Games. The heart of NESG is strategy evaluation neural network (SENN). SENN is able to effectively evaluate leader's strategies against an opponent who may potentially not behave in a perfectly rational way due to certain cognitive biases or limitations. SENN is trained on historical data and does not require any direct prior knowledge regarding the follower's target preferences, payoff distribution or bounded rationality model. NESG was tested on a set of 90 benchmark games inspired by real-world cybersecurity scenario known as deep packet inspections. Experimental results show an advantage of applying NESG over the existing state-of-the-art methods when playing against not perfectly rational opponents. The method provides high quality solutions with superior computation time scalability. Due to generic and knowledge-free construction of NESG, the method may be applied to various real-life security scenarios.

The paper presents a new method for approximating Strong Stackelberg Equilibrium in general-sum sequential games with imperfect information and perfect recall. The proposed approach is generic as it does not rely on any specific properties of a particular game model. The method is based on iterative interleaving of the two following phases: (1) guided Monte Carlo Tree Search sampling of the Follower's strategy space and (2) building the Leader's behavior strategy tree for which the sampled Follower's strategy is an optimal response. The above solution scheme is evaluated with respect to expected Leader's utility and time requirements on three sets of interception games with variable characteristics, played on graphs. A comparison with three state-of-the-art MILP/LP-based methods shows that in vast majority of test cases proposed simulation-based approach leads to optimal Leader's strategies, while excelling the competitive methods in terms of better time scalability and lower memory requirements.

This study proposes a novel biologically plausible mechanism for generating low-dimensional spike-based text representation. First, we demonstrate how to transform documents into series of spikes spike trains which are subsequently used as input in the training process of a spiking neural network (SNN). The network is composed of biologically plausible elements, and trained according to the unsupervised Hebbian learning rule, Spike-Timing-Dependent Plasticity (STDP). After training, the SNN can be used to generate low-dimensional spike-based text representation suitable for text/document classification. Empirical results demonstrate that the generated text representation may be effectively used in text classification leading to an accuracy of $80.19\%$ on the bydate version of the 20 newsgroups data set, which is a leading result amongst approaches that rely on low-dimensional text representations.

This paper introduces a new type of Security Games (SG) played on a plane with targets moving along predefined straight line trajectories and its respective Mixed Integer Linear Programming (MILP) formulation. Three approaches for solving the game are proposed and experimentally evaluated: application of an MILP solver to finding exact solutions for small-size games, MILP-based extension of recently published zero-sum SG approach to the case of generalsum games for finding approximate solutions of medium-size games, and the use of Memetic Algorithm (MA) for mediumsize and large-size game instances, which are beyond MILP’s scalability. Utilization of MA is, to the best of our knowledge, a new idea in the field of SG. The novelty of proposed solution lies specifically in efficient chromosome-based game encoding and dedicated local improvement heuristics. In vast majority of test cases with known equilibrium profiles, the method leads to optimal solutions with high stability and approximately linear time scalability. Another advantage is an iteration-based construction of the system, which makes the approach essentially an anytime method. This property is of paramount importance in case of restrictive time limits, which could hinder the possibility of calculating an exact solution. On a general note, we believe that MA-based methods may offer a viable alternative to MILP solvers for complex games that require application of approximate solving methods.

The paper proposes a neural model for a direct comparison of the two so-called Double Dummy Bridge Problem (DDBP) instances, along with a practical use-case for determining which pair, NS or WE, should propose the higher deal during a bidding phase in a Bridge game. The proposed system is composed of two identical subnetworks combined by a comparator layer placed on top of them. The base of each subnetwork is a shallow autoencoder (AE) which is further connected with a Multilayer Perceptron. The system is trained in two phases - an unsupervised one - used to create a meaningful feature-based input representation in AE compression layer, and a supervised one - meant for fine-tuning of the whole model. Training and test data are composed of pairs of Bridge deals in which the second deal in a pair is the first one rotated by 90 degrees. Since the task is to point which of the two deals promise a higher contract for the NS pair, due to deal rotation within a pair, the system effectively answers the title question "Who should bid higher, NS or WE, in a given deal?". The proposed approach is experimentally compared with two other methods: one relying on a neural system solving the DDBP and the other one employing several estimators of hand strength used by experienced players. The results clearly indicate that both neural network approaches outperform the usage of human-scoring systems by a large margin, most notably in the trump (suit) contract.

A new neural network method for Dimensionality Reduction (DR) of the input feature space in Multilabel Classification (MC) problems is proposed and experimentally evaluated in this paper. The method (abbreviated as TCART-MR) can be used in two possible scenarios: either as a stand-alone DR pre-processing phase, preceding subsequent application of any particular MC algorithm, or as a compact MC approach in which TCART-MR is applied twice - first to DR task and then to MC problem with reduced input space. Extensive experimental results proved statistically relevant advantage of TCART-MR over three state-of-the-art approaches in DR domain (in the context of MC), as well as its superiority over 10 state-of-the-art MC algorithms listed in a recent MC survey paper. The MC tests were performed on a set of 9 benchmark problems and 16 evaluation measures (leading to 144 experimental cases in total).

This paper presents a neural network approach to solving the most common type of human IQ test problems - Raven's Progressive Matrices (RMs). The proposed DeepIQ system is composed of three modules: a deep autoencoder which is trained to learn a feature-based representation of various figure images used in IQ tests, an ensemble of shallow multilayer perceptrons applied to detection of feature differences, and a scoring module use for assessment of candidate answers. DeepIQ is able to learn the underlying principles of solving RMs (the importance of similarity of figures in shape, rotation, size or shading) in a domain-independent way, that allows its subsequent application to test instances constructed based on a different set of figures, never seen before, or another type of IQ problem, with no requirement for additional training. This transfer learning property is of paramount importance due to scarce availability of the real data, and is demonstrated in the paper on two different RM data sets, as well as two distinct types of IQ tasks (solving RMs and odd-one-out problems). Experimental results are promising, excelling human average scores by a large margin on the most challenging subset of RM instances and exceeding 90% accuracy in odd-one-out tests.

In 2006 Zhang and Zhou proposed a multilabel classification model based on the MLP network, which was subsequently improved by Grodzicki et al. This paper further improves both these approaches by introducing a scaling parameter responsible for maintaining a balance between the impacts of particular components of the MLP's error function in the training process. The newly-proposed parameter is autonomously fine-tuned by the system in the nested cross validation process. The proposed approach is tested on a set of well-established benchmarks and demonstrates its superiority over the baseline methods for 16 different error measures used in the experiments. Furthermore, the method proves competitive to 12 other state-of-the-art machine learning approaches which are used for further comparisons. In the combined score composed of ranking positions for all benchmarks and all error functions, the proposed neural network system gains the leading position among all tested methods.

This issue marks the first anniversary issue since I was honored to serve as the Editor-in-Chief (EiC) of the IEEE Transactions on Neural Networks and Learning Systems (TNNLS). I am happy to report that we had a very successful year and here are a few highlights that I would like to share with the community.•The latest impact factor of TNNLS is 4.854 according to the Journal Citation Reports. This marks a record high impact factor for our journal and places TNNLS as the number one scholarly publication in Computer Science (Hardware & Architecture), number three in Computer Science (Theory & Methods), and number ten in Electrical and Electronic Engineering journals.

This paper proposes a neuro-genetic system for trading on Forex market. The main idea is to apply evolutionary methods for selection of the most suitable, in a given macro-economic context, set of variables, whose usefulness is subsequently validated based on a short and simplified training of several perceptron-type networks. Once the indicative training for this selected input data proves effective, the final, more sophisticated and more detailed training is performed on the ensemble of neural predictors. The proposed method was tested on 170 five-day investment periods (spanning over 3 years) with very promising results of both average and worst case performance. Furthermore, an investigation into the way the evolutionary component of the system selects input variables in subsequent trading periods has been performed, leading to interesting observation about the usefulness of particular data sources, as well as their repeatability across independent runs of the system.

In this paper a simulation-based approach to finding optimal defender strategy in multi-act Security Games (SG) played on a graph is proposed. The method employs the Upper Confidence Bounds applied to Trees (UCT) algorithm which relies on massive simulations of possible game scenarios. Three different variants of the algorithm are presented and compared with each other as well as against the Mixed Integer Linear Program (MILP) exact solution in terms of computational efficiency and memory requirements. Experimental evaluation shows that the method has a few times lower memory demands and is faster than MILP approach in majority of test cases while preserving quality of the resulting mixed strategies.

In this article, we present an efficient construction of the Game Description Language (GDL) interpreter. GDL is a first-order logic language used in the General Game Playing (GGP) framework. Syntactically, the language is a subset of Datalog and Prolog, and like those two, is based on facts and rules. Our aim was to achieve higher execution speed than anyone's of the currently available tools, including other Prolog interpreters applied to GDL. Speed is a crucial factor of the state space search methods used by most GGP agents, since the faster the GDL reasoner, the more game states can be evaluated in the allotted time. The cornerstone of our interpreter is the resolution tree which reflects the dependencies between rules. Our paradigm was to expedite any heavy workload to the preprocessing step to optimize the real-time usage. The proposed enhancements effectively maintain a balance between the time needed to build the internal data representation and the time required for data analysis during actual play. Therefore we refrain from using tree-based dictionary approaches such as TRIE to store the results of logical queries in favour of a memory-friendly linear representation and dynamic filters to reduce space complexity. Experimental results show that our interpreter outperforms the two most popular Prolog interpreters used by GGP programs: Yet Another Prolog (YAP) and ECLiPSe, respectively, in 22 and 26 games, out of the 28 tested. We give some insights into possible reasons for the edge of our approach over Prolog.

General game playing (GGP) aims at designing autonomous agents capable of playing any game within a certain genre, without human intervention. GGP agents accept the rules, which are written in the logic-based game definition language (GDL) and unknown to them beforehand, at runtime. The state-of-the-art players use Monte Carlo tree search (MCTS) together with the upper confidence bounds applied to trees (UCT) method. In this paper, we discuss several enhancements to GGP players geared towards more effective playing of single-player games within the MCTS/UCT framework. The main proposed improvements include introduction of a collection of lightweight policies which can be used for guiding the MCTS and a GGP-friendly way of using transposition tables. We have tested our base player and a specialized version of it for single-player games in a series of experiments using ten single-player games of various complexity. It is clear from the results that the optimized version of the player achieves significantly better performance. Furthermore, in the same set of tests against publicly available version of CadiaPlayer, one of the strongest GGP agents, the results are also favorable to the enhanced version of our player.

The term general game playing (GGP) refers to a subfield of AI which aims at developing agents able to effectively play many games from a particular class (finite, deterministic). It is also the name of the annual competition proposed by Stanford Logic Group at Stanford University (Stanford, CA, USA), which provides a framework for testing and evaluating GGP agents. In this paper, we present our GGP player which managed to win four out of seven games in the 2012 preliminary round and advanced to the final phase. Our system (named MINI-Player) relies on a pool of playing strategies and autonomously picks the ones which seem to be best suited to a given game. The chosen strategies are combined with one another and incorporated into the upper confidence bounds applied to trees (UCT) algorithm. The effectiveness of our player is evaluated on a set of games from the 2012 GGP Competition as well as a few other, single-player games. The paper discusses the efficacy of proposed playing strategies and evaluates the mechanism of their switching. The proposed idea of dynamically assigning search strategies during play is both novel and promising.

General game-playing (GGP) competitions provide a framework for building multigame-playing agents. In this paper, we describe an attempt at the implementation of such an agent. It relies heavily on our knowledge-free method of automatic construction of an approximate state evaluation function, based on game rules only. This function is then employed by one of the two game tree search methods: MTD (f) or guided upper confidence bounds applied to trees (GUCT), the latter being our proposal of an algorithm combining UCT with the usage of an evaluation function. The performance of our agent is very satisfactory when compared to a baseline UCT implementation.

This paper describes preliminary studies on automatic music composition. The aim is to investigate if it possible to generate music that would be difficult to distinguish from the music created by human composers. We describe a memetic algorithm used to achieve the goal of the studies, the influence of its parameters on the quality of composed pieces and conclusions stemmed from several tests.

We introduce a classification algorithm based on an idea of ‘multiple-resolution’ (or ‘multiscale’) approach to analysis of the data. In practice, the method uses an average of kernel density estimators where each estimator corresponds to a different data ‘resolution’. First, we examine theoretical properties of this method; next, we propose a practical implementation of such an algorithm with parameters of density estimators adjusted to minimise the misclassification probability. Subsequently, we test the algorithm on artificial data sets characterised by a ‘multiple-resolution’ property. The tests show that the introduced algorithm is superior to the basic version based on one estimator per class. We also test the algorithm on benchmark data sets and compare the results obtained with the results of other classification algorithms.

This book identifies the short-term and long-term challenges facing the field of computational intelligence (CI). After an introductory chapter, the next six chapters describe challenges resulting from attempts to model cognitive and neurocognitive processes. The next three chapters concern foundations of CI, while the last six chapters describe some theoretical problem and special subareas of CI. The book provides new ideas, helpful discussions of new directions of research, and non-trivial challenges for CI. It should be interesting for both established scientists and young researchers.

Artificial neural networks, trained only on sample deals, without presentation of any human knowledge or even rules of the game, are used to estimate the number of tricks to be taken by one pair of bridge players in the so-called double dummy bridge problem (DDBP). Four representations of a deal in the input layer were tested leading to significant differences in achieved results. In order to test networks' abilities to extract knowledge from sample deals, experiments with additional inputs representing estimators of hand's strength used by humans were also performed. The superior network trained solely on sample deals outperformed all other architectures, including those using explicit human knowledge of the game of bridge. Considering the suit contracts, this network, in a sample of 100,000 testing deals, output a perfect answer in 53.11% of the cases and only in 3.52% of them was mistaken by more than one trick. The respective figures for notrump contracts were equal to 37.80% and 16.36%. The above results were compared with the ones obtained by 24 professional human bridge players--members of The Polish Bridge Union--on test sets of sizes between 27 and 864 deals per player (depending on player's time availability). In case of suit contracts, the perfect answer was obtained in 53.06% of the testing deals for ten upper-classified players and in 48.66% of them, for the remaining 14 participants of the experiment. For the notrump contracts, the respective figures were equal to 73.68% and 60.78%. Except for checking the ability of neural networks in solving the DDBP, the other goal of this research was to analyze connection weights in trained networks in a quest for weights' patterns that are explainable by experienced human bridge players. Quite surprisingly, several such patterns were discovered (e.g., preference for groups of honors, drawing special attention to Aces, favoring cards from a trump suit, gradual importance of cards in one suit--from two to the Ace, etc.). Both the numerical figures and weight patterns are stable and repeatable in a sample of neural architectures (differing only by randomly chosen initial weights). In summary, the piece of research described in this paper provides a detailed comparison between various data representations of the DDBP solved by neural networks. On a more general note, this approach can be extended to a certain class of binary classification problems.

This paper presents an application of neural network interleaved training algorithm proposed in [1] in the domain of chess. In order to use the referenced learning method a structure of metric space is introduced in the space of chess moves. Neural network is used as a classifier of a distance from a given move to the optimal one, leading to significant limitation of the set of moves potentially worth to be considered. The method can be used as a supportive tool in effective initial move pre-ordering which is a preliminary step in majority of search-tree methods (e.g. the efficiency of alpha-beta pruning directly depends on the order, in which moves are considered). Proposed neural network-based classification approach can be used as a part of a hybrid AI game-tree search system.

In the last two decades the advancement of AI/CI methods in classical board and card games (such as Chess, Checkers, Othello, Go, Poker, Bridge, …) has been enormous. In nearly all “world famous” board games humans have been decisively conquered by machines (actually Go remains almost the last redoubt of human supremacy). In the above perspective the natural question is whether there is still any need for further development of CI methods in this area. What kind of goals can be achieved on this path? What are (if any) the challenging problems in the field? The paper tries to discuss these issues with respect to classical board mind games and provides (highly subjective) partial answers to some of the open questions. The main conclusion from the arguments specified in the paper is that one of the major, ultimate goals of CI in classical board game research concerns possessing by machines the ability to mimic human approach to game playing. This includes human-specific learning methods (learning from scratch, pattern-based learning, multitask and unsupervised learning) and human-type reasoning and decision making (efficient position estimation, abstraction and generalization of game features, autonomous development of evaluation functions, effective pre-ordering of moves, and selective, contextual search). Three of the above listed issues i.e. autonomous learning, knowledge discovery and intuition are discussed in this paper in more detail.

In this paper a new evolutionary-progressive method for multiple sequence alignment (MSA) is proposed. The method efficiently combines flexibility of evolutionary approach with speed and accuracy of progressive technique. Both stages of introduced hybrid method are described in detail. The results of comparison with several well-known methods show that proposed evolutionary-progressive method is an interesting alternative for purely genetic and purely progressive approaches

The paper presents results of experiments of estimating the number of tricks to be taken by one pair of bridge players in so-called Double Dummy Bridge Problem, using artificial neural networks. In addition to deals presented to neural network's inputs, also some human methods of estimating strength of a hand were applied. Influence of human knowledge on neural network's results depends on the way of coding a deal. One of deal representations tested in the paper definitely outperformed all the other choices even when the remaining representations were additionally supported by human knowledge implemented by several estimators frequently used in professional play. This superior representation output a perfect answer in 53.11% of test deals and only in 3.52% of them was mistaken by more than one trick

A neuro-evolutionary method for a short-term stock index prediction is presented. The data is gathered from the German stock exchange (the target market) and two other markets (Tokyo stock exchange and New York stock exchange) together with EUR/USD and USD/JPY exchange rates. Neural networks supported by genetic algorithm (GA) are used as the prediction engine. The GA is used to find suboptimal set of input variables for a one day prediction. Due to high volatility of mutual relations between input variables, a particular choice of input variables found by the GA is valid only for a short period of time and a new set of inputs is generated every 5 days. The method of selecting input variables works efficiently. Variables which are no longer useful are exchanged with the new ones. On the other hand some particularly useful variables are consequently utilized by the GA in subsequent independent steps. Simulation results of the proposed neuro-evolutionary system applied to prediction of the percentage change of closing value of DAX index are very promising and competitive to the ones obtained by the three other heuristical models implemented and tested for comparison.

Abstract:Two methods of genetic evolution of linear and non‐linear heuristic evaluation functions for the game of checkers and give‐away checkers are presented in the paper. The first method is based on the simplistic assumption that a relation ‘close’ to partial order can be defined over the set of evaluation functions. Hence an explicit fitness function is not necessary in this case and direct comparison between heuristics (a tournament) can be used instead. In the other approach a heuristic is developed step‐by‐step based on the set of training games. First, the end‐game positions are considered and then the method gradually moves ‘backwards’ in the game tree up to the starting position and at each step the best fitted specimen from the previous step (previous game tree depth) is used as the heuristic evaluation function in the alpha‐beta search for the current step. Experimental results confirm that both approaches lead to quite strong heuristics and give hope that a more sophisticated and more problem‐oriented evolutionary process might ultimately provide heuristics of quality comparable to those of commercial programs.

Incremental class learning (ICL) presents a possible solution to the catastrophic interference problem and provides a framework for the development of scalable learning systems. With respect to multi-class classification problems, the ICL approach can be summarized as follows. Initially the system focuses on one category. After it learns this category, it tries to identify a compact subset of features (nodes) in the hidden layers, that are crucial for the recognition of this category. The system then freezes these crucial nodes (features) by fixing their incoming weights. As a result, these features cannot be obliterated in subsequent learning. Moreover, these frozen features are available during subsequent learning and can be shared among a number of categories. Finally, as more categories are learned, the set of features gradually stabilizes and learning a new category requires less effort. We present results of applying the ICL approach to the handwritten digit recognition problem.