Generative Adversarial Networks (GANs) are difficult to train because of pathologies such as mode and discriminator collapse. Similar pathologies have been studied and addressed in competitive evolutionary computation by increased diversity. We study a system, Lipizzaner, that combines spatial coevolution with gradient-based learning to improve the robustness and scalability of GAN training. We study different features of Lipizzaner’s evolutionary computation methodology. Our ablation experiments determine that communication, selection, parameter optimization, and ensemble optimization each, as well as in combination, play critical roles. Lipizzaner succumbs less frequently to critical collapses and, as a side benefit, demonstrates improved performance. In addition, we show a GAN-training feature of Lipizzaner: the ability to train simultaneously with different loss functions in the gradient descent parameter learning framework of each GAN at each cell. We use an image generation problem to show that different loss function combinations result in models with better accuracy and more diversity in comparison to other existing evolutionary GAN models. Finally, Lipizzaner with multiple loss function options promotes the best model diversity while requiring a large grid size for adequate accuracy.

In this paper, we study the problem of constrained min-max optimization in a black-box setting, where the desired optimizer cannot access the gradients of the objective function but may query its values. We present a principled optimization framework, integrating a zeroth-order (ZO) gradient estimator with an alternating projected stochastic gradient descent-ascent method, where the former only requires a small number of function queries and the later needs just one-step descent/ascent update. We show that the proposed framework, referred to as ZO-Min-Max, has a sublinear convergence rate under mild conditions and scales gracefully with problem size. We also explore a promising connection between black-box min-max optimization and black-box evasion and poisoning attacks in adversarial machine learning (ML). Our empirical evaluations on these use cases demonstrate the effectiveness of our approach and its scalability to dimensions that prohibit using recent black-box solvers.

Digital signal processing (DSP) on field-programmable gate arrays (FPGAs) has long been appealing because of the inherent parallelism in these computations that can be easily exploited to accelerate such algorithms. FPGAs have evolved significantly to further enhance the mapping of these algorithms, included additional hard blocks, such as the DSP blocks found in modern FPGAs. Although these DSP blocks can offer more efficient mapping of DSP computations, they are primarily designed for 1-D filter structures. We present a study on spatial convolutional filter implementations on FPGAs, optimizing around the structure of the DSP blocks to offer high throughput while maintaining the coefficient flexibility that other published architectures usually sacrifice. We show that it is possible to implement large filters for large 4K resolution image frames at frame rates of 30-60 FPS, while maintaining functional flexibility.

With the celebrated success of deep learning, some attempts to develop effective methods for detecting malicious PowerShell programs employ neural nets in a traditional natural language processing setup while others employ convolutional neural nets to detect obfuscated malicious commands at a character level. While these representations may express salient PowerShell properties, our hypothesis is that tools from static program analysis will be more effective. We propose a hybrid approach combining traditional program analysis (in the form of abstract syntax trees) and deep learning. This poster presents preliminary results of a fundamental step in our approach: learning embeddings for nodes of PowerShell ASTs. We classify malicious scripts by family type and explore embedded program vector representations.

Random embedding has been applied with empirical success to large-scale black-box optimization problems with low effective dimensions. This paper proposes the EmbeddedHunter algorithm, which incorporates the technique in a hierarchical stochastic bandit setting, following the optimism in the face of uncertainty principle and breaking away from the multiple-run framework in which random embedding has been conventionally applied similar to stochastic black-box optimization solvers. Our proposition is motivated by the bounded mean variation in the objective value for a low-dimensional point projected randomly into the decision space of Lipschitz-continuous problems. In essence, the EmbeddedHunter algorithm expands optimistically a partitioning tree over a low-dimensional — equal to the effective dimension of the problem —search space based on a bounded number of random embeddings of sampled points from the low-dimensional space. In contrast to the probabilistic theoretical guarantees of multiple-run random-embedding algorithms, the finite-time analysis of the proposed algorithm presents a theoretical upper bound on the regret as a function of the algorithm's number of iterations. Furthermore, numerical experiments were conducted to validate its performance. The results show a clear performance gain over recently proposed random embedding methods for large-scale problems, provided the intrinsic dimensionality is low.

Decomposition-based evolutionary algorithms have been applied with success to multi-objective optimization problems where they are broken into several subproblems, and solutions for the original problem are recognized in a coordinated manner. Motivated by the working principle of decomposition-based methods, viz. the “divide-and-conquer” paradigm; this paper is concerned with solving black-box multi-objective problems given a finite number of function evaluations by taking inspiration from the single-objective deterministic sampling method, DIRECT. In particular, we provide a multi-objective algorithmic instance of DIRECT, which we refer to as MO-DIRECT and investigate its performance with respect to established decomposition-based multi-objective techniques. Besides its asymptotic convergence to the Pareto front and its inherent balance between exploration and exploitation of the decision space, the proposed framework is flexible enough to incorporate indicator-based techniques, albeit at the cost of a greater space complexity when compared to the evolutionary counterpart.

This paper provides an empirical analysis of the recently proposed Bayesian Multi-Scale Optimistic Optimization (BaMSOO) algorithm for solving bound-constrained black-box global optimization problems under expensive as well as cheap budgets of function evaluations. The CEC2016 and BBOB benchmarks are used in assessing the algorithm. We also compare BaMSOO with the Simultaneous Optimistic Optimization (SOO) algorithm from which BaMSOO is derived. The results indicate that BaMSOO has a comparable performance with SOO under expensive-budget settings but outperforms it with increasing evaluation budgets. Finally, we provide useful insights regarding the algorithm's relative efficiency and future directions.

Humans seek to select the best decision for a given problem in a process that is highly efficient and often ends with success. This is due to a high-order thinking skill: metacognition, which enables humans to be successful decision makers by constantly monitoring their cognitive activities based on earlier experience. Besides this, the social aspect of metacognition helps humans in monitoring their cognitive activities based on their peers experience and knowledge. Inspired by this, we propose HumanCog: a generic 3-layer architecture for solving optimization problems. HumanCog functions in a way that mimics human cognitive and metacognitive (self as well as social) behavior. The three layers in the network are cognitive layer, metacognitive layer and social cognitive layer. These three layers interact with each other such that accurate decision is made. As an initial work, we provide a simple realization of the HumanCog referred to as HumanCog-ver1, which self-regulates decision based on best experience. The performance evaluation on CEC 2015 and 2013 benchmark problems indicates promising results.

The popularity of GPU programming languages that explicitly express thread-level parallelism leads to the question of whether they can also be used for programming reconfigurable accelerators. This paper describes Guppy, a GPU-like softcore processor based on the in-order LEON3 core. Our long-term vision is to have a unified programming paradigm for accelerators - regardless of whether they are FPGA or GPU based. While others have explored this from a high level hardware synthesis perspective, we chose to adopt the approach of a parametrically reconfigurable softcore. We will discuss the main architecture features of Guppy, compare its performance to the original core. Our design has been synthesized on a Xilinx Virtex 5 FPGA.