AI-based models have been extensively investigated for integration into IoT botnet intrusion detection systems. However, their deployment in enterprise environments remains limited due to their inability to adapt to the non-stationary characteristics of IoT attack traffic, which are often influenced by persistent concept drift. These detection models are typically trained on stationary IoT attack data, leading to a substantial degradation in detection performance when concept drift occurs. One widely adopted strategy for addressing concept drift is incremental learning which involves retraining and updating the classifier whenever a drift occurs. Nonetheless, the rapid evolution of IoT attacks results in frequent concept drifts, which not only impose substantial retraining costs but also heighten the risk of catastrophic forgetting of previously learned information, caused by model shift arising from continuous weight updates. To address these limitations and support practical deployment, this study proposes VLSA-CL, a method that integrates variational latent space alignment with contrastive learning to achieve robust attack detection under continuous concept drift. In the proposed framework, latent representations of incoming traffic are aligned with previously learned latent spaces prior to classification. The alignment model employs contrastive learning to enforce class-wise coherence between new and historical representations, thereby enhancing traffic discrimination. Notably, the classifier is trained once on historical data, eliminating the need for continual retraining and reducing the risk of catastrophic forgetting. Comprehensive experiments conducted on three real-world IoT attack datasets validate the effectiveness of the proposed method, showing that it sustains strong detection performance under concept drift while substantially lowering false alarm rates and missed detections on unseen traffic.

The fast evolution of technologies like wireless sensor networks, cloud computing services, advanced AI driven applications and the Internet of Things (IoT) have led to increased reliance on internet by both individual users and enterprises—both small and large. On the contrary, the advancements in cybersecurity have not matched this pace consequently attracting exponentially rising trends of cyberattacks in the past decade. To enhance network security, this work proposes ResDNViT, a robust model integrating a self-attention-based Vision Transformer (ViT) architecture with a simplified ResNet-based architecture for NetFlow-based attack detection. Motivated by the strong performance of transformers in tasks related to NLP and computer vision, ResDNViT extends the ViT-based architecture for network traffic analysis by expressing NetFlow features as 2D matrices, and splitting them into equal-sized sub-matrices, that are used as input patches for the encoder component. A simplified residual dense network (ResDN) with two residual dense blocks (RDB) is stacked to the encoder’s output layer for classification. The novelty of this approach lies in effectively adapting the ViT-based architecture, originally designed for images, to analyzing NetFlow packets for attack classification. The model was evaluated on four well-studied benchmark datasets: the CICIDS2017_improved, Bot-IoT, CICIoT2022, and N-BaIoT, demonstrating an impressive performance across various classification tasks. The proposed approach’s ability to detect traffic from unseen device kinds was assessed by grouping devices from N-BaIoT into five categories based on usage: Thermostats, Baby Monitors, Doorbells, Security Cameras and Webcams. The model was trained using samples from four categories at a time and tested on samples from the remaining category. A high performance across metrics including accuracy, precision, recall, and F1-score for all categories highlighted the model’s robustness in traffic discrimination. • Combines a ViT-encoder and simplified ResNet-based architecture. • Proposed model removes conv layers, avoiding local spatial dependencies of ResNet. • Enhanced NetFlow attack detection over state-of-the-art detection methods. • Model adapts to unseen device types, removing need for retraining on new devices.

• Proposed an approach for IoT botnet attack detection under non-stationarity. • Deployed latent space alignment to mitigate frequent classifier retraining. • Demonstrated the impact of non-stationarity on detection performance. • Improved attack detection under non-stationary IoT traffic. • Prevents catastrophic forgetting by preserving historical information. Although earlier research has demonstrated that AI-driven models can attain exceptionally high accuracy in identifying IoT-based threats, their practical integration into enterprise environments for real-world attack detection and classification remains limited. This is primarily due to their reliance on stationary datasets for training and evaluation, which fail to capture the dynamic nature of real-world IoT NetFlow traffic, characterized by frequent concept drifts. A common solution is to retrain and update the classifier whenever a concept drift is detected. However, the rapid evolution of IoT attacks translates into frequent drifts, leading to high retraining costs and the risk of the model forgetting historical patterns, making it vulnerable to previously seen attack types. To address these challenges and facilitate real-world adoption, this study proposes a method for detecting novel attacks in non-stationary environments without requiring constant model retraining. The approach involves training a Variational Auto-encoder (VAE) on historical data, followed by training a classifier on the VAE’s latent space representations. Through latent space alignment, new instances are mapped to the established latent space learned from historical data, enabling the classifier to effectively detect attacks without retraining while preserving knowledge of past attack patterns. Our evaluation, using both synthetic and real-world IoT attack traffic, demonstrates the robustness of this approach, achieving improved detection performance under concept drift while significantly reducing false alarms and missed detections on unseen traffic.

Mission Planning for Multi-Uncrewed Ground Vehicle (multi-UGV) missions is a key functional module for achieving effective autonomy and coordination within a fleet of vehicles. However, the complexity of mission planning is compounded by the interconnected sub-problems involved and the challenging environments encountered by UGVs. Aiming to devise efficient and effective techniques to tackle the intricacies of mission planning in complex and cluttered environments, this paper presents an algorithmic architecture tailored for hierarchical multi-UGV mission planning systems. Specifically, this paper designs a Modified Cost Approximation Method integrated with two-layer environmental modelling for fast estimation of the travelling cost graphs of target points. A Hybrid Clustering Method that merges k-means clustering with a marginal cost-based assignment is proposed to streamline task decomposition and task assignment. Furthermore, a three-layer path planner is developed by integrating A*, post-processing steps, and Multi-operator Continuous Ant Colony optimisation, aiming to find paths with reduced cost for UGVs in challenging terrains. To evaluate the proposed techniques, a benchmark set for multi-UGV mission planning problems is designed using the robotic simulation platform CoppeliaSim. Simulation results demonstrate the superior performance of the proposed planning techniques.

We present a modified planning-assisted swarm shepherding method to effectively control multi-robot (sheepdogs) when herding a swarm of reactive agents (sheep) towards a goal and in environments with obstacles. Given a highly-dispersed sheep swarm, a mission planner based on Ant Colony Optimisation and A * is designed and developed to support the shepherding task. To apply the swarm shepherding method to real robots, a multi-layer environmental modelling method is proposed to construct customised environment maps for sheep and sheepdogs according to their physical characteristics. Then, a lookahead - based sub-goal selection method is presented for herding the sheep swarm to follow the A * optimised reference path. Furthermore, a circle-based method for selecting feasible driving/collecting points while avoiding obstacles is designed. Experiments are conducted in numerical simulation environments to compare the proposed method with the state-of-the-art planning-assisted shepherding method, followed by testing in the robot simulation platform CoppeliaSim to demonstrate the effectiveness of the proposed method.

Shepherding is a nature-inspired swarm guidance approach, where one or more sheepdogs act as actuators to guide a swarm towards a goal area. In the real-world, swarm guidance occurs in unknown environments. Context unfolds as the controller agent, the sheepdog, continues to discover new states, causing the state space to unfold during a mission due to the partial observability of the state space by each sheepdog. These individualised experiences could get shared among the shepherds to improve situation awareness. Our prior work introduced an approach to share interpretable knowledge between two agents. In this paper, we extend the two-agent interpretable knowledge fusion algorithm to multi-agent settings; allowing multiple sheepdogs to share their knowledge in an interpretable manner. When an agent receives knowledge from another agent, it decides on whether to integrate this new knowledge with what it already knows, leading to an increase in the size and space complexity of an agent's knowledge base. We propose a modular neural network society of mind architecture to store, update and manipulate the knowledge base of an agent. The architecture stores sub-networks and associate them with situations. When an agent is faced with a state, a gate controller decides based on the situation facing the agent which sub-networks are best suited to make decisions. The contribution is validated on a general classification task utilising the full-state-space for a swarm guidance shepherding problem. When compared to baseline methods, the proposed knowledge transfer algorithm improves generalisation, reduces catastrophic forgetting, and produces smaller models with faster adaptation.

This paper investigates a swarm shepherding problem which aims to herd multiple sub-swarm of robot agents (sheep) in a large-scale cluttered environment to a specific goal area using multiple distance-constrained robots (sheepdogs) located at different depots. We propose to formulate this challenging problem as a Multi-depot, Distance-constrained Close-Open Mixed Vehicle Routing Problem (MDCOMVRP). We also design a Two-phase Ant Colony Optimisation to address it by decomposing MDCOMVRP into a Multi-depot Open Vehicle Routing Problem (MOVRP) and a split problem. In the first phase, the Max-Min Ant System algorithm is employed to find open routes for all robots by transforming the MOVRP into a standard Travelling Salesman Problem using the proposed transformation method. In the second phase, a Modified Split algorithm is presented to construct a set of close or open distance-constrained routes, which are further optimised by the 2-opt local search method to generate the optimised sequence for each sheepdog robot to collect/drive sheep sub-swarms. Experiments are conducted to demonstrate that the proposed algorithm can solve MDCOMVRP successfully and assist the robots to complete the swarm shepherding mission efficiently.

Artificial Intelligence (AI) is the ubiquitous revolutionary technology of this century. AI has revolutionised humanity, including industry and government organisations, and transformed our world into smart digital spaces. As a discipline, one of the definitions of AI is the automation of cognition; or, put simply, the set of technologies required to support the design and implementation of artificial cognition in artificial systems. To design an intelligent system/machine, technologists need to transform the algorithms of AI into a system-of-systems design of cognition, whereby the artificial agent can sense, make sense, make decisions, take decisions, and learn about the contexts it is situated within. Modelling and Simulation (M&S) sit at the core of an artificial agent’s design and implementation components. In this presentation, I will cover the use of M&S within AI from different angles . In doing so, I will bring elements from my research to showcase how M&S not only contributes to AI but also shows that without M&S, AI can’t operate. I will paint futures that range from very concrete and narrowly defined uses of AI to a world of human and artificial cognitive agents. Humans educate AI agents, and AI agents educate humans. I will then conclude with some challenges for the M&S community to support the effort in advancing AI. This presentation will be drawn from many of my published works; below are some critical references for interested readers.

Monocular depth estimation is an important task that can be applied to many robotic applications. Existing methods focus on improving depth estimation accuracy via training increasingly deeper and wider networks, however these suffer from large computational complexity. Recent studies found that edge information are important cues for convolutional neural networks (CNNs) to estimate depth. Inspired by the above observations, we present a novel lightweight Edge Guided Depth Estimation Network (EGD-Net) in this study. In particular, we start out with a lightweight encoder-decoder architecture and embed an edge guidance branch which takes as input image gradients and multi-scale feature maps from the backbone to learn the edge attention features. In order to aggregate the context information and edge attention features, we design a transformer-based feature aggregation module (TRFA). TRFA captures the long-range dependencies between the context information and edge attention features through cross-attention mechanism. We perform extensive experiments on the NYU depth v2 dataset. Experimental results show that the proposed method runs about 96 fps on a Nvidia GTX 1080 GPU whilst achieving the state-of-the-art performance in terms of accuracy.

Many real-world classification problems have imbalanced frequency of class labels; a well-known issue known as the "class imbalance" problem. Classic classification algorithms tend to be biased towards the majority class, leaving the classifier vulnerable to misclassification of the minority class. While the literature is rich with methods to fix this problem, as the dimensionality of the problem increases, many of these methods do not scale-up and the cost of running them become prohibitive. In this paper, we present an end-to-end deep generative classifier. We propose a domain-constraint autoencoder to preserve the latent-space as prior for a generator, which is then used to play an adversarial game with two other deep networks, a discriminator and a classifier. Extensive experiments are carried out on three different multi-class imbalanced problems and a comparison with state-of-the-art methods. Experimental results confirmed the superiority of our method over popular algorithms in handling high-dimensional imbalanced classification problems. Our code is available on https://github.com/TanmDL/SLPPL-GAN

We propose a three-player spectral generative adversarial network (GAN) architecture to afford GAN the ability to manage minority classes under imbalanced conditions. A class-dependent mixture generator spectral GAN (MGSGAN) was developed to force generated samples to remain within the actual distribution of the data. MGSGAN was able to generate minority classes, even when the imbalanced ratio of majority to minority classes was high. A classifier based on lower features was adopted along with a sequential discriminator to develop a three-player GAN game. The generative networks performed data augmentation to improve the classifier ’ s performance. The proposed method was validated using two hyperspectral image data sets and compared with state-of-the-art methods in two class-imbalanced settings corresponding with real data distributions.

A single dataset could hide a significant number of relationships among its feature set. Learning these relationships simultaneously avoids the time complexity associated with running the learning algorithm for every possible relationship, and affords the learner with an ability to recover missing data and substitute erroneous ones by using available data. In our previous research, we introduced the gate-layer autoencoders (GLAEs), which offer an architecture that enables a single model to approximate multiple relationships simultaneously. GLAE controls what an autoencoder learns in a time series by switching on and off certain input gates, thus, allowing and disallowing the data to flow through the network to increase network's robustness. However, GLAE is limited to binary gates. In this article, we generalize the architecture to weighted gate layer autoencoders (WGLAE) through the addition of a weight layer to update the error according to which variables are more critical and to encourage the network to learn these variables. This new weight layer can also be used as an output gate and uses additional control parameters to afford the network with abilities to represent different models that can learn through gating the inputs. We compare the architecture against similar architectures in the literature and demonstrate that the proposed architecture produces more robust autoencoders with the ability to reconstruct both incomplete synthetic and real data with high accuracy.

As an essential component for many autonomous driving and robotic activities such as ego-motion estimation, obstacle avoidance and scene understanding, monocular depth estimation (MDE) has attracted great attention from the computer vision and robotics communities. Over the past decades, a large number of methods have been developed. To the best of our knowledge, however, there is not a comprehensive survey of MDE. This paper aims to bridge this gap by reviewing 197 relevant articles published between 1970 and 2021. In particular, we provide a comprehensive survey of MDE covering various methods, introduce the popular performance evaluation metrics and summarize publically available datasets. We also summarize available open-source implementations of some representative methods and compare their performances. Furthermore, we review the application of MDE in some important robotic tasks. Finally, we conclude this paper by presenting some promising directions for future research. This survey is expected to assist readers to navigate this research field.

Depth estimation from a single RGB image has attracted great interest in autonomous driving and robotics. State-of-the-art methods are usually designed on top of complex and extremely deep network architectures, which require more computational resources. Moreover, the inherent characteristic of the backbone used by the existing approaches results in severe spatial information loss in the produced feature maps, which impairs the accuracy of depth estimation on small sized images. In this study, we aimed to design a novel and efficient Convolutional Neural Network (CNN) to address these problems. Specifically, we stacked two shallow encoder-decoder style subnetworks successively in a unified network. Extensive experiments have been conducted on the NYU depth v2, KITTI, Make3D and Unreal data sets. Experimental results show that the proposed network achieves comparable accuracy to state-of-the-art methods that have extremely deep architectures but runs at a much faster speed on a single, less powerful GPU.

Individual differences in motivation can explain why people act differently in the same situation, and which aspects of a game people with different motive profiles may find most engaging. However, identifying a player’s motive profile from data available during gameplay remains an open research question. Besides a range of subjective and objective techniques for identifying player motivation, electroencephalography (EEG) technology could offer an automatic, objective technique for identifying the profile that best describes a given player. This article proposes a framework to measure player profiles of achievement, affiliation, and power motivation using EEG signals during their engagement within a game. First, an abstract mini-game is proposed to evaluate a player’s motivation. In the mini-game, each human player interacts with four nonplayer characters to gainfortuneorfriendshipthrough an individual play phase and a social network phase. The game is used within an experimental scenario to collect players’ actions and EEG signals. In addition, data from a psychological test are used to establish ground truth. We propose three subject labeling schemes using the output of the psychological test. Based on a player’s motive profile, behavioral indicators and EEG data analysis indicate that assessing a player’s motive profile is more robust from EEG signals than from behavioral data.

Within applications of swarm control, failing to maintain cohesion amongst the agents may lead to mission failure. We study the effect of limited sensing range of swarming agents when guided by a shepherd. A connectivity-aware approach is proposed to enhance the cohesion of the swarm. We combine a graph-based model of the flock with particle swarm optimization (assisted by DBSCAN) to improve the shepherd's performance in herding sheep (swarm members). The approach is evaluated using multiple initial swarm configurations. Simulation results on swarm sizes of 50 and 100 agents show up to 50% reduction in the task completion time and an average improvement of 25% in the success rate for low density sheep initialization scenarios and competitive results for the higher density scenarios.

Traditional oversampling methods are generally employed to handle class imbalance in datasets. This oversampling approach is independent of the classifier; thus, it does not offer an end-to-end solution. To overcome this, we propose a three-player adversarial game-based end-to-end method, where a domain-constraints mixture of generators, a discriminator, and a multi-class classifier are used. Rather than adversarial minority oversampling, we propose an adversarial oversampling (AO) and a data-space oversampling (DO) approach. In AO, the generator updates by fooling both the classifier and discriminator, however, in DO, it updates by favoring the classifier and fooling the discriminator. While updating the classifier, it considers both the real and synthetically generated samples in AO. But, in DO, it favors the real samples and fools the subset class-specific generated samples. To mitigate the biases of a classifier towards the majority class, minority samples are over-sampled at a fractional rate. Such implementation is shown to provide more robust classification boundaries. The effectiveness of our proposed method has been validated with high-dimensional, highly imbalanced and large-scale multi-class tabular datasets. The results as measured by average class specific accuracy (ACSA) clearly indicate that the proposed method provides better classification accuracy (improvement in the range of 0.7% to 49.27%) as compared to the baseline classifier

Controlling a large swarm of agents is a challenging task. Shepherding refers to an active field of research that seeks to address this challenge by using a control agent (sheepdog), which guides a swarm (sheep) towards a goal. Traditional shepherding involves switching between two main behaviours: driving the swarm towards the goal, and collecting stray sheep back to the flock. Evidently, the movement of the agents are dependent on their sensed information. Therefore, effectively controlling a swarm is even more challenging when sensor information or communication channels are unreliable. In this paper, we propose a shepherding methodology to achieve efficient swarm control in the presence of noise in the sensed information. The proposed approach consists of a new resting behaviour and a neural network-based reinforcement learning model. The neural network is used to learn shepherding policies using the new resting behaviour, where the objective is to optimise the frequency of sheep-to-dog interactions with varying levels of noise. The proposed approach is validated through simulations. Numerical experiments show that the proposed approach results in a more effective and stable performance compared to some conventional shepherding models from the literature.

Considerable progress has been made in improving the estimation accuracy of cognitive workload using various sensor technologies. However, the overall performance of different algorithms and methods remain suboptimal in real-world applications. Some studies in the literature demonstrate that a single modality is sufficient to estimate cognitive workload. These studies are limited to controlled settings, a scenario that is significantly different from the real world where data gets corrupted, interrupted, and delayed. In such situations, the use of multiple modalities is needed. Multimodal fusion approaches have been successful in other domains, such as wireless-sensor networks, in addressing single-sensor weaknesses and improving information quality/accuracy. These approaches are inherently more reliable when a data source is lost. In the cognitive workload literature, sensors, such as electroencephalography (EEG), electrocardiography (ECG), and eye tracking, have shown success in estimating the aspects of cognitive workload. Multimodal approaches that combine data from several sensors together can be more robust for real-time measurement of cognitive workload. In this article, we review the published studies related to multimodal data fusion to estimate the cognitive workload and synthesize their main findings. We identify the opportunities for designing better multimodal fusion systems for cognitive workload modeling.

Consensus achievement is a class of problems in which a group of agents, such as a swarm, needs to collectively reach a common decision to select one of the available options. Many consensus achievement strategies were proposed in which an agent forms its opinion and exchanges it with the other agents to reach a collective decision. To facilitate the decision making process, agents which are highly confident in their opinions are commonly given a higher chance to influence the collective's decision making. However, the use of subjective metrics for confidence could degrade the performance of the state-of-theart algorithms in complex scenarios where agents with wrong opinions can be the most confident. To tackle this problem, we propose an objective metric for confidence by using experience to learn the mapping between the information available to an agent and the probability that the agent's opinion is correct. To compute its confidence level, an agent feeds data from its local observations, as well as the received neighbours' opinions, into a fuzzy inference system (FIS) that uses these inputs to estimate confidence. The proposed strategy is distributed and it requires the agents to communicate locally using messages containing only their ID and opinions. Our strategy is evaluated under scenarios with different levels of complexity. The results show that our algorithm outperforms the state-of-the-art algorithms in terms of its accuracy, task time, and ability to reach majority. The proposed approach was also shown to maintain its success, even in the most complex environments.

Swarm guidance, such as the case of guiding a group of sheep away from a field, is a challenging task. As the swarm size increases, it becomes necessary that multiple control points, or sheepdogs, are needed to guide the swarm. In this paper, a swarm of unmanned aerial vehicles (UAVs) acts as a moving safety network (aka a formation) that not only guides the sheep swarm, but also prevents them from dispersing or reversing to the other side of the field. We investigate two types of formations. The first type acts as a baseline, maintains fixed distances from the sheep swarm, and relies on fixed predefined angular structure relative to the sheep's global centre of mass (GCM). The second type is dynamic, where the force vector to control the UAV and the individual distance of each UAV from the sheep's GCM are controlled by a Perceptron, with the weights optimized by a particle swarm optimization algorithm. We evolve five Perceptrons to specialize in relative positions in the formation, which fixes the space cost for the optimization algorithm, while allowing the size of the swarm of UAVs to scale up. We demonstrate that the use of Perceptron-networks for dynamic control scheme reduces the total distance travelled by the UAVs, is transparent when interpreted with Hinton diagrams, and transferable to a larger number of UAVs.

The design of reward functions in reinforcement learning is a human skill that comes with experience. Unfortunately, there is not any methodology in the literature that could guide a human to design the reward function or to allow a human to transfer the skills developed in designing reward functions to another human and in a systematic manner. In this paper, we use Systematic Instructional Design, an approach in human education, to engineer a machine education methodology to design reward functions for reinforcement learning. We demonstrate the methodology in designing a hierarchical genetic reinforcement learner that adopts a neural network representation to evolve a swarm controller for an agent shepherding a boids-based swarm. The results reveal that the methodology is able to guide the design of hierarchical reinforcement learners, with each model in the hierarchy learning incrementally through a multi-part reward function. The hierarchy acts as a decision fusion function that combines the individual behaviours and skills learnt by each instruction to create a smart shepherd to control the swarm.

In the shepherding problem, an external agent (the shepherd) attempts to influence the behavior of a swarm of agents (the sheep) by steering them towards a goal that is known to the shepherd but not the sheep. The problem offers a level of abstraction for Human-Swarm Interaction, where the human is able to shepherd the swarm towards a goal. Similarly, a smart robot could act as a shepherd to replace biological shepherds with ground or air vehicles. In both cases, it is important to preserve the energy of the shepherd by modulating the shepherd's influence vector on the sheep. Therefore, in this paper, we design a force modulation function for the shepherd agent to optimize the energy used by the agent and systematically study the effect of modulating the force of the influence vector on task success and energy used. The problem is further investigated using a bi-objective optimization formulation, where the energy used by the shepherd as well as the time of completion of the task are minimized, subject to a threshold of success rate. The findings demonstrate the coupling between contextual information used by the shepherd to modulate its influence vector and the effectiveness and efficiency of shepherd to complete the task.

Autoencoders (AE) have been used successfully as unsupervised learners for inferring latent information, learning hidden features and reducing the dimensionality of the data. In this paper, we propose a new AE architecture: Gate-Layer AE (GLAE). The novelty of GLAE lies in its ability to encourage learning of the relationships among different input variables, which affords it with an inherent ability to recover missing variables from the available ones and to act as a concurrent multi-function approximator.GLAE uses a network architecture that associates each input with a binary gate acting as a switch that turns on or off the flow to each input unit, while synchronising its action with data flow to the network. We test GLAE with different coding sizes and compare its performance against the Classic AE, Denoising AE and Variational AE. The evaluation uses Electroencephalograph (EEG) data with an aim to reconstruct the EEG signal when some data are missing. The results demonstrate GLAE's superior performance in reconstructing EEG signals with up to 25% missing data in an input stream.

Swarm control is a difficult problem due to the need to guide a large number of agents simultaneously. We cast the problem as a shepherding problem, similar to biological dogs guiding a group of sheep towards a goal. The shepherd needs to deal with complex and dynamic environments and make decisions in order to direct the swarm from one location to another. In this paper, we design a novel curriculum to teach an artificial intelligence empowered agent to shepherd in the presence of the large state space associated with the shepherding problem and in a transparent manner. The results show that a properly designed curriculum could indeed enhance the speed of learning and the complexity of learnt behaviours.

Human and Autonomous control of a swarm of robots is a non-trivial task compounded by the multi-agent nature of a swarm and the tight-coupling in swarm dynamics. Our previous work has been successful in designing controllers to shepherd a swarm of sheep by learning from human demonstrations. However, humans could get disengaged easily if the platform to collect demonstrations is uninteresting. In this paper, we propose a smartphone based Shepherding game as a platform to collect data on the behaviour of Shepherding models. The game is designed to allow both humans and artificial intelligence (AI) empowered shepherds to control the swarm of sheep. The game offers a plug-and-play ability for different human and AI controllers of the shepherd. The game logs all movements made by the human during controlling the shepherd for use by supervised learning algorithms to develop autonomous controllers. We present and evaluate different components of the game.

Highly secure access control requires Swiss-cheese-type multi-layer security protocols. The use of electroencephalogram (EEG) to provide cognitive indicators for human workload and fatigue has created environments where the EEG data are well-integrated into systems, making it readily available for more forms of innovative uses including biometrics. However, most of the existing studies on EEG biometrics rely on resting state signals or require specific and repetitive sensory stimulation, limiting their uses in naturalistic settings. Moreover, the limited discriminatory power of uni-variate measures denies an opportunity to use dependences information inherent in brain regions to design more robust biometric identifiers. In this paper, we proposed a novel model for ongoing EEG biometric identification using EEG collected during a diverse set of tasks. The novelty lies in representing EEG signals as a graph based on within-frequency and cross-frequency functional connectivity estimates, and the use of graph convolutional neural network (GCNN) to automatically capture deep intrinsic structural representations from the EEG graphs for person identification. An extensive investigation was carried out to assess the robustness of the method against diverse human states, including resting states under eye-open and eye-closed conditions and active states drawn during the performance of four different tasks. We compared our method with the state-of-the-art EEG features, classifiers, and models of EEG biometrics. Results show that the representation drawn from EEG functional connectivity graphs demonstrates more robust biometric traits than direct use of uni-variate features. Moreover, the GCNN can effectively and efficiently capture discriminative traits, thus generalizing better over diverse human states.

Entropy has been widely investigated as an effective metric to evaluate the dynamic complexity of signals. EEG is biological signals that contain rich complex dynamics. Transforming the information encoded in the rich dynamics embedded within EEG into appropriate biomarkers with discriminatory powers is important for event detection. It has broad prospects in a wide range of applications including medical diagnosis, therapy, and rehabilitation. This paper proposes a new entropy-based measure, Multi-scale Weighted Inherent Fuzzy Entropy (WIFEn), as an effective EEG biomarker for improving event detection performance. WIFEn first extracts Inherent Mode Functions (IMFs) using the Empirical Mode Decomposition method, then uses a weighted sum scheme to fuse the fuzzy entropy metrics calculated on each IMF. Finally, the multi-scale variation accounts for the multi-timescale dynamics inherent in EEG signals. Since EEG signals are a superposition of series of oscillations where information embedded in these oscillations is useful for estimating signal complexity, the aforementioned decomposition, and weighted sum procedures can improve the estimation results. The proposed method is tested with three entropy-based metrics for two tasks. The first task is eye-open and eye-closed detection with resting state EEG signals recorded from 10 subjects; while the second task is seizure detection for 8 epilepsy patients. The results indicate that the multi-scale WIFEn provides a better discriminatory power that improves detection performance than classic entropy-based measures, with an averaged improvement of 13.7% (p-value <; 0.05) for resting-state classification and 5.9% (p-value <; 0.05) for seizure detection.

Apprenticeship learning is a learning scheme based on the direct imitation of humans. Inverse reinforcement learning is used to learn a reward function from human data. Coupling Inverse reinforcement learning with reinforcement learning has demonstrated production of human-competitive policies. However, obtaining human subjects with the right level of skills for complex tasks can be a challenge. We propose a new learning scheme called Apprenticeship Bootstrapping to learn a composite task using human demonstrations on sub-tasks. The scenario is a ground-air interaction task with an Unmanned Aerial Vehicle that needs to maintain 3 autonomous Unmanned Ground Vehicles within range of an imaging sensor. For validation, we show that the bootstrapped policy performs as good as a policy learnt from a human performing the composite task. The method offers a clear advantage when skilled humans are available for simpler tasks that form the building blocks for a more complex task, where availability of experts is limited.

A formation is a geometric shape that a group of agents spatially organizes themselves into and maintains over time. Swarm Q-Learning (SQL) is a tabular multi-agent reinforcement learning algorithm designed to solve formation control problems. We modify SQL by allowing agents to exchange knowledge they have learnt within the same environment and introduce the Swarm Q-Learning with knowledge Sharing within an Environment (SQL-SIE). The algorithm is tested on a task where a swarm of robots, initially scattered in one side of the environment, needs to navigate through obstacles until they reach their initial positions in the formation within a region of interest. Experimental results show that the proposed SQL-SIE is more efficient than SQL as measured by the time taken by the swarm to complete this part of the mission. Moreover, SQL-SIE scales better than SQL as the number of agents increases.

EEG signals could reveal unique information of an individual's brain activities. They have been regarded as one of the most promising biometric signals for person identification and verification. Steady-State Visual Evoked Potentials (SSVEPs), as EEG responses to visual stimulations at specific frequencies, could provide biometric information. However, current methods on SSVEP biometrics with hand-crafted power spectrum features and canonical correlation analysis (CCA) present only a limited range of individual distinctions and suffer relatively low accuracy. In this paper, we propose convolution neural networks (CNNs) with raw SSVEPs for person identification and verification without the need for any hand-crafted features. We conduct a comprehensive comparison between the performance of CNN with raw signals and a number of classical methods on two SSVEP datasets consisting of four and ten subjects, respectively. The proposed method achieved an averaged identification accuracy of 96.8%±0.01, which outperformed the other methods by an average of 45.5% (p-value <; 0.05). In addition, it achieved an averaged False Acceptance Rate (FAR) of 1.53%±0.01 and True Acceptance Rate (TAR) of 97.09%±0.02 for person verification. The averaged verification accuracy is 98.34% ± 0.01, which outperformed the other methods by an average of 11.8% (p-value <; 0.05). The proposed method based on deep learning offers opportunities to design a general-purpose EEG-based biometric system without the need for complex pre-processing and feature extraction techniques, making it feasible for real-time embedded systems.

Robotic control in a continuous action space has long been a challenging topic. This is especially true when controlling robots to solve compound tasks, as both basic skills and compound skills need to be learned. In this paper, we propose a hierarchical deep reinforcement learning algorithm to learn basic skills and compound skills simultaneously. In the proposed algorithm, compound skills and basic skills are learned by two levels of hierarchy. In the first level of hierarchy, each basic skill is handled by its own actor, overseen by a shared basic critic. Then, in the second level of hierarchy, compound skills are learned by a meta critic by reusing basic skills. The proposed algorithm was evaluated on a Pioneer 3AT robot in three different navigation scenarios with fully observable tasks. The simulations were built in Gazebo 2 in a robot operating system Indigo environment. The results show that the proposed algorithm can learn both high performance basic skills and compound skills through the same learning process. The compound skills learned outperform those learned by a discrete action space deep reinforcement learning algorithm.

In this paper, we propose a deep reinforcement learning algorithm to learn multiple tasks concurrently. A new network architecture is proposed in the algorithm which reduces the number of parameters needed by more than 75% per task compared to typical single-task deep reinforcement learning algorithms. The proposed algorithm and network fuse images with sensor data and were tested with up to 12 movement-based control tasks on a simulated Pioneer 3AT robot equipped with a camera and range sensors. Results show that the proposed algorithm and network can learn skills that are as good as the skills learned by a comparable single-task learning algorithm. Results also show that learning performance is consistent even when the number of tasks and the number of constraints on the tasks increased.

A number of benchmark problems exist for evaluating multi-objective evolutionary algorithms (MOEAs) in the objective space. However, the decision space performance analysis is a recent and relatively less explored topic in evolutionary multi-objective optimization research. Among other implications, such analysis can lead to designing more realistic test problems, gaining better understanding about optimal and robust design areas, and design and evaluation of knowledge-based optimization algorithms. This paper complements the existing research in this area and proposes a new method to generate multi-objective optimization test problems with clustered Pareto sets in hyper-rectangular defined areas of decision space. The test problem is parametrized to control number of decision variables, number and position of optimal areas in the decision space and modality of fitness landscape. Three leading MOEAs, including NSGA-II, NSGA-III, and MOEA/D, are evaluated on a number of problem instances with varying characteristics. A new metric is proposed that measures the performance of algorithms in terms of their coverage of the optimal areas in the decision space. The empirical analysis presented in this research shows that the decision space performance may not necessarily be reflective of the objective space performance and that all algorithms are sensitive to population size parameter for the new test problems.

Growing trend of the dynamic multiobjective optimization research in the evolutionary computation community has increased the need for challenging and conceptually simple benchmark test suite to assess the optimization performance of an algorithm. This paper proposes a new dynamic multiobjective benchmark test suite which contains a number of component functions with clearly defined properties to assess the diversity maintenance and tracking ability of a dynamic multiobjective evolutionary algorithm (MOEA). Time-varying fitness landscape modality, tradeoff connectedness, and tradeoff degeneracy are considered as these properties rarely exist in the current benchmark test instances. Cross-problem comparative study is presented to analyze the sensitivity of a given algorithm to certain fitness landscape properties. To demonstrate the use of the proposed benchmark test suite, three evolutionary multiobjective algorithms, namely nondominated sorting genetic algorithm, decomposition-based MOEA, and recently proposed Kalman-filter-based prediction approach, are analyzed and compared. Besides, two problem-specific performance metrics are designed to assess the convergence and diversity performances, respectively. By applying the proposed test suite and performance metrics, microscopic performance details of these algorithms are uncovered to provide insightful guidance to the algorithm designer.

In this paper, we introduce a novel form of association rules (ARs) that do not require discretization of continuous variables or the use of intervals in either sides of the rule. This rule form captures nonlinear relationships among variables, and provides an alternative pattern representation for mining essential relations hidden in a given data set. We refer to the new rule form as a functional AR (FAR). A new neural network-based, co-operative, coevolutionary algorithm is presented for FAR mining. The algorithm is applied to both synthetic and real-world data sets, and its performance is analyzed. The experimental results show that the proposed mining algorithm is able to discover valid and essential underlying relations in the data. Comparison experiments are also carried out with the two state-of-the-art AR mining algorithms that can handle continuous variables to demonstrate the competitive performance of the proposed method.

Knowledge-based optimization is a recent direction in evolutionary optimization research which aims at understanding the optimization process, discovering relationships between decision variables and performance parameters, and using discovered knowledge to improve the optimization process, using machine learning techniques. A novel evolutionary optimization framework that incorporates a knowledge-based representation to search for Pareto optimal patterns in decision space was proposed earlier. This paper extends this framework to problems with four and more objectives, commonly referred to as many-objective optimization problems, using a hybridization approach with NSGA3. Experimental results on standard test functions are presented to demonstrate the advantages of the proposed hybrid algorithm in both objective and decision spaces.

Neural information recorded from electroencephalogram (EEG) provides new possibilities for diagnosis of brain abnormalities, cognitive monitoring, etc. However, many artifacts, such as eye blink and muscle movements, impact and contaminate EEG data. While traditional techniques proposed for artifact removal identified artifact on two-way data, (spatial x temporal), multidimensional nature of EEG data (spatial x temporal x spectral x condition x trial) is overlooked. In this work, we investigate the use of multiway analysis/tensor factorization on the extended EEG tensor (spatial x temporal x spectral), which is constructed from continuous wavelet transform, using Block Term Decomposition (BTD) of rank-(Lr, Lr, 1) for artifact removal. Eight different carefully designed experiments to study artifact typically produced by voluntarily, and sometimes involuntarily, behaviors using a subject were performed and analyzed. After the BTD decomposition, artifacted components are automatically identified removed using spatial and temporal features. The reconstructed signal from proposed method suppresses artifact while retains the signal texture of eight types of artifact investigated.

Human identity is a prerequisite for trust assurance and assessment, which is essential for effective human-machine interaction in trusted autonomous systems. Unlike conventional authentication methods which do not require users to re-authenticate themselves for sustained access, continuous authentication affirms human identity in real-time, therefore is a solution for continued access monitoring in trusted autonomous systems. Robust continuous authentication needs robust multi-modal data sources. In this paper, we design a multi-modal biometrics system that continuously verifies the presence of a logged-in user. Two types of biometric data are used, face images and Electroencephalography (EEG) signals. Information from individual modalities is fused at matching score level. For face modality, matching scores are calculated by distances between eigenface coefficients. While for EEG signals, an event-related potential (ERP) modality is established by a simple ERP elicitation protocol and calculation of cross-correlation similarities. Scores from the two modalities are normalized and fused using three schemes, namely the sum-score, max-score and min-score scheme. The experiments reveal that individual variations found in the ERPs are detectable and can be used for continuous authentication. This is an interesting finding which indicates that the ERP biometrics are feasible for user authentication and worthy of further research. Results also show that combining ERP biometric with face biometric using sum-score scheme outperforms each modality in isolation. This piece of finding indicates the potential of integrating ERP into multimodal authentication systems.

In this article, we present a new type of classification problem, which we call Comparative Classification Problem (CCP), where we use the term data record to refer to a block of instances. Given a single data record with n instances for n classes, the CCP problem is to map each instance to a unique class. This problem occurs in a wide range of applications where the independent and identically distributed assumption is broken down. The primary difference between CCP and classical classification is that in the latter, the assignment of a translator to one record is independent of the assignment of a translator to a different record. In CCP, however, the assignment of a translator to one record within a block excludes this translator from further assignments to any other record in that block. The interdependency in the data poses challenges for techniques relying on the independent and identically distributed (iid) assumption. In the Pairwise CCP (PWCCP), a pair of records is grouped together. The key difference between PWCCP and classical binary classification problems is that hidden patterns can only be unmasked by comparing the instances as pairs. In this article, we introduce a new algorithm, PWC4.5, which is based on C4.5, to manage PWCCP. We first show that a simple transformation—that we call Gradient-Based Transformation (GBT)—can fix the problem of iid in C4.5. We then evaluate PWC4.5 using two real-world corpora to distinguish between translators on Arabic-English and French-English translations. While the traditional C4.5 failed to distinguish between different translators, GBT demonstrated better performance. Meanwhile, PWC4.5 consistently provided the best results over C4.5 and GBT.

Trust is a fundamental concept that underpins the coherence and resilience of social systems and shapes human behavior. Despite the importance of trust as a social and psychological concept, the concept has not gained much attention from evolutionary game theorists. In this letter, an N-player trust-based social dilemma game is introduced. While the theory shows that a society with no untrustworthy individuals would yield maximum wealth to both the society as a whole and the individuals in the long run, evolutionary dynamics show this ideal situation is reached only in a special case when the initial population contains no untrustworthy individuals. When the initial population consists of even the slightest number of untrustworthy individuals, the society converges to zero trusters, with many untrustworthy individuals. The promotion of trust is an uneasy task, despite the fact that a combination of trusters and trustworthy trustees is the most rational and optimal social state. This letter presents the game and results of replicator dynamics in a hope that researchers in evolutionary games see opportunities in filling this critical gap in the literature.

Convergence performance and parametric sensitivity are two issues that tend to be neglected when extending differential evolution (DE) to multiobjective optimization (MO). To fill this research gap, we develop two novel mutation operators and a new parameter adaptation mechanism. A multiobjective DE variant is obtained through integration of the proposed strategies. The main innovation of this paper is the simultaneous use of individuals across generations from an objective-based perspective. Good convergence-diversity tradeoff and satisfactory exploration-exploitation balance are achieved via the hybrid cross-generation mutation operation. Furthermore, the cross-generation adaptation mechanism enables the individuals to self-adapt their associated parameters not only optimization stage-wise but also objective-space-wise. Empirical results indicate the statistical superiority of the proposed algorithm over several state-of-the-art evolutionary algorithms in handling MO problems.

Challenging multi-modal optimization problems have been very successfully solved by evolutionary computation (EC) techniques. To date, many methods have been proposed on evolutionary optimization for both single and multiobjective large scale problems. In the age of Big Data, there is an urge to take evolutionary optimization techniques to the next level for solving problems with even larger scales: thousands and millions of variables. These problems arise in many domains ranging from bioinformatics, to neuroscience and social simulations. In this paper, we investigate the use of EC to solve Big electroencephalography (EEG) data optimization problems with thousands of variables. The optimization problem attempts to identify maximum information that should be kept from a signal while minimizing the artifact. The high level of epistasis inherent in a signal can slow down the evolution. Therefore, we investigate the advantages of optimizing the problem in the frequency domain with different thresholds as opposed to the time domain. We propose synthetic EEG data sets of various scale and noise level. These data sets were the basis for the Optimization of Big Data 2015 Competition (BigOpt), CEC 2015. Two state-of-art multiobjective evolutionary algorithms (MOEAs) were evaluated. The results of this work suggest that frequency representation of the signals facilitates dimensionality reduction for big scale optimization of time series data, and hence provides faster and better quality solutions for EEG data cleaning. Moreover, the results suggest that existing state-of-art multiobjective evolutionary computation methods are extremely slow. Methods that can optimize the problem faster and with high quality are needed.

Knowledge extraction from a multi-objective optimization process has important implications including a better understanding of the optimization process and the relationship between decision variables. The extant approaches, in this respect, rely on processing the post-optimization Pareto sets for automatic rule discovery using statistical or machine learning methods. However such approaches fall short of providing any information during the progress of the optimization process, which can be critical for decision analysis especially if the problem is dynamic. In this paper, we present a multi-objective optimization framework that uses a knowledge-based representation to search for patterns of Pareto optimal design variables instead of conventional point form solution search. The framework facilitates the online discovery of knowledge during the optimization process in the form of interpretable rules. The core contributing idea of our research is that we apply multi-objective evolutionary process on a population of bounding hypervolumes, or rules, instead of evolving individual point-based solutions. The framework is generic in a sense that any existing multi-objective optimization algorithm can be adapted to evaluate the rule quality based on the sampled solutions from the bounded space. An instantiation of the framework using hyperrectangular representation and non-dominated sorting based rule evaluation is presented in this paper. Experimental results on a specifically designed test function as well as some standard test functions are presented to demonstrate the working and convergence properties of our algorithm.

In many engineering applications, black-box optimization relies on the use of a simulation to obtain a numeric evaluation or a score for a proposed solution. The cost of optimization is mostly a reflection of the cost of running this simulation environment. On the one hand, the higher the fidelity of the simulation environment, the longer it is likely to take to evaluate a single solution. Consequently, less solutions are allowed to be evaluated given a time constraint on the running time of the optimization algorithm. On the other hand, the lesser the fidelity of the simulation environment, the more likely more solutions could be evaluated within the same time constraint. However, understanding the relationship between fidelity and the quality of the final solution obtained by the optimization method is largely an unexplored area of research. In this paper, we present an approach for adjusting taskload of Air traffic controllers (ATC) in real time by using three different shadow simulators of increasing fidelity and Differential Evolution (DE) as the evolutionary optimization algorithm. According to air traffic conditions, DE optimizes a goal programming model to steer the taskload up or down towards a predefined taskload target by generating two ATC requests every 10 minutes. The results demonstrate how a high fidelity simulator can help DE to achieve better solution quality in the absence of any time constraint on running the experiments. However, when there is a tight time constraint imposed, lower fidelity simulators allow DE to explore more solutions in the search space by cutting down on the extra time needed when higher fidelity simulators are used.

Autonomous vehicles are frequently deployed in environments where only certain trajectories are feasible. Classical trajectory generation methods attempt to find a feasible trajectory that satisfies a set of constraints. In some cases the optimal trajectory may be known, but it is hidden from the autonomous vehicle. Under such circumstance the vehicle must discover a feasible trajectory. This paper describes a multi-agent system that uses a combination of reinforcement learning and differential evolution to generate a trajectory that is ε-close to a target trajectory that is hidden.

Air Traffic Control (ATC) is a complex safety critical environment. A tower controller would be making many decisions in real-time to sequence aircraft. While some optimization tools exist to help the controller in some airports, even in these situations, the real sequence of the aircraft adopted by the controller is significantly different from the one proposed by the optimization algorithm. This is due to the very dynamic nature of the environment. The objective of this paper is to test the hypothesis that one can learn from the sequence adopted by the controller some strategies that can act as heuristics in decision support tools for aircraft sequencing. This aim is tested in this paper by attempting to learn sequences generated from a well-known sequencing method that is being used in the real world. The approach relies on a genetic algorithm (GA) to learn these sequences using a society Probabilistic Finite-state Machines (PFSMs). Each PFSM learns a different sub-space; thus, decomposing the learning problem into a group of agents that need to work together to learn the overall problem. Three sequence metrics (Levenshtein, Hamming and Position distances) are compared as the fitness functions in GA. As the results suggest, it is possible to learn the behavior of the algorithm/heuristic that generated the original sequence from very limited information.

Enhanced situational awareness is integral to risk management and response evaluation. Dynamic systems that incorporate both hard and soft data sources allow for comprehensive situational frameworks which can supplement physical models with conceptual notions of risk. The processing of widely available semi-structured textual data sources can produce soft information that is readily consumable by such a framework. In this paper, we augment the situational awareness capabilities of a recently proposed risk management framework (RMF) with the incorporation of soft data. We illustrate the beneficial role of the hard-soft data fusion in the characterization and evaluation of potential vessels in distress within Maritime Domain Awareness (MDA) scenarios. Risk features pertaining to maritime vessels are defined a priori and then quantified in real time using both hard (e.g., Automatic Identification System, Douglas Sea Scale) as well as soft (e.g., historical records of worldwide maritime incidents) data sources. A risk-aware metric to quantify the effectiveness of the hard-soft fusion process is also proposed. Though illustrated with MDA scenarios, the proposed hard-soft fusion methodology within the RMF can be readily applied to other domains.

The concept of trust has attracted the attention of many researchers over the years who studied the impact of trust in many domains. Trust is a ubiquitous concept. It is pervasive in every aspect of our life, from interpersonal relationships to national defence and security applications. However, despite the vast literature on trust, we are not close enough to mastering the dynamics of trust. One reason is that if we define procedural steps for trust, we simultaneously define steps for deception; thus, we simply define a vacuous cycle. Another reason is that, the dynamics of trust change as the world changes. But how can we then study trust? This paper connects the interdisciplinary literature to synthesize a Computational Red Teaming (CRT) based model of trust that defines opportunities whereby computational intelligence techniques, more specifically, evolutionary game theory researchers, can contribute to this vastly growing research area. We offer a position on the topic by reviewing games for trust and introduce a new theoretic game to study influence and transfer of trust.

This paper presents an EEG-based interactive genetic algorithm framework, with the goal of leveraging EEG signals collected from a human expert involved in the evaluation of interactive genetic algorithm as inputs for genetic parameter control. We explain the framework of the system and our cognitive model constructed based on a 19 channel EEG system. An experiment has been performed to test the effectiveness of our framework and our cognitive model. Our work is the first attempt to combine brain-computer interaction with interactive evolutionary computation and parameter control.

Planning problems, such as mission capability planning in defense, can traditionally be modeled as a resource investment project scheduling problem (RIPSP) with unconstrained resources and cost. This formulation is too abstract in some real-world applications. In these applications, the durations of tasks depend on the allocated resources. In this paper, we first propose a new version of RIPSPs, namely extended RIPSPs (ERIPSPs), in which the durations of tasks are a function of allocated resources. Moreover, we introduce a resource proportion coefficient to manifest the contribution degree of various resources to activities. Since the more realistic nature of projects in practice implies that the circumstances under which the plan will be executed are stochastic in nature, we present a stochastic version of ERIPSPs, namely stochastic extended RIPSPs (SERIPSPs). To solve SERIPSPs, we first use scenarios to capture the space of possibilities (i.e., stochastic elements of the problem). We focus on three sources of uncertainty: duration perturbation, resource breakdown, and precedence alteration. We propose a robustness measure for the solutions of SEPIPSPs when uncertainties interact. We then formulate an SERIPSP as a multiobjective optimization model with three optimization objectives: makespan, cost, and robustness. A knowledge-based multiobjective evolutionary algorithm (K-MOEA) is proposed to solve the problem. The mechanism of K-MOEA is simple and time efficient. The algorithm has two main characteristics. The first is that useful information (knowledge) contained in the obtained approximated nondominated solutions is extracted during the evolutionary process. The second is that extracted knowledge is utilized by updating the population periodically to guide subsequent search. The approach is illustrated using a synthetic case study. Randomly generated benchmark instances are used to analyze the performance of the proposed K-MOEA. The experimental results illustrate the effectiveness of the proposed algorithm and its potential for solving SERIPSPs.

Many real-world situations require an agent with limited cognitive resources to be engaged in multiple games simultaneously. In these circumstances, the agent is unable to devote infinite cognitive resources to represent the optimal strategy that needs to be played against each opponent. We investigate this type of game, where a player, called the focal player, playing concurrently against multiple opponents independent 2-player games. Nevertheless, these games are coupled through the shared memory resource available to the agent. Each opponent has a fixed strategy. While the history length may vary from one opponent to another, the focal player doesn't know the opponents' strategies or which history length is used by which opponent. All the focal player can observe is the opponents' actions. We use evolutionary computation to decide on an appropriate allocation of memory to opponents. We show - given that opponents are using fixed strategies - that the memory distribution relies on the number of active bits (i.e., bits which generates a repeated pattern with a favorable payoff) that the focal player needs to exploit its opponents' strategies. We show how our current results relate to the minimum required number of bits for a player to face a set of opponents with fixed strategies, and how the number of active bits increases as the richness of the opponent strategy - measured using entropy - increases.

Bringing the 2012 IEEE World Congress on Computational Intelligence (IEEE-WCCI 2012) for the first time to Australia has been a fulfilling journey of joy and honour. This premier event of the IEEE Computational Intelligence Society (IEEE-CIS) brings together three flagship conferences of the society in even years. It consisted of these conferences: the International Joint Conference on Neural Networks (IJCNN 2012), the IEEE International Conference on Fuzzy Systems (FUZZIEEE 2012) and the 2012 IEEE Congress on Evolutionary Computation (IEEE CEC 2012). This document presents the technical papers from the IEEE CEC 2012 conference, which had 758 submissions, of which, 482 were accepted.

With the continued growth in Air Traffic, researchers are investigating innovative ways to increase airspace capacity while maintaining safety. A key safety indicator for an airspace is its Collision Risk estimate, which is compared against a Target Level of Safety (TLS) to provide a quantitative basis for judging the safety of operations in an airspace. However this quantitative value does not give an insight into the overall collision risk picture for an airspace, and how the risk changes given the interaction of a multitude of factors such as sector/traffic characteristics and controllers actions for flow management. In this paper, we propose an evolutionary framework with multi-objective optimization to evolve collision risk of air traffic scenarios. We attempt to identify, through evolutionary mechanism, the flight events resulting from Air Traffic Controller's actions that can lead to higher collision risks, thereby identifying the contributing factors or the events leading to collision risk. Computational experiments were conducted in an hi-fidelity air traffic simulation environment with collision risk model. Results indicate that “risk-free” traffic scenarios having collision risk below TLS can become “risk-prone” by few flight events, with Climb and Turn maneuvers, specifically during entering and exiting a sector, contributing significantly to increased collision risk.

This paper extends our previous work on evolving stories towards a computational platform for automatic scenario generation. In particular, we address a shortcoming of our earlier work relating to scenario representation: the regular story plot grammar. The use of this grammar, which only captures causal relationships between plot elements from the point of view of a single story character, resulted in the generation of stories that are always associated with one main character only, limiting the scalability of the approach. In addition, the regular grammar employed is not suitable for representing practical scenarios since a practical scenario may not require a character at all. To overcome these problems, we propose a new approach to scenario representation. Firstly, we introduce a set of scenario building blocks based on narrative theory. As a result, a scenario can be represented as a network of these building blocks that captures various relationships in the scenario. The task of generating scenarios is then transformed to the task of generating networks of these building blocks. Secondly, we develop two network representation languages extending Boers' GL-2 graph representation systems to describe networks with edges of more than one type and edges between two groups of nodes. Thirdly, a set of two context-free grammars is introduced to generate sentences, i.e. scenarios in these languages. Finally, we verify our approach to strategic scenario generation by employing an interactive evolution framework, which shows the proposed scenario representation scheme can facilitate the generation of coherent and novel story-like scenarios.

Dynamic Airspace Sectorization (DAS) is a future concept in Air Traffic Management. Its main goal is to increase airspace capacity by reshaping - thus optimizing - airspace sector boundaries based on the specifics of different air traffic situations, weather conditions and other factors. The primary objective for the optimization is to balance and reduce the workload of Air Traffic Controllers (ATCs). Many researchers have made efforts in this topic in the past years. However, air traffic changes continually, and DAS has to be adaptive to each change; be it in terms of aircraft density, dynamic routes, fleet mix, etc. Therefore, instead of sectorizing the airspace each time a change occurs, we should re-sectorize it by maintaining maximum similarities between each sectorization. In this paper, we propose a multi-objective evolutionary computation methodology to re-sectorize an airspace. We use a similarity measure between the existing sectorization and the re-sectorization as an objective to maximize during the evolution.We test the methodology with different air traffic conditions with four objective functions: minimize ATC task load standard deviation, maximize average flight sector time, maximize the minimum distance between traffic crossing points and sector boundaries, and maximize the similarity of two airspace sectorizations. Experimental results show that our re-sectorization method is able to perform airspace re-sectorization under different changes in the air traffic, while satisfying the predefined objectives.

Optimization problems under uncertainty have received considerable attention in recent years due to their practical implications. In real-world applications, a problem is usually confronted with multiple types of uncertainties that are incommensurable with each other. Decision makers in the real-world do not trade-off objectives alone, but also and more importantly trade-off different uncertainties. The contemporary optimization techniques that deal with uncertainty generally treat different types of uncertainties by aggregating them into a single form. In this paper, we introduce a new type of optimization problems which are characterized by multiple conflicting uncertainties. We term them as multi-uncertainty optimization problems. Modeling multiple conflicting uncertainties as an optimization problem can provide analysts a powerful tool to search non-dominated solutions in a risk space in addition to the objective space. This is particularly useful since sources of uncertainties are usually uncontrollable and cannot be optimized as objectives. The concept of a risk operating curve is introduced which provides a unique perspective of the problem to the decision makers allowing them to opt for solutions based on their risk attitude toward different sources of uncertainties. The application of these concepts is demonstrated through a test problem in the resource-constrained project scheduling domain.

One of the major challenges in the field of evolutionary algorithms (EAs) is to characterise which kinds of problems are easy and which are not. Researchers have been attracted to predict the behaviour of EAs in different domains. We introduce fitness landscape networks (FLNs) that are formed using operators satisfying specific conditions and define a new predictive measure that we call motif difficulty (MD) for comparison-based EAs. Because it is impractical to exhaustively search the whole network, we propose a sampling technique for calculating an approximate MD measure. Extensive experiments on binary search spaces are conducted to show both the advantages and limitations of MD. Multidimensional knapsack problems (MKPs) are also used to validate the performance of approximate MD on FLNs with different topologies. The effect of two representations, namely binary and permutation, on the difficulty of MKPs is analysed.

In multiobjective optimization problems (MOPs), the Pareto set consists of efficient solutions that represent the best trade-offs between the conflicting objectives. Many forms of uncertainty affect the MOP, including uncertainty in the decision variables, parameters or objectives. A source of uncertainty that is not studied in the evolutionary multiobjective optimization (EMO) literature is the decision-maker's attitude to risk (DMAR) even though it has great significance in real-world applications. Often the decision-makers change over the course of the decision-making process and thus, some relevant information about preferences of future decision-makers is unknown at the time a decision is made. This poses a major risk to organizations because a new decision-maker may simply reject a decision that has been made previously. When an EMO technique attempts to generate the set of nondominated solutions for a problem, then DMAR-related uncertainty needs to be reduced. Solutions generated by an EMO technique should be robust against perturbations caused by the DMAR. In this paper, we focus on the DMAR as a source of uncertainty and present two new types of robustness in MOP. In the first type, dominance robustness (DR), the robust Pareto solutions are those which, if perturbed, would have a high chance to move to another Pareto solution. In the second type, preference robustness (PR), the robust Pareto solutions are those that are close to each other in configuration space. Dominance robustness captures the ability of a solution to move along the Pareto optimal front under some perturbative variation in the decision space, while PR captures the ability of a solution to produce a smooth transition (in the decision variable space) to its neighbors (defined in the objective space). We propose methods to quantify these robustness concepts, modify existing EMO techniques to capture robustness against the DMAR, and present test problems to examine both DR and PR.

To solve two-dimensional (2D) rectangular packing problems, we introduce a new spatial method based on the discretization of the container into a grid of cells with predefined resolution. Before an item is added, grid cells are checked whether they can accommodate the item. If an appropriate empty cell cluster is found, the item is added and moved towards the bottom-left corner of the container. This placement and sliding method is supplemented by a heuristic that orders the items according to descending size. Order and rotation of items can be improved by hybridizing the heuristic with a genetic algorithm (GA) in which a population of order-rotation chromosomes is evolved. The method is tested on 47 benchmark problems and compared to other methods in the literature. This shows that it is fast and performs very well in finding close to optimal problem solutions. Particularly for large problem sizes, it outperforms some of the currently leading methods, such as heuristic recursive (HR). The hybridization with the GA meta-heuristic results in further performance improvements.

Many planning problems, such as mission capability planning, can be modelled as project scheduling problems. Unlike conventional deterministic project scheduling problems, project scheduling problems involve uncertainty and the execution of the plan is very likely to be perturbed by many factors. In other words, the circumstances under which the plan will be executed are changing and stochastic. In this paper, we first use scenarios to represent the stochastic elements in the problem; these are: perturbation strength and perturbation occurrence time. We define and explain the Stochastic Resource Investment Project Scheduling (SRIPS) problem. A multi-objective optimization model of SRIPS is proposed where three optimization objectives are considered simultaneously: makespan, cost, and robustness. A multi-objective genetic algorithm is employed to solve the problem. Finally, we generate two test problems with 30 and 60 non-dummy activities to validate the performance of the proposed approach and analyze the sensitivity of the results to different parameter settings.

Translator Stylometry is a small but growing area of research in computational linguistics. Despite the research proliferation on the wider research field of authorship attribution using computational linguistics techniques, the translator stylometry problem is more challenging and there is no sufficient literature on the topic. Some authors even claimed that this problem does not have a solution; a claim we will challenge in this paper. We present an innovative set of translator stylometric features that can be used as signatures to detect and identify translators. The features are based on the concept of network motifs: small graph local substructures which have been used successfully in characterizing global network dynamics. The text is transformed into a network, where words become nodes and their adjacencies in a sentence are represented through links. Motifs of size 3 are then extracted from this network and their distribution is used as a signature for the corresponding translator. We then investigate the impact of sample size, method of normalization and imbalance dataset on classification accuracy. We also adopt the Fuzzy Lattice Reasoning Classifier (FLR) among others, where FLR achieved the best performance with a classification accuracy reaching the 70% mark.

Predicting optimal size and mix of future transportation fleets is of great importance to defence logistics planners but faces some challenges. Firstly, the future is uncertain because the environment changes constantly and adversaries are highly adaptive. Secondly, optimising a large heterogeneous transport fleet is inherently complex. Heuristic-based optimisation techniques are therefore often applied that provide approximate solutions to support the decision making in such complex problems. However, heuristic-based methods act as black boxes and do not offer insights into the relationships between future scenarios and the solutions found. In this paper, we use an evolutionary rule-based approach to understand these relationships. A multi-objective Learning Classifier System (LCS) is employed to learn interpretable patterns of future scenarios and to associate them with the best performing heuristics under given conditions. To accomplish this, two novel reward functions are introduced that assign credits to classifiers based on the multi-objective performance of the predicted heuristics. Results show that LCS generalises well to relate scenario characteristics with Pareto optimal heuristics.

Understanding complex networks in the real world is a nontrivial task. In the study of community structures we normally encounter several examples of these networks, which makes any statistical inferencing a challenging endeavor. Researchers resort to computer-generated networks that resemble networks encountered in the real world as a means to generate many networks with different sizes, while maintaining the real-world characteristics of interest. The generation of networks that resemble the real world turns out in itself to be a complex search problem. We present a new rewiring algorithm for the generation of networks with unique characteristics that combine the scale-free effects and community structures encountered in the real world. The algorithm is inspired by social interactions in the real world, whereby people tend to connect locally while occasionally they connect globally. This local-global coupling turns out to be a powerful characteristics that is required for our proposed rewiring algorithm to generate networks with community structures, power law distributions both in degree and in community size, positive assortative mixing by degree, and the rich-club phenomenon.

Being able to learn the synchronization behavior of a networked complex system has profound implications for studying and modeling many natural and artificial phenomena, such as the spread of diseases, emergence of social trends, as well as more effective agent based distillation models. In order to study the practicality of learning synchronization behavior, we utilize the spatial iterated prisoner's dilemma game, which is played on a variety of complex network topologies. Players synchronize their interactions with other players, depending on the strategy they employ in the game. A genetic algorithm is used in order to attempt to learn the synchronization behavior of the players with respect to a target network. Our results indicate that it is impractical to learn the synchronization behavior on a network using only the strategy payoff information, and that more information is likely required to assist the learning process.

In this paper, we propose a computational framework for automated story-based scenario generation. Under this framework, a regular grammar is developed to model various causal relationships inside a given story world. The grammar is then evolved using evolutionary computation techniques to generate novel story plots, i.e. story-based scenarios. To evaluate these newly generated scenarios, a human-in-the-loop model is used. An experimental study was carried out, in which the proposed framework was used to create novel plots based on the famous Little Red Riding Hood fairy tale. The experimental study demonstrated that evolutionary computation can potentially contribute significantly to story generations. Some challenges were identified including the difficulty to quantify such subjective measures as plot interestingness and creativity.

What makes a combinatorial optimization problem hard? The concept of phase transition was introduced in combinatorial decision problems to explain that not all NP-Complete problems are hard, and that there exists a phase transition from solvable to unsolvable problems, within which hard problems exist. Phase transition has been studied using randomly generated problems in which variables have uniform distributions across the different constraints. Real-world problems demonstrate different distributions, however. This paper reveals the relationship between the difficulty of a 3-SAT problem and graph properties. It establishes for the first time a link between the theory of phase-transition in 3-SAT and phase transitions in complex systems and networks. This paper also addresses the question of whether the phase transition phenomenon exists for non-uniform randomly generated problems. A positive answer to this question means in principle that (1) we can generate test problems for combinatorial optimization that are not uniform; (2) we can generate test problems that resemble hard versions of real-world problems; (3) we can identify the features that we need to look for in a problem to test whether or not it is hard. We use a method that we call Adversarial Evolution (AE). In AE, an evolutionary computation method is used to generate hard problem instances by evolving solutions towards the failure of an algorithm and the phase transition region.

Both separated and overlapping communities are useful to analyze real networks in different situations. However, to the best of our knowledge, existing community detection methods based on Evolutionary Algorithms (EAs) can detect separate communities only. This is because it is difficult to represent overlapping communities in ways that are suitable for EAs. In this paper, we first design a representation method that can represent each individual as both separated and overlapping communities without assigning the number of communities in advance. We then design three objective functions to guide the evolutionary process in different conditions. Finally, based on the designed representation and objective functions, we propose a multiobjective evolutionary algorithm to solve CDPs (MEA_CDPs) under the framework of NSGA-II. In the experiments, 4 well-known real-life benchmark networks are used to validate the performance of MEA_CDPs, and the results shown that MEA_CDPs not only can find high quality communities, but also can detect both separated and overlapping communities at the same time, and present multiple types of communities. Moreover, the overlapping nodes identified by MEA_CDPs are really ambiguous according to their edge distributes in different communities. This illustrates the effectiveness of the objective functions we designed.

An efficient design of a Multi-Objective Learning Classifier System for multi-flight navigation is presented. A classifier is represented by a set of rules, which are used to simultaneously navigate all the flights in the airspace. Navigation of a flight is based on the relation of the flight with factors of the air traffic environment such as wind, storm as well as other flights. This system continually learns and refines the rules of classifiers by a multi-objective optimization algorithm - NSGAII - to discover the trade-off set of classifiers which navigate flights without any conflict, minimal distance of flying, minimal discomfort defined by storm level and the time duration of flights passing through storm areas, and minimizing total delay time of flights. We propose to detect conflicts between flights by grouping trajectory segments in 3-D (abscissa-x, ordinate-y, and time-t) boxes. The conflict detection is only implemented in a box, thus the number of conflict detection times approximates to the number of conflicts. Further, conflicts between flights are resolved using a hill climber by propagating delays in the takeoff time of conflicting flights. The advantage of the proposed system is that the classifier outputs its rules in a symbolic representation, making the overall process transparent to the user and reusable. Moreover, the system successfully discovered rules in all runs to optimize its performance.

Any significant decision making process involves dealing with many complexities including spatial interactions, temporal changes, interdependencies between system components and risk and uncertainties associated with decision choices. In this paper, we attempt to capture part of these complexities using an evolutionary game theoretic framework. At the heart of this framework is a game called InterDependent Security (IDS). In IDS games players opt between investing or not investing in securing themselves against possible losses in case of a negative risk event. Two models of IDS games are considered where both investors and non-investors share the risk of loss from a bad event. In the first model, the occurrence of a risk event is of a given probability and the players receive payoffs based on the expected loss. In the second model, events are modelled as stochastic processes and the game dynamics are studied in multi-period encounters. Game dynamics for both of these models are investigated over a range of cost to loss ratios under different levels of exogenous risk. The spatial effects are captured by placing the player population on a regular graph, and the evolutionary dynamics are modelled through a best-takeover update rule. There are significant differences between the dynamics of the two IDS models. The simulation results for deterministic model show three equilibrium regimes which conform to the conditions derived analytically. Stochastic games, on the other hand, show only two stable states for the most part of the parametric range. These states alternate cyclically throughout the iterations of the game.

An important aspect of airlift problems is to find the smallest fleet of aircraft to move cargo from one or more locations to a destination. In critical airlift operations, such as emergency evacuations, disaster relief and defence operations, a compromise needs to be struck between minimizing the time needed for completing all tasks and minimizing the size of the fleet. Usually, the time to complete a task is stochastic. A deterministic model, therefore, will under-estimate fleet size which results in increased levels of risk to achieve the overall airlift mission. In this paper, we introduce a stochastic version of the two-dimensional bin packing problem. We test a number of objective functions to measure different levels of risk. We then use an evolutionary multi-objective algorithm to solve a number of test problems. Analysis demonstrates that the different risk functions and level of variability/uncertainty in performing each task affect solutions non-linearly. Moreover, the multi-objective approach provides the analyst with an estimate of the range of risk; thus solutions can be selected based on criticality of meeting airlift demands.

Machine learning has become a prominent tool in various domains owing to its adaptability. However, this adaptability can be taken advantage of by an adversary to cause dysfunction of machine learning; a process known as Adversarial Learning. This paper investigates Adversarial Learning in the context of artificial neural networks. The aim is to test the hypothesis that an ensemble of neural networks trained on the same data manipulated by an adversary would be more robust than a single network. We investigate two attack types: targeted and random. We use Mahalanobis distance and covariance matrices to selected targeted attacks. The experiments use both artificial and UCI datasets. The results demonstrate that an ensemble of neural networks trained on attacked data are more robust against the attack than a single network. While many papers have demonstrated that an ensemble of neural networks is more robust against noise than a single network, the significance of the current work lies in the fact that targeted attacks are not white noise.

Evaluating conflict resolution algorithms in the air-traffic domain is a challenging task. These algorithms are usually tested using a pair of aircraft or a limited number of geometries involving multiple aircraft. Our previous work demonstrated the use of evolutionary computation for risk assessment of air-traffic conflict detection algorithms using a red-teaming (or playing the devil) approach. This paper extends our previous work to conflict resolution and investigate the effect of symmetry in the representation on the performance of the evolutionary operators.

This paper proposes a framework, that incorporates evolutionary computation and wargame simulation, to investigate the role of information networks in organizing efficient supply chains for military logistics. Under the proposed framework, evolutionary computation is used to evolve the information networks, which are subsequently evaluated by playing simulation wargames. Through a series of simulation studies, in which various supply scenarios have been simulated, we have found that information networks play a substantial role in efficient demand estimation. Depending on the level of information uncertainty, i.e. the hostile force strength distribution, different topological characteristics of the information networks, i.e. different information relationships between supply nodes, are favored. The objective of the paper is to discover the fundamental principles for information networks and their interaction with supply chains. These principles are significant for new and/or future military concepts such as network centric warfare. We believe that the proposal of the framework and the discovery of those emergent topological characteristics would significantly contribute to the organizing of efficient supply chains for military logistic operations.

Over the years, we have been applying multi- objective evolutionary algorithms (MOEAs) to a number of real- world problems, solving multi-objective optimization problems (MOPs) in the real world faces a number of challenges including when to terminate the algorithm. This paper addresses this challenge by introducing what we call a "stability measure". We use this measure to estimate when to stop the multi-objective evolutionary search. For the proposed measure, the non-dominated set obtained by an MOEA will be tested under local variability in the decision variable space. A non-dominated solution found by the MOEA will be assigned a stability value, which corresponds to the number of solutions within the neighborhood that dominate it. Obviously, if the found non-dominated solution lies on the Pareto optimal front (POF) then there cannot be any such dominating solutions in its neighborhood. The average of stability values assigned to all non-dominated solutions will be used as the stability value for the set. In order to validate the proposed measure, we carried out measurements on the obtained non-dominated sets from two MOEAs (NSGA-II and SPEA2), and two other well-known algorithms hill-climber, and a random-walk. We use random- walk in order to create a baseline for judging the performance of the algorithms, where we expect the highest level of variations to occur. To apply this measure at each generation, it incurs additional cost that does not contribute to the evolutionary search per se. This motivated us to add this measure as a local search operator. In this way, the local search operator plays two roles: (1) it attempts to find better solutions than the ones we have in the population; and (2) it acts as a stability measure for the evolutionary search. The results confirmed the usefulness of the proposed measure and algorithm.

Prioritisation is an important technique for resolving planning conflicts between agents with shared resources, such as robots moving through a shared space. This paper explores the use of genetic-based machine learning to assign priority dynamically, to improve performance of a team of agents without unduly impacting individual agents' performance. A decoupled heuristic approach is used for flexibility, whereby individual XCS agents learn to optimise their behaviour first, and then a high-level planner agent is introduced and trained to resolve conflicts by assigning priority. The approach is designed for Partially Observable Markov Decision Process (POMDP) environments and demonstrated on a problem in 3D aircraft path planning.

This paper investigates the use of a framework of local models in the context of noisy evolutionary multi-objective optimization. Within this framework, the search space is explicitly divided into several nonoverlapping hyperspheres. A direction of improvement, which is related to the average performance of the spheres, is used for moving solutions within each sphere. This helps the local models to filter noise and increase the robustness of the evolutionary algorithm in the presence of noise. A wide range of noisy problems we used for testing and the experimental results demonstrate the ability of local models to better filter noise in comparison with that of global models.

This paper focuses on the development of a spatial evolutionary multiagent social network for studying the macroscopic-behavioral dynamics of civil violence, as a result of microscopic game-theoretic interactions between goal-oriented agents. Agents are modeled from multidisciplinary perspectives and their strategies are evolved over time via collective coevolution and independent learning. Spatial and temporal simulation results reveal fascinating global emergence phenomena and interesting patterns of group movement and autonomous behavioral development. Extensions of differing complexity are also used to investigate the impact of various decision parameters on the outcome of unrest. Analysis of the results provides new insights into the intricate dynamics of civil upheavals and serves as an avenue to gain a more holistic understanding of the fundamental nature of civil violence.

This paper examines the comparative performance and adaptability of evolutionary, learning, and memetic strategies to different environment settings in the iterated prisoner's dilemma (IPD). A memetic adaptation framework is developed for IPD strategies to exploit the complementary features of evolution and learning. In the paradigm, learning serves as a form of directed search to guide evolving strategies to attain eventual convergence towards good strategy traits, while evolution helps to minimize disparity in performance among learning strategies. Furthermore, a double-loop incremental learning scheme (ILS) that incorporates a classification component, probabilistic update of strategies and a feedback learning mechanism is proposed and incorporated into the evolutionary process. A series of simulation results verify that the two techniques, when employed together, are able to complement each other's strengths and compensate for each other's weaknesses, leading to the formation of strategies that will adapt and thrive well in complex, dynamic environments.

Learning classifier systems (LCSs) are rule-based inductive learning systems that have been widely used in the field of supervised and reinforcement learning over the last few years. This paper employs sUpervised Classifier System (UCS), a supervised learning classifier system, that was introduced in 2003 for classification tasks in data mining. We present an adaptive framework of UCS on top of a self-organized map (SOM) neural network. The overall classification problem is decomposed adaptively and in real time by the SOM into subproblems, each of which is handled by a separate UCS. The framework is also tested with replacing UCS by a feedforward artificial neural network (ANN). Experiments on several synthetic and real data sets, including a very large real data set, show that the accuracy of classifications in the proposed distributed environment is as good or better than in the nondistributed environment, and execution is faster. In general, each UCS attached to a cell in the SOM has a much smaller population size than a single UCS working on the overall problem; since each data instance is exposed to a smaller population size than in the single population approach, the throughput of the overall system increases. The experiments show that the proposed framework can decompose a problem adaptively into subproblems, maintaining or improving accuracy and increasing speed.

This paper investigates the effect of elitism on multi-objective ant colony optimization algorithms (MACOs). We use a straightforward and systematic approach in this investigation with elitism implemented through the use of local, global, and mixed non-dominated solutions. Experimental work is conducted using a suite of multi-objective traveling salesman problems (mTSP), each with two objectives. The experimental results indicate that elitism is essential to the success of MACOs in solving multi-objective optimization problems. Further, global elitism is shown to play a particularly important role in refining the pheromone information for MACOs during the search process. Inspired by these results, we also propose an adaptation strategy to control the effect of elitism. With this strategy, the solutions most recently added to the global non-dominated archive are given a higher priority in defining the pheromone information. The obtained results on the tested mTSPs indicate improved performance in the elitist MACO when using the adaptive strategy compared to the original version.

Scenarios are pen-pictures of plausible futures, used for strategic planning. The aim of this investigation is to expand the horizon of scenario-based planning through computational models that are able to aid the analyst in the planning process. The investigation builds upon the advances of Information and Communication Technology (ICT) to create a novel, flexible and customizable computational capability-based planning methodology that is practical and theoretically sound. We will show how evolutionary computation, in particular evolutionary multi-objective optimization, can play a central role - both as an optimizer and as a source for innovation.

Capability planning problems are pervasive throughout many areas of human interest with prominent examples found in defense and security. Planning provides a unique context for optimization that has not been explored in great detail and involves a number of interesting challenges which are distinct from traditional optimization research. Planning problems demand solutions that can satisfy a number of competing objectives on multiple scales related to robustness, adaptiveness, risk, etc. The scenario method is a key approach for planning. Scenarios can be defined for long-term as well as short-term plans. This paper introduces computational scenario-based planning problems and proposes ways to accommodate strategic positioning within the tactical planning domain. We demonstrate the methodology in a resource planning problem that is solved with a multi-objective evolutionary algorithm. Our discussion and results highlight the fact that scenario-based planning is naturally framed within a multi-objective setting. However, the conflicting objectives occur on different system levels rather than within a single system alone. This paper also contends that planning problems are of vital interest in many human endeavors and that Evolutionary Computation may be well positioned for this problem domain.

In this paper, we introduce the air traffic operations and management simulator (ATOMS), which is an air traffic and airspace modeling and simulation system for the analysis of free-flight concepts. This paper describes the design, architecture, functionality, and applications of the ATOMS. It is an intent-based simulator that discretizes the airspace in equal-sized hyper-rectangular cells to maintain intent reference points. It can simulate end-to-end airspace operations and air navigation procedures for conventional air traffic, as well as for free flight. Atmospheric and wind data that are modeled in the ATOMS result in accurate trajectory predictions. The ATOMS uses a multiagent-based modeling paradigm for modular design and easy integration of various air traffic subsystems. A variety of advanced air traffic management (ATM) concepts that are envisioned in free flight are prototyped in the ATOMS, including airborne separation assurance (ASA), cockpit display of traffic information (CDTI), weather avoidance, and decision support systems (DSSs). Experimental results indicate that advanced ATM concepts make a sound case for free flight; however, there is a need to investigate and understand their complex interaction under nonnominal scenarios.

UCS is a supervised learning classifier system that was introduced in 2003 for classification in data mining tasks. The representation of a rule in UCS as a univariate classification rule is straightforward for a human to understand. However, the system may require a large number of rules to cover the input space. Artificial neural networks (NNs), on the other hand, normally provide a more compact representation. However, it is not a straightforward task to understand the network. In this paper, we propose a novel way to incorporate NNs into UCS. The approach offers a good compromise between compactness, expressiveness, and accuracy. By using a simple artificial NN as the classifier's action, we obtain a more compact population size, better generalization, and the same or better accuracy while maintaining a reasonable level of expressiveness. We also apply negative correlation learning (NCL) during the training of the resultant NN ensemble. NCL is shown to improve the generalization of the ensemble.

Many complex systems, whether biological, sociological, or physical ones, can be represented using networks. In these networks, a node represents an entity, and an arc represents a relationship/constraint between two entities. In discrete dynamics, one can construct a series of networks with each network representing a time snapshot of interaction among the different components in the system. Understanding these networks is a key to understand the dynamics of real and artificial systems. Network motifs are small graphs-usually three to four nodes-representing local structures. They have been widely used in studying complex systems and in characterizing features on the system level by analyzing locally how the substructures are formed. Frequencies of different network motifs have been shown in the literature to vary from one network to another, and conclusions hypothesized that these variations are due to the evolution/dynamics of the system. In this paper, we show for the first time that in strategy games, each game (i.e., type of dynamism) has its own signature of motifs and that this signature is maintained during the evolution of the game. We reveal that deterministic strategy games have unique footprints (motifs' count) that can be used to recognize and classify the game's type and that these footprints are consistent along the evolutionary path of the game. The findings of this paper have significance for a wide range of fields in cybernetics.

In most real-world problems, we either know little about the problems or the problems are too complex to have a clear vision on how to decompose them by hand. Thus, it is usually desirable to have a method to automatically decompose a complex problem into a set of subproblems and assign one or more specialists to each subproblem. Thecooperative coevolutionary mixture of experts(CCME) model was designed to automatically decompose problems by combining the global optimization power of cooperative coevolution with the divide-and-conquer ability of mixture of experts. This paper analyzes how CCME decomposes complex classification problems through a principal-component-analysis-based visualization tool. The visualization shows that CCME decomposes the problem by driving different experts toward different regions of the input space. The paper also investigates the effect of regularization, usinglearning by forgetting(LF), on CCME. LF significantly reduces the structural complexity of CCME while maintaining the classification accuracy.

Evolutionary games are used to model and understand some complex real world situations in economics, defence, and industry. However, different games are usually studied independently and in isolation of each other. Notwithstanding, in real world situations, an agent is involved in multiple games simultaneously and her action in one game influences her utility in the others. This situation is far more complex when the utility functions for the different games are in conflict with each other. In this paper, we propose for the first time an analysis for the interaction between the iterated prisoner's dilemma and the minority games. We empirically explore the relationship and clarify the mathematical conditions under which the Minority game won't change the dynamics of the iterated prisoner's dilemma game.

Evolutionary games are used in understanding the complex dynamics of real life situations. The iterated prisoner's dilemma game with exit option was used in modeling strategic alliances. The model was solved as an optimization problem, and several observations were considered as to how partners are behaving in the alliance, and what factors are affecting the alliance's cooperation level. In this paper we introduce an evolutionary version of the iterated prisoner's dilemma game with exit option to better understand alliance activity and investigate whether the observations from the mathematical solution continue to hold in the evolutionary environment. The results reveal that in some cases there is a significant difference in the evolutionary dynamics from that predicted by the mathematical solution. The mathematical model was found to be inadequate for alliances which use history information to make their future decisions.

It is well known that evolutionary algorithms often perform much better when augmented with a local search mechanism. While many local search methods exist for combinatorial optimization problems, there are relatively few methods designed to work over continuous fitness landscapes. This paper describes a novel continuous space local search algorithm for evolutionary algorithms that emulates army ant swarm raids. Our preliminary results show the method is remarkably effective.

Recently some algorithms have been proposed to clean post-training rule populations evolved by XCS, a state of the art Learning Classifier System (LCS). We present an algorithm to extract optimal rules, which we refer to as signatures, during the operation of UCS, a recent variant of XCS. In a benchmark binary valued dataset our method seconds the generalization and optimality hypotheses for UCS and provide mechanisms for retrieving all maximally general rules in real time. In real valued problems, where precise realization of decision boundaries is often not possible, our algorithm is able to retrieve near optimal representations with the help of a modified subsumption operator. The algorithm is able to reduce the processing time asymptotically and provides a mechanism for early stopping of the learning process.

In this paper, we present a novel efficient strategy to improve the performance of the differential evolution (DE) algorithm for real parameter optimization, by generating a variable step length based on a probability distribution, instead of using the conventional fixed step length approach. Previous studies investigated uniform and Gaussian distributions. In this study, we compare between these two distributions and a Cauchy distribution. The proposed strategy controls search parameters in a probabilistic manner. Experimental results are carried out on a wide range of fifteen standard test problems with different scenarios. The obtained results showed that the performance of the DE algorithm was best when using a Cauchy distribution (CD); thanks to its thick tails that enable it to generate considerable changes more frequently than other probability distributions and to escape a local optima for multimodal optimization problems.

Planning future force structures is usually associated with a high, but difficult to quantify, risk factor. Among many other reasons for the importance of this planning process, the defence industry requires the military establishment to communicate their decisions on the capabilities needed in the future. This communication enables the industry to shape their R&D programs and tailor their production plans. Overall, the decision maker needs to anticipate the state(s) of the environment across a relatively long time frame. The process of anticipation is surrounded with many risk factors. Moreover, transforming a force structure is not a single-step process. Intermediate force structures need also to lake into account threats in the medium future. This paper presents a temporal risk assessment methodology for planning future force structure. The methodology relics on constructing a topological structure of interfiled into force structures with transitions based on the proximity of these structures to each other and budget constraints. The path with minimal maximum risk is then identified using a dynamic programming min-max path finder algorithm. The methodology is demonstrated with two simple examples

Free flight is a new concept in air traffic management, where pilots are given more freedom in making decisions in the cockpit. This allows air traffic controllers to manage more flights. One of the concepts under investigation in the Australian airspace is moving sectors, where an air traffic controller becomes responsible of a moving volume of the space containing a group of airplanes. Planning flight trajectories in this group is a hard problem. In this paper, we show that XCS can be used as a reliable planning tool. We also propose a novel idea for incorporating hard constraints within XCS to increase its reliability

This paper focuses on the design and development of a spatial evolutionary multi-agent social network (EMAS) to investigate the underlying emergent macroscopic behavioral dynamics of civil violence, as a result of the microscopic local movement and game-theoretic interactions between multiple goal-oriented agents. Agents are modeled from multi-disciplinary perspectives and their behavioral strategies are evolved over time via collective co-evolution and independent learning. Experimental results reveal the onset of fascinating global emergent phenomenon as well as interesting patterns of group movement and behavioral development. Analysis of the results provides new insights into the intricate behavioral dynamics that arises in civil upheavals. Collectively, EMAS serves as a vehicle to facilitate the behavioral development of autonomous agents as well as a platform to verify the effectiveness of various violence management policies which is paramount to the mitigation of casualties.

Red teaming is the process of studying a problem by anticipating adversary behaviors. When done in simulations, the behavior space is divided into two groups; one controlled by the red team which represents the set of adversary behaviors or bad guys, while the other is controlled by the blue team which represents the set of defenders or good guys. Through red teaming, analysts can learn about the future by forward prediction of scenarios. More recently, defense has been looking at evolutionary computation methods in red teaming. The fitness function in these systems is highly stochastic, where a single configuration can result in multiple different outcomes. Operational, tactical and strategic decisions can be made based on the findings of the evolutionary method in use. Therefore, there is an urgent need for understanding the nature of these problems and the role of the stochastic fitness to gain insight into the possible performance of different methods. This paper presents a first attempt at characterizing the search space difficulties in red teaming to shed light on the expected performance of the evolutionary method in stochastic environments.

This paper investigates the use of evolutionary multi-objective optimization methods (EMOs) for solving single-objective optimization problems in dynamic environments. A number of authors proposed the use of EMOs for maintaining diversity in a single objective optimization task, where they transform the single objective optimization problem into a multi-objective optimization problem by adding an artificial objective function. We extend this work by looking at the dynamic single objective task and examine a number of different possibilities for the artificial objective function. We adopt the non-dominated sorting genetic algorithm version 2 (NSGA2). The results show that the resultant formulations are promising and competitive to other methods for handling dynamic environments.

XCS is widely accepted as one of the most reliable Michigan-style learning classifier system for data mining. Many studies found that XCS is able to provide good generalization using a ternary representation for binary inputs as well as interval representation for continuous-valued inputs. Since distributed data mining is becoming more popular due to massive data sets spread across a network at many organizations, we have proposed an XCS system for distributed data mining called DXCS. DXCS has been tested on binary inputs. The results showed that DXCS does not only achieve as good performance as the centralized XCS system, but also reduces data transmission in the network. In this paper, we further examine DXCS with real-valued inputs in stationary and dynamic environments

This paper compares the performance of anti-noise methods, particularly probabilistic and re-sampling methods, using NSGA2. It then proposes a computationally less expensive approach to counteracting noise using re-sampling and fitness inheritance. Six problems with different difficulties are used to test the methods. The results indicate that the probabilistic approach has better convergence to the Pareto optimal front, but it looses diversity quickly. However, methods based on re-sampling are more robust against noise but they are computationally very expensive to use. The proposed fitness inheritance approach is very competitive to re-sampling methods with much lower computational cost.

Evolutionary algorithms (EAs) are widely used to deal with optimization problems in dynamic environments (DE) [3]. When using EAs to solve DE problems, we are usually interested in the algorithm's ability to adapt and recover from the changes. One of the main problems facing an evolutionary method when solving DE problems is the loss of genetic diversity.In this paper, we investigate the use of evolutionary multi-objective optimization methods (EMOs) for single-objective DE problems. For that purpose, we introduce an artificial second objective with the aim to maintain useful diversity in the population. Six different artificial objectives are examined and compared.All the results will be compared against a traditional GA and the random immigrants algorithm[4]. NSGA2 is employed as the evolutionary multi-objective technique.

XCS is a flexible system for data mining due to its ability to deal with environmental changes, learn online with little prior knowledge and evolve accurate and maximally general classifiers. In this paper, we propose DXCS which is an XCS-based distributed data mining system. A MDL metric is proposed to quantify and analyze network load, and study the balance between network load and classifier accuracy in the presence of noise. The DXCS system shows promising results.

In this paper we investigate the emergence of communication in competitive multi-agent systems. A competitive environment is created with two teams of agents competing in an exploration task; the quickest team to explore the largest area wins. One team uses indirect communication and is controlled by an artificial neural network evolved using a Pareto multi-objective approach. The second team uses direct communication and a fixed strategy for exploration. A comparison is made between agents with and without communication. Results show that as the fitness function vary differing exploration strategies emerge. Experiments with communication produced cooperative strategies; while the experiments without communication produced effective strategies but with individuals acting independently.

We propose a sub-structural niching method that fully exploits the problem decomposition capability of linkage-learning methods such as the estimation distribution algorithms and concentrate on maintaining diversity at the sub-structural level. The proposed method consists of three key components: (1) Problem decomposition and sub-structure identification, (2) sub-structure fitness estimation, and (3) sub-structural niche preservation. The sub-structural niching method is compared to restricted tournament selection (RTS)---a niching method used in hierarchical Bayesian optimization algorithm---with special emphasis on sustained preservation of multiple global solutions of a class of boundedly-difficult, additively-separable multimodal problems. The results show that sub-structural niching successfully maintains multiple global optima over large number of generations and does so with significantly less population than RTS. Additionally, the market share of each of the niche is much closer to the expected level in sub-structural niching when compared to RTS.

For many years, Australian researchers have been contributing to the areas of artificial life and complex adaptive systems. This report highlights some of the Australian-based activities in these areas.

We propose a novel perspective on the use of evolutionary multiobjective optimization (EMO) as a paradigm for evolving embodied organisms and as a framework for characterizing complexity. The paper demonstrates novel experiments that show the power of EMO in generating robots with different morphologies, yet with very similar locomotion abilities. The proposed framework for comparing the complexity of an object across different complexity measures allowed meaningful and quantifiable comparisons between the evolved organisms. We show empirically that the partial order feature inherited in the Pareto concept exhibits characteristics which are suitable for comparing between the complexities of artificially evolved embodied organisms.

In evolutionary computation, genetic operators, such as mutation and crossover, are employed to perturb individuals to generate the next population. However these fixed, problem independent genetic operators may destroy the sub-solution, usually called building blocks, instead of discovering and preserving them. One way to overcome this problem is to build a model based on the good individuals, and sample this model to obtain the next population. There is a wide range of such work in genetic algorithms; but because of the complexity of the genetic programming (GP) tree representation, little work of this kind has been done in GP. In this paper, we propose a new method, grammar model-based program evolution (GMPE) to evolved GP program. We replace common GP genetic operators with a probabilistic context-free grammar (SCFG). In each generation, an SCFG is learnt, and a new population is generated by sampling this SCFG model. On two benchmark problems we have studied, GMPE significantly outperforms conventional GP, learning faster and more reliably.

In this paper, we investigate the use of a self-adaptive Pareto evolutionary multi-objective optimization (EMO) approach for evolving the controllers of virtual embodied organisms. The objective of this paper is to demonstrate the trade-off between quality of solutions and computational cost. We show empirically that evolving controllers using the proposed algorithm incurs significantly less computational cost when compared to a self-adaptive weighted sum EMO algorithm, a self-adaptive single-objective evolutionary algorithm (EA) and a hand-tuned Pareto EMO algorithm. The main contribution of the self-adaptive Pareto EMO approach is its ability to produce sufficiently good controllers with different locomotion capabilities in a single run, thereby reducing the evolutionary computational cost and allowing the designer to explore the space of good solutions simultaneously. Our results also show that self-adaptation was found to be highly beneficial in reducing redundancy when compared against the other algorithms. Moreover, it was also shown that genetic diversity was being maintained naturally by virtue of the system's inherent multi-objectivity.

In this paper, we present a comparison between two multiobjective formulations to the formation of neuro-ensembles. The first formulation splits the training set into two nonoverlapping stratified subsets and form an objective to minimize the training error on each subset, while the second formulation adds random noise to the training set to form a second objective. A variation of the memetic Pareto artificial neural network (MPANN) algorithm is used. MPANN is based on differential evolution for continuous optimization. The ensemble is formed from all networks on the Pareto frontier. It is found that the first formulation outperformed the second. The first formulation is also found to be competitive to other methods in the literature.

Global optimization problems continue to be a challenge in computational mathematics. The field is progressing in two streams: deterministic and heuristic approaches. In this paper, we present a hybrid method that uses the discrete gradient method, which is a derivative free local search method, and evolutionary strategies. We show that the hybridization of the two methods is better than each of them in isolation.

We present a boosting genetic algorithm for classification rule discovery. The method is based on the iterative rule learning approach to genetic classifiers. The boosting mechanism increases the weight of those training instances that are not classified correctly by the new rules, so that in the next iteration the algorithm focuses the search on those rules that capture the misclassified or uncovered instances. We show that the boosted genetic classifier has higher accuracy for prediction, or from an alternative and perhaps more important perspective, uses less computational resources for similar accuracy, than the original genetic classifier.

In genetic programming (GP) and most other evolutionary computing approaches, the knowledge learned during the evolutionary processing is implicitly encoded in the population. A small family of approaches, known as estimation of distribution algorithms, learn this knowledge directly in the form of probability distributions. In this research, we proposed a new approach for program synthesis - program evolution with explicit learning (PEEL), belonging to this family. PEEL learns probability distributions from previous generations and stochastically generates new populations according to this distribution. PEEL is intrinsically different from GP systems because it abandons conventional GP genetic operators and does not maintain population. On the benchmark problems we have studied, this approach shows at least comparable performance to GP.

A self-adaptive Pareto evolutionary multiobjective optimization (EMO) algorithm based on differential evolution is proposed for evolving locomotion controllers in an artificially embodied legged creature. The objective is to demonstrate the trade-off between quality of solutions and computational cost. We show empirically that evolving controllers using the proposed algorithm incurs significantly less computational cost compared to a self-adaptive weighted sum EMO algorithm, a self-adaptive single-objective evolutionary algorithm and a hand-tuned Pareto EMO algorithm. The main contribution of the self-adaptive Pareto EMO approach is its ability to produce sufficiently good controllers with different locomotion capabilities in a single run, thereby reducing the evolutionary computational cost dramatically. Moreover, the performance of our proposed Pareto EMO algorithm was found to be comparable against a current state-of-the-art Pareto EMO algorithm, the NSGA-II algorithm, for evolving legged locomotion controllers.

Marriage in Honey-Bees Optimization is a new swarm intelligence technique inspired by the marriage process of honey-bees. It has been shown to be very effective in solving the propositional satisfiability problem known as 3-SAT. The objective of this paper is to test a conventional annealing approach as the basis for determining the pool of drones. The modified algorithm is tested using a group of randomly generated hard 3-SAT problems to compare its behavior and efficiency against previous implementations. The overall performance of the MBO algorithm was found to have improved significantly using the proposed annealing function. Furthermore, a dramatic improvement was noted with the committee machine using this true annealing approach.

The use of backpropagation for training artificial neural networks (ANNs) is usually associated with a long training process. The user needs to experiment with a number of network architectures; with larger networks, more computational cost in terms of training time is required. The objective of this letter is to present an optimization algorithm, comprising a multiobjective evolutionary algorithm and a gradient-based local search. In the rest of the letter, this is referred to as the memetic Pareto artificial neural network algorithm for training ANNs. The evolutionary approach is used to train the network and simultaneously optimize its architecture. The result is a set of networks, with each network in the set attempting to optimize both the training error and the architecture. We also present a self-adaptive version with lower computational cost. We show empirically that the proposed method is capable of reducing the training time compared to gradient-based techniques.

The Pareto differential evolution (PDE) algorithm was introduced and showed competitive results. The behavior of PDE, as in many other evolutionary multiobjective optimization (EMO) methods, varies according to the crossover and mutation rates. In this paper, we present a new version of PDE with self-adaptive crossover and mutation. We call the new version self-adaptive Pareto differential evolution (SPDE). The emphasis of this paper is to analyze the dynamics and behavior of SPDE. The experiments also show that the algorithm is very competitive with other EMO algorithms.

Genetic programming (GP) plays the primary role in the discovery of programs through evolving the program's set of parse trees. We present a new technique for constructing programs through ant colony optimization (ACO) using the tree adjunct grammar (TAG) formalism. We call the method AntTAG and we show that the results are very promising.

Honey-bees are one of the most well studied social insects. They exhibit many features that distinguish their use as models for intelligent behavior. These features include division of labor, communication on the individual and group level, and cooperative behavior. In this paper, we present a unified model for the marriage in honey-bees within an optimization context. The model simulates the evolution of honey-bees starting with a solitary colony (single queen without a family) to the emergence of an eusocial colony (one or more queens with a family). From optimization point of view, the model is a committee machine approach where we evolve solutions using a committee of heuristics. The model is applied to a fifty propositional satisfiability problems (SAT) with 50 variables and 215 constraints to guarantee that the problems are centered on the phase transition of 3-SAT. Our aim in this paper is to analyze the behavior of the algorithm using biological concepts (number of queens, spermatheca size, and number of broods) rather than trying to improve the performance of the algorithm while losing the underlying biological essence. Notwithstanding, the algorithm outperformed WalkSAT, one of the state-of-the-art algorithms for SAT.

The use of evolutionary algorithms (EAs) to solve problems with multiple objectives (known as multi-objective optimization problems (MOPs)) has attracted much attention. Being population based approaches, EAs offer a means to find a group of Pareto-optimal solutions in a single run. Differential evolution (DE) is an EA that was developed to handle optimization problems over continuous domains. The objective of this paper is to introduce a novel Pareto-frontier differential evolution (PDE) algorithm to solve MOPs. The solutions provided by the proposed algorithm for two standard test problems, outperform the Strength Pareto Evolutionary Algorithm, one of the state-of-the-art evolutionary algorithms for solving MOPs.