Many real-world applications often generate unbalanced data. Learning from such data may lead to biased classifiers that perform poorly on the class of interest. Oversampling methods have been shown to be effective in rebalancing unbalanced data to help classifiers avoid performance bias. However, many existing oversampling methods rely on a predesigned linear model structure and the neighborhood information of an original instance. This may lead to the generation of noisy instances when the original data has noise. In this study, we develop a novel oversampling method in which genetic programming is introduced to automatically select good-quality instances and evolve a model structure that combines the selected instances to create a new instance. In the proposed oversampling method, an individual is used to represent a generated instance, which is evaluated by the fitness function designed based on the Euclidean distance and the cosine theorem. In the experiments, we examine the effectiveness of the proposed oversampling method in assisting different types of classifiers to solve the issue of class imbalance, and compare it with popular sampling methods in unbalanced classification. The results have been analyzed comprehensively, indicating that the new method successfully addressed the class imbalance issue by generating a group of good-quality instances for the minority class and outperformed the compared sampling methods in almost all cases.

Diverse human motion prediction (HMP) aims to predict multiple plausible future motions given an observed human motion sequence. It is a challenging task due to the diversity of potential human motions while ensuring an accurate description of future human motions. Current solutions are either low-diversity or limited in expressiveness. Recent denoising diffusion probabilistic models (DDPM) demonstrate promising performance in various generative tasks. However, introducing DDPM directly into diverse HMP incurs some issues. While DDPM can enhance the diversity of potential human motion patterns, the predicted human motions gradually become implausible over time due to significant noise disturbances in the forward process of DDPM. This phenomenon leads to the predicted human motions being unrealistic, seriously impacting the quality of predicted motions and restricting their practical applicability in real-world scenarios. To alleviate this, we propose a novel conditional diffusion-based generative model, called DivDiff, to predict more diverse and realistic human motions. Specifically, the DivDiff employs DDPM as our backbone and incorporates Discrete Cosine Transform (DCT) and Transformer mechanisms to encode the observed human motion sequence as a condition to instruct the reverse process of DDPM. More importantly, we design a diversified reinforcement sampling function (DRSF) to enforce human skeletal constraints on the predicted human motions. DRSF utilizes the acquired information from human skeletal as prior knowledge, thereby reducing significant disturbances introduced during the forward process. Extensive results received in the experiments on two widely-used datasets (Human3.6M and HumanEva-I) demonstrate that our model obtains competitive performance on both diversity and accuracy.

In evolutionary robotics (ER), the evolution of a robot’s morphology (i.e., physical structure) or controller (i.e., control algorithm or instruction sequence) often entails tackling an extensive number of tasks. The use of evolutionary multitasking (EMT) in ER, which optimizes multiple tasks simultaneously by reusing potentially useful knowledge across diverse tasks, could improve the performance of problem-solving to each task. However, existing EMT methods do not fully use intertask correlations, limiting knowledge sharing. In view of this, this study introduces a novel framework, termed adaptive group-based collaborative optimization, tailored for handling optimization problems involving a large number of tasks within the ER domain simultaneously. The proposed framework divides tasks into groups according to their similarity and then proceeds through two principal stages, namely, intergroup knowledge separation and intragroup knowledge reunion. During intergroup knowledge separation stage, an adaptive method for selecting crossover operators enables source tasks to share useful knowledge to the target task across groups. During intragroup knowledge reunion stage, an adaptive knowledge combination strategy facilitates the target task in assimilating knowledge from multiple sources intragroup. We validated the efficacy of the proposed framework in both planar manipulators and hexapod robot experiments. The results indicate that our method outperforms existing state-of-the-art algorithms (i.e., MME, MMKT) on several metrics (e.g., mean fitness and quality diversity metrics). The proposed method can effectively improve the effectiveness and diversity of solutions in solving ER problems with a large number of tasks (e.g., 5 000 or 10 000), and has broad potential in practical ER applications.

Semi-supervised time-series classification could effectively alleviate the issue of lacking labeled data. However, existing approaches usually ignore model interpretability, making it difficult for humans to understand the principles behind the predictions of a model. Shapelets are a set of discriminative subsequences that show high interpretability in time series classification tasks. Shapelet learning-based methods have demonstrated promising classification performance. Unfortunately, without enough labeled data, the shapelets learned by existing methods are often poorly discriminative, and even dissimilar to any subsequence of the original time series. To address this issue, we propose the Diffusion Language-Shapelets model (DiffShape) for semi-supervised time series classification. In DiffShape, a self-supervised diffusion learning mechanism is designed, which uses real subsequences as a condition. This helps to increase the similarity between the learned shapelets and real subsequences by using a large amount of unlabeled data. Furthermore, we introduce a contrastive language-shapelets learning strategy that improves the discriminability of the learned shapelets by incorporating the natural language descriptions of the time series. Experiments have been conducted on the UCR time series archive, and the results reveal that the proposed DiffShape method achieves state-of-the-art performance and exhibits superior interpretability over baselines.

Unbalanced classification is an essential machine learning task, which has attracted widespread attention from both the academic and industrial communities due mainly to its broad applications. Evolutionary computation (EC) has contributed greatly to unbalanced classification. However, to the best of our knowledge, there have not been any comprehensive investigations on the strengths and weaknesses of alternative EC methods in addressing various challenging problems in unbalanced classification. This article reviews the literature which utilize EC techniques for unbalanced classification, with the aim of revealing the contributions of EC to unbalanced classification, providing an overview of recent advances, and identifying limitations of existing works. In addition, we present a series of real-world applications, and identify open challenges as well as possible research directions for the future.

Deep convolutional neural networks have become a dominant solution for numerous image classification tasks. However, a main criticism is the poor explainability due to the black-box characteristic, which hurdles the extensive usage of deep convolutional neural networks. To address this issue, this paper proposes a new evolutionary multi-objective based method, which aims to explain the behaviours of deep convolutional neural networks by evolving local explanations on specific images. To the best of our knowledge, this is the first evolutionary multi-objective method to evolve local explanations. The proposed method is model-agnostic, i.e. it is applicable to explain any deep convolutional neural networks. ImageNet is used to examine the effectiveness of the proposed method. Three well-known deep convolutional neural networks -VGGNet, ResNet, and MobileNet, are chosen to demonstrate the modelagnostic characteristic. Based on the experimental results, it can be observed that the local explanations are understandable to end-users, who need to check the sensibility of the evolved explanations to decide whether to trust the predictions made by the deep convolutional neural networks. Furthermore, the local explanations evolved by the proposed method improves the confidence of deep convolutional neural networks making the predictions. Lastly, the pareto front and convergence analyses indicate that the proposed method can form a good set of nondominated solutions.

The ability to swiftly and precisely reach any user-specified target location is necessary for a robotic arm that can be used in real-world scenarios. To date, many evolutionary optimization algorithms have been used to design controllers for robotic arms. However, when designing a robotic arm to reach multiple targets, most existing methods need to evolve the control strategy from scratch for each target, rather than trying to reuse existing experience. Therefore, computational resources are repeatedly and meaninglessly consumed. To this end, this paper proposes a genetic algorithm based on knowledge transfer (GAKT) dedicated to reusing existing knowledge to optimize a new robotic arm control task. Specifically, the knowledge transfer process can be summarized into the following two steps. First, through sequential transfer, GAKT initializes the population with the help of a knowledge base constructed by a quality diversity algorithm. Second, underperforming individuals are encouraged to acquire knowledge from excellent individuals in the same generation during the optimization process. We tested the effectiveness of GAKT and investigated its average performance by selecting multiple target points in different dimensions. The results show that GAKT can find the most advantageous arrival strategy (that is, make the end of the manipulator the closest to the target) on most of the selected targets. Moreover, we conducted ablation experiments and demonstrated the effectiveness of the knowledge transfer processes.

Deep reinforcement learning has attracted wide interest because of its extraordinary capabilities in multiple fields. However, morphology optimization by using evolutionary computation techniques has not been intensively investigated. In this paper, we explore the use of genetic algorithms (GA) to automatically design the morphology of an agent. Evaluating the performance of an agent is very time-consuming because it needs to be trained from scratch. Moreover, it is computationally infeasible to train separate controllers for all possible different morphologies of agents to identify the optimal ones and is difficult to obtain the accurate cumulative reward of an agent to estimate the performance of the morphologies. To address these issues, we use a morphology comparator as a surrogate model to estimate the probability of one morphology being better than the other, instead of directly predicting the performance of each morphology. A set of surrogate models based on a radial basis function network are developed before evolution to make full use of the data to guide the search. Experimental results indicate that the proposed method is able to efficiently find out optimal morphologies to achieve better performance than the default morphology.

Deep Neural Networks (DNNs) have been criticized because they easily overfit noisy (incorrect) labels. To improve the robustness of DNNs, existing methods for image data regard samples with small training losses as correctly labeled data (small-loss criterion). Nevertheless, time series' discriminative patterns are easily distorted by external noises (i.e., frequency perturbations) during the recording process. This results in training losses of some time series samples that do not meet the small-loss criterion. Therefore, this paper proposes a deep learning paradigm called Scale-teaching to cope with time series noisy labels. Specifically, we design a fine-to-coarse cross-scale fusion mechanism for learning discriminative patterns by utilizing time series at different scales to train multiple DNNs simultaneously. Meanwhile, each network is trained in a cross-teaching manner by using complementary information from different scales to select small-loss samples as clean labels. For unselected large-loss samples, we introduce multi-scale embedding graph learning via label propagation to correct their labels by using selected clean samples. Experiments on multiple benchmark time series datasets demonstrate the superiority of the proposed Scale-teaching paradigm over state-of-the-art methods in terms of effectiveness and robustness.

Human motion prediction intends to predict how humans move given a historical sequence of 3D human motions. Recent transformer-based methods have attracted increasing attentions and demonstrated their promising performance in 3D human motion prediction. However, existing methods generally decompose the input of human motion information into spatial and temporal branches in a separate way and seldom consider their inherent coherence between the two branches, hence often failing to register the dynamic spatio-temporal information during the training process. Motivated by these issues, we propose a spatio-temporal cross-transformer network (STCT) for 3D human motion predictions. Specifically, we investigate various types of interaction methods (i.e., Concatenation Interaction, Msg token interaction, and Cross-transformer) to capture the coherence of the spatial and temporal branches. According to the obtained results, the proposed cross-transformer interaction method shows its superiority over other methods. Meanwhile, considering that most existing works treat the human body as a set of 3D human joint positions, the predicted human joints are proportionally less appropriate to the realistic human body due to unreasonable bone length and non-plausible poses as time progresses. We further resort to the bone constraints of human mesh to produce more realistic human motions. By fitting a parametric body model (i.e., SMPL-X model) to the predicted human joints, a reconstruction loss function is proposed to remedy the unreasonable bone length and pose errors. Comprehensive experiments on AMASS and Human3.6M datasets have demonstrated that our method achieves superior performance over compared methods.

Unbalanced classification has attracted widespread interest because of its broad applications. However, due to mainly the uneven class distribution, constructed classifiers are usually biased toward the majority class, and thereby perform terribly on the minority class. Unfortunately, the minority class is often the class of interest in many real-world applications. High dimensionality often further degrades the classification performance, making it more complicated to address the class imbalance issue. Genetic programming (GP) has been applied to construct classifiers, which can simultaneously select good-quality features to improve the classification performance. To handle the class imbalance issue, cost-sensitive GP classifiers treat the minority class as being more important than the majority class, but this may cause an accuracy decrease in overlapping areas where the prior probabilities of the two classes are almost the same. To date, most cost-sensitive classification methods have not been specifically investigated how the impacts of overlapping areas on cost-sensitive classifiers can be avoided. In this study, we propose a new cost-sensitive GP method, where rough set theory is employed to detect overlapping areas before training cost-sensitive classifiers for classification with unbalanced high-dimensional data. The proposed method is compared with 46 popular classification methods, including 10 GP methods and 36 non-GP methods on 14 datasets that are unbalanced and high dimensional. The experimental results indicate that the proposed method performs better than the compared methods in almost all cases.

High-dimensional unbalanced classification is challenging because of the joint effects of high dimensionality and class imbalance. Genetic programming (GP) has the potential benefits for use in high-dimensional classification due to its built-in capability to select informative features. However, once data are not evenly distributed, GP tends to develop biased classifiers which achieve a high accuracy on the majority class but a low accuracy on the minority class. Unfortunately, the minority class is often at least as important as the majority class. It is of importance to investigate how GP can be effectively utilized for high-dimensional unbalanced classification. In this article, to address the performance bias issue of GP, a new two-criterion fitness function is developed, which considers two criteria, that is, the approximation of area under the curve (AUC) and the classification clarity (i.e., how well a program can separate two classes). The obtained values on the two criteria are combined in pairs, instead of summing them together. Furthermore, this article designs a three-criterion tournament selection to effectively identify and select good programs to be used by genetic operators for generating offspring during the evolutionary learning process. The experimental results show that the proposed method achieves better classification performance than other compared methods.

In classification, when class overlap is intertwined with the issue of class imbalance, it is often challenging to discover useful patterns because of an ambiguous boundary between the majority class and the minority class. This becomes more difficult if the data is high-dimensional. To date, very few pieces of work have investigated how the class overlap issue can be effectively addressed or alleviated in classification with high-dimensional unbalanced data. In this paper, we propose a new genetic programming based method, which is able to automatically and directly detect borderline instances, in order to address the class overlap issue in classification with high-dimensional unbalanced data. In the proposed method, each individual has two trees to be trained together based on different classification rules. The proposed method is examined and compared with baseline methods on high-dimensional unbalanced datasets. Experimental results show that the proposed method achieves better classification performance than the baseline methods in almost all cases.

Class imbalance is an unavoidable issue in many real-world applications. Learning from unbalanced data, classifiers are often biased toward the majority class, while the minority class is important as well (even more important in many cases). How the issue of class imbalance is addressed becomes more challenging if a classification task further encounters the high dimensionality issue. This paper proposes a new genetic programming (GP) approach to high-dimensional unbalanced classification. A new classification mechanism is proposed for GP to improve its classification performance. This new classification mechanism is independent of a classification threshold to separate the majority class and the minority class. The effectiveness of the proposed method is examined on seven high-dimensional unbalanced datasets. Experimental results indicate that the proposed GP method often performs better than other GP methods that use a fitness function to solve the issue of class imbalance, in terms of classification performance and training time.

High-dimensionality and class imbalance represent two main challenges in classification. Recently, there is a growing number of datasets exhibiting the characteristics of the combination of the class imbalance and high-dimensionality. Genetic programming (GP) has been successfully applied to solve high-dimensional classification tasks. However, most existing GP methods may also suffer from a performance bias if the class distribution is unbalanced. Using fitness functions for cost adjustment is one of the most important methods in GP to address the class imbalance issue. This paper develops new fitness functions in GP to address the class imbalance issue in classification with high-dimensional unbalanced data. Two fitness functions are proposed to increase the performance of the traditional accuracy measures, and one fitness function is proposed to approximate Area Under Curve (AUC) with the goal to save the training time. Experiments on six high-dimensional unbalanced datasets show the better performance of the proposed fitness functions, compared to existing fitness functions.