In multimodal multi-objective optimization problems (MMOPs), multiple solutions on different Pareto optimal solution sets (PSs) are mapped to the same point on the Pareto front. Considering these different solutions can provide users with richer decisions, the search of multiple PSs is crucial when solving MMOPs. To this end, many multimodal multi-objective evolutionary algorithms (MMOEAs) often employ intricate mechanisms to maintain the diversity of the offspring in mating selection, but ignore to learn PSs. In this paper, a data-driven evolutionary algorithm based on inductive graph neural networks (DEA-IGNN) is proposed to solve MMOPs, which successfully learns the PSs topology by the graph structure to generate offspring with good performance. Specifically, a graph topology construction method based on Euclidean distance in the decision space is designed. It determines the neighborhood by calculating the Euclidean distance of individuals in the decision space and establishes the topological relationships to construct the graphs representing of population distribution. On this basis, a model based on inductive graph neural networks is constructed to assist offspring reproduction, which can learn unknown nodes by sampling and aggregating existing information. Moreover, a data-driven reproduction strategy is proposed to predict offspring with the good diversity and convergence, which uses the traditional variation operators to generate training data and adopts these data to train the model. The proposed DEA-IGNN is implemented and compared with eleven competitive MMOEAs on three test suites and a practical problem. The experimental results show that DEA-IGNN has good performance.

Single domain generalization aims to develop a robust model trained on a source domain to generalize well on unseen target domains. Recent progress in single domain generalization has focused on expanding the scope of training data through style (e.g., backgrounds) augmentation. However, existing methods are difficult to generate data with large style shifts due to the lack of precise correlation measures between the generated and original data, and they struggle to effectively capture the consistency between the generated and original data when learning feature representations. In this article, we propose a novel Mutual Information-guided Style Augmentation (MISA) based single domain generalization method. Specifically, MISA incorporates a style diversity module, which uses the matrix-based Rényi’s \(\alpha\) -order entropy functionals to compute an approximate mutual information value between the augmented and original data, minimizing it to guide style generator learning. Moreover, MISA combines the merits of the random convolution and affine transformation to further improve the texture diversity of the augmented data. Additionally, MISA introduces a representation learning module, which minimizes the approximate mutual information value between the prediction logits of the original sample and its corresponding residual component to capture the consistency between the generated and original data for feature representation optimization. Using five real-world datasets, the extensive experiments have demonstrated the effectiveness of MISA, in comparison with state-of-the-art methods.

Stable learning aims to leverage the knowledge in a relevant source domain to learn a prediction model that can generalize well to target domains. Recent advances in stable learning mainly proceed by eliminating spurious correlations between irrelevant features and labels through sample reweighting or causal feature selection. However, most existing stable learning methods either only weaken partial spurious correlations or discard part of true causal relationships, resulting in generalization performance degradation. To tackle these issues, we propose the Dual Feature Learning (DFL) algorithm for stable learning, which consists of two phases. Phase 1 first learns a set of sample weights to balance the distribution of treated and control groups corresponding to each feature, and then uses the learned sample weights to assist feature selection to identify part of irrelevant features for completely isolating spurious correlations between these irrelevant features and labels. Phase 2 first learns two groups of sample weights again using the subdataset after feature selection, and then obtains high-quality feature representations by integrating a weighted cross-entropy model and an autoencoder model to further get rid of spurious correlations. Using synthetic and four real-world datasets, the experiments have verified the effectiveness of DFL, in comparison with eleven state-of-the-art methods.

Panoramic driving systems are crucial for autonomous driving but face challenges in real-time performance and reliability. This paper proposes an end-to-end, multi-tasking MtpNet that reduces latency and enhances detection accuracy. The convolution was upgraded using the Efficient Layer Aggregation Network, and precise multi-task loss functions and more effective training strategies were devised. Our results demonstrate improved performance in small object detection, partial occlusion handling, and drivable area segmentation. The recall of the traffic object detection is 1.3% higher than that of the state-of-the-art model, reaching 94.1%, the mAP50is 6.4% higher, reaching 89.8%, and the mIoU of the drivable area segmentation is 2.7% higher, reaching 95.9%. Additionally, the accuracy of lane detection reached 88.7%. The visual comparison using three datasets TuSimple, CityScapes, and CULane demonstrates that MtpNet has good detection segmentation and strong robustness under various conditions. Codes are available at https://github.com/ErLinErYi/mtpnet

Single domain generalization aims to learn a model that generalizes well to unseen target domains by using a related source domain. However, most existing methods only focus on improving the generalization performance of the model during training, making it difficult to achieve satisfactory performance when deployed in the target domain with large domain shifts. In this paper, we propose a Practical Single Domain Generalization (PSDG) method, which first leverages the knowledge in a source domain to establish a model with good generalization ability in the training phase, and subsequently updates the model to adapt to target domain data using knowledge in the unlabeled target domain during the testing phase. Specifically, during training, PSDG leverages a newly proposed style (e.g., background features) generator named StyIN to generate novel domain data. Moreover, PSDG introduces style-diversity regularization to constantly synthesize distinct styles to expand the coverage of training data, and introduces object-consistency regularization to capture consistency between the currently generated data and the original data, making the model filter style knowledge during training. During testing, PSDG uses a sample-aware and sharpness-aware minimization method to seek for a flat entropy minimum surface for further model optimization by using the knowledge in the unlabeled target domain. Using three real-world datasets the experiments have demonstrated the effectiveness of PSDG, in comparison with several state-of-the-art methods.

This article addresses the optimization problem of zeroth-order in a distributed setting, where the gradient information is not available in the edge Internet of Things (IoT) clients. The high communication costs and poorer convergence hinder the use of zeroth-order optimization methods in distributed IoT. This article proposes a communication-efficient Distributed adaptive Zeroth-order optimization method (DaZoo). DaZoo is applied to optimize a class of nonconvex optimization problems, where each client can only access zeroth-order information of local functions. To estimate the global gradient, each client uses a population-based feedback strategy to approximate the first-order gradient, which are then aggregated through a central server. A novel global adaptive optimization scheme is devised for DaZoo, making it with the flexibility to adapt to any landscape without the need for manual parameter tuning. Furthermore, sparsification techniques are incorporated into the local model differences to substantially reduce communication overhead. The theoretical findings suggest that DaZoo can reduce iteration complexity compared to the baselines. Case studies on distributed closed-box attacks and large-scale IoT attack detection demonstrate that DaZoo can outperform state-of-the-art methods.

Federated learning (FL) enables collaborative model training across massively distributed edge devices, such as Internet of Things (IoT) nodes. However, resource constraints impose a major challenge, as there exists a trade-off between maximizing learning accuracy and minimizing communication overhead between the resource-limited devices. In this paper, we present a device selection approach for heterogeneous FL systems based on multi-objective optimization and knowledge transfer. We formulate the resource constraint in federated optimization as a multi-objective problem, and obtain Pareto-optimal solutions balancing resource efficiency and test accuracy. Additionally, we introduce an innovative knowledge transfer mechanism that propagates the globally optimal models obtained during multi-objective optimization to subsequent FL tasks, further expediting convergence. The multi-objective formulation and knowledge transfer provide new insights into efficient and robust federated learning for resource-constrained IoT applications. We conduct extensive experiments on real-world datasets. Results demonstrate that our method achieves up to 11% higher accuracy than state-of-the-art methods, while effectively mitigating resource constraints. Impact Statement–Federated learning is an efficient algorithm that enables everything to be interconnected without sharing data. However, resource constraint is the main challenge for federated optimization problems. Although many works have proposed various solutions from different perspectives, these methods cannot simultaneously minimize the communication resource cost while ensuring algorithm performance. We propose an automatic device selection algorithm for federated systems based on multi-objective optimization and knowledge transfer. This work not only reduces the global resource usage rate of federated learning, but also enables it to converge quickly.

LncRNA-protein interactions are ubiquitous in organisms and play a crucial role in a variety of biological processes and complex diseases. Many computational methods have been reported for lncRNA-protein interaction prediction. However, the experimental techniques to detect lncRNA-protein interactions are laborious and time-consuming. Therefore, to address this challenge, this paper proposes a reweighting boosting feature selection (RBFS) method model to select key features. Specially, a reweighted apporach can adjust the contribution of each observational samples to learning model fitting; let higher weights are given more influence samples than those with lower weights. Feature selection with boosting can efficiently rank to iterate over important features to obtain the optimal feature subset. Besides, in the experiments, the RBFS method is applied to the prediction of lncRNA-protein interactions. The experimental results demonstrate that our method achieves higher accuracy and less redundancy with fewer features.

Almost all existing causal feature selection methods are proposed without considering the problem of sample selection bias. However, in practice, as data-gathering process cannot be fully controlled, sample selection bias often occurs, leading to spurious correlations between features and the class variable, which seriously deteriorates the performance of those existing methods. In this article, we study the problem of causal feature selection under sample selection bias and propose a novel Progressive Causal Feature Selection (PCFS) algorithm which has three phases. First, PCFS learns the sample weights to balance the treated group and control group distributions corresponding to each feature for removing spurious correlations. Second, based on the sample weights, PCFS uses a weighted cross-entropy model to estimate the causal effect of each feature and removes some irrelevant features from the confounder set. Third, PCFS progressively repeats the first two phases to remove more irrelevant features and finally obtains a causal feature set. Using synthetic and real-world datasets, the experiments have validated the effectiveness of PCFS, in comparison with several state-of-the-art classical and causal feature selection methods.

In this study, we investigate a challenging problem, namely, robust domain adaptation, where data from only a single well-labeled source domain are available in the training phase. To address this problem, assuming that the causal relationships between the features and the class variable are robust across domains, we propose a novel causal autoencoder (CAE), which integrates a deep autoencoder and a causal structure learning model to learn causal representations using data from a single source domain. Specifically, a deep autoencoder model is adopted to learn the low-dimensional representations, and a causal structure learning model is designed to separate the low-dimensional representations into two groups: causal representations and task-irrelevant representations. Using three real-world datasets, the experiments have validated the effectiveness of CAE, in comparison with eleven state-of-the-art methods.

Learning a causal structure from observational data is crucial for data scientists. Recent advances in causal structure learning (CSL) have focused on local-to-global learning, since the local-to-global CSL can be scaled to high-dimensional data. The local-to-global CSL algorithms first learn the local skeletons, then construct the global skeleton, and finally orient edges. In practice, the performance of local-to-global CSL mainly depends on the accuracy of the global skeleton. However, in many real-world settings, owing to inevitable data quality issues (e.g. noise and small sample), existing local-to-global CSL methods often yield many asymmetric edges (e.g., given anasymmetric edge containing variables A and B, the learned skeleton of A contains B, but the learned skeleton of B does not contain A), which make it difficult to construct a high quality global skeleton. To tackle this problem, this paper proposes a Bootstrap sampling based Causal Structure Learning (BCSL) algorithm. The novel contribution of BCSL is that it proposes an integrated global skeleton learning strategy that can construct more accurate global skeletons. Specifically, this strategy first utilizes the Bootstrap method to generate multiple sub-datasets, then learns the local skeleton of variables on each asymmetric edge on those sub-datasets, and finally designs a novel scoring function to estimate the learning results on all sub-datasets for correcting the asymmetric edge. Extensive experiments on both benchmark and real datasets verify the effectiveness of the proposed method.

Local causal structure learning aims to discover and distinguish direct causes (parents) and direct effects (children) of a variable of interest from data. While emerging successes have been made, existing methods need to search a large space to distinguish direct causes from direct effects of a target variable T. To tackle this issue, we propose a novel Efficient Local Causal Structure learning algorithm, named ELCS. Specifically, we first propose the concept of N-structures, then design an efficient Markov Blanket (MB) discovery subroutine to integrate MB learning with N-structures to learn the MB of T and simultaneously distinguish direct causes from direct effects of T. With the proposed MB subroutine, ELCS starts from the target variable, sequentially finds MBs of variables connected to the target variable and simultaneously constructs local causal structures over MBs until the direct causes and direct effects of the target variable have been distinguished. Using eight Bayesian networks the extensive experiments have validated that ELCS achieves better accuracy and efficiency than the state-of-the-art algorithms.

Domain adaptation aims to facilitate the learning task in an unlabeled target domain by leveraging the auxiliary knowledge in a well-labeled source domain from a different distribution. Almost existing autoencoder-based domain adaptation approaches focus on learning domain-invariant representations to reduce the distribution discrepancy between source and target domains. However, there is still a weakness existing in these approaches: the class-discriminative information of the two domains may be damaged while aligning the distributions of the source and target domains, which makes the samples with different classes close to each other, leading to performance degradation. To tackle this issue, we propose a novel dual-representation autoencoder (DRAE) to learn dual-domain-invariant representations for domain adaptation. Specifically, DRAE consists of three learning phases. First, DRAE learns global representations of all source and target data to maximize the interclass distance in each domain and minimize the marginal distribution and conditional distribution of both domains simultaneously. Second, DRAE extracts local representations of instances sharing the same label in both domains to maintain class-discriminative information in each class. Finally, DRAE constructs dual representations by aligning the global and local representations with different weights. Using three text and two image datasets and 12 state-of-the-art domain adaptation methods, the extensive experiments have demonstrated the effectiveness of DRAE.

Polarimetric SAR classification is an effective approach in image understanding. This paper proposes a novel semantic method for classification of Polarimetric SAR data. The method combines superpixels and semantic model to benefit from both the object-oriented classification and the high-level semantic information. Firstly, pixels was grouped into superpixels via Simple Linear Iterative Clustering (SLIC). Secondly, the feature vector was generated within the superpixels by considering both polarimetric information and textures. To incorporate semantic information, the feature vectors were further processed via probabilistic Latent Semantic Analysis (pLSA). Finally, Supporting Vector Machine (SVM) was utilized to obtain classification results. The results were evaluated with respect to the accuracy of classification and spatial preservation. The results of this work were analyzed by means of RADARSAT-2 data.

Entropy, alpha, and anisotropy (H/α̅/A) of Cloude decomposition are effective in polarimetric SAR image understanding and geophysical information inversion. As an incoherent target decomposition, the inner sample covariance matrix estimation severely affects the estimated parameters. The contradiction between details preservation and accurate parameters estimation is still a challenge task. In this article, we propose adaptive coherency matrix estimation based on local heterogeneity coefficients, and utilize it to parameters estimation of Cloude decomposition. The results were evaluated with respect to details preservation and the accuracy of parameters estimation. The results of this work were analyzed by means of AIRSAR data.

Polarimetric synthetic aperture radar (SAR) images usually contain a mixture of homogeneous and heterogeneous regions, which makes estimation of the coherency matrix a very challenging task. In this paper, we propose an adaptive coherency matrix estimation method that employs local heterogeneity coefficient and leverages the sample covariance matrix estimation to the homogeneous components and the fixed-point estimation to the heterogeneous components. Evaluations were conducted with synthetic polarimetric data and real-world SAR imagery, including UAVSAR, RADARSAT-2, and ESAR. Our experimental results demonstrated that the heterogeneity coefficient effectively characterizes the scattering property of ground objects, which enables adaptive estimation of the coherency matrix in high-resolution polarimetric SAR imagery. Our method was able to handle single- and multilook polarimetric SAR imagery gracefully. Compared with the sample covariance matrix estimator, the fixed-point estimator, and the Lee sigma filtering, our method achieved the best performance for retaining the spatial structure, suppressing speckles, and preserving polarimetric information of SAR imagery with different degrees of heterogeneity.