In multi-agent reinforcement learning (MARL), trust region (TR) methods are widely used because they effectively mitigate the nonstationarity of multi-agent systems and facilitate collaboration among diverse agent types. Based on the multi-agent advantage decomposition lemma, TR methods adopt a sequential update scheme (i.e., agents’ policy networks are trained with a certain order). However, current TR methods lack a ranking scheme and train the agents in a random order, this results in suboptimal performance and large variances. To solve this issue, based on agents’ observations (the input of agents’ policy networks), we formulate our ranking criteria and furthermore propose our ranking schemes. Specifically, we avoid agents with similar observations being ranked adjacent to each other for training and give higher priority to the agents with more information in their observations. We extend our schemes to popular TR methods and evaluate them on a series of StarCraftII, Google Football and Multi-Agent MuJoCo tasks, results show that our ranking schemes can enhance current TR methods in many tasks, whatever in performance, efficiency or stability, indicating its modeling capability on both homogeneous and heterogeneous agent tasks.

Graph Neural Network (GNN) has been widely applied in multivariate time series forecasting due to its excellent relationship modeling capabilities. However, current methods still face limitations in computational efficiency or time series expression capabilities. To address these issues, we propose an Efficient Graph-Enhanced Neural Network (EGENN), which consists of an adjacency matrix generator, GNN, and projection module. Firstly, EGENN designs a spectral similarity-based graph construction method and further enhances the expressive power of temporal features. Secondly, we introduce an inter-layer attention graph convolutional network, which adaptively aggregates information from different network depths to better capture complex patterns. Finally, a predictive projection strategy fusing wavelet convolutions and patch-wise transformation is proposed to produce compact parameterization and extended receptive fields. Experiments on five datasets from different domains show that our model achieves state-of-the-art prediction performance while maintaining low computational resource consumption.

Image Quality Assessment (IQA) aims to evaluate the perceptual quality of images based on human subjective perception. Existing methods generally combine multiscale features to achieve high performance, but most rely on straightforward linear fusion of these features, which may not adequately capture the impact of distortions on semantic content. To address this, we propose a bottom-up image quality assessment approach based on the Contrastive Language-Image Pre-training (CLIP, a recently proposed model that aligns images and text in a shared feature space), named BPCLIP, which progressively extracts the impact of low-level distortions on high-level semantics. Specifically, we utilize an encoder to extract multiscale features from the input image and introduce a bottom-up multiscale cross attention module designed to capture the relationships between shallow and deep features. In addition, by incorporating 40 image quality adjectives across six distinct dimensions, we enable the pre-trained CLIP text encoder to generate representations of the intrinsic quality of the image, thereby strengthening the connection between image quality perception and human language. Our method achieves superior results on most public Full-Reference (FR) and No-Reference (NR) IQA benchmarks, while demonstrating greater robustness.

Training an effective policy on complex goal-reaching tasks with sparse rewards is an open challenge. It is more difficult for the task of reaching remote goals (RRG), as the unavailability of the original rewards and large Wasserstein distance between the distributions of desired goals and initial states make existing methods for common goal-reaching tasks inefficient or even completely ineffective. In this article, we propose progressively learning to reach remote goals by continuously updating boundary goals (PLUB), which solves RRG tasks by reducing the Wasserstein distance between the distributions of boundary goals and desired goals. Specifically, the concept of boundary goal is introduced, which is the set of the closest achieved goals for each desired goal. In addition, to reduce the computational complexity caused by the Wasserstein distance, the closest moving distance is introduced, which is its upper bound, and also the expectation of the distance between the desired goal and the closest boundary goal. By selecting the appropriate intermediate goal from all boundary goals and continuously updating boundary goals, both the closest moving distance and the Wasserstein distance can be reduced. As a result, RRG tasks degenerate into common goal-reaching tasks that can be efficiently solved by a combination of hindsight relabeling and the learning from demonstrations (LfD) method. Extensive experiments on several robotic manipulation tasks demonstrate that PLUB can bring substantial improvements over the existing methods.

Three-dimensional (3D) reconstruction of neurons is a critical and evolving area in neuroscience, addressing the substantial challenges presented by weak signals, high noise levels, and heterogeneous signal distribution in neuronal optical images. Previous methodologies predominantly focused on reconstructing neuronal fibers but faced significant limitations in integrating both neuronal fibers and somas, making it difficult to handle large-scale neuronal image reconstruction. Furthermore, conventional techniques often employ fixed thresholding to eliminate background noise, inadvertently leading to the loss of valuable low-intensity neuronal signals, which are crucial for comprehensive neuronal analysis. In response to these limitations, we propose ADTAH, an innovative neuron image reconstruction method that leverages the adaptive distance transform combined with the Hessian matrix for robust and precise 3D reconstruction of neurons. ADTAH consists of two branches: nerve fiber reconstruction and soma reconstruction. The nerve fiber reconstruction branch begins with adaptive threshold distance transform, dynamically adjusting the Hessian matrix window size to effectively capture nerve fibers of varying thicknesses, thereby optimizing the extraction of tubular structures. The soma reconstruction branch employs high-threshold distance transform to accurately identify and fill somas, ensuring comprehensive coverage of both fibers and somas. Our experiments on publicly available 3D neuron image datasets demonstrate that ADTAH outperforms existing techniques, providing superior differentiation between neuronal and background signals, enhanced computational efficiency, and greater robustness. Our approach has demonstrated state-of-the-art performance on the publicly available 3D Neuron image dataset Big Neuron and an fMost dataset made available by the Allen Institute for Science.

Deep network-based image Compressive Sensing (CS) has attracted much attention in recent years. However, there still exist the following two issues: 1) Existing methods typically use fixed-scale sampling, which leads to limited insights into the image content. 2) Most pre-trained models can only handle fixed sampling rates and fixed block scales, which restricts the scalability of the model. In this paper, we propose a novel scale-aware scalable CS network (dubbed S2-CSNet), which achieves scale-aware adaptive sampling, fine granular scalability and high-quality reconstruction with one single model. Specifically, to enhance the scalability of the model, a structural sampling matrix with a predefined order is first designed, which is a universal sampling matrix that can sample multi-scale image blocks with arbitrary sampling rates. Then, based on the universal sampling matrix, a distortion-guided scale-aware scheme is presented to achieve scale-variable adaptive sampling, which predicts the reconstruction distortion at different sampling scales from the measurements and select the optimal division scale for sampling. Furthermore, a multi-scale hierarchical sub-network under a well-defined compact framework is put forward to reconstruct the image. In the multi-scale feature domain of the sub-network, a dual spatial attention is developed to explore the local and global affinities between dense feature representations for deep fusion. Extensive experiments manifest that the proposed S2-CSNet outperforms existing state-of-the-art CS methods.

Deep learning (DL)-based human activity recognition (HAR) methods have shown promise in the applications of health Internet of Things (IoT) and wireless body sensor networks (BSNs). However, adapting these methods to new users in real-world scenarios is challenging due to the cross-subject issue. To solve this issue, we propose ActiveSelfHAR, a framework that combines active learning’s benefit of sparsely acquiring informative samples with actual labels and self-training’s benefit of effectively utilizing unlabeled data to adapt the HAR model to the target domain, i.e., the new users. ActiveSelfHAR consists of several key steps. First, we utilize the model from the source domain to select and label the domain invariant samples, forming a self-training set. Second, we leverage the distribution information of the self-training set to identify and annotate samples located around the class boundaries, forming a core set. Third, we augment the core set by considering the spatiotemporal relationships among the samples in the nonself-training set. Finally, we combine the self-training set and augmented core set to construct a diverse training set in the target domain and fine-tune the HAR model. Through leave-one-subject-out validation on three IMU-based data sets and one EMG-based data set, our method achieves mean HAR accuracies of 95.20%, 82.06%, 89.52%, and 92.82%, respectively. Our method demonstrates similar HAR accuracies to the upper bound, i.e., fine-tuning framework with approximately 1% labeled data of the target data set, while significantly improving data efficiency and time cost. Our work highlights the potential of implementing user-independent HAR methods into health IoT and BSN.

The high complexity and wide applications of Cyber-Physical Systems (CPSs) pose a large requirement on both accuracy and interpretability of the time-series anomaly detection algorithms. While a large number of deep learning algorithms have achieved excellent accuracy, the interpretability is often limited, especially when considering retaining correlations in multivariate time-series. In this paper, we propose a novel multivariate time-series anomaly detection method based on adaptive masking mechanism to improve both accuracy and interpretability, which contains a specially designed series saliency module. For more intuitive and interpretable results, a learnable adaptive mask is introduced in the series saliency module, which can disclose the influence on anomalies in both feature and temporal dimensions. The original time-series and their versions with adaptive perturbations added are then mixed via the mask forming an adaptive data augmentation method to improve the accuracy of anomaly detection. Furthermore, the anomaly detection module is model-agnostic, whether based on forecasting or reconstruction. The optimization of the training objectives will lead to more accurate and interpretable detection results. With four real-world datasets, we demonstrate that the adaptive mask can provide more accurate anomaly detection results with meaningful interpretations in the form of a mask matrix.

Matrix low rank approximation is an effective method to reduce or eliminate the statistical redundancy of its components. Compared with the traditional global low rank methods such as singular value decomposition (SVD), local low rank approximation methods are more advantageous to uncover interpretable data structures when clear duality exists between the rows and columns of the matrix. Local low rank approximation is equivalent to low rank submatrix detection. Unfortunately, existing local low rank approximation methods can detect only submatrices of specific mean structure, which may miss a substantial amount of true and interesting patterns. In this work, we develop a novel matrix computational framework called RPSP (Random Probing based submatrix Propagation) that provides an effective solution for the general matrix local low rank representation problem. RPSP detects local low rank patterns that grow from small submatrices of low rank property, which are determined by a random projection approach. RPSP is supported by theories of random projection. Experiments on synthetic data demonstrate that RPSP outperforms all state-of-the-art methods, with the capacity to robustly and correctly identify the low rank matrices when the pattern has a similar mean as the background, background noise is heteroscedastic and multiple patterns present in the data. On real-world datasets, RPSP also demonstrates its effectiveness in identifying interpretable local low rank matrices.

Cloud computing offers a new way of service provision by re-arranging various resources over the Internet. The most important and popular cloud service is data storage. In order to preserve the privacy of data holders, data are often stored in cloud in an encrypted form. However, encrypted data introduce new challenges for cloud data deduplication, which becomes crucial for big data storage and processing in cloud. Traditional deduplication schemes cannot work on encrypted data. Existing solutions of encrypted data deduplication suffer from security weakness. They cannot flexibly support data access control and revocation. Therefore, few of them can be readily deployed in practice. In this paper, we propose a scheme to deduplicate encrypted data stored in cloud based on ownership challenge and proxy re-encryption. It integrates cloud data deduplication with access control. We evaluate its performance based on extensive analysis and computer simulations. The results show the superior efficiency and effectiveness of the scheme for potential practical deployment, especially for big data deduplication in cloud storage.