• A simple and effective multi-site brain disease recognition framework based on tensor decomposition and personalized federated learning is proposed to quickly integrate samples from different hospitals/sites while enabling personalized feature extraction at each site. • A designed Dynamic Prototype Aggregation (DPA) module utilizes a sliding window technique to capture the intrinsic characteristics of time-varying BOLD signals. • A dual-feature aggregation module is designed to aggregate coarse-grained shared features and fine-grained prototype representation features, respectively, to facilitate efficient knowledge sharing among sites. Brain diseases significantly impact physical and mental health, making the development of models to identify biomarkers for early diagnosis essential. However, building high-quality models typically relies on large-scale datasets, while the privacy-sensitive nature of medical data often restricts its sharing and utilization. Multi-site studies provide a potential solution by integrating data from various sources, yet existing methods frequently neglect site-specific private features, such as demographic information. Therefore, in this paper, we propose a simple yet effective framework based on Tensor Decomposition and Personalized Federated Learning (TDPFL) for multi-site brain disease recognition, while protecting these private features. On the central server, we designed a dual feature aggregation module to facilitate efficient knowledge sharing among sites. On the client side, we introduced a personalized branch to safeguard private information ( i.e. , age, gender, and education) and developed a tensor decomposition module to extract features from subjects’ brain scan data. Furthermore, we developed a dynamic prototype aggregation module to monitor evolving brain features over time. This mechanism enhances the model’s capacity to capture these dynamics, thereby improving classification and prediction accuracy. Experiments on two publicly available rs-fMRI datasets across six sites showed that TDPFL outperformed baseline methods with a 4 % improvement in average classification accuracy. Additionally, we identified site-specific brain disease-related biomarkers, offering novel insights into early diagnosis. Code is available at https://github.com/ChaojunZ/TDPFL.git

The recently proposed TrackFormer has established a fully end-to-end framework with the concepts of object query and track query for multi-object tracking (MOT). TrackFormer, which is based on the deformable attention mechanism, heavily depends on the keypoint sampling, where a set of keypoints is sampled around the so-called reference point for the subtasks of object detection and data association in MOT. However, the keypoint sampling is still not effective due to the absence of prior position information and the inaccuracy of the reference point, which leads to degraded tracking performance. In this paper, we propose TrackFormer++ to address this issue of the ineffective keypoint sampling through the strategies of prior position embedding and reference point updating. In the proposed TrackFormer++, the reference point for object detection is utilized as the prior position and explicitly embedded into the object query. Similarly, the reference point for data association is adaptively updated according to a predicted offset relative to the object center in the previous frame. Extensive experiments by the public and private detection on the MOT17 and MOT20 datasets demonstrate that TrackFormer++ achieves superior or comparable performance to the state-of-the-art baselines. Our code is available at:.

Deep learning provides an intelligent analytical approach for Big Data analysis and feature extraction in Industrial Internet of Things (IIoT). However, due to concerns about data security and privacy disclosure, conventional data-centralized deep learning often faces difficulties about data famine and data islands. Federated learning (FL) as a novel privacy-preserving deep learning paradigm breaks the data islands among different smart factories by sharing their model parameters instead of raw data, which essentially solves the data famine problem for training a high-quality deep learning model. Nevertheless, exchanging numerous model parameters not only generates considerable communication overhead but also poses the risk of privacy information disclosure hidden in model parameters due to inference attacks launched by external attackers orhonest-but-curiousservers. The purpose of this article is to build a high-quality trustworthy FL architecture dubbed TrustFedGRU for IIoT while alleviating the communication overhead. First, a multikey decryption assisted privacy-preserving homomorphic encryption scheme is proposed in FL to meet the distinct privacy-preserving requirements of different data owners without impairing model performance. Furthermore, tensor decomposition is leveraged to convert the weight tensor of the gated recurrent unit (GRU) into a low-rank approximation, so as to reduce the communication bandwidth overhead and decrease the storage requirements. Meanwhile, a novel dynamic update-based FL approach is investigated to improve the model performance. The experimental results show that the proposed TrustFedGRU greatly reduces the communication overhead while guaranteeing the model performance and security.

The time-series InSAR technique is commonly used for surface subsidence monitoring, providing insights into the trends of environmental changes. In this study, we employed the time-series InSAR technique in the Hunshandake Sandy Land, which will help evaluate the effectiveness of desertification control measures. The results show that there is no significant widespread subsidence or uplift in the Hunshandake Sandy Land, indicating limited movement of sand particles and minimal impact from human activities. However, within certain small areas of the study region, notable surface subsidence and uplift phenomena were observed, which can be attributed to saline-alkali land presence. Additionally, InSAR monitoring revealed pronounced decorrelation in the crescent-shaped dune region, suggesting rapid surface changes occurring on these dunes.

Mobile crowdsourcing (MC) has emerged as an efficient data collection and processing technique with the growing use of mobile devices. Mobile devices typically have numerous sensors to capture a variety of data types, including location information, speech, picture, and video data. Due to the lack of storage capacity and processing power of mobile devices, conducting in-depth analysis and computation of the data is impossible. Cloud-based MC is a viable solution to the issue of limited resources in data outsourcing. How to effectively represent and process encrypted heterogeneous data is an enormous challenge. To alleviate this matter, a unified encrypted-tensor model is proposed to represent heterogeneous data consisting of unstructured, semistructured, and structured data, which represents data in different formats and from various sources. Due to the heterogeneity of data, we devise the encrypted query index and implement the query scheme for structured, semistructured, and unstructured data by transforming heterogeneous data into a graph. We evaluated the search performance of our proposed scheme on real-world data sets. This article analyzes the aspects of time search efficiency, memory occupation, and approximation accuracy. Theoretical analysis and experimental results show that the searchable encryption method based on heterogeneous data proposed in this article can effectively represent and mine big data.

Synthetic aperture radar (SAR) has a wide range of applications in resource exploration, environmental monitoring, urban and rural planning, among others. However, SAR images often suffer from speckle noise, which requires the use of despeckling techniques. With the increasing availability of SAR time series, there is potential to develop more efficient despeckling methods. Nevertheless, in speckle reduction, the coherence between multitemporal SAR images creates new challenges. In this study, a patch-based low-rank approximation (PLRA) method is proposed for SAR time series despeckling using the RABASAR framework, which effectively eliminates temporal fluctuations and speckles. First, a similar patch search approach in time series is introduced to remove time-dependent changes by analyzing fluctuation models. Then, a low-rank approximation method based on patch stacks is proposed to obtain a low-rank image for noise filtering. Furthermore, the low-rank image is integrated into the RABASAR framework to improve the despeckling process. Experimental results demonstrate the superior performance of the proposed method in preserving image texture details, mitigating temporally correlated disturbances, and reducing speckle noise in comparison to other state-of-the-art methods.

Federated learning enables multiple data providers to collaborate on training models without exposing personal data. During the training process, frequent communication is required between the data provider and the central server, which puts great pressure on federated learning. To reduce the communication pressure of federated learning, we use the CP-decomposition processing model to reduce the size of data that needs to be transmitted during the communication process. In addition, we aggregate the global model based on the Shapley value, and eliminate nodes that are not beneficial to federated learning as soon as possible, which reduces the communication pressure and can stimulate the participating nodes and enhance the enthusiasm of participants in federated learning, thus improving the training results of the global model. We named the system CPSV, which stands for Federated learning of CP-decomposition models based on Shapley value aggregation. Numerous experiments on CPSV have shown that CPSV can motivate and supervise participating nodes to aggregate better global models while reducing the stress of federal learning communication.

The 360-depth estimation with spherical images and LiDAR data has recently become increasingly popular in autonomous driving and scene reconstruction. Compared with perspective images, spherical images have omnidirectional FoV, which exceedingly matches LiDAR data. However, the spherical distortion makes the 360-depth estimation a great challenge. To address this problem, we propose a self-supervised 360 depth estimation network in this paper. The network consists of a spherical convolution branch to extract panoramic image features and a ResNet branch to extract LiDAR features. Then an attention-based decoder is designed to estimate the depth. The reprojection error is used to self-supervise the network training. Experiments on the KITTI-360 dataset demonstrate the effectiveness of the proposed method.

Global visual localization is an important task in geoscience with a plethora of applications such as SLAM and autonomous navigation. Current place recognition approaches restrict the modality of the query data which relies on the database data modality. However, real-world robots are equipped with different sensors in different application scenarios and it is difficult for data from a single fixed modality to accommodate all challenging environments. To overcome this limitation, we propose to build a generalized model that allows spherical images and point clouds to be retrieved under any single-modal query. Our 2D-3D dataset is created based on the KITTI360 dataset with spherical images and corresponding point clouds for training and evaluation. Extensive experimental results demonstrate the effectiveness of our proposed approach.

Deep semi-supervised learning is a fast-growing field with a range of practical applications. This paper provides a comprehensive survey on both fundamentals and recent advances in deep semi-supervised learning methods from perspectives of model design and unsupervised loss functions. We first present a taxonomy for deep semi-supervised learning that categorizes existing methods, including deep generative methods, consistency regularization methods, graph-based methods, pseudo-labeling methods, and hybrid methods. Then we provide a comprehensive review of 60 representative methods and offer a detailed comparison of these methods in terms of the type of losses, architecture differences, and test performance results. In addition to the progress in the past few years, we further discuss some shortcomings of existing methods and provide some tentative heuristic solutions for solving these open problems.

Semi-supervised learning (SSL) has tremendous value in practice due to the utilization of both labeled and unlabelled data. An essential class of SSL methods, referred to as graph-based semi-supervised learning (GSSL) methods in the literature, is to first represent each sample as a node in an affinity graph, and then, the label information of unlabeled samples can be inferred based on the structure of the constructed graph. GSSL methods have demonstrated their advantages in various domains due to their uniqueness of structure, the universality of applications, and their scalability to large-scale data. Focusing on GSSL methods only, this work aims to provide both researchers and practitioners with a solid and systematic understanding of relevant advances as well as the underlying connections among them. The concentration on one class of SSL makes this article distinct from recent surveys that cover a more general and broader picture of SSL methods yet often neglect the fundamental understanding of GSSL methods. In particular, a significant contribution of this article lies in a newly generalized taxonomy for GSSL under the unified framework, with the most up-to-date references and valuable resources such as codes, datasets, and applications. Furthermore, we present several potential research directions as future work with our insights into this rapidly growing field.

The prosperity of deep neural networks in various computer vision applications heavily depends on large-scale high-quality annotated datasets. However, inaccurate annotations are often inevitable when building hand-labeled datasets, which can lead to imperfectly supervision problems. Most existing methods to address these problems mainly use noisy labels to learn latent representations, which may be corrupted when the noise rate is high, thus limiting the predictions for clean test data set. In this paper, we propose a flexible coarse-to-fine representation learning framework to improve the quality of latent representations by exploiting the reliable geometry information from raw features. This framework includes: (i) an anchored representation learner, which is a coarse mechanism for learning reasonable feature-similarity to avoid feature corruption, (ii) a confident representation learner, which is used to learn confident data pairs, and (iii) an exploratory representation learner, which is proposed for learning from feasible unconfident pairs and performing label refurbishment. Comprehensive experiments have demonstrated the effectiveness of our proposed method, especially when the noise level is high and the task is relatively difficult.

The Minkowski log-ratio (MLR) distance admits closed-form formula for mixture model of exponential families (Gaussian family, Wishart family, etc.), and is suitable for measuring the dissimilarity of polarimetric SAR (PolSAR) data. In this paper, MLR distance is introduced for PolSAR image change detection. Specifically, the PolSAR images are estimated by Wishart mixture models and over-segmented into superpixels first. After that, the statistical distribution differences between two corresponding superpixels are measured by MLR distance. Finally, the change detection map is obtained by the Kittler-Illingworth thresholding method based on the generated difference map. Qualitative and quantitative experimental analysis shows the MLR distance can provide a desirable difference map which facilitates the following thresholding stage.

Unsupervised classification plays an important role in understanding polarimetric synthetic aperture radar (PolSAR) images. One of the typical representations of PolSAR data is in the form of Hermitian positive definite (HPD) covariance matrices. Most algorithms for unsupervised classification using this representation either use statistical distribution models or adopt polarimetric target decompositions. In this paper, we propose an unsupervised classification method by introducing a sparsity-based similarity measure on HPD matrices. Specifically, we first use a novel Riemannian sparse coding scheme for representing each HPD covariance matrix as sparse linear combinations of other HPD matrices, where the sparse reconstruction loss is defined by the Riemannian geodesic distance between HPD matrices. The coefficient vectors generated by this step reflect the neighborhood structure of HPD matrices embedded in the Euclidean space and hence can be used to define a similarity measure. We apply the scheme for PolSAR data, in which we first oversegment the images into superpixels, followed by representing each superpixel by an HPD matrix. These HPD matrices are then sparse coded, and the resulting sparse coefficient vectors are then clustered by spectral clustering using the neighborhood matrix generated by our similarity measure. The experimental results on different fully PolSAR images demonstrate the superior performance of the proposed classification approach against the state-of-the-art approaches.

Desertification of semi-arid grasslands is a serious problem for economic development and ecological preservation. Using the Hunshandake Sandy Lands as an example, we present an overview of monitoring of land desertification using two different data sources including TM and MODIS data. The driving mechanisms of Hunshandake Sandy Land desertification are also discussed.

Unsupervised classification of synthetic aperture radar (SAR) imagery is an essential step in SAR image interpretation. There is a growing demand for an efficient way to fuse multi-information of SAR imagery. This paper presents an intensity/coherent information fusion algorithm by using region covariance features for unsupervised classification. More precisely, we firstly extract the intensity properties and coherent characteristics from each pixel of SAR imagery, then use the region covariance descriptor to fuse the intensity and coherent features, and finally exploit the K-means algorithm to obtain the final unsupervised classification map. Experimental results on SAR imagery demonstrate the effectiveness of the proposed fusion scheme.

Hermitian positive definite (HPD) covariance matrices form one of the most widely-used data representations in PolSAR applications. However, most of these applications either use statistical distribution models on the PolSAR covariance matrices or polarimetric target decomposition. In this paper, we study HPD matrices for PolSAR image classification in the context of sparse coding. More specifically, the PolSAR HPD matrices are first represented as sparse linear combinations of elements from a dictionary, where each element itself is an HPD matrix and the representation loss is measured by the affine-invariant Riemannian metric. We then introduce a sparsity induced similarity measure between two HPD matrices. Finally, we propose a supervised classification scheme using support vector machines on the Riemannian sparse codes and an unsupervised classification scheme encompassing a sparsity induced similarity measure followed by spectral clustering. The proposed methods are validated on the NASA/JPL AIRSAR fully PolSAR data. The experimental results demonstrate the effectiveness of our methods.

The change detection of polarimetric synthetic aperture radar (PolSAR) images is a longstanding and challenging task, not only because of the speckle issue but also due to the complex texture, which generally appears highly heterogeneous. There are two widely used approaches for the change detection of PolSAR images: one is the post classification comparison algorithm, and the other is the directly unsupervised change detection algorithm. In this paper, we focus on the latter and propose a region-based change detection method for PolSAR images by means of Wishart mixture models (WMMs). The WMMs fit the distribution of PolSAR images with less errors both in the homogeneous and the extremely heterogeneous area. More precisely, two PolSAR images are first segmented into compact local regions using the customized simple-linear-iterative-clustering algorithm, while the WMMs are used to model each local region. To generate a difference map, statistical distribution differences measured by information theoretic divergence are then computed for corresponding local region pairs. The Cauchy-Schwarz divergence is adopted as its analytic expression can be derived for WMMs. Finally, the change detection results are obtained by the Kittler-Illingworth thresholding method with Markov random field-based smoothing. The proposed scheme is tested on different PolSAR data sets. Qualitative and quantitative evaluations show its superior performance comparing to the traditional pixel-level approach.