Realistic scene reconstruction in driving scenarios poses significant challenges due to fast-moving objects. Most existing methods rely on labor-intensive manual labeling of object poses to reconstruct dynamic objects in canonical space and move them based on these poses during rendering. While some approaches attempt to use 3D object trackers to replace manual annotations, the limited generalization of 3D trackers -- caused by the scarcity of large-scale 3D datasets -- results in inferior reconstructions in real-world settings. In contrast, 2D foundation models demonstrate strong generalization capabilities. To eliminate the reliance on 3D trackers and enhance robustness across diverse environments, we propose a stable object tracking module by leveraging associations from 2D deep trackers within a 3D object fusion strategy. We address inevitable tracking errors by further introducing a motion learning strategy in an implicit feature space that autonomously corrects trajectory errors and recovers missed detections. Experimental results on Waymo-NOTR and KITTI show that our method outperforms existing approaches. Our code will be released on https://lolrudy.github.io/No3DTrackSG/.

In this article, we address the problem of federated learning in the presence of stragglers. For this problem, a coded federated learning framework has been proposed, where the central server aggregates gradients received from the non-stragglers and gradient computed from a privacy-preservation global coded dataset to mitigate the negative impact of the stragglers. However, when aggregating these gradients, fixed weights are consistently applied across iterations, neglecting the generation of the global coded dataset and the dynamic nature of the trained model over iterations. This oversight may result in diminished learning performance. To overcome this drawback, we propose a new method named adaptive coded federated learning (ACFL). In ACFL, before the training, each device uploads a local coded dataset with additive noise to the central server to generate a global coded dataset under privacy-preservation requirements. During each iteration of the training, the central server aggregates the gradients received from the non-stragglers and the gradient computed from the global coded dataset, where an adaptive policy for varying the aggregation weights is designed. Under this policy, we optimize the performance in terms of privacy and learning, where the learning performance is analyzed through convergence analysis and the privacy performance in sharing local coded datasets with the server is characterized via mutual information differential privacy. Finally, we perform simulations to demonstrate the superiority of ACFL compared with the baseline methods.

In this paper, we consider the problem of federated learning (FL) with devices that have intermittent connectivity to the central server. For this problem, the concept of semi-decentralized FL has been proposed in the literature. This paradigm allows non-straggler devices to relay the gradients computed by the stragglers to the server, and enables realization of gradient coding (GC) to mitigate the negative impact of the stragglers that fail to communicate directly to the central server. However, for GC in semi-decentralized FL, the communication overhead caused by information transmission among the devices is significant. To overcome this shortcoming, inspired by the existing communication-optimal exact consensus algorithm (CECA), we propose a new communication-efficient semi-decentralized FL method (COFFEE). In each round, the devices exchange information by taking a certain number of steps towards communication-optimal exact consensus, ensuring that each device obtains the average of the gradients computed by both its previous neighbors and itself. Afterwards, the non-stragglers transmit the local average result to the server for global aggregation to update the global model. We analyze the convergence performance and the communication overhead of COFFEE analytically. Building on this, to further enhance learning performance under a specific communication overhead, we propose an enhanced version of COFFEE with an adaptive aggregation rule at the central server, referred to as A-COFFEE, which adjusts to the straggler pattern of the devices over training rounds. Experiments are conducted to verify that the proposed methods outperform the baseline methods.

In this paper, we study the problem of distributed learning (DL) with devices transmitting sign information of the local gradients to the server under communication constraints, where the devices are susceptible to Byzantine attacks. For this problem, a sign-based gradient descent method with majority vote and stochastic 1-bit quantization (Sign-M-stochastic) has been proposed very recently. However, the Byzantine resilience of Sign-M-stochastic is inherently limited, based on the fact that all Byzantine devices and honest devices participate equally in the training process. To overcome this drawback and enhance the resilience to Byzantine attacks, inspired by the audit-based distributed detection systems, we propose a novel DL method with an audit mechanism (DL-AM). In each iteration, the sign information of the local gradients are obtained by the devices from stochastic 1-bit quantization. All devices, partitioned into groups, send the sign information to the server through multiple paths, both directly and via other devices in the same group. This approach provides the server with additional information about the identities of the devices, which enables the server to form the global model update by aggregating the sign information of different devices with varying weights. We analyze the convergence performance of the proposed method from a theoretical perspective. Finally, numerical results demonstrate the superiority of DL-AM over the baseline methods.

Most of existing studies regarding cross-modal content-based remote sensing image retrieval (CM-CBRSIR) focus on reducing/enlarging the Euclidean distances of cross modal (CM) data with the same/different content in a common feature space. The advantages of using Euclidean distance lie in its straightforwardness. However, the Euclidean distances of CM data features are sensitive to the outlier data and may lead to non-robust retrieval performance, particularly in the case of noisy images with low-quality. To address this issue, we propose a robust Hirschfeld–Gebelein–Rényi maximal correlation (HGRMC) augmented algorithm for CM-CBRSIR in this work, named by HAC. In HAC, not only the projected features of CM data in Euclidean distance space but also maximal correlation information of HGRMC are learned during the training phase of the retrieval model, where HGRMC is additionally used to capture the statistical dependency between CM data to enhance the retrieval performance with the strongly noisy input data. In the retrieval phase, we also develop a fusion scheme based on the Dempster-Shafer (DS) evidence theory to combine the superiorities of Euclidean distance and HGRMC correlation criterions. Extensive experimental results demonstrate that our proposed HAC algorithm provides better and more robust retrieval performance in comparison with existing state-of-the-art CM-CBRSIR methods.

Most existing methods for cross-modal content-based remote sensing image retrieval (CM-CBRSIR) have only focused on implementations by optimizing the projected features in a common space under the Euclidean distance criterion. In this work, to better bridge the heterogeneity gap caused by the modality difference, we propose a Hirschfeld–Gebelein–Rényi (HGR) maximal correlation augmented CM-CBRSIR method by utilizing the HGR maximal correlation between different modalities. Except for optimizing the projected features under the Euclidean distance constraints, another feature projection, which carries the information of the HGR maximal correlation, is learned during the training phase. In the retrieval phase, we combine the information learned by the Euclidean distance criterion and HGR maximal correlation based on the Dempster–Shafer (DS) evidence theory. Experimental results show that the proposed method outperforms the existing state-of-the-art methods.1

This paper considers the problem of change detection (CD) with multi-sensor optical remote sensing (RS) images. Copulas are adopted to characterize the dependence structure between the image pair. For this problem, a conditional copula-based CD technique has been proposed in the literature. However, in this technique, it is difficult to select the best copula function in an analytical framework. Resulting copula misspecification may lead to performance degradation. To deal with this problem, we model the CD problem as a binary hypothesis testing problem and propose a new superpixel-level copula-based statistical method (SCOPS) for CD, where an explicit strategy for copula selection is provided for the proposed method. The effectiveness of the copula selection strategy is verified on CD tasks with simulated multi-sensor optical RS images. Experiments on real RS datasets demonstrate the superiority of SCOPS over the state-of-the-art methods.

In this article, we investigate the problem of decentralized federated learning (DFL) in Internet of Things (IoT) systems, where a number of IoT clients train models collectively for a common task without sharing their private training data in the absence of a central server. Most of the existing DFL schemes are composed of two alternating steps, i.e., model updating and model averaging. However, averaging model parameters directly to fuse different models at the local clients suffers from client-drift, especially when the training data are heterogeneous across different clients. This leads to slow convergence and degraded learning performance. As a possible solution, we propose the DFL via a mutual knowledge transfer (Def-KT) algorithm, where local clients fuse models by transferring their learned knowledge to each other. Our experiments on the MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 data sets reveal that the proposed Def-KT algorithm significantly outperforms the baseline DFL methods with model averaging, i.e., Combo and FullAvg, especially when the training data are not independent and identically distributed (non-IID) across different clients.

In this article, we consider the problem of federated learning (FL) with training data that are non independent and identically distributed (non-IID) across the clients. To cope with data heterogeneity, an iterative federated clustering algorithm (IFCA) has been proposed. IFCA partitions the clients into a number of clusters and lets the clients in the same cluster optimize a shared model. However, in IFCA, the clusters are nonoverlapping, which leads to an inefficient utilization of the local information since the knowledge of a client is used by only one cluster during each round. To capture the complex nature of real-world data, soft clustering methods with overlapping clusters have been proposed that attain superior performance over the hard ones. Motivated by this, we propose a new algorithm named FL with soft clustering (FLSC) by combining the strengths of soft clustering and IFCA, where the clients are partitioned into overlapping clusters and the information of each participating client is used by multiple clusters simultaneously during each round. The experimental results show that FLSC achieves better learning performance on the classification tasks on the MNIST and Fashion-MNIST data sets, compared with the state-of-the-art baseline methods, i.e., the global model method and IFCA.

In this article, the problem of federated learning (FL) in the presence of label quality disparity is considered. To address this problem, the federated opportunistic computing for ubiquitous system (FOCUS) has been proposed very recently. In FOCUS, the central server utilizes its accurately labeled benchmark samples to quantify the credibility of different clients by computing the cross-entropy (CE) loss of the locally updated models on the benchmark data set and the CE loss of the global model on the local data sets. However, FOCUS assumes the availability of the accurate labels of the benchmark data set, which is difficult to guarantee under many practical scenarios. To overcome this limitation of FOCUS, we propose a new algorithm named robust federated opportunistic learning (RFOL), which does not require the benchmark samples at the central server to be labeled. In RFOL, the client credibility is evaluated by computing the Kullback–Leibler (KL) divergence among the soft predictions on the benchmark samples of different locally updated models and the CE loss of the global model on the local data sets. The experimental results on several popular data sets reveal that: 1) with an unlabeled benchmark data set at the server, the proposed RFOL algorithm attains almost the same learning performance as FOCUS, which requires an accurately labeled benchmark data set at the server; 2) with an inaccurately labeled benchmark data set, RFOL outperforms FOCUS, which shows that the former is more robust to the inaccurate labels of the benchmark samples; and 3) RFOL outperforms FedAvg, which assigns equal credibility to all the clients.

In this article, we investigate the problem of federated learning (FL) in a communication-constrained environment of the Internet of Things (IoT), where multiple IoT clients train a global model collectively by communicating model updates with a central server instead of sending raw data sets. To ease the communication burden in IoT systems, several approaches have been proposed for the FL tasks, including sparsification methods and data quantization strategies. To overcome the shortcomings of the existing methods, we propose two new FL algorithms based on compressed sensing (CS) referred to as the CS-FL algorithm and the 1-bit CS-FL algorithm, both of which compress the upstream and downstream data while communicating between the clients and the central server. The proposed algorithms improve upon the existing algorithms by letting the clients send analog and 1-bit data, respectively, to the server after compression with a random measurement matrix. Based on that, in CS-FL and 1-bit CS-FL, the clients update the model locally utilizing the result of sparse reconstruction obtained by iterative hard thresholding (IHT) and binary IHT (BIHT), respectively. Experiments conducted on the MNIST and the Fashion-MNIST data sets reveal the superiority of the proposed algorithm over the baseline algorithms, SignSGD with a majority vote, FL based on sparse ternary compression, and FedAvg.

In this paper, we present a new detector for the detection of sparse stochastic signals using 1-bit data in two-level two- degree tree-structured sensor networks (2L-2D TSNs). Related prior work mostly concentrates on parallel sensor networks (PSNs). However, PSNs may sometime become impractical in many applications including the case where some sensors are beyond the communication range of the fusion center (FC). Therefore, we design the proposed detector for 2L-2D TSNs where information is transmitted hierarchically. To satisfy severe resource constraints, each local sensor performs 1-bit quantization before transmission to the FC. The FC fuses the received 1-bit data employing the locally most powerful test (LMPT). It is shown theoretically and numerically that, compared with the LMPT detector with Q sensors that transmit analog measurements in 2L-2D TSNs, the proposed 1-bit LMPT detector that uses quantization thresholds derived in this paper asymptotically requires 1.74Q sensors to compensate for the performance loss induced by local quantization.

In this letter, we consider the problem of distributed detection of stochastic sparse signals in battery-powered sensor networks (SNs). For this problem, an original locally most powerful test (oLMPT) detector has previously been developed, where compressed measurements are collected from all local sensors and then fused at the fusion center (FC) for making the global decision. However, since the sensors always operate on limited energy resources, allowing all the nodes to send their observations to the FC all the time exerts tremendous pressure on their energy consumption and hinders the longevity of the sensors. To solve this problem, we propose a new censoring LMPT (cen-LMPT) detector by combining the strengths of censoring strategy and the oLMPT detector, where sensors are designated to merely send observations deemed informative enough so as to utilize the local energy more efficiently, and the FC still makes the global decision based on LMPT. We analytically derive the relationship between the detection performance and the communication rate for the proposed detector. It is shown that, compared with the oLMPT detector, the proposed cen-LMPT detector with the same number of nodes can achieve almost the same detection performance with significantly lower communication rate and, therefore, much lower local energy consumption. The simulation results verify our theoretical findings.

In this letter, we consider the detection of sparse stochastic signals with sensor networks (SNs), where the fusion center (FC) collects 1-bit data from the local sensors and then performs global detection. For this problem, a newly developed 1-bit locally most powerful test (LMPT) detector requires 3.3Q sensors to asymptotically achieve the same detection performance as the centralized LMPT (cLMPT) detector with Q sensors. This 1-bit LMPT detector is based on 1-bit quantized observations without any additional processing at the local sensors. However, direct quantization of observations is not the most efficient processing strategy at the sensors since it incurs unnecessary information loss. In this letter, we propose an improved-1-bit LMPT (Im-1-bit LMPT) detector that fuses local 1-bit quantized likelihood ratios (LRs) instead of directly quantized local observations. In addition, we design the quantization thresholds at the local sensors to ensure asymptotically optimal detection performance of the proposed detector. It is shown theoretically and numerically that, with the designed quantization thresholds, the proposed Im-1-bit LMPT detector for the detection of sparse signals requires less number of sensor nodes to compensate for the performance loss caused by 1-bit quantization.