Model pruning has gained traction as a promising defense strategy against backdoor attacks in deep learning. However, existing pruning-based approaches often fall short in accurately identifying and removing the specific parameters responsible for inducing backdoor behaviors. Despite the dominance of fine-tuning-based defenses in recent literature, largely due to their superior performance, pruning remains a compelling alternative, offering greater interpretability and improved robustness in low-data regimes. In this paper, we propose a novel pruning approach featuring a learned \emph{selection} mechanism to identify parameters critical to both main and backdoor tasks, along with an \emph{invertible} pruning mask designed to simultaneously achieve two complementary goals: eliminating the backdoor task while preserving it through the inverse mask. We formulate this as a bi-level optimization problem that jointly learns selection variables, a sparse invertible mask, and sample-specific backdoor perturbations derived from clean data. The inner problem synthesizes candidate triggers using the inverse mask, while the outer problem refines the mask to suppress backdoor behavior without impairing clean-task accuracy. Extensive experiments demonstrate that our approach outperforms existing pruning-based backdoor mitigation approaches, maintains strong performance under limited data conditions, and achieves competitive results compared to state-of-the-art fine-tuning approaches. Notably, the proposed approach is particularly effective in restoring correct predictions for compromised samples after successful backdoor mitigation.

In this paper, given a user's query set and budget, we aim to use the limited budget to help users assemble a set of datasets that can enrich a base dataset by introducing the maximum number of distinct tuples (i.e., maximizing distinctiveness). We prove this problem to be NP-hard. A greedy algorithm using exact distinctiveness computation attains an approximation ratio of (1-e-1 )/2, but it lacks efficiency and scalability due to its frequent computation of the exact distinctiveness marginal gain of any candidate dataset for selection. This requires scanning through every tuple in candidate datasets and thus is unaffordable in practice. To overcome this limitation, we propose an efficient machine learning (ML)-based method for estimating the distinctiveness marginal gain of any candidate dataset. This effectively eliminates the need to test each tuple individually. Estimating the distinctiveness marginal gain of a dataset involves estimating the number of distinct tuples in the tuple sets returned by each query in a query set across multiple datasets. This can be viewed as the cardinality estimation for a query set on a set of datasets, and the proposed method is the first to tackle this cardinality estimation problem. This is a significant advancement over prior methods that were limited to single-query cardinality estimation on a single dataset and struggled with identifying overlaps among tuple sets returned by each query in a query set across multiple datasets. Extensive experiments using five real-world data pools demonstrate that our algorithm, which utilizes ML-based distinctiveness estimation, outperforms all relevant baselines in effectiveness, efficiency, and scalability. A case study on two downstream ML tasks also highlights its potential to find datasets with more useful tuples to enhance the performance of ML tasks.

We investigate the robustness of the recently introduced partialsharing online federated learning (PSO-Fed) algorithm against model-poisoning attacks. To this end, we analyze the performance of the PSO-Fed algorithm in the presence of Byzantine clients, who may clandestinely corrupt their local models with additive noise before sharing them with the server. PSO-Fed can operate on streaming data and reduce the communication load by allowing each client to exchange parts of its model with the server. Our analysis, considering a linear regression task, reveals that the convergence of PSO-Fed can be ensured in the mean sense, even when confronted with model-poisoning attacks. Our extensive numerical results support our claim and demonstrate that PSO-Fed can mitigate Byzantine attacks more effectively compared with its state-of-the-art competitors. Our simulation results also reveal that, when model-poisoning attacks are present, there exists a non-trivial optimal stepsize for PSO-Fed that minimizes its steady-state mean-square error.

In this paper, we study how to acquire labeled data points from a large data pool to enrich a training set for enhancing supervised machine learning (ML) performance. The state-of-the-art solution is the clustering-based training set selection (CTS) algorithm, which initially clusters the data points in a data pool and subsequently selects new data points from clusters. The efficiency of CTS is constrained by its frequent retraining of the target ML model, and the effectiveness is limited by the selection criteria, which represent the state of data points within each cluster and impose a restriction of selecting only one cluster in each iteration. To overcome these limitations, we propose a new algorithm, called CTS with incremental estimation of adaptive score (IAS). IAS employs online learning, enabling incremental model updates by using new data, and eliminating the need to fully retrain the target model, and hence improves the efficiency. To enhance the effectiveness of IAS, we introduce adaptive score estimation, which serves as novel selection criteria to identify clusters and select new data points by balancing trade-offs between exploitation and exploration during data acquisition. To further enhance the effectiveness of IAS, we introduce a new adaptive mini-batch selection method that, in each iteration, selects data points from multiple clusters rather than a single cluster, hence eliminating the potential bias due to using only one cluster. By integrating this method into the IAS algorithm, we propose a novel algorithm termed IAS with adaptive mini-batch selection (IAS-AMS). Experimental results highlight the superior effectiveness of IAS-AMS, with IAS also outperforming other competing algorithms. In terms of efficiency, IAS takes the lead, while the efficiency of IAS-AMS is on par with that of the existing CTS algorithm.

Object classification using LiDAR 3D point cloud data is critical for modern applications such as autonomous driving. However, labeling point cloud data is labor-intensive as it requires human annotators to visualize and inspect the 3D data from different perspectives. In this paper, we propose a semi-supervised cross-domain learning approach that does not rely on manual annotations of point clouds and performs similar to fully-supervised approaches. We utilize available 3D object models to train classifiers that can generalize to real-world point clouds. We simulate the acquisition of point clouds by sampling 3D object models from multiple viewpoints and with arbitrary partial occlusions. We then augment the resulting set of point clouds through random rotations and adding Gaussian noise to better emulate the real-world scenarios. We then train point cloud encoding models on the synthesized and augmented datasets and evaluate their cross-domain classification performance on corresponding real-world datasets. We also introduce PointSyn2Real, a new benchmark dataset for cross-domain learning on point clouds. The results of our extensive experiments with this dataset demonstrate that the proposed cross-domain learning approach for point clouds outperforms the related baseline and state-of-the-art approaches in both indoor and outdoor settings in terms of cross-domain generalizability.1

Monitoring of physiology and behavior of marine animals living undisturbed in their natural habitats can provide valuable information about their well-being and response to environmental stressors. We focus on detecting the feeding behavior in predatory fish using implantable biologgers that record and analyze electrocardiogram (ECG) signals. We propose a novel processing pipeline for resource-constrained embedded systems that can infer higher-level information, such as heart-rate and feeding events, from the ECG signals in situ. Our main contributions are in proposing efficient event detection algorithms that can reliably detect fish feeding events from noisy heart-rate data based on the unique statistical properties of feeding-induced changes in the heart-rate. We evaluate our approaches using an in-house biologger that we surgically implant in twelve coral trout fish and use to collect data during an experiment for a period of ten weeks and show that our signal processing pipeline performs well with noisy ECG signals overall. Specifically, our heart-rate estimation algorithm achieves errors of less than one beat per minute even in scenarios where popular algorithms used by domain specialists perform poorly. Furthermore, our feeding detection algorithms offer improved accuracy compared with the state-of-the-art algorithms while requiring significantly reduced computational and energy resources. We implement the proposed heart-rate estimation and feeding detection algorithms on the biologger and evaluate the associated system overhead. The results show that our proposed heart-rate estimation and feeding detection algorithms can run in-situ on the biologger as they demand rather small computational and energy resources that can conveniently be provisioned. This work is an important first step towards developing effective tools for long-term monitoring of high-level parameters pertaining to the health and behavior of marine animals in the wild.

In this work, we study how to find the k most representative routes over large scale trajectory data, which is a fundamental operation that benefits various real-world applications, such as traffic monitoring and public transportation planning. The operator is time-sensitive as it must be able to adapt the results as traffic conditions change. We first prove the NP-hardness of the problem, and then propose a range of effective approximate solutions that have rapid response times. Specifically, we first build a lookup table that stores the trajectories covered by each edge in a given road network. Rather than performing a depth-first search for all possible routes, we find a 1/η approximate solution by developing a maximum-weight algorithm. Since each edge in a route may be close to several trajectories, we further propose a coverage-first algorithm to locate the edges with the greatest coverage gain in the solution route set. By observing that in the real world each edge is connected to only a few other edges in a road network, we have developed a connect-first algorithm that finds consecutive edges for k representative routes by greedily selecting edges with the maximum marginal gain for each route. Finally, comprehensive experiments over two real-world datasets are conducted to verify the effectiveness and efficiency of our proposed algorithms, and provide evidence of the usefulness of our solution and rapid response times in traffic monitoring tasks.

We describe a multi-modal sensing system for reliable recognition of animal behaviour that can operate in varying environmental conditions. We present the system architecture including the utilised sensors, visualise some intermediate processes, and provide a quantitative performance evaluation in one of the target use cases.

We develop a privacy-preserving distributed algorithm to minimize a regularized empirical risk function when the first-order information is not available and data is distributed over a multi-agent network. We employ a zeroth-order method to minimize the associated augmented Lagrangian function in the primal domain using the alternating direction method of multipliers (ADMM). We show that the proposed algorithm, named distributed zeroth-order ADMM (D-ZOA), has intrinsic privacy-preserving properties. Most existing privacy-preserving distributed optimization/estimation algorithms exploit some perturbation mechanism to preserve privacy, which comes at the cost of reduced accuracy. Contrarily, by analyzing the inherent randomness due to the use of a zeroth-order method, we show that D-ZOA is intrinsically endowed with$(\epsilon,\delta)-$differential privacy. In addition, we employ the moments accountant method to show that the total privacy leakage of D-ZOA grows sublinearly with the number of ADMM iterations. D-ZOA outperforms the existing differentially-private approaches in terms of accuracy while yielding similar privacy guarantee. We prove that D-ZOA reaches a neighborhood of the optimal solution whose size depends on the privacy parameter. The convergence analysis also reveals a practically important trade-off between privacy and accuracy. Simulation results verify the desirable privacy-preserving properties of D-ZOA and its superiority over the state-of-the-art algorithms as well as its network-wide convergence.

Monitoring of physiology and behavior of marine animals living undisturbed in their natural habitats can provide valuable data on their well-being and response to environmental stressors. We focus on detection of feeding of predatory fish using implantable biologgers that record electrocardiogram (ECG) signals. We propose a novel processing pipeline for resource-constrained embedded systems that can infer higher-level information, such as heart-rate and feeding events, from the ECG signals. Our main contribution is a lightweight change-detection algorithm, that can reliably detect fish feeding in noisy heart-rate data based on unique statistical properties of feeding-induced changes in heart-rate. We evaluate our approach using an in-house biologger that we surgically implant in twelve coral trouts over a period of ten weeks. We show that our signal processing pipeline performs well with noisy ECG signals overall. Specifically, our heart-rate estimation algorithm achieves errors of less than one beat per minute even in scenarios where popular algorithms used by domain scientists perform poorly. Furthermore, our feeding detection algorithm achieves good accuracy and matches the performance of state-of-the-art algorithms while requiring significantly less memory and computational resources. This work is an important first step towards long-term monitoring of high-level condition and health of marine animals in the wild.

We propose a new distributed algorithm to solve the total least-squares (TLS) problem when data are distributed over a multi-agent network. To develop the proposed algorithm, named distributed ADMM TLS (DA-TLS), we reformulate the TLS problem as a parametric semidefinite program and solve it using the alternating direction method of multipliers (ADMM). Unlike the existing consensus-based approaches to distributed TLS estimation, DA-TLS does not require careful tuning of any design parameter. Numerical experiments demonstrate that the DA-TLS converges to the centralized solution significantly faster than the existing consensus-based TLS algorithms.

We consider the problem of supervised spectral unmixing with a fully-perturbed linear mixture model where the given endmembers, as well as the observations of the spectral image, are subject to perturbation due to noise, error, or model mismatch. We calculate the Fisher information matrix and the Cramer-Rao lower bound associated with the estimation of the abundance matrix in the considered fully-perturbed linear spectral unmixing problem. We develop an algorithm for estimating the abundance matrix by minimizing a constrained and regularized maximum-log-likelihood objective function using the block coordinate-descend iterations and the alternating direction method of multipliers. We analyze the convergence of the proposed algorithm theoretically and perform simulations with real hyperspectral image data sets to evaluate its performance. The simulation results corroborate the efficacy of the proposed algorithm in mitigating the adverse effects of perturbation in the endmembers.

We consider fusing an arbitrary number of multiband, i.e., panchromatic, multispectral, or hyperspectral, images of the same scene. Using the well-known forward observation and linear mixture models, a vector total-variation penalty, and appropriate constraints, we cast this problem as a convex optimization problem. The total-variation penalty helps cope with the potential ill-posedness of the underlying inverse problem by exploiting the prior knowledge that natural images are mostly piecewise smooth in the spatial domain and comprise relatively few archetypical signatures, i.e., endmembers, in the spectral domain. We solve the formulated convex but non-smooth optimization problem using the alternating direction method of multipliers. Our experiments with multiband images constructed from real hyperspectral datasets demonstrate the performance advantages of the proposed algorithm over the existing algorithms, which need to be used in tandem to fuse more than two multiband images.

Indoor localization using radio signals is challenging. A recently proposed algorithm based on WiFi channel state information is an effective solution. However, it relies on a computationally expensive grid search. We propose a new algorithm based on a modified matrix pencil method that reduces the computational complexity by two orders of magnitude without any loss of accuracy.

We consider the problem of self-localization by a resource-constrained mobile node given perturbed anchor position information and distance estimates from the anchor nodes. We consider normally-distributed noise in anchor position information. The distance estimates are based on the log-normal shadowing path-loss model for the RSSI measurements. The available solutions to this problem are based on complex and iterative optimization techniques such as semidefinite programming or second-order cone programming, which are not suitable for resource-constrained environments. In this paper, we propose a closed-form weighted least-squares solution. We calculate the weights by taking into account the statistical properties of the perturbations in both RSSI and anchor position information. We also estimate the bias of the proposed solution and subtract it from the proposed solution. We evaluate the performance of the proposed algorithm considering a set of arbitrary network topologies in comparison to an existing algorithm that is based on a similar approach but only accounts for perturbations in the RSSI measurements. We also compare the results with the corresponding Cramer-Rao lower bound. Our experimental evaluation shows that the proposed algorithm can substantially improve the localization performance in terms of both root mean square error and bias.

Natural images tend to mostly consist of smooth regions with individual pixels having highly correlated spectra. This information can be exploited to recover hyperspectral images of natural scenes from their incomplete and noisy measurements. To perform the recovery while taking full advantage of the prior knowledge, we formulate a composite cost function containing a square-error data-fitting term and two distinct regularization terms pertaining to spatial and spectral domains. The regularization for the spatial domain is the sum of total variation of the image frames corresponding to all spectral bands. The regularization for the spectral domain is the ℓ1-norm of the coefficient matrix obtained by applying a suitable sparsifying transform to the spectra of the pixels. We use an accelerated proximal-subgradient method to minimize the formulated cost function. We analyze the performance of the proposed algorithm and prove its convergence. Numerical simulations using real hyperspectral images exhibit that the proposed algorithm offers an excellent recovery performance with a number of measurements that is only a small fraction of the hyperspectral image data size. Simulation results also show that the proposed algorithm significantly outperforms an accelerated proximal-gradient algorithm that solves the classical basis-pursuit denoising problem to recover the hyperspectral image.

We develop a fully-distributed iterative algorithm for finding a model-distributed least-squares solution of systems of linear equations over sensor networks. Here, model-distributed means the solution vector is distributed across the network rather than being replicated at each node. For this purpose, we devise a dual regularized least-squares problem via a suitable decomposition of the normal equations associated with the original problem. The resultant dual problem can be solved in a fully-decentralized and iterative manner by means of the diffusion-based Pareto optimization strategy. We verify the usefulness of the proposed algorithm via both theoretical analysis and numerical examples.

Acoustic echo cancellation and feedback cancellation systems require robust and computationally efficient adaptive filtering techniques. In this paper, a new affine projection algorithm with intermittent update of the filter coefficients is proposed where the update interval is determined according to the adaptation state. Simulation results show that the proposed algorithm provides improved performance and reduced average computational complexity compared with other similar algorithms for acoustic echo cancellation and acoustic feedback cancellation applications.

We examine the problem of estimating the frequency of a three-phase power system in an adaptive and low-cost manner when the voltage readings are contaminated with observational error and noise. We assume a widely-linear predictive model for the αβ complex signal of the system that is given by Clarke's transform. The system frequency is estimated using the parameters of this model. In order to estimate the model parameters while compensating for noise in both input and output of the model, we utilize the notions of total least-squares fitting and gradient-descent optimization. The outcome is an augmented gradient-descent total least-squares (AGDTLS) algorithm that has a computational complexity comparable to that of the complex least mean square (CLMS) and the augmented CLMS (ACLMS) algorithms. Simulation results demonstrate that the proposed algorithm provides significantly improved frequency estimation performance compared with CLMS and ACLMS when the measured voltages are noisy and especially in unbalanced systems.

We develop a gradient-descent distributed adaptive estimation strategy that compensates for error in both input and output data. To this end, we utilize the concepts of total least-squares estimation and gradient-descent optimization in conjunction with a recently-proposed framework for diffusion adaptation over networks. The proposed strategy does not require any prior knowledge about the noise variances and has a computational complexity comparable to the diffusion least mean square (DLMS) strategy. Simulation results demonstrate that the proposed strategy provides significantly improved estimation performance compared with the DLMS and bias-compensated DLMS (BC-DLMS) strategies when both the input and output signals are noisy.

A new set-membership adaptive filtering algorithm is developed based on the exponentially-weighted RLS algorithm with a time-varying forgetting factor that is optimized at each iteration by imposing a bounded-magnitude constraint on the a posteriori filter output error. The new algorithm is designed to improve the numerical behavior of the previously proposed BEACON algorithm while delivering the same convergence and tracking performance as the BEACON algorithm. Simulation results for a flat-fading MIMO channel estimation application demonstrate the superiority of the new algorithm over the BEACON algorithm in terms of numerical stability.

Increasing the projection order in the affine projection adaptive filtering algorithm speeds up the convergence but also increases the steady-state misalignment. To address this unfavorable compromise, we propose a new affine projection algorithm with a variable projection order. This algorithm adaptively changes the projection order according to the estimated variance of the filter output error. The error variance is estimated using the exponential window and moving averaging techniques and employing a variable forgetting factor. Simulations demonstrate that the new algorithm provides fast initial convergence and low steady-state misalignment without necessarily trading off one for the other in addition to a significant reduction in average computational complexity.

A new adaptive MIMO channel equalizer is proposed based on adaptive generalized decision-feedback equalization and ordered-successive interference cancelation. The proposed equalizer comprises equal-length subequalizers, enabling any adaptive filtering algorithm to be employed for coefficient updates. A recently proposed computationally-efficient recursive least squares algorithm based on dichotomous coordinate descents is utilized to solve the normal equations associated with the adaptation of the new equalizer. Simulations show that the proposed equalizer is superior to the previously proposed adaptive MIMO channel equalizers by providing both enhanced bit error rate performance and reduced computational complexity. Furthermore, the proposed algorithm exhibits stable numerical behavior and can deliver a trade-off between performance and complexity.

We develop a new linearly-constrained recursive total least squares adaptive filtering algorithm by incorporating the linear constraints into the underlying total least squares problem using an approach similar to the method of weighting and searching for the solution (filter weights) along the input vector. The proposed algorithm outperforms the previously proposed constrained recursive least square (CRLS) algorithm when both input and output data are observed with noise. It also has a significantly smaller computational complexity than CRLS. Simulations demonstrate the efficacy of the proposed algorithm.