Machine unlearning aims to remove the influence of specific training samples from model, motivated by privacy regulations and data revocation requirements.Existing approaches often depend on retain data, which compromises privacy and becomes computationally expensive.To address this challenge, we propose RKLU, a novel retain-data-free unlearning method that fine-tunes models by minimizing the KL divergence between their outputs and a target distribution that suppresses the probabilities of the samples to be forgotten.RKLU achieves near-perfect unlearning with minimal utility loss on diverse vision and text classification benchmarks, offering a privacy-preserving and efficient alternative.

Machine unlearning has garnered significant attention due to its ability to selectively erase knowledge obtained from specific training data samples in an already trained machine learning model. This capability enables data holders to adhere strictly to data protection regulations. However, existing unlearning techniques face practical constraints, often causing performance degradation, demanding brief fine-tuning post unlearning, and requiring significant storage. In response, this paper introduces a novel class of layer-wise partial machine unlearning algorithms that enable selective and controlled erasure of targeted knowledge. Of these, partial amnesiac unlearning integrates layer-wise selective pruning with the state-of-the-art amnesiac unlearning. This method selectively prunes and stores updates made to the model during training, enabling the targeted removal of specific data from the trained model. Other methods assimilates layer-wise partial-updates into label-flipping and optimization-based unlearning, thereby mitigating the adverse effects of specific knowledge deletion on model efficacy. Through a detailed experimental evaluation, we showcase the effectiveness of proposed unlearning methods. Experimental results highlight that the partial amnesiac unlearning not only preserves model efficacy but also eliminates the necessity for brief fine-tuning post unlearning, unlike conventional amnesiac unlearning. Further, employing layer-wise partial updates in label-flipping and optimization-based unlearning techniques demonstrates superiority in preserving model efficacy compared to their naive counterparts.

• This paper aims to enhance the polyp characterization performance within CCE images. Towards this, we introduce a novel AI architecture called ResNet9-KAN, which integrates Kolmogorov-Arnold network into the well-established ResNet9 architecture. • Our proposed ResNet-KAN was evaluated on a new CCE dataset generated using the PillCam Colon 2 system, focusing on a bowel cancer screening population aged 50-74 with a positive fecal immunochemical test. The CCEs were conducted in four outpatient clinics, with subsequent colonoscopies performed in four hospitals in the Region of Southern Denmark: Odense University Hospital, Lillebaelt Hospital, Esbjerg and Grindsted Hospital, and Hospital Sønderjylland. • Our proposed ResNet9-KAN achieved better accuracy, sensitivity, and specificity compared to state-of-the-art AI models such as ResNet101 and Google’s ViT-B-16, while requiring fewer computational resources, paving the way for more accurate and reliable diagnostic practices in clinical settings. Background and Aim: Colon capsule endoscopy (CCE) offers a minimally invasive method for imaging gastrointestinal lesions, including colorectal polyps, which may be precursors to colorectal cancer. However, its low image quality poses challenges for tasks such as polyp characterization. This work develops a low-complexity AI model, ResNet9-KAN, by integrating the Kolmogorov-Arnold network (KAN) into 9-layer residual network (ResNet9) architecture. This model efficiently characterizes polyps as neoplastic or non-neoplastic in CCE images, facilitating real-time patient management. Methods: This work utilized a CCE dataset generated from the PillCam Colon 2 system at four hospitals in the Region of Southern Denmark. It comprises 2089 CCE images of 479 polyps (317 neoplastic, 162 non-neoplastic) from a bowel cancer screening population aged 50 to 74. The proposed ResNet9-KAN and several existing AI models were trained on 1672 CCE images (221 neoplastic, 113 non-neoplastic polyps) and evaluated on 569 test images (48 neoplastic, 25 non-neoplastic polyps). Results: The evaluation revealed that our proposed ResNet9-KAN surpassed existing AI models with per-image characterization accuracy of 97.71 %, demonstrating an excellent balance between sensitivity (97.10 %) and specificity (98.17 %). It also achieved the highest F1 score of 0.9730 and a competitive area under the curve (AUC) of 0.9895. Additionally, ResNet9-KAN exhibited per-polyp characterization accuracy of 99.23 %, with a sensitivity of 99.85 %, specificity of 98.65 %, and an F1 score of 0.9912. Conclusions: This work highlights the efficacy of ResNet9-KAN in accurately characterizing polyps in low-quality CCE images, showing substantial potential for in situ characterization where histological verification currently requires a follow-up colonoscopy.

We investigate the robustness of the recently introduced partialsharing online federated learning (PSO-Fed) algorithm against model-poisoning attacks. To this end, we analyze the performance of the PSO-Fed algorithm in the presence of Byzantine clients, who may clandestinely corrupt their local models with additive noise before sharing them with the server. PSO-Fed can operate on streaming data and reduce the communication load by allowing each client to exchange parts of its model with the server. Our analysis, considering a linear regression task, reveals that the convergence of PSO-Fed can be ensured in the mean sense, even when confronted with model-poisoning attacks. Our extensive numerical results support our claim and demonstrate that PSO-Fed can mitigate Byzantine attacks more effectively compared with its state-of-the-art competitors. Our simulation results also reveal that, when model-poisoning attacks are present, there exists a non-trivial optimal stepsize for PSO-Fed that minimizes its steady-state mean-square error.

Distributed and collaborative machine learning over emerging Internet of Things (IoT) networks is complicated by resource constraints, device, and data heterogeneity, and the need for personalized models that cater to the individual needs of each network device. This complexity becomes even more pronounced when new devices are added to a system that must rapidly adapt to personalized models. Along these lines, we propose a networked federated meta-learning (NF-ML) algorithm that utilizes meta-learning and underlying shared structures across the network to enable fast and personalized model adaptation of newly added network devices. The NF-ML algorithm learns two sets of model parameters for each device in a distributed manner, with devices communicating only with their immediate neighbors. One set of parameters is personalized for the device-specific task, whereas the other is a generic parameter set learned via peer-to-peer communication. The performance of the proposed NF-ML algorithm was validated using both synthetic and real-world data, and the results show that it adapts to new tasks in just a few epochs, using as little as 10% of the available data, significantly outperforming traditional federated learning methods.

Autoencoders (AEs) have attracted significant attention for hyperspectral anomaly detection (HAD) in remote sensing applications due to their ability to unveil small, unique objects scattered across large geographical regions in an unsupervised manner. However, the training and inference processes of AEs are computationally demanding, posing challenges for efficient HAD in resource-constrained onboard applications. Various optimization techniques and parallel computing approaches have been proposed to alleviate the computational burden and enhance the feasibility of AEs for real-time applications in HAD. In this paper, we first present an efficient lightweight autonomous autoencoder (LAutoAE) that addresses the computational challenges of the autonomous hyperspectral anomaly detection autoencoder (AUTO-AD) while maintaining a similar anomaly detection accuracy. To further enhance the accuracy, we introduce LAutoAE+, which integrates kernel principal component analysis (KPCA) based pre-processing methods with the LAutoAE. Experiments on diverse datasets demonstrate that the proposed LAutoAE and LAutoAE+ achieve comparable or superior detection performance compared with conventional Auto-AD, while also achieving reductions of 87% and 89.4%, respectively, in the number of learnable parameters.

Nonlinear adaptive filters (NAFs) exhibit superior modeling capabilities compared to conventional linear adaptive filters, especially in practical applications involving nonlinear input-output relationships. The functional link adaptive filter (FLAF) is an NAF that uses nonlinear functional expansions to achieve nonlinear modelling, however, at the expense of high computational complexity. In response, a low-complexity Hammerstein-type block-oriented functional link adaptive filter (HBO-FLAF) was recently developed, which requires less computation than that of the traditional FLAF. To shed more light on its behaviour and design, we provide a steady-state theoretical analysis of the HBO-FLAF in this paper. We derive the conditions for steady-state mean and mean square convergence of the weight update equations, specifically, an upper bound on the step-size parameter, an expression for the steady-state excess mean square error (EMSE) and a lower bound on the steady-state EMSE of the HBO-FLAF. Numerical simulation results show a close relation with the derived results, thus validating the theoretical analysis.

3D Network-on-Chip (NoC) technology has emerged as a compelling solution in modern System-on-Chip (SoC) designs. This NoC technology effectively addresses the escalating need for high-performance and energy-efficient on-chip communication in various applications, including High-Performance Computing (HPC), Graphics Processing Units (GPUs), and Multi-Processor SoCs (MPSoCs). However, the efficient mapping of applications onto 3D NoCs remains a complex challenge, necessitating the development of improved algorithms to address the issue. In this context, we present a novel neural mapping model with a reinforcement learning (RL) approach (NeurMap3D) to design application-specific 3D NoC-based IC. Additionally, we propose the NCTPAM (neural congestion-aware Through-Silicon Vias (TSVs) placement and application mapping) approach, which not only addresses application mapping but also incorporates TSVs placement and load balance across the TSVs for the specific application. In order to reduce the CPU execution time of NCTPAM algorithm, we propose incorporating a partial model parameter (θ) update mechanism. Experimental results indicate improved performance in terms of minimizing communication cost, load balancing across TSVs and energy consumption, highlighting the potential of our approach to enhance the efficiency of these synthesized network architectures.

Personalized federated learning enables every edge device or group of edge devices within the distributed network to learn a device- or cluster-specific model tailored to their local needs. Data scarcity, however, makes it difficult to learn such individual models, resulting in performance degradation. Since the device- or cluster-specific tasks are distinct but often related, leveraging these similarities through inter-cluster learning alleviates data shortage and enhances learning performance. Although inter-cluster learning can boost performance, uncontrolled intercluster learning may lead to performance degradation due to over- or under-usage of local similarity enforcement. In light of this issue, an intelligent mechanism that performs inter-cluster learning based on device-specific needs is required. To this end, this paper proposes adopting reinforcement learning principles to control device-specific inter-cluster learning in real-time. We propose networked personalized federated learning using reinforcement learning (NPFed-RL) as a general framework and then demonstrate its feasibility by applying it to the ridge regression problem. We conduct numerical experiments to compare the proposed method with the state-of-the-art. The proposed method successfully controls device-specific parameters and offers better learning performance than existing solutions.

Online federated learning (FL) enables geographically distributed devices to learn a global shared model from locally available streaming data. Most online FL literature considers a best case scenario regarding the participating clients and the communication channels. However, these assumptions are often not met in real-world applications. Asynchronous settings can reflect a more realistic environment, such as heterogeneous client participation due to available computational power and battery constraints, as well as delays caused by communication channels or straggler devices. Further, in most applications, energy efficiency must be taken into consideration. Using the principles of partial-sharing-based communications, we propose a communication-efficient asynchronous online FL (PAO-Fed) strategy. By reducing the communication load of the participants, the proposed method renders participation more accessible and efficient. In addition, the proposed aggregation mechanism accounts for random participation, handles delayed updates, and mitigates their effect on accuracy. We study the first- and second-order convergence of the proposed PAO-Fed method and obtain an expression for its steady-state mean square deviation. Finally, we conduct comprehensive simulations to study the performance of the proposed method on both synthetic and real-life data sets. The simulations reveal that in asynchronous settings, the proposed PAO-Fed is able to achieve the same convergence properties as that of the online federated stochastic gradient while reducing the communication by 98%.

This article presents communication-efficient approaches to federated learning (FL) for resource-constrained devices with access to streaming data. In particular, we first propose a partial-sharing-based framework for online federated learning (PSO-Fed), wherein clients update local models from a stream of data and exchange tiny fractions of the model with the server, reducing the communication overhead. In contrast to classical FL approaches, the proposed strategy provides clients who are not part of a global iteration with the freedom to update local models whenever new data arrives. Furthermore, by devising a client-side innovation check, we also propose an event-triggered PSO-Fed (ETPSO-Fed) that further reduces the computational burden of clients while enhancing communication efficiency. We implement the above-mentioned frameworks in the context of kernel regression, where clients perform local learning employing random Fourier features (RFFs)-based kernel least mean squares. In addition, we examine the mean and mean-square convergence of the proposed PSO-Fed. Finally, we conduct experiments to determine the efficacy of the proposed frameworks. Our results show that PSO-Fed and ETPSO-Fed can compete with Online-Fed while requiring significantly less communication overhead. Simulations demonstrate an 80% reduction in PSO-Fed and an 84.5% reduction in ETPSO-Fed communication overhead compared to Online-Fed. Notably, the proposed PSO-Fed strategies show good resilience against model-poisoning attacks without involving additional mechanisms.

Numerous real-life systems exhibit complex nonlinear input-output relationships. Kernel adaptive filters, a popular class of nonlinear adaptive filters, can efficiently model these nonlinear input-output relationships. Their growing network structure, however, poses considerable challenges in terms of their hardware implementation, making them inefficient for real-time applications. Random Fourier features (RFF) facilitate the development of kernel adaptive filters with a fixed network structure. For the first time, this paper attempts to implement the RFF-based kernel least mean square (RFF-KLMS) algorithm on hardware. To this end, we propose several reformulations of the feature functions (FFs) that are computationally expensive in their native form so that they can be implemented in real-time VLSI. Specifically, we reformulate inner product evaluation, cosine, and exponential functions that appear in the implementation of FFs. With these reformulations, the proposed delayed RFF-KLMS (DRFF-KLMS) is then synthesized using 45-nm CMOS technology with 16-bit fixed-point representations. According to the synthesis results, pipelined DRFF-KLMS architectures require minimal hardware increase over the state-of-the-art conventional delayed LMS architecture while significantly improving estimation performance for the nonlinear model. Our results suggest that the cosine feature function-based DRFF-KLMS is appropriate for applications requiring high accuracy, whereas the exponential function-based DRFF-KLMS may be well suited for resource-constrained applications.

Federated learning (FL) literature typically assumes that each client has a fixed amount of data, which is unrealistic in many practical applications. Some recent works introduced a framework for online FL (Online-Fed) wherein clients perform model learning on streaming data and communicate the model to the server; however, they do not address the associated communication overhead. As a solution, this paper presents a partial-sharing-based online federated learning framework (PSO-Fed) that enables clients to update their local models using continuous streaming data and share only portions of those updated models with the server. During a global iteration of PSO-Fed, non-participant clients have the privilege to update their local models with new data. Here, we consider a global task of kernel regression, where clients use a random Fourier features-based kernel LMS on their data for local learning. We examine the mean convergence of the PSO-Fed for kernel regression. Experimental results show that PSO-Fed can achieve competitive performance with a significantly lower communication overhead than Online-Fed.

Many assumptions in the federated learning literature present a best-case scenario that can not be satisfied in most real-world applications. An asynchronous setting reflects the realistic environment in which federated learning methods must be able to operate reliably. Besides varying amounts of non-IID data at participants, the asynchronous setting models heterogeneous client participation due to available computational power and battery constraints and also accounts for delayed communications between clients and the server. To reduce the communication overhead associated with asynchronous online federated learning (ASO-Fed), we use the principles of partial-sharing-based communication. In this manner, we reduce the communication load of the participants and, therefore, render participation in the learning task more accessible. We prove the convergence of the proposed ASO-Fed and provide simulations to analyze its behavior further. The simulations reveal that, in the asynchronous setting, it is possible to achieve the same convergence as the federated stochastic gradient (Online-FedSGD) while reducing the communication tenfold.

Conventional adaptive filters, which assume Gaussian distribution for signal and noise, exhibit significant performance degradation when operating in non-Gaussian environments. Recently proposed fractional-order adaptive filters (FoAFs) address this concern by assuming that the signal and noise are symmetric$\alpha $-stable random processes. However, the literature does not include any VLSI architectures for these algorithms. Toward that end, this article develops hardware-efficient architecture for fractional-order correntropy adaptive filter (FoCAF). We first reformulate the FoCAF for its efficient real-time VLSI implementation and then demonstrate that these reformulations cause negligible performance degradation under the 16-bit fixed-point implementation. Using this reformulated algorithm, we design an FoCAF architecture. Furthermore, we analyze the critical path of the design to select the appropriate level of pipelining based on the sampling rate of the application. According to the critical-path analysis, the FoCAF design is pipelined using retiming techniques to obtain delayed FoCAF (DFoCAF), which is then synthesized using$\mathbf {45}$-nm CMOS technology. Synthesis results reveal that DFoCAF architecture requires a minimal increase in hardware over the prominent least mean square (LMS) filter architecture and achieves a significant increase in the performance in symmetric$\alpha $-stable environments where LMS fails to converge.

This work proposes an efficient batch-based implementation for kernel regression on graphs (KRG) using random Fourier features (RFF) and a low-complexity online implementation. Kernel regression has proven to be an efficient learning tool in the graph signal processing framework. However, it suffers from poor scalability inherent to kernel methods. We employ RFF to overcome this issue and derive a batch-based KRG whose model size is independent of the training sample size. We then combine it with a stochastic gradient-descent approach to propose an online algorithm for KRG, namely the stochastic-gradient KRG (SGKRG). We also derive sufficient conditions for convergence in the mean sense of the online algorithms. We validate the performance of the proposed algorithms through numerical experiments using both synthesized and real data. Results show that the proposed batch-based implementation can match the performance of conventional KRG while having reduced complexity. Moreover, the online implementations effectively learn the target model and achieve competitive performance compared to the batch implementations.

This work revisits the problem of distributed adaptive filtering in multi-agent sensor networks. In contrast to classical approaches, the formulation relaxes the Gaussian assumption on the signal and noise to the generalized setting of α-stable distributions that do not possess second- and higher-order statistical moments. Most importantly, the considered scenario allows for different characteristic exponents throughout the network. Drawing upon ideas from correntropy-type local similarity measures and fractional-order calculus, a novel class of distributed fractional-order correntropy adaptive filters, that are robust against the jittery behavior of α-stable signals, is derived and their convergence criterion is established. The effectiveness of the proposed algorithms, as compared to existing distributed adaptive filtering techniques, is demonstrated via simulation examples.

Multitask diffusion strategies are useful to estimate node-specific, or, multiple parameter vectors over a distributed network by exploiting inter-cluster and intra-cluster cooperation. During cooperation, all nodes transmit their intermediate estimates to their neighboring nodes, resulting in high energy consumption. In this paper, we propose a clustered multitask diffusion affine projection algorithm by transmitting only a subset of the entries of the intermediate estimate vectors among the neighboring nodes. The proposed algorithm, namely, clustered multitask partial diffusion affine projection algorithm provides a trade-off between the estimation performance and the required communication cost. Important results on convergence (in mean and mean square) of the proposed strategy are presented. Numerical simulations reveal that even though the estimation performance deteriorates somewhat as the number of coefficients transmitted to the neighboring nodes decreases, the degradation can be compensated to a large extent by a proportional increase in the magnitude of the regularization strength among the clusters.

This brief proposes robust adaptive filtering algorithms and their VLSI architectures for sparse system identification under impulsive noise. Several robust algorithms are derived by combining error nonlinear adaptive filtering algorithms with proportionate adaptation. We make a comparative study of the derived algorithms and their VLSI architectures in terms of convergence rate and hardware complexity to show that the hardware overhead is negligible for the achieved improvement in robustness.

This paper presents a framework based on the logarithmic number system to implement adaptive filters with error nonlinearities in hardware. The framework is demonstrated through pipelined implementations of two recently proposed adaptive filtering algorithms based on logarithmic cost, namely, least mean logarithmic square (LMLS) and least logarithmic absolute difference (LLAD). To the best of our knowledge, the proposed architectures are the first attempts to implement both LMLS and LLAD algorithms in hardware. We derive error computing algorithms to realize the nonlinear error functions for LMLS and LLAD and map them onto hardware. We also propose a novel variable-α scheme to enhance the original LMLS algorithm and prove its robustness and suitability for VLSI implementations in practical applications. Detailed bit width and error analysis are carried out for the proposed VLSI fixed point implementations. Postlayout implementation results show that with an additional multiplier over conventional least mean square (LMS), 7-dB improvement in steady-state mean square deviation performance can be achieved and with the proposed variable-α scheme, 12-dB improvement can be achieved without compromising the convergence. We will show that LMLS can potentially replace LMS in practical applications, by demonstrating a proof-of-concept by extending the framework to transform domain adaptive filters.