Decentralized federated learning (DFL) realizes cooperative model training among connected clients without relying on a central server, thereby mitigating communication bottlenecks and eliminating the single-point failure issue present in centralized federated learning (CFL). Most existing work on DFL focuses on supervised learning, assuming each client possesses sufficient labeled data for local training. However, in real-world applications, much of the data is unlabeled. We address this by considering a challenging yet practical semi-supervised learning (SSL) scenario in DFL, where clients may have varying data sources: some with few labeled samples, some with purely unlabeled data, and others with both. In this work, we propose SemiDFL, the first semi-supervised DFL method that enhances DFL performance in SSL scenarios by establishing a consensus in both data and model spaces. Specifically, we utilize neighborhood information to improve the quality of pseudo-labeling, which is crucial for effectively leveraging unlabelled data. We then design a consensus-based diffusion model to generate synthesized data, which is used in combination with pseudo-labeled data to create mixed datasets. Additionally, we develop an adaptive aggregation method that leverages the model accuracy of synthesized data to further enhance SemiDFL performance. Through extensive experimentation, we demonstrate the remarkable performance superiority of the proposed DFL-Semi method over existing CFL and DFL schemes in both iid and Non-iid SSL scenarios.

D high-density NAND flash has a limited voltage window that narrows page read margins, making it vulnerable to read disturbance and triggering Read Refresh Operations (RROs). Existing methods ignore repeated voltage states caused by invalid pages on the same wordline, failing to fully utilize the voltage range. This paper proposes an Invalid Page-Aware Refresh (IPAR) scheme to achieve high-performance 3D highdensity NAND flash memory. First, IPAR merges repeated voltage states through reprogramming to widen valid page read margins and improve read endurance thresholds. Moreover, IPAR further adopts a page-level allocation strategy that contains two operations: hot-write data is placed on high-latency pages to allow more reprogramming operations, while hot-read data is allocated to low-latency pages of reprogrammed blocks, effectively mitigating read disturbance and reducing migration overhead. Trace-driven simulations based on FEMU show that IPAR reduces RROs triggers by 38.91%, write amplification by 48.08%, and average response time by 16.93% compared to baseline.

Hyperspectral images (HSIs) and multispectral images (MSIs) possess complementary advantages, with HSI providing rich spectral information and MSI offering fine spatial details. Therefore, fusing HSI and MSI to obtain high-resolution hyperspectral images (HR-HSIs) with both high spatial detail and rich spectral information has drawn increased attention. However, convolutional neural network (CNN)-based methods are limited by their local receptive fields, while traditional Transformer architectures suffer from extremely high computational complexity when applied to HSI with numerous spectral bands. To address these issues, we propose a novel multiscale Spatial-Spectral Dilated Transformer (SSDT) network based on the Transformer framework. The proposed method adopts a dual-branch architecture, consisting of the Spatial Dilated MultiScale Transformer (Spa-DMST) and the Spectral Dilated Positional Embedded Transformer (Spe-DPET) modules. Specifically, Spa-MSDT introduces a multi-level dilated window mechanism to enable cross-scale spatial feature extraction, while Spe-PEDT employs position encoding-enhanced dilated attention to reduce computational complexity. To further optimize the feature fusion process, we design a Gating Fusion Reconstruction (GFR) module that integrates depthwise separable convolutions with a gating mechanism, effectively suppressing redundant information and enhancing detail reconstruction capabilities. Extensive experiments and visualized results on three simulated datasets and one real dataset demonstrate that the proposed method outperforms other state-of-the-art fusion methods. The code of this paper will be available at https://github.com/FxyPd/SSDT.

In SplitFed learning (SFL), a global model is split into two segments, where distributed clients train the first segment in a federated manner and a main server trains the other. Existing studies focus on algorithm development but ignore the important issue of incentives, without which self-interested clients may be unwilling to participate. We fill this gap by presenting a first incentive study in SFL. One challenge is that the design requires an understanding of how clients' participation affects the model performance. To this end, we provide a first convergence analysis for SFL considering partial client participation to guide the mechanism design. Another challenge is that monetary payment may not be viable for large distributed systems. To this end, we propose a model-versioning mechanism where the main server assigns different versions of models (of different qualities) to clients as incentives. The design is further complicated by clients' multi-dimensional private information. To this end, we design the model-versioning mechanism so that it decouples clients' decisions and admits a weakly dominant strategy at equilibrium. We prove that our mechanism is feasible, effective, and incentive compatible. Experimental results show that our mechanism greatly improves client participation and model accuracy compared to a benchmark.

While providing machine learning model as a service to process users’ inference requests, online applications can periodically upgrade the model utilizing newly collected data. Federated learning (FL) is beneficial for enabling the training of models across distributed clients while keeping the data locally. However, existing work has overlooked the coexistence of model training and inference under clients’ limited resources. This paper focuses on the joint optimization of model training and inference to maximize inference performance at clients. Such an optimization faces several challenges. The first challenge is to characterize the clients’ inference performance when clients may partially participate in FL. To resolve this challenge, we introduce a new notion of age of model (AoM) to quantify client-side model freshness, based on which we use FL’s global model convergence error as an approximate measure of inference performance. The second challenge is the tight coupling among clients’ decisions, including participation probability in FL, model download probability, and service rates. Toward the challenges, we propose an online problem approximation to reduce the problem complexity and optimize the resources to balance the needs of model training and inference. Experimental results demonstrate that the proposed algorithm improves the average inference accuracy by up to 12%.

Garbage collection (GC) is critical to improving 3D NAND flash memory space utilization. However, GC is very time-consuming for migrating valid data for error correction, which leads to a sharp decline in system performance. Copyback is an advanced command that can be used to accelerate data migration in GC. However, existing copyback-based GC scheme can not guarantee data reliability for various types of flash pages and cause a large decoding latency in the error correction process. In this paper, we first evaluate the copyback error characteristics that measure the quantitative relationships between the number of copybacks and the raw bit error rate (RBER) of upper, middle, and lower pages of triple level cell NAND flash through practical testing, respectively. Then we propose a multi-threshold copyback-based GC scheme (MCBGC). Under the specific RBER limit, the copyback thresholds of upper, middle, and lower pages are determined respectively based on the copyback error characteristics. Experimental results show that compared with the existing copyback-based GC scheme, the write latency of the proposed scheme can be improved by up to 17.7%.

Split federated learning (SFL) is a recent distributed approach for collaborative model training among multiple clients. In SFL, a global model is typically split into two parts, where clients train one part in a parallel federated manner, and a main server trains the other. Despite the recent research on SFL algorithm development, the convergence analysis of SFL is missing in the literature, and this paper aims to fill this gap. The analysis of SFL can be more challenging than that of federated learning (FL), due to the potential dual-paced updates at the clients and the main server. We provide convergence analysis of SFL for strongly convex and general convex objectives on heterogeneous data. The convergence rates are $O(1/T)$ and $O(1/\sqrt[3]{T})$, respectively, where $T$ denotes the total number of rounds for SFL training. We further extend the analysis to non-convex objectives and where some clients may be unavailable during training. Numerical experiments validate our theoretical results and show that SFL outperforms FL and split learning (SL) when data is highly heterogeneous across a large number of clients.

Knowledge distillation (KD) can enable collaborative learning among distributed clients that have different model architectures and do not share their local data and model parameters with others. Each client updates its local model using the average model output/feature of all client models as the target, known as federated KD. However, existing federated KD methods often do not perform well when clients’ local models are trained with heterogeneous local datasets. In this paper, we propose Federated knowledge distillation enabled by Adversarial Learning (FedAL) to address the data heterogeneity among clients. First, to alleviate the local model output divergence across clients caused by data heterogeneity, the server acts as a discriminator to guide clients’ local model training to achieve consensus model outputs among clients through a min-max game between clients and the discriminator. Moreover, catastrophic forgetting may happen during the clients’ local training and global knowledge transfer due to clients’ heterogeneous local data. Towards this challenge, we design the less-forgetting regularization for both local training and global knowledge transfer to guarantee clients’ ability to transfer/learn knowledge to/from others. Experimental results show thatFedALand its variants achieve higher accuracy than other federated KD baselines.

Multi-access Edge Cloud (MEC) networks are powerful for providing emerging computation-intensive and latency-sensitive applications with low latency leveraging ubiquitous edge devices. These networks enable complex applications to be split into multiple components/subtasks and deployed among multiple edge servers with limited computation and communication resources. However, multiple subtasks within an application are dependent on each other. They cannot be executed in parallel, resulting in non-trivial resource waste when allocating resources to every subtask throughout the lifetime of the application. This paper investigates the multi-component task offloading problem in MEC networks that addresses the dependencies among components and three-dimensional (3D) resource allocation, i.e., computation, communication, and time slots. The problem is NP-hard and challenging to solve due to the complex task dependencies, including triangular dependencies among multiple subtasks and the routing of edges between dependent subtasks. To address the challenge, we first propose a non-destructive task reconfiguration algorithm that transforms a task call graph into multiple sequential layers, breaking out the triangular dependency. Then, we develop a dePendency-awaRe task offloAding algorithm wIth taSk rEconfiguration (PRAISE) algorithm to maximize the total offloading benefit.PRAISEdecouples the original problem into task offloading and 3D convex resource optimization. Simulation results show thatPRAISEoutperforms baselines with higher system benefits and lower resource costs.

Multi-access edge computing (MEC) is a powerful technology that facilitates the provision of services to 6G users with ultra-low latency and high reliability, particularly in supporting artificial intelligence (AI) applications that rely on distributed machine learning (DL). However, the mobility of users poses challenges in offloading DL tasks to the MEC networks while ensuring satisfactory delay and blocking rates. Task replication emerges as a promising technique for achieving a cell-less design for mobile users. Nevertheless, existing research overlooks the replication of DL tasks involving multiple subtasks and users, as well as the high resource cost of task replication. Towards this challenge, this paper investigates the Mobility-awarE mulTi-replicA (META) DL task offloading problem in MEC networks. First, we propose a hybrid resource allocation mechanism that allocates resources to a replica with high access probability in a static manner and dynamically allocates resources to replicas with low access probabilities. Then, we develop an access base station (BS) clustering algorithm for each user to determine the optimal number of replicas. Additionally, we propose the META DL task offloading algorithms with proved approximation ratios to minimize the overall resource cost. Through simulations based on generated and real-world mobile users, we demonstrate the effectiveness of our proposed algorithms.

Distributed ensemble learning (DEL) involves training multiple models at distributed learners, and then combining their predictions to improve performance. Existing related studies focus on algorithm development but ignore the important issue of incentives, without which self-interested learners may be unwilling to participate. We aim to fill this gap by presenting a first study on the incentive mechanism design in DEL. The mechanism specifies both the training data and the reward for learners with heterogeneous computation and communication costs. One challenge is that it is unclear how learners' diversity (in terms of training data) contributes to the ensemble accuracy. To this end, we decompose the ensemble accuracy into a diversity-precision tradeoff to guide the mechanism design. Another challenge is that the mechanism design is a mixed-integer program with a large search space. To this end, we propose an alternating algorithm that iteratively updates each learner's training data size and reward. We prove that the algorithm converges and is polynomial in the number of learners. Numerical results using MNIST dataset are consistent with our analysis. Interestingly, we show that the mechanism may prefer a lower level of learner diversity to achieve a higher ensemble accuracy. Our code is made publicly available.

Cloud Computing (CC) is powerful for the computation offloading of services, promoting the implementation of various modern applications. Mobile Edge Computing (MEC) can provide low-latency services utilizing edge servers locating in proximity to users. The combination of MEC and CC can give play to the dual advantages of both. However, it is a challenging problem to offload service requests to the collaborative edge-cloud networks aiming at minimizing costs due to the resource limitation of edge servers and the online feature of services. To address this issue, we mathematically model the service requests with multiple inter-connected functions. Then, the problem of computation offloading of multi-function service requests in collaborative edge-cloud networks is formulated to be an Integer Linear Programming (ILP) and is proved to be NP-hard. Furthermore, a Cost-minimized Computation Offloading with Reconfiguration (CCOR) algorithm is proposed to minimize the total cost of online services. Finally, simulation results show that the proposed CCOR algorithm can effectively reduce the cost of computation offloading with higher resource utilization of edge cloud compared with baseline algorithms.

Knowledge distillation (KD) has enabled remarkable progress in model compression and knowledge transfer. However, KD requires a large volume of original data or their representation statistics that are not usually available in practice. Data-free KD has recently been proposed to resolve this problem, wherein teacher and student models are fed by a synthetic sample generator trained from the teacher. Nonetheless, existing data-free KD methods rely on fine-tuning of weights to balance multiple losses, and ignore the diversity of generated samples, resulting in limited accuracy and robustness. To overcome this challenge, we propose robustness and diversity seeking data-free KD (RDSKD) in this paper. The generator loss function is crafted to produce samples with high authenticity, class diversity, and inter-sample diversity. Without real data, the objectives of seeking high sample authenticity and class diversity often conflict with each other, causing frequent loss fluctuations. We mitigate this by exponentially penalizing loss increments. With MNIST, CIFAR-10, and SVHN datasets, our experiments show that RDSKD achieves higher accuracy with more robustness over different hyperparameter settings, compared to other data-free KD methods such as DAFL, MSKD, ZSKD, and DeepInversion.

Mobile edge computing (MEC) is powerful for providing services with ultra-low latency and extremely high reliability to support newly emerging applications in 5G and beyond by leveraging network function virtualization (NFV) where each mobile user requests a service function chain (SFC). To guarantee the quality of experience (QoE), e.g., reduce the downtime of moving users, effective placement of virtual network functions (VNFs) in MEC networks is critical to cope with the user mobility. Placing multiple instances of SFC for each user is promising for reducing the downtime during moving. However, an extremely high resource cost is induced. Towards this challenge, this paper investigates the mobility-aware multi-instance (MAMI) VNF placement problem in MEC networks. According to the empirical probabilities that one user stays within the coverage of different edge nodes, both static and dynamic SFC instances are placed to balance the tradeoff between the downtime and resource cost. A MAMI VNF placement algorithm is proposed for resource cost minimization, where resource sharing among the SFC instances of each user is allowed to further reduce resource cost. Simulation results based on real-world-like moving users show the effectiveness of our proposed algorithm.

Edge computing that utilizes ubiquitous edge devices locating in close proximity to users is powerful for providing Quality of Service guaranteed computation offloading services. Toward the limited resources of edge servers and wireless links, large services can be split into multiple interconnected components to be served by multiple edge servers cooperatively. The current works on service placement either assume unsplittable services or ignore the geographically isolated property of edge servers. They also ignore the interference among online services that share the same physical nodes/links in terms of executing delay. Namely, every service adds load to the placed nodes/links and every increment on load of nodes/links risks delay violation of existing services. To overcome above challenges, this article emphasizes on the interference-aware (IA) online multicomponent service placement in edge cloud networks. First, the delay of tree-like services is analyzed considering the dependency among components, based on which the IA residual capacities of physical nodes, links, and paths are defined and formulated theoretically. Furthermore, we reduce the problem of multicomponent service placement to be NP-hard and transform it into an ant colony optimization (ACO) problem to obtain the near-optimal solution. More importantly, a level traversal component ranking method and an IA dynamic pruning method are proposed for ACO to achieve faster convergence, interference awareness, and higher acceptance ratio of services. Simulation results are presented to validate the effectiveness of proposed methods. In addition, the classic artificial intelligence application of image classification is experimented to further strength the motivation of IA investigation in practical.

Federated learning (FL) is an emerging technique for training machine learning models using geographically dispersed data collected by local entities. It includes local computation and synchronization steps. To reduce the communication overhead and improve the overall efficiency of FL, gradient sparsification (GS) can be applied, where instead of the full gradient, only a small subset of important elements of the gradient is communicated. Existing work on GS uses a fixed degree of gradient sparsity for i.i.d.-distributed data within a datacenter. In this paper, we consider adaptive degree of sparsity and non-i.i.d. local datasets. We first present a fairness-aware GS method which ensures that different clients provide a similar amount of updates. Then, with the goal of minimizing the overall training time, we propose a novel online learning formulation and algorithm for automatically determining the near-optimal communication and computation trade-off that is controlled by the degree of gradient sparsity. The online learning algorithm uses an estimated sign of the derivative of the objective function, which gives a regret bound that is asymptotically equal to the case where exact derivative is available. Experiments with real datasets confirm the benefits of our proposed approaches, showing up to 40% improvement in model accuracy for a finite training time.

In cloud and edge computing systems, computation, communication, and memory resources are distributed across different physical machines and can be used to execute computational tasks requested by different users. It is challenging to characterize the capacity of such a distributed system, because there exist multiple types of resources and the amount of resources required by different tasks is random. In this paper, we define the capacity as the number of tasks that the system can support with a given overload/outage probability. We derive theoretical formulas for the capacity of distributed systems with multiple resource types, where we consider the power of d choices as the task scheduling strategy in the analysis. Our analytical results describe the capacity of distributed computing systems, which can be used for planning purposes or assisting the scheduling and admission decisions of tasks to various resources in the system. Simulation results using both synthetic and real-world data are also presented to validate the capacity bounds.

Fiber-Wireless (FiWi) access network, which could provide an anytime-anywhere access for end users with high bandwidth capacity and long distance, is facing the challenge of resource allocation and optimization due to the complexity and diversity of traffic demands. Though network virtualization becomes a promising solution, which allows heterogeneous virtual networks coexisting on the shared substrate network, previous works ignored both varied requirements of Quality of Service (QoS) satisfaction of virtual networks and the flexibility of reconfiguring the resource of substrate FiWi access network. In this paper, we propose a new Virtual Network Embedding (VNE) framework based on QoS satisfaction and network reconfiguration. By equipping each virtual network with a specific QoS satisfaction requirement, the characteristics of virtual network demands are formulated from a more practical point of view. Moreover, the adaptive bandwidth allocation of substrate network and virtual network reconfiguration are exploited to maximize the InP revenue. Simulation results demonstrate that our proposed VNE algorithm outperforms previous approaches with multifold increment of InP revenue.