This paper proposes the Low Upload and Storage Cost (LUSC) scheme, a coded distributed computing (CDC) approach that accelerates Convolutional Neural Network (CNN) inference on resource-constrained edge devices. LUSC introduces a new encoding and decoding mechanism that exploits the periodicity and evenness of cosine functions, reducing communication and storage overhead while ensuring numerical stability. Orthogonal decoding matrices derived from this cosine-based design guarantee stable inversion and preserve numerical precision. By leveraging the duality property of cosine functions, LUSC reduces the number of workers required for decoding, enabling finer-grained task decomposition and further lowering upload and storage costs. When combined with a spatial and channel grid partition (SCGP) strategy, LUSC (including other matrix-multiplication-based CDC schemes) can be applied to convolutions, accelerating CNN inference while maintaining resilience to stragglers. Experimental results demonstrate that LUSC consistently outperforms existing numerically stable CDC schemes, providing efficient inference with reduced communication and storage costs and maintaining robustness under straggler conditions.

In this article, we investigate a decentralized constrained optimization problem over time-varying directed networks. The nodes in the network aim to collaboratively minimize the aggregate of all locally known convex cost functions, subject to local nonidentical and multiple constraint sets, inequality constraints, and equality constraints. Problems of this nature arise in a number of applications in real networks, such as facility location in wireless sensor networks and image deblurring in machine learning. To address these types of problems, we propose an efficient subgradient-rescaling-based decentralized fixed-random projection (SR-DFRP) algorithm, named the SR-DFRP algorithm. In particular, the SR-DFRP algorithm employs Polyak's random projection to handle nonidentical and multiple constraints, which reduces computational load by avoiding the formulation of complex subproblems. Furthermore, by utilizing dynamically constructed row-stochastic matrices, the algorithm employs a subgradient rescaling strategy to mitigate the imbalance induced by the time-varying directed networks. Rigorous theoretical analyses are provided to establish that the SR-DFRP algorithm converges almost surely to the optimal solution. Extensive simulations on facility location and image deblurring problems are presented to validate the efficacy of the algorithm and the validity of the theoretical results.

Existing privacy-preserving multi-keyword retrieval schemes often suffer from reduced retrieval efficiency, lack robust verification mechanisms in dynamic environments, and are prone to symmetric key leakage issues. To address these shortcomings, we propose a dynamic and secure multi-keyword search scheme with a two-factor verification mechanism, named Mimi. Specifically, Mimi first constructs a dynamic verification tree structure to accelerate the verification of the correctness of returned results. Second, it builds an encrypted searchable index that supports sub-linear search time complexity. Third, Mimi incorporates a secure symmetric key exchange protocol to protect the confidentiality of the symmetric key. Furthermore, Mimi supports multi-user search operations without increasing the index construction costs and accommodates dynamic updates to both user roles and data. Through comprehensive security analysis, we demonstrate that Mimi ensures the security of the encrypted searchable inverted index and maintains query indistinguishability for users. Empirical evaluations show that the Mimi scheme is efficient and effective.

Existing CNN inference frameworks based on FHE often suffer from reduced efficiency and accuracy due to the polynomial approximation of activation functions, and they lack effective mechanisms to prevent sensitive information leakage during the final classification stage. To address these limitations, we propose FSAT, a fast and secure inference framework enhanced with adversarial training. Specifically, FSAT employs a private CNN model architecture, where linear layers are computed through an optimized homomorphic ciphertext convolution operation, while non-linear layer operations are efficiently realized using a secure searchable index and an encrypted look-up table, which replace polynomial activation approximations and significantly improve inference accuracy and latency performance. To further mitigate information leakage, we introduce a dual-constraint adversarial training scheme that makes it substantially more difficult for an adversary to infer sensitive attributes of the input data. Experimental results demonstrate that FSAT achieves high inference accuracy and efficiency while substantially reducing the risk of sensitive data leakage.

This article focuses on multiagent distributed asynchronous optimization over directed networks where each agent can only access its individual local function, and the aggregate aim is to minimize the cumulative sum of all local functions. Considering the asynchrony among the agents, we develop an algorithm in which agents compute and communicate individually, without any form of synchronized coordination. Agents perform their local updates by local communication with their immediate neighbors, and this may involve the use of stale information. Since asynchrony naturally leads to latency or packet loss, an asynchronous robust gradient tracking mechanism is developed to guarantee estimating the average of agents’ gradients precisely. Moreover, it employs uncoordinated step-sizes which are more flexible and general than constant or decaying step-size. When the global objective is strongly convex and the local objectives have Lipschitz-continuous gradients, we prove that each agent executing the asynchronous algorithm linearly converges to the consensus optimal point at an$\mathcal {O}(\lambda ^{k})$rate, where$\lambda \in (0,1)$is convergence factor and k represents the iteration number, with a step satisfying a tight explicit upper bound. Numerical experiments demonstrate that our algorithm has better advantages over the state-of-the-art asynchronous algorithms.

Imbalanced classification and label noise are two problems in machine learning and data mining. These two problems are widely present in real-world datasets. Data oversampling constitutes one of the potential solutions. Nonetheless, there are certain inherent limitations of current oversampling techniques. Traditional oversamplers are inadequate in ascertaining the quantity of new samples and can not mitigate the impact of noise on new samples. To solve these two problems theoretically, we propose an optimization framework for imbalance-ratio and position (OIRP). OIRP constructs two optimization models utilizing data on feature distribution, class ratio, quantity, and dimensionality to determine the optimal quantity to generate and the optimal position of new samples. These two models are demonstrated to possess optimal solutions. OIRP is employed to enhance current oversampling algorithms. It is adaptive, devoid of parameters, and universally applicable. The experiments are based on dozens of datasets with different quantities, imbalance ratios, noise rates, classical and sota comparison algorithms. The experimental results show that OIRP significantly improves existing oversamplers. The code, datasets, and experimental results of this work can be found in the https://github.com/adsl305480885/OIRP-binary-Oversampling.

The centralized server in traditional federated learning (FL) is authoritative (i.e. decisive control), which may cause immeasurable damage to the system's security in the event of decision failure or attack. To weaken the authority of the central server, existing studies have proposed blockchain-based federated learning (BFL) approaches. However, existing BFL still suffers from high resource overhead and difficulty in resisting high malicious ratio (more than 50%) attacks. To address the above challenge, this paper proposes an efficient and secure non-authoritative (i.e. highly decentralized) anarchic (i.e. distributed self-governance) federated learning framework which is named NAAFL. During the local process of NAAFL, an area credit-based screening mechanism for participating devices is proposed to ensure that participating devices are always highly trusted devices with higher total credit values. Then, to effectively exclude a high percentage of malicious training gradients, a multi-device validation voting mechanism based on historical information is designed to construct the global gradient. Subsequently, to weaken the central server authority and reduce the resource overhead while guaranteeing security, a secure and low-consumption consensus mechanism based on the federation chain is proposed, and the overhead is further reduced by a momentum acceleration algorithm. Finally, the theoretical analysis and experimental simulation are conducted on the proposed NAAFL. The results further show that the proposed NAAFL outperforms existing studies and can defend against attacks with up to 80% malicious ratio, which exceeds the common threshold (50%) of existing studies. Meanwhile, the overhead of NAAFL is reduced by about 77.51% compared to BFL.

The increasing privacy concerns associated with cloud-assisted image retrieval have captured the attention of researchers. However, a significant number of current research endeavors encounter limitations, including suboptimal accuracy, inefficient retrieval, and a lack of effective result verification mechanisms. To address these limitations, we propose an adaptive verifiable privacy-preserving medical image retrieval (AVPMIR) scheme in the outsourced cloud. Specifically, we utilize the convolutional neural network (CNN) ResNet50 model to extract the feature of each medical image within the dataset of the medical institution, aiming to enhance retrieval accuracy. To enhance retrieval efficiency, we build an encryption searchable index based on a mini-batch$k$-means clustering algorithm. Furthermore, we present an index merging method in which multi-data owners build a different index tree according to different standards. To check the correctness of the returned results from the cloud server, we construct an adaptive verification framework for the obtained results based on chameleon hash and BLS signature. To provide strong security for the medical image datasets, we design an improved logistic chaotic mapping algorithm. The security analysis demonstrates that AVPMIR can defend various threat models. The experiment analysis further indicates that the AVPMIR can improve retrieval efficiency and demonstrate its practicability.

With the widespread use of cloud storage technology by individuals and organizations, data providers usually send their data to cloud for storage to reduce memory pressure, and allow the users to retrieve these data, which has become the trend of rapid data retrieval. To guarantee the data confidentiality, several research works have been developed on encrypted cloud data for ranked multi-keyword retrieval. Nevertheless, most of these schemes are disabled since they cannot resist keyword guessing attacks. Moreover, the ranked top-$K$search results obtained by the subscriber from the encrypted cloud data are inaccurate. To overcome these drawbacks, we design a novel and efficient privacy-preserving ranked multi-keyword retrieval scheme (named as PRMKR) in this paper. With PRMKR, the data and the inverted indexes which belong to the data provider can be securely transferred to the cloud server. In addition, a registered subscriber can request accurate retrieval services without compromising his/her trapdoor information to the cloud server. Specifically, we design an encryption searchable plugin-in server and lower dimensional inverted indexesvector for data owners, which can further guarantee data confidentiality of the data owner and improve search efficiency, respectively. Our rigorous security proof demonstrates that PRMKR can withstand keyword guessing attacks. Finally, experimental evaluations confirm that PRMKR has decent computational and communication efficiency.

This article considers distributed optimization by a group of agents over an undirected network. The objective is to minimize the sum of a twice differentiable convex function and two possibly nonsmooth convex functions, one of which is composed of a bounded linear operator. A novel distributed primal-dual fixed point algorithm is proposed based on an adapted metric method, which exploits the second-order information of the differentiable convex function. Furthermore, by incorporating a randomized coordinate activation mechanism, we propose a randomized asynchronous iterative distributed algorithm that allows each agent to randomly and independently decide whether to perform an update or remain unchanged at each iteration, and thus alleviates the communication cost. Moreover, the proposed algorithms adopt nonidentical stepsizes to endow each agent with more independence. Numerical simulation results substantiate the feasibility of the proposed algorithms and the correctness of the theoretical results.

In this article, we consider a class of decentralized resource allocation problems over directed networks, where each node only communicates with its in-neighbors and attempts to minimize its own cost when network-wide resource constraints as well as local capacity limits are satisfied. Decentralized optimization to solve this problem has been a significant focus within engineering research due to its advantages in scalability, robustness, and flexibility. Most existing methods are synchronous while few works are devoted to asynchronously solving the problem. The problem becomes even more challenging when the networks are directed. To address the resource allocation problem when the above issues are considered, we propose a novel decentralized asynchronous algorithm based on the gossip-based communication protocol and epigraph strategy. An important feature of the algorithm is that it is implemented in a completely decentralized manner in the case of asynchronous communication and directed networks. We provide theoretical proof to guarantee the convergence of the proposed algorithm, which indicates that it can successfully allocate the optimal resource. When solving the resource allocation problem over time-varying directed networks, we further discuss a related decentralized asynchronous algorithm according to the random sleep protocol. Numerical examples are given to demonstrate the viability and performance of the algorithms.

In this article, we consider a class of decentralized nonsmooth composite optimization problems over undirected graphs. The global optimization problem is to minimize the sum of local objective functions consisting of a Lipschitz-differentiable convex function and two possibly nonsmooth convex functions, one of which contains a bounded linear operator. The goal is to solve the global optimization problem through decentralized computation and communication over a network of agents without a central coordinator. Through using triple proximal splitting operators to deal with the nonsmooth terms, we come up with a novel decentralized algorithm with uncoordinated stepsizes, where the stepsizes with independent upper bounds are also distributed for agents or edges over the communication network. Furthermore, we establish the sublinear convergence rate for the proposed algorithm in terms of the first-order optimality residual in a nonergodic sense. Simulation experiments on a constrained quadratic programming problem and an optimal load-sharing problem are carried out to verify the correctness of the theoretical results.

This article investigates the problem of distributed online optimization for a group of units communicating on time-varying unbalanced directed networks. The main target of the set of units is to cooperatively minimize the sum of all locally known convex cost functions (global cost function) while pursuing the privacy of their local cost functions being well masked. To address such optimization problems in a collaborative and distributed fashion, a differentially private-distributed stochastic subgradient-push algorithm, called DP-DSSP, is proposed, which ensures that units interact with in-neighbors and collectively optimize the global cost function. Unlike most of the existing distributed algorithms which do not consider privacy issues, DP-DSSP via differential privacy strategy successfully masks the privacy of participating units, which is more practical in applications involving sensitive messages, such as military affairs or medical treatment. An important feature of DP-DSSP is tackling distributed online optimization problems under the circumstance of time-varying unbalanced directed networks. Theoretical analysis indicates that DP-DSSP can effectively mask differential privacy as well as can achieve sublinear regrets. A compromise between the privacy levels and the accuracy of DP-DSSP is also revealed. Furthermore, DP-DSSP is capable of handling arbitrarily large but uniformly bounded delays in the communication links. Finally, simulation experiments confirm the practicability of DP-DSSP and the findings in this article.

Decentralized dual methods play significant roles in large-scale optimization, which effectively resolve many constrained optimization problems in machine learning and power systems. In this article, we focus on studying a class of totally non-smooth constrained composite optimization problems over multi-agent systems, where the mutual goal of agents in the system is to optimize a sum of two separable non-smooth functions consisting of a strongly-convex function and another convex (not necessarily strongly-convex) function. Agents in the system conduct parallel local computation and communication in the overall process without leaking their private information. In order to resolve the totally non-smooth constrained composite optimization problem in a fully decentralized manner, we devise a synchronous decentralized dual proximal (SynDe-DuPro) gradient algorithm and its asynchronous version (AsynDe-DuPro) based on the randomized block-coordinate method. Both SynDe-DuPro and AsynDe-DuPro algorithms are theoretically proved to achieve the globally optimal solution to the totally non-smooth constrained composite optimization problem relied on the quasi-Fejér monotone theorem. As a main result, AsynDe-DuPro algorithm attains the globally optimal solution without requiring all agents to be activated at each iteration and thus is more robust than most existing synchronous algorithms. The practicability of the proposed algorithms and correctness of the theoretical findings are demonstrated by the experiments on a constrained Decentralized Sparse Logistic Regression (DSLR) problem in machine learning and a Decentralized Energy Resources Coordination (DERC) problem in power systems.

In this article, the problem of the distributed convex optimization is investigated, where the target is to collectively minimize a sum of local convex functions over an unbalanced directed multiagent network. Each agent in the network possesses only its private local objective function, and the sum of all local objective functions constitutes the global objective function. We particularly consider the scenario, where the underlying interaction network is strongly connected and the relevant weight matrix is row stochastic. To collectively figure out the optimization problem, a distributed accelerated convergence algorithm where agents utilize uncoordinated step-sizes is presented by incorporating consensus of multiagent networks into distributed inexact gradient tracking technique. Most of the existing methods require all agents to possess the out-degree information of their in-neighbors, which is impractical and hardly inevitable as interpreted in this article. By utilizing the small-gain theorem, we prove that if the maximum step-size is positive and sufficiently small (constrained by a specific upper bound), the proposed algorithm, termed as SGT-FROST, converges geometrically to the optimal solution given that the objective functions are smooth and strongly convex. A certain convergence rate is also shown. Simulations confirm the findings in this article.

In this article, we concentrate on dealing with the distributed optimization problem over a directed network, where each unit possesses its own convex cost function and the principal target is to minimize a global cost function (formulated by the average of all local cost functions) while obeying the network connectivity structure. Most of the existing methods, such as push-sum strategy, have eliminated the unbalancedness induced by the directed network via utilizing column-stochastic weights, which may be infeasible if the distributed implementation requires each unit to gain access to (at least) its out-degree information. In contrast, to be suitable for the directed networks with row-stochastic weights, we propose a new directed distributed Nesterov-like gradient tracking algorithm, named as D-DNGT, that incorporates the gradient tracking into the distributed Nesterov method with momentum terms and employs nonuniform step-sizes. D-DNGT extends a number of outstanding consensus algorithms over strongly connected directed networks. The implementation of D-DNGT is straightforward if each unit locally chooses a suitable step-size and privately regulates the weights on information that acquires from in-neighbors. If the largest step-size and the maximum momentum coefficient are positive and small sufficiently, we can prove that D-DNGT converges linearly to the optimal solution provided that the cost functions are smooth and strongly convex. We provide numerical experiments to confirm the findings in this article and contrast D-DNGT with recently proposed distributed optimization approaches.

This paper studies a class of distributed convex optimization problems by a set of agents in which each agent only has access to its own local convex objective function and the estimate of each agent is restricted to both coupling linear constraint and individual box constraints. Our focus is to devise a distributed primal-dual gradient algorithm for working out the problem over a sequence of time-varying general directed graphs. The communications among agents are assumed to be uniformly strongly connected. A column-stochastic mixing matrix and a fixed step-size are applied in the algorithm which exactly steers all the agents to asymptotically converge to a global optimal solution. Based on the standard strong convexity and the smoothness assumptions of the objective functions, we show that the distributed algorithm is capable of driving the whole network to geometrically converge to an optimal solution of the convex optimization problem only if the step-size does not exceed some upper bound. We also give an explicit analysis for the convergence rate of the proposed optimization algorithm. Simulations on economic dispatch problems and demand response problems in power systems are performed to illustrate the effectiveness of the proposed optimization algorithm.