Artificial intelligence (AI) is a strategic technology leading the Fourth Industrial Revolution and has been widely applied across various industries. Based on data from Chinese A-share listed manufacturing firms from 2011 to 2023, this paper investigates the relationship between the intelligent development of manufacturing enterprises (IDME) and supply chain resilience (SCR). The study finds that the IDME can significantly enhance its SCR, particularly for large-scale enterprises, firms in strategic emerging industries (SEI), and enterprises located in regions with weak digital infrastructure and low levels of financial development. Mechanism analysis reveals that intelligent development has a positive effect on SCR by strengthening both internal capabilities and external linkages of enterprises. Specifically, it improves labor quality, enhances competitive capability, and optimizes internal governance; moreover, it promotes inter-firm collaborative innovation, reinforces information transparency, and strengthens supply–demand relationships. The conclusions of this study enrich the literature on the impact of intelligent development on SCR in manufacturing enterprises and provide empirical evidence and policy implications for promoting intelligent transformation and improving SCR in the manufacturing sector.

The accuracy of digital twin models hinges on the prompt collection of information from the vehicular environment. However, the high mobility of vehicles and the dynamically changing network environment pose significant challenges. Dynamic twin model migration can reduce the Age of Information (AoI) by bringing twin models closer to their vehicles. Existing works rarely consider the inherent differences in optimization cycles between digital twin model migration and data upload, which potentially leads to suboptimal cost efficiency and information freshness. Specifically, real-time vehicular data must be rapidly uploaded to edge servers to ensure the accuracy and timeliness of digital twin models, while frequent migration of twin models over short periods incurs substantial costs. Therefore, we propose a dual-timescale bilevel learning approach, where the upper-layer learning optimizes twin model migration decisions on a long timescale to achieve forward-looking model migration, and the lower-layer learning optimizes data upload and resource allocation decisions on a short timescale to ensure the accuracy and timeliness of digital twin models. Then, we design a multi-agent selective parameter sharing approach based on spatiotemporal dependency correlations to accelerate model convergence and reduce communication costs among agents. Furthermore, through a rigorous theoretical analysis, we prove the convergence of the dual-timescale bilevel learning with broad applicability. Finally, numerical results demonstrate that our algorithm outperforms comparison algorithms in terms of convergence, AoI, and system cost, achieving at least a 21.30% reduction in AoI and a 14.58% reduction in system cost compared to the benchmark algorithms.

In mobile edge computing (MEC), computation offloading serves as an effective solution to bridge the gap between the stringent latency requirements of computational tasks and the limited processing capabilities of terminal devices (TDs). However, complex inter-task dependencies, dynamic network conditions, and the decentralized architecture of MEC systems pose significant challenges to efficient and adaptive task offloading. To address these challenges, this paper investigates dependency-aware task offloading and resource optimization in MEC environments. First, we propose a task feature extraction method based on dependency-aware graph neural networks (FEDG), which captures the hierarchical structure and varying importance of subtask dependencies by adaptively learning the aggregation weights of predecessor nodes and edges. Then, to address the joint dependency-aware task offloading and resource allocation problem under partial observability in MEC networks, we design a Dependency-aware Graph-based Multi-Agent deep reinforcement learning (DGMA) algorithm. DGMA integrates adaptive prioritized experience replay and correlation-based selective parameter sharing to improve learning efficiency and accelerate convergence in multi-agent environments. Extensive simulations demonstrate that DGMA achieves superior performance in terms of delay, energy consumption, offloading utility, and deadline violation rate.

Mobile Edge Computing (MEC) decentralizes the network's computing and storage capabilities from centralized infrastructure to edge nodes located closer to end-users, enabling context-aware service deployment, low-latency service response, and efficient computation for mobile users. However, achieving energy-efficient service deployment while maintaining service delivery quality remains a significant challenge due to the wide geographic distribution of edge nodes, the dynamic variation of service workloads, and the differences between service deployment and delivery cycles. To address these challenges, we first design a feature encoding strategy and a self-attention-based encoder to extract contextual features, which are fused to support adaptive decision timescale regulation driven by service semantics and system load dynamics. Then, we propose a novel Dual-Timescale Energy-Efficient Service Deployment and Delivery (DT-EESD) framework integrated with hierarchical learning. The upper layer leverages an enhanced decision-making mechanism to optimize proactive service deployment and base station switching on a larger timescale, aiming to reduce long-term network costs. The lower layer employs a fine-grained real-time optimization approach to dynamically handle service delivery and resource allocation on a smaller timescale, effectively responding to dynamic service requests. By incorporating an expected reward-based learning mechanism, the framework efficiently handles the temporal coupling between deployment and delivery cycles. Extensive experiments demonstrate that DT-EESD outperforms baseline algorithms, achieving at least a 10.89% reduction in average system cost while also improving model convergence, reducing delay, and enhancing resource utilization.

Facial avatar animation methods animate virtual characters based on natural human performance and have been widely used in film and game production. Despite its practical relevance, academic research in this area has been scarce, particularly in the deep learning era. To fill this gap, we propose Facial Avatar Animation via Spatial-Temporal Aggregation (FAST), which leverages Memory-based Spatial-Temporal Aggregation (MSTA) to capture both spatial and temporal dependencies in facial animation. However, directly regressing blendshape and pose coefficients introduces uncertainty and reduces interpretability. Thus, we introduce Implicit Latent Representations (ILRs), which learn the semantic correspondence between the predicted results and blendshapes/poses, enhancing both model interpretability and the vivid tracking of facial expressions. Additionally, monocular RGB-based pose estimation suffers from depth ambiguity that destabilizes animation. To address this, we incorporate a Semantic-Aware Rigid Prior (SRP) to enhance the rigid stability of the animation. To tackle the lack of blendshape coefficient annotations in existing datasets and support the advancement of avatar animation methods, we present the BS500 dataset, which includes 500 individuals and over 4.5 million frames with diverse demographic features such as gender, age, expression, and head pose. Extensive experiments show the superiority of the FAST method over existing approaches. The code and dataset will be released.

Face video editing presents significant challenges, requiring precise preservation of facial identity, temporal consistency, and background details. Existing methods encounter three major challenges: difficulty in achieving accurate facial reconstruction, struggles with challenging real-world videos and reliance on a crop-edit-stitch paradigm that confines editing to localized facial regions. In response, we introduce DiffFERV, a novel diffusion-based framework for realistic face video editing that addresses these limitations through three core contributions. (1) A specialization stage that extends large Text-to-Image (T2I) models' general prior to faces while retaining their broad generative capabilities. This enables robust performance on non-aligned and challenging face images. (2) Temporal modeling, implemented through two distinct attention mechanisms, complements the specialization stage to ensure joint and temporally consistent processing of video frames. (3) Finally, we present a holistic editing pipeline and the concept of preservation features, which leverages our model’s enhanced priors and temporal mechanisms to achieve faithful edits of entire video frames without the need for cropping, excelling even in real-world scenarios. Extensive experiments demonstrate that DiffFERV achieves state-of-the-art performance in both reconstruction and editing tasks.

User activity sequences have emerged as one of the most important signals in recommender systems.We present a foundational model, PinFM, for understanding user activity sequences across multiple applications at a billion-scale visual discovery platform.We pretrain a transformer model with 20B+ parameters using extensive user activity data, then fine-tune it for specific applications, efficiently coupling it with existing models.While this pretrainingand-fine-tuning approach has been popular in other domains, such as Vision and NLP, its application in industrial recommender systems presents numerous challenges.The foundational model must be scalable enough to score millions of items every second while meeting tight cost and latency constraints imposed by these systems,.Additionally, it should capture the interactions between user activities and other features and handle new items that were not present during the pretraining stage.We developed innovative techniques to address these challenges.Our infrastructure and algorithmic optimizations, such as the Deduplicated Cross-Attention Transformer (DCAT), improved our throughput by 600% on Pinterest internal data.We demonstrate that PinFM can learn interactions between user sequences and candidate items * Work done at Pinterest.

The advent of Multi-access Edge Computing (MEC) has empowered Internet of Things (IoT) devices and edge servers to deploy sophisticated Deep Neural Network (DNN) applications, enabling real-time inference. Many concurrent inference requests and intricate DNN models demand efficient multi-DNN inference in MEC networks. However, the resource-limited IoT device/edge server and expanding model size force models to be dynamically deployed, resulting in significant undesired energy consumption. In addition, parallel multi-DNN inference on the same device complicates the inference process due to the resource competition among models, increasing the inference latency. In this paper, we propose a Resource-aware and Dynamic DNN Deployment (R3D) scheme with the collaboration of end-edge-cloud. To mitigate resource competition and waste during multi-DNN parallel inference, we develop a Resource Adaptive Management (RAM) algorithm based on the Roofline model, which dynamically allocates resources by accounting for the impact of device-specific performance bottlenecks on inference latency. Additionally, we design a Deep Reinforcement Learning (DRL)-based online optimization algorithm that dynamically adjusts DNN deployment strategies to achieve fast and energy-efficient inference across heterogeneous devices. Experiment results demonstrate that R3D is applicable in MEC environments and performs well in terms of inference latency, resource utilization, and energy consumption.

Semantic communication (SC) has emerged as a promising paradigm for upcoming intelligent applications, enabling mobile devices to collaboratively execute intelligent tasks with edge servers through computation offloading. However, few studies have addressed the problem of collaborative inference in SC networks. Traditional collaborative inference mechanisms may suffer performance decline in SC systems and are vulnerable to eavesdroppers. To address these issues, first, we present an encryptor that encrypts semantic information to avoid privacy leakage and a decryptor for restoration. Besides, we propose a novel SC-empowered edge computing framework enabling mobile devices to deploy a partial semantic encoder and offload the rest to edge servers. Based on this framework, we formulate the collaborative inference optimization problem, jointly optimizing delay, energy consumption, and privacy leakage. DNNPart is devised based on deep deterministic policy gradient to address the problem, which consists of a semantic attention mechanism that enables it to focus on important state variables, a hybrid action representation method that makes it adapt to mixed discrete and continuous action spaces, a dynamic model splitting algorithm that locates the optimal partition layer and adaptively splits the semantic coders. Integrated with these components, DNNPart iteratively optimizes the offloading strategy to find the optimal offloading strategy. Extensive simulations were conducted to verify the effectiveness of the proposed method by comparing it with baseline mechanisms.

Mobile edge computing (MEC) decentralizes the computational and storage capabilities of the network to edge nodes, providing support for the dynamic deployment and rapid response of mobile services. However, the large-scale distributed deployment of edge nodes, their widespread geographic distribution, multi-dimensional and complexly dependent service characteristics, and the dynamically changing network environment pose challenges to energy-efficient service deployment. In this paper, we consider multi-dimensional features for service deployment, including dynamic traffic demand, geography information, service semantics, and service popularity, with the aim of improving the availability of edge services and reducing network energy consumption. Firstly, to handle the large volume of edge service data, we design a multi-dimensional feature extraction approach based on the Transformer model, which does not rely on the sequential order of data and can enhance computational efficiency of edge models through parallel processing. Then, to adapt to the dynamically changing edge network environment, we propose an Energy-Efficient Service Deployment algorithm (EESD) based on the improved Dueling Deep Q-Network, which makes service deployment and base station switching decisions in a learning-based manner. Finally, simulation results demonstrate that EESD outperforms comparison algorithms in terms of model convergence, system total cost, and energy consumption.

In the realm of online advertising, accurately predicting the conversion rate (CVR) is crucial for enhancing advertising efficiency and user satisfaction. This paper addresses the challenge of CVR prediction while adhering to user privacy preferences and advertiser requirements. Traditional methods face obstacles such as the reluctance of advertisers to share sensitive conversion data and the limitations of model training in secure environments like data clean rooms. We propose a novel model training framework that enables collaborative model training without sharing sample-level gradients with the advertising platform. Our approach introduces several innovative components: (1) utilizing batch-level aggregated gradients instead of sample-level gradients to minimize privacy risks; (2) applying adapter-based parameter-efficient fine-tuning and gradient compression to reduce communication costs; and (3) employing de-biasing techniques to train the model under label differential privacy, thereby maintaining accuracy despite privacy-enhanced label perturbations. Our experimental results, conducted on industrial datasets, demonstrate that our method achieves competitive ROC-AUC performance while significantly decreasing communication overhead and complying with both advertisers’ privacy requirements and user privacy choices. This framework establishes a new standard for privacy-preserving, high-performance CVR prediction in the digital advertising landscape.

The prediction of user trajectories is a fundamental component to support urban traffic management and various advanced transportation applications, such as traffic optimization and location-based services. Trajectory data typically contains multiple behavioral patterns and contexts, including different travel purposes, modes of transportation, time intervals, and geographic regions. These complex factors collectively influence the prediction of user trajectories. However, trajectory prediction models face challenges in effectively distinguishing between these various patterns. In this paper, we propose a novel stack Transformer-based multimodal long-term trajectory prediction (SMTTP) scheme for heterogeneous user behavior patterns. First, a learnable trajectory similarity measure method is proposed to estimate the relative distance between multi-attribute variable-length trajectories. Then, to address the instability of trajectory clustering caused by random initialization, a cluster head initialization algorithm based on high confidence nodes is developed to improve clustering stability and reduce convergence time. In addition, a Transformer-based trajectory prediction model with multi-dimensional feature fusion is proposed to achieve accurate and efficient long-term trajectory prediction. Experimental results on the real telecom dataset in Shanghai, China show that the proposed SMTTP scheme can achieve improved performance in trajectory prediction in terms of prediction error, and also has high accuracy and stability in unsupervised trajectory clustering.

As a distributed learning paradigm, federated learning (FL) can be applied in mobile edge computing (MEC) to support real-time artificial intelligence by leveraging edge computation resources while preserving data privacy in the end devices. However, the unpredictable wireless connections between end devices and edge servers in MEC (e.g., frequent handovers and unstable wireless channels) may result in the loss of important model parameters, which slows down the FL training process and degrades the quality of the global model. In this paper, we propose an adaptive collaborative federated learning (ACFL) scheme to accelerate the convergence and improve model reliability by mitigating communication-based parameter loss under a three-layer MEC architecture. First, a dynamic epoch adjustment method is proposed to reduce communication rounds by dynamically adjusting the training epochs in end devices. In addition, to accelerate the FL convergence, we present an edge server collaborative training scheme by leveraging a multi-layer computing architecture, where edge servers utilize their maintained data to collaboratively train models with end devices. Finally, extensive simulations are conducted and show that ACFL can efficiently improve model reliability and accelerate the convergence of the FL process in MEC.

In Mobile Edge Computing (MEC) networks, dynamic service migration can support service continuity and reduce user-perceived delay. However, service migration in MEC networks faces significant challenges due to the uncertainty in future traffic demands, the distributed architecture of MEC networks, high operating costs and the dynamism of network resources. Digital Twins (DT), which achieve the mapping of physical entities to virtual digital models in cyberspace, provide new perspectives for intelligent and efficient service provisioning in MEC networks. In this paper, we propose a traffic prediction-assisted federated deep reinforcement learning scheme to efficiently migrate services and improve the cost efficiency of DT-enabled MEC networks. Specifically, to address the coupled spatio-temporal dependencies of mobile traffic and the imbalance in traffic data, a Multi-order Spatio-temporal information integration-based distributed Traffic Prediction (MSTP) scheme is proposed, which achieves high-accuracy mobile traffic prediction at a low cost. Then, we propose a Federated Cooperative cost-efficient Service Migration (FCSM) algorithm that adaptively adjusts service migration strategies in a distributed manner to respond to future traffic demands. Moreover, a theoretical model is developed to analyze the convergence of FCSM and derive the upper bound of the time-average squared gradient norm. Finally, extensive simulations demonstrate that the proposed schemes achieve excellent traffic prediction performance, enhance users’ Quality of Service (QoS), and significantly reduce the system cost of MEC networks.

Providing privacy protection has been one of the primary motivations of Federated Learning (FL). Recently, there has been a line of work on incorporating the formal privacy notion of differential privacy with FL. To guarantee the client-level differential privacy in FL algorithms, the clients’ transmitted model updates have to be clipped before adding privacy noise. Such clipping operation is substantially different from its counterpart of gradient clipping in the centralized differentially private SGD and has not been well-understood. In this paper, we first empirically demonstrate that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity when training neural networks, which is partly because the clients’ updates become similar for several popular deep architectures. Based on this key observation, we provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients’ updates. To the best of our knowledge, this is the first work that rigorously investigates theoretical and empirical issues regarding the clipping operation in FL algorithms.

Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification. To defend against such attacks, an effective and popular approach, known as adversarial training (AT), has been shown to mitigate the negative impact of adversarial attacks by virtue of a min-max robust training method. While effective, it remains unclear whether it can successfully be adapted to the distributed learning context. The power of distributed optimization over multiple machines enables us to scale up robust training over large models and datasets. Spurred by that, we propose distributed adversarial training (DAT), a large-batch adversarial training framework implemented over multiple machines. We show that DAT is general, which supports training over labeled and unlabeled data, multiple types of attack generation methods, and gradient compression operations favored for distributed optimization. Theoretically, we provide, under standard conditions in the optimization theory, the convergence rate of DAT to the first-order stationary points in general non-convex settings. Empirically, we demonstrate that DAT either matches or outperforms state-of-the-art robust accuracies and achieves a graceful training speedup (e.g., on ResNet-50 under ImageNet). Codes are available at https://github.com/dat-2022/dat.

Mobile-edge computing (MEC) sinks computation and storage capacities to network edge, where it is close to users to support delay-sensitive services. However, due to the dynamic and stochastic properties of MEC networks, the deployed services may be frequently migrated among edge servers to follow the mobility of users, which greatly increases the network operational cost. In this article, considering the service migration cost brought by user mobility, we study the joint optimization problem of service deployment and request routing decisions to maximize the long-term network utility of MEC networks. First, we propose a Lyapunov optimization-based online service migration algorithm to decompose the continuous optimization problem into a number of one-slot online optimization problems. Then, to address the NP-hard issue of one-slot optimization, we use a randomized rounding technique to implement service migration and request routing. Furthermore, through a closed-form theoretical analysis, we prove that the proposed algorithm not only greatly meets the local user requests and enables approximate performance guarantees but also adaptively balances the service migration cost and system performance online. Finally, extensive simulations are conducted, which demonstrate that our algorithm can efficiently utilize the storage and computation resources of edge servers, and maximize the long-term network utility while ensuring the stability of service migration cost.

The Internet of Things (IoT) technology has expanded network space by interconnected devices, which has been widely used in various fields, such as environmental monitoring, object tracking, risk warning, etc. Due to insufficient computing capacity, limited battery life, and unreliable communication environment in IoT, unmanned aerial vehicle (UAV)-enabled edge computing has been recently utilized to provide enhanced coverage and efficient computational support in the scenarios with sparse or unreliable ground infrastructure, such as disaster rescue, emergency response, military fields, etc. However, UAV-enabled edge computing faces many challenges, such as low offloading efficiency, high energy consumption, high complexity, etc. In this article, a distributed computation offloading scheme is proposed to provide computational support to large-scale IoT nodes and optimize the energy efficiency of multiple UAVs. First, to provide accurate and efficient computational support, a real-time intelligent positioning algorithm is designed to obtain the precise location information of IoT nodes. Then, a distributed computation offloading and path planning algorithm is presented, which jointly optimizes the computation offloading of large-scale IoT nodes and trajectory planning of multiple UAVs to reduce the energy consumption of UAVs. Furthermore, we develop a closed-form theoretical analysis model to demonstrate that the algorithm enables a performance guarantee related to energy efficiency. Finally, extensive simulations have been conducted and show that the proposed scheme can greatly improve the system utility and energy efficiency.

In this paper, we study the problem of constrained min-max optimization in a black-box setting, where the desired optimizer cannot access the gradients of the objective function but may query its values. We present a principled optimization framework, integrating a zeroth-order (ZO) gradient estimator with an alternating projected stochastic gradient descent-ascent method, where the former only requires a small number of function queries and the later needs just one-step descent/ascent update. We show that the proposed framework, referred to as ZO-Min-Max, has a sublinear convergence rate under mild conditions and scales gracefully with problem size. We also explore a promising connection between black-box min-max optimization and black-box evasion and poisoning attacks in adversarial machine learning (ML). Our empirical evaluations on these use cases demonstrate the effectiveness of our approach and its scalability to dimensions that prohibit using recent black-box solvers.

Recently, there is a growing interest in the study of median-based algorithms for distributed non-convex optimization. Two prominent examples include signSGD with majority vote, an effective approach for communication reduction via 1-bit compression on the local gradients, and medianSGD, an algorithm recently proposed to ensure robustness against Byzantine workers. The convergence analyses for these algorithms critically rely on the assumption that all the distributed data are drawn iid from the same distribution. However, in applications such as Federated Learning, the data across different nodes or machines can be inherently heterogeneous, which violates such an iid assumption. This work analyzes signSGD and medianSGD in distributed settings with heterogeneous data. We show that these algorithms are non-convergent whenever there is some disparity between the expected median and mean over the local gradients. To overcome this gap, we provide a novel gradient correction mechanism that perturbs the local gradients with noise, which we show can provably close the gap between mean and median of the gradients. The proposed methods largely preserve nice properties of these median-based algorithms, such as the low per-iteration communication complexity of signSGD, and further enjoy global convergence to stationary solutions. Our perturbation technique can be of independent interest when one wishes to estimate mean through a median estimator.

Deep learning models are increasingly popular in many machine learning applications where the training data may contain sensitive information. To provide formal and rigorous privacy guarantee, many learning systems now incorporate differential privacy by training their models with (differentially) private SGD. A key step in each private SGD update is gradient clipping that shrinks the gradient of an individual example whenever its l2 norm exceeds a certain threshold. We first demonstrate how gradient clipping can prevent SGD from converging to a stationary point. We then provide a theoretical analysis on private SGD with gradient clipping. Our analysis fully characterizes the clipping bias on the gradient norm, which can be upper bounded by the Wasserstein distance between the gradient distribution and a geometrically symmetric distribution. Our empirical evaluation further suggests that the gradient distributions along the trajectory of private SGD indeed exhibit such symmetric structure. Together, our results provide an explanation why private SGD with gradient clipping remains effective in practice despite its potential clipping bias. Finally, we develop a new perturbation-based technique that can provably correct the clipping bias even for instances with highly asymmetric gradient distributions.

The adaptive momentum method (AdaMM), which uses past gradients to update descent directions and learning rates simultaneously, has become one of the most popular first-order optimization methods for solving machine learning problems. However, AdaMM is not suited for solving black-box optimization problems, where explicit gradient forms are difficult or infeasible to obtain. In this paper, we propose a zeroth-order AdaMM (ZO-AdaMM) algorithm, that generalizes AdaMM to the gradient-free regime. We show that the convergence rate of ZO-AdaMM for both convex and nonconvex optimization is roughly a factor of $O(\sqrt{d})$ worse than that of the first-order AdaMM algorithm, where $d$ is problem size. In particular, we provide a deep understanding on why Mahalanobis distance matters in convergence of ZO-AdaMM and other AdaMM-type methods. As a byproduct, our analysis makes the first step toward understanding adaptive learning rate methods for nonconvex constrained optimization.Furthermore, we demonstrate two applications, designing per-image and universal adversarial attacks from black-box neural networks, respectively. We perform extensive experiments on ImageNet and empirically show that ZO-AdaMM converges much faster to a solution of high accuracy compared with $6$ state-of-the-art ZO optimization methods.

In this paper, we propose that the quality of the page delivery should play an important role in the page ranking process, especially for users with a slow Internet connection or mobile users. We define several important quality attributes and explain how we rank the web page based on these attributes. The experiment result shows that our proposed algorithm can promote the pages with a higher delivery quality to higher positions in the result list, which is beneficial to users to improve their searching experiences.