The substantial memory demands of model weights and key-value (KV) caches often lead to severe memory bottlenecks in LLM serving. Existing systems address this by offloading KV caches to host memory and rapidly restoring them on demand before decoding. However, these approaches are too coarse-grained and fail to fully exploit the combined computational and storage capabilities of GPUs.

Serving Large Language Models (LLMs) in production faces significant challenges from highly variable request patterns and severe resource fragmentation in serverless clusters. Current systems rely on static pipeline configurations that struggle to adapt to dynamic workload conditions, leading to substantial inefficiencies.

Modern LLM serving systems increasingly adopt elastic inference pipelines where stages frequently join, leave, and migrate across nodes. However, existing GPU communication frameworks like NCCL assume static topologies, causing routing failures and P99 latency spikes during worker transitions that violate sub-millisecond tail latency requirements. We present Connex, a communication system that elevates endpoint mobility from exceptional failure to first-class primitive. Rather than optimizing individual mechanisms in isolation, Connex defines a mobility contract that the communication layer enforces whenever workers join, leave, or migrate while token streams, activations, or KV transfers are in flight. The contract is realized through three cooperating mechanisms: (1) epoch-based routing that bounds staleness without global coordination, (2) explicit handover protocols that preserve stream ordering and provide exactly-once delivery across migrations, and (3) credit-based backpressure with traffic-class isolation that prevents churn-induced interference with latency-critical paths. Evaluation on a 5-node GPU cluster under synthetic and production-derived churn shows that Connex reduces P99 tail spikes by up to 85% compared to NCCL-based baselines, achieves sub-second cutover, and maintains 100% goodput at moderate loads where baselines collapse to 0–28%, while incurring less than 5% steady-state overhead.

ABSTRACT Objective Large Language Models (LLMs) are increasingly deployed in modern AI infrastructure, creating a strong demand for high‐throughput and resource‐efficient serving systems. Disaggregated LLM serving, which decouples prompt prefill from auto‐regressive decode to accommodate their heterogeneous compute and memory characteristics, has emerged as a promising architecture. However, existing disaggregated serving systems suffer from three fundamental limitations: static resource allocation that fails to adapt to highly dynamic workloads, severe load imbalance between compute‐bound prefill and memory‐bound decode stages, and prefix‐cache‐aware routing that skews load distribution and creates performance hotspots. These issues collectively limit resource utilization, scalability, and the ability to meet service level objectives (SLOs) under real‐world workloads. Methods To address these challenges, we propose BanaServe, a dynamic orchestration framework for disaggregated LLM serving that continuously rebalances both computational and memory resources across prefill and decode instances. BanaServe introduces three key mechanisms: (i) layer‐level weight migration to enable coarse‐grained redistribution of computation, (ii) attention‐level Key–Value (KV) cache migration for fine‐grained memory load balancing, and (iii) a Global KV Cache Store with layer‐wise overlapped transmission to decouple routing decisions from cache placement. Together, these mechanisms eliminate cache‐induced hotspots and allow routers to perform purely load‐aware scheduling with minimal latency overhead. BanaServe is implemented on top of state‐of‐the‐art LLM serving frameworks, including vLLM and DistServe. Results We evaluate BanaServe under diverse and challenging workloads, including long‐context inference, bursty request arrivals, and mixed prompt–generation patterns. Experimental results show that, compared to vLLM, BanaServe improves throughput by 1.2–3.9× and reduces total processing time by 3.9%–78.4%. In comparison with DistServe, BanaServe achieves 1.1–2.8× higher throughput while reducing latency by 1.4%–70.1%. These gains are consistent across workload variations, demonstrating BanaServe's robustness under highly dynamic serving conditions. Conclusion BanaServe demonstrates that dynamic, multi‐granularity resource rebalancing and cache‐decoupled routing are essential for efficient disaggregated LLM serving. By jointly addressing resource elasticity, stage imbalance, and cache‐induced load skew, BanaServe substantially improves throughput, latency, and resource utilization in real‐world deployments. This work provides a practical and scalable foundation for next‐generation LLM serving systems operating under dynamic and heterogeneous workloads.

ABSTRACT Objective Microservices transform traditional monolithic applications into lightweight, loosely coupled application components and have been widely adopted in many enterprises. Cloud platform infrastructure providers enhance the resource utilization efficiency of microservices systems by co‐locating different microservices. However, this approach also introduces resource competition and interference among microservices. Designing interference‐aware strategies for large‐scale, co‐located microservice clusters is crucial for enhancing resource utilization and mitigating competition‐induced interference. These challenges are further exacerbated by unreliable metrics, application diversity, and node heterogeneity. Methods In this paper, we first analyze the characteristics of large‐scale and co‐located microservices clusters at Alibaba and further discuss why cycle per instruction (CPI) is adopted as a metric for interference measurement in large‐scale production clusters, as well as how to achieve accurate prediction of CPI through multi‐dimensional metrics. Based on CPI interference prediction and analysis, we also present the design of the C‐Koordinator platform, an open‐source solution utilized in Alibaba cluster, which incorporates co‐location and interference mitigation strategies. Results The interference prediction models consistently achieve over 90.3% accuracy, enabling precise prediction and rapid mitigation of interference in operational environments. As a result, application latency is reduced and stabilized across all percentiles (P50, P90, P99) response time (RT), achieving improvements ranging from 16.7% to 36.1% under various system loads compared with state‐of‐the‐art system. Conclusion These results demonstrate the system's ability to maintain smooth application performance in co‐located environments.

In recent years, the Mixture-of-Experts (MoE) architecture has been widely applied to large language models (LLMs), providing a promising solution that activates only a subset of the model’s parameters during computation, thereby reducing overall memory requirements and allowing for faster inference compared to dense models. Despite these advantages, existing systems still face issues of low efficiency due to static model placement and lack of dynamic workloads adaptation. This leads to suboptimal resource utilization and increased latency, especially during bursty requests periods.To address these challenges, this paper introduces Brownout-Serve, a novel serving framework designed to optimize inference efficiency and maintain service reliability for MoE-based LLMs under dynamic computational demands and traffic conditions. BrownoutServe introduces “united experts” that integrate knowledge from multiple experts, reducing the times of expert access and inference latency. Additionally, it proposes a dynamic brownout mechanism to adaptively adjust the processing of certain tokens, optimizing inference performance while guaranteeing service level objectives (SLOs) are met. Our evaluations show the effectiveness of BrownoutServe under various workloads: it achieves up to 2.46× throughput improvement compared to state-of- the-art systems and reduces SLO violations by up to 90.28%, showcasing its robustness under bursty traffic while maintaining acceptable inference accuracy.

Open source software supply-chain attacks, once successful, can exact heavy costs in mission-critical applications. As open source ecosystems for deep learning flourish and become increasingly universal, they present attackers previously unexplored avenues to code-inject malicious backdoors in deep neural network models. This article proposes Flareon , a small, stealthy, seemingly harmless code modification that specifically targets the data augmentation pipeline with motion-based triggers. Flareon neither alters ground-truth labels, nor modifies the training loss objective, nor does it assume prior knowledge of the victim model architecture, training data, and training hyperparameters. Yet, it has a surprisingly large ramification on training—models trained under Flareon learn powerful target-conditioned (or “ all2all ”) backdoors. We also proposed a learnable variant of Flareon that is even stealthier in terms of added perturbations. The resulting models can exhibit high attack success rates for any target choices and better clean accuracies than backdoor attacks that not only seize greater control but also assume more restrictive attack capabilities. We also demonstrate the resilience of Flareon against a wide range of defenses. Flareon is fully open source and available online to the deep learning community.

In contemporary cloud architectures, an increasing number of cloud service providers are adopting programmable switches to deliver cloud network services (e.g., load-balancing gateways) for millions of tenants. The rate limiters are essential for cloud networks to execute network policies such as congestion control and traffic isolation on programmable switches. Most existing rate limiters utilize the token bucket algorithm, which suffers from scalability issues and substantial control overhead, impacting bandwidth utilization. The WF2Q+ (Worst-case FairWeighted Fair Queueing Plus) algorithm, known for its scalability and accuracy, is gaining traction but faces challenges on programmable switches. The hardware limitations hinder key operations of WF2Q+ (sorting and scheduling) in a single switch pipeline. This paper introduces DRAGONKING, a novel rate-limiting system that enables WF2Q+ through a multi-pipeline sorting and scheduling design. DRAGONKING enhances scalability and throughput while maintaining high accuracy. It achieves this by strategically balancing bandwidth across multiple pipelines, allowing for timely packet scheduling. DRAGONKING is implemented and evaluated on Barefoot Tofino switch. Results show that DRAGONKING supports up to two million entries and delivers line-rate throughput, achieving 1.9× improvements compared with token bucket-based limiters. Moreover, it can maintain over 99% accuracy, precisely enforcing rate limits from 10 Gbps to 100 Gbps.

Large language models increasingly rely on pipeline parallelism for distributed inference, but existing systems face critical challenges in serverless environments: heterogeneous request distributions across pipeline stages and unpredictable workload patterns requiring rapid elasticity. Traditional static resource allocation fails to address pipeline-specific bottlenecks and cold start delays inherent in serverless architectures. We propose QUART, a workload-adapted resource allocation system for LLM distributed serving in serverless clusters. QUART introduces pipeline-aware resource management through: (1) latency-aware critical stage identification using coefficient of variation (CV)-based burst propagation analysis, (2) dynamic replica allocation with proportional-integral-derivative (PID) control for congested stages, and (3) hierarchical parameter caching with copy-on-write mechanisms enabling sub-second serverless scaling. The system addresses serverless-specific challenges through cache-aware scheduling that maintains model parameters in memory, eliminating disk I/O overhead during rapid scaling events. Evaluation with real-world workloads shows QUART reduces average response latency by up to 87.1% compared to existing serverless inference systems while achieving 2.37x improvement in goodput.

To meet strict Service-Level Objectives (SLO), contemporary Large Language Models (LLMs) decouple the prefill and decoding stages and place them on separate GPUs to mitigate the distinct bottlenecks inherent to each stage. However, the heterogeneity of LLM workloads causes producer-consumer imbalance between the two instance types in such disaggregated architecture. To address this problem, we propose DOPD (Dynamic Optimal Prefill/Decoding), a dynamic LLM inference system that adjusts instance allocations to achieve an optimal prefill-to-decoding (P/D) ratio based on real-time load monitoring. Combined with an appropriate request-scheduling policy, DOPD effectively resolves imbalances between prefill and decoding instances and mitigates resource allocation mismatches due to mixed-length requests under high concurrency. Experimental evaluations show that, compared with vLLM and DistServe (representative aggregation-based and disaggregation-based approaches), DOPD improves overall system goodput by up to$1.5\times$, decreases P90 time-to-first-token (TTFT) by up to 67.5%, and decreases P90 time-per-output-token (TPOT) by up to 22.8%. Furthermore, our dynamic P/D adjustment technique performs proactive reconfiguration based on historical load, achieving over 99% SLO attainment while using fewer additional resources.

This paper presents a comprehensive analysis of diffusion model serving challenges in production cloud environments. We examine the unique computational patterns and resource requirements that distinguish diffusion model serving from traditional ML workloads, revealing fundamental systemlevel challenges from their multi-stage pipeline architectures. Our analysis is based on a dataset collected from a commercial image generation service processing 3.5 million requests across 300+ GPUs of production operation.

Advances in large language models (LLMs) have opened up new possibilities across various fields, fueling a new wave of interactive AI applications such as DeepSeek and ChatGPT. Inference serving systems play a crucial role in supporting these applications. Recent research indicates that when crossserver parallelization is enabled in inference serving systems, data synchronization overhead can exceed 65% of the total inference delay, making the reduction of communication overhead essential for speeding up inference. While existing systems accelerate cross-server communications by offloading synchronization operations to programmable switches, they often suffer from limited aggregation throughput under bursty traffic conditions, posing challenges for homogeneous network environments. To address these challenges, we propose HeroServe, an innovative inference serving system that leverages heterogeneous networks to accelerate data synchronization in distributed clusters. Our approach enables a fast and scalable inference serving system by employing an offline planner for joint computation allocation and communication scheduling, along with an online scheduler for dynamic traffic management and load balancing. We implement a prototype on a testbed comprising six servers and two programmable switches. Experimental results demonstrate that HeroServe improves scalability by$1.53 \times$while achieving lower latency compared to state-of-the-art solutions.

In this paper, we present ROCK, a novel system for efficiently serving thousands of LoRA adapters for multimodal models in cloud environments. Through extensive analysis of production workloads, we identify key challenges in current cloud-based image generation services: extreme request burstiness (up to$90 \times$normal rates), heterogeneous task characteristics, and inefficient adapter management that wastes 40 % of GPU memory and increases delays by$3 x$during peak times. ROCK addresses these challenges through a three-layer architecture that decouples hardware, adapters, and requests. Our system features dynamic heterogeneous queues that match tasks to appropriate resources based on multidimensional feature vectors, and a multilevel orchestration framework that intelligently manages adapter placement across heterogeneous storage. Experiments on a 64-GPU testbed demonstrate that ROCK reduces average response latency by$16-26 \%$, and achieves an 84.1 % cache hit rate for LoRA adapters-outperforming traditional approaches while reducing adapter update frequency by up to 77 %.

Transformer models face scalability challenges in causal language modeling (CLM) due to inefficient memory allocation for growing keyvalue (KV) caches, which strains compute and storage resources.Existing methods like Grouped Query Attention (GQA) and tokenlevel KV optimization improve efficiency but rely on rigid resource allocation, often discarding "low-priority" tokens or statically grouping them, failing to address the dynamic spectrum of token importance.We propose mixSGA, a novel mixture-of-expert (MoE) approach that dynamically optimizes token-wise computation and memory allocation.Unlike prior approaches, mixSGA retains all tokens while adaptively routing them to specialized experts with varying KV group sizes, balancing granularity and efficiency.Our key novelties include: (1) a token-wise expert-choice routing mechanism guided by learned importance scores, enabling proportional resource allocation without token discard; (2) weight-sharing across grouped attention projections to minimize parameter overhead; and (3) an auxiliary loss to ensure one-hot routing decisions for training-inference consistency in CLMs.Extensive evaluations across Llama3, TinyLlama, OPT, and Gemma2 model families show mixSGA's superiority over static baselines.On instruction-following and continued pretraining tasks, mixSGA achieves higher ROUGE-L and lower perplexity under the same KV budgets.

Deep learning (DL) inference services are widely recognized as crucial workloads in large-scale cloud clusters. However, due to the stringent latency requirements, cloud providers often over-provision GPU resources, resulting in underutilization of the available GPU potential. Although co-locating tasks on the same device can enhance utilization, ensuring Service Level Objectives (SLOs) guarantees for multiplexing highly dynamic inference services becomes extremely challenging due to significant resource interference.

Occlusion is a key factor leading to detection failures. This paper proposes a motion-assisted detection (MAD) method that actively plans an executable path, for the robot to observe the target at a new viewpoint with potentially reduced occlusion. In contrast to existing MAD approaches that may fail in cluttered environments, the proposed framework is robust in such scenarios, therefore termed clutter resilient occlusion avoidance (CROA). The crux to CROA is to minimize the occlusion probability under polyhedron-based collision avoidance constraints via the convex-concave procedure and duality-based bilevel optimization. The system supports lidar-based MAD with intertwined execution of learning-based detection and optimization-based planning. Experiments show that CROA outperforms various MAD schemes under a sparse convolutional neural network detector, in terms of point density, occlusion ratio, and detection error, in a multi-lane urban driving scenario.

As securities trading systems transition to a microservices architecture, optimizing system performance presents challenges such as inefficient resource scheduling and high service response delays. Existing container orchestration platforms lack tailored performance optimization mechanisms for trading scenarios, making it difficult to meet the stringent 50ms response time requirement imposed by exchanges. This paper introduces SealOS+, a Sealos-based performance optimization approach for securities trading, incorporating an adaptive resource scheduling algorithm leveraging deep reinforcement learning, a three-level caching mechanism for trading operations, and a Long Short-Term Memory (LSTM) based load prediction model. Real-world deployment at a securities exchange demonstrates that the optimized system achieves an average CPU utilization of 78%, reduces transaction response time to 105ms, and reaches a peak processing capacity of 15,000 transactions per second, effectively meeting the rigorous performance and reliability demands of securities trading.

Multilingual Automatic Speech Recognition (ASR) systems face challenges in efficiently integrating new languages while maintaining scalability and performance. This paper introduces an Incremental Tucker-based Low-Rank Adaptation (LoRA) framework designed to address these limitations. By leveraging Tucker decomposition to compress multilingual LoRA parameters and introducing an efficient incremental learning mechanism, the framework enables seamless addition of new languages without re-computing the full decomposition. Experiments on the Mozilla Common Voice dataset demonstrate that the proposed approach reduces parameter requirements by up to 80% compared to traditional LoRA-based methods, while achieving competitive Word Error Rates (WER) across diverse languages. This framework provides a scalable and resource-efficient solution for modern multilingual ASR systems, offering significant advantages in memory efficiency and adaptability to dynamic language expansion.

With the rapid development of large language models (LLMs), there has been a strong interest in integrating other modalities such as image comprehension capabilities. While they have shown impressive performance in various multimodal tasks, the robustness of Visual Language Models (VLMs) has not been thoroughly investigated. We mainly focus on the robustness of VLMs on visual adversarial examples. In this work, we explore the capability of adversarial examples targeting VLMs. We highlight that the multimodal nature of VLMs presents a unique attack surface to manipulate the outputs of the LLMs, and the continuous nature of visual inputs further enhances the effectiveness of adversarial attacks against language generative models. Furthermore, we demonstrate three application scenarios for adversarial examples targeting VLMs: image description, jailbreaking, and information hiding. We conduct experiments on several leading open-source VLMs and demonstrate the successful application of adversarial examples in all the proposed scenarios. We hope that our findings would enable the development of multimodal models more robust to adversarial attacks. Our code is available at https://github.com/lafeat/m3-break.

Industrial anomaly detection for 2D objects has gained significant attention and achieved progress in anomaly detection (AD) methods. However, identifying 3D depth anomalies using only 2D information is insufficient. Despite explicitly fusing depth information into RGB images or using point cloud backbone networks to extract depth features, both approaches struggle to adequately represent 3D information in multimodal scenarios due to the disparities among different modal information. Additionally, due to the scarcity of abnormal samples in industrial data, especially in multimodal scenarios, it is necessary to perform anomaly generation to simulate real-world abnormal samples. Therefore, we propose a novel unified multimodal anomaly detection framework to address these issues. Our contributions consist of 3 key aspects. (1) We extract visible depth information from 3D point cloud data simply and use 2D RGB images to represent appearance, which disentangles depth and appearance to support unified anomaly generation. (2) Benefiting from the flexible input representation, the proposed Multi-Scale Gaussian Anomaly Generator and Unified Texture Anomaly Generator can generate richer anomalies in RGB and depth. (3) All modules share parameters for both RGB and depth data, effectively bridging 2D and 3D anomaly detection. Subsequent modules can directly leverage features from both modalities without complex fusion. Experiments show our method outperforms state-of-the-art (SOTA) on MVTec-3D AD and Eyecandies datasets. Code available at: https://github.com/Xantastic/BridgeNet

ABSTRACT Objective The microservices architecture has become a dominant paradigm in cloud computing due to its advantages in development, deployment, modularity, and scalability. Ensuring Quality of Service (QoS) through efficient Service Level Objective (SLO) resource allocation is a critical challenge. Current frameworks for microservice autoscaling based on SLOs often rely on heavy and complex models that are time‐consuming and resource‐intensive, making them unsuitable for rapidly changing environments and highly dynamic workloads. Methods This study proposes LSRAM (Lightweight SLO Resource Allocation Management), a novel framework designed to overcome the limitations of existing SLO‐based autoscaling methods. LSRAM operates in two stages: 1). Lightweight SLO Resource Allocation Model: Computes optimal SLO resource allocation for each microservice using a gradient descent method, ensuring rapid computation and minimal computational overhead. 2). SLO Resource Update Model: Adapts resource allocation dynamically in response to changes in the cluster environment, such as varying loads and application types, without requiring extensive retraining. Results LSRAM effectively addresses scenarios involving bursty traffic and fluctuating workloads. Compared to state‐of‐the‐art SLO allocation frameworks, LSRAM achieves the following: 1). Reduces resource usage by 17%. 2). Maintains QoS guarantees for users, even under dynamic conditions. 3). Demonstrates faster adaptability to changes in the system environment due to its lightweight design. Conclusion LSRAM offers a scalable, efficient, and adaptive solution for SLO‐based resource allocation in microservices architectures. By reducing resource usage while maintaining QoS, it provides a robust framework for managing dynamic and unpredictable workloads in cloud environments. Its lightweight design ensures practical applicability and superior performance compared to traditional, resource‐intensive methods.

ABSTRACT Background Cloud‐native applications are increasingly becoming popular in modern software design. Employing a microservice‐based architecture into these applications is a prevalent strategy that enhances system availability and flexibility. However, cloud‐native applications introduce new challenges, including frequent inter‐service communication and the management of heterogeneous codebases and hardware, resulting in unpredictable complexity and dynamism. Furthermore, as applications scale, only limited research teams or enterprises possess the resources for large‐scale deployment and testing, which impedes progress in the cloud‐native domain. Aims To address these challenges, we propose CloudNativeSim, a simulator for cloud‐native applications with a microservice‐based architecture. Results CloudNativeSim offers several key benefits: (i) comprehensive and dynamic modeling for cloud‐native applications, (ii) an extended simulation framework with new policy interfaces for scheduling cloud‐native applications, and (iii) support for customized application scenarios and user feedback based on Quality of Service (QoS) metrics. Conclusion CloudNativeSim can be easily deployed on standard computers to manage a high volume of requests and services. Its performance was validated through a case study, demonstrating higher than 94.5% accuracy in terms of response time simulation. The study further highlights the feasibility of CloudNativeSim by illustrating the effects of various scaling policies.

Microservice architecture has transformed traditional monolithic applications into lightweight components. Scaling these lightweight microservices is more efficient than scaling servers. However, scaling microservices still faces the challenges resulting from the unexpected spikes or bursts of requests, which are difficult to detect and can degrade performance instantaneously. To address this challenge and ensure the performance of microservice-based applications, we propose a status-aware and elastic scaling framework called StatuScale , which is based on load status detector that can select appropriate elastic scaling strategies for differentiated resource scheduling in vertical scaling. Additionally, StatuScale employs a horizontal scaling controller that utilizes comprehensive evaluation and resource reduction to manage the number of replicas for each microservice. We also present a novel metric named correlation factor to evaluate the resource usage efficiency. Finally, we use Kubernetes, an open source container orchestration and management platform, and realistic traces from Alibaba to validate our approach. The experimental results have demonstrated that the proposed framework can reduce the average response time in the Sock-Shop application by 8.59% to 12.34% and in the Hotel-Reservation application by 7.30% to 11.97%, decrease service level objective violations, and offer better performance in resource usage compared to baselines.

Distributed machine learning over geo-distributed clouds enables joint training of data located in different regions, alleviating the burden of transferring large volumes of training datasets, which greatly saves bandwidth. However, the limited capacity of WAN links slows down the inter-cloud communications, which significantly decelerates the synchronization of distributed machine learning over geo-distributed clouds. Besides, the multi-tenancy in clouds results in multiple training tasks running simultaneously, whose synchronizations consistently compete for the limited WAN bandwidth with each other, which further aggravates the training performance of each task. While existing works optimize synchronizations through techniques like gradient compression, multi-resource interleaving and so on, none of them targets at the synchronization congestion especially due to multi-tenant learning, which results in inferior training performance.To solve these problems, we propose a simple but effective scheme, SCC, for fast and efficient multi-tenant learning via synchronization congestion control. SCC monitors the cross-cloud network conditions and evaluates the synchronization congestion level based on the round-trip transmission time for each synchronization. Then SCC alleviates synchronization congestion via controlling the synchronization frequency according to the synchronization congestion level in a probabilistic way. Extensive experiments are conducted within our testbeds consisted of 16 NVIDIA V100 GPUs to evaluate the performance of SCC, and comparison results show that SCC can reduce the average training completion time and makespan by up to 28.6% and 43.2% over SAP-SGD [1]. Targeted experiments are conducted to demonstrate the effectiveness and robustness of SCC.

Although the emerging serverless paradigm has the potential to become a dominant way of deploying cloud-service tasks across millions of mobile and IoT devices, the overhead characteristics of executing these tasks on such a volume of mobile devices remain largely unclear. To address this issue, this paper conducts a deep analysis based on the OpenFaaS platform—a popular open-source serverless platform for mobile edge environments—to investigate the overhead of performing deep learning inference tasks on mobile devices. To thoroughly evaluate the inference overhead, we develop a performance benchmark, namedESBench, whereby a set of comprehensive experiments are conducted with respect to a bunch of simulated mobile devices associated with an edge cluster. Our investigation reveals that the performance of deep learning inference tasks is significantly influenced by the model size and resource contention in mobile devices, leading to up to$3\times$degradation in performance. Moreover, we observe that the network environment can negatively impact the performance of mobile inference, increasing the CPU overhead under poor network conditions. Based on our findings, we further propose some recommendations for designing efficient serverless platforms and resource management strategies as well as for deploying serverless computing in the mobile edge environment.

Online map matching (MM) aligns real-time GPS trajectories with digital road networks, playing a vital role in vehicle navigation, route planning, and traffic analysis. Hidden Markov Models (HMMs) are widely used for their interpretability and ability to handle low GPS sampling rates. However, in urban scenarios characterized by complex road networks, significant GPS localization error, and dynamic traffic conditions, existing HMM-based methods face challenges such as large road search spaces due to uniform GPS localization error distributions (GLED) and inaccurate route accessibility estimates stemming from inadequate consideration of real-time traffic conditions. This paper proposes an improved HMM-based MM method, recognizing that urban vehicle trajectories can be categorized into two types: path-free (e.g., taxis, private cars) and path-constrained (e.g., buses). Analyzing path-constrained trajectories helps estimate fine-grained GLED and real-time traffic states of path-free vehicles more precisely. The novelty of our approach lies in two aspects: i) Using a hierarchical spectral clustering algorithm based on GPS localization errors of path-constrained bus trajectories, a city is divided into fine-grained sub-regions with consistent GLED. This enables the HMM an adaptive road search scopes, improving online MM efficiency. ii) Gradient boosting trees, known for their interpretability, estimate free-flow speeds by integrating path-constrained trajectories with the factors like road attributes and time, optimizing HMM state transition probabilities for path-free trajectory MM. Experiments on real-world data demonstrate that the optimized HMM methods, leveraging different trajectory types, significantly enhance MM efficiency and accuracy compared to baseline models.The codebase of our methods and datasets are available at https://github.com/jacklee018/onlineMM-IPCT.

Transport Mode Split (TMS) represents the distribution of trips among transport modes between city regions. Accurate TMS prediction is crucial for urban planning and traffic management. Traditional methods, such as curve models and discrete choice models, often fail to capture user travel mode pReferences due to dataset limitations and inadequate spatiotemporal modeling. While deep learning models have been applied in related domains such as ride-sharing and traffic prediction, they typically focus on a single transport mode and do not model the competitive relationships between modes. We propose a novel Deep Learning-based Framework for Transport Mode Split Prediction (DMSP), integrating restricted-route (bus/subway) and free-route (taxi) data. It introduces two key innovations: 1) efficient extraction of TMS samples from sparse raw data, enriched with spatiotemporal context, and 2) a prediction model extending discrete choice theory via maximum utility principles. Mode utilities are estimated using CNNs for travel mode-specific effects and GNN for spatiotemporal dependencies. Experiments on six months of Shenzhen data show that DMSP reduces MAPE by over 4% compared to existing methods.

Mobile edge computing (MEC) has revolutionized the way computational tasks are offloaded and latency is reduced by leveraging edge servers close to end devices. Efficient resource allocation and task offloading are crucial for enhancing system performance in MEC environments. Traditional reinforcement learning (RL) approaches have shown promise in optimizing resource allocation and task offloading problems. However, they often face challenges such as high computational complexity and the need for extensive training data. Quantum reinforcement learning (QRL) emerges as a promising solution to overcome these limitations by leveraging quantum computing principles to enhance efficiency and scalability. In this paper, we propose a hybrid quantum-classical non-sequential model for joint resource allocation and task offloading in MEC systems. Our model combines the advantages of RL in handling environmental dynamics and quantum computing in reducing adjustable parameters and accelerating the training process. Extensive experiments demonstrate that our proposed algorithm can achieve higher training and inference performance under various parameter settings compared to traditional RL models and previous QRL models.

The shared replay buffer is the core of synergy in evolutionary reinforcement learning (ERL). Existing methods overlooked the objective conflict between population evolution in evolutionary algorithm and ERL, leading to poor quality of the replay buffer. In this article, we propose a strategic ERL algorithm with operator selection and experience filter (SERL-OS-EF) to address the objective conflict issue and improve the synergy from three aspects: 1) an operator selection strategy is proposed to enhance the performance of all individuals, thereby fundamentally improving the quality of experiences generated by the population; 2) an experience filter is introduced to filter the experiences obtained from the population, maintaining the long-term high quality of the buffer; and 3) a dynamic mixed sampling strategy is introduced to improve the efficiency of RL agent learning from the buffer. Experiments in four MuJoCo locomotion environments and three Ant-Maze environments with deceptive rewards demonstrate the superiority of the proposed method. In addition, the practical significance of the proposed method is verified on a low-carbon multienergy microgrid (MEMG) energy management task.

As Large Language Models (LLMs) continue to advance, their parameter sizes are growing exponentially-far outpacing hardware capabilities. This widening gap necessitates distributed computing through pipeline parallelism for efficient inference. However, the uneven distribution of requests across pipeline stages creates significant performance bottlenecks in real-world deployments. To address this challenge, we presentPlanck, a performance optimization framework specifically designed for distributed LLM inference.Planck implements fine-grained control through two key mechanisms: a progressive SLO allocation strategy that dynamically adjusts time constraints based on workload patterns, and stage-specific performance controllers that prevent bottlenecks before they cascade through the system. By intelligently balancing resources across pipeline stages,Planck effectively eliminates queue buildup-essentially preventing traffic congestion before it forms. Evaluation using diverse workloads in real cloud environments demonstrates thatPlanck reduces P99 tail latency by up to 18% and decreases the longest queue lengths by as much as 47.8% across pipeline stages, significantly improving both system responsiveness and resource utilization.

Cloud native solutions are widely applied in various fields, placing higher demands on the efficient management and utilization of resource platforms. To achieve the efficiency, load forecasting and elastic scaling have become crucial technologies for dynamically adjusting cloud resources to meet user demands and minimizing resource waste. However, existing prediction-based methods lack comprehensive analysis and integration of load characteristics across different time scales. For instance, long-term trend analysis helps reveal long-term changes in load and resource demand, thereby supporting proactive resource allocation over longer periods, while short-term volatility analysis can examine short-term fluctuations in load and resource demand, providing support for real-time scheduling and rapid response. In response to this, our research introduces TempoScale, which aims to enhance the comprehensive understanding of temporal variations in cloud workloads, enabling more intelligent and adaptive decision-making for elastic scaling. TempoScale utilizes the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise algorithm to decompose time-series load data into multiple Intrinsic Mode Functions (IMF) and a Residual Component (RC). First, we integrate the IMF, which represents both long-term trends and short-term fluctuations, into the time series prediction model to obtain intermediate results. Then, these intermediate results, along with the RC, are transferred into a fully connected layer to obtain the final result. Finally, this result is fed into the resource management system based on Kubernetes for resource scaling. Our proposed approach can reduce the Mean Square Error by 5.80% to 30.43% compared to the baselines, and reduce the average response time by 5.58% to 31.15%. The results demonstrate the effectiveness of our proposed method in reducing violations of service-level objectives and providing better performance in terms of resource utilization.

Publishing commuting trajectory data, including information of home and workplace locations, commuting distances and working hours, provides valuable insights for urban transportation planning. However, the data also contains sensitive personal information, raising privacy concerns even after the removal of unique identifiers. While traditional privacy-preserving methods, such as k-anonymity and differential privacy, have been widely applied, they mainly focus on single-trip travel patterns (e.g., point sequences or paths) and fail to capture the unique characteristics of commuting behavior. In this paper, we propose RCDP, a novel differential privacy-based model for synthesizing realistic commuting data using a prefix tree structure. RCDP introduces two key innovations: (1) it models round-trip commuting patterns through adaptive spatio-temporal generalization and a prefix tree, ensuring that the synthesized data retains the key commuting characteristics; (2) it employs a hierarchical privacy budget allocation mechanism that dynamically adjusts the budget across tree levels, along with a distribution-based node insertion method to maintain tree consistency, effectively balancing privacy and utility. Validation using public transport smart card data in Shenzhen, China demonstrates that RCDP outperforms existing k-anonymity and differential privacy approaches in preserving essential commuting features while ensuring strong privacy protection.

Model inference in edge is often regarded as an effective method to alleviate high latency and enhance data privacy in edge-cloud collaborative computing environment. In this paper, we demonstrate that optimizing network communication in edge-cloud environment with limited bandwidth can enhance model inference performance. We first analyze network bottlenecks and the characteristics in model inference, then design a serverless inference system - EINS, to support collaborative optimization of network transmission and inference performance in edge-cloud environment. This system identifies concurrent network communication bottlenecks in multi-model deployment, dynamically scales capacity, and optimizes placement strategies and model transfer sequences. Real-world workload evaluation reveal that EINS can achieve a 5.7x throughput improvement and an average reduction of 62% latency in model instance startup.

Pipeline parallelism is a key mechanism to ensure the performance of large model serving systems. These systems need to deal with unpredictable online workloads with low latency and high good put. However, due to the specific characteristics of large models and resource constraints in pipeline parallelism, existing systems struggle to balance resource allocation across pipeline stages. The primary challenge resides in the differential distribution of requests across various stages of the pipeline. We propose QUART, a large model serving system that focuses on optimizing the performance of key stages in pipeline parallelism. QUART dynamically identifies the key stages of the pipeline and introduces an innovative two-level model parameter caching system based on forks to achieve rapid scaling of key stages within seconds. In evaluations with real-world request workloads, QUART reduces average response latency by up to 87.1%) and increases good put by 2.37x compared to the baseline. The experiments demonstrate that QUART effectively reduces tail latency and the average queue length of the pipeline.

Through providing cheap rack and network hosting services, third-party internet data centers (IDCs) have gained significant popularity among cloud service providers. Real-time monitoring of the quality of the IDC network and proactively alarming is crucial to guaranteeing the reliability of cloud services. The prevailing approach to addressing this problem involves utilizing active probes and making evaluations based on the results of single-link or multi-link probing. However, the existing efforts still tend to generate a significant number of unnecessary alerts, resulting in enormous operational costs. For this reason, we first build a large-scale distributed ping-based dial test system that enables monitoring the quality of the IDC network in a many-to-one probe mode. We develop an efficient exporter tool based on the standard Prometheus' data interface to ensure real-time and precise measurement data collection. To quickly and accurately detect potential network issues, we also design a multi-step heuristic-based fault detection and alarm method. Furthermore, we propose a comprehensive alarm life-cycle model based on the results of multi-link probing to guide alarm management in production practice. This system has been successfully deployed in the production environment of Sangfor company's managed cloud for over a year, enabling proactive diagnosis of hundreds of IDC gateway IP addresses. The actual statistical results indicate a significant improvement in the mean time to repair (MTTR) for IDC network failures, reducing it from a few hours to just a few minutes. The average daily number of alarms generated by this system is less than 15, decreasing approximately 85 % compared to before. The alarm accuracy exceeds 95 % and the false negative rate is less than 2 % ■

Pipeline parallelism is an important strategy for improving inference performance in Large Language Models (LLMs). However, we find that different stages of LLM pipelines exhibit distinct performance and request characteristics, posing challenges to system performance in online inference scenarios. To address this issue, we propose Planck, a performance optimization framework tailored for LLM pipeline inference. By balancing request traffic, queue length, and execution time at each stage, Planck introduces a progressive SLO (Service Level Objective) allocation method and a stage instance performance controller. Planck fine-grainedly allocates SLOs to each pipeline stage and dynamically adjusts according to request distribution to control queue length. Through optimizing queue lengths across different stages of the model pipeline, Planck effectively reduces waiting time and tail latency. Evaluations conducted on a real cloud cluster using diverse workloads demonstrate that Planck effectively reduces P99 latency and queue length for each pipeline stage.

Ensuring efficient resource allocation while providing service level agreement (SLA) guarantees for end-to-end (E2E) latency is crucial for microservice applications. Although existing studies have made significant contributions towards achieving this objective, they primarily concentrate on static graphs. However, microservice graphs are inherently dynamic during runtime in production environments, necessitating more effective and scalable resource management solutions.In this paper, we present Derm, a new resource management system designed for microservice applications with highly dynamic graphs. Our principal finding is that prioritizing different microservice graphs can lead to a substantial reduction in resource allocation. To take advantage of this opportunity, we develop three main components. The first is a performance model that describes uncertainties of microservice latency through a conditional exponential distribution. The second is a probabilistic quantification of the dynamics of microservice graphs. The third is an optimization method for adjusting the resource allocation of microservices to minimize resource usage. We evaluate Derm in our cluster using real microservice benchmarks and production traces. The results highlight that Derm reduces the resource usage by $68.4 \%$ and lowers SLA violation probability by $6.7 \times$, compared to existing approaches.

Service Level Objectives (SLOs) aim to set threshold for service time in cloud services to ensure acceptable quality of service (QoS) and user satisfaction. Currently, many studies consider SLOs as a system resource to be allocated, ensuring QoS that to meet the SLOs. Existing microservice auto-scaling frameworks that rely on SLO resources often utilize complex and computationally intensive models, requiring significant time and resources to determine appropriate resource allocation. This paper aims to rapidly allocate SLO resources and minimize resource costs while ensuring application QoS meets the SLO requirements in a dynamically changing microservice environment. We propose MSARS, a framework that leverages meta-learning to quickly derive SLO resource allocation strategies and employs reinforcement learning for adaptive scaling of microservice resources. It features three innovative components: first, MSARS uses graph convolutional networks to predict the most suitable SLO resource allocation scheme for the current environment. Second, MSARS utilizes meta-learning to enable the graph neural network to quickly adapt to environmental changes ensuring adaptability in highly dynamic microservice environments. Third, MSARS generates auto-scaling policies for each microservice based on an improved Twin Delayed Deep Deterministic Policy Gradient (TD3) model. The adaptive auto-scaling policy integrates the SLO resource allocation strategy into the scheduling algorithm to satisfy SLOs. Finally, we compare MSARS with state-of-the-art resource auto-scaling algorithms that utilize neural networks and reinforcement learning, MSARS takes 40% less time to adapt to new environments, 38% reduction of SLO violations, and 8% less resources cost.

The deployment of ML serving applications, featuring multiple inference functions on serverless platforms, has gained substantial popularity, leading to numerous developments of new systems. However, these systems often focus on optimizing resource provisioning and cold start management separately, ultimately resulting in higher monetary costs. This paper introduces SMIless, a highly efficient serverless system tailored for serving DAG-based ML inference in heterogeneous environments. SMIless effectively co-optimizes resource configuration and cold-start management in the context of dynamic invocations. This is achieved by seamlessly integrating adaptive pre-warming windows, striking an effective balance between performance and cost. We have implemented SMIless on top of OpenFaaS and conducted extensive evaluations using real-world ML serving applications. The experimental results demonstrate that SMIless can achieve up to a $5.73 \times$ reduction in the overall costs while meeting the SLA requirements for all user requests, surpassing the performance of state-of-the-art solutions.

With the advancement of new industrialization, production processes or scenarios characterized by automation and intelligence require monitoring by more powerful visual inspection tools to maintain production continuity, safety, and efficiency. Spike vision sensors, with their high temporal resolution and high dynamic range, can capture subtle changes of high-speed moving objects in complex industrial environments, providing precise data support for equipment maintenance, fault detection, and condition monitoring. However, in industrial scenarios, factors such as mechanical vibrations, electromagnetic interference, and extreme strong light affect the quality of sensor data, leading to higher noise levels and increased probability of data loss and errors. To address this, we propose a multi-stage processing method for spike data processing and image reconstruction. It first performs preliminary feature extraction and denoising through an anti-noise feature encoder. Building on this, it integrates features and enhances consistency through spatiotemporal information utilization. Finally, it reconstructs the images through an fine-tuned spike reconstruction model. Experimental results show that our method can effectively reduce noise and improve data quality, achieving good performance on industrial datasets. This provides higher quality data support for downstream tasks such as predictive maintenance and high-speed production inspection in industrial scenarios.

Summary Real‐time estimation of crowd counting in underground metro systems, constrained by limited space, is crucial for managing heightened pedestrian volumes and responding promptly to emergencies. To address this challenge, we propose a passenger state transition‐based model, called STRmt, designed for the seamless and continuous monitoring of real‐time crowd movement within service areas of stations and trains, leveraging auto fare collection systems (AFC) as a comprehensive sensor network. Our innovation lies in modeling the dynamic movement of passengers within a metro system over time as a state transition process aligned with the train schedule. To achieve this, we introduce a spatio‐temporal deep learning framework, denoted as STnet, designed to dynamically predict these state transitions. The performance of our method is rigorously assessed through extensive experiments conducted spanning 2 years in Shenzhen, China, utilizing AFC data, train schedule data, and weather data. The results demonstrate that the proposed method surpasses baseline methods, achieving an estimation precision of 0.92.

A common approach to improving resource utilization in data centers is to adaptively provision resources based on the actual workload. One fundamental challenge of doing this in microservice management frameworks, however, is that different components of a service can exhibit significant differences in their impact on end-to-end performance. To make resource management more challenging, a single microservice can be shared by multiple online services that have diverse workload patterns and SLA requirements. We present an efficient resource management system, namely Erms, for guaranteeing SLAs with high probability in shared microservice environments. Erms profiles microservice latency as a piece-wise linear function of the workload, resource usage, and interference. Based on this profiling, Erms builds resource scaling models to optimally determine latency targets for microservices with complex dependencies. Erms also designs new scheduling policies at shared microservices to further enhance resource efficiency. Experiments across microservice benchmarks as well as trace-driven simulations demonstrate that Erms can reduce SLA violation probability by 5× and more importantly, lead to a reduction in resource usage by 1.6×, compared to state-of-the-art approaches.

Deploying federated learning (FL) in edge clouds poses challenges, especially when multiple models are concurrently trained in resource-constrained edge environments. Existing research on federated edge learning has predominantly focused on client selection for training a single FL model, typically with a fixed learning topology. Preliminary experiments indicate that FL models with adaptable topologies exhibit lower learning costs compared to those with fixed topologies. This paper delves into the intricacies of jointly selecting participants and learning topologies for multiple FL models simultaneously trained in the edge cloud. The problem is formulated as an integer non-linear programming problem, aiming to minimize total learning costs associated with all FL models while adhering to edge resource constraints. To tackle this challenging optimization problem, we introduce a two-stage algorithm that decouples the original problem into two sub-problems and iteratively addresses them separately with efficient heuristics. Our method enhances resource competition and load balancing in edge clouds by allowing FL models to choose participants and learning topologies independently. Extensive experiments conducted with real-world networks and FL datasets affirm the better performance of our algorithm, demonstrating lower average total costs with up to 33.5% and 39.6% compared to previous methods designed for multi-model FL.

Microservices have transformed monolithic applications into lightweight, self-contained, and isolated application components, establishing themselves as a dominant paradigm for application development and deployment in public clouds such as Google and Alibaba. Autoscaling emerges as an efficient strategy for managing resources allocated to microservices’ replicas. However, the dynamic and intricate dependencies within microservice chains present challenges to the effective management of scaled microservices. Additionally, the centralized autoscaling approach can encounter scalability issues, especially in the management of large-scale microservice-based clusters. To address these challenges and enhance scalability, we propose an innovative distributed resource provisioning approach for microservices based on the Twin Delayed Deep Deterministic Policy Gradient algorithm. This approach enables effective autoscaling decisions and decentralizes responsibilities from a central node to distributed nodes. Comparative results with state-of-the-art approaches, obtained from a realistic testbed and traces, indicate that our approach reduces the average response time by 15% and the number of failed requests by 24%, validating improved scalability as the number of requests increases.

Mobile edge computing emerges to serve mobile users with low-latency computation offloading in edge networks, which are resource-constrained with massive users and workloads. However, existing communication and computing resource allocation schemes for offloaded tasks aren't efficient enough, where finished tasks still occupy resources, wasting constrained resources. Besides, the multi-user offloading is usually for scenarios of one task per user, ignoring real-worldmulti-taskoffloading scenarios where each user has multiple tasks, lack generality and flexibility. Meanwhile, local computing resource allocation schemes in multi-task scenarios ignore resource readjustment, causing low resource utilization. To solve these problems, we propose ECO-GAME, an efficient multi-task offloading scheme, which dynamically allocates bandwidth and computing resources to unfinished tasks, resulting in high resource utilization. We initially formulate the multi-task offloading problem as the game minimizing each user's cost, which is NP-hard. Thus we re-formulate the game utilizing potential games to optimize user's objective either locally or globally, and prove the existence of its Nash equilibrium. We then design an efficient multi-task offloading algorithm to obtain an approximate solution in polynomial time, together with computational complexity analysis. We further conduct performance evaluation on ECO-GAME utilizing price of anarchy. Numerical results demonstrate the efficiency of ECO-GAME, and show ECO-GAME reduces 49.2% cost over the state-of-the-art work, and scales well with the increasing number of tasks and users.

A common approach to improving resource utilization in data centers is to adaptively provision resources based on the actual workload. One fundamental challenge of doing this in microservice management frameworks, however, is that different components of a service can exhibit significant differences in their impact on end-to-end performance. To make resource management more challenging, a single microservice can be shared by multiple online services that have diverse workload patterns and SLA requirements.

To fully utilize computing resources, cloud providers such as Google and Alibaba choose to co-locate online services with batch processing applications in their data centers. By implementing unified resource management policies, different types of complex computing jobs request resources in a consistent way, which can help data centers achieve global optimal scheduling and provide computing power with higher quality. To understand this new scheduling paradigm, in this paper, we first present an in-depth study of Alibaba's unified scheduling workloads. Our study focuses on the characterization of resource utilization, the application running performance, and scheduling scalability. We observe that although computing resources are significantly over-committed under unified scheduling, the resource utilization in Alibaba data centers is still low. In addition, existing resource usage predictors tend to make severe overestimations. At the same time, tasks within the same application behave fairly consistently, and the running performance of tasks can be well-profiled with respect to resource contention on the corresponding physical host.

Previous work on adversarial examples typically involves a fixed norm perturbation budget, which fails to capture the way humans perceive perturbations. Recent work has shifted towards natural unrestricted adversarial examples (UAEs) that breaks ℓpperturbation bounds but nonetheless remain semantically plausible. Current methods use GAN or VAE to generate UAEs by perturbing latent codes. However, this leads to loss of high-level information, resulting in low-quality and unnatural UAEs. In light of this, we propose AdvDiffuser, a new method for synthesizing natural UAEs using diffusion models. It can generate UAEs from scratch or conditionally based on reference images. To generate natural UAEs, we perturb predicted images to steer their latent code towards the adversarial sample space of a particular classifier. We also propose adversarial inpainting based on class activation mapping to retain the salient regions of the image while perturbing less important areas. On CIFAR-10, CelebA and ImageNet, we demonstrate that it can defeat the most robust models on the RobustBench leaderboard with near 100% success rates. Furthermore, The synthesized UAEs are not only more natural but also stronger compared to the current state-of-the-art attacks. Specifically, compared with GA-attack, the UAEs generated with AdvDiffuser exhibit 6× smaller LPIPS perturbations, 2 ~ 3× smaller FID scores and 0.28 higher in SSIM metrics, making them perceptually stealthier. Finally, adversarial training with AdvDiffuser further improves the model robustness against attacks with unseen threat models.1

Low response latency holds a pivotal role in the landscape of edge deep learning model inference, yet the constrained resources within edge computing environments often limit its full potential. Within the scope of this paper, we substantiate that even amid the resource limitations inherent to edge computing environments, it remains feasible to curtail model response latency by elevating multi-core parallel efficiency. Our research encompasses a comprehensive analysis of the parallel acceleration effects observed across models featuring diverse parameter magnitudes during the inference process. This analysis culminates in the development of FLASH, an online deep model inference system tailored for Serverless edge inference, strategically optimized through the utilization of multi-core parallelism. FLASH exhibits dynamic adaptability by modulating the number of CPU cores within computational instances in accordance with traffic request loads. It also employs a dynamic scaling mechanism to finely adjust model placement, ultimately facilitating inference acceleration and mitigating the concomitant cold start overhead. Empirical experimentation conducted across a spectrum of burst-level workloads serves to underscore FLASH’s capacity, resulting in an average reduction in response latency by 33% and a maximum reduction of 75%, while concurrently realizing a throughput enhancement of 2.94x.

Diffusion models have become popular in computer vision applications due to their ability to generate high-quality images quickly, with some models achieving a high degree of realism. However, the use of these models for novel image manipulation and generation poses significant risks of being used for harmful purposes, such as the spread of misinformation or copyright infringement. Recent research has proposed techniques that involve adding imperceptible perturbations to images to mitigate privacy and copyright concerns. However, trivial image processing methods can largely destroy these protective perturbations. In this paper, we propose a new method called DiffCleaner, a destruction-restoration pipeline, that combines the advantages of image corruption and restoration methods. It can not only remove protective perturbations but also further preserve high quality generation, as well as fidelity of the original features in generated images. DiffCleaner produces specialized images with visualization and quantitative metrics closely matching unprotected results. We pitch a range of circumventing methods vs. two state-of-the-art data protection methods against diffusion models, and show that both are fragile and offer inadequate data protection. By providing a benchmark comprising a range of protective perturbations removal methods, we hope this paper will inspire and facilitate new research towards developing more resilient methods for data protection.

Although the importance of using static taint analysis to detect taint-style vulnerabilities in Linux-based embedded firmware is widely recognized, existing approaches are plagued by following major limitations: (a) Existing works cannot properly handle indirect call on the path from attacker-controlled sources to security-sensitive sinks, resulting in lots of false negatives. (b) They employ heuristics to identify mediate taint source and it is not accurate enough, which leads to high false positives.

A common strategy for improving efficiency in training deep learning entails multiplexing tasks on a single GPU. To mitigate the interference caused by multiplexing, existing approaches primarily employ kernel-level solutions to regulate GPU kernel execution, or harness hardware-level techniques to explicitly restrict GPU streaming multiprocessors and memory. Nevertheless, none of them perform satisfactorily in optimizing the completion time of tasks.

Data scarcity hinders the usability of data-dependent algorithms when tackling IoT intrusion detection (IID). To address this, we utilize the data-rich network intrusion detection (NID) domain to facilitate more accurate intrusion detection for IID domains. In this article, a geometric graph alignment (GGA) approach is leveraged to mask the geometric heterogeneities between domains for better intrusion knowledge transfer. Specifically, each intrusion domain is formulated as a graph where vertices and edges represent intrusion categories and category-wise inter-relationships, respectively. The overall shape is preserved via a confused discriminator incapable to identify adjacency matrices between different intrusion domain graphs. A rotation avoidance mechanism and a center point matching mechanism are used to avoid graph misalignment due to rotation and symmetry, respectively. Besides, category-wise semantic knowledge is transferred to act as vertex-level alignment. To exploit the target data, a pseudo-label (PL) election mechanism that jointly considers network prediction, geometric property, and neighborhood information is used to produce fine-grained PL assignment. Upon aligning the intrusion graphs geometrically from different granularities, the transferred intrusion knowledge can boost IID performance. Comprehensive experiments on several intrusion data sets demonstrate state-of-the-art performance of the GGA approach and validate the usefulness of GGA-constituting components.

In this article, we propose a joint semantic transfer network (JSTN) toward effective intrusion detection (ID) for large-scale scarcely labeled Internet of Things (IoT) domain. As a multisource heterogeneous domain adaptation (MS-HDA) method, the JSTN integrates a knowledge-rich network intrusion (NI) domain and another small-scale IoT intrusion (II) domain as source domains and preserves intrinsic semantic properties to assist target II domain ID. The JSTN jointly transfers the following three semantics to learn a domain-invariant and discriminative feature representation. The scenario semantic endows source NI and II domains with characteristics from each other to ease the knowledge transfer process via a confused domain discriminator and categorical distribution knowledge preservation. It also reduces the source–target discrepancy to make the shared feature space domain invariant. Meanwhile, the weighted implicit semantic transfer boosts discriminability via a fine-grained knowledge preservation, which transfers the source categorical distribution to the target domain. The source–target divergence guides the importance weighting during knowledge preservation to reflect the degree of knowledge learning. Additionally, the hierarchical explicit semantic alignment performs centroid-level and representative-level alignment with the help of a geometric similarity-aware pseudo-label refiner, which exploits the value of the unlabeled target II domain and explicitly aligns feature representations from a global and local perspective in a concentrated manner. Comprehensive experiments on various tasks verify the superiority of the JSTN against state-of-the-art comparing methods, on average a 10.3% of accuracy boost is achieved. The statistical soundness of each constituting component and the computational efficiency is also verified.

Short-term Origin-Destination(OD) matrix prediction in metro systems aims to predict the number of passenger demands from one station to another during a short time period. That is crucial for dynamic traffic operations, e.g., route recommendation, metro scheduling. However, existing methods need further improvement due to that they fail to take full use of the real-time traffic information and model the complex spatiotemporal correlation of traffic flows. In this paper, aMulti-ViewPassengerFlow (MVPF) evolution trend based OD matrix prediction method is proposed. It consists of two components focusing on individual station and cross-station learning. Specifically, the individual station level part uses Gate Recurrent Unit and Extended Graph Attention Networks combined model to learn the high-level spatiotemporal-dependent representation of each station as the roles of origin and destination respectively, by considering multiple views of real-time traffic information (i.e., Inflow, destination allocation of Inflow, Outflow, origin allocations of Outflow). The cross-station part aims to learn passenger mobility pattern from each origin to destination through defining a transition matrix under spatiotemporal context. Compared with state-of-the-art solutions, MVPF increases the OD prediction performance metric of WMAPE by 2.5% on average. The experimental results demonstrate the superiority of MVPF against other competitors. The source code is available athttps://github.com/zfrInSIAT/MVPF-code.

Flow scheduling and congestion control are two important techniques to reduce flow completion time in data center networks. While existing works largely treat them independently, the interactions between flow scheduling and congestion control are in general overlooked which leads to sub-optimal solutions, especially given that the link capacity is increasing faster than the switch port buffer size. In this paper, we presentFlash, a simple yet effective scheme that integrates scheduling and congestion control. Specifically,Flashputs forward a congestion-aware scheduling scheme to determine the priority of flows based on the latest network congestion extent and the flow’s bytes sent. Besides,Flashproposes a priority-based packet dropping scheme in switch port buffers and implements a priority-aware congestion control scheme. Experiment results show thatFlashhas superior performance: (1) it has 35.8% lower tail latency than PIAS and performs similar with pFabric in a 10G network without knowing the flow size, (2) in 100G networks with shallow buffers, the information agnosticFlashhas 6.8% lower average FCT than the information-aware pFabric, (3) it outperforms pFabric by 13.5% in FCT if flow size is also known toFlash.

Traffic classification is essential for cybersecurity maintenance and network management, and has been widely used in QoS (Quality of Service) guarantees, intrusion detection, and other tasks. Recently, with the emergence of SSL/TLS encryption protocols in the modern Internet environment, the traditional payload-based classification methods are no longer effective. Some researchers have used machine learning methods to model the flow features of encrypted traffics (e.g. message type, length sequence, statistical features, etc.), and achieved good results in some cases. However, these high-level hand-designed features cannot be used for more fine-grained operations and may lead to the loss of important information, thus affecting the classification accuracy. To overcome this limitation, in this paper, we designed a novel multimodal deep learning framework for encrypted traffic classification called PEAN. PEAN uses the raw bytes and length sequence as the input, and uses the self-attention mechanism to learn the deep relationship among network packets in a biflow. Furthermore, unsupervised pre-training was introduced to enhance PEAN’s ability to characterize network packets. Experiments on a real trace set captured in a large data center demonstrate the effectiveness of PEAN, which achieves better results than the state-of-the-art methods.

Serverless computing is growing in popularity by virtue of its lightweight and simplicity of management. It achieves these merits by reducing the granularity of the computing unit to the function level. Specifically, serverless allows users to focus squarely on the function itself while leaving other cumbersome management and scheduling issues to the platform provider, who is responsible for striking a balance between high-performance scheduling and low resource cost. In this article, we conduct a comprehensive survey of serverless computing with a particular focus on its infrastructure characteristics. Whereby some existing challenges are identified, and the associated cutting-edge solutions are analyzed. With these results, we further investigate some typical open-source frameworks and study how they address the identified challenges. Given the great advantages of serverless computing, it is expected that its deployment would dominate future cloud platforms. As such, we also envision some promising research opportunities that need to be further explored in the future. We hope that our work in this article can inspire those researchers and practitioners who are engaged in related fields to appreciate serverless computing, thereby setting foot in this promising area and making great contributions to its development.

Deep learning is one of the most widely used modeling approach for spatio-temporal prediction. Traditional deep learning models get discrete features or hidden states from discrete data. However such discretization method cannot work well if the data is irregularly sampled or incompletely observed. It is difficult to obtain continuous features and hidden states for deep learning models. In this paper, we demonstrate how to use neural differential equations to solve this problem. We first validated the ability of Neural Controlled Differential Equations (Neural CDE) to process real-world irregular spatiotemporal data. Neural CDE is a type of neural differential equation that can obtain continuous hidden states. To make Neural CDE can obtain the spatial correlation of spatio-temporal data, we combined Graph Attention Network (GAT) with Neural CDE to propose a new model for spatio-temporal prediction. Furthermore, we used neural differential equations to designed a module to reduce the error of the prediction results. Experimental results show our proposed model outperforms the standard baseline by up to 37.40%.

When deploying microservices in production clusters, it is critical to automatically scale containers to improve cluster utilization and ensure service level agreements (SLA). Although reactive scaling approaches work well for monolithic architectures, they are not necessarily suitable for microservice frameworks due to the long delay caused by complex microservice call chains. In contrast, existing proactive approaches leverage end-to-end performance prediction for scaling, but cannot effectively handle microservice multiplexing and dynamic microservice dependencies.

Predicting resource usage of workloads in large scale production clusters is very important to understand the characteristics of applications. It is also very important for cluster operators to manage cluster resources more efficiently. Traditional statistical-based prediction methods face challenges in predicting resource usage in large scale dynamic and complex clusters. The current commonly used deep learning methods such as Recurrent Neural Networks (RNN) usually use the historical data in single node to predict the future resource usage. While most of the modern applications (e.g., microservices) are distributively deployed to the cluster, the traditional single node resource prediction methods cannot predict the resource usages well. To solve this problem, in this paper we propose a new deep learning model called GraphGRU which is based on graph attention networks (GAT) to predict the resource usages from the cluster perspective. We use the Dynamic Time Warping (DTW) algorithm to construct a graph structure for multiple physical nodes in the cluster and also use a method similar to data compensation to co-train the model with both horizontal and vertical data. We validate our model on the Alibaba microservices dataset which is captured from a large scale production cluster. Compared to the traditional deep learning methods, our model improves the prediction accuracy by up to 48.27%.

Abstract As one of the most useful online processing techniques, the theta‐join operation has been utilized by many applications to fully excavate the relationships between data streams in various scenarios. As such, constant research efforts have been put to optimize its performance in the distributed environment, which is typically characterized by reducing the number of Cartesian products as much as possible. In this article, we design and implement a novel fast theta‐join algorithm, calledPrefap, by developing two distinct techniques—prefilteringandamalgamated partitioning—based on the state‐of‐the‐art FastThetaJoin algorithm to optimize the efficiency of the theta‐join operation. Firstly, we develop a prefiltering strategy before data streams are partitioned to reduce the amount of data to be involved and benefit a more fine‐grained partitioning. Secondly, to avoid the data streams being partitioned in a coarse‐grained isolated manner and improve the quality of the partition‐level filtering, we introduce an amalgamated partitioning mechanism that can amalgamate the partitioning boundaries of two data streams to assist a fine‐grained partitioning. With the integration of these two techniques into the existing FastThetaJoin algorithm, we design and implement a new framework to achieve a decreased number of Cartesian products and a higher theta‐join efficiency. By comparing with existing algorithms, FastThetaJoin in particular, we evaluate the performance ofPrefapon both synthetic and real data streams from two‐way to multiway theta‐join to demonstrate its superiority.

Abstract Semantic segmentation is a hot topic in computer vision and various deep learning networks are designed to achieve higher accuracy on that by fully exploring the capability of neural networks. This paper aims to address the issue and proposes the substructures with novelty for popular networks. Meanwhile, we present a cross‐channel structure, which simultaneously reduces parameter while the kernel size becomes larger. After that, to overcome the weakness of insufficient dataset which refers to satellite image data, we propose a feature attention mechanism with generative adversarial network to enhance the images' features. We show the recognition result on the satellite image dataset with a large picture. This paper evaluates substructures on the PASCAL VOC2012 dataset and improves the mIOU from 74.68% to 88.15%.

Abstract As a key technology in the 5G era, mobile edge computing (MEC) has developed rapidly in recent years. MEC aims to reduce the service delay of mobile users, while alleviating the processing pressure on the core network. MEC can be regarded as an extension of cloud computing on the user side, which can deploy edge servers and bring computing resources closer to mobile users, and provide more efficient interactions. However, due to the user's dynamic mobility, the distance between the user and the edge server will change dynamically, which may cause fluctuations in Quality of Service. Therefore, when a mobile user moves in the MEC environment, certain approaches are needed to schedule services deployed on the edge server to ensure the user experience. In this article, we model service scheduling in MEC scenarios and propose a delay‐aware and mobility‐aware service management approach based on concise probabilistic methods. This approach has low computational complexity and can effectively reduce service delay and migration costs. Furthermore, we conduct experiments by utilizing multiple realistic datasets and use iFogSim to evaluate the performance of the algorithm. The results show that our proposed approach can optimize the performance on service delay, with 8%–20% improvement and reduce the migration cost by more than 75% compared with baselines during the rush hours.

We present an extension of the bankers algorithm to resolve deadlock for programs whose resource-request graph can be modeled as a recursion tree for parallel execution. Our algorithm implements the bankers logic, with the key difference being that some properties of the tree are fully exploited to improve the resource utilization and safety check in deadlock avoidance. For an n-node tree modeled program making requests to m types of resources, our recursion-tree based algorithm can obtain a time complexity of O(mn loglogn) on average in safety check while reducing the conservativeness in resource utilization. We reap these benefits by proposing a concept of the resource critical tree and leverage it to localize the maximum claim associated with each node in the tree. To tackle the case when the tree model is not statically known, we relax the definition of a local maximum claim by sacrificing some resource utilization. With this trade-off, the algorithm can resolve the deadlock and achieve more efficient safety checks within time of O(m loglogn). Our empirical studies on a two-dimensional integration problem on sparse grids show that the proposed algorithms can reduce resource utilization conservativeness and improve avoidance performance by minimizing the number of safety checks.

Reducing tail latency becomes increasingly important to improve the user-perceived service experience. User-facing latency-sensitive cloud applications typically contain multiple interactive tiers (e.g., Web, App, Database) running in different virtual machines (VMs) with complex interaction patterns. However, such interactions between VMs in different tiers are often neglected in previous VM consolidation methods, resulting in poor application performance. In this article, we study the consolidation of multi-tier interactive workloads from a new perspective of user-perceived tail latency. We propose a novel profiling-based consolidation methodology to satisfy tail latency requirements while reducing the number of used physical machines. To achieve such a goal, we first perform large-scale profiling experiments under various consolidation settings in a KVM virtualized private cluster to establish the empirical performance values. We consider two key factors that affect the tail latency of multi-tier workloads:interferencewith co-located VMs andinteractionbetween tiers. We model the consolidation of multi-tier workloads as an optimization problem with different objectives and constraints, and derive the consolidation schedule. We implement and evaluate the proposed models, as well as comparing with other methods (i.e.,withoutprofiling orwithoutconsidering interaction influence). Extensive experimental results show that the proposed method is able to reduce up to5Xtail latency, compared with the methodwithoutprofiling and up to1.3Xtail latency, compared with the methodwithoutconsidering the interaction influence between different tiers.

In recent years, various deep learning architectures have been proposed to solve complex challenges (e.g. spatial dependency, temporal dependency) in traffic domain, which have achieved satisfactory performance. These architectures are composed of multiple deep learning techniques in order to tackle various challenges in traffic tasks. Traditionally, convolution neural networks (CNNs) are utilized to model spatial dependency by decomposing the traffic network as grids. However, many traffic networks are graph-structured in nature. In order to utilize such spatial information fully, it’s more appropriate to formulate traffic networks as graphs mathematically. Recently, various novel deep learning techniques have been developed to process graph data, called graph neural networks (GNNs). More and more works combine GNNs with other deep learning techniques to construct an architecture dealing with various challenges in a complex traffic task, where GNNs are responsible for extracting spatial correlations in traffic network. These graph-based architectures have achieved state-of-the-art performance. To provide a comprehensive and clear picture of such emerging trend, this survey carefully examines various graph-based deep learning architectures in many traffic applications. We first give guidelines to formulate a traffic problem based on graph and construct graphs from various kinds of traffic datasets. Then we decompose these graph-based architectures to discuss their shared deep learning techniques, clarifying the utilization of each technique in traffic tasks. What’s more, we summarize some common traffic challenges and the corresponding graph-based deep learning solutions to each challenge. Finally, we provide benchmark datasets, open source codes and future research directions in this rapidly growing field.

In this paper, we investigate an efficient way for identifying passengers in a metro system across two heterogeneous but complementary trajectory data sources: AFC data recording two points per trip about when and where a passenger enters or leaves the metro system, and WiFi data recording a few points passed by a passenger in the way of some of his/her trips. The identification result can help us to complete individuals’ mobility, and benefits to lots of services, e.g., individual route choice analysis, epidemic case detecting and so on. The problem is similar to calculate the similarity between two trajectories from two data sources, where a trajectory refers to a sequence of points where a passenger appeared in a metro system on observed days. However, due to the small location space in a metro network, large number of passengers with similar travel pattern, and so on, there are lots of trip overlaps or point co-occurrences between different passengers. That results in a large number of passengers mismatched by existing trajectory similarity measurement. To address the problem, this paper proposes a novel global-local correlation based trajectory similarity measurement GLTC. Specifically, GLTC first extracts all overlapping trip pairs of two trajectories by considering the spatiotemporal inclusions from global level. Then it gets the similarity by aggregating each overlapping trip pair’s local similarity, which is calculated by considering some data-driven insights helpful to uniquely identify a passenger (e.g., uneven passenger flow distribution in different cross-sections of a metro network, the number of trips in same travel pattern of a trajectory, and so on). We evaluate GLTC based on real-world data, and the experimental result shows that GLTC outperforms other baselines.

The emerging trend towards moving from monolithic applications to microservices has raised new performance challenges in cloud computing environments. Compared with traditional monolithic applications, the microservices are lightweight, fine-grained, and must be executed in a shorter time. Efficient scaling approaches are required to ensure microservices’ system performance under diverse workloads with strict Quality of Service (QoS) requirements and optimize resource provisioning. To solve this problem, we investigate the trade-offs between the dominant scaling techniques, including horizontal scaling, vertical scaling, and brownout in terms of execution cost and response time. We first present a prediction algorithm based on gradient recurrent units to accurately predict workloads assisting in scaling to achieve efficient scaling. Further, we propose a multi-faceted scaling approach using reinforcement learning called CoScal to learn the scaling techniques efficiently. The proposed CoScal approach takes full advantage of data-driven decisions and improves the system performance in terms of high communication cost and delay. We validate our proposed solution by implementing a containerized microservice prototype system and evaluated with two microservice applications. The extensive experiments demonstrate that CoScal reduces response time by 19%-29% and decreases the connection time of services by 16% when compared with the state-of-the-art scaling techniques for Sock Shop application. CoScal can also improve the number of successful transactions with 6%-10% for Stan’s Robot Shop application.

Cloud computing has been regarded as a successful paradigm for IT industry by providing benefits for both service providers and customers. In spite of the advantages, cloud computing also suffers from distinct challenges, and one of them is the inefficient resource provisioning for dynamic workloads. Accurate workload predictions for cloud computing can support efficient resource provisioning and avoid resource wastage. However, due to the high-dimensional and high-variable features of cloud workloads, it is difficult to predict the workloads effectively and accurately. The current dominant work for cloud workload prediction is based on regression approaches or recurrent neural networks, which fail to capture the long-term variance of workloads. To address the challenges and overcome the limitations of existing works, we proposed an e fficient supervised learning-based D eep N eural Network ( esDNN ) approach for cloud workload prediction. First, we utilize a sliding window to convert the multivariate data into a supervised learning time series that allows deep learning for processing. Then, we apply a revised Gated Recurrent Unit (GRU) to achieve accurate prediction. To show the effectiveness of esDNN, we also conduct comprehensive experiments based on realistic traces derived from Alibaba and Google cloud data centers. The experimental results demonstrate that esDNN can accurately and efficiently predict cloud workloads. Compared with the state-of-the-art baselines, esDNN can reduce the mean square errors significantly, e.g., 15%. rather than the approach using GRU only. We also apply esDNN for machines auto-scaling, which illustrates that esDNN can reduce the number of active hosts efficiently, thus the costs of service providers can be optimized.

Loosely-coupled and light-weight microservices running in containers are replacing monolithic applications gradually. Understanding the characteristics of microservices is critical to make good use of microservice architectures. However, there is no comprehensive study about microservice and its related systems in production environments so far. In this paper, we present a solid analysis of large-scale deployments of microservices at Alibaba clusters. Our study focuses on the characterization of microservice dependency as well as its runtime performance. We conduct an in-depth anatomy of microservice call graphs to quantify the difference between them and traditional DAGs of data-parallel jobs. In particular, we observe that microservice call graphs are heavy-tail distributed and their topology is similar to a tree and moreover, many microservices are hot-spots. We also discover that the structure of call graphs for long-term developed applications is much simpler so as to provide better performance. Our investigation on microservice runtime performance indicates most microservices are much more sensitive to CPU interference than memory interference. Moreover, we design resource management policies to efficiently tune memory resources.

Urban traffic network is a typical complex network. Traffic states data (e.g., traffic flow, traffic occupancy, traffic speed, etc.) has strong temporal and spatial correlation. To accurately predict urban traffic state, it is very important to extract the road features in the traffic network. The existing methods use separated temporal and spatial components or Spatio-temporal fusion components to predict traffic. Graph Convolution Network (GCN) is usually used to obtain the correlation between spatial nodes or Spatio-temporal nodes. However, the message aggregation method of GCN cannot assign different weights to neighbor nodes. While Graph Attention Network (GAT) can pay attention to different neighbor nodes. To better explain the existing traffic prediction models, we carried out experiments on the model framework based on GCN. We use a new proposed GAT instead of GCN, and find that the new GAT has better performance in multi-features traffic prediction tasks. We also made a theoretical analysis on the improvement of the performance and carried out experiments on four real datasets, which can provide strong support for the theoretical analysis. Our method improves the interpretability of the Graph Neural Network (GNN) model in extracting spatial features of the traffic networks.

Loosely-coupled and light-weight microservices running in containers are replacing monolithic applications gradually. Understanding the characteristics of microservices is critical to make good use of microservice architectures. However, there is no comprehensive study about microservice and its related systems in production environments so far. In this paper, we present a solid analysis of large-scale deployments of microservices at Alibaba clusters. Our study focuses on the characterization of microservice dependency as well as its runtime performance. We conduct an in-depth anatomy of microservice call graphs to quantify the difference between them and traditional DAGs of data-parallel jobs. In particular, we observe that microservice call graphs are heavy-tail distributed and their topology is similar to a tree and moreover, many microservices are hot-spots. We reveal three types of meaningful call dependency that can be utilized to optimize microservice designs. Our investigation on microservice runtime performance indicates most microservices are much more sensitive to CPU interference than memory interference. To synthesize more representative microservice traces, we build a mathematical model to simulate call graphs. Experimental results demonstrate our model can well preserve those graph properties observed from Alibaba traces.

Resource management is challenging in clouds due to the dynamics and sharing characteristics. The crucial problem is how to allocate resources accurately and satisfy demands of workloads timely. The traditional solution is to use historical data to predict future resource usage. Although these resource prediction methods can predict the periodicity, they can not accurately predict mutation points due to the high dynamics and uncertainty of resource usage. To tackle this issue, in this paper we propose a resource usage prediction method - RPTCN, which is based on a deep learning method - temporal convolutional networks (TCNs) in cloud systems. We add a fully connected layer and attention mechanism to TCNs to improve the prediction accuracy. In order to explore the relationship between the usage of different resources in the temporal dimension, we use correlation analysis to screen performance indicators as multidimensional feature input for prediction. Finally, we evaluate the performance of this method on Alibaba trace v2018. Evaluations show that RPTCN improves the overall MAE and MSE by 6.50%~89.03% and 0.41%~68.82% respectively compared to baselines in dynamic and long-term prediction of resource usage. Moreover, the convergence and generalization of RPTCN are also better than the baselines.

The core network is the most basic facility in the entire telecommunications network, which is consists of large number of routers, switches and firewalls. Network management like re-planning routes or adjusting policies is very important to avoid failures. However, the timing of intervention is very challenging. Too early intervention will incur unnecessary overheads, and too late intervention will cause serious disaster. In this paper, we analyzed a large data set from a real-world core telecommunications network and proposed Linear Bayesian Neural Networks (LBNN)11Code available at https://github.com/YanyingLin/Lbnn to perceive the core network state changes and make decisions about network intervention. In particular, we considered three aspects of complexity, including the weight of the mutual effect between devices, the dependence on the time dimension of the network states, and the randomness of the network state changes. The entire model is extended to a probability model based on the Bayesian framework to better capture the randomness and variability of the data. Experimental results on real-world data set show that LBNN achieves very high detection accuracy, with an average of 92.1%.

With the improvement of data volume, computing power and algorithms, deep learning has achieved rapid development and showing excellent performance. Recently, many deep learning models are proposed to solve the problems in different areas. A recurrent neural network (RNN) is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit temporal dynamic behavior, which makes it applicable to tasks such as handwriting recognition or speech recognition. However, the RNN relies heavily on the automatic learning ability to update parameters that concentrate on the data flow but seldom considers the feature extraction capability of the gate mechanism. In this paper, we propose a novel architecture to build the forget gate which is generated by multiple bases. Instead of using the traditional single-layer fully-connected network, we use a Multiple Attention (MA) based network to generate the forget gate which refines the optimization space of gate function and improve the granularity of the recurrent neural network to approximate the map in the ground truth. Due to the benefit of MA structure on the gate mechanism, the proposed MA-LSTM model achieves better feature extraction capability than other known models.

Stock price movement prediction is commonly accepted as a very challenging task due to the volatile nature of financial markets. Previous works typically predict the stock price mainly based on its own information, neglecting the cross effect among involved stocks. However, it is well known that an individual stock price is correlated with prices of other stocks in complex ways. To take the cross effect into consideration, we propose a deep learning framework, called Multi-GCGRU, which comprises graph convolutional network (GCN) and gated recurrent unit (GRU) to predict stock movement. Specifically, we first encode multiple relationships among stocks into graphs based on financial domain knowledge and utilize GCN to extract the cross effect based on these pre-defined graphs. To further get rid of prior knowledge, we explore an adaptive relationship learned by data automatically. The cross-correlation features produced by GCN are concatenated with historical records and then fed into GRU to model the temporal dependency of stock prices. Experiments on two stock indexes in China market show that our model outperforms other baselines. Note that our model is rather feasible to incorporate more effective stock relationships containing expert knowledge, as well as learn data-driven relationship.

Graph Convolutional Network (GCN) is adopted to tackle the problem of convolution operation in non-Euclidean space. Previous works on GCN have made some progress, however, one of their limitations is that the design of Adjacency Matrix (AM) as GCN input requires domain knowledge and such process is cumbersome, tedious and error-prone. In addition, entries of a fixed Adjacency Matrix are generally designed as binary values (i.e., ones and zeros) which can not reflect the real relationship between nodes. Meanwhile, many applications require a weighted and dynamic Adjacency Matrix instead of an unweighted and fixed AM, and there are few works focusing on designing a more flexible Adjacency Matrix. To that end, we propose an end-to-end algorithm to improve the GCN performance by focusing on the Adjacency Matrix. We first provide a calculation method callednodeinformationentropyto update the matrix. Then, we perform the search strategy in a continuous space and introduce the Deep Deterministic Policy Gradient (DDPG) method to overcome the drawback of the discrete space search. Finally, we integrate the GCN and reinforcement learning into an end-to-end framework. Our method can automatically define the Adjacency Matrix without prior knowledge. At the same time, the proposed approach can deal with any size of the matrix and provide a better AM for network. Four popular datasets are selected to evaluate the capability of our algorithm. The method in this paper achieves the state-of-the-art performance onCoraandPubmeddatasets, with the accuracy of 84.6% and 81.6% respectively.

Graph Neural Networks (GNNs) have attracted increasing attention due to the significant representation learning capacity for graphs. The traffic forecasting is a typical graph representation learning task, but it is challenging to model the complex spatial and temporal relationships in traffics. Traditional spectral approaches get filters based on the eigendecomposition, which depends on the Laplacian matrix of the graph. However, these approaches have expensive matrix operation on graph convolutions neural networks and are insufficient to tackle the spatial dependency. In this paper, we propose a novel graph neural network - Spatial-Temporal Multi-head Graph ATtention network (ST-MGAT), to deal with the traffic forecasting problem. We build convolutions on the graph directly. We consider the features of neighborhood nodes and the weights of the edges to generate new node representation. More specifically, there are two main modules: i) Temporal convolution blocks to capture the dynamic time correlations; ii) Graph attention networks to capture the dynamic spatial relations between nodes. Experimental results show that our model achieves up to 13% improvement over the state-of-the-art approaches in short-term, medium-term, and long-term highway traffic forecasting.

Subway passenger flow forecasting, an essential component of intelligent transportation system, is critical for traffic management, public safety, urban planning. However, it is very challenging due to the high nonlinearities and complex dynamic spatio-temporal dependencies of passenger flows. In this paper, we model the subway system as a directed weighted graph and propose a novel spatio-temporal deep learning framework, Multi-STGCnet, for forecasting short-term subway passenger flow at a station level. Specifically, Multi-STGCnet is mainly composed of two components, temporal component and spatial component. (1) The temporal component employs three long short-term memory network (LSTM)-based modules to capture three temporal properties of the target station, which are the interval closeness, daily periodicity, weekly trend. (2) The spatial component designs three spatial matrixes to extract spatial correlation of a target station with all other stations classified as near neighbors, middle neighbors and distant neighbors. Respectively, it adopts three graph convolution network (GCN) and LSTM combined modules to capture the spatio-temporal influences from different neighbors. Finally, the outputs of the two components are fused with different weights to generate prediction. We evaluate Multi-STGCnet on a real world dataset from the metro system in Shenzhen, China. Experiment results demonstrate that our model outperforms multiple baselines.

Computation offloading between the edge and the cloud is an effective way for deployed service to fully utilize the resources at both sides for its QoS improvement and overall cost reduction. Although the offloading problem has been intensively studied in the context of mobile computing, existing algorithms in most cases cannot be effectively migrated to the edge-cloud environment because their inter-partition communication costs are always deemed as symmetric, and their intra-partition communication costs are often ignored, which, though reasonable to the traditional case, are not valid to our settings anymore. In this article, we propose a new algorithmic approach to the offloading problem in the edge-cloud environment, where a heterogeneous model is advocated to incorporate the communication cost between co-resident tasks while considering the asymmetry of communication costs between non-coresident tasks. We prove the offloading problem with respect to this model is NP-hard, and thereby designing an efficient algorithm to obtain a sub-optimal solution. Additionally, we also show that in a homogeneous case when the intra-partition and inter-partition communication costs between any pair of interactive tasks are symmetric, an optimal offloading algorithm can be devised by transforming the problem into a classical min-cut problem. We implemented and evaluated the algorithms by offloading a PageRank-based application in a controlled edge-cloud setting. Our empirical results show that the proposed algorithm for the heterogeneous case is always efficient to find a better offloading scheme, compared with the selected existing algorithms, while for the homogeneous case, the proposed solution can efficiently achieve the optimal strategy.

Traditional alarm prediction in telecommunication networks mainly relies on expert knowledge. However, with the increasing complexity of telecommunication network, the traditional methods may not work well. It's necessary to study new automatic association rules extraction methods. In this paper, we proposed a method called DCSA (Density-based Clustering and Sequential Association Analysis) for alarm association rules mining. We use time density-based clustering and FP-Growth algorithm to mine the association rules in alarm data, which overcomes the drawbacks of sliding windows method. In addition, we design a sequential rules filtering module to eliminate the items that do not meet the sequential conditions in the original rules. Experiments on 7.5 million real alarm items from a telecommunication company of China show the sequential rules filtering module can greatly reduce the redundancy of association rules. We also demonstrate that the proposed DCSA method could effectively predict the occurrence of alarms.

Docker container is experiencing rapid development with the support from the industry like Google and Alibaba and is being widely used in large scale production cloud environment. For example, Alibaba has deployed millions of containers for its internal business, and most of the online services are already migrated to the containers. Those services are usually very complex, spanning multiple containers with complex interaction and dependency relationship. Detecting potential anomalies in such a large container-based cloud platform is very challenging. Traditional detection models usually use system resource metrics like CPU and memory usage, but rarely consider the relationship among components, causing high false positive rate. In this paper, we present a novel Anomaly Detection and root cause localization method based on Graph Similarity (ADGS) in the container-based cloud environment. We first monitor the response time and resource usage of each component in the application to determine whether the system status is normal or not. Then, we propose a new mechanism to locate the root cause of the anomalies based on graph similarity, investigating the anomaly propagation rules among cluster components. We implement and evaluate our method in a container-based environment. The results show that the proposed method can detect and determine the root cause of anomalies efficiently and accurately.

In this paper, we design an efficient deduplication algorithm based on the distributed storage architecture of Ceph. The algorithm uses on-line block-level data deduplication technology to complete data slicing, which neither affects the data storage process in Ceph nor alter other interfaces and functions in Ceph. Without relying on any central node, the algorithm maintains the characteristics of Ceph by designing a special hash object to store the data fingerprint, and uses the CRUSH algorithm to judge the data duplication based on calculation, instead of global search. The algorithm replaces the duplicate data with the deduplicated objects, which storage their fingerprints with less storage space. We compare the effects of different block sizes with respect to the performance and deduplication rates through experimental studies, and select the most appropriate block size in our prototype implementation. The experimental results show that the algorithm can not only effectively save the storage space but also improve the bandwidth utilization when reading and writing the duplicate data.

Reducing tail latency becomes increasingly important to improve user-perceived service experience. User-facing latency-sensitive cloud applications typically contain multiple interactive tiers running in different virtual machines (VMs) with complex interaction patterns. Consolidation of those applications is a challenge. In this paper we study the consolidation of multi-tier interactive workloads from a new perspective of user-perceived tail latency. We propose a novel profiling-based consolidation methodology. The objective is to satisfy tail latency while reducing the number of physical machines. We consider two key factors that affecting the tail latency of multi-tier workloads: interference with neighboring VMs and interaction between different tiers. We model the consolidation of multi-tier workloads as an optimization problem with different objectives and constraints. We implement and evaluate the proposed models, as well as comparing with other methods (i.e., without profiling or without considering interaction influence). Experimental results show that the proposed method is able to greatly reduce the tail latency compared with the traditional consolidation method.

Cloud computing technology is widely used in today's datacenters due to the benefits such as high scalability, on-demand services and low cost. An in-depth understanding of the characteristics of workloads running in production cloud environments is very important for improving the resource management efficiency. In this paper, we make a detailed analysis with visualization techniques and clustering methods on the trace dataset released by Alibaba which contains 11089 online services and 12951 batch jobs running on 1313 machines. Our methodology for clustering workloads contains: i) Select effective feature vectors as the dimensions of clustering; ii) Identify the cluster boundaries of each dimension using K-Means algorithm; iii) Classify jobs by combining the feature vectors which uses the results from previous step; iv) Analyze the characteristics of workload groups at runtime. Our analysis reveals several insights which previous work has not found on Alibaba cluster trace. For batch jobs: a) Average CPU cores of all batch jobs show bimodal-distribution obviously. b) At a random sampling time, more than 50 % machines only run one group of jobs with a short duration, medium CPU cores and small memory utilization, the remaining machines run mixed groups of jobs. For online instances: a) The resource usage (CPU, Memory, and Disk) of most online instances is low; b) There are up to six groups running on the same machine according to our clustering method at a random sampling time.

Docker container is experiencing a rapid development with the support from industry like Google and is being widely used in large scale production cloud environments. However the performance of applications running in Docker containers is still not clear due to the complex relationship between container resource allocation and application performance. In this paper, we first study the impact of key parameters in container resource allocation that affect the performance of containerized applications. Then, we present modeling techniques over CPU, memory and I/O resources to characterize the performance of applications running in containers. To address this multi-dimensional modeling problem, we propose three machine learning techniques, i.e. Linear Regression (LR), Support Vector Machine (SVM) and Artificial Neural Network (ANN). We implement and evaluate the modeling techniques for four complex benchmark workloads from Spark. Experimental results demonstrate the proposed models can achieve as low as 2.27% prediction error, with an average of 10.13% for most applications. Furthermore, the prediction accuracy of SVM and ANN models are substantially better than LR based approaches, with 48.13% and 29.30% improvement.

To improve resource efficiency and design intelligent scheduler for clouds, it is necessary to understand the workload characteristics and machine utilization in large-scale cloud data centers. In this paper, we perform a deep analysis on a newly released trace dataset by Alibaba in September 2017, consists of detail statistics of 11089 online service jobs and 12951 batch jobs co-locating on 1300 machines over 12 hours. To the best of our knowledge, this is one of the first work to analyze the Alibaba public trace. Our analysis reveals several important insights about different types of imbalance in the Alibaba cloud. Such imbalances exacerbate the complexity and challenge of cloud resource management, which might incur severe wastes of resources and low cluster utilization. 1) Spatial Imbalance: heterogeneous resource utilization across machines and workloads. 2) Temporal Imbalance: greatly time-varying resource usages per workload and machine. 3) Imbalanced proportion of multi-dimensional resources (CPU and memory) utilization per workload. 4) Imbalanced resource demands and runtime statistics (duration and task number) between online service and offline batch jobs. We argue accommodating such imbalances during resource allocation is critical to improve cluster efficiency, and will motivate the emergence of new resource managers and schedulers.

Docker container is experiencing a rapid development with the support from industry and being widely used in large scale production cloud environment, due to the benefits of speedy launching time and tiny memory footprint. However the performance of big data applications (e.g., Spark) running in Docker containers is still not clear due to the complex parameter configuration and interference between neighbor containers. This paper investigates the impacts of docker configuration and resource interference on the performance of big data applications in a typical container environment. In particular, we first conduct a series of experiments to measure the performance impact by adjusting the docker configuration parameters, such as resource limits, and observe the Spark performance is not linear with increasing resource allocation for containers. Then, we evaluate the interference between multiple containers by controlling the resource competition and detect the performance interference phenomenon between multiple containers. Finally, we propose a performance prediction model based on the Support Vector Regression (SVR) to predict the application performance with different configurations and resource competition settings. Experimental results show the prediction error is less than 10% for all the four typical Spark applications.

Temperature prediction can enhance datacenter thermal management towards minimizing cooling power draw. Traditional approaches achieve this through analyzing task-temperature profiles or resistor-capacitor circuit models to predict CPU temperature. However, they are unable to capture task resource heterogeneity within multi-tenant environments and make predictions under dynamic scenarios such as virtual machine migration, which is one of the main characteristics of Cloud computing. This paper proposes virtual machine level temperature prediction in Cloud datacenters. Experiments show that the mean squared error of stable CPU temperature prediction is within 1.10, and dynamic CPU temperature prediction can achieve 1.60 in most scenarios.

Improving energy efficiency of data centers has become increasingly important nowadays due to the significant amounts of power needed to operate these centers. An important method for achieving energy efficiency is server consolidation supported by virtualization. However, server consolidation may incur significant degradation to workload performance due to virtual machine (VM) co-location and migration. How to reduce such performance degradation becomes a critical issue to address. In this paper, we propose a profiling-based server consolidation framework which minimizes the number of physical machines (PMs) used in data centers while maintaining satisfactory performance of various workloads. Inside this framework, we first profile the performance losses of various workloads under two situations: running in co-location and experiencing migrations. We then design two modules: (1) consolidation planning module which, given a set of workloads, minimizes the number of PMs by an integer programming model, and (2) migration planning module which, given a source VM placement scenario and a target VM placement scenario, minimizes the number of VM migrations by a polynomial time algorithm. Also, based on the workload performance profiles, both modules can guarantee the performance losses of various workloads below configurable thresholds. Our experiments for workload profiling are conducted with real data center workloads and our experiments on our two modules validate the integer programming model and the polynomial time algorithm.

With the rapid development of big data and cloud computing, big data analytics as a service in the cloud is becoming increasingly popular. More and more individuals and organizations tend to rent virtual cluster to store and analyze data rather than building their own data centers. However, in virtualization environment, whether scaling out using a cluster with more nodes to process big data is better than scaling up by adding more resources to the original virtual machines (VMs) in cluster is not clear. In this paper, we study the scalability performance issues of hadoop virtual cluster with cost consideration. We first present the design and implementation of VirtualMR platform which can provide users with scalable hadoop virtual cluster services for the MapReduce based big data analytics. Then we run a series of hadoop benchmarks and real parallel machine learning algorithms to evaluate the scalability performance, including scale-up method and scale-out method. Finally, we integrate our platform with resource monitoring module and propose a system tuner. By analyzing the monitored data, we dynamically adjust the parameters of hadoop framework and virtual machine configuration to improve resource utilization and reduce rent cost. Experimental results show that the scale-up method outperforms the scale-out method for CPU-bound applications, and it is opposite for I/O-bound applications. The results also verify the efficiency of system tuner to increase resource utilization and reduce rent cost.

CloudSim is one of the most powerful simulation platforms for cloud computing. It supports the energy-conscious scheduling and network simulation in the latest version. However, it still faces several limitations: 1) Current CloudSim cannot support both the power model and the network model at the same time. 2) The network components in current CloudSim do not support power-aware simulation. 3) The simulation of migration does not take into account the network overheads. To overcome these limitations, we design and implement an enhanced cloud simulation platform called DartCSim+ that supports the energy-aware network simulation and network-aware live migration. Further, we also implement a resubmit mechanism for packets transmission to provide a more real network behavior to solve transmission failure which is caused by migration or network failure. Finally, three groups of experiments are performed to demonstrate the effectiveness of DartCSim+.

Virtualization technology is currently becoming increasingly popular and valuable in cloud computing environments due to the benefits of server consolidation, live migration, and resource isolation. Live migration of virtual machines can be used to implement energy saving and load balancing in cloud data center. However, to our knowledge, most of the previous work concentrated on the implementation of migration technology itself while didn't consider the impact of resource reservation strategy on migration efficiency. This paper focuses on the live migration strategy of multiple virtual machines with different resource reservation methods. We first describe the live migration framework of multiple virtual machines with resource reservation technology. Then we perform a series of experiments to investigate the impacts of different resource reservation methods on the performance of live migration in both source machine and target machine. Additionally, we analyze the efficiency of parallel migration strategy and workload-aware migration strategy. The metrics such as downtime, total migration time, and workload performance overheads are measured. Experiments reveal some new discovery of live migration of multiple virtual machines. Based on the observed results, we present corresponding optimization methods to improve the migration efficiency.

Cloud computing has recently received considerable attention. With the fast development of cloud computing, the data center is becoming larger in scale and consumes more energy. There is an emergency need to develop efficient energy-saving methods to reduce the huge energy consumption in the cloud data center. In this paper, we achieve this goal by dynamically allocating resources based on utilization analysis and prediction. We use ``Linear Predicting Method" (LPM) and ``Flat Period Reservation-Reduced Method" (FPRRM) to get useful information from the resource utilization log, and make M/M/1 queuing theory predicting method have better response time and less energy-consuming. Experimental evaluation performed on CloudSim cloud simulator shows that the proposed methods can effectively reduce the violation rate and energy-consuming in the cloud.

Virtualization technology has been widely applied in data centers and IT infrastructures, with advantages of server consolidation and live migration. Through live migration, data centers could flexibly move virtual machines among different physical machines to balance workloads, reduce energy consumption and enhance service availability.

Virtualization technology is currently widely used due to its benefits on high resource utilization, flexible manageability and powerful system security. However, its use for high performance computing (HPC) is still not popular due to the unclearness of the virtualization overheads. It's worthy to evaluate the virtualization cost and to find the performance bottleneck when running HPC applications in virtual cluster. We first evaluate the basic performance overheads due to virtualization. Then we create a 16-node virtual cluster and perform a performance evaluation for both para-virtualization and full virtualization. After that, we evaluate the MPI (Message Passing Interface) scalability to investigate the impact of MPI and network communication between virtual machines. In addition to the macro assessment, we use the Oprofile/Xenoprof to investigate the architecture characterization like CPU cycle, L2 cache misses, DTLB misses and ITLB misses which is an auxiliary explanation to the performance bottleneck. Experimental results indicate that performance overheads of virtualization are acceptable for HPC, para-virtualization is very suitable for HPC due to the high virtualization efficiency and efficient inter-domain communication. Finally, we use the non-linear regression modeling technology to present a performance model for network latency and bandwidth to predict the performance in virtual cluster environment.

Virtualization brings many benefits such as improving system utilization and reducing cost through server consolidation. However, it also introduces isolation problem when running multiple virtual machine workloads in one physical platform. Additionally, with the advent of multi-core technology, more and more cores are built into one die in today's data center that will share and compete for the resource like cache. It's worthy to study the isolation of server consolidation in modern multi-core platform. However, to our knowledge there are few work done on the isolation property especially the fault isolation property when one of the virtual machine workloads is attacked in server consolidation. In this paper, we study the isolation property from performance perspective and provide two optimization methods to improve the isolation property. We first define the isolation property and quantify the performance isolation in consolidation and propose a VM-level optimization method. Then we study the fault isolation by introducing a misbehavior virtual machine in server consolidation scenario and propose a core-level cache-aware optimization method to improve the fault isolation. Experimental results show that our two optimization methods can effectively improve the performance isolation and fault isolation with 29.39% and 19.52% respectively. What's more, Oprofile/Xenoprof toolkits are used to find out the factors affecting isolation property from the hardware events level.