ABSTRACT The exponential growth of IoT devices poses unprecedented challenges to cloud‐edge collaboration, particularly in microservice‐based industrial scenarios where real‐time sensor data flows through complex service chains. Current approaches suffer from critical issues: Inefficient service deployment, load imbalance, and service instability. Specifically, the performance degradation is particularly severe when critical path services experience sudden load spikes, often leading to cascading delays across entire service chains. This paper presents SDT‐MCS (Service Dependency Topology‐aware Microservice Collaborative Scheduling), a framework that combines topology‐oriented federated learning with lightweight Actor‐Critic reinforcement mechanisms. For resource collaboration, we propose a topology‐aware pre‐deployment algorithm that leverages federated learning to optimize global resource orchestration while considering both resource constraints and service dependencies. For service collaboration, we design a chain‐based scheduling mechanism that employs Actor‐Critic reinforcement learning for local dynamic adjustment, enabling rapid response to workload variations while maintaining service chain stability. We implement our framework on EdgeCloudSim and evaluate it using production workload traces from industrial robotics and smart city scenarios, with additional validation on a physical testbed. Experimental results demonstrate that our topology‐aware pre‐deployment reduces average latency by 31.6% in ETL scenarios compared to baseline approaches. Furthermore, our chain‐based scheduling achieves 35.4% latency reduction under high concurrency while maintaining service stability between 65%–70% through dynamic load balancing.

Cloud service providers typically co-locate various workloads within the same production cluster to improve resource utilization and reduce operational costs. These workloads primarily consist of batch analysis jobs composed of multiple parallel short-running tasks and long-running applications (LRAs) that continuously reside in the system. The adoption of microservice architecture has led to the emergence of distributed long-running applications (DLRAs), which enhance deployment flexibility but pose challenges in detecting and investigating QoS violations due to workload variability and performance propagation across microservices. State-of-the-art resource managers are only responsible for resource allocation among applications/jobs and do not prioritize runtime QoS aspects, such as applicationlevel latency. To address this, we introduce Prank, a QoSdriven resource management framework for co-located workloads. Prank incorporates a non-intrusive performance anomaly detection mechanism for DLRAs and proposes a root cause localization algorithm based on PageRank-weighted analysis of performance anomalies. Moreover, it dynamically balances resource allocation between DLRAs and co-located batch jobs on nodes hosting critical microservices, optimizing for both DLRA performance and overall cluster efficiency. Experimental results demonstrate that Prank outperforms state-of-the-art baselines, reducing DLRA tail latency by over 38% while increasing batch job completion time by no more than 21% on average

Accurate environmental perception is essential for the effective operation of autonomous vehicles. However, visual images captured in dynamic environments or adverse weather conditions often suffer from various degradations. Image restoration focuses on reconstructing clear and sharp images by eliminating undesired degradations from corrupted inputs. These degradations typically vary in size and severity, making it crucial to employ robust multi-scale representation learning techniques. In this paper, we propose Multi-Scale Feature Modulation (MSFM), a novel deep convolutional architecture for image restoration. MSFM modulates multi-scale features in both frequency and spatial domains to make features sharper and closer to that of clean images. Specifically, our multi-scale frequency attention module transforms features into multiple scales and then modulates each scale in the implicit frequency domain using pooling and attention. Moreover, we develop a multi-scale spatial modulation module to refine pixels with the guidance of local features. The proposed frequency and spatial modules enable MSFM to better handle degradations of different sizes. Experimental results demonstrate that MSFM achieves state-of-the-art performance on 12 datasets for a range of image restoration tasks, i.e., image dehazing, image defocus/motion deblurring, and image desnowing. Furthermore, the restored images significantly improve the environmental perception of autonomous vehicles.

Finding performance bottlenecks through bench-marking is one of the driving forces to improve the resource provision efficiency of cloud computing. Although existing benchmarks have been designed to improve the effectiveness in system performance evaluation, the following problems still exist in these benchmarks due to insufficient consideration of the characteristics of jobs in the production environment: (i) lacking of understanding for the details of workloads composition in the production environment, which reduces the authenticity of the job. (ii) the design of workloads submission patterns lacks quantization and reproducibility, which often relies on a random setting. In our benchmarking, multiple workloads are generated by analyzing and fine-grained matching the composition of workloads in the real production, and a design of workloads submission pattern based on LSTM time series prediction is proposed to simulate the real submission behavior. We finally demonstrate the effectiveness of our work by evaluating the impact of different workloads submission patterns on system performance evaluation.

The clustering method has been widely used in data mining, pattern recognition, and image identification. Fuzzy c-means (FCM) is a soft clustering method that introduces the concept of membership. In this method, the fuzzy membership matrix is obtained by calculating the distance between data points in the original space. However, these methods may yield suboptimal results owing to the influence of redundant features. Moreover, FCM is always sensitive to noise points and heavily subject to outliers. In this article, we propose a method called sparsity FCM clustering with principal component analysis embedding (P_SFCM). We simultaneously conduct principal component analysis and membership learning, and then add an additional weighting factor for each data point. The goal of this operation is to identify the noise or outliers. Overall, the benefit of our framework is that it retains most of the information in the subspace while improving the robustness of the noise. In this article, we employ an iterative optimization algorithm to efficiently solve our model. To verify the reliability of the proposed method, we conduct a convergence analysis, noise robustness analysis, and multicluster experiments. Furthermore, comparative experiments are conducted on both synthetic and real benchmark datasets. The experimental results show that the P_SFCM is competitive with comparable methods.

As one of the classical clustering techniques, spectral embedding boasts extensive applicability across numerous domains. Traditional spectral embedding techniques entail the mapping of graph models to low-dimensional vector spaces (indicator vectors) to facilitate hard partitioning. However, data boundaries occasionally exhibit ambiguity, thereby constraining the utility of hard partitioning. In this article, we introduce an innovative spectral embedding method, namely, joint learning of anchor graph-based fuzzy spectral embedding model and fuzzy K-means (AFSEFK). Drawing inspiration from fuzzy logic, our method employs a membership vector in lieu of the conventional indicator vector for spectral embedding, amalgamating it with fuzzy K-means to concurrently optimize membership, thereby simultaneously learning the local and global structures inherent in the data. Moreover, to enhance the quality of similarity graphs and augment clustering performance, we implement the balanced K-means-based hierarchical K-means technique to generate representative anchors. Subsequently, an anchor-based similarity graph is devised through a parameter-free neighbor assignment strategy. Comprehensive extensive experimentation with synthetic and real-world datasets substantiates the efficacy of the AFSEFK algorithm.

In traditional graph embedding methods, graph construction is sensitive to high-dimensional data with noise and outliers, making an effective exploration of the neighborhood structure of the data difficult. Besides, with these methods, constructing graphs and reducing dimensions are disconnected and cannot be mutually optimized. To address these problems, we propose an unsupervised dimensionality reduction method based on bipartite graph, named unsupervised adaptive bipartite graph embedding (UABGE). First, the anchors are generated from the raw data by K-means or random sampling. Second, the bipartite graph, which is constructed between the samples and the anchors in the low-dimensional subspace, utilizes the adaptive allocation method to assign neighbors for each sample, so that the local structure of high-dimensional data can be captured effectively. Third, we present an objective function that combines bipartite graph construction and projection matrix learning to achieve mutual optimization between them, which can be solved with an alternating optimization algorithm. Finally, the computational complexity and the convergence of the algorithm are analyzed. Experimental results on synthetic data and publicly available datasets illustrate the effectiveness of the proposed method.

Graph embedding is a widely used method for dimensionality reduction due to its computational effectiveness. The quality of the graph and the efficiency of graph construction will directly affect the performance and the efficiency of the graph embedding methods. However, in the unsupervised graph embedding methods, the graph is not considered as an optimized graph since there is no label that can be used to construct this graph. In addition, the running of traditional graph embedding methods become very time-consuming on large-scale datasets due to the high computational cost in the step of graph construction. Aiming to solve these problems, we propose an unsupervised dimensionality reduction method based on bipartite graph, called Unsupervised Optimized Bipartite Graph Embedding (UOBGE). Representative anchors are firstly identified in the data. Then, we construct the bipartite graph between the projected samples and the projected anchors and the intrinsic graph connecting all the projected sample pairs with equal weights, which keep the local and global geometric structures of the data, respectively. Finally, the bipartite graph and the projection matrix are optimized simultaneously by introducing an alternating optimization procedure. Extensive experiments on several datasets demonstrate that the effectiveness and efficiency of the proposed method.

Summary Striking a balance between improved cluster utilization and guaranteed application QoS is a long‐standing research problem in multi‐tenants shared cluster. The typical solution is to detect performance degradation and investigate the root cause to conduct performance isolation. Existing efforts rely on lots of prior knowledge of applications and the assumption of interference‐free workload placement is possible. Performance interference is usually mitigated through application‐level approaches such as centralized rescheduling, which is usually an hindsight and a waste of resources. In this article, we present ScaleReactor, a graceful runtime agent on a per node basis that decouples the performance isolation from centralized resource management, and migrates the performance interference of scale‐out workloads in container‐based cluster using a lightweight black‐box approach. ScaleReactor analyzes the degree of contention for multi‐dimensional resources among co‐located workloads to detect the performance degradation without additional prior information, and uses correlation analysis to locate the cause of contention, while isolating resources in a graceful manner to reduce system overhead and the performance degradation of intrusive workloads. Experiments have demonstrated that ScaleReactor effectively reduces the job completion time of scale‐out applications in shared clusters, with the maximum value up to 36% and low system overhead against the existing isolation mechanism.

To achieve a high degree of resource utilization, production clusters need to co-schedule diverse workloads – including both batch analytic jobs with short-lived tasks and long-running applications (LRAs) that execute for a long time frame from hours to months – onto the shared resources. Microservice architecture advances the manifestation of distributed LRAs (DLRAs), comprising multiple interconnected microservices that are executed in long-lived distributed containers and serve massive user requests. Detecting and mitigating QoS violation become even more intractable due to the network uncertainties and latency propagation across dependent microservices. However, current resource managers are only responsible for resource allocation among applications/jobs but agnostic to runtime QoS such as latency at application level. The state-of-the-art QoS-aware scheduling approaches are dedicated for monolithic applications, without considering the temporal-spatio performance variability across distributed microservices. In this paper, we presentToposch, a new scheduling and execution framework to prioritize the QoS of DLRAs whilst balancing the performance of batch jobs and maintaining high cluster utilization through harvesting idle resources.Toposchtracks footprints of every single request across microservices and uses critical path analysis, based on the end-to-end latency graph, to identify microservices that have high risk of QoS violation. Based on microservice and node level risk assessment, we intervene the batch scheduling by adaptively reducing the visible resources to batch tasks and thus delaying their execution to give way to DLRAs. We propose a prediction-based vertical resource auto-scaling mechanism, with the aid of resource-performance modeling and fine-grained resource inference and access control, for prompt recovery of QoS violation. A cost-effective task preemption is leveraged to ensure a low-cost task preemption and resource reclamation during the auto-scaling.Toposchis integrated with Apache YARN and experiments show thatToposchoutperforms other baselines in terms of performance guarantee of DLRAs, at an acceptable cost of batch job slowdown. The tail latency of DLRAs is merely 1.12x of the case of executing alone on average inToposchwith a 26% JCT increase of Spark analytic jobs.

Striking a balance between improved cluster utilization and guaranteed application QoS is a long-standing research problem in cluster resource management. The majority of current solutions require a large number of sandboxed experimentation for different workload combinations and leverage them to predict possible interference for incoming workloads. This results in non-negligible time complexity that severely restricts its applicability to complex workload co-locations. The nature of pure offline profiling may also lead to model aging problem that drastically degrades the model precision. In this paper, we present Perph, a runtime agent on a per node basis, which decouples ML-based performance prediction and resource inference from centralized scheduler. We exploit the sensitivity of long-running applications to multi-resources for establishing a relationship between resource allocation and consequential performance. We use Online Gradient Boost Regression Tree (OGBRT) to enable the continuous model evolution. Once performance degradation is detected, resource inference is conducted to work out a proper slice of resources that will be reallocated to recover the target performance. The integration with Node Manager (NM) of Apache YARN shows that the throughput of Kafka data-streaming application is 2.0x and 1.82x times that of isolation execution schemes in native YARN and pure cgroup cpu subsystem. In TPC-C benchmarking, the throughput can also be improved by 35% and 23% respectively against YARN native and cgroup cpu subsystem.

Balancing resource utilization and application QoS is a long-standing research topic in cluster resource management. Big data YARN clusters need to co-schedule diverse workloads on shared resources including batch processing jobs, streaming jobs, and other long-running applications such as web services, database services, etc. Current resource managers are only responsible for resource allocation among applications/jobs but completely unaware of runtime QoS requirements of interactive and latency-sensitive applications. Prior works to maximize the QoS of monolithic applications ignore inherent dependencies and temporal-spatio performance variability of components, characteristics of distributed applications primarily driven by microservices. In this paper, we present Toposch, a new resource management system to adaptively co-locate batch tasks and microservices by harvesting runtime latency. In particular, Toposch tracks full footprints of every request across microservices over time. A latency graph is periodically generated for identifying victim microservices through an end-to-end latency critical path analysis. We then exploit per-microservice and per-node risk assessment to gauge the visible resources to the capacity scheduler in YARN. Execution of batch tasks are adaptively throttled or delayed, thereby avoiding latency increase due to node over-saturation. TOPOSCH is integrated with YARN and experiments show that the latency of DLRAs can be reduced by up to 39.8% against the default capacity scheduling in YARN.

Cluster administrators are facing great pressures to improve cluster utilization through workload co-location. Guaranteeing performance of long-running applications (LRAs), however, is far from settled as unpredictable interference across applications is catastrophic to QoS [2]. Current solutions such as [1] usually employ sandboxed and offline profiling for different workload combinations and leverage them to predict incoming interference. However, the time complexity restricts the applicability to complex co-locations. Hence, this issue entails a new framework to harness runtime performance and mitigate the time cost with machine intelligence: i) It is desirable to explore a quantitative relationship between allocated resource and consequent workload performance, not relying on analyzing interference derived from different workload combinations. The majority of works, however, depend on offline profiling and training which may lead to model aging problem. Moreover, multi-resource dimensions (e.g., LLC contention) that are not completely included by existing works but have impact on performance interference need to be considered [3]. ii) Workload co-location also necessitates fine-grained isolation and access control mechanism. Once performance degradation is detected, dynamic resource adjustment will be enforced and application will be assigned an access to specific slices of each resources. Inferring a "just enough" amount of resource adjustment ensures the application performance can be secured whilst improving cluster utilization.

A long-standing challenge in cluster scheduling is to achieve a high degree of utilization of heterogeneous resources in a cluster. In practice there exists a substantial disparity between perceived and actual resource utilization. A scheduler might regard a cluster as fully utilized if a large resource request queue is present, but the actual resource utilization of the cluster can be in fact very low. This disparity results in the formation of idle resources, leading to inefficient resource usage and incurring high operational costs and an inability to provision services. In this paper we present a new cluster scheduling system, ROSE, that is based on a multi-layered scheduling architecture with an ability to over-subscribe idle resources to accommodate unfulfilled resource requests. ROSE books idle resources in a speculative manner: instead of waiting for resource allocation to be confirmed by the centralized scheduler, it requests intelligently to launch tasks within machines according to their suitability to oversubscribe resources. A threshold control with timely task rescheduling ensures fully-utilized cluster resources without generating potential task stragglers. Experimental results show that ROSE can almost double the average CPU utilization, from 36.37% to 65.10%, compared with a centralized scheduling scheme, and reduce the workload makespan by 30.11%, with an 8.23% disk utilization improvement over other scheduling strategies.

The presentation of Bluetooth Low Energy (BLE; e.g., Bluetooth 4.0) makes Bluetooth based indoor positioning have extremely broad application prospects. In this paper, we propose a received signal strength indication (RSSI) based Bluetooth positioning method. There are two phases in the procedure of our positioning: offline training and online locating. In the phase of offline training, we use piecewise fitting based on the lognormal distribution model to train the propagation model of RSSI for every BLE reference nodes, respectively, in order to reduce the influence of the positioning accuracy because of different locations of BLE reference nodes. Here we design a Gaussian filter to pre-process the receiving signals in different sampling points. In the phase of online locating, we use weighted sliding window to reduce fluctuations of the real-time signals. In addition, we propose a distance weighted filter based on triangle trilateral relations theorem, which can reduce the influence of positioning accuracy due to abnormal RSSI and improve the location accuracy effectively. Besides, in order to reduce the errors of targets coordinates caused by ordinary least squares method, we propose a collaborative localization algorithm based on Taylor series expansion. Another important feature of our method is the active learning ability of BLE reference nodes. Every reference node adjusts its pre-trained model according to the received signals from detecting nodes actively and periodically, which improve the accuracy of positioning greatly. Experiments show that the probability of locating error less than 1.5 meter is higher than 80% using our positioning method.