Quantifying energy consumption is a fundamental element of green computing. Power models trained by resource utilization allow quantifying the energy number and enable energy-efficient resource management systems without raising the concerns of complexity, cost, and security. However, energy consuming behavior on different machines varies by several factors. In this paper, we address the challenges of power modeling for cloud instances where information about these factors is obscured or unseen in the training set, and propose a best-effort method to train and serve a power model as precise as possible by leveraging a large, industry-standard power database. The proposed method prioritizes the modeling precision, and offers similarity and uncertainty indicators to elucidate the confidence level when serving an unseen instance. The results have demonstrated feasibility and precision of the proposed method against comparable approaches.

Sustainable computing is gaining priority for data centers due to need for mandatory compliance with carbon emission reporting regulations. Hence, accurately estimating the energy consumption of servers in data centers has become quintessential. Apart from estimating carbon emissions at the server level, quantifying carbon emissions at the virtualization layer is also crucial because it provides a more precise and detailed understanding of the possible hotspots and mismatched utilization which can help in taking remedial actions. In this paper, we take a first step towards modeling energy consumption at the Logical PARtitions (LPARs) level of data centers driven by IBM POWER9 Systems. We experimentally validate our approach on utilization metrics from the data center of the CIO office of IBM and compare it with the instrumented energy measurements wherein we demonstrate on average 90–95 % prediction accuracy for our model.

Enterprises world-wide are increasingly prioritizing sustainability due to the growing focus on carbon neutrality as well as the requirement to adhere to emerging strict regulations from governments across the globe. With many enterprise workloads deployed on cloud and data centers, to fulfill the mandatory carbon reporting requirements of their clients, it is becoming inevitable for cloud providers and data center operators to quantify each client’s share of the total carbon emission from their facility. Accurate carbon quantification requires power measurements to be available at the lowest level of the hardware infrastructure such as physical servers and network switches. However, power sensing is quite limited in many data centers, with measurements normally available only at an aggregated level such as the rack level. To drill down to the level of a workload to capture the correct power usage per workload, it is very important to dis-aggregate this power across servers. In this paper, we propose a software based non-linear model using the Newton-Raphson method to estimate the power model parameters of individual servers using server utilizations when the overall rack level power measurements are given. The methodology is applicable to data centers with multiple types of servers in a rack and is light-weight in the sense that it does not require mechanisms such as shutting down individual servers in order to estimate idle power. The method is also generalized to account for the real world scenario where the time granularity of rack power and server utilization measurements may not match. We have conducted detailed evaluations of the methods proposed and find good convergence for parameter estimation even when tested with multiple different initial conditions.

Automated crop classification and mapping is currently a topic of significant research interest worldwide due to the following two factors. First, it is one of the key tasks on which the success of digital agriculture hinges, and second, there is wider availability of remote-sensed imagery, both optical and radar-based, that can help with remote monitoring of crops. Several different models have been developed for the purpose, but a hitherto unexplored geography or time period generally requires fresh ground data specific to the space and time, and, in many cases, fresh feature engineering as well, due to lack of intra-class compactness and inter-class separability. A near-universal model that can be applied with minimal fine-tuning using free and open access satellite imagery and requiring no new ground data, which can reduce data costs, is elusive. This paper underscores the challenges involved via a case study with two different classification approaches at various regions. Methods and techniques that can ameliorate the problems are discussed.

Real-time monitoring of agricultural crops is an important exercise because of the huge economic impact. Identification of crop during early stage of the crop cycle can help formulate better agriculture policies and management strategies. In this context, the objective of this article is to evaluate the potential of Sentinel-1 Synthetic Aperture Radar (SAR) and Sentinel-2 optical imagery in crop identification for an Indian region. A multi-class classification algorithm based on random forest is applied to the features extracted from the above mentioned satellite data sets. Initial experimental suggest that the Sentinel-1 SAR data is promising in achieving high classification accuracy (85%).

Semi-partitioned real-time scheduling algorithms extend partitioned ones by allowing a (usually small) subset of tasks to migrate. The first such algorithm to be proposed was directed at soft real-time (SRT) sporadic task systems where bounded deadline tardiness is acceptable. That algorithm, called EDF-fm, has the desirable property that migrations are boundary-limited, i.e., they can only occur at job boundaries. However, it is not optimal because per-task utilization restrictions are required. In this paper, a new optimal semi-partitioned scheduling algorithm for SRT sporadic task systems is proposed that eliminates such restrictions. This algorithm, called EDF-os, preserves the boundary-limited property. In overhead-aware schedulability experiments presented herein, EDF-os proved to be better than all other tested alternatives in terms of schedulability in almost all considered scenarios. It also proved capable of ensuring very low tardiness bounds, which were near zero in most considered scenarios.

Over-the-top estimation of available bandwidth (AB) in a network path has been well studied for wired networks. The AB of a path denotes its slack capacity, i.e., the bandwidth available for use in the path without impacting the existing traffic. This estimation problem has been receiving attention only recently in cellular networks, which are increasingly becoming one of the main modes of access for a large number of applications. In this paper, we discuss the challenges posed by the problem, and why existing techniques developed for wired networks cannot be applied. We show that, interestingly, it may not even be feasible to estimate AB using over-the-top approaches under certain conditions, even when the wireless channel and traffic conditions are non-varying. We then present a novel AB estimation technique for cellular networks, which typically use proportional fair scheduling at base stations. When the wireless channel and traffic conditions are non-varying, our technique can accurately determine AB when it exceeds the “fair share” due to a new flow. We also extend the basic technique for estimation under conditions that are time-varying. The proposed methods can as well be used when one or more bottleneck links in a network path are fair-scheduled using algorithms such as weighted-fair queueing. We evaluate our methods using simulations and over operational networks, and present the results. In simulations, our technique is capable of detecting AB close to 90% of the time under feasible conditions even with bursty traffic.

We design and implement a novel system called Async, which enables a mobile network operator (MNO) to efficiently manage the growth of mobile data by leveraging the delay-elastic nature of certain applications and the price-sensitive nature of certain users. Specifically, Async introduces an alternate “asynchronous” content-delivery paradigm for heavy content (e.g., videos), and facilitates an MNO to negotiate with users a delay in delivery in exchange for appropriate incentives. The MNO uses the negotiated delays to actively manage Async flows to reduce congestion and improve the quality-of-experience (QoE) of both delayed and regular flows. We show that in comparison to state-of-the-art, Async's network-based flow management enhances QoE for more than 30% of the regular flows, with up to 60% improvement in per-flow QoE metric, while still meeting the negotiated delivery times of 95% of the delayed flows. Async also lowers the delivery times of delayed flows by ∼67% and significantly increases robustness to traffic unpredictability. Our design is robust to disconnections and does not require any modifications to existing network infrastructure and protocols. Our prototype deployment (using Apache's mod_proxy and an Android app) on live networks confirms Async's efficacy in meeting EDTs for diverse deployment scenarios.

The increasing demand for high-quality streaming video delivered to mobile clients necessitates efficient bandwidth utilization and allocation at not only the wireless channel but also the wired backhaul of broadband cellular networks. In this context, we propose techniques for increasing the link utilization and enhancing the quality-of-experience (QoE) for end users while multiplexing video streams over a wired link. For increasing the link utilization, we present a generic multi-tiered bandwidth estimation and scheduling scheme that can guarantee lower bounds on loss for flows at lower tiers. This scheme can be used for supporting heterogeneous loss classes, providing differentiated losses for different layers of video streams, or providing per-flow guarantees using lower aggregate bandwidth than schemes proposed in the literature. For enhancing the end-user QoE, we present a scheme for minimizing correlated losses and improving the smoothness of video quality by minimizing the maximum loss suffered by any logical unit of a stream and also the variability in loss across the length of the stream. In extensive simulations performed using video sources encoded in various formats, our multi-tiered scheme could lower the estimated bandwidth and improve statistical multiplexing gains by up to 25% with two and three classes and over 30% in the context of providing per-flow guarantees and differentiated loss for different layers. Our loss-minimization approach could lower the maximum loss by a factor of five and the loss variance by more than an order of magnitude.

Video objects are much larger in size than traditional web objects and tend not to be viewed in entirety. Hence, caching them partially is a promising approach. Also, the projected growth in video traffic over wireless cellular networks calls for resource-efficient caching mechanisms in the wireless edge to lower traffic over the cellular backhaul and peering links and their associated costs. An evaluation of traditional partial caching solutions proposed in the literature shows that known solutions are not robust to video viewing patterns, increasing object pool size, changing object popularity, or limitation in the resources available for caching at the wireless network elements. In this paper, to overcome the limitations, we propose a novel approach that adopts a flexible segmentation policy and generalizes both LRU and LFU when applied to segmented accesses, and in our simulations, is shown to significantly lower wireless backhaul traffic (by around 20-30% and in some cases even higher).

The increasing demand for high-quality streaming video delivered to mobile clients necessitates efficient bandwidth utilization and allocation at not only the wireless channel but also the wired backhaul of broadband wireless networks. In this context, we propose techniques for increasing the link utilization and enhancing the quality-of-experience (QoE) for end users while multiplexing video streams over a wired link. For increasing the link utilization, we present a generic multi-tiered bandwidth estimation and scheduling scheme that can guarantee lower bounds on loss for flows at lower tiers. This scheme can be used for supporting heterogeneous loss classes, differentiated losses for different layers of video streams, or per-flow guarantees using lower aggregate bandwidth than schemes proposed in the literature. For enhancing the end-user QoE, we present a scheme for minimizing correlated losses and improving the smoothness of video quality by minimizing the maximum loss suffered by any logical unit of a stream and also the variability in loss across the length of the stream. In simulations performed using MPEG-4 sources, our loss-minimization approach could lower the maximum loss by a factor of five and the loss variance by more than an order of magnitude. Our multi-tiered scheme could lower the estimated bandwidth and improve statistical multiplexing gains by 10-20% with three classes, 5-20% with two classes, and over 30% in the context of providing deterministic per-flow guarantees.

The Earliest Deadline First scheduling algorithm (EDF) is known to not be optimal under global scheduling on multiprocessor platforms. Results have been obtained that bound the maximum tardiness- the amount of time by which deadlines may be missed- of any feasible system of implicit-deadline sporadic tasks scheduled using global EDF. However, it is known that these bounds are not tight. In this paper, we derive an algorithm for obtaining tardiness bounds that are superior to previously known bounds. In contrast to prior algorithms, which compute a single tardiness bound for all the tasks in the system, our algorithm derives a separate tardiness bound for each task. Particularly upon task systems in which the parameters of the tasks are very dissimilar, our bounds are significantly better than prior bounds. Our algorithm also yields a simple sufficient test for the tardiness verification problem: given a task system with maximum acceptable tardiness bounds per task, is the system guaranteed to be scheduled by global EDF such that these tardiness constraints are not violated?

We consider lifetime quality-of-experience (QoE) management for video-on-demand (VoD) users in WiMAX networks. For efficient resource utilization while enhancing user experience, we propose run-time load balancing through joint scheduling among multiple base stations (BSs), referred to as macro scheduling. Macro scheduling employs a utility-maximization approach. To achieve long-term proportional fairness (PF) and manage lifetime QoE, user and flow utilities are modeled as functions of their past service rates, in addition to current channel conditions and bandwidth needs. We show that scheduling flows across multiple BSs jointly to achieve PF is NP-hard in the strong sense. Since approximation algorithms proposed in prior work are computationally expensive for online use, we design efficient heuristics that perform as well as the approximation algorithms. A simulation-based evaluation shows that our overall macro scheduling scheme can improve the number of satisfied users by up to 35% in comparison to other approaches, while only minimally sacrificing on throughput.

We consider schemes for enacting task share changes - a process called reweighting - on real-time multiprocessor platforms. Our particular focus is reweighting schemes that are deployed in environments in which tasks may frequently request significant share changes. Prior work has shown that fair scheduling algorithms are capable of reweighting tasks with minimal allocation error and that partitioning-based scheduling algorithms can reweight tasks with better average-case performance, but greater error. However, preemption and migration overheads can be high in fair schemes. In this paper, we consider the question of whether global scheduling techniques can improve the accuracy of reweighting relative to partitioning-based schemes and provide improved average-case performance relative to fair-scheduled systems. Our conclusion is that, for soft real-time systems, global scheduling techniques provide a good mix of accuracy and average-case performance.

We consider coordinating accesses to shared data structures in multiprocessor real-time systems scheduled under preemptive global EDF. To our knowledge, prior work on global EDF has focused only on systems of independent tasks. We take an initial step here towards a generic resource-sharing framework by considering simple shared objects, such as queues, stacks, and linked lists. In many applications, the predominate use of synchronization constructs is for sharing such simple objects. We analyze two synchronization methods for such objects, one based on queue-based spin locks and a second based on lock-free algorithms

The earliest-deadline-first (EDF) scheduling of a sporadic real-time task system on a multiprocessor may require that the total utilization of the task system, U/sub sum/, not exceed (m + 1)/2 on m processors if every deadline needs to be met. In recent work, we considered the alleviation of this under-utilization for task systems that can tolerate deadline misses by bounded amounts (i.e., bounded tardiness). We showed that if U/sub sum/ /spl les/ m and tasks are not pinned to processors, then the tardiness of each task is bounded under both preemptive and non-preemptive EDF. However, the tardiness bounds derived are applicable to every task in the task system, i.e., any task may incur maximum tardiness. In this paper, we consider supporting tasks whose tolerances to tardiness are less than that known to be possible under EDF. We propose a new scheduling policy, called EDF-hl, which is a variant of EDF, and show that under EDF-hl, any tardiness, including zero tardiness, can be ensured for a limited number of privileged tasks, and that bounded tardiness can be guaranteed to the remaining tasks if their utilizations are restricted. EDF-hl reduces to EDF in the absence of privileged tasks. The tardiness bound that we derive is a function of U/sub sum/, in addition to individual task parameters. Hence, tardiness for all tasks can be lowered by lowering U/sub sum/. A simulation-based evaluation of the tardiness bounds that are possible is provided.

We present a real-time, Linux-based testbed called LITMUS, which we have developed for empirically evaluating multiprocessor real-time scheduling algorithms. We also present the results from such an evaluation, in which partitioned earliest-deadline-first (EDF) scheduling, preemptive and nonpreemptive global EDF scheduling, and two variants of the global PD2 Pfair algorithm were considered. The tested algorithms were compared based on both raw performance and schedulability (with real overheads considered) assuming either hard- or soft-real-time constraints. To our knowledge, this paper is the first attempt by anyone to compare partitioned and global real-time scheduling approaches using empirical data

In hard real-time systems, a significant disparity in schedulability exists between EDF-based scheduling algorithms and Pfair scheduling, which is the only known way of optimally scheduling recurrent real-time tasks on multiprocessors. This is unfortunate because EDF-based algorithms entail lower scheduling and task-migration overheads. In work on hard real-time systems, it has been shown that the disparity in schedulability can be lessened by placing caps on per-task utilizations. In this paper, we show that it can also be lessened by easing the requirement that all deadlines be met. Our main contribution is a new EDF-based scheme that ensures bounded deadline tardiness. In this scheme, per-task utilizations must be capped, but overall utilization need not be restricted. The required cap is quite liberal. Hence, our scheme should enable a wide range of soft real-time applications to be scheduled with no constraints on total utilization. We also propose techniques and heuristics that can be used to reduce tardiness.

This paper considers the scheduling of soft real-time sporadic task systems under global EDF on an identical multiprocessor. Prior research on global EDF has focused mostly on hard real-time systems, where, to ensure that all deadlines are met, approximately 50% of the available processing capacity will have to be sacrificed in the worst case. This may be overkill for soft real-time systems that can tolerate bounded tardiness. In this paper, we derive tardiness bounds under preemptive and non-preemptive global EDF on multiprocessors when the total utilization of a task system is not restricted and may equal the number of processors. Our tardiness bounds depend on per-task utilizations and execution costs - the lower these values, the lower the tardiness bounds. As a final remark, we note that global EDF may be superior to partitioned EDF for multiprocessor-based soft real-time systems in that the latter does not offer any scope to improve system utilization even if bounded tardiness can be tolerated

Summary form only given. The earliest-pseudo-deadline-first (EPDF) Pfair algorithm is more efficient than other known Pfair scheduling algorithms, but is not optimal on more than two processors. Srinivasan and Anderson established a sufficient per-task utilization restriction for ensuring a tardiness of at most one quantum under EPDF. They also conjectured that a tardiness bound of one quantum applies to systems that are not restricted in any way. We present counterexamples that show that this conjecture is false. We also present sufficient utilization restrictions that are more liberal than theirs.

Prior work on Pfair scheduling has resulted in three optimal multiprocessor scheduling algorithms, and one algorithm, EPDF, that is less expensive but not optimal. EPDF is still of interest in soft real-time systems, however, due to its ability to guarantee bounded tardiness. In particular, it has been shown that a tardiness bound of t quanta is possible under EPDF if all task weights (i.e., shares or utilizations) are restricted to a value specified as a function of t. In an actual system, however, different tasks may be subject to different tardiness bounds. If such a system is scheduled under EPDF, then the tardiness of a task with a higher bound may cause the tardiness bound of a task with a lower bound to be violated; that is, temporal isolation among the various tardiness classes may not be guaranteed. In this paper, we propose an algortihm based on EPDF for scheduling task classes with different tardiness bounds on a multiprocessor. Our algorithm provides temporal isolation among classes, allows the available processing capacity to be fully utilized, and does not require that previously established per-task weight restrictions be made more stringent.

We present a sufficient linear-time schedulability test for preemptable, asynchronous, periodic task systems with arbitrary relative deadlines, scheduled on a uniprocessor by an optimal scheduling algorithm. We show that this test is more accurate than the commonly-used density condition. We also present and discuss the results of our test with that of a pseudo-polynomial-time schedulability test presented previously for a restricted class of task systems in which utilization is strictly less than one.