Data centers consume a large amount of energy, much of which is wasted due to idle servers. Turning off idle servers might be an effective power-saving solution; however, there is a trade-off between energy savings and system performance . Hence, we propose a setup queueing model with a batching policy that allows servers to process a set of jobs simultaneously to minimize power consumption while maintaining acceptable performance. We consider an M/M/ c / SET–BATCH queue, a multi-server batch service queue with a fixed batch size and setup times, and some variants, including systems in which idle servers delay before turning off or systems in which the batch size is dynamic. We analyze the steady-state probabilities and system performance of the M/M/ c / SET–BATCH system and its variants. Our analysis of the M/M/ c / SET–BATCH system with lower computational complexity is made possible by utilizing the special structure of the model. In addition, we use simulations to compare the M/M/ c / SET–BATCH model with some other variants with different setup time distributions. The results suggest that the model performs better when the setup time has a larger coefficient of variation . Our results indicate that the batching policy enhances the system performance, especially when we allow servers to be idle before turning them off.

As machine learning models have been deployed in many vision systems, including autonomous vehicles and robots, designing architectures for machine learning systems (MLSs) has emerged as a critical concern. Previous studies have shown that enhancing the reliability of MLS outputs can be achieved by comparing multiple inference results on distinct inputs. Nevertheless, the architectures facilitating multiple inferences incur non-negligible performance overhead and energy consumption that have been less investigated. This article delves into the trade-offs among reliability, performance, and energy efficiency of architectures for two-input MLSs through real experiments conducted on image classification and object detection tasks. Specifically, we scrutinize the comparison between parallel- and shared-type architectures of two-input MLSs for vision tasks. The experiments confirm that the shared-type architecture can achieve a shorter response time and smaller energy consumption by using a shared machine learning module for both image classification and object detection tasks. However, the parallel-type architecture can benefit the redundant machine learning modules for improving throughput and fault tolerance. Our empirical results also show the service time distributions of image classification and object detection tasks fit well with a log-normal distribution and a mixture of the Gaussian model, respectively.

The multiple-input machine learning system (MLS) is a system architecture exploiting data diversity to improve the output reliability of the system by comparing prediction results on multiple input data. While the output reliability is enhanced by redundancy, the architecture imposes additional costs and non-negligible processing overheads. The performance of multiple-input MLSs has been theoretically investigated in the previous study using queueing analysis. However, it is little known how real MLSs are impacted by the multiple predictions and comparison processes needed in the architecture. In this paper, we implement two-input MLSs in two different configurations, a parallel type architecture and a shared type architecture, and evaluate the reliability, performance, and energy consumption of the system by experiments. Our empirical results unveil several advantages of the shared type architecture that can suppress the increases in response time and energy consumption by using a shared machine learning module for predictions of two inputs. We also compare the results of the performance simulation of two-input MLS with the empirical results. While we confirm the effectiveness of the simulation, we also find some gaps in the real observations. For example, we observe that the inference time distribution fits well in the log-normal distribution rather than the exponential distribution assumed in the simulation. Our findings could be useful for developing performance models for multiple-input MLSs.

In 5G, network functions can be scaled out/in dynamically to adjust the capacity for network slices. The scale-out/-in procedure, namely autoscaling, enhances performance by scaling out instances and reduces operational costs by scaling in instances. However, the autoscaling problems in 5G networks are different from those in traditional cloud computing. The 5G network functions must be considered the simultaneous deployment of multiple instances; moreover, the deployment of 5G network functions is more frequent than that of traditional cloud computing. Both the number and timing of deployment will substantially affect the cost-effectiveness of the system. In this paper, we first identify the autoscaling issues specifically based on the 3GPP standards. We develop a low-complexity analytical queuing model to formulate the problem and quantify a set of performance metrics with closed-form solutions. The proposed analytical model and closed-form solutions are cross-validated by extensive simulations. The analytical model offers design insights and theoretical guidelines, helping us study the effectiveness of reservations. We proposed a dynamic block-setup reservation algorithm (DBRA) to find the optimal reserved number and threshold value of network slices. Therefore, mobile operators can balance the system's cost-effectiveness without large-scaled testing and real deployment, saving cost on time and money.

We calculate asymptotics of the distribution of the number of customers in orbit in a two-class priority retrial M ∕ G ∕ 1 -type queueing model. In this model, priority customers wait in line while non-priority customers join an orbit and retry later. Although the generating function and moments of the number of customers in orbit have been analyzed before, asymptotics of the distribution have not been thoroughly investigated, mainly because of the complex nature of the generating function. We show that we can use singularity analysis of the probability generating function to do just that. Our results show that different regimes exist for these asymptotics in case of light-tailed service times: in what we call the ‘priority regime’, the tail asymptotics have the same decay ( ∼ c n − 3 ∕ 2 R − n ) as in the priority non-retrial queue and the retrial rate only influences the constant c . In the ‘retrial regime’, the retrial rate also influences the sub-exponential factor of the asymptotics. In this regime, asymptotics are very similar to asymptotics in retrial queues without (priority) waiting line. Finally, we also analyze the case that the service time distribution is power law (with or without exponential cut-off) using the same technique.

In cloud computing paradigm, virtual resource autoscaling approaches have been intensively studied recent years. Those approaches dynamically scale in/out virtual resources to adjust system performance for saving operation cost. However, designing the autoscaling algorithm for desired performance with limited budget, while considering the existing capacity of legacy network equipment, is not a trivial task. In this paper, we propose a Deadline and Budget Constrained Autoscaling (DBCA) algorithm for addressing the budget-performance tradeoff. We develop an analytical model to quantify the tradeoff and cross-validate the model by extensive simulations. The results show that the DBCA can significantly improve system performance given the budget upper-bound. In addition, the model provides a quick way to evaluate the budget-performance tradeoff and system design without wide deployment, saving on cost and time.

Network Function Virtualization (NFV) enables mobile operators to virtualize their network entities as Virtualized Network Functions (VNFs), offering fine-grained on-demand network capabilities. VNFs can be dynamically scale-in/out to meet the performance desire and other dynamic behaviors. However, designing the auto-scaling algorithm for desired characteristics with low operation cost and low latency, while considering the existing capacity of legacy network equipment, is not a trivial task. In this paper, we propose a VNF Dynamic Auto Scaling Algorithm (DASA) considering the tradeoff between performance and operation cost. We develop an analytical model to quantify the tradeoff and validate the analysis through extensive simulations. The results show that the DASA can significantly reduce operation cost given the latency upper-bound. Moreover, the models provide a quick way to evaluate the cost- performance tradeoff and system design without wide deployment, which can save cost and time.