Secure Neural Network Inference (SNNI) enables privacy-preserving inference on encrypted data with strong cryptographic guarantees. However, practical deployments suffer from high preprocessing overhead, significant communication costs, and sequential execution. These limitations lead to low throughput, underutilized system resources, long queueing delays, and poor scalability. This work introduces SwiftSNNI, a unified, resource-aware scheduling framework for SNNI. It implements a hybrid offline–online strategy that orchestrates offline preprocessing (Tpre,i) and online inference (Ton,i) jobs to maximize parallelism. By formulating SNNI scheduling as a constrained optimization problem, SwiftSNNI overlaps Tpre,i phase execution of future requests with active Ton,j, jobs. SwiftSNNI also incorporates optional advance notices to enable proactive Tpre,i, which further reduces average input delay (D). Evaluations using five benchmark neural networks (M1, M2, HiNet, AlexNet, VGG-16) under diverse workloads and stochastic arrival rates confirm substantial performance gains. Compared to a parallelized sequential baseline (MS-SHARK), SwiftSNNI achieves up to 97% lower average input delay (D), a 81% reduction in makespan (≈ 5.4 × speedup), and delivers 5.6 × increase in throughput. Furthermore, SwiftSNNI reduces average waiting time (W) by over 99%, demonstrating robust starvation prevention for high-concurrency workloads. SwiftSNNI supports concurrent execution, scales to larger neural networks, and provides an efficient runtime for SNNI deployments. The SwiftSNNI implementation is available online.

Secure neural network inference (SNNI) enables the use of deep neural networks in scenarios involving multiple stakeholders, protecting the confidentiality of client data and of the neural network’s parameters. The cryptographic techniques used introduce high computational overhead, leading to significant energy consumption. Reducing the energy consumption of SNNI is thus an important objective. A prerequisite for energy optimization is the ability to profile the energy consumption of SNNI. However, this is challenging due to the complexity of the cryptographic techniques, neural networks, and technical setup involved. This paper is the first to propose an energy profiling approach for SNNI. Our approach measures the energy consumption for securely processing individual layers of the neural network, thus providing fine-grained insights into the energy profile of SNNI. We evaluate our approach using the ResNet50 neural network and the Cheetah SNNI framework. Our results show that we can reliably measure the energy consumption of individual layers. By introducing short periods of inactivity between layers to disentangle them, we achieve high correlation between execution time and energy consumption, suggesting that, under appropriate conditions, execution time may be used as a proxy for energy consumption. Our approach and insights can foster the design of more energy-efficient SNNI protocols.

IT systems are exposed to a rapidly changing landscape of serious security risks. Given the limited resources available to an organization, it is becoming more and more important to properly prioritize security risks, so that the organization can focus its efforts on the most critical risks. Traditionally, risks are assessed in terms of two aspects: occurrence probability and caused damage. However, for real-time risk prioritization, a third aspect is also of critical importance: urgency. Urgency stems from time-related considerations, such as the time needed by adversaries to exploit a vulnerability or the time needed for system administrators to put a countermeasure in place. These time-related considerations are orthogonal to the traditional aspects of occurrence probability and caused damage, and are largely ignored by existing risk management approaches. This paper proposes a way for introducing the notion of urgency into risk assessment. Our aim is to devise an intuitive approach for assessing risks, taking urgency into account, based on a solid theoretical underpinning. We establish a mathematical model using probability theory, and derive formulas for time-aware risk assessment in different settings.

In this paper, we present the SECURED project11Funded in part by the European Union (EU), Grant Agreement no. 10109571. Views and opinions expressed are those of the authors and do not necessarily reflect those of the EU or the Health and Digital Executive Agency. Neither the EU nor the granting authority are responsible for them., aimed at improving privacy-preserving processing of data in the health domain. The technologies developed in the project will be demonstrated in four health-related use cases and with the involvement of SME's selected through an open funding call.

Patient monitoring in hospitals, nursing centers, and home care can be largely automated using cameras and machine-learning-based video analytics, thus considerably increasing the efficiency of patient care. In particular, Facial-expression-based Pain Assessment Systems (FePAS) can automatically detect pain and notify medical personnel. However, current FePAS solutions using cloud-based video analytics offer very limited security and privacy protection. This is problematic, as video feeds of patients constitute highly sensitive information. To address this problem, we introduce SecFePAS, the first FePAS solution with strong security and privacy guarantees. SecFePAS uses advanced cryptographic protocols to perform neural network inference in a privacy-preserving way. To counteract the significant overhead of the used cryptographic protocols, SecFePAS uses multiple optimizations. First, instead of a cloud-based setup, we use edge computing with a 5G connection to benefit from lower network latency. Second, we use a combination of transfer learning and quantization to devise neural networks with high accuracy and optimized inference time. Third, SecFePAS quickly filters out unessential frames of the video to focus the in-depth analysis on key frames. We tested SecFePAS with the SqueezeNet and ResNet50 neural networks on a real pain estimation benchmark. SecFePAS outperforms state-of-the-art FePAS systems in accuracy and optimizes secure processing time.

In edge computing, application components can be placed over a range of computational devices from cloud data centers to nodes at the network edge. Application placement can have significant impact on important metrics like latency and resource utilization. Thus, application placement is an important optimization problem. In edge computing, the characteristics of both the infrastructure and the application may change over time, which may require the dynamic re-optimization of the application placement. Most algorithms suggested so far for the dynamic re-optimization of edge application placement are centralized, i.e., they rely on one entity collecting information from the whole infrastructure and making decisions centrally. However, centralized approaches suffer from limited scalability and are vulnerable to failures. In this paper, we present a decentralized approach for the dynamic re-optimization of edge application placement. We adopt an algorithm of Malek et al. for distributed systems and modify it to make it applicable to edge computing. In this approach, each node makes decisions autonomously, using auctions for coordination. Our empirical results demonstrate that the proposed algorithm is very effective in optimizing edge application placement. In an edge system with 637 edge nodes and 563 end devices, our algorithm achieves 54% higher reduction of application latency than a previous decentralized algorithm.

An edge data center can host applications that require low-latency access to nearby end devices. If the resource requirements of the applications exceed the capacity of the edge data center, some non-latency-critical application components may be offloaded to the cloud. Such offloading may incur financial costs both for the use of cloud resources and for data transfer between the edge data center and the cloud. Moreover, such offloading may violate data protection requirements if components process sensitive data. The operator of the edge data center has to decide which components to keep in the edge data center and which ones to offload to the cloud. In this paper, we formalize this problem and prove that it is strongly NP-hard. We introduce an optimization algorithm that is fast enough to be run online for dynamic and automatic offloading decisions, guarantees that the solution satisfies hard constraints regarding latency, data protection, and capacity, and achieves near-optimal costs. We also show how the algorithm can be extended to handle multiple edge data centers. Experiments show that the cost of the solution found by our algorithm is on average only 2.7% higher than the optimum.

Data processing systems operate in increasingly dynamic environments, such as in cloud or edge computing. In such environments, changes at run time can result in the dynamic appearance of data protection vulnerabilities, i.e., configurations in which an attacker could gain unauthorized access to confidential data. An autonomous system can mitigate such vulnerabilities by means of automated self-adaptations. If there are several data protection vulnerabilities at the same time, the system has to decide which ones to address first. In other areas of cybersecurity, risk-based approaches have proven useful for prioritizing where to focus efforts for increasing security. Traditionally, risk assessment is a manual and time-consuming process. On the other hand, addressing run-time risks requires timely decision-making, which in turn necessitates automated risk assessment.

Data produced by end devices like smartphones, sensors or IoT devices can be stored and processed across a continuum of compute resources, from end devices via fog nodes to the cloud, enabling reduced latency, increased processing speed and energy savings. However, the data may be sensitive (e.g., personal data or confidential commercially sensitive information), with regulatory or other requirements for its protection. Protecting sensitive data in the dynamic, heterogeneous, and decentralized cloud-to-edge continuum is very challenging. This paper describes a solution: FogProtect, an integrated set of four technologies to protect data in the cloud-to-edge continuum. Fog-Protect addresses four concerns: (i) control and enforcement of distributed data access and usage; (ii) management of distributed data protection policies; (iii) risk assessment for data assets in the cloud-to-edge continuum; (iv) automated optimisation and adaptation to address identified risks. FogProtect operates dynamically, reacting to system changes or detected vulnerabilities to keep the data secure across the cloud- to-edge continuum. This paper describes an overview of the FogProtect concept, discusses each of the four approaches, and illustrates their usage for the protection of data in three real-world use cases.

Edge computing enables the processing of data - frequently personal data - at the edge of the network. For personal data, legislation such as the European General Data Protection Regulation requires data protection by design. Hence, data protection has to be accounted for in the design of edge computing systems whenever personal data is involved. This leads to specific requirements for modeling the architecture of edge computing systems, e.g., representation of data and network properties. To the best of our knowledge, no existing modeling language fulfils all these requirements. In our previous work we showed that the commonly used UML profile UMLsec fulfils some of these requirements, and can thus serve as a starting point. The aim of this paper is to create a modeling language which meets all requirements concerning the design of the architecture of edge computing systems accounting for data protection. Thus, we extend UMLsec to satisfy all requirements. We call the resulting UML profile UMLsec4Edge. We follow a systematic approach to develop UMLsec4Edge. We app UMLsec4Edge to real-world use cases from different domains, and create appropriate deployment diagrams and class diagrams. These diagrams show UMLsec4Edge is capable of meeting the requirements.

In recent years, cloud computing concepts have been extended towards the network edge, leading to paradigms like fog and edge computing. As a result, applications can be placed on a variety of resources, including fog nodes and cloud data centers. Application placement has significant impact on important metrics like latency. Finding an optimal application placement is computationally challenging, particularly because of the potentially huge number of infrastructure nodes and application components. To overcome the limited scalability of application placement algorithms, optimization can be decentralized, i.e., performed separately for different parts of the infrastructure. The infrastructure can be split into fog colonies, where a fog colony consists of the computational resources in a given geographical region. Application placement can then be performed for the individual fog colonies, thus mitigating the scalability problem. However, independent optimization of application placement in different fog colonies may lead to missed synergies and thus to sub-optimal overall results. Hence, some kind of coordination between fog colonies may be beneficial. In this article, we analyze the effects of decentralization and coordination on the optimization results. In particular, we compare empirically four different approaches: (i) centralized decision-making, where decisions are made in one go for the entire infrastructure, (ii) independent fog colonies, where optimization is carried out in each fog colony independently from each other, (iii) fog colonies with communication, where excess application components in one fog colony can be sent to a neighboring fog colony, and (iv) fog colonies with overlaps, where shared resources may be dynamically distributed between neighboring fog colonies. Our experiments show that, for large problem instances, decentralization combined with coordination leads to the best results.

Abstract Workload consolidation is an important method for the efficient operation of cloud data centers, impacting important quality attributes such as resource utilization and power consumption. Many different approaches have been proposed for workload consolidation, but few comparative studies were executed to date. Therefore, it is unclear which of the proposed approaches work best in which situation. In this article, we present a comprehensive simulation‐based comparison of five workload consolidation techniques. We introduce a general framework for workload consolidation techniques to the DISSECT‐CF simulator to foster the development and comparison of efficient data center consolidation algorithms. We use this framework to evaluate the effectiveness of a first fit best fit decreasing heuristic, a custom heuristic, and three population‐based metaheuristics (genetic algorithm, artificial bee colony, and particle swarm optimization). The evaluation is based on a wide variety of real‐world workload traces. The five algorithms are compared in terms of total energy consumption, the duration of the simulation, and the number of migrations. Based on the results, there is no generally best consolidation technique. The results deliver insight into the pros and cons of the algorithms as well as the impact of different parameters. In particular, the results show that population‐based metaheuristics do not offer a significant gain in terms of solution quality to compensate for the increased simulation time.

The risk of unauthorized access to confidential data is a major problem in cloud computing. In previous work, the notion of risk patterns was introduced to capture configurations of cloud systems that are prone to data protection issues. In this paper, we devise a program for the automatic detection of risk patterns in cloud system models. Our program makes use of the Eclipse Modeling Framework, the model transformation library Henshin, and the modeling workbench Sirius to (i) enable security experts to describe cloud risk patterns in a compact way, (ii) enable the efficient automatic detection of risk patterns in the model of a cloud system, and (iii) support cloud experts in experimenting with the security implications of different cloud configurations. A case study and experiments demonstrate the applicability and scalability of the proposed approach.

Protecting sensitive data is a key concern for the adoption of cloud solutions. Protecting data in the cloud is made particularly challenging by the dynamic changes that cloud systems may undergo at run-time, as well as the complex interactions among multiple software and hardware components, services, and stakeholders. Conformance to data protection requirements in such a dynamic environment cannot any longer be ensured during design time; e.g. due to the dynamic changes imposed by replication and migration of components. It requires run-time data protection mechanisms. This paper proposes combining multiple existing data protection approaches and extending them to run-time, ultimately delivering an end-to-end architecture for run-time data protection in the cloud. We validate the practical applicability of our approach by a commercial case study.

Summary In recent years, many algorithms have been proposed for the optimized allocation of virtual machines in cloud data centers. Such algorithms are usually implemented and evaluated in a cloud simulator. This paper investigates the impact of the choice of cloud simulator on the implementation of the algorithms and on the evaluation results. In particular, we report our experiences with porting an algorithm and its evaluation framework from one simulator (CloudSim) to another (DISSECT‐CF). Our findings include limitations in the design of the simulators and in existing algorithm implementations. Based on this experience, we propose architectural guidelines for the integration of virtual machine allocation algorithms into cloud simulators.

To adapt to continuously changing workloads in networks, components of the running network services may need to be replicated (scaling the network service) and allocated to physical resources (placement) dynamically, also necessitating dynamic re-routing of flows between service components. In this paper, we propose joint optimization of scaling, placement, and routing (JASPER), a fully automated approach to jointly optimizing scaling, placement, and routing for complex network services, consisting of multiple (virtualized) components. JASPER handles multiple network services that share the same substrate network; services can be dynamically added or removed and dynamic workload changes are handled. Our approach lets service designers specify their services on a high level of abstraction using service templates. JASPER automatically makes scaling, placement and routing decisions, enabling quick reaction to changes. We formalize the problem, analyze its complexity, and develop two algorithms to solve it. Extensive empirical results show the applicability and effectiveness of the proposed approach.

Previous work on optimizing resource provisioning in virtualized environments focused either on mapping virtual machines (VMs) to physical machines (PMs) or mapping application components to VMs. In this paper, we argue that these two optimization problems influence each other significantly and in a highly non-trivial way. We define a sophisticated problem formulation for the joint optimization of the two mappings, taking into account sizing aspects, colocation constraints, license costs, and hardware affinity relations. As demonstrated by the empirical evaluation on a real-world workload trace, the combined optimization leads to significantly better overall results than considering the two problems in isolation.

The management of complex network services requires flexible and efficient service provisioning as well as optimized handling of continuous changes in the workload of the services. To adapt to changes in the demand, service components need to be replicated (scaling) and allocated to physical resources (placement) dynamically. In this paper, we propose a fully automated approach to the joint optimization problem of scaling and placement, enabling quick reaction to changes. We formalize the problem, analyze its complexity, and develop two algorithms to solve it. Empirical results show the applicability and effectiveness of the proposed approach.

Concerns about protecting personal data and intellectual property are major obstacles to the adoption of cloud services. To ensure that a cloud tenant's data cannot be accessed by malicious code from another tenant, critical software components of different tenants are traditionally deployed on separate physical machines. However, such physical separation limits hardware utilization, leading to cost overheads due to inefficient resource usage. Secure hardware enclaves offer mechanisms to protect code and data from potentially malicious code deployed on the same physical machine, thereby offering an alternative to physical separation. We show how secure hardware enclaves can be employed to address data protection concerns of cloud tenants, while optimizing hardware utilization. We provide a model, formalization and experimental evaluation of an efficient algorithmic approach to compute an optimized deployment of software components and virtual machines, taking into account data protection concerns and the availability of secure hardware enclaves. Our experimental results suggest that even if only a small percentage of the physical machines offer secure hardware enclaves, significant cost savings can be achieved.

Summary One of the key problems for Infrastructure‐as‐a‐Service providers is finding the optimal allocation of virtual machines on the physical machines available in the provider's data center. Since the allocation has significant impact on operational costs as well as on the performance of the accommodated applications, several algorithms have been proposed for the virtual machine placement problem. So far, no objective comparison of the proposed algorithms has been provided; therefore, it is not known which one works best or what factors influence the performance of the algorithms. In this paper, we present an environment and methodology for such comparisons and compare 7 different algorithms using the proposed environment and methodology. Our results showcase differences of up to 66% between the effectiveness of different algorithms on the same real‐world workload traces, thus underlining the importance of objectively comparing the performance of competing algorithms.

Finding the best way to map virtual machines (VMs) to physical machines (PMs) in a cloud data center is an important optimization problem, with significant impact on costs, performance, and energy consumption. In most situations, the computational capacity of PMs and the computational load of VMs are a vital aspect to consider in the VM-to-PM mapping. Previous work modeled computational capacity and load as one-dimensional quantities. However, today's PMs have multiple processor cores, all of which can be shared by cores of multiple multicore VMs, leading to complex scheduling issues within a single PM, which the one-dimensional problem formulation cannot capture. In this paper, we argue that at least a simplified model of these scheduling issues should be taken into account during VM placement. We show how constraint programming techniques can be used to solve this problem, leading to significant improvement over non-multicore-aware VM placement. Several ways are presented to hybridize an exact constraint solver with common packing heuristics to derive an effective and scalable algorithm.

In “Process placement in multicore clusters: Algorithmic issues and practical techniques,” Jeannot, Mercier, and Tessier presented an algorithm called TreeMatch for determining the best placement of a set of communicating processes on a hierarchically structured computing architecture, described by a tree. In order to speed up the algorithm, it was suggested to decompose levels of the tree with high arity into several levels of smaller arity. The authors conjectured what the optimal strategy for decomposition is. In this contribution, we prove that their conjecture was right.

In the distributed cloud paradigm, data centers are geographically dispersed and interconnected over a wide-area network. Due to the geographical distribution of data centers, communication networks play an important role in distributed clouds in terms of communication cost and QoS. Large-scale, processing-intensive tasks require the cooperation of many VMs, which may be distributed in more than one data center and should communicate with each other. In this setting, the number of data enters serving the given task and the network distance among those data centers have critical impact on the communication cost, traffic and even completion time of the task. In this paper, we present the NACER algorithm, a Network-Aware Cost-Efficient Resource allocation method for optimizing the placement of largemulti-VM tasks in distributed clouds. NACER builds on ideas of the A* search algorithm from Artificial Intelligence research in order to obtain better results than typical greedy heuristics. We present extensive simulation results to compare the performance of NACER with competing heuristics and show its effectiveness.

One of the most crucial steps in the design of embedded systems is hardware/software partitioning, that is, deciding which components of the system should be implemented in hardware and which ones in software. Most formulations of the hardware/software partitioning problem are NP-hard, so the majority of research efforts on hardware/software partitioning has focused on developing efficient heuristics.This article considers the combinatorial structure behind hardware/software partitioning. Two similar versions of the partitioning problem are defined, one of which turns out to be NP-hard, whereas the other one can be solved in polynomial time. This helps in understanding the real cause of complexity in hardware/software partitioning. Moreover, the polynomial-time algorithm serves as the basis for a highly efficient novel heuristic for the NP-hard version of the problem. Unlike general-purpose heuristics such as genetic algorithms or simulated annealing, this heuristic makes use of problem-specific knowledge, and can thus find high-quality solutions rapidly. Moreover, it has the unique characteristic that it also calculates lower bounds on the optimum solution . It is demonstrated on several benchmarks and also large random examples that the new algorithm clearly outperforms other heuristics that are generally applied to hardware/software partitioning.

Abstract Traditional tracing mechanisms, usually developed for use in a single‐computer environment, are bound to a specific programming language. Today's highly distributed and heterogeneous computing environments require new tracing methodologies. This paper addresses the problem by reviewing ways in which the middleware might—and should—support tracing. In particular, CORBA (Common Object Request Broker Architecture) meta‐objects that can be applied for tracing are studied. One meta‐object, namely the interceptor concept, is presented in more depth, followed by a detailed description of an interceptor‐based tracing architecture for CORBA applications. Implementation details and evaluation experience are given. Copyright © 2004 John Wiley & Sons, Ltd.