Abstract The scope of the Internet of Things (IoT) environment has been expanding from private to public spaces, where selecting the most appropriate service by predicting the service quality has become a timely problem. However, IoT services can be physically affected by (1) uncertain environmental factors such as obstacles and (2) interference among services in the same environment while interacting with users. Using the traditional modeling-based approach, analyzing the influence of such factors on the service quality requires modeling efforts and lacks generalizability. In this study, we propose Learning Physical Environment factors based on the Attention mechanism to Select Services for UsERs (PLEASSURE), a novel framework that selects IoT services by learning the uncertain influence and predicting the long-term quality from the users’ feedback without additional modeling. Furthermore, we propose fingerprint attention that extends the attention mechanism to capture the physical interference among services. We evaluate PLEASSURE by simulating various IoT environments with mobile users and IoT services. The results show that PLEASSURE outperforms the baseline algorithms in rewards consisting of users’ feedback on satisfaction and interference.

The Internet of Things (IoT) has gained more attention for enhancing users’ daily lives in public spaces by providing services using shareable devices. However, uncertain factors and other services in the environment may affect the service severely, resulting in users’ low satisfaction. Based on multiagent reinforcement learning and cluster-based federated learning, autonomous service agents may learn the complex influence of the factors from user feedback without sophisticated modeling and detection processes. However, conventional approaches are limited in dealing with multiple clustering dimensions of service agents and dynamic environmental contexts affecting the agents. In this work, we propose the multidimension and multiagent federated reinforcement learning (MultiFedRL) for efficient training of service agents in public IoT environments. First, we suggest a parallel structure of neural networks for multiple clustering dimensions to share parameters independently, solving the limitation of conventional cluster-based federated learning. Second, we suggest an environment-centric learnable communication protocol for the agents to summarize and interpret physical contexts consisting of static characteristics and dynamic states. To evaluate MultiFedRL, we developed a simulation framework for IoT services provided to mobile users in public spaces, imitating the user-service interaction based on crucial physics phenomena. Experimental results show that MultiFedRL increases user satisfaction by 82.9% and training efficiency by 24.5% compared to state-of-the-art cluster-based federated learning.

As the number of IoT devices and the volume of data increase, distributed computing systems have become the primary deployment solution for large-scale Internet of Things (IoT) environments. Federated learning (FL) is a collaborative machine learning framework that allows for model training using data from all participants while protecting their privacy. However, traditional FL suffers from low computational and communication efficiency in large-scale hierarchical cloud-edge collaborative IoT systems. Additionally, due to heterogeneity issues, not all IoT devices necessarily benefit from the global model of traditional FL, but instead require the maintenance of personalized levels in the global training process. Therefore we extend FL into a horizontal peer-to-peer (P2P) structure and introduce our P2PFL framework: efficient peer-to-peer federated learning for users (EPFLU). EPFLU transitions the paradigms from vertical FL to a horizontal P2P structure from the user perspective and incorporates personalized enhancement techniques using private information. Through horizontal consensus information aggregation and private information supplementation, EPFLU solves the weakness of traditional FL that dilutes the characteristics of individual client data and leads to model deviation. This structural transformation also significantly alleviates the original communication issues. Additionally, EPFLU has a customized simulation evaluation framework, and uses the EUA dataset containing real-world edge server distribution, making it more suitable for real-world large-scale IoT. Within this framework, we design two extreme data distribution scenarios and conduct detailed experiments of EPFLU and selected baselines on the MNIST and CIFAR-10 datasets. The results demonstrate that the robust and adaptive EPFLU framework can consistently converge to optimal performance even under challenging data distribution scenarios. Compared with the traditional FL and selected P2PFL methods, EPFLU achieves communication time improvements of 39% and 16% respectively.

The emergence of fog computing has shown promising results for reducing network latency and congestion in the cloud. In this environment, effective autoscaling to handle spatio-temporal load bursts in geographically distributed and resource-constrained fog nodes has become a timely problem. A typical strategy for autoscaling is based on centralized monitoring of the fog nodes and the deployed service instances. However, centralized collection and analysis of the metrics for autoscaling can become infeasible with the increasing number of fog nodes. Moreover, the dynamic and fluctuating characteristics of the fog nodes make effective autoscaling challenging in fog computing environments. In this work, we propose HiDRA, a Hierarchical Decentralized Autoscaler that scales and places microservice instances based on multi-agent reinforcement learning. In HiDRA, agents are divided into scaling and placement agents that collaborate with each other to effectively handle spatio-temporal load bursts in fog computing. These Deep Q-Network-based autoscaling agents are trained solely based on their regional observations at runtime, eliminating the need for a centralized collection of metrics. We evaluated HiDRA in multiple simulated fog environments created using a real-world dataset. The environments were divided into three levels of sparsity, each consisting of 20, 15, and 10 initial instances and unstable nodes. The result shows that comparatively by ratio against the baseline, HiDRA increased the average request success rate by 10.7%, 16.4%, and 36.7% and reduced the number of created instances by 12.3%, 15.9%, and 16.8% in environments with 20, 15, and 10 initial instances and unstable nodes, respectively.

Pre-trained models (PTMs) have revolutionized machine learning by significantly enhancing reusability and reducing resources required for model training. Despite their advantages, selecting appropriate PTMs for specific system requirements remains difficult due to application heterogeneity and variable task performance. This has led to the proposal of PTM compositions to enhance capabilities beyond individual models, with an on-the-fly approach being applied further to meet dynamic requirements. However, composing PTMs on-the-fly can result in objective mismatch, leading to inefficiencies and errors in constituent PTMs. This necessitates the timely and efficient replacement of underperforming PTMs. The process is a major challenge due to the vast number of candidates and the extensive time required for evaluation. Therefore, we propose the Sample-Infer-Predict framework for efficient on-the-fly PTM replacement which comprises three phases: sampling, inference, and prediction. First, our novel Density and Diversity sampling algorithm efficiently selects representative user inputs. Second, the inference phase evaluates candidate PTMs from model hubs for the prediction dataset. Third, the prediction phase utilizes the dataset to predict the optimal PTM replacements. We evaluate our methodology using a vehicle detection PTM composition and a dataset of 5849 vehicle images, focusing on the efficiency of the replacement process, the quality of the meta-predictions, and the effectiveness of our sampling technique. Our approach reduces replacement time (155s vs. 28293s), achieves high precision@k values, and lowers mean absolute error values compared to other sampling techniques.

While the emergence of fog computing has shown promising results for reducing the network latency and congestion in the cloud, effective autoscaling to handle spatio-temporal bursts of load has become a timely problem. In this work, we propose HiDRA, a Hierarchical Decentralized Reactive Autoscaler that scales and places microservice instances based on multi-agent reinforcement learning. In HiDRA, the agents are divided into scaling and placement agents that collaborate with each other to effectively handle spatio-temporal bursts of load in fog computing. These hierarchical agents are trained solely based on their regional observations at runtime, eliminating the need for a centralized collection of metrics. We evaluated HiDRA in 20 simulated fog environments to show that HiDRA reduces the created number of instances by 36.70% while resulting in similar scaling performance in terms of response success rate.

In the context of the recent emergence of the Internet of Things (IoT), human users and IoT-based services are interacting via physical effects, such as light and sound. Therefore, it is necessary to consider the quality of the delivery of physical effects to users by IoT devices for selecting services in IoT environments. However, traditional service-selection algorithms focus primarily on the network-level Quality of Service (QoS), such as latency and throughput. In this study, we improve on the visual-service effectiveness metric developed in our previous work to measure the effectiveness of the personalized delivery of physical effects of visual services to users by considering user- and application-specific factors. We evaluate the metric by conducting a user study, and the results show that the metric reflects users’ perceived effectiveness with high accuracy. We also investigate the use of virtual reality (VR) to imitate physical environments for efficient evaluation of the metric. Based on this metric, we develop a dynamic effect-driven output-service selection agent (DEOSA) that selects output services dynamically by considering the effectiveness of service-effect delivery. By adopting a state-of-the-art reinforcement-learning algorithm, DEOSA can learn the optimal policy for selecting output services that can be generalized to various environments. We evaluate DEOSA in simulated IoT environments and show that it can learn the optimal policy successfully; it generally performs better than traditional greedy algorithms in terms of the visual service effectiveness metric and the replacement overhead in randomly generated test environments.

With the increasing number of Web of Things devices, the network and processing delays in the cloud have also increased. As a solution, fog computing has emerged, placing computational resources closer to the user to lower the communication overhead and congestion in the cloud. In fog computing systems, microservices are deployed as containers, which require an orchestration tool like Kubernetes to support service discovery, placement, and recovery. A key challenge in the orchestration of microservices is automatically scaling the microservices in case of an unpredictable burst of load. In cloud computing, a centralized autoscaler can monitor the deployed microservice instances and make scaling actions based on the monitored metric values. However, monitoring an increasing number of microservices in fog computing can cause excessive network overhead and thereby delay the time to scaling action. We propose DESA, a fully DEcentralized Self-adaptive Autoscaler through which microservice instances make their own scaling decisions, cloning or terminating themselves through self-monitoring. We evaluate DESA in a simulated fog computing environment with different numbers of fog nodes. Furthermore, we conduct a case study with the 1998 World Cup website access log, examining DESA’s performance in a realistic scenario. The results show that DESA successfully reduces the scaling reaction time in large-scale fog computing systems compared to the centralized approach. Moreover, DESA resulted in a similar maximum number of instances and lower average CPU utilization during bursts of load.

In highly dynamic IoT environments such as Vehicle-to-Everything (V2X), discovering and providing Internet of Things (IoT) services to users is a challenging problem because the environments alter too quickly and unpredictably. For a fast and effective service discovery in a highly dynamic environment, we propose a cached service registry on mobile entities along with new metrics for evaluating the effectiveness of service discoveries. We conducted an experiment on a simulated V2X environment, and showed that both the rate of finding accessible services and the utilization of services can be improved comparing to the traditional service discovery method.

The functions, capabilities, and effects produced by the application services of cyber physical systems (CPS) are usually consumed by users performing their daily activities in a variety of environmental conditions. Thus, it is critical to ensure that those systems neither interfere with human activities nor harm the users involved. In this paper, we propose a framework for testing and verifying the safety and reliability of CPS services from the perspectives of CPS environments and users. The framework provides an environmentaware testing method by which the efficiency of testing CPS services can be improved by prioritizing CPS environments and by applying machinelearning techniques. The framework also includes a metric by which we can automate the test of the most effective services that deliver effects from physical devices to users. Additionally, the framework provides a computational model that assesses mental workloads to test whether a CPS service can cause cognitive depletion or contention problems for users. We conducted a series of experiments to show the effectiveness of the proposed approaches for ensuring the safety and reliability of CPS application services during the development and operation phases.

Recent advances in Internet of Things (IoT) technologies have encouraged web services to expand their provision boundary to physical environments by utilizing IoT devices. IoT services that generate and deliver physical effects to users via a space utilize such IoT devices as media to interact within a physical environment. Existing studies on dynamic service selection have only considered network-level quality of service (QoS) attributes, which cannot be used to evaluate the quality of the delivery of physical effects from a user perspective. Furthermore, to provide the services in a continuous manner, a dynamic selection of physical media is essential. Herein we propose a new metric called visual service effectiveness to evaluate how well a visual effect, generated using an IoT device as a medium, can be delivered to a user. Based on this metric, we also propose an effect-driven dynamic medium selection agent (EDMS-Agent) that conducts medium selection to maximize the visual service effectiveness during runtime and can be trained using reinforcement learning algorithms. We evaluated our EDMS-Agent by conducting several experiments in simulated IoT environments. The results show that simple distance-based metric is insufficient to measure the quality of physical effects in the user's perspective, and EDMS-Agent performs better than the baselines in terms of effectiveness, by learning the optimal policy of selecting media.