Optimization is critical for improving the operations of large-scale socio-technical infrastructures such as those found in energy, mobility, and information systems. In particular, understanding the performance of multi-agent discrete-choice combinatorial optimization under distributed adversarial attacks is a compelling and underexplored problem. Multi-agent systems involve a large number of remote control variables that can influence the cost-effectiveness of distributed optimization heuristics. This paper unravels, for the first time, the trajectories of distributed optimization from resilience to vulnerability, and finally to collapse under varying adversarial influence. Using real-world and synthetic data to generate over 112 million multi-agent optimization scenarios, we systematically assess how the number of agents with varying levels of adversarial severity and network positioning influences optimization performance, with particular attention to the impact on Pareto optimality. With this large-scale dataset, made openly available as a benchmark, we disentangle how optimization systems remain resilient to adversaries and which adversary conditions make optimization vulnerable or cause collapse. These findings can support the design of self-healing strategies for fault tolerance and fault correction, addressing a critical gap in adversarial distributed optimization.

Gaming environments are popular testbeds for studying human interactions and behaviors in complex artificial intelligence systems. Particularly, in multiplayer online battle arena (MOBA) games, individuals collaborate in virtual environments of high realism that involves real-time strategic decision-making and trade-offs on resource management, information collection and sharing, team synergy and collective dynamics. This paper explores whether collective intelligence, emerging from cooperative behaviours exhibited by a group of individuals, who are not necessarily skillful but effectively engage in collaborative problem-solving tasks, exceeds individual intelligence observed within skillful individuals. This is shown via a case study in League of Legends, using machine learning algorithms and statistical methods applied to large-scale data collected for the same purpose. By modeling and visualizing systematically game-specific metrics but also new game-agnostic topological and graph spectra measures of cooperative interactions, we demonstrate compelling insights about the superior performance of collective intelligence.

The transformation of smart mobility is unprecedented—Autonomous, shared and electric-connected vehicles, along with the urgent need to meet ambitious net-zero targets by shifting to low-carbon transport modalities result in new traffic patterns and requirements for real-time computation at large-scale, for instance, self-driving cars and augmented reality applications. The cloud computing paradigm can neither respond to such low-latency requirements nor adapt resource allocation to such dynamic spatiotemporal service requests. This article addresses this grand challenge by introducing a novel decentralized optimization framework for mobility-aware edge-to-cloud resource allocation, service offloading, provisioning, and load-balancing. In contrast to related work, this framework comes with superior efficiency and cost-effectiveness under evaluation in real-world traffic settings and mobility datasets. This breakthrough capability of “computing follows vehicles” proves to be superior to balance resource utilization in a highly cost-effective way, while preventing service deadline violations by 14%–34%.

Distributed sensing by cooperative drone swarms is crucial for several Smart City applications, such as traffic monitoring and disaster response. Using an indoor lab with inexpensive drones, a testbed supports complex and ambitious studies on these systems while maintaining low cost, rigor, and external validity. This paper introduces the Multi-drone Sensing Experimentation Testbed (M-SET), a novel platform designed to prototype, develop, test, and evaluate distributed sensing with swarm intelligence. M-SET addresses the limitations of existing testbeds that fail to emulate collisions, thus lacking realism in outdoor environments. By integrating a collision avoidance method based on a potential field algorithm, M-SET ensures collision-free navigation and sensing, further optimized via a multi-agent collective learning algorithm. Extensive evaluation demonstrates accurate energy consumption estimation and a low risk of collisions, providing a robust proof-of-concept. New insights show that M-SET has significant potential to support ambitious research with minimal cost, simplicity, and high sensing quality.

This study addresses integrating blockchain technology into lightweight devices, specifically on clustered Wireless Sensor Networks (WSNs). Integrating blockchain in the WSNs solved the problem of heterogeneity, data integrity, and data confidentiality. However, no blockchain integration considers network lifetime in WSNs. This research focuses on developing a permissioned blockchain system that incorporates a consensus mechanism known as Proof-of-Authority (PoA) within clustered WSNs with two main features. The first is to enhance the network lifetime by introducing a rotational selection of block proposers using an Energy-Aware PoA (EA-PoA) weighting mechanism. Known as the Multi-Level Blockchain Model (MLBM), the subsequent feature is the creation of a hierarchical network model within a blockchain network. The MLBM network comprises both local and master blockchains. Each cluster inside a WSN possesses its local blockchain network. In MLBM, the local blockchain creates a block on the main blockchain by proposing the headers of every ten blocks to improve data integrity. Each local blockchain has its leader, which can increase block production. The results show that the proposed solution can overcome traditional PoA performance and is suitable for clustered WSNs. In terms of lifetime, the EA-PoA selection method can extend network lifetime by up to 10%. In addition, MLBM can increase block production by up to twice each additional cluster compared to a single blockchain network used in traditional PoA.

The research initiative “self-improving system integration” (SISSY) was established with the goal to master the ever-changing demands of system organisation in the presence of autonomous subsystems, evolving architectures, and highly-dynamic open environments. It aims to move integration-related decisions from design-time to run-time, implying a further shift of expertise and responsibility from human engineers to autonomous systems . This introduces a qualitative shift from existing self-adaptive and self-organising systems, moving from self-adaptation based on predefined variation types, towards more open contexts involving novel autonomous subsystems, collaborative behaviours, and emerging goals. In this article, we revisit existing SISSY research efforts and establish a corresponding terminology focusing on how SISSY relates to the broad field of integration sciences. We then investigate SISSY-related research efforts and derive a taxonomy of SISSY technology. This is concluded by establishing a research road-map for developing operational self-improving self-integrating systems.

Large-scale decentralized systems of autonomous agents interacting via asynchronous communication often experience the following self-healing dilemma: fault detection inherits network uncertainties making a remote faulty process indistinguishable from a slow process. In the case of a slow process without fault, fault correction is undesirable as it can trigger new faults that could be prevented with fault tolerance that is a more proactive system maintenance. But in the case of an actual faulty process, fault tolerance alone without eventually correcting persistent faults can make systems underperforming. Measuring, understanding and resolving such self-healing dilemmas is a timely challenge and critical requirement given the rise of distributed ledgers, edge computing, the Internet of Things in several energy, transport and health applications. This paper contributes a novel and general-purpose modeling of fault scenarios during system runtime. They are used to accurately measure and predict inconsistencies generated by the undesirable outcomes of fault correction and fault tolerance as the means to improve self-healing of large-scale decentralized systems at the design phase. A rigorous experimental methodology is designed that evaluates 696 experimental settings of different fault scales, fault profiles and fault detection thresholds in a prototyped decentralized network of 3000 nodes. Almost 9 million measurements of inconsistencies were collected in a network, where each node monitors the health status of another node, while both can defect. The prediction performance of the modeled fault scenarios is validated in a challenging application scenario of decentralized and dynamic in-network data aggregation using real-world data from a Smart Grid pilot project. Findings confirm the origin of inconsistencies at design phase and provide new insights how to tune self-healing at an early stage. Strikingly, the aggregation accuracy is well predicted as shown by high correlations and low root mean square errors.

The pervasiveness of Internet of Things results in vast volumes of personal data generated by smart devices of users (data producers) such as smart phones, wearables and other embedded sensors. It is a common requirement, especially for Big Data analytics systems, to transfer these large in scale and distributed data to centralized computational systems for analysis. Nevertheless, third parties that run and manage these systems (data consumers) do not always guarantee users’ privacy. Their primary interest is to improve utility that is usually a metric related to the performance, costs and the quality of service. There are several techniques that mask user-generated data to ensure privacy, e.g. differential privacy. Setting up a process for masking data, referred to in this paper as a ‘privacy setting’, decreases on the one hand the utility of data analytics, while, on the other hand, increases privacy. This paper studies parameterizations of privacy settings that regulate the trade-off between maximum utility, minimum privacy and minimum utility, maximum privacy, where utility refers to the accuracy in the estimations of aggregation functions. Privacy settings can be universally applied as system-wide parameterizations and policies (homogeneous data sharing). Nonetheless they can also be applied autonomously by each user or decided under the influence of (monetary) incentives (heterogeneous data sharing). This latter diversity in data sharing by informational self-determination plays a key role on the privacy-utility trajectories as shown in this paper both theoretically and empirically. A generic and novel computational framework is introduced for measuring privacy-utility trade-offs and their Pareto optimization. The framework computes a broad spectrum of such trade-offs that form privacy-utility trajectories under homogeneous and heterogeneous data sharing. The practical use of the framework is experimentally evaluated using real-world data from a Smart Grid pilot project in which energy consumers protect their privacy by regulating the quality of the shared power demand data, while utility companies make accurate estimations of the aggregate load in the network to manage the power grid. Over 20,000 differential privacy settings are applied to shape the computational trajectories that in turn provide a vast potential for data consumers and producers to participate in viable participatory data sharing systems.

The Internet of Things (IoT) comes along with new challenges for experimenting, testing, and operating decentralized socio-technical systems at large-scale. In such systems, autonomous agents interact locally with their users, and remotely with other agents to make intelligent collective choices. Via these interactions they self-regulate the consumption and production of distributed (common) resources, e.g., self-management of traffic flows and power demand in Smart Cities. While such complex systems are often deployed and operated using centralized computing infrastructures, the socio-technical nature of these decentralized systems requires new value-sensitive design paradigms; empowering trust, transparency, and alignment with citizens’ social values, such as privacy preservation, autonomy, and fairness among citizens’ choices. Currently, instruments and tools to study such systems and guide the prototyping process from simulation, to live deployment, and ultimately to a robust operation of a high Technology Readiness Level (TRL) are missing, or not practical in this distributed socio-technical context. This paper bridges this gap by introducing a novel testbed architecture for decentralized socio-technical systems running on IoT. This new architecture is designed for a seamless reusability of (i) application-independent decentralized services by an IoT application, and (ii) different IoT applications by the same decentralized service. This dual self-integration promises IoT applications that are simpler to prototype, and can interoperate with decentralized services during runtime to self-integrate more complex functionality, e.g., data analytics, distributed artificial intelligence. Additionally, such integration provides stronger validation of IoT applications, and improves resource utilization, as computational resources are shared, thus cutting down deployment and operational costs. Pressure and crash tests during continuous operations of several weeks, with more than 80K network joining and leaving of agents, 2.4M parameter changes, and 100M communicated messages, confirm the robustness and practicality of the testbed architecture. This work promises new pathways for managing the prototyping and deployment complexity of decentralized socio-technical systems running on IoT, whose complexity has so far hindered the adoption of value-sensitive self-management approaches in Smart Cities.

Communication structure plays a key role in the learning capability of decentralized systems. Structural self-adaptation, by means of self-organization, changes the order as well as the input information of the agents' collective decision-making. This paper studies the role of agents' repositioning on the same communication structure, i.e. a tree, as the means to expand the learning capacity in complex combinatorial optimization problems, for instance, load-balancing power demand to prevent blackouts or efficient utilization of bike sharing stations. The optimality of structural self-adaptations is rigorously studied by constructing a novel large-scale benchmark that consists of 4000 agents with synthetic and real-world data performing 4 million structural self-adaptations during which almost 320 billion learning messages are exchanged. Based on this benchmark dataset, 124 deterministic structural criteria, applied as learning meta-features, are systematically evaluated as well as two online structural self-adaptation strategies designed to expand learning capacity. Experimental evaluation identifies metrics that capture agents with influential information and their optimal positioning. Significant gain in learning performance is observed for the two strategies especially under low-performing initialization. Strikingly, the strategy that triggers structural self-adaptation in a more exploratory fashion is the most cost-effective.

The Internet of Things equips citizens with a phenomenal new means for online participation in sharing economies. When agents self-determine options from which they choose, for instance, their resource consumption and production, while these choices have a collective systemwide impact, optimal decision-making turns into a combinatorial optimization problem known as NP-hard. In such challenging computational problems, centrally managed (deep) learning systems often require personal data with implications on privacy and citizens’ autonomy. This article envisions an alternative unsupervised and decentralized collective learning approach that preserves privacy, autonomy, and participation of multi-agent systems self-organized into a hierarchical tree structure. Remote interactions orchestrate a highly efficient process for decentralized collective learning . This disruptive concept is realized by I-EPOS, the Iterative Economic Planning and Optimized Selections , accompanied by a paradigmatic software artifact. Strikingly, I-EPOS outperforms related algorithms that involve non-local brute-force operations or exchange full information. This article contributes new experimental findings about the influence of network topology and planning on learning efficiency as well as findings on techno-socio-economic tradeoffs and global optimality. Experimental evaluation with real-world data from energy and bike sharing pilots demonstrates the grand potential of collective learning to design ethically and socially responsible participatory sharing economies.

The emergence of the Internet of Things in Smart Cities questions how the future citizens will perceive their predominant living and working environments and what quality of living they can experience within it, for instance the level of everyday stress. However, perception and experienced stress levels are challenging metrics to measure and are even more challenging to correlate with an underlying causal-effectual relationship in such stimulus abundant environments. The Internet of Things, enabled by several pervasive and ubiquitous devices such as smart phones and smart sensors, can provide real-time contextual information that can be used by advanced data science methodologies to generate new insights about urban qualities in Smart Cities and how they can be improved. The goal of this study is to show the predominant factors, which influence perceptual qualities of inhabitants in a Smart City equipped with sensing capabilities by the Internet of Things. To serve this goal, a novel data collection process for Smart Cities is introduced that involves (i) environmental data, such noise, dust, illuminance, temperature, relative humidity, (ii) location/mobility data, such as GNSS and citizens density detected via WiFi, and (iii) perceptual social data collected by citizens' responses in smart phones. These fine-grained real-time data can provide invaluable insights about the spatial correlations of the sensor measurements as well as the spatial and citizens' similarity illustrated. The data analysis illustrated reveals significant links between stress level and environmental changes observed.

The pervasiveness of Internet of Things devices in techno-socio-economic domains such as Smart Cities and Smart Grids results in a massive scale of data about our society. Decision-making by system operators or policy-makers requires a sophisticated understanding of these data with real-time data analytics methods. However, common data analytics methods often serve exclusively corporate and commercial interests and result in privacy-intrusion, surveillance, profiling and discriminatory actions. This paper illustrates an alternative data analytics approach that relies on participatory citizens to contribute Internet of Things data and crowdsourced computational resources in order to compute aggregation functions in a collective fashion. This democratization calls for a fully decentralized and privacy-preserving system design with which a local data management mechanism implemented in smart phones can guarantee highly accurate computations under highly dynamic data streams. Experimental evaluation with real-world Smart Grid data illustrates the performance trade-offs and shows how they can be managed in an automated and empirical way using decision trees.

The Internet of Things empowers citizens to interconnect their devices, such as smart phones, into large-scale participatory decentralized networks, which they can use to make real-time collective measurements as public good, for instance, crowd-sourcing the monitoring of traffic in a city. This approach is an alternative to big data analytics systems that are often expensive to access, privacy-intrusive and allow discriminatory and profiling actions over citizens' data. On the contrary, large-scale decentralized networks are complex to manage and collective measurements, i.e. computations of aggregation functions, need to encounter several dynamics such as continuously changing input data streams and highly varying temporal demand for access to the collective measurements. This paper proposes a highly reactive self-adaptation model to tackle the challenge of dynamic computational demand in large-scale decentralized in-network aggregation. The self-adaptation process makes nodes self-aware about other nodes that join and leave the network and therefore it makes them capable of self-orchestrating the communication to improve accuracy and minimize communication cost. The model is simple, yet agile. This is shown when applied in DIAS, the Dynamic Intelligent Aggregation Service without introducing architectural changes. Evaluation using data from a real-world smart grid pilot project as well as extreme demand profiles that scale up and down the demand 50% on average confirm the cost-effectiveness of in-network aggregation empowered by self-adaptation. The findings are confirmed both in simulation and a large-scale live deployment in a cluster infrastructure with 3000 independent Java virtual machines each running a DIAS node. Overall, the results encourage new promising pathways towards the broader adoption of self-adaptive participatory data analytics in large-scale decentralized networks.

The introduction of active devices in Smart Grids, such as smart transformers, powered by intelligent software and networking capabilities, brings paramount opportunities for online automated control and regulation. However, online mitigation of disruptive events, such as cascading failures, is challenging. Local intelligence by itself cannot tackle such complex collective phenomena with domino effects. Collective intelligence coordinating rapid mitigation actions is required. This paper introduces analytical results from which two optimization strategies for self-repairable Smart Grids are derived. These strategies build a coordination mechanism for smart transformers that runs in three healing modes and performs collective decision-making of the phase angles in the lines of a transmission system to improve reliability under disruptive events, i.e., line failures causing cascading failures. Experimental evaluation using self-repairability envelopes in different case networks, ac power flows, and varying number of smart transformers confirms that the higher the number of smart transformers participating in the coordination, the higher the reliability and the capability of a network to self-repair.

The opportunities to empirically study temporal networks nowadays are immense thanks to Internet of Things technologies along with ubiquitous and pervasive computing that allow a real-time fine-grained collection of social network data. This empowers data analytics and data scientists to reason about complex temporal phenomena, such as disease spread, residential energy consumption, political conflicts etc., using systematic methologies from complex networks and graph spectra analysis. However, a misuse of these methods may result in privacy-intrusive and discriminatory actions that may threaten citizens' autonomy and put their life under surveillance. This paper studies highly sparse temporal networks that model social interactions such as the physical proximity of participants in conferences. When citizens can self-determine the anonymized proximity data they wish to share via privacy-preserving platforms, temporal networks may turn out to be highly sparse and have low quality. This paper shows that even in this challenging scenario of privacy-by-design, significant information can be mined from temporal networks such as the correlation of events happening during a conference or stable groups interacting over time. The findings of this paper contribute to the introduction of privacy-preserving data analytics in temporal networks and their applications.

The increase in the deployment of smart meters has enabled collection of fine-grained energy consumption data at consumer premises. Analysis of this real-time energy consumption data bestows new opportunities for better demand-response (DR) programs. This paper offers a new perspective to study energy demand and helps in designing novel mechanisms for decentralized demand-side management. Specifically, a new concept of finding the demand states using energy consumption of consumers over time and feasible transitions therein is introduced. It is shown that the orchestration of temporal transitions between the demand states can meet broad range of smart grid objectives. An online demand regulation model is developed that captures the temporal dynamics of energy demand to identify target consumers for different DR programs. This methodology is empirically evaluated and validated using data from more than 4000 households, which were part of a real-world smart grid project. This paper is the first one to comprehensively analyze the temporal dynamics of demands.

The introduction of ubiquitous systems, wearable computing and 'Internet of Things' technologies in our digital society results in a large-scale data generation. Environmental, home, and mobile sensors are only a few examples of the significant capabilities to collect massive data in real-time from a plethora of heterogeneous social environments. These capabilities provide us with a unique opportunity to understand and tackle complex problems with new novel approaches based on reasoning about data. However, existing 'Big Data' approaches often turn this opportunity into a threat of citizens' privacy and open participation by surveilling, profiling and discriminating people via closed proprietary data mining services. This paper illustrates how to design and build an open participatory platform for privacy-preserving social mining: the Planetary Nervous System. Building such a complex platform in which data sharing and collection is self-determined by the user and is performed in a decentralized fashion within different ubiquitous environments is a challenge. This paper tackles this challenge by introducing a modular and compositional design approach based on a model of virtual sensors. Virtual sensors provide a holistic approach to build the core functionality of the Planetary Nervous System but also social mining applications that extend the core functionality. The holistic modeling approach with virtual sensors has the potential to simplify the engagement of citizens in different innovative crowd-sourcing activities and increase its adoption by building communities. Performance evaluations of virtual sensors in the Planetary Nervous System confirm the feasibility of the model to build real-time ubiquitous social mining services.

The robustness of smart grids is challenged by unpredictable power peaks or temporal demand oscillations that can cause blackouts and increase supply costs. Planning of demand can mitigate these effects and increase robustness. However, the impact on consumers in regards to the discomfort they experience as a result of improving robustness is usually neglected. This paper introduces a decentralized agent-based approach that quantifies and manages the tradeoff between robustness and discomfort under demand planning. Eight selection functions of plans are experimentally evaluated using real data from two operational smart grids. These functions can provide different quality of service levels for demand-side energy self-management that capture both robustness and discomfort criteria.

Distributed management of complex, distributed systems is the focus of this paper. Adaptation through local deliberation by software agents within a hierarchical virtual organization is the approach taken. Global stabilization of resource utilization is the goal. Electricity networks are used to illustrate the potential of two fitness functions on the basis of which local choices for resource utilization are made: minimizing oscillations is the first function considered, reversing oscillations the second. Results reveal considerable increase in the stabilization of resource utilization compared to a system that utilizes resources in a greedy manner.