Human-AI coevolution, defined as a process in which humans and AI algorithms continuously influence each other, increasingly characterises our society, but is understudied in artificial intelligence and complexity science literature. Recommender systems and assistants play a prominent role in human-AI coevolution, as they permeate many facets of daily life and influence human choices through online platforms. The interaction between users and AI results in a potentially endless feedback loop, wherein users' choices generate data to train AI models, which, in turn, shape subsequent user preferences. This human-AI feedback loop has peculiar characteristics compared to traditional human-machine interaction and gives rise to complex and often “unintended” systemic outcomes. This paper introduces human-AI coevolution as the cornerstone for a new field of study at the intersection between AI and complexity science focused on the theoretical, empirical, and mathematical investigation of the human-AI feedback loop. In doing so, we: (i) outline the pros and cons of existing methodologies and highlight shortcomings and potential ways for capturing feedback loop mechanisms; (ii) propose a reflection at the intersection between complexity science, AI and society; (iii) provide real-world examples for different human-AI ecosystems; and (iv) illustrate challenges to the creation of such a field of study, conceptualising them at increasing levels of abstraction, i.e., scientific, legal and socio-political.

Human-AI coevolution, defined as a process in which humans and AI algorithms continuously influence each other, increasingly characterises our society, but is understudied in artificial intelligence and complexity science literature. Recommender systems and assistants play a prominent role in human-AI coevolution, as they permeate many facets of daily life and influence human choices through online platforms. The interaction between users and AI results in a potentially endless feedback loop, wherein users' choices generate data to train AI models, which, in turn, shape subsequent user preferences. This human-AI feedback loop has peculiar characteristics compared to traditional human-machine interaction and gives rise to complex and often “unintended” systemic outcomes. This paper introduces human-AI coevolution as the cornerstone for a new field of study at the intersection between AI and complexity science focused on the theoretical, empirical, and mathematical investigation of the human-AI feedback loop. In doing so, we: (i) outline the pros and cons of existing methodologies and highlight shortcomings and potential ways for capturing feedback loop mechanisms; (ii) propose a reflection at the intersection between complexity science, AI and society; (iii) provide real-world examples for different human-AI ecosystems; and (iv) illustrate challenges to the creation of such a field of study, conceptualising them at increasing levels of abstraction, i.e., scientific, legal and socio-political.

We consider a type of information-seeking dialogue between a seeker agent and a respondent agent, where the seeker estimates the respondent to not be willing to share a particular set of sought-after information. Hence, the seeker postpones (hides) its goal topic, related to the respondent's sensitive information, until the respondent is perceived as willing to talk about it. In the intermediate process, the seeker opens other topics to steer the dialogue tactfully towards the goal. Such dialogue strategies, which we refer to as goal-hiding strategies, are common in diverse contexts such as criminal interrogations and medical assessments, involving sensitive topics. Conversely, in malicious online interactions like social media extortion, similar strategies might aim to manipulate individuals into revealing information or agreeing to unfavorable terms. This paper proposes a formal dialogue framework for understanding goal-hiding strategies. The dialogue framework uses Quantitative Bipolar Argumentation Frameworks (QBAFs) to assign willingness scores to topics. An initial willingness for each topic is modified by considering how topics promote (support) or demote (attack) other topics. We introduce a method to identify relations among topics by considering a respondent's shared information. Finally, we introduce a gradual semantics to estimate changes in willingness as new topics are opened. Our formal analysis and empirical evaluation show the system's compliance with privacy-preserving safety properties. A formal understanding of goal-hiding strategies opens up a range of practical applications; For instance, a seeker agent may plan with goal-hiding to enhance privacy in human-agent interactions. Similarly, an observer agent (third-party) may be designed to enhance social media security by detecting goal-hiding strategies employed by users' interlocutors.

An important issue in deploying an autonomous system is how to enable human users and stakeholders to develop an appropriate level of trust in the system. It has been argued that a crucial mechanism to enable appropriate trust is the ability of a system to explain its behaviour. Obviously, such explanations need to be comprehensible to humans. Due to the perceived similarity in functioning between humans and autonomous systems, we argue that it makes sense to build on the results of extensive research in social sciences that explores how humans explain their behaviour. Using similar concepts for explanation is argued to help with comprehensibility, since the concepts are familiar. Following work in the social sciences, we propose the use of a folk-psychological model that utilises beliefs, desires, and ``valuings''. We propose a formal framework for constructing explanations of the behaviour of an autonomous system, present an (implemented) algorithm for giving explanations, and present evaluation results.

The increase in computational and communication power of pervasive devices is also enabling to embed increasing intelligence in devices, there included the capability to act in autonomy, and possibly interacting with each other, in order to achieve specific goals, thus leaving humans out of the decision loop. In this context, the goal of the panel is thus reasoning about the possible implications (technical, ethical, and lega) of assigning great and often critical decision power to pervasive autonomous systems.

This short paper reports the summary of the panel held in the context of the PerCom 2022 conference.

Virginia Dignum has given a Keynote Talk at The ACM Web Conference 2022 on Thursday 28th April 2022. This paper provides a summary of the topics she addressed during her talk.

Evidence-driven agent-based modeling plays a useful part in understanding social phenomena. By integrating social-cognitive theories in our agent models, we bear evidence from social and psychological studies on our models for human decision-making. Social practice theory (SPT) provides a socio-cognitive theory that emphasizes three empirically and theoretically grounded aspects of behavior: habituality, sociality, and interconnectivity. Previous work has emphasized the importance of SPT for agents, has made abstract models of SPT, or used SPT to study energy systems. This article provides a set of requirements for integrating SPT in agent models and an evaluation of 11 current agent models with respect to these requirements. We find that current agent models do not fully capture habituality, sociality, or interconnectivity, nor is there a model that aims to integrate all three aspects. For example, current models do not support context-dependent habits, use a comprehensive set of collective concepts, and support hierarchies of activities. Our evaluation allows researchers to pick one of the current agent models depending on their needs regarding habituality, sociality, and interconnectivity. Furthermore, this article shows the usefulness of an agent model that integrates SPT and provides requirements that help modelers to achieve this model.

Artificial Intelligence (AI) applications are being used to predict and assess behaviour in multiple domains which directly affect human well-being. However, if AI is to improve people’s lives, then people must be able to trust it, by being able to understand what the system is doing and why. Although transparency is often seen as the requirement in this case, realistically it might not always be possible, whereas the need to ensure that the system operates within set moral bounds remains. In this paper, we present an approach to evaluate the moral bounds of an AI system based on the monitoring of its inputs and outputs. We place a ‘Glass-Box’ around the system by mapping moral values into explicit verifiable norms that constrain inputs and outputs, in such a way that if these remain within the box we can guarantee that the system adheres to the value. The focus on inputs and outputs allows for the verification and comparison of vastly different intelligent systems; from deep neural networks to agent-based systems. The explicit transformation of abstract moral values into concrete norms brings great benefits in terms of explainability; stakeholders know exactly how the system is interpreting and employing relevant abstract moral human values and calibrate their trust accordingly. Moreover, by operating at a higher level we can check the compliance of the system with different interpretations of the same value.

Ethics by Design concerns the methods, algorithms and tools needed to endow autonomous agents with the capability to reason about the ethical aspects of their decisions, and the methods, tools and formalisms to guarantee that an agent's behavior remains within given moral bounds. In this context some questions arise: How and to what extent can agents understand the social reality in which they operate, and the other intelligences (AI, animals and humans) with which they co-exist? What are the ethical concerns in the emerging new forms of society, and how do we ensure the human dimension is upheld in interactions and decisions by autonomous agents?. But overall, the central question is: "Can we, and should we, build ethically-aware agents?" This paper presents initial conclusions from the thematic day of the same name held at PRIMA2017, on October 2017.

Understanding the social contexts in which actions and interactions take place is of utmost importance for planning one’s goals and activities. People use social practices as means to make sense of their environment, assessing how that context relates to past, common experiences, culture and capabilities. Social practices can therefore simplify deliberation and planning in complex contexts. In the context of patient-centered planning, hospitals seek means to ensure that patients and their families are at the center of decisions and planning of the healthcare processes. This requires on one hand that patients are aware of the practices being in place at the hospital and on the other hand that hospitals have the means to evaluate and adapt current practices to the needs of the patients. In this paper we apply a framework for formalizing social practices of an organization to an emergency department that carries out patient-centered planning. We indicate how such a formalization can be used to answer operational queries about the expected outcome of operational actions.

Current developments in governance and policy setting are challenging traditional top-down models of decision-making. Whereas, on the one hand, citizens are increasingly demanding and expected to participate directly on governance questions, social networking platforms are, on the other hand, increasingly providing podia for the spread of unfounded, extremist and/or harmful ideas. Participatory deliberation is a form of democratic policy making in which deliberation is central to decision-making using both consensus decision-making and majority rule. However, by definition, it will lead to socially accepted results rather than ensuring the moral acceptability of the result. In fact, participation per se offers no guidance regarding the ethics of the decisions taken, nor does it provide means to evaluate alternatives in terms of their moral “quality.” This article proposes an open participatory model, Massive Open Online Deliberation (MOOD), that can be used to solve some of the current policy authority deficits. MOOD taps on individual understanding and opinions by harnessing open, participatory, crowd-sourced, and wiki-like methodologies, effectively producing collective judgements regarding complex political and social issues in real time. MOOD offers the opportunity for people to develop and draft collective judgements on complex issues and crises in real time. MOOD is based on the concept of Ethics by Participation , a formalized and guided process of moral deliberation that extends deliberative democracy platforms to identify morally acceptable outcomes and enhance critical thinking and reflection among participants.

Autonomous agents are increasingly required to be able to make moral decisions. In these situations, the agent should be able to reason about the ethical bases of the decision and explain its decision in terms of the moral values involved. This is of special importance when the agent is interacting with a user and should understand the value priorities of the user in order to provide adequate support. This paper presents a model of agent behavior that takes into account user preferences and moral values.

As intelligent systems are increasingly making decisions that directly affect society, perhaps the most important upcoming research direction in AI is to rethink the ethical implications of their actions. Means are needed to integrate moral, societal and legal values with technological developments in AI, both during the design process as well as part of the deliberation algorithms employed by these systems. In this paper, we describe leading ethics theories and propose alternative ways to ensure ethical behavior by artificial systems. Given that ethics are dependent on the socio-cultural context and are often only implicit in deliberation processes, methodologies are needed to elicit the values held by designers and stakeholders, and to make these explicit leading to better understanding and trust on artificial autonomous systems.

Institutions regulate societies. Comprising Searle's constitutive counts-as rules, "A counts-as B in context C", an institution ascribes from brute and institutional facts (As), a social reality comprising institutional facts (Bs) conditional on the social reality (contexts Cs). When brute facts change an institution evolves from one social reality to the next. Rule changes are also regulated by rule-modifying counts-as rules ascribing rule change in the past/present/future (e.g. a majority rule change vote counts-as a rule change). Determining rule change legality is difficult, since changing counts-as rules both alters and is conditional on the social reality, and in some cases hypothetical rule-change effects (e.g. not retroactively criminalising people). However, without a rigorous account of rule change ascriptions, AI agents cannot support humans in understanding the laws imposed on them. Moreover, advances in automated governance design for socio-technical systems, are limited by agents' ability to understand how and when to enact institutional changes. Consequently, we answer "when do rule changes count-as legal rule changes?" in a temporal setting with a novel formal framework.

Normative multi-agent systems offer the ability to integrate social and individual factors to provide increased levels of fidelity with respect to modelling social phenomena, such as cooperation, coordination, group decision making, and organization, in both human and artificial agent systems. An important open research issue refers to group norms, that is, norms that govern groups of agents. Depending on the interpretation, group norms may be intended to affect the group as a whole, each member of a group, or some members of the group. Moreover, upholding group norms may require coordination among the members of the group. We have identified three sets of agents affected by group norms, namely, (i) the addressees of the norm, (ii) those that will act on it, and (iii) those that are responsible for ensuring norm compliance. We present a formalism to represent these, connecting it to a minimalist agent organisation model. We use our formalism to develop a reasoning mechanism that enables agents to identify their position with respect to a group norm to further support agent autonomy and coordination when deciding on possible courses of action.

Security policies in organisations typically take the form of obligations for the employees. However, it is often unclear what the purpose of such obligations is, and how these can be integrated in the operational processes of the organisation. This can result in policies that may be either too strong or too weak, leading to unnecessary productivity loss, or the possibility of becoming victim to attacks that exploit the weaknesses, respectively. In this paper, we propose a framework in which the security obligations of employees are linked directly to prohibitions that prevent external agents (attackers) from reaching their goals. We use logic-based and graph-based approaches to formalise and reason about such policies, and show how the framework can be used to verify correctness of the associated refinements. Finally, we extend the graph-based model with quantitative policies and associated quantitative analysis, based on the time an adversary needs for an attack. The framework can assist organisations in aligning security policies with their threat model.

Agent organizations can be seen as a set of entities regulated by mechanisms of social order and created by more or less autonomous actors to achieve common goals. Just like agents, organizations should also be able to adapt themselves to changing environments. In order to develop a theory of how this reorganization should be performed we need a formal framework in which organizations, organizational performance and the reorganization itself can be described. In this article, we present a formal description of reorganization actions in Logic forAgent Organization. We show how this formalization can support the preservation of some nice properties of organizations while it can also be used to reason about which reorganization is needed to achieve some basic organizational properties.

Security policies in organisations typically take the form of obligations for the employees. However, it is often unclear what the purpose of such obligations is, and how these can be integrated in the operational processes of the organisation. This can result in policies that may be either too strong or too weak, leading to unnecessary productivity loss, or the possibility of becoming victim to attacks that exploit the weaknesses, respectively. In this paper, we propose a framework in which the security obligations of employees are linked directly to prohibitions that prevent external agents (attackers) from reaching their goals. We use graph-based and logic-based approaches to formalise and reason about such policies, and show how the framework can be used to verify correctness of the associated refinements. The framework can assist organisations in aligning security policies with their threat model.

Teamwork has become a widely accepted metaphor for describing the nature of multi-robot and multi-agent cooperation. By virtue of teamwork models, team members attempt to manage general responsibilities and commitments to each other in a coherent fashion that both enhances performance and facilitates recovery when unanticipated problems arise. Whereas early research on teamwork focused mainly on interaction within groups of autonomous agents or robots, there is a growing interest in leveraging human participation effectively. Unlike autonomous systems designed primarily to take humans out of the loop, many important applications require people, agents, and robots to work together in close and relatively continuous interaction. For software agents and robots to participate in teamwork alongside people in carrying out complex real-world tasks, they must have some of the capabilities that enable natural and effective teamwork among groups of people. Just as important, developers of such systems need tools and methodologies to assure that such systems will work together reliably and safely, even when they have been designed independently.