Fairness in multi-agent systems (MAS) focuses on equitable reward distribution among agents in scenarios involving sensitive attributes such as race, gender, or socioeconomic status. This paper introduces fairness in Proximal Policy Optimization (PPO) with a penalty term derived from a fairness definition such as demographic parity, counterfactual fairness, or conditional statistical parity. The proposed method, which we call Fair-PPO, balances reward maximisation with fairness by integrating two penalty components: a retrospective component that minimises disparities in past outcomes and a prospective component that ensures fairness in future decision-making. We evaluate our approach in two games: the Allelopathic Harvest, a cooperative and competitive MAS focused on resource collection, where some agents possess a sensitive attribute, and HospitalSim, a hospital simulation, in which agents coordinate the operations of hospital patients with different mobility and priority needs. Experiments show that Fair-PPO achieves fairer policies than PPO across the fairness metrics and, through the retrospective and prospective penalty components, reveals a wide spectrum of strategies to improve fairness; at the same time, its performance pairs with that of state-of-the-art fair reinforcement-learning algorithms. Fairness comes at the cost of reduced efficiency, but does not compromise equality among the overall population (Gini index). These findings underscore the potential of Fair-PPO to address fairness challenges in MAS.

This paper introduces a novel method for estimating the self-interest level of Markov social dilemmas. We extend the concept of self-interest level from normal-form games to Markov games, providing a quantitative measure of the minimum reward exchange required to align individual and collective interests. We demonstrate our method on three environments from the Melting Pot suite, representing either common-pool resources or public goods. Our results illustrate how reward exchange can enable agents to transition from selfish to collective equilibria in a Markov social dilemma. This work contributes to multi-agent reinforcement learning by providing a practical tool for analysing complex, multistep social dilemmas. Our findings offer insights into how reward structures can promote or hinder cooperation, with potential applications in areas such as mechanism design.

Recent interest in Multi-Agent Systems of Large Language Models (MAS LLMs) has led to an increase in frameworks leveraging multiple LLMs to tackle complex tasks. However, much of this literature appropriates the terminology of MAS without engaging with its foundational principles. In this position paper, we highlight critical discrepancies between MAS theory and current MAS LLMs implementations, focusing on four key areas: the social aspect of agency, environment design, coordination and communication protocols, and measuring emergent behaviours. Our position is that many MAS LLMs lack multi-agent characteristics such as autonomy, social interaction, and structured environments, and often rely on oversimplified, LLM-centric architectures. The field may slow down and lose traction by revisiting problems the MAS literature has already addressed. Therefore, we systematically analyse this issue and outline associated research opportunities; we advocate for better integrating established MAS concepts and more precise terminology to avoid mischaracterisation and missed opportunities.

Fine-grained text classification requires models to distinguish between many fine-grained classes that are hard to tell apart. However, despite the increased risk of models relying on confounding features and predictions being especially difficult to interpret in this context, existing work on the interpretability of fine-grained text classification is severely limited. Therefore, we introduce our visual analysis system, SemLa, which incorporates novel visualization techniques that are tailored to this challenge. Our evaluation based on case studies and expert feedback shows that SemLa can be a powerful tool for identifying model weaknesses, making decisions about data annotation, and understanding the root cause of errors.

Some imitation learning methods combine behavioural cloning with self-supervision to infer actions from state pairs. However, most rely on a large number of expert trajectories to increase generalisation and human intervention to capture key aspects of the problem, such as domain constraints. In this paper, we propose Continuous Imitation Learning from Observation (CILO), a new method augmenting imitation learning with two important features: (i) exploration, allowing for more diverse state transitions, requiring less expert trajectories and resulting in fewer training iterations; and (ii) path signatures, allowing for automatic encoding of constraints, through the creation of non-parametric representations of agents and expert trajectories. We compared CILO with a baseline and two leading imitation learning methods in five environments. It had the best overall performance of all methods in all environments, outperforming the expert in two of them.

Abstract Social dilemmas present a significant challenge in multi-agent cooperation because individuals are incentivised to behave in ways that undermine socially optimal outcomes. Consequently, self-interested agents often avoid collective behaviour. In response, we formalise social dilemmas and introduce a novel metric, the general self-interest level, to quantify the disparity between individual and group rationality in such scenarios. This metric represents the maximum proportion of their individual rewards that agents can retain while ensuring that a social welfare optimum becomes a dominant strategy. Our approach diverges from traditional concepts of altruism, instead focusing on strategic reward redistribution. By transferring rewards among agents in a manner that aligns individual and group incentives, rational agents will maximise collective welfare while pursuing their own interests. We provide an algorithm to compute efficient transfer structures for an arbitrary number of agents, and introduce novel multi-player social dilemma games to illustrate the effectiveness of our method. This work provides both a descriptive tool for analysing social dilemmas and a prescriptive solution for resolving them via efficient reward transfer contracts. Applications include mechanism design, where we can assess the impact on collaborative behaviour of modifications to models of environments.

Abstract In natural language processing (NLP), text classification tasks are increasingly fine‐grained, as datasets are fragmented into a larger number of classes that are more difficult to differentiate from one another. As a consequence, the semantic structures of datasets have become more complex, and model decisions more difficult to explain. Existing tools, suited for coarse‐grained classification, falter under these additional challenges. In response to this gap, we worked closely with NLP domain experts in an iterative design‐and‐evaluation process to characterize and tackle the growing requirements in their workflow of developing fine‐grained text classification models. The result of this collaboration is the development of SemLa, a novel Visual Analytics system tailored for 1) dissecting complex semantic structures in a dataset when it is spatialized in model embedding space, and 2) visualizing fine‐grained nuances in the meaning of text samples to faithfully explain model reasoning. This paper details the iterative design study and the resulting innovations featured in SemLa. The final design allows contrastive analysis at different levels by unearthing lexical and conceptual patterns including biases and artifacts in data. Expert feedback on our final design and case studies confirm that SemLa is a useful tool for supporting model validation and debugging as well as data annotation.

Designing AI systems with the capacity to explain their behaviour is paramount to enable human oversight, facilitate trust, promote acceptance of technology and, ultimately, empower users and improve their experience. There are, however, several challenges to explainable AI, one of which is the generation and selection of explanations from the causal history of a given event. Causal attribution, among other cognitive processes, has been found to be influenced by socio-cultural factors, which suggests that there could be systematic differences in preferences for AI explanations between communities of users according to their cognitive style and socio-cultural characteristics. In this paper, we investigate the relationship between preferences in the explanations provided by belief-desire-intention AI agents, cognitive style (holistic vs analytical), and socio-cultural factors, such as gender, education, social class, and political and religious beliefs. We found a relationship between explanation preference, cognitive style and various socio-cultural characteristics. Holistic cognitive style is associated with preference for goal explanations while analytic cognitive style is associated with preference for belief explanations. Socio-cultural variables that affect explanation preference are gender, religious beliefs, educational attainment, some fields of education, and political party affiliation.

Abstract This paper presents a novel model, called TomAbd, that endows autonomous agents with Theory of Mind capabilities. TomAbd agents are able to simulate the perspective of the world that their peers have and reason from their perspective. Furthermore, TomAbd agents can reason from the perspective of others down to an arbitrary level of recursion, using Theory of Mind of $$n^{\text {th}}$$ n th order. By combining the previous capability with abductive reasoning, TomAbd agents can infer the beliefs that others were relying upon to select their actions, hence putting them in a more informed position when it comes to their own decision-making. We have tested the TomAbd model in the challenging domain of Hanabi, a game characterised by cooperation and imperfect information. Our results show that the abilities granted by the TomAbd model boost the performance of the team along a variety of metrics, including final score, efficiency of communication, and uncertainty reduction.

We consider a one-to-many persuasion setting, where a persuader presents arguments to a multi-party audience, aiming to convince them of some particular goal argument. The individual audience members each have differing personal knowledge, which they use, together with the arguments presented by the persuader, to determine whether they are convinced of the goal. The persuader must, therefore, carefully consider its strategy, i.e., which arguments to assert, in order to maximise the number of convinced audience members. Here, we use evolutionary search to find (near-)optimal strategies for the persuader. We implement our approach using search-based model engineering, which provides a natural and efficient encoding for such problems. We investigate the performance of our approach on a range of settings, considering different structures and sizes of argumentation frameworks (representing the underlying knowledge available to the persuader and audience members), and varying the size of audience and of the audience members' personal knowledge bases. We show that we can find effective strategies for problems with more than 200 arguments and more than 100 audience members. Further, we show that the approach supports multiple persuader objectives, finding persuader strategies that aim to minimise arguments to assert while still maximising the number of convinced audience members.

Abstract Quality of Service (QoS) properties play an important role in distinguishing between functionally equivalent services and accommodating the different expectations of users. However, the subjective nature of some properties and the dynamic and unreliable nature of service environments may result in cases where the quality values advertised by the service provider are either missing or untrustworthy. To tackle this, a number of QoS estimation approaches have been proposed, using the observation history available on a service to predict its performance. Although the context underlying such previous observations (and corresponding to both user and service related factors) could provide an important source of information for the QoS estimation process, it has only been used to a limited extent by existing approaches. In response, we propose a context‐aware quality learning model, realized via a learning‐enabled service agent, exploiting the contextual characteristics of the domain to provide more personalized, accurate, and relevant quality estimations for the situation at hand. The experiments conducted demonstrate the effectiveness of the proposed approach, showing promising results (in terms of prediction accuracy) in different types of changing service environments.

Norms are a valuable means of establishing coherent cooperative behaviour in decentralised systems in which there is no central authority. Axelrod’s seminal model of norm establishment in populations of self-interested individuals provides some insight into the mechanisms needed to support this through the use of metanorms, but considers only limited scenarios and domains. While further developments of Axelrod’s model have addressed some of the limitations, there is still only limited consideration of such metanorm models with more realistic topological configurations. In response, this paper tries to address such limitation by considering its application to different topological structures. Our results suggest that norm establishment is achievable in lattices and small worlds, while such establishment is not achievable in scale-free networks, due to the problematic effects of hubs. The paper offers a solution, first by adjusting the model to more appropriately reflect the characteristics of the problem, and second by offering a new dynamic policy adaptation approach to learning the right behaviour. Experimental results demonstrate that this dynamic policy adaptation overcomes the difficulties posed by the asymmetric distribution of links in scale-free networks, leading to an absence of norm violation, and instead to norm emergence.

Argument-based persuasion dialogues provide an effective mechanism for agents to communicate their beliefs, and their reasons for those beliefs, in order to convince another agent of some topic argument. In such dialogues, the persuader has strategic considerations, and must decide which of its known arguments should be asserted, and the order in which they should be asserted. Recent works consider mechanisms for determining an optimal strategy for persuading the responder. However, computing such strategies is expensive, swiftly becoming impractical as the number of arguments increases. In response, we present a strategy that uses heuristic information of the domain arguments and can be computed with high numbers of arguments. Our results show that not only is the heuristic strategy fast to compute, it also performs significantly better than a random strategy.

Preferences have been widely studied in several areas including computer science, as they play an important role in many computational tasks, such as decision making support, providing recommendations and personalizing applications. Although many approaches consider particular preference representation models to be used as input for algorithms to address these tasks, there is a need for identifying a model that provides adequate constructions for users to express their preferences. In this paper, we propose a preference meta-model that provides various preference constructs, which include end-user expressions. We start by describing an exploratory study of how people express their preferences in natural language, which provides the basis for the meta-model. After describing the meta-model, we evaluate it with two user studies in different domains. The results of this evaluation indicate that the preference statements that can be expressed with the meta-model are adequate for allowing users to indicate their preferences to a computational system.

Recently, research has focused on processing tasks that require continuous execution to produce data in a real-time manner. Such tasks often also need to be executed for long periods of time such as years, requiring large amounts of resources (e.g. CPUs) that can be found in a Grid. However, a Grid may be unwilling or unable to allocate resources for continuous usage far in advance, because of high fluctuations in resource availability and/or resource demand. Therefore, a client must relax its requirements in terms of long-term execution, and negotiate a shorter period of execution time; when this period ends, the client must negotiate again to continue task's execution. We propose a negotiation strategy, ConTask, which helps to increase the periods of execution time, and reduce the length of interruptions between them.

In many cases, intermediaries play a major role in linking between service providers and their target users. Yet, attracting intermediaries at a marketplace to promote a service to their existing customers can be very challenging, since they are usually very busy and would incur additional cost as a result of such promotion. In response, this paper presents an information-based incentivisation framework, which combines financial rewards with other motivating information, in order to incentivise intermediaries at a marketplace to undertake service promotion. Specifically, the intermediaries are associated with a group of incentivising agents, capable of learning the individual motivational needs of these intermediaries, and accordingly target them with the most effective incentives. The incentivising agents collaborate with each other to gather motivational information, by sharing their observations on intermediaries. The proposed incentivisation approach is evaluated through a corresponding agent-based simulation, and the experimental results obtained demonstrate its effectiveness.

Explanations play an essential role in decision support and recommender systems as they are directly associated with the acceptance of those systems and the choices they make. Although approaches have been proposed to explain automated decisions based on multi-attribute decision models, there is a lack of evidence that they produce the explanations users need. In response, in this paper we propose an explanation generation technique, which follows user-derived explanation patterns. It receives as input a multi-attribute decision model, which is used together with user-centric principles to make a decision to which an explanation is generated. The technique includes algorithms that select relevant attributes and produce an explanation that justifies an automated choice. An evaluation with a user study demonstrates the effectiveness of our approach.

Today there is an increase in the number of cooperative initiatives in different domains to make tools and data available to global communities free or charge. Such cooperative systems are open, heterogeneous, dynamic, and lack a formal payment system. Incentivising cooperation in these scenarios is essential to maintain their effectiveness. Therefore, there is a recognised need to move away from an ad hoc approach to one in which cooperation is supported and encouraged. The agent-oriented paradigm has been advocated as a natural way to design and implement systems that are distributed and heterogeneous. However, developing an agent-oriented system for today's cooperative systems is challenging. It requires a means not only to provide non-monetary incentives for service providers, but also to consider the level of quality of cooperations, in terms of the quality of provided and received services. In this context, the key contribution of this paper is a framework for non-monetary interactions among self-interested agents, in which the motivation to cooperate and the bases for analysing cooperations come from Piaget's theory of exchange values. Our framework includes a computational model of these values, which defines how exchange values are accumulated and spent by interacting agents. We illustrate how our framework can be used by agents to analyze cooperations and to take decisions about them, and provide an empirical evaluation.

Accounting for quality correlations among web services when performing service composition is essential to obtain more accurate quality estimations of service combinations, thus providing users with better composite solutions. Yet, most current composition approaches fail to address such correlations by assuming independence between services regarding their quality values. In response, this paper presents a correlation-aware composition approach, where quality dependencies among services are modelled and considered during composite service selection. Moreover, to improve selection efficiency, correlation-aware search space reduction techniques are introduced, which prune out uninteresting service compositions prior to selection. The effectiveness of the approach, in terms of time and optimality, is demonstrated via experimental results.

Just as conventional institutions are organisational structures for coordinating the activities of multiple interacting individuals, electronic institutions provide a computational analogue for coordinating the activities of multiple interacting software agents. In this paper, we argue that open multi-agent systems can be effectively designed and implemented as electronic institutions, for which we provide a comprehensive computational model. More specifically, the paper provides an operational semantics for electronic institutions, specifying the essential data structures, the state representation and the key operations necessary to implement them. We specify the agent workflow structure that is the core component of such electronic institutions and particular instantiations of knowledge representation languages that support the institutional model. In so doing, we provide the first formal account of the electronic institution concept in a rigorous and unambiguous way.

The specification of acceptable behaviour can be achieved via the use of obligations, permissions and prohibitions, collectively known as norms, which identify the states of affairs that should, may, or should not hold. Norms provide the ability to constrain behaviour while preserving individual agent autonomy. While much work has focused on the semantics of norms, the design of normative systems, and in particular understanding the impact of norms on a system, has received little attention. Since norms often interact with each other (for example, a permission may temporarily derogate an obligation, or a prohibition and obligation may conflict), understanding the effects of norms and their interactions becomes increasingly difficult as the number of norms increases. Yet this understanding can be critical in facilitating the design and development of effective or efficient systems. In response, this paper addresses the problem of norm explanation for Naïve users by providing of a graphical norm representation that can explicate why a norm is applicable, violated or complied with, and identify the interactions between permissions and other types of norms. We adopt a conceptual graph based semantics to provide this graphical representation while maintaining a formal semantics.

Dynamic composition of services provides the ability to build complex distributed applications at run time by combining existing services, thus coping with a large variety of complex requirements that cannot be met by individual services alone. However, with the increasing amount of available services that differ in granularity (amount of functionality provided) and qualities, selecting the best combination of services becomes very complex. In response, this paper addresses the challenges of service selection, and makes a twofold contribution. First, a rich representation of compositional planning knowledge is provided, allowing the expression of multiple decompositions of tasks at arbitrary levels of granularity. Second, two distinct search space reduction techniques are introduced, the application of which, prior to performing service selection, results in significant improvement in selection performance in terms of execution time, which is demonstrated via experimental results.

Abstract argument frameworks have been used for various applications within multi-agent systems, including reasoning and negotiation. Different argument frameworks make use of different inter-argument relations and semantics to identify some subset of arguments as coherent, yet there is no easy way to map between these frameworks; most commonly, this is done manually according to human intuition. In response, in this paper, we show how a set of arguments described using Dung's or Nielsen's argument frameworks can be mapped from and to an argument framework that includes both attack and support relations. This mapping preserves the framework's semantics in the sense that an argument deemed coherent in one framework is coherent in the other under a related semantics. Interestingly, this translation is not unique, with one set of arguments in the support based framework mapping to multiple argument sets within the attack only framework. Additionally, we show how EAF can be mapped into a subset of the argument interchange format (AIF). By using this mapping, any other argument framework using this subset of AIF can be translated into a DAF while preserving its semantics.

The adoption of agent technologies and multi-agent systems constitutes an emerging area in bioinformatics. In this article, we report on the activity of the Working Group on Agents in Bioinformatics (BIOAGENTS) founded during the first AgentLink III Technical Forum meeting on the 2nd of July, 2004, in Rome. The meeting provided an opportunity for seeding collaborations between the agent and bioinformatics communities to develop a different (agent-based) approach of computational frameworks both for data analysis and management in bioinformatics and for systems modelling and simulation in computational and systems biology. The collaborations gave rise to applications and integrated tools that we summarize and discuss in context of the state of the art in this area. We investigate on future challenges and argue that the field should still be explored from many perspectives ranging from bio-conceptual languages for agent-based simulation, to the definition of bio-ontology-based declarative languages to be used by information agents, and to the adoption of agents for computational grids.

Summary form only given. Agents are computer systems capable of flexible autonomous action in dynamic and open environments. They are a natural extension of component-based approaches, but can be considered at several different levels: as a source of specific technologies; as a metaphor for abstraction and design, and as a means of simulation. While the CSCW community has focused on human-centred approaches to coordination and interaction, agent-based computing has addressed the infrastructure for support of machine interaction. Yet as the richness of computing increases, and as an increasing number of social metaphors are being used as inspiration for enabling and regulating machine-oriented interactions, the divisions between the two are decreasing. Motivated by this convergence, we review the current state of the art in agent-based computing, consider some applications in relation to collaboration across different domains, assess some of the challenges that face the different communities, and propose some research activities that can contribute to both.

Notification services (NSs) are middleware components providing asynchronous message delivery between publishers and consumers. Multiple interconnected NSs form a distributed NS, with each NS routing notifications between publishers and consumers at different locations, enabling consumers to share subscriptions, reducing the number of messages sent. Consumers can specify quality of service (QoS) levels when subscribing to a NS, using negotiation to find QoS levels acceptable to both parties. However, if consumers specify sufficiently different QoS levels, notifications cannot be shared and new subscriptions must be made. Chained negotiation can be used to negotiate QoS levels through intermediate NSs, enabling the reuse of existing subscriptions for additional consumers. In this paper, we present a chained negotiation engine, evaluating its performance and behaviour, showing that it enables negotiation over QoS while still sharing notifications, and that it provides better results for a consumer by negotiation directly with the publisher.

This paper highlights intrusiveness as a key issue in the field of pervasive computing environments and presents a multiagent approach to tackling it. Specifically, we discuss how interruptions can impact on individual and group tasks and how they can be managed by taking into account user and group preferences through negotiation between software agents. The system we develop is implemented on the Jabber platform and is deployed in the context of a meeting room scenario.

My Grid is an e-Science Grid project that aims to help biologists and bioinformaticians to perform workflow-based in silico experiments, and help them to automate the management of such workflows through personalisation, notification of change and publication of experiments. In this paper, we describe the architecture of my Grid and how it will be used by the scientist. We then show how my Grid can benefit from agents technologies. We have identified three key uses of agent technologies in my Grid: user agents, able to customize and personalise data, agent communication languages offering a generic and portable communication medium, and negotiation allowing multiple distributed entities to reach service level agreements.

In large-scale service monitoring, automated dynamic (re)distribution of running monitoring applications is likely to push the limits of scalability. In the aggregation nets approach, we associate each partition of a distributed application with an autonomous agent capable of relocating the partition fully or partially, or modifying it to accommodate the dynamics of its local environment. Coordinated agent behaviour is to result in maintaining acceptable performance for the whole application. Aggregation nets is a typical Grid application that relies on the availability of distributed computing power and a network sensing infrastructure that provides information for agent decision-making. In order to evaluate our approach, we emulate aggregation nets on top the physical network simulator Berkeley ns. We describe our model of aggregation nets, and its implementation.

The use of agents of many different kinds in a variety of fields of computer science and artificial intelligence is increasing rapidly and is due, in part, to their wide applicability. The richness of the agent metaphor that leads to many different uses of the term is, however, both a strength and a weakness: its strength lies in the fact that it can be applied in very many different ways in many situations for different purposes; the weakness is that the term agent is now used so frequently that there is no commonly accepted notion of what it is that constitutes an agent. This paper addresses this issue by applying formal methods to provide a defining framework for agent systems. The Z specification language is used to provide an accessible and unified formal account of agent systems, allowing us to escape from the terminological chaos that surrounds agents. In particular, the framework precisely and unambiguously provides meanings for common concepts and terms, enables alternative models of particular classes of system to be described within it, and provides a foundation for subsequent development of increasingly more refined concepts.

As student numbers on computer science courses continue to increase, the corresponding demands placed on teaching staff in terms of assessment grow ever stronger. In particular, the submission and assessment of practical work on large programming courses can present very significant problems. In response to this, we have developed a networked suite of software utilities that allow on-line submission, testing and marking of coursework. It has been developed and used over the course of five years, and has evolved into a mature tool that has greatly reduced the administrative time spent managing the processes of submission and assessment. In this paper, we describe the software and its implementation, and discuss the issues involved in its construction. Copyright © 1999 John Wiley & Sons, Ltd.

In response to the demands of increasing student numbers, the BOSS system for submission and assessment has been constructed to enable student programming assignments to be submitted and tested on-line. More recent developments of this system have been concerned with the addition of electronic marking facilities that incorporate both automated marking, resulting from the automated testing, and manual marking in a secure environment. This paper briefly reviews the system and describes in detail the electronic marksheets, their functionality, and their user-interface.

Perhaps the most successful agent architectures, and certainly the best known, are those based on the Belief-Desire-Intention (BDI) framework. Despite the wealth of research that has accumulated on both formal and practical aspects of this framework, however, there remains a gap between the formal models and the implemented systems. In this paper, we build on earlier work by Rao aimed at narrowing this gap, by developing a strongly-typed, formal, yet computational model of the BDI-based AgentSpeak(L) language. AgentSpeak(L) is a programming language, based on the Procedural Reasoning System (PRS) and the Distributed Multi-Agent Reasoning System (dMARS), which determines the behaviour of the agents it implements. In developing the model, we add to Rao's work, identify some omissions, and progress beyond the description of a particular language by giving a formal specification of a general BDI architecture that can be used as the basis for providing further formal specifications of more sophisticated systems.

Previous work has addressed the development of a framework to categorise and understand agent-based systems. It described and formalised an agent-hierarchy that included objects, agents and autonomous agents, each with different levels of functionality, and provided a precise vocabulary with which to discuss agent systems. This paper reviews a large variety of further work that has built on that foundation in several ways. First, the framework itself has been refined to detail important aspects of agent functionality such as goal generation and adoption. Second, the structures and relationships between agents have been specified and analysed allowing a more complete understanding of the dynamics of agent systems. Third, existing systems and theories have been formalised within the framework so that they may be evaluated and compared in a coherent and consistent way. Finally, some steps have been taken in attempting to construct a methodology for the development of agent-based systems. Though this work spans a large range of concerns, it is based on a single set of basic concepts providing fundamental structure.