We address the problem of learning autonomous safe agent behaviour with unknown dynamics and reward functions, where traditional Reinforcement Learning is impossible. We present DROPJ, a human-centred algorithm that maximises safety during both training and deployment. We first learn a world model (a learned simulation) from a set of past real-world trajectories. A user then plays the game in the simulation to draw several informative virtual trajectories. From these, we extract pairs of trajectory segments and present them to a user to elicit their preference over these segments and the reason (justification) for that preference. With this feedback, a reward model is trained, which is used to deploy the agent with Model Predictive Control. We find that generating trajectories from user trials significantly reduces the computational cost of training, and significantly improves performance during deployment. In that context, we show that the use of preferences rather than other types of feedback substantially improves the performance. We further demonstrate that the use of justifications associated with safety requirements results in safer policies.

Emergent communication enables agents to develop bespoke languages that improve communication efficiency. Despite the known importance of temporal structure in natural language, there is no existing evidence of temporal references in emergent communication. This paper addresses this gap, by exploring how agents communicate about temporal relationships. We analyse three potential factors for the emergence of temporal references: environmental, external, and architectural. Our experiments demonstrate that altering the loss function is insufficient for temporal references to emerge; rather, architectural changes are necessary. A minimal change in agent architecture, using a different batching method, allows the emergence of temporal references. This modified design is compared with the standard architecture in a temporal referential games environment, which emphasises temporal relationships. The analysis shows that over 95% of the agents with the modified batching method develop temporal references, without changes to their loss function. We consider temporal referencing necessary for future improvements to the agents’ communication efficiency, enabling future agents to use a closer to optimal coding as compared to purely compositional languages. These insights provide the basis for incorporation of temporal references into other emergent communication settings, and investigation of other aspects of language.

As mobile applications grow in complexity, there is an increasing need to perform computationally intensive tasks. However, User Devices (UDs), such as tablets and smartphones, have limited capacity to carry out the required computations. Task offloading in Mobile Edge Computing (MEC) is a strategy that meets this demand by distributing tasks between UDs and servers. Deep Reinforcement Learning (DRL) is a promising solution for this strategy because it can adapt to dynamic changes and minimize online computational complexity. However, the combination of continuous-valued soft constraints and discrete-valued hard constraints on UDs and MEC servers poses significant challenges for designing efficient DRL algorithms. Existing DRL-based task-offloading algorithms focus on the constraints of the UDs, assuming the availability of enough resources on the server. Moreover, existing Multiagent DRL (MADRL)-based task-offloading algorithms are homogeneous agents and consider homogeneous constraints as a penalty in their reward function. We propose a novel Client–Master MADRL (CMMADRL) algorithm for task offloading in MEC that uses client agents at the UDs to decide on their resource requirements and a master agent at the server to make a combinatorial action selection based on the decision of the UDs. CMMADRL is shown to achieve up to 59% improvement in performance over existing benchmark and heuristic algorithms.

We introduce a novel method for assessing agent teamwork based on their spatial coordination. Our approach models the influence of spatial proximity on team formation and sustained spatial dominance over adversaries using a Multi-agent Markov Decision Process. We develop an algorithm to derive efficient teamwork strategies by combining Monte Carlo Tree Search and linear programming. When applied to team defence in football (soccer) using real-world data, our approach reduces opponent threat by 21%, outperforming optimised individual behaviour by 6%. Additionally, our model enhances the predictive accuracy of future attack locations and provides deeper insights compared to existing teamwork models that do not explicitly consider the spatial dynamics of teamwork.

Evaluating individual contributions from team members is a critical challenge across many domains, such as security and team sports. While progress has been made in valuing contributions, such as target defence in security or on-ball performance in football (soccer), many aspects of performance, such as off-ball football actions, remain difficult to quantify. We introduce GAPP, a Graph Attention Network model that predicts football pass reception probabilities and provides interpretable insights into off-ball defending. Using attention mechanisms, GAPP captures player interactions and introduces two new metrics to quantify defender contributions. We tested GAPP on 306 English Premier League matches, and showed it reduces binary cross-entropy loss by 6.4 percent compared to multiple baselines for pass reception prediction, while offering unique insights for off-ball defender evaluation for coaches, scouts and teams. This work shows the potential of graph attention networks for analysing complex multi-agent systems like football.

Multi-Agent Policy Gradient (MAPG) has made significant progress in recent years. However, centralized critics in state-of-the-art MAPG methods still face the centralized-decentralized mismatch (CDM) issue, which means sub-optimal actions by some agents will affect other agent's policy learning. While using individual critics for policy updates can avoid this issue, they severely limit cooperation among agents. To address this issue, we propose an agent topology framework, which decides whether other agents should be considered in policy gradient and achieves compromise between facilitating cooperation and alleviating the CDM issue. The agent topology allows agents to use coalition utility as learning objective instead of global utility by centralized critics or local utility by individual critics. To constitute the agent topology, various models are studied. We propose Topology-based multi-Agent Policy gradiEnt (TAPE) for both stochastic and deterministic MAPG methods. We prove the policy improvement theorem for stochastic TAPE and give a theoretical explanation for the improved cooperation among agents. Experiment results on several benchmarks show the agent topology is able to facilitate agent cooperation and alleviate CDM issue respectively to improve performance of TAPE. Finally, multiple ablation studies and a heuristic graph search algorithm are devised to show the efficacy of the agent topology.

Future reward estimation is a core component of reinforcement learning agents; i.e., Q-value and state-value functions, predicting an agent’s sum of future rewards. Their scalar output, however, obfuscates when or what individual future rewards an agent may expect to receive. We address this by modifying an agent’s future reward estimator to predict their next N expected rewards, referred to as Temporal Reward Decomposition (TRD). This unlocks novel explanations of agent behaviour. Through TRD we can: estimate when an agent may expect to receive a reward, the value of the reward and the agent’s confidence in receiving it; measure an input feature’s temporal importance to the agent’s action decisions; and predict the influence of different actions on future rewards. Furthermore, we show that DQN agents trained on Atari environments can be efficiently retrained to incorporate TRD with minimal impact on performance.

Effective communication requires the ability to refer to specific parts of an observation in relation to others. While emergent communication literature shows success in developing various language properties, no research has shown the emergence of such positional references. This paper demonstrates how agents can communicate about spatial relationships within their observations. The results indicate that agents can develop a language capable of expressing the relationships between parts of their observation, achieving over 90% accuracy when trained in a referential game which requires such communication. Using a collocation measure, we demonstrate how the agents create such references. This analysis suggests that agents use a mixture of non-compositional and compositional messages to convey spatial relationships. We also show that the emergent language is interpretable by humans. The translation accuracy is tested by communicating with the receiver agent, where the receiver achieves over 78% accuracy using parts of this lexicon, confirming that the interpretation of the emergent language was successful.

This paper discusses the importance of ethical alignment in AI systems, particularly those designed with citizen end users in mind. It explores the intersection of responsible AI, socio-technical systems, and citizen-centric design, proposing that addressing the ethical aspect of decisions in citizen-centric AI systems enhances trust and acceptance of AI technologies. We focus on four key areas: (1) the formal specification of ethical principles, (2) processes to extract and elicit individual users’ ethical preferences, (3) aggregating these ethical preferences for a collective, and (4) mechanisms to ensure that the behaviour of AI systems aligns with the collective ethical preferences. We put forward a research roadmap by identifying challenges in these areas and highlighting solution concepts with the potential to address them.

Soccer is a sport characterised by open and dynamic play, with player actions and roles aligned according to team strategies simultaneously and at multiple temporal scales with high spatial freedom. This complexity presents an analytics challenge, which to date has largely been solved by decomposing the game according to specific criteria to analyse specific problems. We propose a more holistic approach, utilising Transformer or RNN components in the novel Seq2Event model, in which the next match event is predicted given prior match events and context. We show metric creation using a general purpose context-aware model as a deployable practical application, and demonstrate development of the poss-util metric using a Seq2Event model. Summarising the expectation of key attacking events (shot, cross) during each possession, our metric is shown to correlate over matches (r=0.91, n=190) with the popular xG metric. Example practical application of poss-util to analyse behaviour over possessions and matches is made. Potential in sports with stronger sequentiality, such as rugby union, is discussed.

In this paper, we present a new application-focused benchmark dataset and results from a set of baseline Natural Language Processing and Machine Learning models for prediction of match outcomes for games of football (soccer). By doing so we give a baseline for the prediction accuracy that can be achieved exploiting both statistical match data and contextual articles from human sports journalists. Our dataset is focuses on a representative time-period over 6 seasons of the English Premier League, and includes newspaper match previews from The Guardian. The models presented in this paper achieve an accuracy of 63.18% showing a 6.9% boost on the traditional statistical methods.

In this paper we describe a novel approach to team formation based on the value of inter-agent interactions. Specifically, we propose a model of teamwork that considers outcomes from chains of interactions between agents. Based on our model, we devise a number of network metrics to capture the contribution of interactions between agents. This is then used to learn the value of teamwork from historical team performance data. We apply our model to predict team performance and validate our approach using real-world team performance data from the 2018 FIFA World Cup. Our model is shown to better predict the real-world performance of teams by up to 46% compared to models that ignore inter-agent interactions.

Online communities use trust and reputation systems to assist their users in evaluating other parties. Due to the preponderance of these systems, malicious entities have a strong incentive to attempt to influence them, and strategies employed are increasingly sophisticated. Current practice is to evaluate trust and reputation systems against known attacks, and hence are heavily reliant on expert analysts. We present a novel method for automatically identifying vulnerabilities in such systems by formulating the problem as a derivative-free optimisation problem and applying efficient sampling methods. We illustrate the application of this method for attacks that involve the injection of false evidence, and identify vulnerabilities in existing trust models. In this way, we provide reliable and objective means to assess how robust trust and reputation systems are to different kinds of attacks.

editorial Free Access Share on Introduction to the Special Section on Trust and AI Editors: Jie Zhang NTU, Singapore NTU, SingaporeView Profile , Jamal Bentahar Concordia University, Canada Concordia University, CanadaView Profile , Rino Falcone ISTC-CNR, Italy ISTC-CNR, ItalyView Profile , Timothy J. Norman University of Southampton, UK University of Southampton, UKView Profile , Murat Şensoy Blue Prism Labs, UK Blue Prism Labs, UKView Profile Authors Info & Claims ACM Transactions on Internet TechnologyVolume 19Issue 4November 2019 Article No.: 44epp 1–3https://doi.org/10.1145/3365675Online:09 November 2019Publication History 0citation344DownloadsMetricsTotal Citations0Total Downloads344Last 12 Months114Last 6 weeks6 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF

We introduce CISpaces.org, a tool to support situational understanding in intelligence analysis that complements but not replaces human expertise. The system combines natural language processing, argumentation-based reasoning, and natural language generation to produce intelligence reports from social media data, and to record the process of forming hypotheses from relationships among information. In this paper, we show how CISpaces.org meets the desirable requirements elicited from senior professionals, and demonstrate its usage and capabilities to support analysts in delivering effective and tailored intelligence to decision makers.

This paper briefly describes AIF-EL, an OWL2-EL compliant ontology for the Argument Interchange Format.

We demonstrate CISpaces.org, a tool to support situational understanding in intelligence analysis that complements but not replaces human expertise, for the first time applied to a judicial context. The system combines argumentation-based reasoning and natural language generation to support the creation of analysis and summary reports, and to record the process of forming hypotheses from relationships among information.

In making practical decisions, agents are expected to comply with ideals of behaviour, or norms. In reality, it may not be possible for an individual, or a team of agents, to be fully compliant—actual behaviour often differs from the ideal. The question we address in this paper is how we can design agents that act in such a way that they select collective strategies to avoid more critical failures (norm violations), and mitigate the effects of violations that do occur. We model the normative requirements of a system through contrary-to-duty obligations and violation severity levels, and propose a novel multi-agent planning mechanism based on Decentralised POMDPs that uses a qualitative reward function to capture levels of compliance: N-Dec-POMDPs. We develop mechanisms for solving this type of multi-agent planning problem and show, through empirical analysis, that joint policies generated are equally as good as those produced through existing methods but with significant reductions in execution time.

In this paper we combine fuzzy set theory and argumentation to facilitate the use of fuzzy arguments and attacks. Unlike many existing approaches, our work does not require the use of any parameters, bringing it closer to Dung's work in spirit. We begin by introducing Fuzzy Argumentation Frameworks, and specialise them using the Gödel t-norm. We then examine this framework's properties and show that the standard Dung extensions are obtained, though the stable semantics coincide with the preferred. Finally, we examine the relationship between our framework and Dung's original system, as well as the existing fuzzy frameworks, describing where they overlap and differ.

Models of decentralized online planning vary in the information that individual agents use to make local action decisions. Some models consider only local observations, eschewing coordination through communication. Others use communication to ensure that all agents are aware of the action decisions of others, but assume costless and delay-free communication. In this paper, we propose a model of online planning (OB-MAP) that uses estimates of the value of communicating to manage coordination through communication as costs vary. We compare this approach to existing models in widely employed benchmark problems, demonstrating that OB-MAP performs significantly better in many scenarios regardless of varying (including infinite) cost of communication.

Bootstrapping trust assessment where there is little or no evidence regarding a subject is a significant challenge for existing trust and reputation systems. When direct or indirect evidence is absent, existing approaches usually assume that all agents are equally trustworthy. This naive assumption makes existing approaches vulnerable to attacks such as Sybil and whitewashing. Inspired by real‐life scenarios, we argue that malicious agents may share some common patterns or complex features in their descriptions. If such patterns or features can be detected, they can be exploited to bootstrap trust assessments. Based on this idea, we propose the use of frequent subgraph mining and state‐of‐the‐art knowledge representation formalisms to estimate a priori trust for agents. Our approach first discovers significant patterns that may be used to characterize trustworthy and untrustworthy agents. Then, these patterns are used as features to train a regression model to estimate the trustworthiness of agents. Last, a priori trust for unknown agents (e.g., newcomers) is estimated using the discovered features based on the trained model. Through empirical evaluation, we show that the proposed approach significantly outperforms well‐known trust approaches if trustworthiness of agents is correlated with patterns in their descriptions or social networks. Furthermore, we show that the proposed approach performs at least as good as the existing approaches if such correlations do not exist.

Truth discovery is an important technique for enabling reliable crowdsourcing applications. It aims to automatically discover the truths from possibly conflicting crowdsourced claims. Most existing truth discovery approaches focus oncategoricalapplications, such as image classification. They use the accuracy, i.e., rate of exactly correct claims, to capture the reliability of participants. As a consequence, they are not effective for truth discovery inquantitativeapplications, such as percentage annotation and object counting, where similarity rather than exact matching between crowdsourced claims and latent truths should be considered. In this paper, we propose two unsupervised Quantitative Truth Finders (QTFs) for truth discovery in quantitative crowdsourcing applications. One QTF explores an additive model and the other explores a multiplicative model to capture different relationships between crowdsourced claims and latent truths in different classes of quantitative tasks. These QTFs naturally incorporate the similarity between variables. Moreover, they use the bias and the confidence instead of the accuracy to capture participants’ abilities in quantity estimation. These QTFs are thus capable of accurately discovering quantitative truths in particular domains. Through extensive experiments, we demonstrate that these QTFs outperform other state-of-the-art approaches for truth discovery in quantitative crowdsourcing applications and they are also quite efficient.

The ubiquity of smartphones has led to the emergence of mobile crowdsourcing tasks such as the detection of spatial events when smartphone users move around in their daily lives. However, the credibility of those detected events can be negatively impacted by unreliable participants with low-quality data. Consequently, a major challenge in mobile crowdsourcing is truth discovery, i.e., to discover true events from diverse and noisy participants’ reports. This problem is uniquely distinct from its online counterpart in that it involves uncertainties in both participants’mobilityandreliability. Decoupling these two types of uncertainties through location tracking will raise severe privacy and energy issues, whereas simply ignoring missing reports or treating them as negative reports will significantly degrade the accuracy of truth discovery. In this paper, we propose two new unsupervised models, i.e., Truth finder for Spatial Events (TSE) and Personalized Truth finder for Spatial Events (PTSE), to tackle this problem. In TSE, we model location popularity, location visit indicators, truths of events, and three-way participant reliability in a unified framework. In PTSE, we further model personal location visit tendencies. These proposed models are capable of effectively handling various types of uncertainties and automatically discovering truths without any supervision or location tracking. Experimental results on both real-world and synthetic datasets demonstrate that our proposed models outperform existing state-of-the-art truth discovery approaches in the mobile crowdsourcing environment.

To enable reliable crowdsourcing applications, it is of great importance to develop algorithms that can automatically discover the truths from possibly noisy and conflicting claims provided by various information sources. In order to handle crowdsourcing applications involving big or streaming data, a desirable truth discovery algorithm should not only beeffective, but also bescalable. However, with respect to quantitative crowdsourcing applications such as object counting and percentage annotation, existing truth discovery algorithms are not simultaneously effective and scalable. They either address truth discovery in categorical crowdsourcing or perform batch processing that does not scale. In this paper, we propose new parallel and streaming truth discovery algorithms for quantitative crowdsourcing applications. Through extensive experiments on real-world and synthetic datasets, we demonstrate that 1) both of them are quite effective, 2) the parallel algorithm can efficiently perform truth discovery on large datasets, and 3) the streaming algorithm processes data incrementally, and it can efficiently perform truth discovery both on large datasets and in data streams.

Information about quantitative characteristics in local businesses and services, such as the number of people waiting in line in a cafe and the number of available fitness machines in a gym, is important for informed decision, crowd management and event detection. In this paper, we investigate the potential of leveraging crowds as sensors to report such quantitative characteristics and investigate how to recover the true quantity values from noisy crowdsourced information. Through experiments, we find that crowd sensors have both bias and variance in quantity sensing, and task difficulties impact the sensing accuracy. Based on these findings, we propose an unsupervised probabilistic model to jointly assess task difficulties, ability of crowd sensors and true quantity values. Our model differs from existing categorical truth finding models as ours is specifically designed to tackle quantitative truth. In addition to devising an efficient model inference algorithm in a batch mode, we also design an even faster online version for handling streaming data. Experimental results in various scenarios demonstrate the effectiveness of our model.

Securing provenance has recently become an important research topic, resulting in a number of models for protecting access to provenance. Existing work has focused on graph transformation mechanisms that supply a user with a provenance view that satisfies both access control policies and validity constraints of provenance. However, it is not always possible to satisfy both of them simultaneously, because these two conditions are often inconsistent which require sophisticated conflict resolution strategies to be put in place. In this paper we develop a new access control model tailored for provenance. In particular, we explicitly take into account validity constraints of provenance when specifying certain parts of provenance to which access is restricted. Hence, a provenance view that is granted to a user by our authorisation mechanism would automatically satisfy the validity constraints. Moreover, we propose algorithms that allow provenance owners to deploy fine-grained access control for their provenance data.

No abstract available.

The ubiquity of smartphones has led to the emergence of mobile crowdsourcing tasks such as the detection of spatial events when smartphone users move around in their daily lives. However, the credibility of those detected events can be negatively impacted by unreliable participants with low-quality data. Consequently, a major challenge in quality control is to discover true events from diverse and noisy participants' reports. This truth discovery problem is uniquely distinct from its online counterpart in that it involves uncertainties in both participants' mobility and reliability. Decoupling these two types of uncertainties through location tracking will raise severe privacy and energy issues, whereas simply ignoring missing reports or treating them as negative reports will significantly degrade the accuracy of the discovered truth. In this paper, we propose a new method to tackle this truth discovery problem through principled probabilistic modeling. In particular, we integrate the modeling of location popularity, location visit indicators, truth of events and three-way participant reliability in a unified framework. The proposed model is thus capable of efficiently handling various types of uncertainties and automatically discovering truth without any supervision or the need of location tracking. Experimental results demonstrate that our proposed method outperforms existing state-of-the-art truth discovery approaches in the mobile crowdsourcing environment.

We present the CISpaces framework, a collaborative virtual space for intelligence analysts for the elaboration of information to explain a situation. CISpaces supports the analysis of conflicting information in collaboration exploiting argumentation schemes to structure and share analyses, crowd-sourcing to collect information and provenance to establish the credibility of hypotheses.

Systems for allowing users to manage access to their personal data are important for a wide variety of applications including healthcare, where authorised individuals may need to share information in ways that the owner had not anticipated. Simply denying access in unknown cases may hamper critical decisions and affect service delivery. Rather, decisions can be made considering the risk of a given sharing request, and the trustworthiness of the requester. We propose a trust- and risk-aware access control mechanism (TRAAC) and a sparse zone-based policy model, which together allow decision-making on the basis of the requester's trustworthiness with regards to both the information to be shared, and the completion of obligations designed to mitigate risk. We formalise our approach and compare it with an existing approach that does not model trust through simulation.

Large-scale multiagent systems have the potential to be highly dynamic. Trust and reputation are crucial concepts in these environments, as it may be necessary for agents to rely on their peers to perform as expected, and learn to avoid untrustworthy partners. However, aspects of highly dynamic systems introduce issues which make the formation of trust relationships difficult. For example, they may be short-lived, precluding agents from gaining the necessary experiences to make an accurate trust evaluation. This article describes a new approach, inspired by theories of human organizational behavior, whereby agents generalize their experiences with previously encountered partners as stereotypes , based on the observable features of those partners and their behaviors. Subsequently, these stereotypes are applied when evaluating new and unknown partners. Furthermore, these stereotypical opinions can be communicated within the society, resulting in the notion of stereotypical reputation . We show how this approach can complement existing state-of-the-art trust models, and enhance the confidence in the evaluations that can be made about trustees when direct and reputational information is lacking or limited. Furthermore, we show how a stereotyping approach can help agents detect unwanted biases in the reputational opinions they receive from others in the society.

The increasing need to share information in dynamic environments has created a requirement for risk-aware access control systems. In this paper, we present a metamodel for risk-aware authorization that captures the key aspects of a system in relation to risk mitigation. In particular, we develop various risk-aware models as instances of the metamodel that broadly differ in the form of risk mitigation that is used (system obligations and user obligations respectively), and study how those obligations are applied to reduce and account for the risk incurred by granting access. Unlike system obligations, an access control system cannot guarantee that user obligations are fulfilled. We propose two approaches to defining risk-aware authorization semantics that takes unfulfilled obligations into account: one is to restrict users' future access because of prior failure to fulfill obligations, and the other is to “reward” users who have been diligent in fulfilling their obligations by permitting risky access requests.

Trust is crucial in dynamic multi-agent systems, where agents may frequently join and leave, and the structure of the society may often change. In these environments, it may be difficult for agents to form stable trust relationships necessary for confi-dent interactions. Societies may break down when trust between agents is too low to motivate inter-actions. In such settings, agents should make de-cisions about who to interact with, given their de-gree of trust in the available partners. We propose a decision-theoretic model of trust decision mak-ing allows controls to be used, as well as trust, to increase confidence in initial interactions. We con-sider explicit incentives, monitoring and reputation as examples of such controls. We evaluate our ap-proach within a simulated, highly-dynamic multi-agent environment, and show how this model sup-ports the making of delegation decisions when trust is low. 1

In this paper we describe a software assistant agent that can proactively assist human users situated in a time-constrained environment to perform normative reasoning-reasoning about prohibitions and obligations-so that the user can focus on her planning objectives. In order to provide proactive assistance, the agent must be able to 1) recognize the user's planned activities, 2) reason about potential needs of assistance associated with those predicted activities, and 3) plan to provide appropriate assistance suitable for newly identified user needs. To address these specific requirements, we develop an agent architecture that integrates user intention recognition, normative reasoning over a user's intention, and planning, execution and replanning for assistive actions. This paper presents the agent architecture and discusses practical applications of this approach.

Many organizations depend on critical sensory information to achieve their tasks. As the number of those tasks increases, efficient determination and allocation of required resources in sensor networks become crucial. In this paper, we propose means to describe tasks semantically with their requirements and constraints so that software agents can reason about those tasks and determine what type of sensor resources they may need. Based on the semantic description and reasoning mechanisms, we propose a distributed agent-based approach to efficiently allocate sensor resources to tasks. Our evaluation of the proposed approach shows that not only it enables fully automated determination and allocation of resources for tasks, but also the resulting allocation is efficient and close to optimum.

In this paper, we describe how agents can support collaborative planning within international coalitions, formed in an ad hoc fashion as a response to military and humanitarian crises. As these coalitions are formed rapidly and without much lead time or co-training, human planners may be required to observe a plethora of policies that direct their planning effort. In a series of experiments, we show how agents can support human planners, ease their cognitive burden by giving advice on the correct use of policies and catch possible violations. The experiments show that agents can effectively prevent policy violations with no significant extra cost.

This article presents a formal framework and outlines a method that autonomous agents can use to negotiate the semantics of their communication language at run–time. Such an ability is needed in open multi–agent systems so that agents can ensure they understand the implications of the utterances that are being made and so that they can tailor the meaning of the primitives to best fit their prevailing circumstances. To this end, the semantic space framework provides a systematic means of classifying the primitives along multiple relevant dimensions. This classification can then be used by the agents to structure their negotiation (or semantic fixing) process so that they converge to the mutually agreeable semantics that are necessary for coherent social interactions.

In this paper, we present an abstract formal model of decision‐making in a social setting that covers all aspects of the process, from recognition of a potential for cooperation through to joint decision. In a multi‐agent environment, where self‐motivated autonomous agents try to pursue their own goals, a joint decision cannot be taken for granted. In order to decide effectively, agents need the ability to (a) represent and maintain a model of their own mental attitudes, (b) reason about other agents' mental attitudes, and (c) influence other agents' mental states. Social mental shaping is advocated as a general mechanism for attempting to have an impact on agents' mental states in order to increase their cooperativeness towards a joint decision. Our approach is to specify a novel, high‐level architecture for collaborative decision‐making in which the mentalistic notions of belief, desire, goal, intention, preference and commitment play a central role in guiding the individual agent's and the group's decision‐making behaviour. We identify preconditions that must be fulfilled before collaborative decision‐making can commence and prescribe how cooperating agents should behave, in terms of their own decision‐making apparatus and their interactions with others, when the decision‐making process is progressing satisfactorily. The model is formalized through a new, many‐sorted, multi‐modal logic.

This paper presents a framework that captures how the social nature of agents that are situated in a multi‐agent environment impacts upon their individual mental states. Roles and social relationships provide an abstraction upon which we develop the notion of social mental shaping. This allows us to extend the standard Belief‐Desire‐Intention model to account for how common social phenomena (e.g. cooperation, collaborative problem‐solving and negotiation) can be integrated into a unified theoretical perspective that reflects a fully explicated model of the autonomous agent’s mental state.

This paper describes work undertaken in the ADEPT (Advanced Decision Environment for Process Tasks) project towards developing an agent-based infrastructure for managing business processes. We describe how the key technology of negotiating, service providing, autonomous agents was realized and demonstrate how this was applied to the BT (British Telecom) business process of providing a customer quote for network services.