Understanding the intent is an essential step for maintaining effective communications. This essential feature is used in communications for assembling, patrolling, and surveillance. A fused and interactive multimodal system for human-robot communication, used in assembly applications, is presented in this paper. Communication is multimodal. Having the options of multiple communication modes such as gestures, text, symbols, graphics, images, and speech increase the chance of effective communication. The intent is the main component that we are aiming to model, specifically in human machine dialogues. For this, we extract the intents from spoken dialogues and fuse the intent with any detected matching gesture that is used in interaction with the robot. The main components of the presented system are: (1) a speech recognizer system using Kaldi, (2) a deep-learning based Dual Intent and Entity Transformer (DIET) based classifier for intent and entity extraction, (3) a hand gesture recognition system, and (4) a dynamic fusion model for speech and gesture based communication. These are evaluated on contextual assembly situation using a simulated interactive robot.

Given the need for multimodal autonomous human-robot interactive systems serving complex situations, a noise-robust and context- aware situated multimodal spoken dialogue system (MSSDS) in emergency evacuation missions is presented. The MSSDS system is composed of: (1) a context and noise robust Wake-Up-Word (WUW) (a) to initiate the dialogue, and (b) to detect the context switching by explicit use of WUW during emergency evacuation mission steps, (2) multi-turn user- initiated interactive text-and-spoken dialog communication system, and (3) an interactive voice and text interface for human-robot communication designed based on dialogues used in real-world emergency situations. In an emergency environment, speech is mixed with different noises, and therefore communication using speech in such an environment is challenging. We handle the noise by using Team Connect Ceiling (TCC) beam-forming microphone arrays. Innovative and useful applications of spoken dialogue systems, presented as proof of concept, constitute another contribution. Numerous digital assistants, keyword spotting, and wake-up-words-based technologies have already been developed, but these are mainly used indoors. Our objective is to support communication in complex environments, e.g., indoors and outdoors, in human and machine teaming, via a wake-up-word-based multimodal interactive system. The development of the real-world application to communicate with the robot using multimodalities in complex situations based on the presented approach is in progress while the presented simulated approach is reflecting parts of this development. Numerous machine learning technologies and toolkits have been applied in this ongoing development process. The novelty of WUW-based MSSDS is discussed in this paper. Our Markov Decision Process (MDP) evaluation shows that the WUW-based MSSDS performs better.

The Distributed Constraint Optimization Problem (DCOP) lies at the foundations of multiagent cooperation. With DCOPs, the optimization in distributed resource allocation problems is formalized using constraint optimization problems. The solvers for the problem are designed based on decentralized cooperative algorithms that are performed by multiple agents. In a conventional DCOP, a single objective is considered. The Multiple Objective Distributed Constraint Optimization Problem (MODCOP) is an extension of the DCOP framework, where agents cooperatively have to optimize simultaneously multiple objective functions. In the conventional MODCOPs, a few objectives are globally defined and agents cooperate to find the Pareto optimal solution. However, such models do not capture the interests of each agent. On the other hand, in several practical problems, the share of each agent is important. Such shares are modeled as preference values of agents. This class of problems can be defined using the MODCOP on the preferences of agents. In particular, we define optimization problems based on leximin ordering and Asymmetric DCOPs (Leximin AMODCOPs). The leximin defines an ordering among vectors of objective values. In addition, Asymmetric DCOPs capture the preferences of agents. Because the optimization based on the leximin ordering improves the equality among the satisfied preferences of the agents, this class of problems is important. We propose several solution methods for Leximin AMODCOPs generalizing traditional operators into the operators on sorted objective vectors and leximin. The solution methods applied to the Leximin AMODCOPs are based on pseudo trees. Also, the investigated search methods employ the concept of boundaries of the sorted vectors.

Distributed Constraint Optimization Problem (DCOP) has been studied as a fundamental component of multiagent systems. With DCOPs, various applications on multiagent systems are formalized as constraint optimization problems where variables and functions are distributed among agents. Leximin AMODCOP has been proposed as a class of Multiple Objective DCOPs, where multiple objectives for individual agents are optimized based on the leximin operator. This problem also relates to Asymmetric DCOPs based on its the criteria of fairness among agents. Previous studies explore only Leximin AMODCOPs on constraint graphs limited to functions with unary or binary scopes. We address the Leximin AMODCOPs on factor graphs that directly represent n-ary functions. A dynamic programming method on factor graphs is investigated as an exact solution method. In addition, for relatively dense problems, we also investigate several approximate/inexact algorithms.

We focus on the problem of selecting reviewers (or raters) that are considered by a recommender system (or a user) under the aspect of security. Malicious reviewers can exert unreasonable influence, and can bias online consumers unfairly against an attacked item or competitor. This paper proposes an approach where a meta-recommender maximizes the aggregated diversity of reviewers when deciding which reviews should be considered by a recommender system or an online consumer. This problem can be of interest in many domains where producers or service providers may seek advantages by compromising competitors with fake reviews or ratings such as tourism and hospitality industries or even free open-source software. A solution is proposed for users linked in social networks, such as unstructured P2P societies. In order to evaluate the proposed solution, we describe a mechanism of selecting reviewers of software updates such that not all end-users of a software are impacted by a potentially malicious strict subset of all available reviewers, and we experimentally assess resistance to attacks.

Much of the Distributed Constraint Satisfaction Problem (DisCSP) solving research has addressed cooperating agents, and privacy was frequently mentioned as a significant motivation of the decentralization. While privacy may have a role for cooperating agents, it is easier understood in the context of self-interested utility-based agents, and this is the situation considered here. With utility-based agents, the DisCSP framework can be extended to model privacy and satisfaction under the concept of utility. We introduce Utilitarian Distributed Constraint Satisfaction Problems (UDisCSP), an extension of the DisCSP that exploits the rewards for finding a solution and the costs for losing privacy as guidance for the utility-based agents. A parallel can be drawn between Partially Observable Markov Decision Processes (POMDPs) and the problems solved by individual agents for UDisCSPs. Common DisCSP solvers are extended to take into account the utility function. In these extensions we assume that the planning problem is further restricting the set of communication actions to only the ones available in the corresponding solver protocols. The solvers obtained propose the action to be performed in each situation, defining thereby the policy of the agents.

Peer-to-peer infrastructures in a world where many systems are mobile, or found behind NATs and firewalls that cannot be programmed by users, require significant support from computers with appropriate connectivity, called supernodes. We introduce a new fully decentralized unstructured peer-to-peer (P2P) approach to open-source instant messaging systems (that is employed in the DirectDemocracyP2P system). Here each human owning a peer can control the traffic supported by her system. The control may be based on criteria such as: (a) her desire to help the endpoints of the communication based on friendship, (b) her desire to help a cause based on the content/topic of the communication, (c) reputation and rewards, or (d) her interest in the handled data itself. Providing intrinsic incentives for peers to help with traffic is important in order for an open-source freeware P2P system to eventually be viable. In non-incentive P2P systems like Skype, availability of open-source versions can potentially starve the system of supernodes (once users learn how to disable the resource consuming supernode-function). This paper analyzes key functions of the solution such as integration of incentive management into an extension of the STUN protocol, connection establishment and message transfer under different network setups. We show how to use the results of surveys, in conjunction with simulations, to quantify the effects of various incentives on the survivability of an open approach.

DirectDemocracyP2P is an open source platform developed in JAVA and offering peer-to-peer and mobile ad hoc wireless communication capabilities. The platform offers an API supporting plugins, beside its main application: deliberative petition drives (aka citizens' initiatives with integrated argumentation) [1]. An authentication-by-reputation technique based on digital signatures and peer review [2], [3] is integrated into the platform via this main application. Each peer manages independently its database of items of interest. The items of interest are encapsulated as self-contained pieces of information and uniquely identifiable using a system of global identifiers (GIDs). Each GID consists of a combination of public keys with creation dates, or digest values. Communication is based on a combination of push and pull mechanisms. [1].

Forming effective coalitions is a major research challenge in AI and multi-agent systems (MAS). Coalition Structure generation (CSG) involves partitioning a set of agents into coalitions so that social surplus (the sum of the rewards of all coalitions) is maximized. A partition is called a Coalition Structure (CS). In traditional works, the value of a coalition is given by a black box function called a characteristic function. In this paper, we propose a novel formalization of CSG, i.e., we assume the value of a characteristic function is given by an optimal solution of a distributed constraint optimization problem (DCOP) among the agents of a coalition. A DCOP is a popular approach for modeling cooperative agents, since it is quite general and can formalize various application problems in MAS. At first glance, one might assume that the computational costs required in this approach would be too expensive, since we need to solve an NP-hard problem just to obtain the value of a single coalition. To optimally solve a CSG, we might need to solve n-th power of 2 DCOP problem instances, where n is the number of agents. However, quite surprisingly, we show that an approximation algorithm, whose computational cost is about the same as solving just one DCOP, can find a CS with quality guarantees. More specifically, we develop an algorithm with parameter k that can find a CS whose social surplus is at least max(k/(w*+1), 2k/n) of the optimal CS, where w* is the tree width of a constraint graph. When k=1, the complexity of this algorithm is about the same as solving just one DCOP. These results illustrate that the locality of interactions among agents, which is explicitly modeled in the DCOP formalization, is quite useful in developing an efficient CSG algorithm with quality guarantees.

Distributed problems raise privacy issues. The user would like to specify securely his constraints (desires, availability, money) on his computer once. The computer is expected to compute and communicate for searching an acceptable solution while maintaining the privacy of the user. Even without computers infested with spy viruses that capture the interaction with the user, most agent based approaches reveal parts of one agent's secret data to its partners in distributed computations [Using privacy loss to guide decisions in distributed CSP search]. Some cryptographic multi-party computation protocols [Completeness theorems for non-cryptographic fault-tolerant distributed computating] succeed to avoid leaking secrets at the computation of some functions with private inputs. They have been applied to find the set of all solutions for the meeting scheduling problem [On securely scheduling a meeting]. However, nobody yet succeeded to apply those techniques for finding a random solution to the meeting scheduling problem. Note that revealing all solutions, when you only need a single one, leaks a lot of data about when others are, or are not, available. Some answers were proposed in our previous approaches to distributed constraint problems [Solving a distributed CSP with cryptographic multi-party computations, without revealing constraints and without involving trusted servers]. They guarantee that no agent can infer with certitude a secret from the identity of the solution of the problem (other than the acceptance of the solution), but guarantee nothing about inference of probabilistic information about secrets. Our new technique answers this problem, too.

This paper addresses the problem of detecting keywords in unconstrained speech without explicit modeling of non-keyword segments. The proposed algorithm is based on recent developments in confidence measures using local posterior probabilities, and searches for the segment maximizing the average observation posteriori along the most likely path in the hypothesized keyword model. As known, this approach (sometimes referred to as sliding model method) requires a relaxation of the begin/endpoints of the Viterbi matching, as well as a time normalization of the resulting score, making dynamic programming sub-optimal or more complex (more computation and/or more memory). We present here an alternative (quite simple and efficient) solution to this problem, using an iterative form of Viterbi decoding algorithm, but which does not require scoring for all possible begin/endpoints. Convergence proof of this algorithm is available (Silaghi and Bourlard, 1999). Results obtained with this method on 100 keywords chosen at random from the BREF database are reported.