We propose an automated negotiation for a reinforcement learning agent to adapt the agent to unexpected situations such as demand changes in supply chain management (SCM). Existing studies that consider reinforcement learning and SCM assume a centralized environment where the coordination of chain components is hierarchical rather than through negotiations between agents. This study focused on a negotiation agent that considered the value function of reinforcement learning for SCM as its utility function in automated negotiation. We demonstrated that the proposed approach could avoid inventory shortages under increased demand requests from the terminal customer.

This study proposed a novel reward-based negotiating agent strategy using an issue-based represented deep policy network. We compared the negotiation strategies with reinforcement learning (RL) by the tournaments toward heuristics-based champion agents in multi-issue negotiation. A bilateral multi-issue negotiation in which the two agents exchange offers in turn was considered. Existing RL architectures for a negotiation strategy incorporate rich utility function that provides concrete information even though the rewards of RL are considered as generalized signals in practice. Additionally, in existing reinforcement learning architectures for negotiation strategies, both the issue-based representations of the negotiation problems and the policy network to improve the scalability of negotiation domains are yet to be considered. This study proposed a novel reward-based negotiation strategy through deep RL by considering an issue-based represented deep policy network for multi-issue negotiation. Comparative studies analyzed the significant properties of negotiation strategies with RL. The results revealed that the policy-based learning agents with issue-based representations achieved comparable or higher utility than the state-of-the-art baselines with RL and heuristics, especially in the large-sized domains. Additionally, negotiation strategies with RL based on the policy network can achieve agreements by effectively using each step.

Previous research on the comprehensive negotiation strategy using deep reinforcement learning (RL) has scalability issues of not performing effectively in the large-sized domains. We improve negotiation strategy via deep RL by considering an issue-based represented deep policy network to deal with multi-issue negotiation. The architecture of the proposed learning agent considers the characteristics of multi-issue negotiation domains and policy-based learning. We demonstrate that proposed method achieve equivalent or higher utility than existing negotiation agents in the large-sized domains.

Existing research in the field of automated negotiation considers a negotiation architecture in which some of the negotiation components are designed separately by reinforcement learning (RL), but comprehensive negotiation strategy design has not been achieved. In this study, we formulated an RL model based on a Markov decision process (MDP) for bilateral multi-issue negotiations. We propose a versatile negotiating agent that can effectively learn various negotiation strategies and domains through comprehensive strategies using deep RL. We show that the proposed method can achieve the same or better utility than existing negotiation agents.

With the availability of domain specific historical negotiation data, the practical applications of machine learning techniques can prove to be increasingly effective in the field of automated negotiation. Yet a large portion of the literature focuses on domain independent negotiation and thus passes the possibility of leveraging any domain specific insights from historical data. Moreover, during sequential negotiation, utility functions may alter due to various reasons including market demand, partner agreements, weather conditions, etc. This poses a unique set of challenges and one can easily infer that one strategy that fits all is rather impossible in such scenarios. In this work, we present a simple yet effective method of learning an end-to-end negotiation strategy from historical negotiation data. Next, we show that transfer learning based solutions are effective in designing adaptive strategies when underlying utility functions of agents change. Additionally, we also propose an online method of detecting and measuring such changes in the utility functions. Combining all three contributions we propose an adaptive automated negotiating agent framework that enables the automatic creation of transfer learning based negotiating agents capable of adapting to changes in utility functions. Finally, we present the results of an agent generated using our framework in different ANAC domains with 100 different utility functions each and show that our agent outperforms the benchmark score by domain independent agents by 6%.

We address the joint task allocation and path planning problem whereby an operator with a fleet of vehicles must assign multiple tasks to each vehicle, while ensuring collision-free paths for them such that the total travel cost is minimized. Instead of sequentially solving the task allocation problem first, and then resolving all predicted collisions, i.e. conflicts between vehicles, we propose a novel method that solves in a simultaneous way task allocation and multi-agent path planning. Specifically, we introduce an extension of the Time Dependent Vehicle Routing Problem (TDVRP) whereby we propose to integrate conflicts information into a time dependent cost function used in the task allocation resolution. We compare our approach to two baseline approaches that both use a standard Capacitated VRP (CVRP) solver, a “one-shot” method and a “multi-shot” method. We perform simulations on benchmark realistic warehouse scenarios and the obtained results show that our proposed approach is able to generate improvements in the solutions costs compared to the baseline approaches.

This paper addresses the problem of path negotiation among self-interested multirobot operators in shared space. In conventional multirobot path planning problems, most of the research thus far has focused on the coordination and planning of collision-free paths for multiple robots with some common objectives. On the contrary, the recent progress of technologies in autonomous vehicles, including automated guidance vehicles, unmanned aerial vehicles, and manned autonomous cars, has increased demand for solving coordination and conflict avoidance in these autonomous and self-interested agents that pursue their own objectives. In this research, we tackle this problem from the operator perspective. We assume a problem setting where collisions between robots are avoided based on path reservation and negotiation. Under that circumstance, we propose a task-oriented utility function and a path negotiation algorithm for robot operators to maximize their task utility during path negotiation. The simulation and experiment results demonstrate the effectiveness of our task-based negotiation method over a simple path-based negotiation approach.

Given a large amount of table data, how can we find the tables that contain the contents we want? A naive search fails when the column names are ambiguous, such as if columns containing stock price information are named “Close” in one table and named “P” in another table.One way of dealing with this problem that has been gaining attention is the semantic annotation of table data columns by using canonical knowledge. While previous studies successfully dealt with this problem for specific types of table data such as web tables, it still remains for various other types of table data: (1) most approaches do not handle table data with numerical values, and (2) their predictive performance is not satisfactory.This paper presents a novel approach for table data annotation that combines a latent probabilistic model with multilabel classifiers. It features three advantages over previous approaches due to using highly predictive multi-label classifiers in the probabilistic computation of semantic annotation. (1) It is more versatile due to using multi-label classifiers in the probabilistic model, which enables various types of data such as numerical values to be supported. (2) It is more accurate due to the multi-label classifiers and probabilistic model working together to improve predictive performance. (3) It is more efficient due to potential functions based on multi-label classifiers reducing the computational cost for annotation.Extensive experiments demonstrated the superiority of the proposed approach over state-of-the-art approaches for semantic annotation of real data (183 human-annotated tables obtained from the UCI Machine Learning Repository).

Automatic negotiation is gaining more interest recently thanks to the wider deployment of intelligent systems and the need for them to cooperate/compete on behalf of their users. A central assumption of most autonomous negotiation agents is that the utility function of the user is perfectly known to the agent. That is an often unmet assumption in real situations. Utility elicitation is the process of learning about the utility function of the user incrementally and has a long history in decision support research. Recently, some utility elicitation systems capable of incrementally eliciting the utility function of the user during the negotiation were presented. This work expands this body of research by optimizing the elicitation algorithm to realistic elicitation strategies. The proposed method extends the optimal elicitation algorithm to the - practical - case where queries to the user only reduce the uncertainty in the utility function without removing it completely. Extensive evaluation shows that the proposed extension outperforms two state-of-the-art elicitation algorithms and several baseline alternatives.

Given a collection of basic customer demographics (e.g., age and gender) andtheir behavioral data (e.g., item purchase histories), how can we predictsensitive demographics (e.g., income and occupation) that not every customermakes available?This demographics prediction problem is modeled as a classification task inwhich a customer's sensitive demographic y is predicted from his featurevector x. So far, two lines of work have tried to produce a"good" feature vector x from the customer's behavioraldata: (1) application-specific feature engineering using behavioral data and (2) representation learning (such as singular value decomposition or neuralembedding) on behavioral data. Although these approaches successfullyimprove the predictive performance, (1) designing a good feature requiresdomain experts to make a great effort and (2) features obtained fromrepresentation learning are hard to interpret. To overcome these problems, we present a Relational Infinite SupportVector Machine (R-iSVM), a mixture-of-experts model that can leveragebehavioral data. Instead of augmenting the feature vectors of customers, R-iSVM uses behavioral data to find out behaviorally similar customerclusters and constructs a local prediction model at each customer cluster. In doing so, R-iSVM successfully improves the predictive performance withoutrequiring application-specific feature designing and hard-to-interpretrepresentations. Experimental results on three real-world datasets demonstrate the predictiveperformance and interpretability of R-iSVM. Furthermore, R-iSVM can co-existwith previous demographics prediction methods to further improve theirpredictive performance.

Data-intensive computing systems like MapReduce and Dryad have emerged as a framework for leveraging computing resources of a cluster. I/O bottlenecks need to be eased to improve performance in data-intensive computing systems. State-of-the-art frameworks for data-intensive computing have tackled the issue with a data locality based task scheduling policy. However, locality-aware scheduling does not always work good to mitigate I/O bottlenecks when different I/O characteristic jobs run concurrently. This paper presents LoadAtomizer, a locality and I/O load aware task scheduler for MapReduce. LoadAtomizer mitigates the I/O bottlenecks of a cluster with locality and I/O load aware map task assignment and storage selection. LoadAtomizer quickly assigns a slave a map task whose input data is stored in a lightly loaded storage and commands the slave to read the input data from the storage. LoadAtomizer maintains the load information of storages and the network with a topology-aware load tree. A topology-aware load tree enables LoadAtomizer to select quickly a lightly loaded storage that a slave can access through a lightly loaded network path. Experimental results demonstrated that our prototype of LoadAtomizer shortened completion time of multiple jobs by up to 18.6 %.

Recently, MapReduce has been used to parallelize machine learning algorithms. To obtain the best performance for these algorithms, tuning the parameters of the algorithms is required. However, this is time consuming because it requires executing a MapReduce program multiple times using various parameters. Such multiple executions can be assigned to a cluster in various ways, and the execution time varies depending on the assignments. To achieve the shortest execution time, we propose a method for optimizing the assignment of MapReduce jobs to a cluster assuming machine learning targeted runtime. We developed an execution cost model to predict the total execution time of jobs and obtained the optimal assignment by minimizing the cost model. To evaluate the proposed method, we implemented an experimental MapReduce runtime based on Message Passing Interface and executed logistic regression in four cases. The results showed that the proposed method can correctly predict the optimal job assignment. We also confirmed that the optimal assignment reduced execution time by a maximum 77% compared to the worst assignment.

With the proliferation of mobile-network services, mobile networks have become one of the core social infrastructures and are therefore required to operate stably. Conventional mobile-network-management systems can detect and recover from faults according to previously formulated rules. However, such semi-static rules are vulnerable to large variation in quality of networks and to sudden increases in traffic, which are inherent to radio access networks. Maintenance personnel often had to perform case-by-case analyses for the vast numbers of abnormal cases and such analyses and creation of rules may have taken more time when new faults appeared. In addition, providing network elements timely is essential to maintaining network quality but it is becoming more difficult as mobile networks become larger and more complicated. To overcome these problems, we have developed a system that enhances the conventional UMTS Terrestrial Radio Access Network (UTRAN) management system of mobile networks and improves stability during network operation. In this paper, we present main features of the system and its fundamental technologies: fault detection based on statistical reliability, root-cause analysis of the fault, a rule editor easy to create and modify fault-analysis rules for maintenance personnel, and guidance on expansion based on long-term analysis of trends. We tested this system then confirmed it could reduce fault-detection errors in a conventional management system through theoretical calculations and trials in a cellular mobile network.

Web sites occasionally experience sharp fluctuation in load. The quality of such services can be maintained by allocating servers according to the load. Such autonomic service level management requires server capacity planning. However, existing capacity planning functions cannot appropriately calculate capacity in a heterogeneous server cluster, nor can they facilitate the prioritizing of services. As a result, high-priority services may deteriorate, while the quality of low-priority services remains high. Our approach achieves appropriate capacity planning for a heterogeneous server cluster by the resolution of integer programming induced by a certain status in a system model The status is specified by the consideration of a weighted round-robin algorithm of a managed load balancer. The priority allocation function is facilitated by fuzzy control.