The informative role of the context and its efficient representation as “world models” are key for the strategic behavior of software agents engaged in automated negotiations to reach agreements aligned with established preferences and goals in complex and competitive environment such as e-markets. Existing approaches tackle many of the problems related to opponent modeling, preference elicitation, domain narrowing and protocol selection, as well as the definition of ontological models. However, these agents face problems to cope with dynamic contexts as they often lack of a context-aware nature and cognitive abilities to act intelligently, which raises serious concerns about their rationality to act and explain the decision-making process used for negotiations. This paper explores the potential of Knowledge Graphs (KGs) to enhance the semantic contextual understanding, adaptability to dynamic contexts and the informed decision-making process in negotiation agents. The proposal suggests leveraging the expandable nature of KGs to integrate information and metadata from the context while improving the agents' cognitive abilities with contextual understanding, reasoning and inference. Promising results are presented in a smart grid case study, envisioning a future of self-explainable negotiation agents that make rational decisions given the contextual circumstances encoded in their KGs.

With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for obtaining good performances regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the \textit{Q}-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions.