Recent research has shown that integrating domain knowledge into deep learning architectures is effective; It helps reduce the amount of required data, improves the accuracy of the models' decisions, and improves the interpretability of models. However, the research community lacks a convened benchmark for systematically evaluating knowledge integration methods. In this work, we create a benchmark that is a collection of nine tasks in the domains of natural language processing and computer vision. In all cases, we model external knowledge as constraints, specify the sources of the constraints for each task, and implement various models that use these constraints. We report the results of these models using a new set of extended evaluation criteria in addition to the task performances for a more in-depth analysis. This effort provides a framework for a more comprehensive and systematic comparison of constraint integration techniques and for identifying related research challenges. It will facilitate further research for alleviating some problems of state-of-the-art neural models.

Structured learning algorithms usually involve an inference phase that selects the best global output variables assignments based on the local scores of all possible assignments. We extend deep neural networks with structured learning to combine the power of learning representations and leveraging the use of domain knowledge in the form of output constraints during training. Introducing a non-differentiable inference module to gradient-based training is a critical challenge. Compared to using conventional loss functions that penalize every local error independently, we propose an inference-masked loss that takes into account the effect of inference and does not penalize the local errors that can be corrected by the inference. We empirically show the inference-masked loss combined with the negative log-likelihood loss improves the performance on different tasks, namely entity relation recognition on CoNLL04 and ACE2005 corpora, and spatial role labeling on CLEF 2017 mSpRL dataset. We show the proposed approach helps to achieve better generalizability, particularly in the low-data regime.

Complex services in service-oriented architectures such as the grid typically require to be configured in multiple ways that cannot be anticipated by service designers; we illustrate this requirement by studying the myGrid registry, a grid registry capable of supporting annotations of service descriptions by third-party users. Instead, services have to be conceived so that they can be configured at deployment and run time. We argue that KAoS is a powerful and flexible language that can help define such configurations. Using our registry case study, we examine the requirements that the definition of such complex configurations brings on policy languages and explain how they can be satisfied. Specifically, we use role-value maps to express constraints between property values; we introduce a notion of PolicySet with associated parameters that support constraints within a well defined scope; finally, we define a notion of context that allows us to refer to property values that were extant in past execution environments. Essentially, these concepts allow us to add constraints to values in policy definitions, to organise policies in coherent and structure blocks, and to refer to the execution history. The paper discusses these concepts and how they are implemented in a binding of the KAoS policy language to the myGrid registry.

The next decade will bring radical changes to the way we do information processing, as applications composed of many cooperating distributed subsystems, exploiting different computational paradigms, become more common. This situation increases substantially demands in the area of system management, correctness analysis, understanding, debugging and performance evaluation. Managed Object-based Distributed Monitoring System (MODIMOS) is a project aimed at development of an adaptable platform for visualization of distributed applications, built of interoperating heterogeneous components. MODIMOS is expandable, allowing to add new monitored environments. It employs various management mechanisms to cope with gathered information size and complexity.