The proliferation of news sources on the web amplifies the problem of disinformation and misinformation, impacting public perception and societal stability. These issues necessitate the identification of bias in news broadcasts, whereby the analysis and understanding of speaker roles and news contexts are essential prerequisites. Although there is prior research on multimodal speaker role recognition (mostly) in the news domain, modern feature representations have not been explored yet, and no comprehensive public dataset is available. In this paper, we propose novel approaches to classify speaker roles (e.g., "anchor," "reporter," "expert") and categorise scenes into news situations (e.g., "report," "interview") in news videos, to enhance the understanding of news content. To bridge the gap of missing datasets, we present a novel annotated dataset for various speaker roles and news situations from diverse (national) media outlets. Furthermore, we suggest a rich set of features and employ aggregation and post-processing techniques. In our experiments, we compare classifiers like Random Forest and XGBoost for identifying speaker roles and news situations in video segments. Our approach outperforms recent state-of-the-art methods, including end-to-end multimodal deep network and unimodal transformer-based models. Through detailed feature combination analysis, generalisation and explainability insights, we underscore our models' capabilities and set new directions for future research.

We present the Meta-Ontology for Introspection (MOI): Inspired by fundamental processes of the human mind, cognitive architectures (CAs) explore ever more methods to leverage metacognition. Still, an ontological model to trace metacognitive experiences for learning or as input for metacognitive control routines has yet to be developed. Based on a review of existing standards, we formally identify the relevant scope in the form of Competency Questions (CQs) and extend SOMA, a well-established formal ontology initially designed to interpret episodic memories of a robotic CA. The resulting MOI can model a CA’s software and capabilities of single components, trace information processing and inter-component communication, label self-lived mental events, and capture causal relationships. We evaluate MOI via the CQs and exemplarily demonstrate its reasoning capabilities.

Abstract As the use and diversity of diagrams across many disciplines grows, there is an increasing interest in the diagrams research community concerning how such diversity might be documented and explained. In this article, we argue that one way of achieving increased reliability, coverage, and utility for a general classification of diagrams is to draw on recently developed semiotic principles developed within the field of multimodality. To this end, we sketch out the internal details of what may tentatively be termed the diagrammatic semiotic mode . This provides a natural account of how diagrammatic representations organising lines of research integrate natural language, various forms of graphics, diagrammatic elements such as arrows, lines and other expressive resources into coherent organisations, while still respecting the crucial diagrammatic contributions of visual organisation. We illustrate the proposed approach using two recent diagram corpora and show how a multimodal approach supports the empirical analysis of diagrammatic representations, especially in identifying diagrammatic constituents and describing their interrelations in a manner that may be generalised across diagram types and be used to characterise distinct kinds of functionality.

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave many pieces of information necessary for performing the task unspecified. Humans can solve such problems fast as we reduce the search space by recourse to prior knowledge such as a connected collection of plans that describe how certain goals can be achieved at various levels of abstraction. Rather than enumerating fine-grained physical contexts SOMA sets out to include socially constructed knowledge about the functions of actions to achieve a variety of goals or the roles objects can play in a given situation. As the human cognition system is capable of generalizing experiences into abstract knowledge pieces applicable to novel situations, we argue that both physical and social context need be modeled to tackle these challenges in a general manner. The central contribution of this work, therefore, lies in a comprehensive model connecting physical and social entities, that enables flexibility of executions by the robotic agents via symbolic reasoning with the model. This is, by and large, facilitated by the link between the physical and social context in SOMA where relationships are established between occurrences and generalizations of them, which has been demonstrated in several use cases in the domain of everyday activites that validate SOMA.

Functional relations such as containment or support have proven difficult to formalize. Although previous efforts have attempted this using hybrids of several theories, from mereology to temporal logic, we find that such purely symbolic approaches do not account for the embodied nature of functional relations, i.e. that they are used by embodied agents to describe fragments of a physical world. We propose a formalism that combines descriptions of a high level of abstraction with generative models that can be used to instantiate or recognize arrangements of objects and trajectories conforming to qualitative descriptions. The formalism gives an account of how a qualitative description of a scene or arrangement of objects can be converted into a quantitative description amenable to simulation, and how simulation results can be qualitatively interpreted. We use this to describe functional relations between objects in terms of spatial arrangements, expectations on behavior, and counterfactual expectations for when one of the participants is absent. Our method is able to tackle important questions facing an agent operating in the world, such as what would happen if an arrangement of objects is created and why. This gives the agent a deeper understanding of functional relations, including what role background objects, not explicitly asserted to participate in a functional relation such as containment, play in enabling or hindering the relation from holding.

The situated interpretation of natural language concerning space, spatial relationships and spatial activities is a complex problem spanning contributions from several disciplines. Space plays a central role in many theories of cognition and the spatial language observed in actual contexts of use is extremely flexible. In our work on spatial representations of all kinds, principles drawn from ontological engineering play a central role. Moreover, we have been led to augment those principles in particular ways: most specifically with respect to ontological modularity, ontological heterogeneity, and multiple ontological levels or strata. In the presentation accompanying this position statement, I give an overview of our work on space and the ontologically-relevant conclusions that we have drawn. I also suggest that these conclusions are relevant for ontological work in general, and not just for that concerned with spatial issues.

The spoken language interface is now becoming an increasingly serious research topic with application to a wide range of highly engineered systems. Such systems not only include innocuous human-computer interactions, but also encompass shared-control safety critical devices such as automotive vehicles and robotic systems. Spoken dialogue systems (SDS) are the language architecture used to provide linguistic interaction in these applications, but they have to date been notoriously difficult to engineer in a robust and safe manner. In this paper we report on our efforts to improve the safety and overall usability of dialogue enabled applications through the employment of formal methods in SDS development and testing. Specifically, we use communicating sequential processes (CSP) as the basis of a new approach to the specification, design and verification of dialogue manager control. Moreover, to support this approach, we introduce FDMSC - the Formal Dialogue Management for Shared Control toolkit - and illustrate its use in the construction of formal methods based spoken dialogue systems.

This paper presents a multilingual Natural Language Generation system that produces technical instruction texts in Bulgarian, Czech and Russian. It generates several types of texts, common for software manuals, in two styles. We illustrate the system’s functionality with examples of its input and output behaviour. We discuss the criteria and procedures adopted for evaluating the system and summarise their results. The system embodies novel approaches to providing multilingual documentation, ranging from the re-use of a large-scale, broad coverage grammar of English in order to develop the lexico-grammatical resources necessary for the generation in the three target languages, through to the adoption of a ‘knowledge editing’ approach to specifying the desired content of the texts to be generated independently of the target languages in which those texts finally appear.

Combining elements appropriately within a coherent page layout is a well-recognized and crucial aspect of sophisticated information presentation. The precise function and nature of layout has not, however, been sufficiently addressed within computational approaches; attention is often restricted to relatively local issues of typography and text-formatting, leaving broader issues of layout unaddressed. In this paper we focus on the selection and function of layout in pages that appropriately combine textual and graphical representation styles to yield coherent presentation designs. We demonstrate that layout offers a rich resource for achieving presentational coherence, alongside more traditional resources such as text-formatting and the text-internal marking of discourse connections. We also introduce an integrated approach to layout, text, and diagram generation. Our approach is developed on the basis of a preliminary empirical investigation of professionally produced layouts, followed by implementation within a prototype information system in the area of art history.

Previous algorithms for the generation of referring expressions have been developed specifically for this purpose. Here we introduce an alternative approach based on a fully generic aggregation method also motivated for other generation tasks. We argue that the alternative contributes to a more integrated and uniform approach to content determination in the context of complete noun phrase generation.

Natural language generation is now moving away from research prototypes into more practical applications. Generation functionality is also being asked to play a more significant role in established applications such as machine translation. In both cases, multilingual generation techniques have much to offer. However, the take-up of multilingual generation is being restricted by a critical lack both of large-scale linguistic resources suited to the generation task and of appropriate development environments. This paper describes KPML, a multilingual development environment that offers one possible solution to these problems. KPML aims to provide generation projects with standardized, broad-coverage, reusable resources and a basic engine for using such resources for generation. A variety of focused debugging aids ensure efficient maintenance, while supporting multilingual work such as contrastive language development and automatic merging of independently developed resources. KPML is based on a new, generic approach to multilinguality in resource description that extends significantly beyond previous approaches. The system has already been used in a number of large generation projects and is freely available to the generation community.

The inquiry semantics approach of the Nigel computational systemic grammar of English has proved capable of revealing distinctions within propositional content that the text planning process needs to control in order for adequate text to be generated. An extension to the chooser and inquiry framework motivated by a Japanese clause generator capable of expressing levels of politeness makes this facility available for revealing the distinctions necessary among interpersonal, social meanings also. This paper shows why the previous inquiry framework was incapable of the kind of semantic control Japanese politeness requires and how the implemented extension achieves that control. An example is given of the generation of a sentence that is appropriately polite for its context of use and some implications for future work are suggested.