Ashim Gupta, Carter Blum, Temma Choji, Yingjie Fei, Shalin Shah, Alakananda Vempala, Vivek Srikumar. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Vivek Gupta, Shuo Zhang, Alakananda Vempala, Yujie He, Temma Choji, Vivek Srikumar. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

Abstract This article presents a two-step methodology to annotate temporally anchored spatial knowledge on top of OntoNotes. We first generate potential knowledge using semantic roles or syntactic dependencies and then crowdsource annotations to validate the potential knowledge. The resulting annotations indicate how long entities are or are not located somewhere and temporally anchor this spatial information. We present an in-depth corpus analysis comparing the spatial knowledge generated by manipulating roles or dependencies. Experiments show that working with syntactic dependencies instead of semantic roles allows us to generate more potential entity-related spatial knowledge and obtain better results in a realistic scenario, that is, with predicted linguistic information.

This article presents a corpus and experiments to extract spatial timelines from biographies. Spatial timelines capture where someone is and is not located, and specify when this spatial information is true. We work with 100 Wikipedia biographies, and consider intersentential (location, year) pairs as well as years that are not explicitly stated in the biographies. Experimental results show that a combination of LSTMs outperforms SVM with linguistically motivated features.

Text in social media posts is frequently accompanied by images in order to provide content, supply context, or to express feelings.This paper studies how the meaning of the entire tweet is composed through the relationship between its textual content and its image.We build and release a data set of image tweets annotated with four classes which express whether the text or the image provides additional information to the other modality.We show that by combining the text and image information, we can build a machine learning approach that accurately distinguishes between the relationship types.Further, we derive insights into how these relationships are materialized through text and image content analysis and how they are impacted by user demographic traits.These methods can be used in several downstream applications including pre-training image tagging models, collecting distantly supervised data for image captioning, and can be directly used in end-user applications to optimize screen estate.

This paper presents a framework to infer spatial knowledge from semantic role representations. We infer whether entities are or are not located somewhere, and temporally anchor this spatial information. A large crowdsourcing effort on top of OntoNotes shows that these temporally-anchored spatial inferences are ubiquitous and intuitive to humans. Experimental results show that inferences can be performed automatically and semantic features bring significant improvement.

This paper complements semantic role representations with spatial knowledge beyond indicating plain locations.Namely, we extract where entities are (and are not) located, and for how long (seconds, hours, days, etc.).Crowdsourced annotations show that this additional knowledge is intuitive to humans and can be annotated by non-experts.Experimental results show that the task can be automated.

This paper presents a framework to infer spatial knowledge from verbal semantic role representations.First, we generate potential spatial knowledge deterministically.Second, we determine whether it can be inferred and a degree of certainty.Inferences capture that something is located or is not located somewhere, and temporally anchor this information.An annotation effort shows that inferences are ubiquitous and intuitive to humans.