The availability of datasets annotated with verified events by the public is a necessary prerequisite for unleashing the potential of multimodal deep learning for news event detection. Publicly available datasets are either incompletely annotated due to expensive cost, or ignore the verifiability of event labels, which are susceptible to bias and errors introduced by a limited number of annotators. In this article, we provide a YouTube dataset labelled by real-world news events that can be verified by Wikipedia-like crowd sourcing platforms, with the target of advancing temporal event detection. The events in our dataset cover a wide range of event topics including public security, natural disasters, elections, sports, and entertainment events, etc. In the dataset, each sample is labelled with real-world event that is verifiable by the public. We extensively evaluate the performance of 13 state-of-the-art algorithms on our dataset in a temporal manner, involving the multiple relationships between training and testing event labels, and provide a thorough analysis of the findings.

In this paper, we present a new dataset with the target of advancing temporal real-world event detection from video-sharing social media. The collected temporal event dataset (TED) has certain characteristics. 1) Certainty of event labels that can be recognized as real-world happening. 2) Wide range of event categories covering public security, natural disasters, elections, sports and entertainment events, etc. 3) High overlap among event topics ensuring the difficulties in distinguishing the labels. 4) Multiple data modalities involving textual, acoustic, and visual information, etc. More specifically, two scenarios are defined based on the close or open domains, i.e., temporal event detection without/with new events, denoted as TED-W and TED-N, respectively. For comparisons, a few benchmarks are investigated on the two scenarios with the dataset.