Video assets from archives or online platforms can provide relevant content for embedding into immersive scenes or for generation of 3D objects or scenes. However, XR content creators lack tools to find relevant video segments for their chosen topic. In this paper, we explore the use case of journalists creating immersive experiences for news stories and their need to find related video material to create and populate a 3D scene. An innovative approach creates text and video embeddings and matches textual input queries to relevant video shots. This is provided via a Web dashboard for search and retrieval across video collections, with selected shots forming the input to content creation tools to generate and populate an immersive scene, meaning journalists do not need specialist knowledge to communicate stories via XR.

This demonstration will show how video shot discovery through joint text-video embedding has been integrated into a journalistic workflow through a news monitoring dashboard with the purpose of identifying suitable video material for the creation of immersive scenes around a chosen news story or topic.

This paper presents an updated iteration of the VERGE interactive video retrieval system. It offers various search options like free text and concept-based text search, color similarity, people and face detection, and visual and semantic similarity search. The system is designed to handle large amounts of data efficiently using advanced indexing techniques and state-of-the-art AI technology for visual content analysis. This paper describes enhancements made to improve usability for non-expert users, particularly through changes to the search and browsing interface.

Abstract To optimise digital content marketing for broadcasters, the Horizon 2020 funded ReTV project developed an end-to-end process termed “Trans-Vector Publishing” and made it accessible through a Web-based tool termed “Content Wizard”. This paper presents this tool with a focus on each of the innovations in data and AI-driven media analysis to address each key step in the digital content marketing workflow: topic selection, content search and video summarisation. First, we use predictive analytics over online data to identify topics the target audience will give the most attention to at a future time. Second, we use neural networks and embeddings to find the video asset closest in content to the identified topic. Third, we use a GAN to create an optimally summarised form of that video for publication, e.g. on social networks. The result is a new and innovative digital content marketing workflow which meets the needs of media organisations in this age of interactive online media where content is transient, malleable and ubiquitous.

In order to optimise the distribution of video assets online, media organizations need tailor their offerings for specific digital channels and better understand the interests of their audiences at particular points in time, which are often influenced by contemporary new stories and trends on social media. For this purpose, the research project ReTV has developed a Web-based tool termed ’Content Wizard’ which demonstrates an end-to-end, semi-automated workflow for video content creation, adaptation and distribution across digital channels. Digital assets can be selected based on predicted future trending topics, re-purposed according to the different digital channels they will be published upon and scheduled for the optimal future publication date. The result is an innovative video publication workflow that meets the marketing needs of media organisations in this age of transient online media spread across multiple channels.

In this paper, the problem of unlabeled video retrieval using textual queries is addressed. We present an extended dual encoding network which makes use of more than one encodings of the visual and textual content, as well as two different attention mechanisms. The latter serve the purpose of highlighting temporal locations in every modality that can contribute more to effective retrieval. The different encodings of the visual and textual inputs, along with early/late fusion strategies, are examined for further improving performance. Experimental evaluations and comparisons with state-of-the-art methods document the merit of the proposed network.

Zero-example event detection is a problem where, given an event query as input but no example videos for training a detector, the system retrieves the most closely related videos. In this paper we present a fully-automatic zero-example event detection method that is based on translating the event description to a predefined set of concepts for which previously trained visual concept detectors are available. We adopt the use of Concept Language Models (CLMs), which is a method of augmenting semantic concept definition, and we propose a new concept-selection method for deciding on the appropriate number of the concepts needed to describe an event query. The proposed system achieves state-of-the-art performance in automatic zero-example event detection.

This paper presents a fully-automatic method that combines video concept detection and textual query analysis in order to solve the problem of ad-hoc video search. We present a set of NLP steps that cleverly analyse different parts of the query in order to convert it to related semantic concepts, we propose a new method for transforming concept-based keyframe and query representations into a common semantic embedding space, and we show that our proposed combination of concept-based representations with their corresponding semantic embeddings results to improved video search accuracy. Our experiments in the TRECVID AVS 2016 and the Video Search 2008 datasets show the effectiveness of the proposed method compared to other similar approaches.