Knowledge about the presence of people in a video is a valuable source of information in many applications, such as video annotation, retrieval and summarisation. The contribution of this paper goes in the direction of demonstrating how AI-based face processing technologies can be profitably used to perform video annotation of television content. To validate our vision, we developed the Face Management Framework (FMF), which implements an end-to-end pipeline for face analysis and content annotation based on few-shot or zero-shot face embedding extraction models. The results of the test campaign of the system show that the key performance indicators that we defined were exceeded by a wide margin, demonstrating how media workflows could greatly benefit from the tool and the efficiency improvements it brings.

Timeline summarization aims at presenting long news stories in a compact manner. State-of-the-art approaches first select the most relevant dates from the original event timeline then produce per-date news summaries. Date selection is driven by either per-date news content or date-level references. When coping with complex event data, characterized by inherent news flow redundancy, this pipeline may encounter relevant issues in both date selection and summarization due to a limited use of news content in date selection and no use of high-level temporal references (e.g., the past month). This paper proposes a paradigm shift in timeline summarization aimed at overcoming the above issues. It presents a new approach, namely Summarize Date First, which focuses on first generating date-level summaries then selecting the most relevant dates on top of summarized knowledge. In the latter stage, it performs date aggregations to consider high-level temporal references as well. The proposed pipeline also supports frequent incremental timeline updates more efficiently than previous approaches. We tested our unsupervised approach both on existing benchmark datasets and on a newly proposed benchmark dataset describing the COVID-19 news timeline. The achieved results were superior to state-of-the-art unsupervised methods and competitive against supervised ones.

In this work we exploit a recently introduced nonmonotonic extension of Description Logics, able to deal with the problem of knowledge invention via commonsense concept combination, to dynamically generate novel editorial contents in the context of a real broadcasting company: RAI – Radiotelevisione Italiana, the Italian public broadcaster. In particular, we introduce the system implementing such logic, i.e. DENOTER: Dynamic gEnerator of NOvel contents in mulTimEdia bRoadcasting (available online at the URL: http://di.unito.it/denoter), that has been applied and tested in the online multimedia platform of RAI (i.e. RaiPlay) as a tool for both the generation/suggestion of novel genres of multimedia on-demand contents and the reclassification of the available items within such new genres. Our system works by extracting the typical properties characterizing the available genres (with a standard information extraction pipeline) and by building novel classes of genres as the result of a creative combination of such extracted representations. We have tested DENOTER (i) by reclassifying the available contents in RaiPlay with respect to the new generated genres (ii) with an evaluation, in the form of a controlled user study experiment, of the feasibility of using the obtained reclassifications as recommended contents (iii) with a qualitative evaluation done with a small group of experts of RAI. The obtained results are encouraging and pave the way to many possible further improvements and research directions.

Finding good representations for videos is becoming increasingly more important to enable an efficient analysis and comparison, with potential applications in sports, surveillance, news, or web services. This paper proposes a new representation of videos based on human pose. Rather than looking at conventional features, our method relies only on human pose detections to characterize the video. This approach provides a powerful tool for the efficient analysis of videos of human activities, particularly for video summarization and retrieval. We evaluate the proposed representation on the following tasks: 1) computing video statistics, such as the main poses and viewpoint preferences; 2) partitioning videos into a collection of short clips that will compose the video summary; and 3) retrieving frames or scenes with specific poses from videos. Results show that the proposed approach is able to successfully perform these tasks.

This paper describes AgileRAI, a framework for searching, organising and accessing multimedia data in a fast and semantic-driven way. AgileRAI supports real time ingestion of video streams on which different machine learning techniques (such as global and local visual features extraction and matching) are applied in a parallel and scalable way. Extracted features are matched to a reference database of visual patterns (e.g. buildings, paintings, monuments) in order to produce a set of meta-tags describing the ingested contents. Furthermore, these tags are semantically enriched using open semantic data repositories. The system is designed with a scale-out pattern architecture based on Apache Spark, ensuring high-performance in Big Data management environments.

This paper presents a novel video access and retrieval system for edited videos. The key element of the proposal is that videos are automatically decomposed into semantically coherent parts (called scenes) to provide a more manageable unit for browsing, tagging and searching. The system features an automatic annotation pipeline, with which videos are tagged by exploiting both the transcript and the video itself. Scenes can also be retrieved with textual queries; the best thumbnail for a query is selected according to both semantics and aesthetics criteria.

This paper describes a content set that has been made available by the European Broadcasting Union (EBU). The content in the set consists of broadcast media content collected from different broadcasters around the world. This content set is made available to the research community in order to evaluate automatic information extraction tools on this broadcast media. The set also contains ground truth data and annotations for several automatic information extraction tasks.

This demo paper describes a system for contextual advertising on aggregations of multimodal news items. The prototype is intended to demonstrate how modern content analysis techniques can be profitably used to automate tasks commonly performed by humans such as the planning of the computer-assisted advertising content.

Automatic quality control for audiovisual media is an important tool in the media production process. In this paper we present tools for assessing the quality of audiovisual content in order to decide about the reusability of archive content. We first discuss automatic detectors for the common impairments noise and grain, video breakups, sharpness, image dynamics and blocking. For the efficient viewing and verification of the automatic results by an operator, three approaches for user interfaces are presented. Finally, we discuss the integration of the tools into a service oriented architecture, focusing on the recent standardization efforts by EBU and AMWA's Joint Task Force on a Framework for Interoperability of Media Services in TV Production (FIMS).

The emergence of cloud environments has made feasible the delivery of Internet-scale services by addressing a number of challenges such as live migration, fault tolerance and quality of service. However, current approaches do not tackle key issues related to cloud storage, which are of increasing importance given the enormous amount of data being produced in today's rich digital environment (e.g. by smart phones, social networks, sensors, user generated content). In this paper we present the architecture of a scalable and flexible cloud environment addressing the challenge of providing data-intensive storage cloud services through raising the abstraction level of storage, enabling data mobility across providers, allowing computational and content-centric access to storage and deploying new data-oriented mechanisms for QoS and security guarantees. We also demonstrate the added value and effectiveness of the proposed architecture through two real-life application scenarios from the healthcare and media domains.

Despite the number of efforts, managing content for media production and distribution can be still considered a time-consuming and hard task. From here derives the need for effective data documentation and retrieval systems. Hyper Media News provides a set of integrated tools for large-scale acquisition, analysis, indexing, browsing and recommendation of news content from both television and the Web.

The ACM AIEMPro 2011 workshop presents research on automated media content analysis and production for, amongst others, the development of novel TV services. The program of the workshop has two sessions. The first one is composed of three papers on video and TV content structuring and indexing. The second session is also composed of three papers on media production and retrieval systems and applications.

The third Workshop on Automated Information Extraction in Media Production (AIEMPro10) aims at fostering exchange of ideas and of practices between leading experts in research and leading actors in the media community, in order to catalyze the migration towards new ways of producing media content, aided by large scale introduction of tools for automated multimedia analysis and understanding. Furthermore, the workshop helps researchers in better understanding what are some real-life key requirements which would enable their scientific developments come into wider adoption.

Current Web technology has enabled the distribution of informative content through dynamic media platforms.In addition, the availability of the same content in the form of digital multimedia data has dramatically increased.Content-based, cross-media retrieval applications are needed to efficiently access desired information from this variety of data sources. This paper presents a novel approach for cross-media information aggregation, and describes a prototype system implementing this approach. The prototype adopts online newspaper articles and TV newscasts as information sources, to deliver a service made up of items including both contributions. Extensive experiments prove the effectiveness of the proposed approach in a real-world business context.

The demonstration will be about an integrated sytem for the acquisition, automatic annotation and web publication of television broadcast news programmes named ANTS (Automatic Newscast Transcription System). The system consists of several analysis components integrated in a unified architecture. Users have the possibility to access a large daily-growing database of news stories from the main national channels, all identified, categorised and published in a fully automatic way. The system identifies story boundaries, extracts texts from spoken content, classifies stories by subject and links external relevant information coming from the web. ANTS integrates several heterogeneous analysis components in a performant and ready-for-production environment, capable of elaborating many hours of material per day without significant service drops and with sufficiently good accuracy for industrial employment.

The quantity of multimedia content available either through the traditional distribution channels, such as broadcast television, or through new platforms like the Internet, is continuously increasing. Tools for user-oriented access to desired information are thus needed. In this paper, we illustrate a novel fuzzy multimedia mining technique for genre characterisation, aimed at overcoming limitations of conventional crisp classification systems. Effective results in genre classification and characterisation are presented based on the extraction of structural and cognitive properties of the content.