Determining whether multimedia content related to a specific retrieval topic or a developing news story contains novel information is a crucial task in media monitoring and production. While visual content gains importance, most of the existing work focuses on text documents. We propose a method for determining novel visual information in videos, embedded in a workflow using multimodal topic threading and video to text, and compare two approaches to measure novelty. We present a web application for determining the novelty in multimedia content being ingested into a collection, and evaluate the approach on a subset of the FIVR-200K dataset.

A web-based application for incremental training of landmark recognition, suitable for domain experts without machine learning expertise is presented. Its backend uses the fine-grained image classification network API-Net in a few-shot setting, making use of the two-stage fine-tuning paradigm. The aim is to enable rapid training of a classifier for recognizing new landmarks in video content, supporting needs in media production. Using base models trained on two different datasets, we demonstrate the speed and effectiveness of the application in training the networks to detect new landmarks. In addition, our application provides the ability to retrain a previously learned landmark with further data, e.g. to improve performance for views or imaging conditions not well supported by the model. This feature ensures that the model remains up-to-date with evolving datasets and environmental conditions, thereby improving its accuracy and adaptability over time.

In this paper we propose ÖWF-OD, a new dataset for object detection in archival film content. The dataset enables the evaluation of object detection methods on image data with very different image qualities. 1,000 selected keyframes from 100 hours of video material have been annotated, 4,480 bounding boxes are labeled. In addition to these annotations, image quality measures were calculated for the keyframes in order to assess the influence of image quality on object detection. We evaluate different versions of the YOLO object detector to provide a baseline of object detection results for this dataset. The annotation data and the code to extract the keyframes are provided at https://github.com/TailoredMediaProject/OEWF_ObjectDetection.

Film and video archives hold vast collections of historically relevant content, and the (co-)appearance of specific persons is often one of the most interesting properties. However, while archive documentation does not always contain metadata which person is depicted in the content (and if so, where), it is often desired to obtain a list of persons related to the entire broadcast. We propose to obtain a set of sample images from web searches, and then perform a face verification task using an AI-based method to determine whether (and where) a person appears in the content. We analyse the influence of different annotation conditions and types of errors in automated face sample retrieval on the end-to-end performance of the face verification process. To give the user insights into the success or failure of the verification, we apply a recent explainable face verification method. Thereby we investigate how the explainability performs on a completely different dataset.

The importance of automated intrusion detection systems, not only in network infrastructures, but also in critical and industrial infrastructures is becoming more evident with the significant increase of cyber-attacks targeting such infrastructures. The most recent research initiatives in this field focus on unsupervised learning methods, due to a constant lack of labelled datasets of a good quality. This paper proposes an enhanced autoencoder based anomaly detection approach for water distribution cyber-attack detection. The proposed approach contains a pipeline of methods, including feature engineering as a pre-processing step, anomaly estimation based on autoencoder, and scores smoothing as a post-processing step. The obtained results are very promising compared to existing approaches.

We demonstrate a toolchain for visualising detailed road traffic data from multimodal sensors consisting of (i) the automatic, real-time extraction of movement paths of road users and noteworthy events from traffic monitoring cameras or LIDAR sensors, (ii) extraction of audio events from individual microphones and microphone arrays, (iii) a spatial data managment system storing the extracted information together with a geographic information system (GIS), and (iv) a web based viewer allowing to interactively visualise all these data in the context of a high-definition digital twin of the traffic environment. This system enables the collection of a considerable amount of objective data on road use and can be used for planning changes to traffic facilities as well as for assessing changes in the traffic environment.

In this demonstration, we present the Annotation Tool that is being developed in the porTiVity project to annotate video objects for Interactive Television programs. This tool includes various video processing components to structure and speed-up the annotation process, such as shot segmentation, key-frame extraction, object tracking and object redetection. A specific feature is that the tool includes a preprocessing phase where a quantity of information is precomputed, so that the annotation itself can be done quite rapidly.

The GMF4iTV project (Generic Media Framework for Interactive Television) is an IST European project that developed an end-to-end broadcasting platform providing interactivity on heterogeneous multimedia devices such as Set-Top-Boxes, PCs and PDAs according to the Multimedia Home Platform (MHP) part of the DVB standard. The developed platform allows the content providers to create enhanced audiovisual contents with a degree of interactivity at moving object level or shot changes in a video. The end user is then able to interact with moving objects from the video or individual shots allowing the enjoyment of additional contents associated to them (MHP applications, HTML pages, JPEG, MPEG-4 files,...).

Abstract Automatic generation of play lists for commercial broadcast radio stations has become a major research topic. Audio identification systems have been around for a while, and they show good performance for clean audio files. However, songs transmitted by commercial radio stations are highly distorted to cause greater impact on the casual listener. This impact helps increase the probability that the listener will stay tuned in, but the price we have to pay is a severe modification in the audio itself. This causes the failure oftraditionalidentification systems. Another problem is the fact that songs are never played from the beginning to the end. Actually, they are put on the air several seconds after their real beginning and almost always under the voice of a speaker. The same thing happens at the end. In this article, we present the RAA project, which was conceived to deal with real broadcast audio problems. The idea behind this project is to extract automatically an audio fingerprint (the so‐called AudioDNA) that identifies the fragment of audio. This AudioDNA has to be robust enough to appear almost the same under several degrees of distortion. Once this AudioDNA is extracted from the broadcast audio, a matching algorithm is able to find its fragments inside a database. With this approach, the system can find not only a whole song but also small fragments of it, even with high distortion caused by broadcast (and DJ) manipulations.