Recognizing blended emotions in audiovisual data is important for various applications such as targeted advertising, social media analysis, and deepfake detection, yet many estimation models still assume that each clip expresses a single dominant emotion. The Blended Emotion Recognition Challenge (BlEmoRe) addresses this limitation with a multimodal benchmark where systems must predict both the presence of one or two emotions and their relative salience. In this paper, we present CANDOR, a set of innovations that address three structural obstacles often apparent in conventional estimation pipelines taking advantage of a fusion of pre-trained features: (i) pre-trained features encode actor identity far more strongly than emotion content, (ii) feature-level concatenation reduces per-model discriminative power, and (iii) the training data exhibits a regular per-actor class distribution that can serve as an inductive bias. Therefore, CANDOR introduces identity normalization via unsupervised centroid subtraction, decision-level fusion that preserves each model’s discriminative geometry, a KL distributional regularizer informed by the observed per-actor class distribution, and test-time distributional optimization that refines predictions via gradient descent. Additionally, using the pre-trained features, we investigate how different fusion strategies, such as attention mechanisms, gated methods, and sparse mixture-of-experts architectures, impact classification performance. We then create an ensemble system by combining the top-performing models using fold-corrected weighted voting for emotion presence detection and confidencegated aggregation for salience estimation; this fused ensemble serves as an additional baseline. On the test set of the BlEmoRe challenge, CANDOR achieves first place with an average score of 0.457 (ACCpresence = 0.641, ACCsalience = 0.272), improving over the fusion ensemble by 67% which achieves a test score of 0.274.

The rise of generative AI has enabled the creation of synthetic audio, images, and videos that are virtually indistinguishable from authentic media, presenting new threats to digital trust, privacy, and security. While deepfake detection has advanced, most solutions focus on binary classification performed by data-driven approaches, which are insufficient for attribution and explainability required in high-stakes scenarios. This dissertation aims to develop robust, generalizable, and explainable forensic frameworks that (1) not only detect AI-generated media but also attribute attacks to specific models or methods, (2) provide interpretable evidence for forensic and legal contexts, and (3) are resilient to adversarial manipulations. Our approach integrates data-driven and model-based techniques, leverages external data, and builds on extensive prior work in multimedia forensics. In this paper, we present key challenges, objectives and early findings that support the advancement of trustworthy AI and strengthen digital media security.

The rapid development of generative AI and in particular deepfake technology enables the seamless creation and manipulation of visual content. As the resulting syntheses are often indistinguishable from authentic images, they threaten the integrity of visual evidence. While forensic detectors can be used to detect syntheses, they can become targets of adversarial attacks. In the ''Adversarial Attacks on Deepfake Detectors'' challenge, competitors were tasked with perturbing a dataset of AI-synthesized images so that four classifiers would mistakenly accept them as authentic. In this paper, we introduce our solution, a white-box adversarial framework that injects globally distributed, data-driven noise perturbations optimized via additional surrogate Vision Transformer and EfficientNet classifiers. Empirical comparisons to both conventional post-processing transforms and localized adversarial patches demonstrate that our approach based on globally distributed noise achieves the highest attack success rates across all public detectors while preserving superior SSIM, confirming its efficacy and visual imperceptibility. In the final evaluation of the challenge, our proposed approach placed third with a final score of 2679.

Neural networks have become essential to modern applications, excelling in various tasks such as image recognition, language translation, and predictive analytics. In security, they are, among other things, widely used as part of identity verification that often combines automatic recognition with human verification. However, automatized methods are facing challenges from adversarial attacks, where malicious modifications to an input can deceive networks, thus compromising their reliability. With defenses evolving to detect and reverse such attempts, as well as attacked samples not passing manual verification by a human, it raises the question, whether deepfakes such as face swapping and facial reenactment can serve as a new kind of adversarial attacks. In this paper, we explore if deepfakes can deceive neural networks and humans, by analyzing state-of-the-art methods and introducing a novel one-shot face swapping technique that blends reenactment and swapping for high-quality results and improved attack success rates of up to 11% in comparison to current state-of-the-art face swapping techniques.

Detecting image forgeries in JPEG encoded images has been a research topic in the field of media forensics for a long time. Until today, it still holds a high importance as tools to create convincing manipulations of images have become more and more accessible to the public, which in return might be used to e.g. generate fake news. In this paper, a passive forensic detection framework to detect image manipulations is proposed based on compression artefacts and Benfords First Digit Law. It incorporates a supervised approach to reconstruct the compression history as well as provides an un-supervised detection approach to detect double compression for unknown quantization tables. The implemented algorithms were able to achieve high AUC values when classifying high quality images exceeding similar state-of-the-art methods.