In this paper, we present a novel difference-spotting strategy for video action recognition inspired by the cognitive challenges posed by the childhood puzzle game "Spot the Difference". Our approach aims to enhance the model’s capability to capture time-series variation and intricate details by gradually integrating distinctive information between action and non-action segments in a temporal "coarse-to-fine-to-finer" manner within a discriminative learning framework. To achieve this, we propose a model-agnostic discriminative learning mechanism that can be easily integrated into existing action recognition networks. Firstly, we incorporate coarse-level discriminative information of action and non-action segments across all videos in a corpus using novel booster nets. Secondly, we introduce a fine-level discrimination objective in the penultimate layer of the network through a novel contrastive learning approach, increasing the distinction between different segments within the same video. Lastly, we incorporate finer discrimination through a novel clip matching mechanism, enhancing the distinction of different consecutive clips within an action segment. Experimental results on multiple benchmark datasets (ActivityNet, HACS, FineAction) and backbone architectures (TSN, TSM, TANet, TPN, Timesformer, VideoSwin) demonstrate the effectiveness of our proposed mechanism. We consistently achieve significant improvements (0.33 - 4%) over the baselines, with competitive single crop results on ActivityNet (87.9%) and HACS (90.21%) datasets. Moreover, our technique achieves stateof-the-art classifier results (94.8%) in the ActivityNet 2022 challenge’s validation set.

Recent advances in generative modeling with diffusion processes (DPs) enabled breakthroughs in image synthesis. Despite impressive image quality, these models have various prompt compliance problems, including low recall in generating multiple objects, difficulty in generating text in images, and meeting constraints like object locations and pose. For fine-grained editing and manipulation, they also require fine-grained semantic or instance maps that are tedious to produce manually. While prompt compliance can be enhanced by addition of loss functions at inference, this is time consuming and does not scale to complex scenes. To overcome these limitations, this work introduces a new family of $\textit{Factor Graph Diffusion Models}$ (FG-DMs) that models the joint distribution of images and conditioning variables, such as semantic, sketch, depth or normal maps via a factor graph decomposition. This joint structure has several advantages, including support for efficient sampling based prompt compliance schemes, which produce images of high object recall, semi-automated fine-grained editing, explainability at intermediate levels, ability to produce labeled datasets for the training of downstream models such as segmentation or depth, training with missing data, and continual learning where new conditioning variables can be added with minimal or no modifications to the existing structure. We propose an implementation of FG-DMs by adapting a pre-trained Stable Diffusion (SD) model to implement all FG-DM factors, using only COCO dataset, and show that it is effective in generating images with 15\% higher recall than SD while retaining its generalization ability. We introduce an attention distillation loss that encourages consistency among the attention maps of all factors, improving the fidelity of the generated conditions and image. We also show that training FG-DMs from scratch on MM-CelebA-HQ, Cityscapes, ADE20K, and COCO produce images of high quality (FID) and diversity (LPIPS).