The rapid development of image manipulation technologies poses significant challenges to multimedia forensics, especially in accurate localization of manipulated regions. Existing methods often fail to fully explore the intrinsic discrepancies between manipulated and authentic regions, resulting in sub-optimal performance. To address this limitation, we propose the Focus Region Discrepancy Network (FRD-Net), a novel and efficient framework that significantly enhances manipulation localization by amplifying discrepancies at both macro- and micro-levels. Specifically, our proposed Iterative Clustering Module (ICM) groups features into two discriminative clusters and refines representations via backward propagation from cluster centers, improving the distinction between tampered and authentic regions at the macro level. Thereafter, our Differential Progressive Module (DPM) is constructed to capture fine-grained structural inconsistencies within local neighborhoods and integrate them into a Central Difference Convolution, increasing sensitivity to subtle manipulation details at the micro level. Finally, these complementary modules are seamlessly integrated into a compact architecture that achieves a favorable balance between accuracy and efficiency. Extensive experiments on multiple benchmarks demonstrate that FRD-Net consistently surpasses state-of-the-art methods in terms of manipulation localization performance while maintaining a lower computational cost.

With the popularity of image editing techniques, synthetic images may have inharmonious regions due to color/illumination differences between the manipulated area and the background. The inharmonious region localization task aims to find these regions, which is crucial for blind image harmonization. Existing methods rely on single-view images and do not fully explore multi-scale fusion, which limits their performance. To address these issues, in this paper, we propose a novel multi-view perception and shrinkage aggregation network (MSANet) for the inharmonious region localization task that fully utilizes multi-view images and multi-scale fusion information and can mine subtle cues between candidate objects and the background. Specifically, we first design a multi-view ensemble encoder to fully perceive the inharmonious regions by multi-view interactive learning and then aggregate the feature representations of inharmonious regions. Moreover, we propose a multi-scale shrinkage fusion decoder, where multi-scale features with multi-view prior information are utilized to aggregate adjacent features, adaptively select high-quality information, reduce background interference and gradually locate inharmonious regions. Extensive experimental results on four public datasets (HDobe5K, HCOCO, HFlickr, and Hday2Night) demonstrate that the proposed MSANet can outperform all the SOTA methods in terms of average F1 and average IoU score, while maintaining a lower computational cost1.

Knowledge distillation transfers knowledge from the teacher model to the student one, significantly enhancing the capabilities of the student network. However, alleviating the information gap between the corresponding stages of the student and teacher during the distillation process poses a challenge. This challenge is particularly noticeable at deeper levels, where the limited capability of the student may result in capturing less information, thus leading to poor learning performance. To overcome this limitation, we introduce a novel multi-stage local feature distillation method, which leverages fused multiple feature maps named tutor to bridge the gap. Additionally, we have designed the Value Attention-Based Fusion module (Value-ABF) to enhance feature fusion reasonably. Compared with other distillation methods, our approach achieves comparable or even superior results and demonstrates better training efficiency on CIFAR-100 and COCO2017 datasets for tasks such as image classification, object detection, and instance segmentation.

Image manipulation localization is of vital importance to public order protection. One dominant approach is to detect the anomalies in images, i.e., visual artifacts, as the tampered edge clue for aiding manipulation prediction. Nevertheless, we argue that these methods struggle with the modeling of spatial inconsistency within multi-scale, resulting in sub-optimal model performance. To overcome this problem, in this paper, we propose a novel end-to-end method to identify the multi-scale spatial inconsistency for image manipulation localization (abbreviated as MSI) where the multi-scale edge-guided attention stream (MEA) and multi-scale context-aware search stream (MCS) are jointly explored in a unified framework, moreover, multi-scale information is efficiently used. In the former, the edge-attention module is designed to precisely locate the tampered regions based upon multi-scale edge boundary features. In the latter, the context-aware search module is designed to model spatial contextual information within multiple scales. To validate the effectiveness of the proposed method, we conduct extensive experiments on six image manipulation localization datasets including NIST-2016, Columbia, CASIA1.0, COVER, DEF-12K, and IMD2020. The experimental results demonstrate that our proposed method can outperform state-of-the-art methods by a significant margin in terms of average F1 score while maintaining robustness with respect to various attacks. Compared with MVSS-Net (Published in ICCV 2021) on the NIST-2016, CASIA1.0, DEF-12K, and IMD2020 datasets, the improvements in F1 score can reach 6.7%, 9.5%, 5.4%, and 8.4%, respectively.

Learning manipulation skills from observing human demonstration videos is a promising aspect for intelligent robotic systems. Recent advances in video to command provide an end-to-end approach to translate a video into robot plans. However, the general video captioning methods focus more on the understanding of the full frame, while they lack the consideration of the spatio-temporal features in videos. In this paper, we proposed the two-stream 2D/3D residual networks for robots to learn manipulation tasks from human demonstration videos. We integrate spatial features with 2D residual network and temporal features with 3D residual network as inputs for RNN layers. An encoder-decoder architecture is then used to encode the spatio-temporal features and sequentially generate the command words. Experimental results on an extended manipulation dataset show that our approach outperforms the state-of-the-art methods. Real-world experiments results on a Baxter robotic arm indicate that our method could produce more accurate commands from video demonstrations.