Image manipulation localization (IML) faces a fundamental trade-off between minimizing annotation cost and achieving fine-grained localization accuracy. Existing fully-supervised IML methods depend heavily on dense pixel-level mask annotations, which limits scalability to large datasets or real-world deployment. In contrast, the majority of existing weakly-supervised IML approaches are based on image-level labels, which greatly reduce annotation effort but typically lack precise spatial localization. To address this dilemma, we propose BoxPromptIML, a novel weakly-supervised IML framework that effectively balances annotation cost and localization performance. Specifically, we propose a coarse region annotation strategy, which can generate relatively accurate manipulation masks at lower cost. To improve model efficiency and facilitate deployment, we further design an efficient lightweight student model, which learns to perform fine-grained localization through knowledge distillation from a fixed teacher model based on the Segment Anything Model (SAM). Moreover, inspired by the human subconscious memory mechanism, our feature fusion module employs a dual-guidance strategy that actively contextualizes recalled prototypical patterns with real-time observational cues derived from the input. Instead of passive feature extraction, this strategy enables a dynamic process of knowledge recollection, where long-term memory is adapted to the specific context of the current image, significantly enhancing localization accuracy and robustness. Extensive experiments across both in-distribution and out-of-distribution datasets show that BoxPromptIML outperforms or rivals fully-supervised models, while maintaining strong generalization, low annotation cost, and efficient deployment characteristics.

With the rapid evolution of deepfake technologies and the wide dissemination of digital media, personal privacy is facing increasingly serious security threats. Deepfake proactive forensics, which involves embedding imperceptible watermarks to enable reliable source tracking, serves as a crucial defense against these threats. Although existing methods show strong forensic ability, they rely on an idealized assumption of single watermark embedding, which proves impractical in real-world scenarios. In this paper, we formally define and demonstrate the existence of Multi-Embedding Attacks (MEA) for the first time. When a previously protected image undergoes additional rounds of watermark embedding, the original forensic watermark can be destroyed or removed, rendering the entire proactive forensic mechanism ineffective. To address this vulnerability, we propose a general training paradigm named Adversarial Interference Simulation (AIS). Rather than modifying the network architecture, AIS explicitly simulates MEA scenarios during fine-tuning and introduces a resilience-driven loss function to enforce the learning of sparse and stable watermark representations. Our method enables the model to maintain the ability to extract the original watermark correctly even after a second embedding. Extensive experiments demonstrate that our plug-and-play AIS training paradigm significantly enhances the robustness of various existing methods against MEA.

Deep learning-based image manipulation localization (IML) methods have achieved remarkable performance in recent years, but typically rely on large-scale pixel-level annotated datasets. To address the challenge of acquiring high-quality annotations, some recent weakly supervised methods utilize image-level labels to segment manipulated regions. However, the performance is still limited due to insufficient supervision signals. In this study, we explore a form of weak supervision that improves the annotation efficiency and detection performance, namely scribble annotation supervision. We re-annotated mainstream IML datasets with scribble labels and propose the first scribble-based IML (Sc-IML) dataset. Additionally, we propose the first scribble-based weakly supervised IML framework. Specifically, we employ self-supervised training with a structural consistency loss to encourage the model to produce consistent predictions under multi-scale and augmented inputs. In addition, we propose a prior-aware feature modulation module (PFMM) that adaptively integrates prior information from both manipulated and authentic regions for dynamic feature adjustment, further enhancing feature discriminability and prediction consistency in complex scenes. We also propose a gated adaptive fusion module (GAFM) that utilizes gating mechanisms to regulate information flow during feature fusion, guiding the model toward emphasizing potential tampered regions. Finally, we propose a confidence-aware entropy minimization loss. This loss dynamically regularizes predictions in weakly annotated or unlabeled regions based on model uncertainty, effectively suppressing unreliable predictions. Experimental results show that our method outperforms existing fully supervised approaches in terms of average performance both in-distribution and out-of-distribution.

Weather Foundation Models (WFMs) have recently attracted significant attention for their exceptional performance and inference efficiency in global-scale weather forecasting. However, their coarse spatial resolution and inherent biases constrain their utility for station-level forecasting, which is crucial for applications such as renewable energy management and aviation safety. To address these limitations, we propose the Adaptive Spatiotemporal Alignment Fusion Network (ASTAFN), a novel framework designed for accurate station-level weather forecasting through the synergistic integration of WFMs and station observations. ASTAFN incorporates two complementary data sources: (1) recent station observations, which offer fine-grained local trend information, and (2) WFM-generated forecasts, which provide broad-scale weather patterns. The core innovation of ASTAFN lies in its proxy station learning mechanism, which aligns the spatial structure and corrects the biases of WFMs relative to actual station data, facilitating the extraction of homogeneous spatiotemporal features from both sources. These features are dynamically fused at each forecasting step using an adaptive strategy, effectively compensating for WFM biases and enhancing predictive accuracy. Experimental evaluations on three real-world datasets demonstrate that ASTAFN reduces mean absolute error by 20%–35% compared to baseline WFMs for station-level wind speed forecasting. ASTAFN has been deployed on the regional station-level weather forecasting and analysis platform of the Chinese Academy of Meteorological Sciences, currently serving the Guangdong and Yunnan provinces in southern China.

Deepfake technology has great potential in the field of media and entertainment, but it also brings serious risks, including privacy disclosure and identity fraud. To counter these threats, proactive forensic methods have become a research hotspot by embedding invisible watermark signals to build active protection schemes. However, existing methods are vulnerable to watermark destruction under malicious distortions, which leads to insufficient robustness. Moreover, embedding strong signals may degrade image quality, making it challenging to balance robustness and imperceptibility. Although watermarked images look natural, their underlying structures are often different from the original images, which is ignored by traditional watermarking methods. To address these issues, this paper proposes a proactive watermarking framework called WaveGuard, which explores frequency domain embedding and graph-based structural consistency optimization. In this framework, the watermark is embedded into the high-frequency sub-bands by dual-tree complex wavelet transform (DT-CWT) to enhance the robustness against distortions and deepfake forgeries. By leveraging joint sub-band correlations and selected sub-band combinations, the framework enables robust source tracing and semi-robust deepfake detection. To enhance imperceptibility, we propose a Structural Consistency Graph Neural Network (SC-GNN) that constructs graph representations of the original and watermarked images to ensure structural consistency and reduce perceptual artifacts. Experimental results show that the proposed method performs exceptionally well in face swap and face replay tasks. The code has been published at https://github.com/vpsg-research/WaveGuard.

Image manipulation localization (IML) aims to segment manipulated regions in suspicious images. However, most existing methods rely solely on intrinsic features extracted from the input image and passively model local or global inconsistencies, making it difficult to accurately delineate manipulated regions with ambiguous boundaries. To address these challenges, we propose a prototype memory-based neighboring feature fusion network (PNF-Net), which is inspired by a biological memory mechanism. PNF-Net simulates selective preference by learning manipulation-trace prototypes as memory priors, thereby guiding representation learning toward consistent and discriminative manipulation cues. Specifically, we propose a memory-guided localization module (MLM) that models the consistencies and anomalies between manipulated regions and the background as memory priors, enabling precise localization. We then propose a neighboring feature interaction module (NFIM) that preserves fine-grained details from neighboring shallow features, enhances global semantics from neighboring deep features, and effectively fuses them. Finally, a verification fusion module (VFM) is designed to enrich contextual semantics and improve the completeness and accuracy of localization results. Extensive experiments on multiple benchmark datasets show that our PNF-Net outperforms most state-of-the-art IML models. Our code is available on https://github.com/vpsg-research/PNF-Net.

Active defense strategies have been developed to counter the threat of deepfake technology. However, a primary challenge is their lack of persistence, as their effectiveness is often short-lived. Attackers can bypass these defenses by simply collecting protected samples and retraining their models. This means that static defenses inevitably fail when attackers retrain their models, which severely limits practical use. We argue that an effective defense not only distorts forged content but also blocks the model’s ability to adapt, which occurs when attackers retrain their models on protected images. To achieve this, we propose an innovative Two-Stage Defense Framework (TSDF). Benefiting from the intensity separation mechanism designed in this paper, the framework uses dual-function adversarial perturbations to perform two roles. First, it can directly distort the forged results. Second, it acts as a poisoning vehicle that disrupts the data preparation process essential for an attacker’s retraining pipeline. By poisoning the data source, TSDF aims to prevent the attacker’s model from adapting to the defensive perturbations, thus ensuring the defense remains effective long-term. Comprehensive experiments show that the performance of traditional interruption methods degrades sharply when these methods are subjected to adversarial retraining. However, our framework shows a strong dual defense capability, which can improve the persistence of active defense. Our code will be available at https://github.com/vpsg-research/TSDF.

The denoising diffusion probabilistic model (DDPM) for image inpainting transforms the input image to Gaussian noise by adding noise in the forward process, and reconstitution masked regions that are consistent with unmasked regions from pure noise in the reverse process. Recently, on inpainting methods based on DDPM, the unmasked regions failed to completely degrade into the same pure noise as the masked regions in the forward process, while in the reverse process, the unmasked and masked regions began to denoise from the pure noise at the same time, which led to the inconsistency between the forward and reverse processes. To this end, we propose BlurPaint, which completely degrades unmasked regions in the forward process and keeps consistent with masked regions that become Gaussian noise. In the reverse process, the unmasked regions are combined to complete the image. To better extract useful information from pure noise, a multi-scale feature fusion module (MFFM) is designed as the component of the denoise network, which learns local and non-local features through multi-scale receptive fields. Experimental results on multiple benchmark datasets demonstrate the performance of the proposed model. Our code is available at https://github.com/vpsg-research/BlurPaint.

Recently, the image super-resolution (SR) has made remarkable progress. However, due to the proliferation of resource-constrained scenarios, the computational-intensive SR technology is limited in portable devices. Therefore, high efficiency and lightweight become the key factors of image SR in the real world. To overcome these problems, we propose a hierarchical perceptual distillation network (HPDN), which is a lightweight solution that includes two efficient designs. Firstly, we construct the context cooperation perception attention (CCPA), which adds rich information structure by introducing various pooling modes to obtain more accurate content. Secondly, the proposed multi-scale Dconv sparse attention (MSDSA) captures input features at multiple scales, and pays attention to global information at different receptive fields for image reconstruction. A large number of experiments show that our network almost achieves the best results compared with the SOTA methods.

Recently, Transformer-based methods have gained popularity in skeleton-based action recognition due to their advantages in modeling long-range dependencies. However, Transformer lacks the inductive biases towards skeletal topology and tends to capture salient features, potentially overlooking subtle inter-class variations in similar actions, leading classifications. To address these issues, we first propose Hybrid Spatial-Frequency Attention Network (HSFA-Net) with spatial and temporal structure for fine-grained action recognition. The spatial structure includes a Frequency Domain Channel Enhancement module, which utilizes the Discrete Cosine Transform to convert grouped skeleton data into frequency domain and refines it with multiple high-frequency components. Next, we propose Spatial-Frequency Attention Fusion module to integrate spatial and frequency features for a more comprehensive feature representation. In the temporal structure, we apply the concept of temporal differences to human skeletons and design a Temporal Motion-Sensitive module to highlight motion-sensitive features. Finally, we propose Temporal Frequency Attention module for richer temporal features in the frequency domain. Extensive experiments on NTU-RGB+D and NTU-RGB+D 120 datasets validate the effectiveness of HSFA-Net.

In recent years, it has been found that "grandmother cells" in the primary visual cortex (V1) of macaques can directly recognize visual input with complex shapes. This inspires us to examine the value of these cells in promoting the research of medical image segmentation. In this paper, we design a Similarity Memory Prior Network (Sim-MPNet) for medical image segmentation. Specifically, we propose a Dynamic Memory Weights-Loss Attention (DMW-LA), which matches and remembers the category features of specific lesions or organs in medical images through the similarity memory prior in the prototype memory bank, thus helping the network to learn subtle texture changes between categories. DMW-LA also dynamically updates the similarity memory prior in reverse through Weight-Loss Dynamic (W-LD) update strategy, effectively assisting the network directly extract category features. In addition, we propose the Double-Similarity Global Internal Enhancement Module (DS-GIM) to deeply explore the internal differences in the feature distribution of input data through cosine similarity and euclidean distance. Extensive experiments on four public datasets show that Sim-MPNet has better segmentation performance than other state-of-the-art methods. Our code is available on https://github.com/vpsg-research/Sim-MPNet.

When detecting defects in insulator images, a large number of methods use convolutional neural network (CNN) based structures and few vision transformer (ViT) based methods. Meanwhile, the ViT based methods are difficult to accurately recognize complex insulator defects because the self-attention (SA) mechanism is weaker than CNN in capturing valid information. Additionally, frequent use of the attention module to improve the extraction of valid information leads to large model parameters. To address these issues, we propose an insulator defect detection model based on vision transformer, which is called ID-DETR. Firstly, the proposed fast multi-scale extraction module (FMSE) utilizes partial convolution (PConv) and the efficient multi-scale attention module (EMA) to reduce redundant computations and extract spatial features efficiently. Secondly, a cross-weighted bidirectional feature pyramid network (CWB-FPN) is constructed to retain more detailed information of different layers, which compensates for the shortcoming of the SA. Thirdly, the efficient feature fusion module (EFFM) further enhances the fusion of valid information. Extensive experimental results verify the effectiveness of the proposed method on the synthetic foggy insulator dataset (SFID), the self-made insulator dataset (SID), and the Pascal VOC2007. The code will be released after acceptance.

Face forgery detection has become a new research hotspot. Though existing detection works have achieved impressive performance, they are difficult to achieve a proper trade-off between detection accuracy and model complexity. To solve this problem, we design some low-complexity modules and construct a lightweight dynamic fusion network (LDFnet) to achieve high accuracy and lightweight face forgery detection. Firstly, we regard significant local visual artifacts as a correct semantic feature needed for detection. A spatial group-wise enhance (SGE) module is introduced as a supervision to suppress possible noise and capture local artifacts. Secondly, we design a manipulation trace extraction block (TraceBlock), which can replace vanilla convolution to achieve global inference, thus capturing the texture information in the global scope. Based on TraceBlock, we construct a global texture representation (GTR) network to extract global manipulation features hierarchically. Finally, we design a dynamic fusion mechanism (DFM) to fully fuse local and global clues, and dynamically generate a more discriminating feature representation. Extensive experimental results show that the proposed LDFnet is significantly superior to the previous detection works on some popular face forgery datasets, such as FF++, DFDC, CelebDF and HFF. In particular, LDFnet only uses 963k model parameters and 801M FLOPs, which is far lower than the calculation cost of face forgery detection based on large model, and achieves better detection results.

The serious concerns over the negative impacts of Deepfakes have attracted wide attentions in the community of multimedia forensics. The existing detection works achieve deepfake detection by improving the traditional backbone networks to capture subtle manipulation traces. However, there is no attempt to construct new backbone networks with different structures for Deepfake detection by improving the internal feature representation of convolution. In this work, we propose a novel Space-Frequency Interactive Convolution (SFIConv) to efficiently model the manipulation clues left by Deepfake. To obtain high-frequency features from tampering traces, a Multichannel Constrained Separable Convolution (MCSConv) is designed as the component of the proposed SFIConv, which learns space-frequency features via three stages, namely generation, interaction and fusion. In addition, SFIConv can replace the vanilla convolution in any backbone networks without changing the network structure. Extensive experimental results show that seamlessly equipping SFIConv into the backbone network greatly improves the accuracy for Deepfake detection. In addition, the space-frequency interaction mechanism does benefit to capturing common artifact features, thus achieving better results in cross-dataset evaluation. Our code will be available athttps://github.com/EricGzq/SFIConv.

Image steganography is a technique used to conceal secret information within cover images without being detected. However, the advent of convolutional neural networks (CNNs) has threatened the security of image steganography. Due to the inherent properties of adversarial examples, adding perturbations to stego images can mislead the CNN-based image steganalysis, but it also easily leads to some errors when extracting secret information. Recently, some adversarial embedding methods have been proposed for improving image steganography security. In this work, we aim at furthering enhance the security of adversarial embedding-based image steganography by exploiting the strong correlation between adjacent pixels. Specifically, we divide the cover image into four non-overlapping parts for four-stage information embedding. During the adversarial embedding process, we cluster the modification directions of adjacent pixels and select only those with relatively larger amplitudes of gradients and smaller embedding costs to update their original embedding costs. Experimental results demonstrate that our proposed method can effectively fool targeted steganalyzers and outperform state-of-the-art techniques under different scenarios.

For Deepfake detection, residual-based features can preserve tampering traces and suppress irrelevant image content. However, inappropriate residual prediction brings side effects on detection accuracy. Meanwhile, residual-domain features are easily affected by some image operations such as lossy compression. Most existing works exploit either spatial-domain or residual-domain features, which are fed into the backbone network for feature learning. Actually, both types of features are mutually correlated. In this work, we propose an adaptive fusion based guided residuals network (AdapGRnet), which fuses spatial-domain and residual-domain features in a mutually reinforcing way, for Deepfake detection. Specifically, we present a fine-grained manipulation trace extractor (MTE), which is a key module of AdapGRnet. Compared with the prediction-based residuals, MTE can avoid the potential bias caused by inappropriate prediction. Moreover, an attention fusion mechanism (AFM) is designed to selectively emphasize feature channel maps and adaptively allocate the weights for two streams. Experimental results show that AdapGRnet achieves better detection accuracies than the state-of-the-art works on four public fake face datasets including HFF, FaceForensics++, DFDC and CelebDF. Especially, AdapGRnet achieves an accuracy up to 96.52% on the HFF-JP60 dataset, which improves about 5.50%. That is, AdapGRnet achieves better robustness than the existing works.