At present, most hyperspectral (HS) sharpening methods have not fully utilized the feature correlation between adjacent bands in HS images, nor have they explored the problem of feature uncertainty generated by the model during the fusion process. This may lead to inaccurate fusion features generated by the model, resulting in spatial and spectral distortions in the fusion results. To address these issues, we propose an uncertainty-guided memory network (UMNet) for HS pansharpening. A spatial-spectral recurrent fusion unit (SRFU) is designed based on the concept of temporal data modeling, which utilizes the correlation between adjacent bands to fuse spectral and spatial features from PAN and LRHS images. In SRFU, a state memory interaction unit (SMIU) is constructed based on non-negative matrix factorization (NMF) to learn the global spatial-spectral dependency of PAN and HS images in the recurrent state space. Moreover, based on uncertainty theory, we define two spatial-spectral uncertainty-guided loss functions for the HS pansharpening task to train the model step by step, ensuring that the network can reconstruct more accurate spectral and spatial features. Extensive experiments on three widely used datasets demonstrate that, compared with some state-of-the-art (SOTA) methods, the proposed UMNet has achieved significant improvements in both spatial and spectral quality metrics.

Multi-character role-playing aims to equip models with the capability to simulate diverse roles. Existing methods either use one shared parameterized module across all roles or assign a separate parameterized module to each role. However, the role-shared module may ignore distinct traits of each role, weakening personality learning, while the role-specific module may overlook shared traits across multiple roles, hindering commonality modeling. In this paper, we propose a novel HyCoRA: Hyper-Contrastive Role-Adaptive learning framework, which efficiently improves multi-character role-playing agents' ability by balancing the learning of distinct and shared traits. Specifically, we propose a Hyper-Half Low-Rank Adaptation structure, where one half is a role-specific module generated by a lightweight hyper-network, and the other half is a trainable role-shared module. The role-specific module is devised to represent distinct persona signatures, while the role-shared module serves to capture common traits. Moreover, to better reflect distinct personalities across different roles, we design a hyper-contrastive learning mechanism to help the hyper-network distinguish their unique characteristics. Extensive experimental results on both English and Chinese available benchmarks demonstrate the superiority of our framework. Further GPT-4 evaluations and visual analyses also verify the capability of HyCoRA to capture role characteristics.

Infrared and visible image fusion (IVIF) aims to fuse thermal target information in infrared images and spatial texture information in visible images, improving the observability and comprehensibility of the fused images. Currently, most IVIF methods suffer from the loss of salient target information and texture details in fused images. To alleviate this problem, a progressive cross-modal feature fusion network (PCFFusion) for IVIF is proposed, which comprises two stages: feature extraction and feature fusion. In the feature extraction stage, to enhance the network’s feature representation capability, a feature decomposition module (FDM) is constructed to extract two modal features of different scales by defining a feature decomposition operation (FDO). In addition, by establishing correlations between the high- frequency and low-frequency components of two modal features, a cross-modal feature enhancement module (CMFEM) is built to realize correction and enhancement of the two features at each scale. The feature fusion stage achieves the fusion of two modal features at each scale and the supplementation of adjacent scale features by constructing three cross-domain fusion module (CDFMs). To constrain the fused results preserve more salient targets and richer texture details, a dual-feature fidelity loss function is defined by constructing a salient weight map to balance the two loss terms. Extensive experiments demonstrate that fusion results of the proposed method highlight prominent targets from infrared images while retaining rich background details from visible images, and the performance of PCFFusion is superior to some advanced methods. Specifically, compared to the optimal results obtained by other comparison methods, the proposed network achieves an average increase of 30.35 % and 10.9 % in metrics Mutual Information (MI) and Standard deviation (SD) on the TNO dataset, respectively.

Text-to-image (T2I) diffusion models enable high-quality image generation conditioned on textual prompts. However, fine-tuning these pre-trained models for personalization raises concerns about unauthorized dataset usage. To address this issue, dataset ownership verification (DOV) has recently been proposed, which embeds watermarks into fine-tuning datasets via backdoor techniques. These watermarks remain dormant on benign samples but produce owner-specified outputs when triggered. Despite its promise, the robustness of DOV against copyright evasion attacks (CEA) remains unexplored. In this paper, we investigate how adversaries can circumvent these mechanisms, enabling models trained on watermarked datasets to bypass ownership verification. We begin by analyzing the limitations of potential attacks achieved by backdoor removal, including TPD and T2IShield. In practice, TPD suffers from inconsistent effectiveness due to randomness, while T2IShield fails when watermarks are embedded as local image patches. To this end, we introduce CEAT2I, the first CEA specifically targeting DOV in T2I diffusion models. CEAT2I consists of three stages: (1) motivated by the observation that T2I models converge faster on watermarked samples with respect to intermediate features rather than training loss, we reliably detect watermarked samples; (2) we iteratively ablate tokens from the prompts of detected samples and monitor feature shifts to identify trigger tokens; and (3) we apply a closed-form concept erasure method to remove the injected watermarks. Extensive experiments demonstrate that CEAT2I effectively evades state-of-the-art DOV mechanisms while preserving model performance. The code is available at https://github.com/csyufei/CEAT2I.

Currently, most Hyperspectral (HS) pansharpening methods have two problems, namely the lack of consideration the spatial variations of HS images and inaccurate feature reconstruction in multi-channel complex mapping relationships, leading to spectral and spatial distortions in the fusion results. To address these issues, we propose a dynamic network based on feature modulation and probability mask (FMPM-DNet) for HS pansharpening, including two stages of spectral-spatial feature modulation and feature reconstruction. In the first stage, to increase the feature representation ability of the model, a wave function is defined based on complex transformation to convert spatial features into wave-like features. On this basis, considering the spatial variations of HS images, a dynamic feature modulation unit (DFMU) is constructed to achieve adaptive modulation and coarse fusion of features by dynamically generating spectral-spatial correction matrix. In the second stage, a feature probability mask unit (FPMU) is designed to realize global feature embedding at different depths and local feature embedding at the same depth to obtain refined fused features. Extensive experiments on three widely used datasets demonstrate that the proposed FMPM-Net achieves significant improvements in both spatial and spectral quality metrics compared to some state-of-the-art (SOTA) methods.

Multimodal summarization (MS) combines text and visuals to generate summaries.Recently, many-to-many multimodal summarization (M3S) garnered interest as it enables a unified model for multilingual and cross-lingual MS.Existing methods have made progress by facilitating the transfer of common multimodal summarization knowledge.While, prior M3S models that fully share parameters neglect the language-specific knowledge learning, where potential interference between languages may limit the flexible adaptation of MS modes across different language combinations and hinder further collaborative improvements in joint M3S training.Based on this observation, we propose Language Constrained Multimodal Hyper Adapter (LCMHA) for M3S.LCMHA integrates language-specific multimodal adapters into multilingual pre-trained backbones via a language constrained hypernetwork, enabling relaxed parameter sharing that enhances language-specific learning while preserving shared MS knowledge learning.In addition, a language-regularized hypernetwork is designed to balance intra-and inter-language learning, generating language-specific adaptation weights and enhancing the retention of distinct language features through the regularization of generated parameters.Experimental results on the M3Sum benchmark show LCMHA's effectiveness and scalability across multiple multilingual pre-trained backbones.

Nayu Liu, Junnan Zhu, Yiming Ma, Zhicong Lu, Wenlei Xu, Yong Yang, Jiang Zhong, Kaiwen Wei. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Although Transformer has achieved considerable results in image deraining tasks, the quadratic complexity of self-attention in this structure limits its ability to process high-resolution rainy images. The Receptance Weighted Key Value (RWKV) in the field of natural language processing (NLP) solves the expensive computational cost of self-attention in Transformers by using linear computational complexity to learn long-range dependencies of features. Based on advanced design of RWKV, we propose an image deraining RWKV (ID-RWKV) that gradually obtains fine rainless images by constructing multiple deraining stages. At each stage, a U-shaped rain removal network (U-RRNet) is constructed to encode and decode image features, and output the rain removal results for the current stage. Each layer of U-RRNet consists of Fourier enhancement module (FFM) and local-global RWKV Block (LGRB). FEM is constructed to extract and enhance the features in the frequency domain. LGRB is designed based on RWKV structure to improve the performance and efficiency of deraining models. To better learn local and global contextual information, we propose a LG-WKV attention mechanism to enhance local and global dependencies. To reduce the loss of background information and increase the stability of the network, we construct a deep-shallow feature fusion module (DSFFM) to supplement shallow features. Following extensive experimentation, we demonstrate that our method not only outperforms the current state-of-the-art methods, but also requires fewer parameters and less computation than the Transformer-based method.

Fusion of a high-spatial-resolution multispectral image (MSI) and a low-spatial-resolution hyperspectral image (HSI) aims to generate a high-spatial-resolution HSI (HR-HSI). Most fusion methods use simple upsampling techniques to increase the resolution of HSI without guidance, which can introduce unwanted artifacts and lead to spectral distortion. Additionally, they face challenges in generalization and robustness. To address these challenges, this paper introduces a cross-modal fusion network for MSI and HSI, named CSGAV, which is built on similarity-guided graph attention (SGA) and a variational autoencoder-Transformer (VAET). Specifically, we develop a similarity measure algorithm to compute the similarity degree between the source images and construct an SGA module to mitigate modal differences, producing precise upsampled outputs. Moreover, we present an adaptive weighted Transformer, with the weights guided by a variational autoencoder, thereby enhancing the generalization and robustness of the model. The SGA and VAET are integrated in the cross-modal interactive architecture to achieve the final HR-HSI image. Experimental results conducted on four public datasets show that CSGAV is superior compared to existing state-of-the-art fusion methods both in fusion performance and generalization. The code of this work is available at https://github.com/yotick/CSGAV.

Hyperspectral (HS) pansharpening aims to fuse high-spatial-resolution panchromatic (PAN) images with low-spatial-resolution hyperspectral (LRHS) images to generate high-spatial-resolution hyperspectral (HRHS) images. Due to the lack of consideration for the modal feature difference between PAN and LRHS images, most deep leaning-based methods suffer from spectral and spatial distortions in the fusion results. In addition, most methods use upsampled LRHS images as network input, resulting in spectral distortion. To address these issues, we propose a dual-stage feature correction fusion network (DFCFN) that achieves accurate fusion of PAN and LRHS images by constructing two fusion sub-networks: a feature correction compensation fusion network (FCCFN) and a multi-scale spectral correction fusion network (MSCFN). Based on the lattice filter structure, FCCFN is designed to obtain the initial fusion result by mutually correcting and supplementing the modal features from PAN and LRHS images. To suppress spectral distortion and obtain fine HRHS results, MSCFN based on 2D discrete wavelet transform (2D-DWT) is constructed to gradually correct the spectral features of the initial fusion result by designing a conditional entropy transformer (CE-Transformer). Extensive experiments on three widely used simulated datasets and one real dataset demonstrate that the proposed DFCFN achieves significant improvements in both spatial and spectral quality metrics over other state-of-the-art (SOTA) methods. Specifically, the proposed method improves the SAM metric by 6.4%, 6.2%, and 5.3% compared to the second-best comparison approach on Pavia center, Botswana, and Chikusei datasets, respectively. The codes are made available at: https://github.com/EchoPhD/DFCFN.

Infrared small target detection (IRSTD) involves identifying targets that are typically small in spatial extent, have low signal-to-clutter ratios, and are often embedded in dynamic and complex backgrounds, making the task particularly challenging. Benefiting from the powerful feature extraction and multiscale feature fusion characteristics, U-Net performs well in the IRSTD task. However, existing U-Net methods often focus solely on optimizing backbone feature extraction or skip connections, which limits their performance in complex scenes and makes it difficult to recognize small targets effectively. To address this limitation, we propose a novel hierarchical attention fusion network based on the U-Net architecture, namely HAFNet. Specifically, a dual-branch semantic perception module (DSPM) is designed as the feature extraction backbone to enhance contextual semantic interactions. This module integrates dual-branch feature extraction using standard and dilated convolutions while utilizing spatial and channel attention modules (CAMs) to effectively separate small targets from background noise. In addition, we extend the skip connection by merging a hierarchical feature fusion encoder (HFFE) and a hierarchical feature fusion decoder (HFFD). These modules utilize hierarchical attention-guided and encoded feature injection skip connections (ESCs) to achieve effective fusion of multiscale and multilevel semantic features between the encoder and decoder. Extensive experiments on three public datasets (NUAA-SIRST, IRSTD-1K, and NUDT-SIRST) demonstrate that the proposed HAFNet outperforms the existing IRSTD methods and achieves state-of-the-art (SOTA) detection performance. The code will be released onhttps://github.com/Wangtao-Bao/HAFNet

This study investigates the pattern of information interaction at the cortical level during deception, aiming to reveal the cognitive processes involved in the deception task. Our study involves the 64-channel EEG signals of 28 subjects (14 for innocent and 14 for guilty groups) acquired under the guilty knowledge test (GKT) lie-detection protocol. Additionally, we establish the functional connectivity network at the cortical level considering volume conduction effects, use a data-driven approach to select the regions of interest (ROIs) on the subject's cortex based on scalp electrical activity, and perform cortical current density estimation on 15 ROIs. The nonlinear dependence between the cortical waveforms of the ROIs is quantified based on mutual information, and a network of cortical mutual information connections is constructed in four frequency bands: delta, theta, alpha, and beta. The feature extraction and classification process are performed in each frequency band, and the mutual information connections statistically different between the innocent and guilty groups are first selected as features using statistical tests. Moreover, the optimal feature subset (OFS) is found by combining the SVM classifier and the wrapper feature selection strategy. Furthermore, the most important mutual information connections (MIMICs) per frequency band are obtained by refining the OFS according to the classification performance curve. The average test accuracies of MIMICs in the delta, theta, alpha, and beta bands reached 99.76%, 96.42%, 84.04%, and 97.61%, respectively. Finally, the physiological significance of each frequency sub-band and the physiological function of MIMICs are combined to explore the cognitive mechanism of lies and provide new evidence for cognitive activity in lying states.

Due to the unique environment and inherent properties of magnetic resonance imaging (MRI) instruments, MR images typically have lower resolution. Therefore, improving the resolution of MR images is beneficial for assisting doctors in diagnosing the condition. Currently, the existing MR image super-resolution (SR) methods still have the problem of insufficient detail reconstruction. To overcome this issue, this paper proposes a multi-level feature transfer network (MFTN) based on MRI-Transformer to realize SR of low-resolution MRI data. MFTN consists of a multi-scale feature reconstruction network (MFRN) and a multi-level feature extraction branch (MFEB). MFRN is constructed as a pyramid structure to gradually reconstruct image features at different scales by integrating the features obtained from MFEB, and MFEB is constructed to provide detail information at different scales for low resolution MR image SR reconstruction by constructing multiple MRI-Transformer modules. Each MRI-Transformer module is designed to learn the transfer features from the reference image by establishing feature correlations between the reference image and low-resolution MR image. In addition, a contrast learning constraint item is added to the loss function to enhance the texture details of the SR image. A large number of experiments show that our network can effectively reconstruct high-quality MR Images and achieves better performance compared to some state-of-the-art methods. The source code of this work will be released on GitHub.

Given the power of vision transformers, a new learning paradigm, pre-training and then prompting, makes it more efficient and effective to address downstream visual recog-nition tasks. In this paper, we identify a novel security threat towards such a paradigm from the perspective of back-door attacks. Specifically, an extra prompt token, called the switch token in this work, can turn the backdoor mode on, i.e., converting a benign model into a backdoored one. Once under the backdoor mode, a specific trigger can force the model to predict a target class. It poses a severe risk to the users of cloud API, since the malicious behavior can not be activated and detected under the benign mode, thus making the attack very stealthy. To attack a pre-trained model, our proposed attack, named SWARM, learns a trigger and prompt tokens including a switch token. They are optimized with the clean loss which encourages the model always be-haves normally even the trigger presents, and the backdoor loss that ensures the backdoor can be activated by the trig-ger when the switch is on. Besides, we utilize the cross-mode feature distillation to reduce the effect of the switch token on clean samples. The experiments on diverse vi-sual recognition tasks confirm the success of our switchable backdoor attack, i.e., achieving 95%+ attack success rate, and also being hard to be detected and removed. Our code is available at https://github.com/20000yshust/SWARM.

Hyperspectral image super-resolution (HISR) aims to fuse a low-resolution hyperspectral image (LR-HSI) with a high-resolution multispectral image (HR-MSI) to obtain a high-resolution hyperspectral image (HR-HSI). Due to some existing HISR methods ignoring the significant feature difference between LR-HSI and HR-MSI, the reconstructed HR-HSI typically exhibits spectral distortion and blurring of spatial texture. To solve this issue, we propose a multi-scale feature transfer network (MFTN) for HISR. Firstly, three multi-scale feature extractors are constructed to extract features of different scales from the input images. Then, a multi-scale feature transfer module (MFTM) consisting of three improved feature matching Transformers (IMatchFormers) is designed to learn the detail features of different scales from HR-MSI by establishing the cross-model feature correlation between LR-HSI and degraded HR-MSI. Finally, a multiscale dynamic aggregation module (MDAM) containing three spectral aware aggregation modules (SAAMs) is constructed to reconstruct the final HR-HSI by gradually aggregating features of different scales. Extensive experimental results on three commonly used datasets demonstrate that the proposed model achieves better performance compared to state- of-the-art (SOTA) methods.

Single hyperspectral image super-resolution aims to reconstruct a high-resolution hyperspectral image (HRHSI) from an observed low resolution hyperspectral image (LRHSI). Most current methods combine CNN and Transformer structures to directly extract features of all channels in LRHSI for image reconstruction, but they do not consider the interference of redundant information in adjacent bands, resulting in spectral and spatial distortions in the reconstruction results and an increase in model computational complexity. To address this issue, this paper proposes a spectral clustering-based pyramid super-resolution network (SCPSN) to progressively reconstruct HRHSI at different scales. In each image reconstruction layer, a clustering super-resolution block (CSRB) consisting of spectral clustering block (SCB), patch non local attention block (PNAB), and dynamic fusion block (DFB) is designed to achieve the reconstruction of detail features. Specifically, for the high correlation between adjacent spectral bands in LRHSI, a SCB is first constructed to achieve clustering of spectral channels and filtering of hyperchannels. This can reduce the interference of redundant spectral information and the computational complexity of the model. Then, by utilizing the non-local similarity of features within the channel, a patch non-local attention block (PNAB) is constructed to enhance the features of hyperchannels. Next, a dynamic fusion block (DFB) is designed to reconstruct the features of all channels in LRHSI by establishing correlations between enhanced hyperchannels and other channels. Finally, the reconstructed channels are upsampled and added to the corresponding channels to obtain the reconstructed HRHSI. Extensive experiments validate that the performance of SCPSN is superior to that of some other state-of-the-art (SOTA) HSSR methods in terms of visual effects and quantitative metrics. In addition, our model does not require training on large-scale datasets compared to other methods. The dataset and code will be released on GitHub.

In this letter, to better supplement the advantages of features at different levels and improve the feature extraction ability of the network, a novel multi-level feature interaction transformer network (MFITN) is proposed for pansharpening, aiming to fuse multispectral (MS) and panchromatic (PAN) images. In MFITN, a multi-level feature interaction transformer encoding module is designed to extract and correct global multi-level features by considering the modality difference between source images. These features are then fused using the proposed multi-level feature mixing (MFM) operation, which enables features to fuse interactively to obtain richer information. Furthermore, the global features are fed into a CNN-based local decoding module to better reconstruct high-spatial-resolution multispectral (HRMS) images. Additionally, based on the spatial consistency between MS and PAN images, a band compression loss is defined to improve the fidelity of fused images. Numerous simulated and real experiments demonstrate that the proposed method has the optimal performance compared to state-of-the-art methods. Specifically, the proposed method improves the SAM metric by 7.89% and 6.41% compared to the second-best comparison approach on Pléiades and WorldView-3, respectively.

Pansharpening technology aims to extract spatial information from a panchromatic (PAN) image and integrate it into a multispectral (MS) image to generate a high-spatial resolution MS image. To overcome the problem of spatial and spectral distortion of traditional methods, this letter presents a pansharpening method based on fuzzy logic and edge activity. To obtain more accurate spatial details of source images, details extracted using the component substitution and multiresolution analysis methods are fused via the proposed fuzzy logic algorithm. To preserve the edges of the fusion result, the edge maps of the source images are detected and fused based on the edge activity. The optimized detail maps are obtained by multiplying the fused details and edge maps, which are then injected into the upsampled MS image to obtain the final pansharpened image. Reduced- and full-scale experimental results on the Pléiades and IKONOS datasets demonstrate the effectiveness of our method compared with state-of-the-art pansharpening algorithms. Specifically, the proposed method improves the ERGAS metrics by 9.0% and 11.1% compared to the second-best comparison approach on Pléiades and IKONOS, respectively.

Multimodal summarization (MS) for videos aims to generate summaries from multi-source information (e.g., video and text transcript), showing promising progress recently. However, existing works are limited to monolingual scenarios, neglecting non-native viewers' needs to understand videos in other languages. It stimulates us to introduce multimodal cross-lingual summarization for videos (MCLS), which aims to generate cross-lingual summaries from multimodal input of videos. Considering the challenge of high annotation cost and resource constraints in MCLS, we propose a knowledge distillation (KD) induced triple-stage training method to assist MCLS by transferring knowledge from abundant monolingual MS data to those data with insufficient volumes. In the triple-stage training method, a video-guided dual fusion network (VDF) is designed as the backbone network to integrate multimodal and cross-lingual information through diverse fusion strategies in the encoder and decoder; What's more, we propose two cross-lingual knowledge distillation strategies: adaptive pooling distillation and language-adaptive warping distillation (LAWD), designed for encoder-level and vocab-level distillation objects to facilitate effective knowledge transfer across cross-lingual sequences of varying lengths between MS and MCLS models. Specifically, to tackle lingual sequences of varying lengths between MS and MCLS models. Specifically, to tackle the challenge of unequal length of parallel cross-language sequences in KD, LAWD can directly conduct cross-language distillation while keeping the language feature shape unchanged to reduce potential information loss. We meticulously annotated the How2-MCLS dataset based on the How2 dataset to simulate MCLS scenarios. Experimental results show that the proposed method achieves competitive performance compared to strong baselines, and can bring substantial performance improvements to MCLS models by transferring knowledge from the MS model.

Pansharpening aims to obtain a high-spatial-resolution multispectral (MS) image by fusing a lower-spatial resolution MS image with a high-spatial-resolution panchromatic (PAN) image. Currently, the results obtained by most pansharpening methods still suffer from spatial and spectral distortion issues. The diffusion model has shown outstanding performance in various image-processing tasks. However, maintaining the full image size throughout the diffusion process imposes a large computational burden, and the simultaneous use of PAN and MS images acquired by different sensors as a condition for guiding noise prediction leads to spatial and spectral distortions. To solve these problems, a variable-scale diffusion model (VSDM) based on dynamic condition guidance for pansharpening is proposed, which achieves better fusion performance by improving the diffusion manner of the diffusion model and injecting dynamic conditions to guide the reverse process. In VSDM, a variable-scale diffusion manner (VSDMN) is designed to reduce the computational complexity of the model by reducing the size of the image in the diffusion process. A condition generator (CG) is constructed to generate dynamic conditions using the features learned from the PAN and upsampled MS images. In CG, a cross-attention dynamic convolution is built to extract features from the PAN image by designing a spatial and spectral attention mechanism, which can improve the spatial and spectral consistency in the dynamic condition. Extensive experiments validate the effectiveness of the proposed VSDM against other state-of-the-art (SOTA) pansharpening methods in both quantitative and qualitative assessments. The source code will be released athttps://github.com/MELiMZ/VSDM.

Images captured in low-light environments have problems of insufficient brightness and low contrast, which will affect subsequent image processing tasks. Although most current enhancement methods can obtain high-contrast images, they still suffer from noise amplification and color distortion. To address these issues, this paper proposes a low-light image enhancement network based on multi-scale feature complementation (LIEN-MFC), which is a U-shaped encoder-decoder network supervised by multiple images of different scales. In the encoder, four feature extraction branches are constructed to extract features of low-light images at different scales. In the decoder, to ensure the integrity of the learned features at each scale, a feature supplementary fusion module (FSFM) is proposed to complement and integrate features from different branches of the encoder and decoder. In addition, a feature restoration module (FRM) and an image reconstruction module (IRM) are built in each branch to reconstruct the restored features and output enhanced images. To better train the network, a joint loss function is defined, in which a discriminative loss term is designed to ensure that the enhanced results better meet the visual properties of the human eye. Extensive experiments on benchmark datasets show that the proposed method outperforms some state-of-the-art methods subjectively and objectively.

Backdoor defenses have been studied to alleviate the threat of deep neural networks (DNNs) being backdoor attacked and thus maliciously altered. Since DNNs usually adopt some external training data from an untrusted third party, a robust backdoor defense strategy during the training stage is of importance. We argue that the core of training-time defense is to select poisoned samples and to handle them properly. In this work, we summarize the training-time defenses from a unified framework as splitting the poisoned dataset into two data pools. Under our framework, we propose an adaptively splitting dataset-based defense (ASD). Concretely, we apply loss-guided split and meta-learning-inspired split to dynamically update two data pools. With the split clean data pool and polluted data pool, ASD successfully defends against backdoor attacks during training. Extensive experiments on multiple benchmark datasets and DNN models against six state-of-the-art backdoor attacks demonstrate the superiority of our ASD. Our code is available at https://github.com/KuofengGao/ASD.

Pansharpening is to fuse a panchromatic (PAN) image with a multispectral (MS) image to obtain a high-spatial-resolution multispectral (HRMS) image. The deep learning-based pansharpening methods usually apply the convolution operation to extract features and only consider the similarity of gradient information between PAN and HRMS images, resulting in the problems of edge blur and spectral distortion in the fusion results. To solve this problem, a multi-supervised mask protection network (MMPN) is proposed to prevent spatial information from being damaged and overcome spectral distortion in the learning process. Firstly, by analyzing the relationships between high-resolution images and corresponding degraded images, a mask protection strategy (MPS) for edge protection is designed to guide the recovery of fused images. Then, based on the MPS, an MMPN containing four branches is constructed to generate the fusion and mask protection images. In MMPN, each branch employs a dual-stream multi-scale feature fusion module (DMFFM), which is built to extract and fuse the features of two input images. Finally, different loss terms are defined for the four branches, and combined into a joint loss function to realize network training. Experiments on simulated and real satellite datasets show that our method is superior to state-of-the-art methods both subjectively and objectively.

Pansharpening is to fuse a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LRMS) image to obtain an enhanced LRMS image with high spectral and spatial resolution. The current Transformer-based pansharpening methods neglect the interaction between the extracted long- and short-range features, resulting in spectral and spatial distortion in the fusion results. To address this issue, a novel cross Transformer and convolutional neural network (CNN) for pansharpening (CTCP) is proposed to achieve better fusion results by designing a cross mechanism, which can enhance the interaction between long- and short-range features. First, a dual branch feature extraction module (DBFEM) is constructed to extract the features from the LRMS and PAN images, respectively, reducing the aliasing of the two image features. In the DBFEM, to improve the feature representation ability of the network, a cross long-short-range feature module (CLSFM) is designed by combining the feature learning capabilities of Transformer and CNN via the cross mechanism, which achieves the integration of long-short-range features. Then, to improve the ability of spectral feature representation, a spectral feature enhancement fusion module (SFEFM) based on a frequency channel attention is constructed to realize feature fusion. Finally, the shallow features from the PAN image are reused to provide detail features, which are integrated with the fused features to obtain the final pansharpened results. To the best of our knowledge, this is the first attempt to introduce the cross mechanism between Transformer and CNN in pansharpening field. Numerous experiments show that our CTCP outperforms some state-of-the-art (SOTA) approaches both subjectively and objectively. The source code will be released at https://github.com/zhsu99/CTCP.

Pansharpening is to fuse high-resolution panchromatic (PAN) images with low-resolution multispectral (LR-MS) images to generate high-resolution multispectral (HR-MS) images. Most of the deep learning-based pansharpening methods did not consider the inconsistency of the PAN and LR-MS images and used simple concatenation to fuse the source images, which may cause spectral and spatial distortion in the fused results. To address this problem, a multi-scale spatial-spectral attention guided fusion network for pansharpening is proposed. First, the spatial features from the PAN image and spectral features from the LR-MS image are independently extracted to obtain the shallow features. Then, a spatial-spectral attention feature fusion module (SAFFM) is constructed to guide the reconstruction of spatial-spectral features by generating a guidance map to achieve the fusion of reconstructed features at different scales. In SAFFM, the guidance map is designed to ensure the spatial-spectral consistency of the reconstructed features. Finally, considering the difference between multiply scale features, a multi-level feature integration scheme is proposed to progressively achieve fusion of multi-scale features from different SAFFMs. Extensive experiments validate the effectiveness of the proposed network against other state-of-the-art (SOTA) pansharpening methods in both quantitative and qualitative assessments. The source code will be released at https://github.com/MELiMZ/ssaff.

Many security applications require unsupervised anomaly detection, as malicious data are extremely rare and often only unlabeled normal data are available for training (i.e., zero-positive). However, security operators are concerned about the high stakes of trusting black-box models due to their lack of interpretability. In this paper, we propose a post-hoc method to globally explain a black-box unsupervised anomaly detection model via rule extraction. First, we propose the concept of distribution decomposition rules that decompose the complex distribution of normal data into multiple compositional distributions. To find such rules, we design an unsupervised Interior Clustering Tree that incorporates the model prediction into the splitting criteria. Then, we propose the Compositional Boundary Exploration (CBE) algorithm to obtain the boundary inference rules that estimate the decision boundary of the original model on each compositional distribution. By merging these two types of rules into a rule set, we can present the inferential process of the unsupervised black-box model in a human-understandable way, and build a surrogate rule-based model for online deployment at the same time. We conduct comprehensive experiments on the explanation of four distinct unsupervised anomaly detection models on various real-world datasets. The evaluation shows that our method outperforms existing methods in terms of diverse metrics including fidelity, correctness and robustness.

Postoperative delirium is a common and preventable complication after cardiovascular surgery and is associated with increased risk of morbidity and mortality. However, strategies for identifying at-risk patients are limited. In this prospective observational study, intraoperative electroencephalography data of 50 patients undergoing cardiovascular surgery were collected. Twenty-five patients of them experienced delirium after surgery and 25 patients did not. The partial directional coherence method was used to evaluate the effective connectivity within the default mode network (DMN) regions in four frequency bands. Statistically significant features were considered as input signals in the CatBoost classifier to predict postoperative delirium. Compared with patients without delirium, patients with postoperative delirium had enhancement of causal effects in the DMN area, especially in the delta band. The accuracy rate of distinguishing patients with postoperative delirium from patients without postoperative delirium could reach 89.1%. These findings might help to explain why information processing was disturbed in patients with delirium and predict postoperative delirium.

Deep learning (DL)-based pansharpening methods have shown great advantages in extracting spectral–spatial features from multispectral (MS) and panchromatic (PAN) images compared with traditional methods. However, most DL-based methods ignore the local inner connection between the source images and the high-resolution MS (HRMS) image, which cannot fully extract spectral–spatial information and attempt to improve the quality of fusion by increasing the complexity of the network. To solve these problems, a lightweight network based on adaptive weighted feature learning network (AWFLN) is proposed for pansharpening. Specifically, a novel detail extraction model is first built by exploring the local relationship between HRMS and source images, thereby improving the accuracy of details and the interpretability of the network. Guided by this model, we then design a residual multiple receptive-field structure to fully extract spectral–spatial features of source images. In this structure, an adaptive feature learning block based on spectral–spatial interleaving attention is proposed to adaptively learn the weights of features and improve the accuracy of the extracted details. Finally, the pansharpened result is obtained by a detail injection model in AWFLN. Numerous experiments are carried out to validate the effectiveness of the proposed method. Compared to traditional and state-of-the-art methods, AWFLN performs the best both subjectively and objectively, with high efficiency. The code is available athttps://github.com/yotick/AWFLN.

Pansharpening is a process of fusing a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LRMS) image to obtain a high-resolution multispectral (HRMS) image. Convolutional neural networks (CNNs) have been commonly utilized in this field because of their remarkable learning capabilities. However, their convolutional operators limit the long-range feature extraction ability of CNN. Meanwhile, the Transformer models have exhibited strong capabilities in modeling long-range representations, but there are shortcomings in modeling local-range feature dependencies. To this end, we propose a novel synergistic transformer and CNN for pansharpening (STCP). First, a parallel U-shaped feature extraction module (PUFEM) is constructed for extracting the features of the LRMS and PAN images, which improves the feature representation ability for the two source images. In the PUFEM, combining the different feature learning capabilities of the CNN and transformer, we design a long-short-range feature integration block (LSFIB) to extract the short-range features and long-range features at different scales in parallel. Then, a channel attention module (CAM)-based feature fusion module (CFFM) is constructed to integrate the features extracted by the PUFEM. Finally, the shallow features from the PAN image are reused to provide detailed features, which are integrated with the fused features from the CFFM to achieve the final pansharpened results. Numerous experiments show that our STCP outperforms some state-of-the-art approaches both subjectively and objectively.

Although analyzing the brain's functional and structural network has revealed that numerous brain networks are necessary to collaborate during deception, the directionality of these functional networks is still unknown. This study investigated the effective connectivity of the brain networks during deception and uncovers the information-interaction patterns of lying neural oscillations. The electroencephalography (EEG) data of 40 lying persons and 40 honest persons were used to create the weight- directed functional brain networks (WDFBN). Specifically, the connecting edge weight was defined based on the normalized phase transfer entropy (dPTE) between each electrode pair, where the network nodes involved 30 electrode channels. Additionally, the signal connectivity matrices were constructed in four frequency bands: delta, theta, alpha, and beta and were subjected to a difference analysis of entropy values between the groups. Statistical analysis of the classification results revealed that all frequency bands correctly detect deception and innocence with an accuracy of 92.83%, 94.17%, 85.93%, and 92.25%, respectively. Therefore, dPTE can be considered a valuable feature for identifying lying. According to WDFBN analysis, deception has stronger information flow in the frontoparietal, frontotemporal and temporoparietal networks compare to honest people. Furthermore, the prefrontal cortex was also found to be activated in all frequency ranges. This study examined the critical pathways of brain information interaction during deception, providing new insights into the underlying neural mechanisms. Our analysis offers significant evidence for the development of brain networks that could potentially be used for lie detection.

Deep neural networks (DNNs) have demonstrated their superiority in practice. Arguably, the rapid development of DNNs is largely benefited from high-quality (open-sourced) datasets, based on which researchers and developers can easily evaluate and improve their learning methods. Since the data collection is usually time-consuming or even expensive, how to protect their copyrights is of great significance and worth further exploration. In this paper, we revisit dataset ownership verification. We find that existing verification methods introduced new security risks in DNNs trained on the protected dataset, due to the targeted nature of poison-only backdoor watermarks. To alleviate this problem, in this work, we explore the untargeted backdoor watermarking scheme, where the abnormal model behaviors are not deterministic. Specifically, we introduce two dispersibilities and prove their correlation, based on which we design the untargeted backdoor watermark under both poisoned-label and clean-label settings. We also discuss how to use the proposed untargeted backdoor watermark for dataset ownership verification. Experiments on benchmark datasets verify the effectiveness of our methods and their resistance to existing backdoor defenses.

Cover Caption: The cover image is based on the Research Article Active forgetting via influence estimation for neural networks by Xianjia Meng et al., https://doi.org/10.1002/int.22981.

The rapidly exploding of user data, especially applications of neural networks, involves analyzing data collected from individuals, which brings convenience to life. Meanwhile, privacy leakage in the applications as a potential threat needs to be addressed urgently. However, removing private information from models is difficult once the user's sensitive data enters machine learning models, particularly neural networks. Most of the previous amnestic methods based on retraining require full access to the training set of the target model and have limited improvements in computational resources and time improvement. In this paper, we propose Scrubber, which removes sensitive data from the original model via influence estimation to produce an unlearning model that is approximately indistinguishable from the retrained model. S crubber builds on the essential concept of influence function and reformulates the influence estimation as a closed-form update of forgetting. For learned models with strictly convex loss functions, our approach theoretically guarantees the effectiveness of forgetting while empirically demonstrating forgetting performance. For models with non-convex losses, we relax strictly convex assumptions by applying a damping term that allows us to make approximate estimates with negligible errors from the original assumption. Furthermore, experiments show that S crubber only causes less than 1% and 3% accuracy drop with more than 80% forgetting rate on average for logistic regression models and convolutional neural networks. The accuracy drop is reduced by 2%–3% compared to most state-of-the-art methods.

Pansharpening aims at fusing a multispectral (MS) image and panchromatic (PAN) image to obtain a high spatial resolution multispectral (HRMS) image. To obtain accurate details and reduce spectral distortion, this letter proposes an efficient pansharpening approach based on texture correction (TC) and detail refinement. First, a TC model is constructed based on spatial and spectral fidelity constraints to obtain a texture image that is highly correlated with the MS image. Second, a detail acquisition model is proposed by the consecutive parameter regression to adaptively refine the extracted details. Finally, the extracted details are injected into the up-sampled MS (UPMS) image to obtain the fused HRMS image. Experimental results demonstrate that the proposed method can obtain the high-quality results with high efficiency.

To reduce the loss of detail and spectral information during the network propagation and better extract detail and spectral features at different scales, a novel dual-information compensation network for pansharpening (DCNP) is proposed for fusing multispectral (MS) and panchromatic (PAN) images. In the network, the domain-specific knowledge is considered to design our DCNP architecture by focusing on the two aims of the pansharpening: spatial and spectral preservation. Specifically, a cascaded U-shaped structure is constructed to improve the feature representation ability of the network. To preserve more spatial details in the pansharpened image, the details of the PAN image are extracted based on Laplacian operator and then compensated into the network. Furthermore, for spectral preservation, the MS image is conducted by the transposed convolution as the compensation information of the network. Experiments on the full- and reduced-scale data indicate that the proposed DCNP achieves significant improvement over state-of-the-art methods in terms of subjective and objective evaluation. Specifically, DCNP improves the PSNR and ERGAS metrics by 12.9% and 44.5% respectively compared to the deep learning-based approach with the best average values on Pléiades.

Pansharpening is a technology involving information integration and processing in remote sensing imagery. It is applied to generate a high-resolution multispectral (HRMS) image through an effective fusion of a low spatial resolution multispectral image and a panchromatic (PAN) image. In this paper, we propose an end-to-end multi-scale and multi-distillation dilated network (MMDN) for pansharpening. In MMDN, to extract more abundant spatial details from source images, a clique structure-based multi-scale dilated block (CSMDB) is presented. The clique structure in CSMDB can fully transfer the information between feature maps obtained by the multi-scale dilated convolutional filters. Then, a multi-distillation residual information block (MRIB) is constructed to help the network capture the spatial structure of different scales in MS and PAN images. Finally, to reuse and supplement the feature information, a feature embedding strategy is designed by feeding the sum result of the output of cascaded CSMDBs and the shallow features to each MRIB. Experimental results verify that the proposed MMDN outperforms other compared state-of-the-art approaches in terms of objective and subjective evaluations.

Deep-learning-based pansharpening methods have achieved remarkable results due to their powerful feature representation ability. However, the existing deep-learning-based pansharpening methods not only lack information exchange and sharing between features of different resolutions but also cannot effectively use the residual information at different levels. These disadvantages may lead to the loss of spatial information and spectral information in the pansharpened image. To address the above problems, we propose a novel dual-stream convolutional neural network with residual information enhancement (DSCNN-RIE) for pansharpening. The proposed network is mainly composed of a set of dual-stream information complementation blocks (DSICBs), which can extract various spatial details at two different resolutions using convolutional filters of various sizes simultaneously, and can transfer complementary information effectively between two different resolutions. Furthermore, to improve the learning ability of the network and enhance the feature extraction, an RIE strategy is presented to stack different levels of residuals into the outputs of cascaded DSICBs. The final pansharpened image is obtained by integrating the extracted features using the shallow feature information of the source images. Experimental results on three datasets demonstrate that DSCNN-RIE outperforms ten other state-of-the-art pansharpening methods in both subjective and objective image-quality evaluations.

Pansharpening is used to fuse a panchromatic (PAN) image with a multispectral (MS) image to obtain a high-spatial-resolution multispectral (HRMS) image. Traditional pansharpening methods face difficulties in obtaining accurate details and have low computational efficiency. In this study, a unified pansharpening model based on the band-adaptive gradient and detail correction is proposed. First, a spectral fidelity constraint is designed by keeping each band of the HRMS image consistent with that of the MS image. Then, a band-adaptive gradient correction model is constructed by exploring the gradient relationship between a PAN image and each band of the MS image, so as to adaptively obtain an accurate spatial structure for the estimated HRMS image. To refine the spatial details, a detail correction constraint is defined based on the parameter transfer by designing a reduced-scale parameter acquisition model. Finally, a unified model is constructed based on the gradient and detail corrections, which is then solved by an alternating direction multiplier method. Both reduced-scale and full-scale experiments are conducted on several datasets. Compared with state-of-the-art pansharpening methods, the proposed method can achieve the best results in terms of fusion quality and has high efficiency. Specifically, our method improves the SAM and ERGAS metrics by 17.6% and 21.2% respectively compared to the traditional approach with the best average values, and improves these two metrics by 4.3% and 10.3% respectively compared to the learning-based approach with the best average values.

Methods of rain removal based on deep learning have rapidly developed, and the image quality after rain removal is continuously improving. However, the results of most methods have some common problems, including a loss of details, a blurring of edges, and the existence of artifacts. To remove rain-related information more thoroughly and retain more edge details, this paper proposes an end-to-end rain removal network based on the progressive residual detail supplement (ERRN-PRDS) approach. The entire network structure is designed in an iterative manner to obtain higher-quality rain removal images from coarse to fine. In the network, a diamond residual block is constructed as the main module of iteration to learn the feature information of the background layer. Meanwhile, to keep more texture details in the background layer, a detail supplement mechanism is designed between the iterative layers to transfer more information to the next iterative operation. Experimental results show that this method can remove the rain information more completely and better retain the image edges compared with previous state-of-the-art methods. In addition, because of the sparsity of the detail injection, our network also achieves high-quality results for image denoising tasks.

Infrared and visible image fusion can integrate the complementary information of infrared and visible images to realize a more accurate scene interpretation. In this paper, a novel infrared and visible image fusion framework is proposed, which is based on a modal feature fusion network (MFFN) and a dual visual decision fusion module. Firstly, the infrared and visible images are decomposed by side window filtering to obtain the modal feature components, which can emphasize the target information of the source images. Secondly, MFFN is designed to merge the modal feature components to get a modal fused image. Then, a dual visual decision fusion module is built to obtain a supplement image which can provide more visual supplementary information for the modal fused image. Lastly, the final fusion result is achieved by combining the supplement image and the modal fused image. Experimental results show that the proposed method can generate fusion results with clearer targets and richer texture details, compared with other state-of-the-art fusion methods.

The purpose of infrared and visible image fusion is to combine the complementary information of an infrared image and a visible image into a single image. In this paper, we propose an infrared and visible image fusion method based on dual-channel information cross fusion block (DICFB), which is developed to crossly extract and preliminarily fuse the multi-scale features of the source images. With the cascaded DICFB, we can obtain a series of fusion feature maps of the source images at different scales. Then, a progressive feature reconstruction module (PFRM) is designed to reconstruct the multi-scale fusion features to obtain the final fused image. Moreover, to better train the network, we design a joint loss function, in which a saliency map-based loss term is proposed to enhance the saliency targets in the fused images. Experimental results show that the proposed method has better performance than other state-of-the-art image fusion methods both objectively and subjectively.

Face sketch synthesis aims to generate a face sketch image from a corresponding photo image and has wide applications in law enforcement and digital entertainment. Despite the remarkable achievements that have been made in face sketch synthesis, most existing works pay main attention to the facial content transfer, at the expense of facial detail information. In this paper, we present a new generative adversarial learning framework to focus on detail preservation for realistic face sketch synthesis. Specifically, the high-resolution network is modified as generator to transform a face image from photograph to sketch domain. Except for the common adversarial loss, we design a detail loss to force the synthesized face sketch images have proximate details to its corresponding photo images. In addition, the style loss is adopted to restrain the synthesized face sketch images have vivid sketch style as the hand-drawn sketch images. Experimental results demonstrate that the proposed approach achieves superior performance, compared to state-of-the-art approaches, both on visual perception and objective evaluation. Specifically, this study indicated the higher FSIM values (0.7345 and 0.7080) and Scoot values (0.5317 and 0.5091) than most comparison methods on the CUFS and CUFSF datasets, respectively.

This paper proposes an effective infrared and visible image fusion method based on a texture conditional generative adversarial network (TC-GAN). The constructed TC-GAN generates a combined texture map for capturing gradient changes in image fusion. The generator in the TC-GAN is designed as a codec structure for extracting more details, and a squeeze-and-excitation module is applied to this codec structure to increase the weight of significant texture information in the combined texture map. The generator loss function is designed by combing the gradient loss and adversarial loss to retain the texture information of the source images. The discriminator brings the texture of the generated image closer to the visible image. To obtain significant texture information from the source images, a multiple decision map-based fusion strategy is proposed using a combined texture map and an adaptive guided filter. Extensive experiments on the public TNO and RoadScene datasets demonstrate that the proposed method is superior to other state-of-the-art algorithms in terms of a subjective evaluation and quantitative indicators.

Pansharpening is to fuse the existing low spatial resolution multi-spectral (MS) image with high spatial resolution panchromatic (PAN) image, so as to obtain high spatial resolution MS (HRMS) image. An efficient pansharpening model based on conditional random fields (CRFs) is proposed in this paper. In the model, a state feature function is designed to force the blurred HRMS image in accordance with the upsampled MS (UPMS) image to keep the spectral fidelity. Meanwhile, a transition feature function is defined to keep the sharpness of fused image. Besides, a new Gaussian filter acquisition algorithm is proposed to effectively satisfy the blur function in the model. To improve the efficiency of algorithm, a new initialization method based on fitting normal distribution is presented. Experiments are conducted on both reduced-scale images and full-scale images. Compared with some classical and state-of-art pansharpening methods, the proposed method achieves the best results in terms of fusion quality and efficiency.

Remote sensing image fusion is to fuse low spatial resolution multispectral (MS) images with high spatial resolution panchromatic (PAN) images to get high spatial resolution multispectral images. The component substitute (CS)-based methods are popular approaches for their high efficiency and high spatial resolution. However, they may produce spectral distortion, especially when there are large radiometric differences between PAN images and MS images. For tackling this problem, a new framework based on the CS model with fuzzy logic and salience measure is proposed. In order to get details that are highly relevant to MS image, a novel fuzzy logic rule based on the global salience measure is designed to fuse the details extracted from both PAN image and MS image. Furthermore, to better preserve the edges of the fused image, a new edge-preserving algorithm is defined to fuse the edges from the PAN image and MS image according to the local salience measure. A series of experiments are conducted and analyzed on both simulated images and real images from the data sets of four satellites to illustrate the effectiveness of the proposed method. Compared with some state-of-the-art methods, our method performs the best in both objective and subjective evaluations. Besides, our method has a low computational cost and is suitable for practical application.

Being different from the most methods of guided depth map enhancement based on deep convolutional neural network which focus on increasing the depth of networks, this paper is to improve the effectiveness of intensity guidance when the network goes deep. Overall, the proposed network upsamples the low-resolution depth maps from coarse to fine. Within each refinement stage of certain-scale depth features, the current-scale and all coarse-scales of the guidance features are revisited by dense connection. Therefore, the multi-scale guidance is efficiently maintained as the propagation of features. Furthermore, the proposed network maintains the intensity features in the high-resolution domain from which the multi-scale guidance is directly extracted. This design further improves the quality of intensity guidance. In addition, the shallow depth features upsampled via transposed convolution layer are directly transferred to the final depth features for reconstruction, which is called global residual learning in feature domain. Similarly, the global residual learning in pixel domain learns the difference between the depth ground truth and the coarsely upsampled depth map. Also, the local residual learning is to maintain the low frequency within each refinement stage and progressively recover the high frequency. The proposed method is tested for noise-free and noisy cases which compares against 16 state-of-the-art methods. Our experimental results show the improved performances based on the qualitative and quantitative evaluations.

In recent years, deep convolutional neural networks (CNNs) have achieved great success in Single Image Super-Resolution (SISR). Most existing networks for Super-Resolution (SR) concentrate on wider or deeper network designs, leading to neglect of the feature correlations of intermediate layers. In this letter, a novel Multilevel and Multiscale Network for SISR (M2SR) is presented. The proposed network framework consists of four parts, including the feature extraction network, the cascade residual U-shaped blocks, the channel-wise attention U-shaped block and the fusion reconstruction network. Specially, the residual U-shaped blocks are designed to extract different scales of features, which are stacked to better refine the multifeatures. Then, to fully exploit the different levels of features, a channel-wise attention U-shaped block (At-U) is proposed to adjust the feature weights, which can adaptively enhance the feature expression and correlation learning. Finally, a fusion reconstruction network is constructed to fuse the different scales of the enhanced features to achieve the reconstructed result. Quantitative and qualitative evaluations of four public datasets show that the proposed method can achieve better performance compared with the state-of-the-art SR methods.

Extensive evidence suggests the feasibility of lie detection using electroencephalograms (EEGs). However, it is largely unknown whether there are any differences in the nonlinear features of EEGs between guilty and innocent subjects. In this study, we proposed a complexity-based method to distinguish lying from truth telling. A total of 35 participants were randomly divided into two groups, and their EEG signals were recorded with 14 electrodes. Averages for sequential sets of five trials were first calculated for the probe responses within each subject. Next, a common wavelet entropy (WE) measure and an improved one were used to quantify complexity from each five-trial average. The results show that for both measures, the WE values in the guilty subjects are statistically lower than those in the innocent subjects for most of the 14 electrodes. More importantly, using the improved measure, the difference in WE between the two groups of subjects significantly increases for 11 brain regions compared with the values from the common measure. Finally, the highest balanced classification accuracy, 89.64%, is achieved when using the combined WE feature vector in five brain regions from the sites of Pz, P3, C4, Cz, and C3. Our findings indicate that the lying task elicits a more ordered brain activity in some specific brain regions than the task of telling the truth. This study not only demonstrates that improved WE measurements could be a powerful quantitative index for detecting lying but also sheds light on the brain mechanisms underlying deceptive behaviors.

Multimodal medical image fusion, emerging as a hot topic, aims to fuse images with complementary multi-source information. In this paper, we propose a novel multimodal medical image fusion method based on structural patch decomposition (SPD) and fuzzy logic technology. First, the SPD method is employed to extract two salient features for fusion discrimination. Next, two novel fusion decision maps called an incomplete fusion map and supplemental fusion map are constructed from salient features. In this step, the supplemental map is constructed by our defined two different fuzzy logic systems. The supplemental and incomplete maps are then combined to construct an initial fusion map. The final fusion map is obtained by processing the initial fusion map with a Gaussian filter. Finally, a weighted average approach is adopted to create the final fused image. Additionally, an effective color medical image fusion scheme that can effectively prevent color distortion and obtain superior diagnostic effects is also proposed to enhance fused images. Experimental results clearly demonstrate that the proposed method outperforms state-of-the-art methods in terms of subjective visual and quantitative evaluations.

Cyber physical systems (CPS) is a tightly coupled integration and interaction between computational and physical components. In many cases, information collection in CPS is provided through a group of distributed sensors and all of them change continuously with time. Thus the sensor information is usually in the form of time series. One particularly interesting application in time series analysis is use of complex networks to represent and study behaviors of system. Complex networks has been playing an important role for analyzing complex systems as it helps understanding the topology structure of systems with different interacting units. In this paper, we proposed a reliable method for constructing complex networks from multivariate time series (MTSs) in the cases of single and multisensor based on information geometry theory, which allows the information in the time series to be extracted by analyzing the associated complex network. We first estimate covariance matrices and then a geodesic-based distance between the covariance matrices is introduced. Consequently, the network can be constructed on a Riemannian manifold where the nodes and edges correspond to the covariance matrix and the geodesic-based distance, respectively. The proposed method provides us with a nonlinear relationship and intrinsic geometry viewpoint to understand the MTSs and also an alternative approach to fuse, model, represent, and visualize the multisensor data in CPS. A number of experimental studies and numerical examples are presented to demonstrate the generality and the effectiveness of our approach with both synthetic and real datasets.

Remote sensing image fusion has a potential spectral distortion problem due to the global/local spectral and spatial correlations between panchromatic (PAN) and multispectral (MS) images. To overcome this problem, in this letter, a compensation details-based injection (CDI) fusion model is presented from a new perspective of compensatory learning. In contrast to the traditional method, the two categories of details, namely, the PAN details and the CD, are considered to compensate for the spatial and spectral differences between low-resolution MS (LRMS) and high-resolution MS images. To obtain the CD, a robust sparse representation was employed to calculate the difference between the PAN and MS images during the fusion. The CD combined with the PAN details extracted by a multiscale-guided filter are then injected into the upsampled LRMS image to achieve a fused image. Extensive experiments were undertaken on several image data sets, and the results demonstrate the effectiveness of the proposed CDI method.

The organization of the brain functional network is associated with mental fatigue, but little is known about the brain network topology that is modulated by the mental fatigue. In this study, we used the graph theory approach to investigate reconfiguration changes in functional networks of different electroen-cephalography (EEG) bands from 16 subjects performing a simulated driving task. Behavior and brain functional networks were compared between the normal and driving mental fatigue states. The scores of subjective self-reports indicated that 90 min of simulated driving-induced mental fatigue. We observed that coherence was significantly increased in the frontal, central, and temporal brain regions. Furthermore, in the brain network topology metric, significant increases were observed in the clustering coefficient (Cp) for beta, alpha, and delta bands and the character path length (Lp) for all EEG bands. The normalized measures γ showed significant increases in beta, alpha, and delta bands, and λ showed similar patterns in beta and theta bands. These results indicate that functional network topology can shift the network topology structure toward a more economic but less efficient configuration, which suggests low wiring costs in functional networks and disruption of the effective interactions between and across cortical regions during mental fatigue states. Graph theory analysis might be a useful tool for further understanding the neural mechanisms of driving mental fatigue.

With various emerging mobile devices, the visual content have be to resized into different sizes or aspect ratios for good viewing experiences. In this paper, we propose a new multioperator retargeting algorithm by using four retargeting operators of seam carving, cropping, warping, and scaling iteratively. To determine which retargeting operator should be used at each iteration, we adopt structural similarity (SSIM) to evaluate the similarity between the original and retargeted images. The retargeting operator sequence is constructed based on the four types of retargeting operators by an optimization process. Since the sizes of original and retargeted images are different, scale-invariant feature transform flow is used for dense correspondence between the original and retargeted images for similarity evaluation. Additionally, visual saliency is used to weight SSIM results based on the characteristics of the human visual system. Experimental results on a public image retargeting database have shown the promising performance of the proposed multioperator retargeting algorithm.

It is very challenging to accurately detect smoke from images because of large variances of smoke colour, textures, shapes and occlusions. To improve performance, the authors combine dual threshold AdaBoost with staircase searching technique to propose and implement an image smoke detection method. First, extended Haar‐like features and statistical features are efficiently extracted from integral images from both intensity and saturation components of RGB images. Then, a dual threshold AdaBoost algorithm with a staircase searching technique is proposed to classify the features of smoke for smoke detection. The staircase searching technique aims at keeping consistency of training and classifying as far as possible. Finally, dynamic analysis is proposed to further validate the existence of smoke. Experimental results demonstrate that the proposed system has a good robustness in terms of early smoke detection and low false alarm rate, and it can detect smoke from videos with size of 320 × 240 in real time.