Recent advances in causal interpretability have extended from language models to vision-language models (VLMs), seeking to reveal their internal mechanisms through input interventions. While textual interventions often target semantics, visual interventions typically rely on coarse pixel-level perturbations, limiting semantic insights on multimodal integration. In this study, we introduce V-SEAM, a novel framework that combines Visual Semantic Editing and Attention Modulating for causal interpretation of VLMs. V-SEAM enables concept-level visual manipulations and identifies attention heads with positive or negative contributions to predictions across three semantic levels: objects, attributes, and relationships. We observe that positive heads are often shared within the same semantic level but vary across levels, while negative heads tend to generalize broadly. Finally, we introduce an automatic method to modulate key head embeddings, demonstrating enhanced performance for both LLAVA and InstructBLIP across three diverse VQA benchmarks. Our data and code are released at: https://github.com/petergit1/V-SEAM.

In image denoising research, discriminative models have achieved impressive results which mainly owes to the powerful ability of convolutional networks in local feature extraction. However, there is still room for improvement due to insufficient utilization of global information. Although using fully connected layers or increasing network depth can supplement global information, this results in a significant increase in parameters and computational cost. To address these issues, we propose a global information compensation network (GICN) for image denoising in this paper. Firstly, at the shallow network part, we propose a global feature mining block that enhances the network's ability to extract global information by combining non-local blocks and the Fourier transform while improving the interpretability of the model. Secondly, between the encoder and decoder, we propose a cross-scale feature aggregation block to fuse information at different scales. Finally, we employ attention blocks to improve skip connections to better capture long-distance dependencies. Extensive experimental results show that our proposed GICN effectively compensates for global information, achieves a balance between denoising efficiency and effect, and surpasses mainstream methods in multiple benchmark tests.

Convolutional neural networks (CNN) have achieved remarkable performance in image denoising. However, most existing CNNs cannot accurately capture and remove tiny noises during the denoising process and lose edge detail information easily. In this paper, we propose a fine-grained residual network guided by wavelet and adaptive coordinate attention (WACAFRN) for image denoising. Firstly, we propose an adaptive coordinate attention mechanism and combine it with cascaded Res2Net residual blocks to form an encoder network for more accurate noise removal. Secondly, we propose a wavelet attention mechanism that combines global and local residual blocks to form a decoder network, aiming to address the problem of edge detail information loss. At last, we complement the noise information through a noise estimation block to further enhance the model’s ability to adapt to noise. Extensive experiment results demonstrate that our proposed method outperforms existing denoising methods in both qualitative and quantitative aspects. Notably, our method significantly improves real-world noise removal tasks on the CC dataset, with an average increase of 2.08 dB in PSNR and 0.0264 in SSIM over the state-of-the-art methods. Additionally, WACAFRN exhibits faster inference speeds, underscoring its efficiency in real-world applications.

Image denoising methods based on convolutional neural networks have been popular and achieved relatively excellent performance. However, most of the existing methods cannot fully obtain and use the shallow feature information when removing noise, and cannot better combine information between various network layers. In this paper, we propose an image denoising algorithm based on a feature enhancement network and multi-scale convGRU, named a shallow feature enhancement and multi-scale convGRU denoising network (SFEMGN), through an in-depth study of convolutional networks and GRU networks. We first propose a feature enhancement block to extract richer shallow features and enhance the protection of image details. Furthermore, the proposed SFEMGN integrates a multi-scale convolution GRU module, which can combine spatial features and temporal features at the same time. Comparative experiments and ablation studies demonstrate that our proposed model can achieve competitive performance in both gray and color image denoising tasks.

OBJECTIVE: Accurate segmentation of liver tumors, which could help physicians make appropriate treatment decisions and assess the effectiveness of surgical treatment, is crucial for the clinical diagnosis of liver cancer. In this study, we propose a 4-dimensional (4D) deep learning model based on 3D convolution and convolutional long short-term memory (C-LSTM) for hepatocellular carcinoma (HCC) lesion segmentation. METHODS: The proposed deep learning model utilizes 4D information on dynamic contrast enhanced (DCE) magnetic resonance imaging (MRI) images to assist liver tumor segmentation. Specifically, a shallow U-net based 3D CNN module was designed to extract 3D spatial domain features from each DCE phase, followed by a 4-layer C-LSTM network module for time domain information exploitation. The combined information of multi-phase DCE images and the manner by which tissue imaging features change on multi-contrast images allow the network to more effectively learn the characteristics of HCC, resulting in better segmentation performance. RESULTS: The proposed model achieved a Dice score of 0.825± 0.077, a Hausdorff distance of 12.84± 8.14 mm, and a volume similarity of 0.891± 0.080 for liver tumor segmentation, which outperformed the 3D U-net model, RA-UNet model and other models in the ablation study in both internal and external test sets. Moreover, the performance of the proposed model is comparable to the nnU-Net model, which showed state-of-the-art performance in many segmentation tasks, with significantly reduced prediction time. CONCLUSION: The proposed 3D convolution and C-LSTM based model can achieve accurate segmentation of HCC lesions.

Counters have been widely adopted in modern circuit design. Also, they have been used to facilitate the implementation of hardware Trojan. It is necessary to reverse engineer the counter design from a chip so as to detect the possible Trojan. However, the existing reverse engineering technique is expensive and intrusive. In this work, we first propose to rely on the scan dump data to identify the registers included in a counter. In this scheme, the scan data can be collected by operating the chip in testing mode, thus the proposed scheme never impacts or impairs a chip as the traditional reverse engineering techniques. The regularity of data from different types of counters is first analyzed. The relationship between the data from a pair of registers is explored so that the counter registers can be distinguished from other normal registers. The proposed method is applied on several circuits containing counters from OpenCores. The experiment results show the proposed method can accurately identify the counter registers with a perfect true positive rate and true negative rate.

Scan-based side-channel attack has become a new threat to cryptographic chips. Many countermeasures are proposed to safeguard scan design against the scan-based attacks. Among which, the methods based on lock and key scheme present more effective and popular. In this paper, we propose a new lock and key scheme which adopts physical unclonable function (PUF) design to generate a unique key for each design. The uniqueness of PUF enables each chip taped out from one mask to possess a different golden key. Once the PUF is invoked for the first time, the PUF response will be hardcoded into the design so that even the environment changes, the PUF-based key maintains. The proposed secure scan design with PUF-based key can protect the cryptographic chips against all known scan-based side-channel attacks while incurring negligible overhead.