Multimodal retrieval-augmented generation (RAG) systems enhance large vision-language models by integrating cross-modal knowledge, enabling their increasing adoption across real-world multimodal tasks. These knowledge databases may contain sensitive information that requires privacy protection. However, multimodal RAG systems inherently grant external users indirect access to such data, making them potentially vulnerable to privacy attacks, particularly membership inference attacks (MIAs). Existing MIA methods targeting RAG systems predominantly focus on the textual modality, while the visual modality remains relatively underexplored. To bridge this gap, we propose MrM, the first black-box MIA framework targeted at multimodal RAG systems. It utilizes a multi-object data perturbation framework constrained by counterfactual attacks, which can concurrently induce the RAG systems to retrieve the target data and generate information that leaks the membership information. Our method first employs an object-aware data perturbation method to constrain the perturbation to key semantics and ensure successful retrieval. Building on this, we design a counterfact-informed mask selection strategy to prioritize the most informative masked regions, aiming to eliminate the interference of model self-knowledge and amplify attack efficacy. Finally, we perform statistical membership inference by modeling query trials to extract features that reflect the reconstruction of masked semantics from response patterns. Experiments on two visual datasets and eight mainstream commercial visual-language models (e.g., GPT-4o, Gemini-2) demonstrate that MrM achieves consistently strong performance across both sample-level and set-level evaluations, and remains robust under adaptive defenses.

Restoring degraded images has long been a significant focus in image preprocessing, aiming to recover high-quality images from their degraded counterparts. While numerous studies have addressed specific types of degradation, All-in-One methods have achieved a notable milestone by enabling a single model to handle various degradations without requiring prior knowledge of the degradation types. However, these approaches often fall short of delivering satisfactory performance in complex and multiple degradation scenarios. To address these challenges, this article proposes a Multi-Prompt Guided Diffusion Network for All-in-One image restoration, referred to as MPGNet. MPGNet consists of two key components: the Image Restoration Part (IRP) and the Prompt Generation Part (PGP). The IRP employs a diffusion model to perform the image restoration task, while the PGP generates tailored prompts to guide image restoration from degradation. By leveraging meticulously designed prompts, MPGNet empowers the diffusion network to adaptively handle complex and multiple degradation scenarios. To enhance adaptability to various degradation types, content specificity, and overall performance, we design three types of prompts: Style Prompt, Content Prompt, and Learnable Prompt. The first two prompts are carefully generated using the Image Encoder from the pre-trained vision-text model CLIP, with contrastive learning applied during respective training, while the Learnable Prompt is custom-designed separately. We also introduce a Prompt Encoder that integrates Style, Content, and Learnable Prompts into a unified representation, facilitating image restoration for IRP. The experimental results demonstrate that the proposed method significantly outperforms recent approaches in both single and multiple degradation scenarios.

Multi-object tracking faces a major challenge in handling the variations of tracked targets within complex scenes. In existing transformer-based tracking methods, typically each tracked target is only associated with one track query. However, trajectories in crowded scenes often experience varying levels of occlusion, making the association brittle for using a single track query to identify the tracked target. Therefore, we argue that relying on a single track query to track a target in complex scenes is inadequate. In this paper, we introduce TGFormer, with the core idea of designing a Track Query Group for each tracked target. Each group encompasses track queries that handle the same tracked target across different levels of occlusion scenes. To achieve long-term robust association, we propose a novel updater that integrates temporal memories and occlusion-aware features to update the Track Query Group, ensuring the tracked target can be consistently captured in complex scenes. Additionally, we introduce a Position Predictor that allows TGFormer to forecast motion trends, helping the model accurately locate moving tracklets. Experimental results show that our method achieves competitive performance on the MOT Challenge and DanceTrack datasets.

Infrared and visible image fusion (IVIF) aims to extract fine details from visible images and complementary information from infrared images. Most existing methods directly extract relevant and complementary features from each modality using neural networks, often overlooking the guidance process and the distinct frequency-domain characteristics of these features. To address this, we propose HRFusion-a novel frequency-domain framework that extracts complementary features from hybrid features using prior-constrained relevant features, effectively enhancing complementary information and reducing redundancy. In HRFusion, hybrid and relevant features are robustly extracted to guide the subsequent fusion stage. By leveraging frequency differences between complementary and relevant features, we introduce the Enhanced Complementary Frequency Network (ECFNet), which uses optimized Variational Mode Decomposition (VMD) to effectively separate and process these signals for fusion. The overall architecture is built with the proposed DTBlock, which captures both global and local features. Extensive experiments show that our method achieves state-of-the-art performance on the TNO, MSRS, M3FD, and Harvard Brain datasets, outperforming recent approaches. Code is available at https://github.com/liuuuuu777/HRFusion.

Monocular depth estimation stands as a fundamental pursuit in computer vision. Recently, some methods have attempted to introduce the text-to-image diffusion model into the domain of monocular depth estimation and achieved impressive results. However, these methods typically employ pre-defined templates as text prompts to guide the learning of denoising networks, resulting in limited flexibility and scalability. In this paper, we propose OGDepth, a diffusion-based monocular depth estimation network with object prompts generated by taking advantage of the object detection information from the scene. Specifically, we design an Object Prompt Module (OPM) to encode the object detection information into prompts that are more closely aligned with the image content, offering richer contextual information while circumventing the monotony and redundancy inherent in template-generated prompts. Moreover, we employ bounding box information for each object to filter and localize objects, enabling the model to grasp relative positional information within the scene. This facilitates the creation of a more precise depth map. Additionally, we design a Global-Local Interaction Decoder (GLID) to facilitate the mutual exchange of features at different scales, enabling efficient feature fusion. Our approach underwent rigorous experiments across multiple datasets, with results showcasing its state-of-the-art performance. Notably, on the KITTI dataset, our model achieves an RMSE of 1.967 and a REL of 0.047, and both metrics are the best among all compared methods. On the NYU Depth V2 dataset, our method achieves an RMSE score of 0.221, representing a notable 12.9% enhancement compared to the baseline method (VPD).

Stereo matching plays an important role in advancing substantial applications such as 3D reconstruction, autonomous driving, depth sensing, and has recently made tremendous progress. It still poses a significant challenge, however, in the extraction of disparities from a pair of rectified images to achieve accurate stereo matching. In this paper, we propose an attention-fused cost volume construction and progressive cost aggregation network, namely AP-Net, to elevate the accuracy of stereo matching. Specifically, we introduce attention fusion to learn feature similarity between corresponding points on the left and right images while capturing long-dependent contextual information to construct attention-fused cost volumes. Moreover, we propose progressive cost aggregation, which consists of two stages to fully exploit information from different cost volumes. Furthermore, to make more accurate adjustments to the initial disparity, we implement an information entropy-guided disparity refinement module, which leverages information entropy as an explicit confidence measure of the initial disparity estimation to guide the refinement module in calculating the disparity residual more accurately and improving the overall accuracy of disparity estimation. Our implemented disparity refinement module can also be seamlessly embedded into a variety of stereo matching networks, significantly improving the model’s performance with only a small increase in computation and parameters. Experimental results across various stereo datasets confirm that our proposed AP-Net consistently delivers competitive performance. The code is available at https://github.com/zhuys-bupt/AP-Net .

Recent online Transformer-based multi-object tracking methods achieve end-to-end optimization by jointly performing detection and association. However, these trackers apply query augmentations uniformly to detect queries and track queries during training, which may limit their ability to fully exploit distinct features. The different augmentations serve distinct purposes and may have conflicting effects on detection and association. Moreover, the detect queries and the track queries with augmentations contain different semantic information. Jointly feeding these queries into self-attention modules for feature interaction may lead to suppression between the two types of queries. In this article, we present a novel Transformer-based end-to-end model, DQFormer, which mitigates conflicts and effectively learns task-specific features through the proposed Decoupled Query Augmentation (DQA) strategy. DQA categorizes query augmentations into separate detection and association branches, generating more discriminative queries tailored specifically for detection and association. To align with DQA, we decompose the self-attention module into a Dual Cross-Attention module. Furthermore, a Targeted Label Assignment strategy is applied to the augmented queries in each cross-attention module, helping the model to learn distinct features for tracking. DQFormer achieves competitive detection and association results across the MOT17, MOT20, and DanceTrack datasets.

Recently, the popular RGB-based 6D pose estimation methods follow the two-stage paradigm, in which the sparse or dense 2D-3D correspondences between the RGB image and the CAD model are first established, and then the 6D pose is estimated based on the PnP/RANSAC algorithm or its variants. Although such methods are considered effective and achieve excellent performance, this two-stage pipeline cannot be end-to-end trainable and is time-consuming. In this work, we propose a one-stage Dual-Channel Feature Fusion Network for End-to-End 6D Pose Estimation (DFFNet), in which an Implicit Pose Feature Extraction Channel is designed to extract the implicit pose features inside each 2D-3D correspondence and an Implicit Topological Feature Extraction Channel is designed to extract the geometric topological features implicitly expressed by each group of 2D-3D correspondences. The different features extracted by the two channels are fused to regress the 6D pose of the object. DFFNet is generic and can be combined with the existing sparse 2D-3D correspondence-extraction network to eliminate the cumbersome RANSAC process and form an overall end-to-end trainable framework. Experiments show that our proposed method outperforms the one-stage and some other two-stage 6D pose estimation methods.

The outstanding performance of deep convolutional neural networks comes from their effective extraction and learning ability. Although binary neural networks (BNNs) have the obvious advantages of low storage and high efficiency over their full-precision counterparts on resource-constrained hardware devices, the accuracy degradation brought by binary quantization is still an unavoidable problem. The activation distribution in BNNs is a key factor affecting network performance. To elevate the accuracy of BNNs, in this paper, we propose to regulate the activation distribution to strengthen the representation ability of BNNs. We first propose an Information Entropy enhancement Basic block (IEBlock) to build a competitive baseline model with higher information entropy of output activation distribution. Specifically, we build the IEBlock by deliberately reorganizing the position of the elements in the normal basic block based on a deep analysis of the information flow. After that, we propose a Depth-aware Activation Distribution Amendment (DADA) module, which learns the interdependencies of feature channels to amend the activation distribution with information loss after binary convolution. Extensive experiments demonstrate that our method effectively improves the information entropy of binary activations and elevates the accuracy of BNNs. Our method has outperformed the state-of-the-art methods on CIFAR-10 and ImageNet datasets. Code is available at: https://github.com/tomorrow-rain/RAD-BNN

Memristor based crossbar (MBC) can execute neural network computations in an extremely energy efficient manner. However, stuck-at faults make memristors cannot represent network weight correctly, thus degrading classification accuracy of the network deployed on the MBC significantly. By carefully analyzing all the possible fault combinations in a pair of differential crossbars, we found that most of the stuck-at faults can be accommodated perfectly by mapping a zero value weight onto the memristors. Based on such observation, in this paper we propose a target sparsifying based fault tolerant scheme for the MBC which executes neural network applications. We first exploit a heuristic algorithm to map weight matrix onto the MBC, aiming at minimizing weight variations in the presence of stuck-at faults. After that, some weights mapped onto the faulty memristors which still have large variations will be purposefully forced to zero value. Network retraining is then performed to recover classification accuracy. For a 4-layer CNN designed for MNIST digit recognition, experimental results demonstrate that our scheme can achieve almost no accuracy loss when 10% of memristors in the MBC are faulty. As the faulty memristors increasing to 20%, accuracy loss is only within 3%.

Due to the complexity of fabrication and bonding processes, TSVs are susceptible to delay defects and have become a serious concern in 3D ICs. To assure the quality and yield of 3D ICs, it is imperative to conduct effective delay testing for TSVs. In this paper, we present a novel on-chip ring oscillator based TSV delay measurement scheme for TSV delay defect detection. In the proposed scheme, three TSVs and the associated logic gates are implemented to construct a basic TSV propagation delay measurement unit. By configuring the control signals, three different oscillators can be created in the unit. By firstly measuring the oscillation periods of the three oscillators, the propagation delay of the target TSV can then be calculated with high resolution. Experimental results are presented to verify the effectiveness of the proposed scheme.

Voltage-frequency island (VFI)-based design has been widely exploited for optimizing system energy of embedded multicore chip in recent years. The existing work either constructed a single static VFI partition for all kinds of applications or required per-core voltage domain configuration. However, the former solution is hard to find one optimal VFI partition for diverse applications while the latter one suffers from high hardware cost. In this article, we propose a cost effective energy optimization framework based on dynamically reconfigurable VFI (D-VFI). Our framework treats a small number of cores as dynamic cores (D-cores) and configures each of them with an independent voltage domain. At runtime, the D-cores can be pieced together with neighboring static VFIs by scaling their operating voltages. This can dynamically construct the optimal VFI partitions for different kinds of applications, thus achieving more aggressive energy optimization under low cost. To identify the D-cores, we propose a rules constrained task scheduling and VFI partitioning algorithm. Moreover, we analyze the task schedules to determine the optimal scaling intervals which can accommodate voltage scaling induced latency. Experimental results demonstrate that the effectiveness of the proposed scheme.

On-chip faster-than-at-speed delay testing provides a promising way for small delay defect detection. However, the frequency of on-chip generated test clock would be impacted by process variations. Hence, it requires determining the actual frequency of generated test clock to ensure the effectiveness of faster-than-at-speed delay testing. In this paper, we present a novel test clock generation scheme, namely Enhanced LCCG, for faster-than-at-speed delay testing. In the proposed scheme, faster-than-at-speed test clock is firstly generated by configuring the corresponding control information specified in the test pattern into Enhanced LCCG. Then, by constructing oscillation paths and counting the corresponding oscillation iteration numbers, the actual frequency of test clock can be measured and calculated with high resolution. Experimental results are presented to validate the proposed method.

Various types of defects are prone to be occurred inside the TSV during the manufacturing and bonding steps, thereby severely impacting the yield of 3D-stacked ICs. Moreover, several types of TSV defects are latent and may easily escape detection during the manufacturing test. However, these latent TSVs are prone to degrade during the field operation and may eventually become faulty and then destroy the entire 3D-stacked IC. To tackle the above problems, in this paper, we present an effective TSV self-repair scheme for 3D-stacked ICs. By designing redundant TSVs and a TSV self-repair architecture, the proposed scheme can effectively repair faulty TSVs detected by manufacturing test for improving the yield of 3D-stacked ICs. Moreover, the latent TSVS failed and then detected during the in-field operation can also be self-repaired, thereby elevating the 3D ICs' quality and reliability. Experimental results are presented to validate the proposed method.

Energy efficiency is a primary design concern for embedded multiprocessor system-on-chips (MPSoCs). Recently, Voltage-Frequency Island (VFI) - based design paradigm was introduced for fine-grained power management, which can seamlessly combine with the task scheduling algorithm to optimize system energy. However, the ever-increasing variabilities cause large uncertainty on delay and power. Such statistical nature in performance parameters easily makes deterministic energy optimization hard to achieve desirable performance yield, defined as the probability of the design meeting timing constraints of the system. In this paper, we propose a variation-aware statistical energy optimization framework, which takes account of performance yield constraints in energy-aware task scheduling, voltage assignment and VFI partitioning process. Energy optimization sensitivity, defined as energy variations of the task under voltage scaling, combines with the statistical slack of the task to guide the overall optimization flow. Experimental results demonstrate the effectiveness of the proposed scheme.

In this paper, we present a novel on-chip path delay measurement architecture for efficiently detecting and debugging of delay faults in the fabricated integrated circuits. Several delay stages are employed in the proposed on-chip path delay measurement (OCDM) circuit, whose delay ranges are increased by a factor of two gradually from the last to the first delay stage. Thus, the proposed OCDM circuit can achieve a large delay measurement range with a small quantity of delay stages. A calibration circuit is incorporated into the proposed on-chip path delay measurement technique to calibrate the delay range of the delay stage under process variations. In addition, delay calibration for import lines is conducted to improve the precision of path delay measurement. Experimental results are presented to validate the proposed path delay measurement architecture.

We propose a flip-flop selection method to reduce the overall volume of transition delay test data, by replacing a small number of selected regular scan cells with enhanced scan cells. Relative measures are presented to reflect the gains when controlling a scan cell to a certain value, and guide the scan cell selection. Experimental results on larger IWLS 2005 benchmark circuits show that, to achieve the same fault coverage of the pure launch on capture (LOC) approach, the volume of test data can be reduced to a half on average by replacing only 1% of regular scan cells to enhanced scan cells. The transition delay fault coverage can also be improved using the proposed method with equally low area overhead.

This paper proposes a unified delay test architecture, in which the design resources for on-line delay fault detection can be reused to support off-line delay testing. A stability checker, which has low hardware overhead, is presented to monitor the stability violation from each critical combinational output. A global error generator, which is shared among stability checkers, can produce a global error signal from individual stability checkers to indicate whether a delay fault appears. A local scan enable generator is incorporated into the scan chain to support scan-based off-line delay testing. Experimental results are presented to validate the effectiveness of the proposed approach.

Faster-than-at-speed testing provides an effective way for detecting and debugging small delay defects in modern fabricated chips. However, the use of external automatic test equipment for faster-than-at-speed delay testing could be costly. In this paper, we present an on-chip clock generation scheme which facilitates faster-than-at-speed delay testing for both launch on capture and launch on shift test frameworks. The required test clock frequency with a high resolution can be obtained by specifying the information in the test patterns, which is then shifted into the delay control stages to configure the launch and capture clock generation circuit (LCCG) embedded on-chip. Similarly, the control information for selecting various test frameworks and clock signals can also be embedded in the test patterns. Experimental results are presented to validate the proposed scheme.

In this paper, we present a novel on-chip path delay measurement circuit for efficiently detecting and debugging of delay faults in the fabricated integrated circuits. Several delay stages are employed in the proposed circuit, whose delay ranges are increased by a factor of two gradually from the last to the first delay stage. Thus, the proposed method can achieve a large delay measurement range with a small quantity of delay stages. Experimental results show that a significant reduction in both delay measurement time and area overhead can be obtained compared to the previous Vernier Delay Line based delay measurement schemes. In addition, by conducting delay compensation, the proposed method can achieve both improved delay measurement resolution and measurement accuracy.

Enhanced scan delay testing approach can achieve high transition delay fault coverage by a small size of test pattern set but with significant hardware overhead. Although the implementation cost of launch on capture (LOC) approach is relatively low, the generated pattern set for testing delay faults is typically very large. In this paper, we present a novel flip-flop selection method to combine the respective advantages of the two approaches, by replacing a small number of selected regular scan cells with enhanced scan cells, thus to reduce the overall volume of transition delay test patterns effectively. Moreover, higher fault coverage can also be obtained by this approach compared to the standard LOC approach. Experimental results on larger ISCAS-89 and ITC-99 benchmark circuits using a commercial test generation tool show that the volume of test patterns can be reduced by over 70% and the transition delay fault coverage can be improved by up to 8.7%.