We consider a critical issue of false negatives in Vision-Language Pre-training (VLP), a challenge that arises from the inherent many-to-many correspondence of image-text pairs in large-scale web-crawled datasets. The presence of false negatives can impede achieving optimal performance and even lead to a significant performance drop. To address this challenge, we propose MAFA (MAnaging FAlse negatives), which consists of two pivotal components building upon the recently developed GRouped mIni-bcTch sampling (GRIT) strategy: 1) an efficient connection mining process that identifies and converts false negatives into positives, and 2) label smoothing for the image-text contrastive (ITC) loss. Our comprehensive experiments verify the effectiveness of MAFA across multiple downstream tasks, emphasizing the crucial role of addressing false negatives in VLP, potentially even surpassing the importance of addressing false positives. In addition, the compatibility of MAFA with the recent BLIP-family model is also demonstrated. Code is available at https://github.com/jaeseokbyun/MAFA.

We consider the challenging blind denoising problem for Poisson-Gaussian noise, in which no additional information about clean images or noise level parameters is available. Particularly, when only "single" noisy images are available for training a denoiser, the denoising performance of existing methods was not satisfactory. Recently, the blind pixelwise affine image denoiser (BP-AIDE) was proposed and significantly improved the performance in the above setting, to the extent that it is competitive with denoisers which utilized additional information. However, BP-AIDE seriously suffered from slow inference time due to the inefficiency of noise level estimation procedure and that of the blind-spot network (BSN) architecture it used. To that end, we propose Fast Blind Image Denoiser (FBI-Denoiser) for Poisson-Gaussian noise, which consists of two neural network models; 1) PGE-Net that estimates Poisson-Gaussian noise parameters 2000 times faster than the conventional methods and 2) FBI-Net that realizes a much more efficient BSN for pixelwise affine denoiser in terms of the number of parameters and inference speed. Consequently, we show that our FBI-Denoiser blindly trained solely based on single noisy images can achieve the state-of-the-art performance on several real-world noisy image benchmark datasets with much faster inference time (× 10), compared to BP-AIDE.

Convolutional neural network (CNN)-based denoisers are recently shown to overwhelm the denoising performances of the conventional prior- or optimization-based methods, particularly for the additive white Gaussian noise (AWGN) case. However, those typical CNN-based denoisers had two main limitations: (1) the superb denoising performances for AWGN do not necessarily carry over to the signal-dependent real-noise cases and (2) the clean source images are required as targets for supervised training. Both limitations are critical in practical real-noise denoising setting, since neither the noise characteristics nor the clean source images are readily available for supervised training. While some recent work proposed CNN denoisers that are trained solely with noisy images, they still fall short of the classical baselines, e.g., BM3D, for blind real-noise denoising settings, in which nothing is known about the noise and only noisy images are available for training. In this paper, we propose Blind Pixelwise Affine Image DEnoiser (BP-AIDE), which tackles above two limitations by combining variance stabilizing transformation (VST) technique with a recently developed unbiased estimate of the mean-squared error (MSE) for a pixelwise affine denoiser. As a result, we show BP-AIDE can be blindly trained solely with single real-noise corrupted images and, to the best of our knowledge, first outperforms both CNN-based and conventional denoisers for the same setting, on several real-world noisy image benchmark datasets.