The rise of the generative models quality during the past years enabled the generation of edited variations of images at an important scale. To counter the harmful effects of such technology, the Image Difference Captioning (IDC) task aims to describe the differences between two images. While this task is successfully handled for simple 3D rendered images, it struggles on real-world images. The reason is twofold: the training data-scarcity, and the difficulty to capture fine-grained differences between complex images. To address those issues, we propose in this paper a simple yet effective framework to both adapt existing image captioning models to the IDC task and augment IDC datasets. We introduce BLIP2IDC, an adaptation of BLIP2 to the IDC task at low computational cost, and show it outperforms two-streams approaches by a significant margin on real-world IDC datasets. We also propose to use synthetic augmentation to improve the performance of IDC models in an agnostic fashion. We show that our synthetic augmentation strategy provides high quality data, leading to a challenging new dataset well-suited for IDC named Syned11The code, weights and dataset are available at https://github.com/gautierevn/BLIP2IDC.

In this paper, we introduce a new transform for image processing, based on wavelets and the lifting paradigm. The lifting steps of a unidimensional wavelet are applied along a local orientation defined on a quincunx sampling grid. To maximize energy compaction, the orientation minimizing the prediction error is chosen adaptively. A fine-grained multiscale analysis is provided by iterating the decomposition on the low-frequency band. In the context of image compression, the multiresolution orientation map is coded using a quad tree. The rate allocation between the orientation map and wavelet coefficients is jointly optimized in a rate-distortion sense. For image denoising, a Markov model is used to extract the orientations from the noisy image. As long as the map is sufficiently homogeneous, interesting properties of the original wavelet are preserved such as regularity and orthogonality. Perfect reconstruction is ensured by the reversibility of the lifting scheme. The mutual information between the wavelet coefficients is studied and compared to the one observed with a separable wavelet transform. The rate-distortion performance of this new transform is evaluated for image coding using state-of-the-art subband coders. Its performance in a denoising application is also assessed against the performance obtained with other transforms or denoising methods.

This paper presents a new image coding technique based on oriented ID multiscale decompositions on a quincunx sampling grid. A transform obtained by adapting the lifting steps of a ID wavelet transform along a local orientation is described. A quad-tree structure is used to efficiently code the orientation map, and optimized in the rate-distortion sense. A gain of up to 1.3 dB over the separable wavelet transform is observed for a set of images, with the rate estimated from the stationary entropy of the subbands.

This paper presents a new coding technique based on a mixed contourlet and wavelet transform. The redundancy of the transform is controlled by using the contourlet at fine scales and by switching to a separable wavelet transform at coarse scales. The transform is then optimized through an iterative projection process in the transform domain in order to minimize the quantization error in the image domain. A gain of respectively up to 0.5 dB and to 1 dB over respectively contourlet and wavelet based coding has been observed for images with directional features.