Recent studies have discovered that widely used text-to-image diffusion models can replicate training samples during image generation, a phenomenon known as memorization. Existing detection methods primarily focus on identifying memorized prompts. However, in real-world scenarios, image owners may need to verify whether their proprietary or personal images have been memorized by the model, even in the absence of paired prompts or related metadata. We refer to this challenge as image-level memorization detection, where current methods relying on original prompts fall short. In this work, we uncover two characteristics of memorized images after perturbing the inference procedure: lower similarity of the original images and larger magnitudes of TCNP. Building on these insights, we propose Inversion-based Inference Perturbation (IIP), a new framework for image-level memorization detection. Our approach uses unconditional DDIM inversion to derive latent codes that contain core semantic information of original images and optimizes random prompt embeddings to introduce effective perturbation. Memorized images exhibit distinct characteristics within the proposed pipeline, providing a robust basis for detection. To support this task, we construct a comprehensive setup for the image-level memorization detection, carefully curating datasets to simulate realistic memorization scenarios. Using this setup, we evaluate our IIP framework across three different memorization settings, demonstrating its state-of-the-art performance in identifying memorized images in various settings, even in the presence of data augmentation attacks.

Diffusion model based language-guided image editing has achieved great success recently. However, existing state-of-the-art diffusion models struggle with rendering correct text and text style during generation. To tackle this problem, we propose a universal self-supervised text editing diffusion model (DiffUTE), which aims to replace or modify words in the source image with another one while maintaining its realistic appearance. Specifically, we build our model on a diffusion model and carefully modify the network structure to enable the model for drawing multilingual characters with the help of glyph and position information. Moreover, we design a self-supervised learning framework to leverage large amounts of web data to improve the representation ability of the model. Experimental results show that our method achieves an impressive performance and enables controllable editing on in-the-wild images with high fidelity. Our code will be avaliable in \url{https://github.com/chenhaoxing/DiffUTE}.

In this work, we propose Geometry Normal Consistency Loss (GNC) for 3D face reconstruction and dense alignment. The existing methods based on the strong constraints of the 3DMM parameter regression only consider reducing the error between 68 landmarks, while they rarely consider the geometric contour structure relation of the face. Instead, we take into account the discrete 68 landmarks as loss constrain by introducing geometry area and normal consistency loss, which naturally defines the holistic and local geometric contour structure of the face. In detail, we select the inverted triangle formed by the leftmost and rightmost landmarks on the cheek and the lowest point of the chin to globally constrain the entire facial feature. In addition, we triangulate the facial landmarks to construct triangular patches, and calculate the normals of the patches, which aims to make use of normal consistency between the local patch and the corresponding ground truth to reconstruct rich local details. Extensive experimental results on AFLW2000-3D and AFLW datasets demonstrate that our GNC achieves compelling performance compared to the state of the arts.

Multi-view stereo based on deep learning is mostly dedicated to improving the accuracy of point clouds, whlile complex scenes, occlusion, and other factors limit their reconstruction completeness, especially in the area with drastic changes in depth direction. In this paper, we propose a multi-view stereo network based on depth edge flow (DEF-MVSNet), using the reference image as a guide to dynamically infer the edge coordinates to improve reconstruction completeness. First, we ignore the boundaries in the depth prediction stage to generate better initial depth inference results. Then, we use a EdgeDetect module to extract the obviously features of the reference image and predict the pixel offset of the depth map. Finally, the EdgeFlow module modifies the initial depth map coordinates according to the offset and uses multiple iterations to dynamically update the depth map. The experimental results prove that our method has a great improvement in the completeness of reconstruction compared with MVSNet and R-MVSNet without increasing the memory and time overhead. Code and models are publicly available at https://github.com/linkuizzZ/EF-MVSNet.

3D face reconstruction based on a single image is a longstanding challenging problem in computer vision. Existing end-to-end methods are difficult to reconstruct rich 3D face details. To solve this problem, we propose a two-stream convolutional neural network combined with a face super-resolution method, which can effectively restore the image’s 3D position information. Our method combines an attention fusion mechanism, which can learn the individual attention mapping of each feature subspace, and effectively learn cross-channel information while learning multi-scale and multi-frequency features. Meanwhile, our module obtains the most discriminative features in different local areas, and enhances the consistency and correlation between the attention areas. Experimental results show that our SRCNet has made significant improvements in the 3D face reconstruction and face alignment of the AFLW2000-3D and AFLW datasets.

In this paper we present a novel direct visual-inertial odometry algorithm, for estimating motion in unknown environments. The algorithm utilizes image patches extracted around image features, and formulates measurement residuals in the image intensity space directly. One key characteristic of the proposed method is that it models the true irradiance at each pixel as a random variable to be estimated and marginalized out. The formulation of the photometric residual explicitly accounts for the camera response function and lens vignetting (which can be calibrated in advance), as well as unknown illumination gains and biases, which are estimated on a per-feature or per-image basis. We present a detailed evaluation of our algorithm on 50 datasets with high-precision ground truth, which amount to approximately 1.5 hours of localization data. Through a direct comparison with a point-feature based method, we demonstrate that the use of photometric residuals results in increased pose estimation accuracy, with approximately 23% lower estimation errors, on average.

In this paper, we propose a high-precision pose estimation algorithm for systems equipped with low-cost inertial sensors and rolling-shutter cameras. The key characteristic of the proposed method is that it performs online self-calibration of the camera and the IMU, using detailed models for both sensors and for their relative configuration. Specifically, the estimated parameters include the camera intrinsics (focal length, principal point, and lens distortion), the readout time of the rolling-shutter sensor, the IMU's biases, scale factors, axis misalignment, and g-sensitivity, the spatial configuration between the camera and IMU, as well as the time offset between the timestamps of the camera and IMU. An additional contribution of this work is a novel method for processing the measurements of the rolling-shutter camera, which employs an approximate representation of the estimation errors, instead of the state itself. We demonstrate, in both simulation tests and real-world experiments, that the proposed approach is able to accurately calibrate all the considered parameters in real time, and leads to significantly improved estimation precision compared to existing approaches.

Packet loss is inevitable in video multicast. In this paper, we propose and study an effective feedback-free loss recovery scheme for layered video which combines forward error correction (FEC) and stream replication. In our scheme, the server multicasts the video in parallel with FEC packets and a number of replicated delayed (ReD) version of the stream. Receivers autonomously and dynamically join the FEC and ReD streams to repair their losses. On the server side, we analyze and optimize the number of replicated streams and FEC packets to meet a certain residual loss requirement (i.e., error after correction). On the receiver side, we analyze the optimal combination of FEC and ReD packets to minimize its loss. We also present a fast yet accurate approximation algorithm for receiver to make such decision. We show that FEC combined with merely one or two replicated streams can effectively reduce the residual error rate (by as much as 50%) as compared with pure FEC or replication alone. Both subjective and objective video measures confirm that our recovery scheme achieves much better visual quality.

In video streaming over multicast networks, error recovery is essential to alleviate the effect of packet loss. In this paper, we study a feedback-free recovery scheme which combines the strength of FEC (namely, a parity packet can repair any lost packet) and pseudo-ARQ (namely, incremental recovery). To account for the receiver heterogeneity, the server multicasts layered video streams for the receivers to join. For each layer, the receivers may join dynamically additional multicast channels of FEC and pseudo-ARQ packets to recover local losses. In order to offer quality video, we address the following issues: 1) "menu creation": given a certain maximum error rate after correction (i.e., residual error rate), what is the combination of FEC and pseudo-ARQ packets for the server to send so as to minimize a target receiver's bandwidth; and 2) "menu selection": given the server's menu, what's the combination of FEC and pseudo-ARQ packets for the receiver to join so as to minimize its residual error rate given its local probability, bandwidth and loss pattern. We present the analysis of the scheme and show that our scheme can substantially reduce a receiver's residual error rate as compared with pure FEC or pure pseudo-ARQ alone (by cutting it as much as half).