Particle Image Velocimetry (PIV) aims to infer underlying velocity fields from time-separated particle images, forming a PDE-constrained inverse problem governed by advection dynamics. Traditional cross-correlation methods and deep learning-based feature matching approaches often struggle with ambiguity, limited resolution, and generalization to real-world conditions. To address these challenges, we propose a PIV Neural Operator (PIVNO) framework that directly approximates the inverse mapping from paired particle images to flow fields within a function space. Leveraging a position informed Galerkin-style attention operator, PIVNO captures global flow structures while supporting resolution-adaptive inference across arbitrary subdomains. Moreover, to enhance real-world adaptability, we introduce a self-supervised fine-tuning scheme based on physical divergence constraints, enabling the model to generalize from synthetic to real experiments without requiring labeled data. Extensive evaluations demonstrate the accuracy, flexibility, and robustness of our approach across both simulated and experimental PIV datasets. Our code is at https://github.com/ZXS-Labs/PIVNO.

Existing score distillation methods are sensitive to classifier-free guidance (CFG) scale, manifested as over-smoothness or instability at small CFG scales, while over-saturation at large ones. To explain and analyze these issues, we revisit the derivation of Score Distillation Sampling (SDS) and decipher existing score distillation with the Wasserstein Generative Adversarial Network (WGAN) paradigm. With the WGAN paradigm, we find that existing score distillation either employs a fixed sub-optimal discriminator or conducts incomplete discriminator optimization, resulting in the scale-sensitive issue. We propose the Adversarial Score Distillation (ASD), which maintains an optimizable discriminator and updates it using the complete optimization objective. Experiments show that the proposed ASD performs favorably in 2D distillation and text-to-3D tasks against existing methods. Furthermore, to explore the generalization ability of our paradigm, we extend ASD to the image editing task, which achieves competitive results. The project page and code are at this link.

Current depth completion methods usually employ a pair of calibrated RGB and depth sensors to reconstruct a dense depth map. Although RGB (dense) and depth (sparse) measurements are collected from the same underlying scene, they reflect different physical characteristics and thus it remains rather intricate how the devised RGB guidance scheme can effectively leads to a faithful depth recovery. Different from existing 3D geometry representations, such as point cloud, voxels or meshes, we propose to define 3D scenes as vector-valued functions, f : Ω ∋ (u, v) ↦ (r, g, b, d) ∈ ℝ4, mapping from the image plane Ω to RGBD vectors. This scene function representation brings two benefits: 1) allowing for the adaptation of the Galerkin method to explore the nodal basis of the scene function space, and 2) transforming the irregularly scattered (X, Y, Z) points in the Euclidean space into the depth function defined over the regular grid in the image plane. We further leverage these two benefits within a deep neural network, characterized by an efficient Galerkin attention-based RGBD function embedding to effectively explore the interaction of color and depth information, and by the utilization of equivariant convolution operation on the RGBD feature map as efficient basic blocks. Experiments show that the proposed method achieves significant performance improvement over state-of-the-arts. Code at https://github.com/ZXS-Labs/DCGA.

Recent advances in continuous super-resolution (SR) has made a substantial progress towards universal SR models, which are characterized by using a single deep neural network (DNN) to fulfill arbitrary scale SR tasks. When deployed on resource stringent platforms, however, a trained DNN model usually requires experience-demanding and laborious manual efforts to compress the models following a predetermined compute budget. This paper proposes an inference-time adaptive network width optimization method for arbitrary scale SR modules, dubbed as Scalable Super-Resolution Neural Operator (SSRNO), which is capable of efficient performance-preserving deployment on various mobile or edge devices with only a user input parameter indicating the desired compression rate. SSRNO realizes the continuous parameterization of SRNO (CVPR2023) by virtue of two novel contributions. First, we propose the Integral Neural Network (INN) formulation for the Galerkin type attention, which is an indispensable component for spatial discretization invariant SR neural networks. Second, we further propose an adaptive layer-wise compression rate estimation mechanism, which allows for flexible adaptation to variant capacity through the neural network layers. Extensive experiments validate the outperforming overall performances over existing continuous SR models in terms of reconstruction accuracy, model scalability as well as throughput. For instance, compared with the baseline SRNO, a typical configuration of SSRNO can achieve a model size compression up to 62% and an over 2x speedup in situations where resources are limited, while it can also expand itself to keep the PSNR degradation within 0.1 dB when the limitations are alleviated. The code is available at https://github.com/ZXS-Labs/SSRNO.

We propose Super-resolution Neural Operator (SRNO), a deep operator learning framework that can resolve high-resolution (HR) images at arbitrary scales from the low-resolution (LR) counterparts. Treating the LR-HR image pairs as continuous functions approximated with different grid sizes, SRNO learns the mapping between the corresponding function spaces. From the perspective of approximation theory, SRNO first embeds the LR input into a higher-dimensional latent representation space, trying to capture sufficient basis functions, and then iteratively approximates the implicit image function with a kernel integral mechanism, followed by a final dimensionality reduction step to generate the RGB representation at the target coordinates. The key characteristics distinguishing SRNO from prior continuous SR works are: 1) the kernel integral in each layer is efficiently implemented via the Galerkin-type attention, which possesses non-local properties in the spatial domain and therefore benefits the grid-free continuum; and 2) the multilayer attention architecture allows for the dynamic latent basis update, which is crucial for SR problems to “hallucinate” high-frequency information from the LR image. Experiments show that SRNO outperforms existing continuous SR methods in terms of both accuracy and running time. Our code is at https://github.com/2y7c3/Super-Resolution-Neural-Operator.

Stereo matching is a fundamental and challenging task which has various applications in autonomous driving, dense reconstruction and other depth related tasks. Contextual information with discriminative features is crucial for accurate stereo matching in the ill-posed regions (textureless, occlusion, etc.). In this paper, we propose an efficient horizontal attention module to adaptively capture the global correspondence clues. Compared with the popular non-local attention, our horizontal attention is more effective for stereo matching with better performance and lower consumption of computation and memory. We further introduce a deformable module to refine the contextual information in the disparity discontinuous areas such as the boundary of objects. Learning-based method is adopted to construct the cost volume by concatenating the features of two branches. In order to offer explicit similarity measure to guide learning-based volume for obtaining more reasonable unimodal matching cost distribution we additionally combine the learning-based volume with the improved zero-centered group-wise correlation volume. Finally, we regularize the 4D joint cost volume by a 3D CNN module and generate the final output by disparity regression. The experimental results show that our proposed HDA-Net achieves the state-of-the-art performance on the Scene Flow dataset and obtains competitive performance on the KITTI datasets compared with the relevant networks.

Given training data, convolutional dictionary learning (CDL) seeks a translation-invariant sparse representation, which is characterized by a set of convolutional kernels. However, even a small training set with moderate sample size can render the optimization process both computationally challenging and memory starving. Under a biconvex optimization strategy for CDL, we propose to diagonally precondition the system matrices in the filter learning sub-problem that can be solved by the alternating direction method of multipliers (ADMM). This method leads to the substitution of matrix inversion ([Formula: see text] and matrix multiplication ([Formula: see text] involved in ADMM with an element-wise operation ([Formula: see text], which significantly reduces the computational complexity as well as the memory requirement. Numerical experiments validate the performance advantage of the proposed method over the state-of-the-arts. Code is available at https://github.com/baopingli/Efficient-Convolutional-Dictionary-Learning-using-PADMM .

Over the last decade various compressive snapshot hyperspectral imaging methods have been proposed. The limited reconstruction quality from severely compressed measurements, however, has been a practical barrier to real applications. This paper proposes a compressive chromo-tomography framework that incorporates the convolutional sparse coding (CSC) prior into the classical total variation and L1regularization functionals. Such a combination allows excellent high-frequency recovery capabilities of CSC, while effectively suppressing ghost artifacts in tomographic reconstructions. Since nondifferentiable regularizers are employed, we propose a preconditioned alternating direction method of multipliers (ADMM) for flexible and efficient solutions, both for the reconstruction task and for hyperspectral convolutional dictionary learning. We demonstrate in our numerical experiments that just 25 learned 3D CSC filters can fulfill a rather effective hyperspectral imagery representation and that the proposed method is capable of high accuracy reconstructions.

In this paper, we propose a patch-wise super-resolution (SR) method that combines an external-sample classification tree and a nonlinear-mapping learning stage to simultaneously guarantees reconstruction quality and speed at the stage of patch representation and mapping. We use the low-resolution (LR) to high-resolution (HR) mapping kernel of each patch-pair sample (called SIMK) to complete classification by binary tree branching and provide reasonable training sets for mapping-learning. Then a high accuracy but low cost lightweight network is learned for each tree node to choose the reasonable branch path for the testing LR patches. In the mapping-learning stage, the nonlinear mapping for each class is represented as a full-connected network, which provides satisfying generalization ability for LR patch reconstruction. Comparing with state-of-the-art methods, our approach achieves real-time (>24fps) SR of realistic vision and high quality for different upscaling factors.

Recent advances in compressive spectral imagers have demonstrated the potential of spatio-spectral modulation (SSM) for improved reconstruction performance. Existing SSM techniques, however, use either a color filter array or a complex optical arrangement, both of which can only provide limited modulation bandwidth in the spectral dimension. This paper proposes a practical SSM method to help address the "missing cone" problem of chromo-tomography. A high-resolution binary coded aperture is used to modulate the dispersed images, which in the Fourier domain fulfills a 3D convolution of the probed spectrum with the aperture's wide spectrum. This spectrum spreading process facilitates the compressed sensing strategy determined by the Fourier Slice Theorem and we demonstrate the advantages of the proposed approach with numerical experiments.

This paper proposes a new approach to the image blind super-resolution (BSR) problem in the case of affine interframe motion. Although the tasks of image registration, blur identification, and high-resolution (HR) image reconstruction are coupled in the imaging process, when dealing with nonisometric interframe motion or without the exact knowledge of the blurring process, classic SR techniques generally have to tackle them (maybe in some combinations) separately. The main difficulty is that state-of-the-art deconvolution methods cannot be straightforwardly generalized to cope with the space-variant motion. We prove that the operators of affine warping and blur commute with some additional transforms and derive an equivalent form of the BSR observation model. Using this equivalent form, we develop an iterative algorithm to jointly estimate the triple-coupled variables, i.e., the motion parameters, blur kernels, and HR image. Experiments on synthetic and real-life images illustrate the performance of the proposed technique in modeling the space-variant degradation process and restoring local textures.

The size of training set as well as the usage thereof is an important issue of learning-based super-resolution. In this paper, we presented an adaptive learning method for face hallucination using locality preserving projections (LPP). By virtue of the ability to reveal the non-linear structure hidden in the high-dimensional image space, LPP is an efficient manifold learning method to analyze the local intrinsic features on the manifold of local facial areas. By searching out patches online in the LPP sub-space, which makes the resultant training set tailored to the testing patch, our algorithm performed the adaptive sample selection and then effectively restored the lost high-frequency components of the low-resolution face image by patch-based eigen transformation using the dynamic training set. Finally, experiments fully demonstrated that the proposed method can achieve good performance of super-resolution reconstruction by utilizing a relative small sample.