Discrete latent representation techniques, such as Vector Quantization (VQ) and Sparse Coding (SC), have demonstrated superior image reconstruction and generation quality compared to continuous representation methods in Variational Autoencoders (VAEs). However, existing approaches often treat the latent representations of an image independently in their discrete representation space, neglecting both the inherent structural information within each representation and the correlations among them. This oversight leads to coarse representations and suboptimal generated results. In this paper, we address these limitations by introducing correlations among and within the latent representations of individual images in the latent discrete space of VAEs using sparse coding. We impose two-dimensional structural information through adaptive thresholding, enhancing local structure in image representations while suppressing noise via parsimonious representation with a learned dictionary. Empirical studies on three real benchmark datasets, including a clinical Ultrasound dataset, BSDS500, and mini-Imagenet, demonstrate that our proposed model preserves fine-grained details in image reconstruction and significantly outperforms baseline models of SC-VAE and VQ-VAE across objective and subjective image quality metrics. Particularly noteworthy are the substantial performance improvements observed on the ultrasound dataset, where structure information is crucial. Specifically, we observe significant performance improvements of 7.68 % and 17.03 % in SSIM, 3.25 dB and 6.58 dB in PSNR, 0.15 and 0.24 in LPIPS, 45.38 and 84.05 in FID over SC-VAE and VQ-VAE, respectively, indicating the superiority of our method in terms of image reconstruction quality and fidelity.

Current vision-language models may grasp basic spatial cues and simple directions (e.g. left, right, front, back), but struggle with the multi-dimensional spatial reasoning necessary for human-like understanding and real-world applications. To address this gap, we develop SPHERE (Spatial Perception and Hierarchical Evaluation of REasoning), a hierarchical evaluation framework supported by a new human-annotated dataset. SPHERE systematically probes models across increasing levels of complexity, from fundamental skills to multi-skill integration and high-level reasoning that combines spatial, visual, and logical understanding. Benchmark evaluation of state-of-the-art models reveals significant deficiencies, especially in reasoning about distance and proximity, understanding both egocentric and allocentric perspectives, and applying spatial logic in physical contexts. These findings expose critical blind spots in existing models and underscore the need for more advanced spatial reasoning techniques, driving the development of vision-language models that align more closely with human spatial cognition.

Following the success of Large Language Models (LLMs) in language tasks, LLMs have been adapted for reasoning in math word problems (MWPs). MWP is a complex task that requires both semantic understanding of text and mathematical reasoning, such that achieving high accuracy in MWP remains a challenge. We find that MWP performance can be improved by step-by-step reasoning where the LLM is trained to generate smaller and more manageable steps. We further propose a post-processing correction model to edit the initial solutions given by the LLM. Our correction model, designed to detect and rectify mistakes in these steps, is firstly pretrained using heuristically generated model-agnostic error data and further finetuned with model-specific errors generated through self-supervised augmentation. The correction model iteratively refines the solution step-by-step by analyzing the problem statement and steps up until the current one, making corrections as needed, and repeating the process until all steps in the solution are processed. Experimental results demonstrate that the step-by-step reasoning significantly improves MWP performance compared to one-step solutions. The combination of pretraining and finetuning effectively aligns the correction model with the error patterns of the reasoning model, resulting in further accuracy improvements through error correction.

Precipitation nowcasting predicts future radar sequences based on current observations, which is a highly challenging task driven by the inherent complexity of the Earth system. Accurate nowcasting is of utmost importance for addressing various societal needs, including disaster management, agriculture, transportation, and energy optimization. As a complementary to existing non-autoregressive nowcasting approaches, we investigate the impact of prediction horizons on nowcasting models and propose SimCast, a novel training pipeline featuring a short-to-long term knowledge distillation technique coupled with a weighted MSE loss to prioritize heavy rainfall regions. Improved nowcasting predictions can be obtained without introducing additional overhead during inference. As SimCast generates deterministic predictions, we further integrate it into a diffusion-based framework named CasCast, leveraging the strengths from probabilistic models to overcome limitations such as blurriness and distribution shift in deterministic outputs. Extensive experimental results on three benchmark datasets validate the effectiveness of the proposed framework, achieving mean CSI scores of 0.452 on SEVIR, 0.474 on HKO-7, and 0.361 on MeteoNet, which outperforms existing approaches by a significant margin.

Techniques of dictionary learning and sparse representation are popular in recent study on image denoising, including classic K-SVD and its variants. The extension of K-SVD to its deep structure learned in an end-to-end way shows the state-of-the-art denoising performance with a great computation efficiency. However, we notice that the current learning framework takes images patches as independent samples, which ignores the inherent correlation among the patches. In this paper, we propose a deep K-SVD denoising network with attention mechanism to enhance the correlation within and among the patches. We impose the two-dimensional correlation on the intermediate parameters during the sparse representation procedure to achieve more smoothing and local-structure enhanced image features. Extensive numerical experiments using public data are conducted. The results on two datasets show that the proposed network achieves an average improvement of 0.81dB in peak signal-to-noise ratio (PSNR), 1.66% in the structural similarity (SSIM) and more than 90% in the convergence rate comparing to its counterpart, which demonstrate the efficiency and the competitive performance of our proposed network.

The interferometry synthetic aperture radar (InSAR) technique can generate the digital elevation model (DEM) using the interferometric phase of two SAR observations. In this study, we propose an elevation reconstruction algorithm based on deep learning without phase unwrapping and phase-to-elevation conversion. We also incorporate the intensity information and interferometric phase to obtain a more accurate DEM. Our results show that the studied network can effectively reconstruct the elevation even in rugged terrains. Comparative results show that a better DEM can be reconstructed with the inclusion of the intensity data.

Due to the range and azimuth velocity of moving targets, severe defocusing occurs in synthetic aperture radar (SAR) images. The traditional ground moving target imaging algorithm generally needs to estimate the parameters of the moving target, and then conduct the refocusing of the moving target according to the estimated parameters. In this paper, a SAR moving target refocusing algorithm based on generative adversarial network (GAN) is proposed without estimating the motion parameters of the targets. To get a sufficiently trained network, we propose to use simulated moving target data to train the model and evaluate its performance using real data. The results of numerical experiments show that the trained network using simulated data can be well transferred to real test data and effectively achieve to refocus multiple moving targets with distinct velocities at the circumstance of heavy noise.

Airborne synthetic aperture radar images are easily smeared by the phase error due to the unsteady platform movement. Autofocusing by traditional methods is unsatisfied in critical condition of homogenous targets with large degree of defocusing. This paper proposes a one-step end-to-end autofocus method base on Unet with residual blocks (Res-Unet). We use smeared SAR image of a certain area of one scene for model training and test the trained network to autofocus the images of the remaining areas. Numerical experiments are conducted on real airborne SAR data and the results show that the method can achieve well-focused images for target scene with a large degree of defocusing. Comparison results also demonstrate that the proposed improved U-net structure with residual blocks far outperforms the conventional U-net in the task of SAR image autofocusing.

Phase unwrapping is essential in interferometric synthetic aperture radar (InSAR) data processing. Currently, deep learning is widely used in the phase unwrapping process. For instance, the one-step phase unwrapping method is excellent because of its strong noise adaptability. The method treats the unwrapping process as a regression problem, which uses the l1 or l2 loss function to constrain the reconstructed phase to be close to the ground truth of the absolute phase. However, no matter whether the l1 or l2 loss function is used, the result may lack details in texture, and the details cannot be well preserved. This is because the l1 or l2 smoothens the output greatly, and the texture detail loss is not intentionally considered. Due to the noise of the wrapped phase, there is speckle noise in the valley part of the unwrapping result. To solve these problems, we study the generative adversarial network (GAN) with mixed loss functions. The texture details are preserved with the trained GAN, and the speckle noise in the valley is significantly reduced.

Traditional range-instantaneous Doppler (RID) methods for maneuvering target imaging suffer from the problems of low resolution and poor noise suppression. We propose a new super-resolution inverse synthetic aperture radar (ISAR) imaging method based on deep-learning-assisted time–frequency analysis (TFA). Our deep neural network resembles the basic structure of a U-net with two additional convolutional-upsampling layers and$l_{1}$-norm loss function for super-resolution generation and noise suppression. The neural network is trained in advance to learn the mapping function between the low-resolution time–frequency spectrum inputs and their high-resolution references. Then, the linear TFA assisted by the pretrained network is integrated into the RID-based ISAR imaging system and is found to achieve sharply focused and denoised target image with super-resolution. Both the simulated and real radar data are used to evaluate the performance of the proposed method. Numerical experimental results demonstrate the superiority of the proposed ISAR imaging method over traditional ones.

In a multipath propagation environment, prevalent target localization methods are mainly based on the classical two-ray propagation model without considering other reflected waves. Because the received target echoes are considerably corrupted by multipath reflections in the case of complex terrain, these prevalent methods might fail to work or achieve poor performance. To solve this problem, we first consider a practical multipath propagation scenario to reveal the dynamic structural relationship of the spatial paths based on the spherical earth model. Subsequently, a target localization algorithm based on low-rank decomposition (LRD) and sparse representation (SR) framework is proposed. The proposed algorithm can effectively mitigate the effects of complex multipath interference without using any prior knowledge on the illuminated terrain and the reflecting paths. Experiments on synthetic data and real data validate the effectiveness of the proposed algorithm.

An automatic multi-temporal method is proposed in this paper for cloud detection without known the reference image in prior. A series of reference images are provided by fitting robustly of the pixels of multi-temporal images contaminated by clouds to show the inherent gradual change of the landscape with time instants. Then the cloud is detected by thresholding the difference between the target and the reference images, which is found to be merely composed of the regression model error modeled as Gaussian noise and outliers corresponding to cloud and its shadow. The proposed method is compared with state-of-the-art algorithms on the LANDSAT dataset, and shows a better discrimination of cloud and cloud shadow covered pixels from the uncontaminated ones.

Traditional feature extractions, such as mel-frequency cepstral coefficients (MFCCs), are susceptible to acoustic channel effects, reverberation, and additive environmental noises when applied to passive moving target classification in underwater environment. A spectra multiplication method (SMM) is proposed in this letter to replace the estimated spectrum of MFCCs. SMM suppresses the time-variant noise, and remarkably improves the discriminability of features with historical signals. Compared with traditional MFCCs, the proposed method shows consistent performance improvements in experiments with measured data under different parameter settings. The effect of multiplication order of SMM on classification accuracy has also been discussed. Since the SMM is generalized to a filter-based view, the relation between cutoff frequency and multiplication order is given. The change of classification accuracy induced by the multiplication order is in accord with the variation of cutoff frequency when designed by IIR filters.

This paper presents a new SAR ground moving target imaging (GMTIm) algorithm based on an unified framework of Keystone transformation (KT). To combat the inherent range-azimuth coupling, an tandem two-step strategy is designed, where the range decoupling is implemented by polar format algorithm (PFA) and the azimuth decoupling is finished by an novel time-frequency representation method that is Lv's distribution (LVD). We show, mathematically, that the azimuth resampling of PFA has inherently the same mechanism as the KT, and also, the LVD achieves the optimal performance when it is performed in accordance with the KT principle. Therefore, multiple moving targets can be imaged simultaneously. Focused targets' responses can be obtained in both range and azimuth dimensions. Isotropic point target simulation is designed, and experiments are carried out to validate our proposed SAR-GMTIm algorithm.

This paper considers the problem of estimating multiple frequency hopping signals with unknown hopping pattern. By segmenting the received signals into overlapped measurements and leveraging the property that frequency content at each time instant is intrinsically parsimonious, a sparsity-inspired high-resolution time-frequency representation (TFR) is developed to achieve robust estimation. Inspired by the sparse Bayesian learning algorithm, the problem is formulated hierarchically to induce sparsity. In addition to the sparsity, the hopping pattern is exploited via temporal-aware clustering by exerting a dependent Dirichlet process prior over the latent parametric space. The estimation accuracy of the parameters can be greatly improved by this particular information-sharing scheme and sharp boundary of the hopping time estimation is manifested. Moreover, the proposed algorithm is further extended to multi-channel cases, where task-relation is utilized to obtain robust clustering of the latent parameters for better estimation performance. Since the problem is formulated in a full Bayesian framework, labor-intensive parameter tuning process can be avoided. Another superiority of the approach is that high-resolution instantaneous frequency estimation can be directly obtained without further refinement of the TFR. Results of numerical experiments show that the proposed algorithm can achieve superior performance particularly in low signal-to-noise ratio scenarios compared with other recently reported ones.

Compressive sensing (CS) based Inverse Synthetic Aperture Radar (ISAR) imaging exploits the sparsity of the target scene to achieve high resolution and effective denoising with limited measurements. This paper extends the CS based ISAR imaging to further include the continuity structure of the target scene within a Bayesian framework. A correlated prior is imposed to statistically encourage the continuity structures in both the cross-range and range domains of the target region and the Gibbs sampling strategy is used for Bayesian inference. Because the resulted method requires to recover the whole target scene at a time with heavy computational complexity, an approximate strategy is proposed to alleviate the computational burden. Experimental results demonstrate that the proposed algorithm can achieve substantial improvements in terms of preserving the weak scatterers and removing noise over other reported CS based ISAR imaging algorithms.

This letter addresses the recovery of hierarchical sparse signals in a Bayesian framework. Hierarchical sparse signals exhibit two levels of sparsity, i.e., block-sparsity among different blocks and internal sparsity within each individual block. As in sparse Bayesian learning, each component of the coefficient vector is firstly modeled as a Gaussian distributed variable with zero mean. To enforce the two-level hierarchical sparsity, the variance is further modeled by two classes of hidden variables controlling the block-sparsity and the internal sparsity, respectively. Finally, variational Bayesian inference is used to recover the coefficient vector from the noise corrupted data. Numerical simulation and experimental results show that the proposed method outperforms those recently reported recovery methods.

This paper presents a novel inverse synthetic aperture radar (ISAR) imaging method by exploiting the inherent continuity of the scatterers on the target scene to obtain enhanced target images within a Bayesian framework. A simplified radar system is utilized by transmitting the sparse probing frequency signal, where the ISAR imaging problem can be converted to deal with underdetermined linear inverse scattering. Following the Bayesian compressive sensing (BCS) theory, a hierarchical Bayesian prior is employed to model the scatterers in the range-Doppler plane. In contrast to the independent prior on each scatterer in the conventional BCS, a correlated prior is proposed to statistically encourage the continuity structure of the scatterers in the target region. To overcome the intractability of the posterior distribution, the Gibbs sampling strategy is used for Bayesian inference. The parameters of the signal model are inferred efficiently from samples obtained by the Gibbs sampler. Because the proposed method is a data-driven learning process, the tedious parameter tuning process required by the convex optimization-based approaches can be avoided. Both the synthetic and the experimental results demonstrate that the proposed algorithm can achieve substantial improvements in the scenarios of limited measurements and low signal-to-noise ratio compared with other reported algorithms for ISAR imaging problems.

For inverse synthetic aperture radar imagery, the inherent sparsity of the scatterers in the range-Doppler domain has been exploited to achieve a high-resolution range profile or Doppler spectrum. Prior to applying the sparse recovery technique, preprocessing procedures are performed for the minimization of the translational-motion-induced Doppler effects. Due to the imperfection of coarse motion compensation, the autofocus technique is further required to eliminate the residual phase errors. This paper considers the phase error correction problem in the context of the sparse signal recovery technique. In order to encode sparsity, a multitask Bayesian model is utilized to probabilistically formulate this problem in a hierarchical manner. In this novel method, a focused high-resolution radar image is obtained by estimating the sparse scattering coefficients and phase errors in individual and global stages, respectively, to statistically make use of the sparsity. The superiority of this algorithm is that the uncertainty information of the estimation can be properly incorporated to obtain enhanced estimation accuracy. Moreover, the proposed algorithm achieves guaranteed convergence and avoids a tedious parameter-tuning procedure. Experimental results based on synthetic and practical data have demonstrated that our method has a desirable denoising capability and can produce a relatively well-focused image of the target, particularly in low signal-to-noise ratio and high undersampling ratio scenarios, compared with other recently reported methods.

The detection and estimation of harmonic sinusoidal signals with multiple unknown fundamental frequencies are of great importance in many applications. In this paper, a cascaded sparse processing method is proposed for joint recovery and order estimation of harmonic sinusoidal signals. The cascaded sparse processing is performed by the following two steps. Firstly, group Lasso method is applied to estimate and recover the fundamental frequencies based on the characteristics of the block-sparsity of the harmonics. Then, Lasso estimator is used for each block corresponding to its fundamental frequency for further noise suppression. The theoretical conditions under which the proposed method can give a correct estimate of the signal support is derived under the Fourier basis. Simulation results of the proposed method are given to show the desirable performance.

This letter considers the multiplicative perturbation problem in compressive sensing, which has become an increasingly important issue on obtaining robust performance for practical applications. The problem is formulated in a probabilistic model and an auto-calibration sparse Bayesian learning algorithm is proposed. In this algorithm, signal and perturbation are iteratively estimated to achieve sparsity by leveraging a variational Bayesian expectation maximization technique. Results from numerical experiments have demonstrated that the proposed algorithm has achieved improvements on the accuracy of signal reconstruction.