Multi-view indoor radar perception has drawn attention due to its cost-effectiveness and low privacy risks. Existing methods often rely on implicit cross-view radar feature association, such as proposal pairing in RFMask or query-to-feature cross-attention in RETR, which can lead to ambiguous feature matches and degraded detection in complex indoor scenes. To address these limitations, we propose REXO (multi-view Radar object dEtection with 3D bounding boX diffusiOn), which lifts the 2D bounding box (BBox) diffusion process of DiffusionDet into the 3D radar space. REXO utilizes these noisy 3D BBoxes to guide an explicit cross-view radar feature association, enhancing the cross-view radar-conditioned denoising process. By accounting for prior knowledge that the person is in contact with the ground, REXO reduces the number of diffusion parameters by determining them from this prior. Evaluated on two open indoor radar datasets, our approach surpasses state-of-the-art methods by a margin of +4.22 AP on the HIBER dataset and +11.02 AP on the MMVR dataset. Our implementation is available at https://github.com/merlresearch/radar-bbox-diffusion.

This paper addresses the integration of millimeter-wave (mmWave) Wi-Fi communication and sensing during data transmission intervals (DTIs). We leverage prior knowledge from codebook beam training conducted during preceding beacon transmission intervals (BTIs) and association beamforming training (A-BFT) intervals to design a transceiver array response that meets both requirements on downlink communication SNR and targeted sensing area. By formulating it as a first-order array response optimization with constraints on power, codebook, communication SNR, and limited RF chains, this paper introduces a two-stage solution. First, we introduce a two-way communication-sensing matching pursuit to determine a set of codewords that prioritize the communication SNR constraint. Then, using the selected codewords, we employ an alternating minimization over an auxiliary phase term and beamforming weights to further minimize an array-response distance loss. Numerical results validate the effectiveness of the proposed DMG beamforming design over baseline methods.

The Radar dEtection TRansformer (RETR) has recently been introduced to fuse multi-view millimeter-wave radar heatmaps by leveraging the detection transformer architecture and a geometric learning framework for indoor radar perception. A notable feature of RETR is its tunable positional encoding (TPE), which allows for adjusting the significance of depth positional embedding across multiple views to promote depth-prioritized feature association. However, the TPE ratio is predetermined, rather than being optimized during the training process. In this paper, we propose a differentiable positional encoding (DiPE) scheme for RETR by automatically adjusting the TPE ratio during the training for enhanced performance and avoiding exhaustive grid value search. DiPE can be applied along with either pre-fixed (e.g., sinusoidal) or learnable positional embeddings, achieved by multiplying dual differentiable masks over the depth and angular positional embedding vectors. Comprehensive evaluations on the open MMVR dataset demonstrate that the proposed DiPE not only simplifies the determination of the TPE ratio but also enhances the overall detection performance.

Radar-based indoor 3D human pose estimation typically relied on fine-grained 3D keypoint labels, which are costly to obtain especially in complex indoor settings involving clutter, occlusions, or multiple people. In this paper, we propose \textbf{RAPTR} (RAdar Pose esTimation using tRansformer) under weak supervision, using only 3D BBox and 2D keypoint labels which are considerably easier and more scalable to collect. Our RAPTR is characterized by a two-stage pose decoder architecture with a pseudo-3D deformable attention to enhance (pose/joint) queries with multi-view radar features: a pose decoder estimates initial 3D poses with a 3D template loss designed to utilize the 3D BBox labels and mitigate depth ambiguities; and a joint decoder refines the initial poses with 2D keypoint labels and a 3D gravity loss. Evaluated on two indoor radar datasets, RAPTR outperforms existing methods, reducing joint position error by $34.3$\% on HIBER and $76.9$\% on MMVR. Our implementation is available at \url{https://github.com/merlresearch/radar-pose-transformer}.

Wi-Fi channel measurements across different bands, e.g., sub-7-GHz and 60-GHz bands, are asynchronous due to the uncoordinated nature of distinct standards protocols, e.g., 802.11ac/ax/be and 802.11ad/ay. Multi-band Wi-Fi fusion has been considered before on a frame-to-frame basis for simple classification tasks, which does not require fine-time-scale alignment. In contrast, this paper considers asynchronous sequence-to-sequence fusion between sub-7-GHz channel state information (CSI) and 60-GHz beam signal-to-noise-ratio (SNR)s for more challenging tasks, such as continuous coordinate estimation. To handle the timing disparity between asynchronous multi-band Wi-Fi channel measurements, this paper proposes a multi-band neural dynamic fusion (NDF) framework. This framework uses separate encoders to embed the multi-band Wi-Fi measurement sequences to separate initial latent conditions. Using a continuous-time ordinary differential equation (ODE) modeling, these initial latent conditions are propagated to the respective latent states of the multi-band channel measurements at the same time instances for a latent alignment and a post-ODE fusion, and at their original time instances for measurement reconstruction. We derive a customized loss function based on the variational evidence lower bound (ELBO) that balances between the multi-band measurement reconstruction and continuous coordinate estimation. We evaluate the NDF framework using an in-house multi-band Wi-Fi testbed and demonstrate substantial performance improvements over a comprehensive list of single-band and multi-band baseline methods.

Conventional radar feature extraction faces limitations due to low spatial resolution, noise, multipath reflection, the presence of ghost targets, and motion blur. Such limitations can be exacerbated by nonlinear object motion, particularly from an ego-centric viewpoint. It becomes evident that to address these challenges, the key lies in exploiting temporal feature relation over an extended horizon and enforcing spatial motion consistency for effective association. To this end, this paper proposes SIRA (Scalable Inter-frame Relation and Association) with two designs. First, inspired by Swin Transformer, we introduce extended temporal relation, generalizing the existing temporal relation layer from two consecutive frames to multiple inter-frames with temporally regrouped window attention for scalability. Second, we propose motion consistency track with the concept of a pseudo-tracklet generated from observational data for better trajectory prediction and subsequent object association. Our approach achieves 58.11 [email protected] for oriented object detection and 47.79 MOTA for multiple object tracking on the Radiate dataset, surpassing previous state-of-the-art by a margin of +4.11 [email protected] and +9.94 MOTA, respectively.

This paper considers object detection with millimeter-wave (mmWave) Wi-Fi beam training frames, e.g., beacon frames, in a monostatic passive directional multi-gigabit (DMG) sensing configuration. We derive an explicit signal model that accounts for the preamble, frame-to-frame antenna gains, and clutter. Given the signal model, we develop a hypothesis testing-based object detection that directly leverages symbol-level preamble waveforms and explores the Kronecker structure between the range steering vector and the Doppler steering vector weighted by the antenna gain. Numerical results confirm the effectiveness of the proposed detector and evaluate the impact of frame-to-frame antenna gains due to the beam scanning.

In contrast to existing multi-band Wi-Fi fusion in a frame-to-frame basis for simple classification, this paper considers asynchronous sequence-to-sequence fusion between sub-7GHz channel state information (CSI) and 60GHz beam SNR for more challenging downstream tasks such as continuous regression. To handle the timing disparity between the two channel measurements, we extend our recently proposed dual-decoder neural dynamic (DDND) framework with latent ordinary differential equations (ODEs), align the distinct latent dynamic states at the same time instances, and introduce a post-ODE fusion framework. The resulting neural dynamic fusion (NDF) framework is trained in an end-to-end fashion with a modified variational autoencoder loss function. Evaluation over a newly collected in-house multi-band Wi-Fi dataset shows the advantage of the proposed NDF method over frame-based and DDND methods.

Due to the noise and low spatial resolution in automotive radar data, exploring temporal relations of learnable features over consecutive 2 radar frames has shown performance gain on downstream tasks (e.g., object detection and tracking) in our previous study [1]. In this paper, we further enhance radar perception by significantly extending the time horizon of temporal relations. To this end, we propose a scalable connective temporal radar (SCTR) method that consists of 1) a standard temporal relation layer (TRL), 2) a connective TRL with shifted window attention, and 3) a window merging operation, to facilitate feature connectivity between radar frames over an extended time interval. Our complexity analysis and comprehensive evaluation of the Radiate dataset demonstrate that the SCTR achieves a great tradeoff between the complexity and downstream detection performance.

Indoor radar perception has seen rising interest due to affordable costs driven by emerging automotive imaging radar developments and the benefits of reduced privacy concerns and reliability under hazardous conditions (e.g., fire and smoke). However, existing radar perception pipelines fail to account for distinctive characteristics of the multi-view radar setting. In this paper, we propose Radar dEtection TRansformer (RETR), an extension of the popular DETR architecture, tailored for multi-view radar perception. RETR inherits the advantages of DETR, eliminating the need for hand-crafted components for object detection and segmentation in the image plane. More importantly, RETR incorporates carefully designed modifications such as 1) depth-prioritized feature similarity via a tunable positional encoding (TPE); 2) a tri-plane loss from both radar and camera coordinates; and 3) a learnable radar-to-camera transformation via reparameterization, to account for the unique multi-view radar setting. Evaluated on two indoor radar perception datasets, our approach outperforms existing state-of-the-art methods by a margin of 15.38+ AP for object detection and 11.91+ IoU for instance segmentation, respectively. Our implementation is available at https://github.com/merlresearch/radar-detection-transformer.

We consider the problem of simultaneously learning a convex penalty function and its proximity operator for image reconstruction from incomplete measurements. Our goal is to apply Accelerated Proximal Gradient Method (APGM) using a learned proximity operator in place of the true proximity operator of the learned penalty function. Starting from a Gaussian image denoiser, we learn an associated penalty function and its proximity operator. The learned penalty function offers provable reconstruction guarantees, whereas access to its proximity operator presents the opportunity to achieve APGM convergence rates, which are faster than those of subgradient descent approaches.

This paper considers spatial-domain detector design for mutual interference mitigation among automotive MIMO-FMCW radars. This detector design is based on our previously derived interference signal model that fully accounts for the time-frequency incoherence and the slow-time code incoherence between the victim and interfering radars. Compared with our previous spatial-domain detector in [1], the proposed detector further exploits the structural property of both transmit and receive steering vectors of the interference for stronger interference mitigation. Preliminary numerical results confirm the performance of our proposed detector and show advantages over baseline detectors.

In frequency-modulated continuous-wave (FMCW) lidar, the distance to an illuminated target is proportional to the beat frequency of the interference signal. Laser phase noise often limits the range accuracy of FMCW lidar, and existing frequency estimation methods make overly simplistic assumptions about the noise model. In this work, we propose an algorithm that performs frequency estimation via phase unwrapping by explicitly accounting for correlations in the phase noise. Given a candidate frequency, we approximately recover the maximum likelihood unwrapping sequence using the Viterbi algorithm and the phase noise statistics. The algorithm then alternates between unwrapping and frequency estimate refinement until convergence. Compared to state-of-the-art alternatives, our algorithm consistently achieves superior performance at long range or with large-linewidth lasers when the signal-to-noise ratio is sufficiently high.

We leverage standards-compliant beam training measurements from commercial-of-the-shelf (COTS) 802.11ad/ay devices for localization of a moving object. Two technical challenges need to be addressed: (1) the beam training measurements are intermittent due to beam scanning overhead control and contention-based channel-time allocation, and (2) how to exploit underlying object dynamics to assist the localization. To this end, we formulate the trajectory estimation as a sequence regression problem. We propose a dual-decoder neural dynamic learning framework to simultaneously reconstruct Wi-Fi beam training measurements at irregular time instances and learn the unknown dynamics over the latent space in a continuous-time fashion by enforcing strong supervision at both the coordinate and measurement levels. The proposed method was evaluated on an in-house mmWave Wi-Fi dataset and compared with a range of baseline methods, including traditional machine learning methods and recurrent neural networks.

Recent developments in wave-based sensor technologies, such as ground penetrating radar (GPR), provide new opportunities for accurate imaging of underground scenes. Given measurements of the scattered electromagnetic wavefield, the goal is to estimate the spatial distribution of the permittivity of the underground scenes. However, such problems are highly ill-posed, difficult to formulate, and computationally expensive. In this paper, we propose a physics-inspired machine learning-based method to learn the wave-matter interaction under the GPR setting. The learned forward model is combined with a learned signal prior to recover the permittivity distribution of the unknown underground scenes. We test our approach on a dataset of 400 permittivity maps with a three-layer background, which is challenging to solve using existing methods. We demonstrate via numerical simulation that our method achieves a 50% improvement in mean squared error over benchmark machine learning-based solvers for reconstructing layered underground scenes.

We introduce a scalable multi-modal approach to learn dense, i.e., pixel-level, correspondences and occlusion maps, between images in a video sequence. The problems of finding dense correspondences and occlusion maps are fundamental in computer vision. In this work we jointly train a deep network to tackle both, with a shared feature extraction stage. We use depth and color images with ground truth optical flow and occlusion maps to train the network end-to-end. From the multi-modal input, the network learns to estimate occlusion maps, optical flows, and a correspondence embedding providing a meaningful latent feature space. We evaluate the performance on a dataset of images derived from synthetic characters, and perform a thorough ablation study to demonstrate that the proposed components of our architecture combine to achieve the lowest correspondence error. The scalability of our proposed method comes from the ability to incorporate additional modalities, e.g., infrared images.

Antenna position ambiguity is a common problem that affects radar imaging systems that are mounted on mobile platforms. Existing approaches that aim to recover a sharp radar image despite this ambiguity aim to estimate the shift in the antenna position by modeling the radar scene as a sparse image with a small number of targets using explicit analytical models for the statistical distribution of the targets in a radar image. The radar imaging problem is then solved by alternating between estimating the radar image, followed by estimating the shift in the antenna positions, until convergence is reached. While such approaches have shown tremendous success, they still struggle to recover the true target positions and may arrive at incorrect local optima when the measurement noise level is high. In this work, we develop a data-driven learning-based strategy for modeling the image of the radar scene instead of relying on explicit analytical models. We adopt a residual Unet architecture of a neural network to act as a denoising operator which takes a backprojected radar image as input and outputs a true target image. While deep denoisers may generally result in unstable iterative algorithms, we introduce a simple filtering step that suppresses noise belonging to the null space of the radar operator from the iterates to stabilize the iterative procedure. We evaluate the effectiveness of our solution using simulated numerical experiments and demonstrate its superiority over the analytic signal prior.

Optical coherence tomography (OCT) using Fourier domain processing can resolve micrometer-scale depth information. However, the conventional volumetric reconstruction approach is unnecessary for opaque samples with only one reflector per lateral position, and the required sample interpolation degrades performance. In this paper, we show that surface depth profilometery with a Fourier-domain OCT system simplifies to a sinusoidal parameter estimation problem. We derive approximate maximum likelihood estimators for the sample depth and reflectivity, which can easily be computed by backprojecting the data without interpolating. Iterative refinement further improves results at high signal-to-noise ratio (SNR). We demonstrate the performance of the technique compared to the conventional Fourier transform approach on both simulated and experimental data collected with a spectral-domain OCT system. Our results show that maximum likelihood profilometry is fast and more robust to noise than the Fourier approaches at moderate SNR.

Blind pansharpening addresses the problem of generating a high spatial-resolution multi-spectral (HRMS) image given a low spatial-resolution multi-spectral (LRMS) image with the guidance of its associated spatially misaligned high spatial-resolution panchromatic (PAN) image without parametric side information. In this article, we propose a fast approach to blind pansharpening and achieve the state-of-the-art image reconstruction quality. Typical blind pansharpening algorithms are often computationally intensive since the blur kernel and the target HRMS image are often computed using iterative solvers and in an alternating fashion. To achieve fast blind pansharpening, we decouple the solution of the blur kernel and of the HRMS image. First, we estimate the blur kernel by computing the kernel coefficients with minimum total generalized variation that blur a downsampled version of the PAN image to approximate a linear combination of the LRMS image channels. Then, we estimate each channel of the HRMS image using local Laplacian prior (LLP) to regularize the relationship between each HRMS channel and the PAN image. Solving the HRMS image is accelerated by both parallelizing across the channels and by fast numerical algorithms for each channel. Due to the fast scheme and the powerful priors we used on the blur kernel coefficients (total generalized variation) and on the cross-channel relationship (LLP), numerical experiments demonstrate that our algorithm outperforms the state-of-the-art model-based counterparts in terms of both computational time and reconstruction quality of the HRMS images.

Recent advances in solving imaging inverse problems have witnessed the combination of deep learning models with classical image models for better signal representation. One such approach, DeepRED, combines the deep image prior (DIP) with the regularization by denoising (RED) framework to boost the performance of image deblurring and super resolution tasks. In this paper, we formulate DeepRED as a consensus equilibrium problem and set up a fixed-point algorithm for solving the equilibrium equations. We also derive sufficient conditions that the DIP generative prior should satisfy to ensure that the corresponding fixed-point operator is non-expansive. We then demonstrate that the fixed-point algorithm that solves the CE equations results in improved image reconstruction quality in a deblurring setting compared to state-of-the-art methods.

In several applications, including imaging of deformable objects while in motion, simultaneous localization and mapping, and unlabeled sensing, we encounter the problem of recovering a signal that is measured subject to unknown permutations. In this paper we take a fresh look at this problem through the lens of optimal transport (OT). In particular, we recognize that in most practical applications the unknown permutations are not arbitrary but some are more likely to occur than others. We exploit this by introducing a regularization function that promotes the more likely permutations in the solution. We show that, even though the general problem is not convex, an appropriate relaxation of the resulting regularized problem allows us to exploit the well-developed machinery of OT and develop a tractable algorithm.

This paper introduces a B-spline chained ellipses model representation for extended object tracking (EOT) using high-resolution automotive radar measurements. With offline automotive radar training datasets, the proposed model parameters are learned using the expectation-maximization (EM) algorithm. Then the probabilistic multi-hypothesis tracking (PMHT) along with the unscented transform (UT) is proposed to deal with the nonlinear forward-warping coordinate transformation, the measurement-to-ellipsis association, and the state update step. Numerical validation is provided to verify the effectiveness of the proposed EOT framework with automotive radar measurements.

Plug-and-play (PnP) denoising for solving inverse problems has received significant attention recently thanks to its state of the art signal reconstruction performance. However, the performance improvement hinges on carefully choosing the noise level of the Gaus-sian denoiser and the descent step size in every iteration. We propose a strategy for training a Gaussian denoiser inspired by an unfolded proximal quasi-Newton algorithm, where the noise level of the input signal to the denoiser is estimated in each iteration and at every entry in the signal. Our scheme deploys a small convolutional neural network (mini-CNN) to estimate an element-wise noise level, mimicking a diagonal approximation of the Hessian matrix in quasi-Newton methods. Empirical simulation results on image deblurring demonstrate that our proposed approach achieves approximately 1dB improvement over state of the art methods, such as, BM3D-PnP and proximal gradient descent-PnP that are supplied with the true noise level, as well as over an end-to-end retrained FFDNet architecture that was trained to estimate the noise level and recover the deblurred images.

We study the problem of reconstructing an object from phaseless measurements in the context of inverse multiple scattering. Our formulation explicitly decouples the variables that represent the unknown object image and the unknown phase, respectively, in the forward model. This enables us to simultaneously optimize over both unknowns with appropriate regularization for each. The resulting optimization problem is nonconvex due to the nonlinear propagation model for multiple scattering and the nonconvex regularization of the phase variables. Nevertheless, we demonstrate experimentally that we can solve the optimization problem using a variation of the fast iterative shrinkage-thresholding algorithm (FISTA)-a convex algorithm, popular for its speed and simplicity-that converges well in our experiments. Numerical results with both simulated and experimentally measured data show that the proposed method outperforms the state-of-the-art phaseless inverse scattering method.

This paper considers object detection in the case of imperfect waveform separation, in the context of automotive radars with a slow-time MIMO-FMCW signaling scheme. We develop an explicit signal model that accounts for waveform separation residuals and propose a Kronecker subspace-based object detector in the framework of generalized likelihood ratio test (GLRT). Our exact theoretical analysis under both hypotheses shows that the proposed detector holds the desired property of constant false alarm rate (CFAR). Numerical simulations validate our proposed object detection scheme.

Motivated by real-world automotive radar measurements that are distributed around object (e.g., vehicles) edges with a certain volume, a novel hierarchical truncated Gaussian measurement model is proposed to resemble the underlying spatial distribution of radar measurements. With the proposed measurement model, a modified random matrix-based extended object tracking algorithm is developed to estimate both kinematic and extent states. In particular, a new state update step and an online bound estimation step are proposed with the introduction of pseudo measurements. The effectiveness of the proposed algorithm is verified in simulations.

Parameter estimation of damped exponential signals has wide applications including fault detection and system parameter identification, etc. However, existing methods for estimating parameters of damped exponentials are either sensitive to noise or restricted to dealing with a certain type of noise such as Gaussian noise. In this paper we aim to estimate parameters of damped exponentials from contaminated signal, i.e., a mixture of damped exponentials, random Gaussian noise, and spike interference. We propose two robust approaches, a convex one solved by the alternating direction method of multipliers (ADMM) and a non-convex one solved by coordinate descent, to recovering a low-rank Hankel matrix of damped exponentials from noisy measurements for further parameter estimation using the matrix pencil technique. Numerical experiments show that our proposed methods outperform classical ones in detecting small damped fault signatures from noisy measurements. While the convex approach is amenable to theoretical analysis and global convergence guarantees, the non-convex one exhibits more robustness and computational efficiency.

We address the problem of sharpening low spatial-resolution multi-spectral (MS) images with their associated misaligned high spatial-resolution panchromatic (PAN) image, based on priors on the spatial blur kernel and on the cross-channel relationship. In particular, we formulate the blind pan-sharpening problem within a multi-convex optimization framework using total generalized variation for the blur kernel and local Laplacian prior for the cross-channel relationship. The problem is solved by the alternating direction method of multipliers (ADMM), which alternately updates the blur kernel and sharpens intermediate MS images. Numerical experiments demonstrate that our approach is more robust to large misalignment errors and yields better super resolved MS images compared to state-of-the-art optimization-based and deep-learning-based algorithms.

The performance of synthetic aperture radar degrades when its moving platform is perturbed with unknown position errors or received signals are interfered by strong random noise. Therefore, it is desirable to perform robust imaging with noisy radar echoes even under large position perturbations. In this paper, we propose a graph-based denoising method, which regularizes both the smoothness in the graph domain and the sparse gradients in the time domain. Different from previous GSP-based methods, our graph model is built in the radar signal domain instead of the image domain, so that we can jointly estimate position perturbations of the radar platform and denoise the received signals, providing focused imaging results. Simulation results demonstrate that our method improves the radar imaging quality from 13.3dB provided by coherence analysis to 21.6dB in terms of PSNR.

In this paper, we develop a robust three dimensional tomographic imaging framework to estimate the ionospheric electron density using ground-based total electron content (TEC) measurements from GPS receivers. In order to increase the sampling rate of the domain, we incorporate into the tomographic measurements the TEC readings observed from low-angle satellites that fall outside of the target ionospheric domain. We discount the proportion of the TEC measurements that originate outside of the target domain using the simulation-based NeQuick2 model as reference. We also employ a diffusion kernel regularization function to robustify the reconstruction against errors in the NeQuick2 model. Finally, we demonstrate through simulations that our framework delivers superior reconstruction of the ionospheric electron density compared to existing schemes. We also demonstrate the applicability of our approach on real TEC measurements.

Reflection tomography is an inverse scattering technique that estimates the spatial distribution of an object's permittivity by illuminating it with a probing pulse and measuring the scattered wavefields by receivers located on the same side as the transmitter. Unlike conventional transmission tomography, the reflection regime is severely ill-posed since the measured wavefields contain far less spatial frequency information about the object. In this paper, we propose an incremental frequency inversion framework that requires no initial target model, and that leverages spatial regularization to reconstruct the permittivity distribution of highly scattering objects. Our framework solves a wave-equation constrained, total-variation (TV) regularized nonlinear least squares problem that solves a sequence of subproblems that incrementally enhance the resolution of the estimated object model. With each subproblem, higher frequency wavefield components are incorporated in the inversion to improve the recovered model resolution. We validate the performance of our approach using synthetically generated data for retrieving high-contrast material such as water in an underground radar imaging setup.

In this paper we develop an optimization-based solution to the problem of distributed radar imaging using antennas with asynchronous clocks. In particular, we consider a distributed radar imaging MIMO system observing a sparse scene under an unknown, but bounded, delay between the transmitter and receiver clocks. Most existing approaches pose the problem as the recovery of a phase shift, leading to non-convex formulations. Instead, inspired by recent work in blind deconvolution, we exploit the realization that synchronization errors in the received data can be modeled as a convolution with an unknown 1-sparse delay signal to be estimated in addition to the image. Thus, we formulate a convex optimization problem that simultaneously recovers all the pair-wise drifts between transmit/receive pairs, as well as the sparse scene being imaged. We verify the validity and performance of our proposed model and recovery method through numerical simulations on synthetic data.

In this paper, we consider the problem of fusing low spatial resolution multi-spectral (MS) aerial images with their associated high spatial resolution panchromatic image. To solve this problem, various methods have been proposed, using either model-based or model-agnostic algorithms such as deep learning techniques. In this paper, we aim to utilize more interpretable architectures to solve the MS fusion problem by integrating existing ideas from image processing with deep learning. In particular, we develop a signal processing-inspired learning solution, where we unroll the iterations of the projected gradient descent (PGD) algorithm, and each iteration contains a projection operation carried out by a deep convolutional neural network. We observe that our proposed method provides a new perspective on existing deep-learning solutions, and under certain circumstance it reduces to current black-box deep learning methods. Our extensive experimental results show significant improvements of the proposed approach over several baselines.

Image registration is of crucial importance in image fusion such as pan-sharpening. Mutual information (MI)-based methods have been widely used and demonstrated effectiveness in registering multi-spectral or multi-modal images. However, MI-based methods may fail to converge in searching registration parameters, resulting mis-registration. In this paper, we propose an outlier robust method to improve the robustness of MI-based registration for multiple rigid transformed images. In particular, we first generate registration parameter matrices using a MI-based approach, then we decompose each parameter matrix into a low-rank matrix of inlier registration parameters and a sparse matrix corresponding to outlier parameter errors. Results of registering multi-spectral images with random rigid transformations show significant improvement and robustness of our method.

The problem of reconstructing an object from the measurements of the light it scatters is common in numerous imaging applications. While the most popular formulations of the problem are based on linearizing the object-light relationship, there is an increased interest in considering nonlinear formulations that can account for multiple light scattering. In this paper, we propose an image reconstruction method, called CISOR, for nonlinear diffractive imaging, based on our new variant of fast iterative shrinkage/thresholding algorithm (FISTA) and total variation (TV) regularization. We prove that CISOR reliably converges for our nonconvex optimization problem, and systematically compare our method with other state-of-the-art methods on simulated as well as experimentally measured data.

The radar autofocus problem arises in situations where radar measurements are acquired of a scene using antennas that suffer from position ambiguity. Current techniques model the antenna ambiguity as a global phase error affecting the received radar measurement at every antenna. However, the phase error signal model is only valid in the far field regime where the position error can be approximated by a one dimensional shift in the down-range direction. We propose in this paper an alternate formulation where the antenna position error is modeled using a two-dimensional shift operator in the image-domain. The radar autofocus problem then becomes a multichannel two-dimensional blind deconvolution problem where the static radar image is convolved with a two dimensional shift kernel for each antenna measurement. We develop an alternating minimization framework that leverages the sparsity and piece-wise smoothness of the radar scene, as well as the one-sparse property of the two dimensional shift kernels.

Multi-spectral (MS) image super-resolution aims to reconstruct super-resolved multi-channel images from their low-resolution images by regularizing the image to be reconstructed. Recently data-driven regularization techniques based on sparse modeling and deep learning have achieved substantial improvements in single image reconstruction problems. Inspired by these data-driven methods, we develop a novel coupled analysis and synthesis dictionary (CASD) model for MS image super-resolution, by exploiting a regularizer that operates within, as well as across, multiple spectral channels using convolutional dictionaries. To learn the CASD model parameters, we propose a deep dictionary learning framework, named DeepCASD, by unfolding and training an end-to-end CASD based reconstruction network over an image data set. Experimental results show that the DeepCASD framework exhibits improved performance on multi-spectral image super-resolution compared to state-of-the-art learning based super-resolution algorithms.

Recently emerging applications, such as autonomous navigation, mapping, and home entertainment, have increased the demand for inexpensive and high quality depth sensing. In this paper we fundamentally re-examine the problem, considering recent advances in photoelectric devices, increased availability of fast electronics, reduced computation cost, and developments in sensing theory. Our main contribution is a real-time hardware architecture for time-of- flight (ToF) depth sensors that exploits random modulation to significantly reduce the acquisition burden. The proposed design is able to acquire compressive, critical, or redundant measurements, without requiring any hardware modifications, at the expense of small reduction in the system frame rate. The architecture we propose is sufficiently flexible to be operable in a variety of conditions and with a variety of reconstruction algorithms.

In remote sensing systems, exact knowledge of the sensor locations is critical for generating focused images. In order to accurately locate misplaced or perturbed sensors from their received signal data, we proposed a robust sensor localization method based on low-rank Euclidean distance matrix (EDM) reconstruction. To this end, an EDM of sensors and objects under detection is defined and partially initialized by computing distances between the inaccurate sensor locations and distances from the sensors to the objects using signal coherence analysis. We then decompose the noisy EDM with missing entries into a low-rank EDM corresponding to true sensor locations and a sparse matrix of distance errors by solving a constrained optimization problem using the alternating direction method of multipliers (ADMM). We verify our method with simulations on a uniform linear array with unknown perturbations up to several wavelengths.

We propose a new compressive imaging method for reconstructing 2D or 3D objects from their scattered wave-field measurements. Our method relies on a novel, nonlinear measurement model that can account for the multiple scattering phenomenon, which makes the method preferable in applications where linear measurement models are inaccurate. We construct the measurement model by expanding the scattered wave-field with an accelerated-gradient method, which is guaranteed to converge and is suitable for large-scale problems. We provide explicit formulas for computing the gradient of our measurement model with respect to the unknown image, which enables image formation with a sparsity-driven numerical optimization algorithm. We validate the method both analytically and with numerical simulations.

Computational imaging methods that can exploit multiple modalities have the potential to enhance the capabilities of traditional sensing systems. In this paper, we propose a new method that reconstructs multimodal images from their linear measurements by exploiting redundancies across different modalities. Our method combines a convolutional group-sparse representation of images with total variation (TV) regularization for high-quality multimodal imaging. We develop an online algorithm that enables the unsupervised learning of convolutional dictionaries on large-scale datasets that are typical in such applications. We illustrate the benefit of our approach in the context of joint intensity-depth imaging.

We consider the problem of fusing multiple cloud-contaminated aerial images of a 3D scene to generate a cloud-free image, where the images are captured from multiple unknown view angles. In order to fuse these images, we propose an end-to-end framework incorporating epipolar geometry and low-rank matrix completion. In particular, we first warp the multi-angular images to single-angle ones based on the estimated fundamental matrices that relate the multi-angular images according to their projective relations to the 3D scene. Then we formulate the fusion process of the warpped images as a low-rank matrix completion problem where each column of the matrix corresponds to a vectorized image with missing entries corresponding to cloud or occluded areas. Results using DigitalGlobe high spatial resolution images demonstrate that our algorithm outperforms existing approaches.

Compression of multispectal images is of great importance in an environment where resources such as computational power and memory are scarce. To that end, we propose a new extremely low-complexity encoding approach for compression of multispectral images, that shifts the complexity to the decoding. Our method combines principles from compressed sensing and distributed source coding. Specifically, the encoder compressively measures blocks of the band of interest and uses syndrome coding to encode the bitplanes of the measurements. The decoder has access to side information, which is used to predict the bitplanes and to decode them. The side information is also used to guide the reconstruction of the image from the decoded measurements. Our experimental results demonstrate significant improvement in the rate-distortion trade-off when compared to coding schemes with similar complexity.

Multi-modal sensing is increasingly becoming important in a number of applications, providing new capabilities and processing challenges. In this paper, we explore the benefit of combining a low-resolution depth sensor with a high-resolution optical video sensor, in order to provide a high-resolution depth map of the scene. We propose a new formulation that is able to incorporate temporal information and exploit the motion of objects in the video to significantly improve the results over existing methods. In particular, our approach exploits the space-time redundancy in the depth and intensity using motion-adaptive low-rank regularization. We provide experiments to validate our approach and confirm that the quality of the estimated high-resolution depth is improved substantially. Our approach can be a first component in systems using vision techniques that rely on high-resolution depth information.

We present a new method for joint automatic extrinsic calibration and sensor fusion for a multimodal sensor system comprising a LIDAR and an optical camera. Our approach exploits the natural alignment of depth and intensity edges when the calibration parameters are correct. Thus, in contrast to a number of existing approaches, we do not require the presence or identification of known alignment targets. On the other hand, the characteristics of each sensor modality, such as sampling pattern and information measured, are significantly different, making direct edge alignment difficult. To overcome this difficulty, we jointly fuse the data and estimate the calibration parameters. In particular, the joint processing evaluates and optimizes both the quality of edge alignment and the performance of the fusion algorithm using a common cost function on the output. We demonstrate accurate calibration in practical configurations in which depth measurements are sparse and contain no reflectivity information. Experiments on synthetic and real data obtained with a three-dimensional LIDAR sensor demonstrate the effectiveness of our approach.

We propose a new method for low-complexity compression of multispectral images. We develop on a novel approach to coding signals with side information based on recent advances in compressed sensing and universal scalar quantization. Our approach can be interpreted as a variation of quantized compressed sensing, where the most significant bits are discarded at the encoder and recovered at the decoder from the side information. The image is reconstructed using weighted total variation minimization, incorporating side information in the weights while enforcing consistency with the recovered quantized coefficient values. Our experiments validate our approach and confirm the improvements in rate-distortion performance.

We propose a new method for low-complexity compression of multispectral images based on universal vector quantization. Our approach generalizes the recently developed theory of universal scalar quantization to vector quantization, and uses it in the context of distributed coding. We exploit the availability of side information on the decoder to reduce the encoding rate of a vector quantizer, applied to compressed measurements of the image. The encoding reuses quantization labels to label multiple quantization cells and leverages the side information to select the correct cell at the decoder. The image is reconstructed using weighted total variation minimization, incorporating side information in the weights while enforcing consistency with the recovered quantization cell.

The iterative Born approximation (IBA) is a well-known method for describing waves scattered by semitransparent objects. In this letter, we present a novel nonlinear inverse scattering method that combines IBA with an edge-preserving total variation regularizer. The proposed method is obtained by relating iterations of IBA to layers of an artificial multilayer neural network and developing a corresponding error backpropagation algorithm for efficiently estimating the permittivity of the object. Simulations illustrate that, by accounting for multiple scattering, the method successfully recovers the permittivity distribution where the traditional linear inverse scattering fails.

Several convex formulation methods have been proposed previously for statistical estimation with structured sparsity as the prior. These methods often require a carefully tuned regularization parameter, often a cumbersome or heuristic exercise. Furthermore, the estimate that these methods produce might not belong to the desired sparsity model, albeit accurately approximating the true parameter. Therefore, greedy-type algorithms could often be more desirable in estimating structured-sparse parameters. So far, these greedy methods have mostly focused on linear statistical models. In this paper, we study the projected gradient descent with a non-convex structured-sparse parameter model as the constraint set. Should the cost function have a stable model-restricted Hessian, the algorithm produces an approximation for the desired minimizer. As an example, we elaborate on application of the main results to estimation in generalized linear models.

Summary form only given. Visual inference over a transmission channel is increasingly becoming an important problem in a variety of applications. In such applications, low latency and bit-rate consumption are often critical performance metrics, making data compression necessary. In this paper, we examine feature compression for support vector machine (SVM)-based inference using quantized randomized embeddings. We demonstrate that embedding the features is equivalent to using the SVM kernel trick with a mapping to a lower dimensional space. Furthermore, we show that universal embeddings - a recently proposed quantized embedding design - approximate a radial basis function (RBF) kernel, commonly used for kernel-based inference. Our experimental results demonstrate that quantized embeddings achieve 50% rate reduction, while maintaining the same inference performance. Moreover, universal embeddings achieve a further reduction in bit-rate over conventional quantized embedding methods, validating the theoretical predictions.

Continuous improvement in optical sensing components, as well as recent advances in signal acquisition theory provide a great opportunity to reduce the cost and enhance the capabilities of depth sensing systems. In this paper we propose a new depth sensing architecture that exploits a fixed coded aperture to significantly reduce the number of sensors compared to conventional systems. We further develop a modeling and reconstruction framework, based on model-based compressed sensing, which characterizes a large variety of depth sensing systems. Our experiments demonstrate that it is possible to reduce the number of sensors by more than 85%, with negligible reduction on the sensing quality.

Compressive sensing (CS) is a data acquisition technique that measures sparse or compressible signals at a sampling rate lower than their Nyquist rate. Results show that sparse signals can be reconstructed using greedy algorithms, often requiring prior knowledge such as the signal sparsity or the noise level. As a substitute to prior knowledge, cross validation (CV), a statistical method that examines whether a model overfits its data, has been proposed to determine the stopping condition of greedy algorithms. This paper analyses cross validation in a general compressive sensing framework. Furthermore, we provide both theoretical analysis and numerical simulations for a cross-validation modification of orthogonal matching pursuit, referred to as OMP-CV, which has good performance in sparse recovery.

We consider the problem of extracting descriptors that represent visually salient portions of a video sequence. Most state-of-the-art schemes generate video descriptors by extracting features, e.g., SIFT or SURF or other keypoint-based features, from individual video frames. This approach is wasteful in scenarios that impose constraints on storage, communication overhead and on the allowable computational complexity for video querying. More importantly, the descriptors obtained by this approach generally do not provide semantic clues about the video content. In this paper, we investigate new feature-agnostic approaches for efficient retrieval of similar video content. We evaluate the efficiency and accuracy of retrieval when k-means clustering is applied to image features extracted from video frames. We also propose a new approach in which the extraction of compact video descriptors is cast as a Non-negative Matrix Factorization (NMF) problem. Initial experiments on video-based matching suggest that compact descriptors obtained via low-rank matrix factorization improve discriminability and robustness to parameter selection compared to k-means clustering.

In this paper, we fundamentally re-examine 3D SAR imaging and propose a CS-based approach aiming to reduce the data collection cost and increase the elevation resolution. In particular, our approach significantly reduces the number of baselines required to acquire the scene of interest, as well as the pulsing rate in each baseline. The baselines are collected using multiple passes of a single or multiple SAR platforms such that their elevations are randomly distributed in the available elevation space. Each baseline uses a fixed pulse repetition frequency (PRF) which can be different from the PRFs used in other baselines. Using the collected multi-baseline data in its entirety we generate a high resolution 3D reflectivity map, using a CS-based iterative imaging algorithm. Our simulation results demonstrate that the proposed method can improve elevation resolution significantly by fusing data from multiple platforms due to the very large virtual elevation aperture even with a small number of baselines.

Traditional rate-distortion theory is focused on how to best encode a signal using as few bits as possible and incurring as low a distortion as possible. However, very often, the goal of transmission is to extract specific information from the signal at the receiving end, and the distortion should be measured on that extracted information. In this paper we examine the problem of encoding signals such that sufficient information is preserved about their pair wise distances. For that goal, we consider randomized embeddings as an encoding mechanism and provide a framework to analyze their performance. We also propose the recently developed universal quantized embeddings as a solution to that problem and experimentally demonstrate that, in image retrieval experiments, universal embedding can achieve up to 25% rate reduction over the state of the art.

In classical spotlight-mode synthetic aperture radar (SAR), a mobile sensor array is steered to always focus on the same area (spot) as it moves, transmitting pulses to illuminate the spot and receiving and processing their reflections. The result is a high resolution image, covering a relatively small area. In this paper, we propose using a randomly steerable sensor array for synthetic aperture imaging, aiming to increase the coverage area without sacrificing the imaging resolution. This is realized by steering the beam of the array such that each transmitted pulse illuminates one of two or more spots, randomly selected with equal probability. Each of those spots has the same size as a single spot in a classical array, effectively doubling (or more) the total area illuminated. Using principles from compressive sensing (CS) we demonstrate that it is possible to reconstruct the images of all illuminated areas by exploiting the structure of the reconstructed images. Our experimental results demonstrate that our random steerable array can double coverage with almost the same imaging resolution.

In this paper, we demonstrate that recently-developed sparse recovery algorithms can be used to improve source localization in reverberant environments. By formulating the localization problem in the frequency domain, we are able to efficiently incorporate information that exploits the reverberation instead of considering it a nuisance to be eliminated. In this formulation, localization becomes a joint-sparsity support recovery problem which can be solved using model-based methods. We also develop a location model which further improves performance. Using our approach, we are able to recover more sources that the number of sensors. In contrast to conventional wisdom, we demonstrate that reverberation is beneficial in source localization, as long as it known and properly accounted for.

In this paper, we develop a new approach to synthetic aperture imaging inspired by recently developed compressive sensing (CS) methods. Our approach modifies the beam steering pattern of conventional sliding spotlight-mode systems and randomizes it such that with each pulse the beam illuminates a different, randomly chosen, part of the imaged area. The randomization allows the acquisition of the area of interest with a significantly larger effective aperture compared to the conventional sliding spotlight mode and, therefore, with significantly larger resolution. The reconstruction estimates the signal using a model that combines a sparse and a dense component. This model captures the structure of SAR images better than conventional sparse models, typically used in CS, and provides superior reconstruction performance. Our experimental results demonstrate that the proposed randomly steered spotlight array can improve imaging resolution, as measured by the reconstruction SNR and the phase error, without compromising the covered area size.

The compressive sensing (CS) framework aims to ease the burden on analog-to-digital converters (ADCs) by reducing the sampling rate required to acquire and stably recover sparse signals. Practical ADCs not only sample but also quantize each measurement to a finite number of bits; moreover, there is an inverse relationship between the achievable sampling rate and the bit depth. In this paper, we investigate an alternative CS approach that shifts the emphasis from the sampling rate to the number of bits per measurement. In particular, we explore the extreme case of 1-bit CS measurements, which capture just their sign. Our results come in two flavors. First, we consider ideal reconstruction from noiseless 1-bit measurements and provide a lower bound on the best achievable reconstruction error. We also demonstrate that i.i.d. random Gaussian matrices provide measurement mappings that, with overwhelming probability, achieve nearly optimal error decay. Next, we consider reconstruction robustness to measurement errors and noise and introduce the binary$\epsilon $-stable embedding property, which characterizes the robustness of the measurement process to sign changes. We show that the same class of matrices that provide almost optimal noiseless performance also enable such a robust mapping. On the practical side, we introduce the binary iterative hard thresholding algorithm for signal reconstruction from 1-bit measurements that offers state-of-the-art performance.

We examine the use of active coherent sensing-an increasingly available technology-for sensing the depth of scenes. A scene is a sparse signal but also exhibits significant structure which cannot be exploited using standard sparse recovery algorithms. Instead, inspired by the model-based compressive sensing literature we develop a scene model that incorporates occlusion constraints in recovering the depth map. Our model is computationally tractable; we develop a variation of the well-known model-based Compressive Sampling Matching Pursuit (CoSaMP) algorithm, and we demonstrate that our approach significantly improves reconstruction performance.

Pan-sharpening is an image fusion process in which high resolution (HR) panchromatic (Pan) imagery is used to sharpen the corresponding low resolution (LR) multi-spectral (MS) imagery. Pan-sharpened MS images generally have high spatial resolutions, but exhibit color distortions. In this paper, we propose a dictionary learning based pan-sharpening process to reduce the color distortion caused by the interpolation of the MS imagery. Instead of interpolating the LR MS image before fusion, we generate an improved MS image which is sparse with respect to a dictionary learned from the image data. Our experiments on degraded QuickBird and IKONOS images demonstrate that the distortion in the MS images produced using our approach is significantly reduced.

We develop a new method for tracking narrowband signals acquired via compressive sensing. The compressive sensing phase-locked loop (CS-PLL) enables one to track oscillating signals in very large bandwidths using sub-Nyquist sampling. A key feature of the approach is the fact that we perform the frequency tracking directly on the compressive measurements without ever recovering the signal. The CS-PLL has a wide variety of potential applications, including communications, phase tracking, and robust control.

In satellite image processing, pan-sharpening is the fusion process in which a low resolution (LR) multi-spectral (MS) image is sharpened using the corresponding high resolution (HR) panchromatic (Pan) image to obtain a HR MS image. In this paper we propose a novel pan-sharpening method which combines the ideas of classical wavelet-based pan-sharpening with recently developed dictionary learning (DL) methods. The HR MS image is generated using wavelet-based pan-sharpening, regulated by promoting sparsity with respect to a dictionary. The dictionary is obtained using DL on the multi-scale wavelet tree vectors of the image to be pan-sharpened. A significant advantage of our approach compared to most DL-based approaches is that it does not require a large database of images on which to train the dictionary. Experiments on degraded satellite images demonstrate that our method significantly reduces color distortions and wavelet artifacts compared to the state of the art.

Randomized embeddings of scale-invariant image features are proposed for retrieval of object-specific meta data in an augmented reality application. The method extracts scale invariant features from a query image, computes a small number of quantized random projections of these features, and sends them to a database server. The server performs a nearest neighbor search in the space of the random projections and returns meta-data corresponding to the query image. Prior work has shown that binary embeddings of image features enable efficient image retrieval. This paper generalizes the prior art by characterizing the tradeoff between the number of random projections and the number of bits used to represent each projection. The theoretical results suggest a bit allocation scheme under a total bit rate constraint: It is often advisable to spend bits on a small number of finely quantized random measurements rather than on a large number of coarsely quantized random measurements. This theoretical result is corroborated via experimental study of the above tradeoff using the ZuBuD database. The proposed scheme achieves a retrieval accuracy up to 94% while requiring the mobile device to transmit only 2.5 kB to the database server, a significant improvement over 1-bit quantization schemes reported in prior art.

Scalar quantization is the most practical and straightforward approach to signal quantization. However, it has been shown that scalar quantization of oversampled or compressively sensed signals can be inefficient in terms of the rate-distortion tradeoff, especially as the oversampling rate or the sparsity of the signal increases. In this paper, we modify the scalar quantizer to have discontinuous quantization regions. We demonstrate that with this modification it is possible to achieve exponential decay of the quantization error as a function of the oversampling rate instead of the quadratic decay exhibited by current approaches. Our approach is universal in the sense that prior knowledge of the signal model is not necessary in the quantizer design, only in the reconstruction. Thus, we demonstrate that it is possible to reduce the quantization error by incorporating side information on the acquired signal, such as sparse signal models or signal similarity with known signals. In doing so, we establish a relationship between quantization performance and the Kolmogorov entropy of the signal model.

The advent of Compressive Sensing has provided significant mathematical tools to enhance the sensing capabilities of hardware devices. In this paper we apply Compressive Sensing to improve over-the-air ultrasonic sensing capabilities. We demonstrate that using an appropriate scene model it is possible to pose three-dimensional surface reconstruction of a scene as a sparse recovery problem. By transmitting incoherent wideband ultrasonic pulses and receiving their reflections a sensor array can sense the scene and reconstruct it using standard CS reconstruction algorithms. We further demonstrate that it possible to construct virtual arrays that exploit the sensors' motion. Thus we can obtain three-dimensional scene reconstruction using a linear mobile array.

The formation of synthetic aperture radar (SAR) images is formulated as an inverse problem, a flexible approach suitable for a variety of acquisition systems and signal models. This paper focuses on increasing robustness to data saturation, specifically by optimizing a one-sided quadratic cost function to promote consistency with the received data. We model the SAR acquisition process using a linear function and we present an efficient implementation of this function and its adjoint for use in iterative optimization algorithms. Improved image quality and robustness to saturation are observed in experiments on synthetic images. Preliminary work on controlling azimuth ambiguities and incorporating image models enables saturation-robust reconstruction from satellite SAR data as well.

A distributed coding algorithm is presented for compression of wavelet-transformed data. Data structures based on zerotrees are exploited for efficient compression of the significance map of wavelet coefficients. The coefficients are scanned in two stages, with a significance pass and refinement pass, similar to the SPIHT algorithm. The bits resulting from these passes are Slepian-Wolf coded using an LDPC syndrome code selected from a bank of available codes. A key realization is that, for each bitplane of the wavelet coefficients, the significance pass of the source data can be synchronized with that of the side information. This allows distributed compression of the significance pass. This is substantially different from previous mixed approaches in which the refinement pass was Slepian-Wolf coded, but the significance pass was coded independently. Rate-distortion results are presented for images from the ALOS AVNIR-2 multispectral dataset and compared against those obtained with SPIHT and JPEG2000.

Synthetic Aperture Radar (SAR) is a fundamental technology with significant impact in remote sensing applications. SAR relies on the motion of the radar platform to synthesize a large aperture, and achieve high resolution imaging of a large area. However, current strip-map SAR designs, relying on uniform pulsing, suffer from a fundamental trade-off between the azimuth resolution and the range coverage length. In this paper we overcome this trade-off using a randomized pulsing scheme combined with non-linear compressive sensing (CS) reconstruction. Our experimental results demonstrate significant improvement in the azimuth resolution using the proposed approach, without compromise on the range length of the imaged area.

Sparse representations have emerged as a powerful tool in signal and information processing, culminated by the success of new acquisition and processing techniques such as compressed sensing (CS). Fusion frames are very rich new signal representation methods that use collections of subspaces instead of vectors to represent signals. This work combines these exciting fields to introduce a new sparsity model for fusion frames. Signals that are sparse under the new model can be compressively sampled and uniquely reconstructed in ways similar to sparse signals using standard CS. The combination provides a promising new set of mathematical tools and signal models useful in a variety of applications. With the new model, a sparse signal has energy in very few of the subspaces of the fusion frame, although it does not need to be sparse within each of the subspaces it occupies. This sparsity model is captured using a mixedl1/l2norm for fusion frames. A signal sparse in a fusion frame can be sampled using very few random projections and exactly reconstructed using a convex optimization that minimizes this mixedl1/l2norm. The provided sampling conditions generalize coherence and RIP conditions used in standard CS theory. It is demonstrated that they are sufficient to guarantee sparse recovery of any signal sparse in our model. More over, a probabilistic analysis is provided using a stochastic model on the sparse signal that shows that under very mild conditions the probability of recovery failure decays exponentially with in creasing dimension of the subspaces.

In this paper we show that, surprisingly, it is possible to recover sparse signals from nonlinearly distorted measurements, even if the nonlinearity is unknown. Assuming just that the nonlinearity is monotonic, we use the only reliable information in the distorted measurements: their ordering. We demonstrate that this information is sufficient to recover the signal with high precision and present two approaches to do so. The first uses order statistics to compute the minimum mean square (MMSE) estimate of the undistorted measurements and use it with standard compressive sensing (CS) reconstruction algorithms. The second uses the principle of consistent reconstruction to develop a deterministic nonlinear reconstruction algorithm that ensures that measurements of the reconstructed signal have ordering consistent with the ordering of the distorted measurements. Our experiments demonstrate the superior performance of both approaches compared to standard CS methods.

The ability of Compressive Sensing (CS) to recover sparse signals from limited measurements has been recently exploited in computational imaging to acquire high-speed periodic and near-periodic videos using only a low-speed camera with coded exposure and intensive off-line processing. Each low-speed frame integrates a coded sequence of high-speed frames during its exposure time. The high-speed video can be reconstructed from the low-speed coded frames using a sparse recovery algorithm. This paper presents a new streaming CS algorithm specifically tailored to this application. Our streaming approach allows causal on-line acquisition and reconstruction of the video, with a small, controllable, and guaranteed buffer delay and low computational cost. The algorithm adapts to changes in the signal structure and, thus, outperforms the off-line algorithm in realistic signals.

This paper investigates the application of lossy distributed source coding to high resolution multispectral images. The choice of distributed source coding is motivated by the need for very low encoding complexity on space and airborne platforms. The data consists of red, blue, green and infra-red channels and is compressed in an asymmetric Wyner-Ziv setting. One image channel is compressed using traditional JPEG and transmitted to the ground station where it is available as side information for Wyner-Ziv coding of the other channels. Encoding is accomplished by quantizing the image data, applying a Low-Density Parity Check code to the remaining three image channels, and transmitting the resulting syndromes. At the ground station, the image data is recovered from the syndromes by exploiting the correlation in the frequency spectrum of the band being decoded and the JPEG-decoded side information band. In experiments with real uncompressed images obtained by a satellite, the rate-distortion performance is found to be vastly superior to JPEG compression of individual image channels and rivals that of JPEG2000 at much lower encoding complexity.

Classical sampling records the signal level at pre-determined time instances, usually uniformly spaced. An alternative implicit sampling model is to record the timing of pre-determined level crossings. Thus the signal dictates the sampling times but not the sampling levels. Logan's theorem provides sufficient conditions for a signal to be recoverable, within a scaling factor, from only the timing of its zero crossings. Unfortunately, recovery from noisy observations of the timings is not robust and usually fails to reproduce the original signal. To make the reconstruction robust this paper introduces the additional assumption that the signal is sparse in some basis. We reformulate the reconstruction problem as a minimization of a sparsity inducing cost function on the unit sphere and provide an algorithm to compute the solution. While the problem is not convex, simulation studies indicate that the algorithm converges in typical cases and produces the correct solution with very high probability.

We address post-silicon characterization of the unique gate delays and their timing distributions on each manufactured IC. Our proposed approach is based upon the new theory of compressed sensing. The first step in performing timing measurements is to find the sensitizable paths by traditional testing methods. Next, we show that the timing variations are sparse in the wavelet domain. The sparsity is exploited for estimation of the gate delays using the compressed sensing theory. This estimation method requires significantly less number of timing measurements compared to the case where the dependence between the gate delays is not directly integrated within the estimation framework. We discuss a number of applications for the new post-silicon timing characterization method. Experimental results on benchmark circuits show that using compressed sensing theory can characterize the post-silicon variations with a mean accurately of 95% in the pertinent sparse basis.

We introduce a new methodology for noninvasive post-silicon characterization of the unique static power profile (tomogram) of each manufactured chip. The total chip leakage is measured for multiple input vectors in a linear optimization framework where the unknowns are the gate leakage variations. We propose compressive sensing for fast extraction of the unknowns since the leakage tomogram contains correlations and can be sparsely represented. A key advantage of our approach is that it provides leakage variation estimates even for inaccessible gates. Experiments show that the methodology enables fast and accurate noninvasive extraction of leakage power characteristics.

Compressive sensing (CS) is a new signal acquisition technique for sparse and compressible signals. Rather than uniformly sampling the signal, CS computes inner products with randomized basis functions; the signal is then recovered by a convex optimization. Random CS measurements are universal in the sense that the same acquisition system is sufficient for signals sparse in any representation. This paper examines the quantization of strictly sparse, power-limited signals and concludes that CS with scalar quantization uses its allocated rate inefficiently. The results complement related work on the quantization of CS measurements of compressible signals.

This paper presents an algorithm to generate autoregressive random binary processes with predefined mean and predefined all-pole power spectrum, subject to specific constraints on the parameters of the all-pole spectrum. The process is generated recursively using a linear combination of the previously generated values to bias the generation of the next value. It is shown that an all-zero filter whitens the process, and, therefore, the process has an all-pole spectrum. The process is also described using an ergodic Markov chain, which is used to determine the appropriate initialization and to prove convergence if the algorithm is not initialized properly. The all-pole parameter range for which the algorithm is guaranteed to work is also derived. It is shown to be a linear constraint on the all-pole parameters and their magnitude, subject to the desired mean for the process. The example and simulations presented elucidate and confirm the theoretic developments.

In this paper, we tackle the problem of source localization by example. We present a methodology that allows a user to train a microphone array system using signals from a set of positions and trajectories and subsequently recall the localization information when presented with new input signals. To do so we present a new statistical model which is capable of accurately describing features from the cross spectra of the microphone signals so as to model the room responses from all positions of interest. We further extend this model to allow modeling of sequences of positions, thereby also enabling the learning and recognition of trajectories. Because of its learning nature this method provides practical advantages in setting up a microphone array, by not requiring favorable room acoustics, careful element positioning or uniformity of sensors. It also introduces an approach to localization which can be extended to other problems requiring models of transfer functions. We present tests on synthetic and real-world data and present the resulting recognition rates for a variety of situations

This work explores low complexity systems to compensate for coefficient erasures in frame representations of signals. Assuming linear synthesis with a pre-specified frame it is demonstrated that erasures can be compensated for even if the origin of the representation coefficients is not known. If the transmitter is aware of the erasure occurrence, the compensation is performed by projecting the erasure error to the remaining coefficients. Furthermore, it is demonstrated that the same compensation can be executed using a transmitter/receiver combination in which the transmitter is not aware of the erasure occurrence. The transmitter compensates for all the coefficients using projections, assuming an erasure will occur. The receiver undoes the compensation for the coefficients that have not been erased, thus maintaining the compensation only of the erased coefficients

Quantization noise shaping is commonly used in oversampled A/D and D/A converters. This paper considers quantization noise shaping for arbitrary frame expansions of signals based on generalizing the view of first order noise shaping as a compensation of the quantization error through a projection. Two levels of generalization are developed, one a special case of the other, and two different cost models are proposed to evaluate the quantizer structures. Within our framework, the implementation of the quantizer and reconstruction are computationally straightforward. The computational complexity is in the initial determination of frame vector ordering, which is part of the quantizer design and carried out off-line. We show that in the case of frame representation corresponding to uniform oversampling, the natural ordering implied by sequential time sampling is optimal. Furthermore, for general finite frame expansions, the problem of optimal ordering corresponds to known problems in graph theory.