Recovering multiband spectra from sub-Nyquist sampling is a prominent research area in signal processing, driven by its wide range of applications and the technical challenges it presents. These challenges demand novel algorithmic approaches tailored to specific scenarios. The problem becomes even more complex when spectral locations are unknown, leading to the development of Blind Multi-Band Sampling techniques. Despite several proposed solutions, a notable research gap persists. Signals with varying energies across different spectral bands often exhibit high-dynamic-range (HDR) features, and with a fixed bit budget, there is a trade-off between optimizing digital resolution and spanning HDR. In this paper, we address this challenge by leveraging the Unlimited Sensing Framework (USF). The interaction between modulo non-linearity and sub-Nyquist sampling induces aliasing in both the domain and range of the signal, further complicating the recovery process, especially with unknown spectral locations. To tackle these challenges, we propose a novel algorithm for blind multiband spectrum recovery from folded samples at sub-Nyquist rates. Importantly, we provide a theoretically guaranteed, perfect recovery at sub-Nyquist sampling rates. Our proof is constructive and leads to an efficient algorithm supported by a novel multi-channel sampling architecture. We validate our approach through numerical experiments, opening up new directions in theory, algorithms, and real-world applications for the field.

The problem of sub-Nyquist multiband sensing has numerous applications across various fields. Despite substantial algorithmic pursuits, the practical implementation of these methods is subject to limitations of digital acquisition via analog-to-digital converters (ADCs). Multiband reconstruction for high-dynamic-range signals that may saturate the ADC is a practical yet challenging problem in many applications. From a different perspective, the Unlimited Sensing Framework (USF) focuses on reconstructing large signals from folded samples yet requires oversampling. The key challenge of multiband reconstruction via sub-Nyquist USF lies in the fundamental stalemate between sub-Nyquist acquisition and the oversampling assumptions required for unfolding. In this paper, we propose a hardware-software co-design approach that enables multiband reconstruction from folded samples at sub-Nyquist rates. The key insight here is to trade the channel redundancy for temporal sampling rate. We extend the multi-coset sampling strategy to the USF context and design a novel reconstruction algorithm. We demonstrate the robustness of our method via Monte-Carlo experiments. Beyond numerical experiments, we build customized hardware and validate our approach through lab experiments. This demonstrates the capabilities of our method in real-world scenarios while creating new avenues and opportunities for the field.

Mainstream signal processing theory and methods are primarily designed for synchronous sampling architectures, where samples are captured at predefined time instants. While this fits well with Shannon’s framework, in the absence of synchronous structure, even fundamental tools like filtering and convolution break down. Alternatively, event-driven or time-encoded sampling offers a more efficient method by capturing signals only when an event occurs. This approach, reminiscent of the "spiking neuron" behavior in the brain, can lead to low-power electronic implementations. Unlike Shannon’s framework, measurements in this scheme are defined by asynchronous sampling, presenting unique challenges. One such open problem is performing spectral estimation from asynchronous samples. In this paper, we propose a novel approach that directly enables spectral estimation from asynchronous measurements. Empirically, our algorithm offers robust, high-resolution spectral information, with a lower sampling rate on trigger times. Beyond numerical experiments, we build an event-driven sampling hardware utilizing asynchronous sigma-delta modulators to validate our approach. These hardware experiments further demonstrate the robustness and practical applicability of our method.

Abstract. In recent years, computational time-of-flight (ToF) imaging has emerged as an exciting and novel imaging modality that offers new and powerful interpretations of natural scenes, with applications extending to three-dimensional, light-in-flight, and non-line-of-sight imaging. Mathematically, ToF imaging relies on algorithmic super-resolution, as the back-scattered sparse light echoes lie on a finer time resolution than what digital devices can capture. Traditional methods necessitate knowledge of the emitted light pulses or kernels and employ sparse deconvolution to recover scenes. Unlike previous approaches, this paper introduces a novel, blind ToF imaging technique that does not require kernel calibration and recovers sparse spikes on a continuum, rather than a discrete grid. By studying the shared characteristics of various ToF modalities, we capitalize on the fact that most physical pulses approximately satisfy the Strang–Fix conditions from approximation theory. This leads to a new mathematical formulation for sparse super-resolution. Our recovery approach uses an optimization method that is pivoted on an alternating minimization strategy. We benchmark our blind ToF method against traditional kernel calibration methods, which serve as the baseline. Extensive hardware experiments across different ToF modalities demonstrate the algorithmic advantages, flexibility, and empirical robustness of our approach. We show that our work facilitates super-resolution in scenarios where distinguishing between closely spaced objects is challenging, while maintaining performance comparable to known kernel situations. Examples of light-in-flight imaging and light-sweep videos highlight the practical benefits of our blind super-resolution method in enhancing the understanding of natural scenes.

In this paper, we address the problem of reconstructing multiband signals from modulo-folded, pointwise samples within the Unlimited Sensing Framework (USF). Focusing on a low-complexity, single-channel acquisition setup, we establish recovery guarantees demonstrating that sub-Nyquist sampling is achievable under the USF paradigm. In doing so, we also tighten the previous sampling theorem for bandpass signals. Our recovery algorithm demonstrates up to a 13x dynamic range improvement in hardware experiments with up to 6 spectral bands. These results enable practical high-dynamic-range multiband acquisition in scenarios previously limited by dynamic range and excessive oversampling.

Bandpass signals are fundamental to a number of applications. Such signal classes are characterized by high-frequency content. Thus, their digital acquisition is demanding due to the higher sampling rate requirement. Relaxing this requirement has been widely investigated in the literature in the context of bandpass sampling theory. However, such investigations do not address the problem of handling high dynamic range (HDR) inputs, which is inherent to real-life applications and the restrictions imposed by analog-to-digital converters (ADCs). Recent work has shown that one can acquire arbitrarily large bandpass signals at sub-Nyquist rates by leveraging the Unlimited Sensing Framework (USF). Taking a step further, in this paper, a dual-channel architecture that integrates USF and second-order bandpass sampling (SO-BPS) is presented. The result extends the valid sampling frequencies and enhances design flexibility. Furthermore, we show that the sampling rate of one channel can be further reduced, and provide a complementary recovery algorithm. Lastly, we validate the theory, asses algorithmic performance and demonstrate the effectiveness of the proposed solution for HDR bandpass sampling.

The problem of frequency estimation from sub-Nyquist samples has numerous applications across various disciplines and has been extensively studied in signal processing literature. Despite the existence of several algorithmic approaches, the full potential of these methods has not been realized due to the limitations of analog-to-digital converters (ADCs). In particular, accurately estimating frequency for high-dynamic-range signals that may saturate the ADC is still an interesting problem, regardless of the sub-Nyquist aspect. On a different note, the Unlimited Sensing Framework (USF) focuses on recovering large signals from folded samples but requires oversampling. In this paper, we propose a hardware-software co-design approach that allows for frequency estimation from folded samples at sub-Nyquist rates. Our key insight is that temporal redundancy can be eliminated by introducing channel redundancy. Surprisingly, our recovery guarantees are independent of the sampling rate. To achieve this, we introduce a novel multi-channel sampling pipeline coupled with a reconstruction algorithm. Beyond numerical experiments, we build customized hardware and validate our approach through lab experiments. This demonstrates the capabilities of our method in real-world scenarios while opening up new questions for the field.

Sampling theory in fractional Fourier Transform (FrFT) domain has been studied extensively in the last decades. This interest stems from the ability of the FrFT to generalize the traditional Fourier Transform, broadening the traditional concept of bandwidth and accommodating a wider range of functions that may not be bandlimited in the Fourier sense. Beyond bandlimited functions, sampling and recovery of sparse signals has also been studied in the FrFT domain. Existing methods for sparse recovery typically operate in the transform domain, capitalizing on the spectral features of spikes in the FrFT domain. Our paper contributes two new theoretical advancements in this area. First, we introduce a novel time-domain sparse recovery method that avoids the typical bottlenecks of transform domain methods, such as spectral leakage. This method is backed by a sparse sampling theorem applicable to arbitrary FrFT-bandlimited kernels and is validated through a hardware experiment. Second, we present Cramér–Rao Bounds for the sparse sampling problem, addressing a gap in existing literature.

In this paper, the trade-off between the quantization noise and the dynamic range of ADCs used to acquire radar signals is revisited using the Unlimited Sensing Framework (USF) in a practical setting. Trade-offs between saturation and resolution arise in many applications, like radar, where sensors acquire signals which exhibit a high degree of variability in amplitude. To solve this issue, we propose the use of the co-design approach of the USF which acquires folded version of the signal of interest and leverages its structure to reconstruct it after its acquisition. We demonstrate that this method outperforms other standard acquisition methods for Doppler radars. We show this theoretically by providing mathematical insights on why the perfect reconstruction of Doppler signals from their folded measurements is possible. Our findings are corroborated via numerical simulations. Taking our theory all the way to practice, we develop a prototype USF-enabled Doppler Radar and show the clear benefits of our method. In each experiment, we show that using the USF increases sensitivity compared to a classic acquisition approach.

Unlimited Sampling Framework (USF) is a digital acquisition protocol that recovers high dynamic range (HDR) input signals from their low dynamic range, modulo samples. Current USF theory and algorithms are predominantly focused on bandlimited signal classes that rely on a relatively high sampling rate. Recently, the "Fourier-Prony" algorithm was proposed and validated via hardware experiments with modulo ADCs. It was shown that this algorithm offers competitive performance in the presence of system noise and quantization, especially when periodic boundary conditions are satisfied.In practice, signals are often measured over a finite observation window and this implies leakage in the Fourier domain. Depending on the severity of spectral leakage, Fourier domain algorithms may fail to reconstruct. To overcome this bottleneck, in this paper, we propose an Iterative Signal Sieving Algorithm (ITER-SIS) that solely operates in the time domain. By utilizing a continuous-domain characterization of modulo samples, ITER-SIS achieves a robust, low-sampling-rate, FFT-free recovery of signals with a finite time observation window, even when there is considerable spectral leakage. Hardware experiments with the modulo ADC demonstrate the robustness of our method in a realistic, noisy and low-sampling rate settings, thus validating its high practical utility in a variety of applications.

To achieve High Dynamic Range (HDR) sensing, the Unlimited Sampling Framework (USF) was recently proposed. In the USF, modulo encoding of the continuous-time input signal prevents the analog-to- digital converter (ADC) from saturation. For recovering the HDR signal from folded samples, reconstruction algorithms are utilized. Current USF pipeline is highly focused on bandlimited signal classes and requires considerable oversampling. In contrast, in this paper, we consider non-bandlimited signals, in particular, sparse inputs with finite-rate-of-innovation (FRI). By devising a novel, dual-channel modulo sampling architecture we show that, surprisingly, sparse signal recovery from modulo samples can be performed independent of the sampling rate. We validate the effectivity of our sampling scheme and show that perfect signal reconstruction is achieved up to machine precision.

The Unlimited Sampling Framework (USF) is an alternative sampling and reconstruction protocol that allows for recovery of signals that are arbitrarily larger than the dynamic range of the analog-to-digital converter (ADC). In the USF, modulo non-linearity prior to sampling folds high dynamic range inputs that may potentially saturate a conventional ADC. In the recovery step, algorithms perform signal unfolding. Current theory backing the USF is predominantly focused towards Fourier bandlimited function classes.By generalizing the notion of bandlimitedness to a much larger class of unitary transforms, in this work, we propose Unlimited Sampling in Phase Space. The advantages are two-fold. Firstly, we generalize the USF to a much larger class of unitary transforms. Secondly, functions such us chirps are non-bandlimited in the Fourier domain but bandlimited in the phase space. This allows for practical applications that can not be handled by current, Fourier domain theory. Our main contribution in this paper is a novel modulo acquisition pipeline in phase space and a mathematically guaranteed recovery algorithm that is also backwards compatible with Fourier domain theory. Computer experiments validate our theory and reinforce its potential for new application areas such as radar and optics.

Massive multiple-input multiple-output (M-MIMO) architecture is the workhorse of modern communication systems. Currently, two fundamental bottlenecks, namely, power consumption and receiver saturation, limit the full potential achievement of this technology. These bottlenecks are intricately linked with the analog-to-digital converter (ADC) used in each radio frequency (RF) chain. The power consumption in M-MIMO systems grows exponentially with the ADC’s bit budget while ADC saturation causes permanent loss of information. This motivates the need for a solution that can simultaneously tackle the above-mentioned bottlenecks while offering advantages over existing alternatives such as low-resolution ADCs. Taking a radically different approach to this problem, we propose$\lambda $–MIMO architecture which uses modulo ADCs ($\mathscr {M}_{\lambda} $–ADC) instead of a conventional ADC. Our work is inspired by the Unlimited Sampling Framework.$\mathscr {M}_{\lambda} $–ADC in the RF chain folds high dynamic range signals into low dynamic range modulo samples, thus alleviating the ADC saturation problem. At the same time, digitization of modulo signal results in high resolution quantization. In the novel$\lambda $–MIMO context, we discuss baseband signal reconstruction, detection and uplink achievable sum-rate performance. The key takeaways of our work include, (a) leveraging higher signal-to-quantization noise ratio (SQNR), (b) detection and average uplink sum-rate performances comparable to a conventional, infinite-resolution ADC when using a 1–2 bit$\mathscr {M}_{\lambda} $–ADC. This enables higher order modulation schemes e.g., 1024 QAM that seemed previously impossible, (c) superior trade-off between energy efficiency and bit budget, thus resulting in higher power efficiency. Numerical simulations and modulo ADC based hardware experiments corroborate our theory and reinforce the clear benefits of$\lambda $–MIMO approach.

Unlimited Sensing is a sampling protocol that recovers high dynamic range input signals from their low dynamic range, modulo samples. Bridging the gap between theory and practice, recently, a hardware validation of the unlimited sampling method was presented. Taking another step in this direction, in this paper, we study the problem of recovery from modulo samples contaminated by sparse outliers (noise). Our hardware experiments suggest that impulsive and jump or reset noise can be sources of sparse outliers in the measurements. Such a noise model has not been considered in literature and can lead to the breakdown of the conventional recovery methods. To overcome this problem, we present a mathematically guaranteed algorithm that is based on spectral estimation. Our method perfectly recovers the signal (up to a constant) when the sampling criterion is met and no other noise sources are present. In real experiments where quantization and system noise (e.g. additive Gaussian) play a role, our approach offers a competitive performance. Hardware experiments with our modulo ADC validate the practical utility of our method.

Signal saturation or clipping is a fundamental bottleneck that limits the capability of analog-to-digital converters (ADCs). The problem arises when the input signal dynamic range is larger than ADC’s dynamic range. To overcome this issue, an alternative acquisition protocol called the Unlimited Sensing Framework (USF) was recently proposed. This non-linear sensing scheme incorporates signal folding (via modulo non-linearity) before sampling. Reconstruction then entails "unfolding" of the high dynamic range input. Taking an end-to-end approach to the USF, a hardware validation called US-ADC was recently presented. US-ADC experiments show that, in some scenarios, the samples can be more accurately modelled as local averages than ideal, pointwise measurements. In particular, this happens when the input signal frequency is much larger than the operational bandwidth of the US-ADC. Pushing such hardware limits using computational approaches motivates the study of modulo sampling and reconstruction via local averages. By incorporating a modulo-hysteresis model, both in theory and in hardware, we present a guaranteed recovery algorithm for input reconstruction. We also explore a practical method suited for low sampling rates. Our approach is validated via simulations and experiments on hardware, thus enabling a step closer to practice.

In event-driven sampling (EDS) the signal is represented in terms of a series of spikes at non-uniform time locations. Owing to the limited dynamic range (DR), just like how conventional analog-to-digital converters (ADC) suffer from signal saturation, in EDS a similar manifestation is observed. Namely, when the input exceeds a threshold, no output spikes are generated. Recently, the Unlimited Sensing Framework (USF) was presented to overcome the DR limitation. In USF, the key idea is to fold the signal using a modulo non-linearity so that its DR is fixed. Subsequently, we combined EDS with USF leading to a new architecture called Modulo Event-Driven Sampling (MEDS), where a modulo signal is input to the EDS model. The goal of this work is to bridge the gap between theory and practice for a MEDS model. Our hardware experiments suggest that for the MEDS approach to work, there are system parameters that must be identified beforehand so that accurate reconstruction of the input is possible. To this end, we introduce a system identification methodology for MEDS that is backed by theoretical guarantees. Using synthetic and experimental data, we validate the performance of our approach, thus demonstrating the utility of system identification.

In the recent years, practitioners in the area of tomography have proposed high dynamic range (HDR) solutions that are inspired by the multi-exposure fusion strategy in computational photography. To this end, multiple Radon Transform projections are acquired at different exposures that are algorithmically fused to facilitate HDR re-construction. A single-shot alternative to multi-exposure fusion approach has been proposed in our recent line of work which is based on the Modulo Radon Transform (MRT). In this case, Radon Transform projections are folded via modulo non-linearity. This folding allows HDR values to be mapped into the dynamic range of the sensor and, thus, avoids saturation or clipping. The folded measurements are then mapped back to their ambient range using algorithms. The main goal of this paper is to introduce a novel, Fourier domain recovery method, namely, the OMP-FBP method, which is based on the Orthogonal Matching Pursuit (OMP) algorithm and Filtered Back Projection (FBP) formula. The proposed OMP-FBP method offers several advantages; it is agnostic to the modulo threshold or the number of folds, can handle much lower sampling rates than previous approaches and is empirically stable to noise and outliers. Computer simulations as well as hardware experiments in the paper validate the effectivity of the OMP-FBP recovery method.

Time-of-Flight (ToF) imagers, e.g. Microsoft Kinect, are active devices that offer a portable, efficient and a consumer-grade solution to three dimensional imaging problems. As the name suggests, in ToF imaging, back scattered light from an active illumination source (typically a sinusoid) is used to measure the ToF, thus resulting in depth information. Despite its prevalence in applications such as autonomous navigation and scientific imaging, current ToF sensors are limited in their dynamic range. Computational imaging solutions enabling high dynamic range (HDR) ToF imaging are largely unexplored. We take a step in this direction by proposing a novel architecture for HDR ToF imaging; we combine ToF imaging with the recently introduced Unlimited Sensing Framework. By considering modulo sampling at each ToF pixel, HDR signals are folded back in the conventional dynamic range. Our work offers a single-shot solution for HDR ToF imaging. We report a sampling density criterion that guarantees inversion of modulo non-linearity. Furthermore, we also present a new algorithm for ToF recovery that circumvents the need for unfolding of modulo samples. Numerical examples based on the Stanford 3D Scanning Repository highlight the merits of our approach, thus paving a path for a novel imaging architecture.

The Unlimited Sensing Framework (USF) provides an alternative protocol for high dynamic range (HDR) acquisition of real world signals. By incorporating a modulo non-linearity prior to sampling, an HDR signal, which is prone to clipping due to sensor saturation, is folded back into the dynamic range of the analog-to-digital converter (ADC). Thereafter, the modulo samples are algorithmically unfolded to their native dynamic range, thus allowing for recovery of HDR input signals. For bandlimited functions, a sampling density criterion akin to the Shannon-Nyquist theorem guarantees recovery from the modulo samples. Recently, a hardware implementation of the modulo ADC has motivated a generalized acquisition model called modulo-hysteresis that can handle non-idealities observed in practice. The recovery guarantees for this model are currently based on finite difference filters. Such filters can not handle practical scenarios where noise and perturbation play a role. The main goal of this work is to introduce a principled approach that explains the role of general filters for signal recovery. Based on the modulo-hysteresis acquisition model, we formulate input recovery guarantees for thresholding with general filters and give numerical simulations to show cases where the finite difference filter is not an optimal choice for reconstruction.

Recently, experiments have been reported where researchers were able to perform high dynamic range (HDR) tomography in a heuristic fashion, by fusing multiple tomographic projections. This approach to HDR tomography has been inspired by HDR photography and inherits the same disadvantages. Taking a computational imaging approach to the HDR tomography problem, we here suggest a new model based on the modulo Radon transform (MRT), which we rigorously introduce and analyze. By harnessing a joint design between hardware and algorithms, we present a single-shot HDR tomography approach, which to our knowledge, is the only approach that is backed by mathematical guarantees. On the hardware front, instead of recording the Radon transform projections that may potentially saturate, we propose to measure modulo values of the same. This ensures that the HDR measurements are folded into a lower dynamic range. On the algorithmic front, our recovery algorithms reconstruct the HDR images from folded measurements. Beyond mathematical aspects such as injectivity and inversion of the MRT for different scenarios including band-limited and approximately compactly supported images, we also provide a first proof-of-concept demonstration. To do so, we implement MRT by experimentally folding tomographic measurements available as an open source dataset using our custom designed modulo hardware. Our reconstruction clearly shows the advantages of our approach for experimental data. In this way, our MRT based solution paves a path for HDR acquisition in a number of related imaging problems.

The Unlimited Sensing Framework (USF) is a digital acquisition protocol that allows for sampling and reconstruction of high dynamic range signals. By acquiring modulo samples, the USF circumvents the clipping or saturation problem that is a fundamental bottleneck in conventional analog-to-digital converters (ADCs). In the context of the USF, several works have focused on bandlimited function classes and recently, a hardware validation of the modulo sampling approach has been presented. In a different direction, in this paper we focus on non-bandlimited function classes and consider the well-known super-resolution problem; we study the recovery of sparse signals (Dirac impulses) from low-pass filtered, modulo samples. Taking an end-to-end approach to USF based super-resolution, we present a novel recovery algorithm (US-SR) that leverages a doubly sparse structure of the modulo samples. We derive a sampling criterion for the US-SR method. A hardware experiment with the modulo ADC demonstrates the empirical robustness of our method in a realistic, noisy setting, thus validating its practical utility.

In contrast to Shannon sampling theory, where measurements are recorded at equally-spaced time instants, event-driven sampling records values at non-uniform instants dependent on the input. However, both sampling schemes are subject to input dynamic range constraints. This represents a fundamental bottleneck, which can only be alleviated via adjustments in the encoder architecture. Here we explore an alternative strategy based on the recent work on Unlimited Sampling theory, which uses a modulo non-linearity to guarantee a predefined input amplitude range. We propose a cascade model comprising a modulo non-linearity in series with an integrate-and-fire (IF) event-driven encoder. The modulo component does not act on inputs within the IF dynamic range, thus our model is fully compatible with the existing IF methodology. For inputs outside the IF dynamic range, the modulo output is discontinuous, and it currently cannot be recovered from the IF output with existing methods. We introduce theoretical conditions for which the input of the proposed cascade model can be recovered with arbitrary precision. Through numerical simulations, we show the performance of the reconstruction algorithm. The proposed methodology paves the way for a new generation of event-driven models suitable for a much wider range of applications.

The fractional Fourier transform has found applications in a variety of topics linked with science and engineering. In this context, sampling theory is one of the most well-studied subjects. Since the fractional Fourier transform or the FrFT generalizes the notion of bandlimitedness, extension of Shannon's sampling theorem to the FrFT domain generalizes the classical result for the Fourier domain. These ideas have further been extended to the class of non-bandlimited functions via shift-invariant subspaces and sparse models. In this paper, we discuss a different approach to sampling theory in the FrFT domain. For the first time, we propose sampling and recovery of bandlimited functions in the FrFT domain that is based on one-bit samples. Our work is inspired by the Sigma-Delta quantization scheme. In particular, we capitalize on the idea of noise shaping and develop a one-bit sampling architecture that allows for recovery of bandlimited functions in the FrFT domain by pushing quantization noise to the higher frequencies. Since the FrFT generalizes the Fourier transform, our work results in a generalized Sigma-Delta architecture. We validate our theoretical concepts through computer experiments and provide an approximation theoretic error bound.

The topic of high dynamic range (HDR) tomography is starting to gain attention due to recent advances in the hardware technology. Registering high-intensity projections that exceed the dynamic range of the detector cause sensor saturation. Existing methods rely on the fusion of multiple exposures. In contrast, we propose a one-shot solution based on the Modulo Radon Transform (MRT). By exploiting the modulo non-linearity, the MRT encodes folded Radon Transform projections so that the resulting measurements do not saturate. Our recovery strategy is pivoted around a property we call compactly λ-supported, which is motivated by practice; in many applications the object to be recovered is of finite extent and the measured quantity has approximately compact support. Our theoretical results are illustrated by numerical simulations with an open-access X-ray tomographic dataset and lead to substantial improvement in the HDR recovery problem. For instance, we report recovery of objects with projections 1000x larger in amplitude than the detector threshold.

Quantization is an integral part of image acquisition but also a major performance bottleneck due to the trade-off between dynamic range and resolution. As we discuss in this paper, in contrast, quantization noise can be acquired reliably even beyond the dynamic range by re-purposing recent hardware development. In this paper, we introduce and mathematically analyze an algorithm to recover images from this information, thus giving rise to a novel, single-shot, high-dynamic-range (HDR) imaging approach. Our method directly works with a refined model for sensor outputs at the digitization stage and crucially exploits smoothing anti-aliasing artifacts. We derive recovery guarantees and demonstrate the validity of our approach via computer experiments. Our work suggests re-thinking of the imaging pipeline as seeming sensing artifacts can lead to improved reconstruction when combined with proper computational methodology.

Spatial resolution is one of the fundamental bottlenecks in the area of time-resolved imaging. Since each pixel measures a scene-dependent time profile, there is a technological limit on the size of pixel arrays that can be simultaneously used to perform measurements. To overcome this barrier, in this paper, we propose a low-complexity, one-bit sensing scheme. On the data capture front, the time-resolved measurements are mapped to a sequence of +1 and -1. This leads to an extremely simple implementation and at the same time poses a new form of information loss. On the image recovery front, our one-bit time-resolved imaging scheme is complemented with a non-iterative recovery algorithm that can handle the case of single and multiple light paths. Extensive computer simulations and physical experiments benchmarked against conventional Time-of-Flight imaging data corroborate our theoretical framework. Thus, our low-complexity alternative to time-resolved imaging can indeed potentially lead to a new imaging methodology.

Super-resolution is the art of recovering spikes from their low-pass projections. Over the last decade specifically, several significant advancements linked with mathematical guarantees and recovery algorithms have been made. Most super-resolution algorithms rely on a two-step procedure: deconvolution followed by high-resolution frequency estimation. However, for this to work, exact bandwidth of low-pass filter must be known; an assumption that is central to the mathematical model of super-resolution. On the flip side, when it comes to practice, smoothness rather than bandlimitedness is a much more applicable property. Since smooth pulses decay quickly, one may still capitalize on the existing super-resolution algorithms provided that the essential bandwidth is known. This problem has not been discussed in literature and is the theme of our work. In this paper, we start with an experiment to show that super-resolution in the presence of noise is sensitive to bandwidth selection. This raises the question of how to select the optimal bandwidth. To this end, we propose a bandwidth selection criterion which works by minimizing a proxy of estimation error that is dependent of bandwidth. Our criterion is easy to compute, and gives reasonable results for experimentally acquired data, thus opening interesting avenues for further investigation, for instance the relationship to Cramér-Rao bounds.

Conventional analog-to-digital converters (ADCs) are limited in dynamic range. If a signal exceeds some prefixed threshold, the ADC saturates and the resulting signal is clipped, thus becoming prone to aliasing artifacts. Recent developments in ADC design allow to overcome this limitation: using modulo operation, the so called self-reset ADCs fold amplitudes which exceed the dynamic range. A new (unlimited) sampling theory is currently being developed in the context of this novel class of ADCs. In this paper, we make a further step in this direction by coupling modulo sampling with one-bit ΣΔ quantization, or, in other words, consider one-bit unlimited sampling. We show that our scheme overcomes the dynamic range limitations of conventional one-bit quantizer, where no recovery guarantees are possible if the signal's dynamic range substantially exceeds the range of its one-bit output. We provide a constructive recovery algorithm for bandlimited signals from one-bit modulo samples complemented with a bound on the reconstruction error.

In a recent paper [1], we introduced the concept of “Unlimited Sampling”. This unique approach circumvents the clipping or saturation problem in conventional analog-to-digital converters (ADCs) by considering a radically different ADC architecture which resets the input voltage before saturation. Such ADCs, also known as Self-Reset ADCs (SR-ADCs), allow for sensing modulo samples. In analogy to Shannon's sampling theorem, the unlimited sampling theorem proves that a bandlimited signal can be recovered from modulo samples provided that a certain sampling density criterion, that is independent of the ADC threshold, is satisfied. In this way, our result allows for perfect recovery of a bandlimited function whose amplitude exceeds the ADC threshold by orders of magnitude. By capitalizing on this result, in this paper, we consider the inverse problem of recovering a sparse signal from its low-pass filtered version. This problem frequently arises in several areas of science and engineering and in context of signal processing, it is studied in several flavors, namely, sparse or FRI sampling, super-resolution and sparse deconvolution. By considering the SR-ADC architecture, we develop a sampling theory for modulo sampling of lowpass filtered spikes. Our main result consists of a new sparse sampling theorem and an algorithm which stably recovers a K -sparse signal from low-pass, modulo samples. We validate our results using numerical experiments.

In parallel to Shannon's sampling theorem, the recent theory of unlimited sampling yields that a bandlimited function with high dynamic range can be recovered exactly from oversampled, low dynamic range samples. In this way, the unlimited sampling methodology circumvents the dynamic range problem that limits the use of conventional analog-to-digital converters (ADCs) which are prone to clipping or saturation problem. The unlimited sampling theorem is made practicable by using a unique ADC architecture-the self-reset ADC or the SR-ADC-which resets voltage before clipping, thus producing modulo or wrapped samples. While retaining full dynamic range of the input signal, surprisingly, the sampling density prescribed by the unlimited sampling theorem is independent of the maximum recordable voltage of the new ADC and depends only on the signal bandwidth. As the corresponding problem of signal recovery from such modulo samples arises in various applications with different signal models, where the original result does not directly apply, the original paper continues to trigger research follow-ups. In this paper, we investigate the case of sampling and reconstruction of a mixture of K sinusoids from such modulo samples. This problem is at the heart of spectral estimation theory and application areas include active sensing, ranging, source localization, interferometry and direction-of-arrival estimation. By relying on the SR-ADCs, we develop a method for recovery of K-sparse, sum-of-sinusoids from finitely many wrapped samples, thus avoiding clipping or saturation. As our signal model is completely characterized by K pairs of amplitudes and frequencies, we obtain a parametric sampling theorem; we complement it with a recovery algorithm. Numerical demonstrations validate the effectivity of our approach.

The field of signal processing is replete with exemplary problems where the measurements amount to time-delayed and amplitude scaled echoes of some template function or a pulse. When the inter-pulse spacing is favorable, something as primitive as a matched filter serves the purpose of identifying time-delays and amplitudes. When the inter-pulse spacing poses an algorithmic challenge, high-resolution methods such as finite-rate-of-innovation (FRI) may be used. However, in many practical cases of interest, the template function may be distorted due to physical properties of propagation and transmission. Such cases can not be handled well by existing signal models. Inspired by problems in spectroscopy, radar, photoacoustic imaging and ultra-wide band arrays, on which we base our case studies, in this work we take a step towards recovering spikes from time-varying pulses. To this end, we re-purpose the FRI method and extend its utility to the case of phase distorted pulses. Application of our algorithm on the above-mentioned case studies results in substantial improvement in peak-signal-to-noise ratio, thus promising interesting future directions.

This paper considers the problem of sampling and reconstruction of a continuous-time sparse signal without assuming the knowledge of the sampling instants or the sampling rate. This topic has its roots in the problem of recovering multiple echoes of light from its low-pass filtered and auto-correlated, time-domain measurements. Our work is closely related to the topic of sparse phase retrieval and in this context, we discuss the advantage of phase-free measurements. While this problem is ill-posed, cues based on physical constraints allow for its appropriate regularization. We validate our theory with experiments based on customized, optical time-of-flight imaging sensors. What singles out our approach is that our sensing method allows for temporal phase retrieval as opposed to the usual case of spatial phase retrieval. Preliminary experiments and results demonstrate a compelling capability of our phaseretrieval based imaging device.

When multiple light paths combine at a given location on an image sensor, an image mixture is created. Demixing or recovering the original constituent components in such cases is a highly ill-posed problem. A number of elegant solutions have thus been developed in the literature, relying on measurement diversity such as polarization, shift, motion, or scene features. In this paper, we approach the image-mixing problem as a time-resolved phenomenon-if every photon arriving at the sensor could be time-stamped, the demixing problem would then amount to separating transient events in time. Based on this idea, we first show that, while acquiring measurements is prohibitive and challenging in the time domain, this task is surprisingly straightforward in the frequency domain. We then establish a link between frequency-domain measurements and consumer time-of-flight (ToF) imaging. Finally, we propose a demixing algorithm, relying only on magnitude information of the ToF sensor. We show that our problem is closely tied to the topic of phase retrieval and that for K-image mixture, (K2-K)/2+1 magnitude-only ToF measurements suffice to demix images exactly in noiseless settings. Our developments are corroborated with experiments on synthetic and ToF data acquired using the Microsoft Kinect sensor.

Finite-Rate-of-Innovation (FRI) sampling theory prescribes a procedure for exact recovery of Dirac impulses from linear measurements in the form of orthogonal projections of streams of Dirac impulses onto the subspace of Fourier—bandlimited functions. This enables recovery of a continuous time sparse signals at sub-Nyquist rates. In many cases, the transform domain of interest may be more general than the Fourier domain. Recent work has extended FRI sampling theory to the spherical Fourier Transform, fractional Fourier Transform and the Laplace Transform. In this paper, we develop a broad FRI framework applicable to a general class of transformations that includes Fourier, Laplace, Fresnel, fractional Fourier, Bargmann and Gauss—Weierstrass transforms, among others. For this purpose, we consider the Special Affine Fourier Transform (SAFT) which parametrically generalizes a number of well known unitary transforms linked with signal processing and optics. We first derive a version of Shannon's sampling theory based on the convolution structure tailored for the SAFT domain. Having identified the subspace of SAFT—bandlimited functions, we apply FRI sampling theory to the SAFT and study recovery of sparse signals, thus providing a unified view of FRI sampling theory for a large class of disparately studied operations.

Optical time-of-flight (ToF) sensors can measure scene depth accurately by projection and reception of an optical signal. The range to a surface in the path of the emitted signal is proportional to the delay time of the light echo or the reflected signal. In practice, a diverging beam may be subject to multi-echo backscatter, and all these echoes must be resolved to estimate the multiple depths. In this paper, we propose a method for super-resolution of optical ToF signals. Our contributions are twofold. Starting with a general image formation model common to most ToF sensors, we draw a striking analogy of ToF systems with sampling theory. Based on our model, we reformulate the ToF super-resolution problem as a parameter estimation problem pivoted around the finite-rate-of-innovation framework. In particular, we show that super-resolution of multi-echo backscattered signal amounts to recovery of Dirac impulses from low-pass measurements. Our theory is corroborated by analysis of data collected from a photon counting, LiDAR sensor, showing the effectiveness of our non-iterative and computationally efficient algorithm.

This work considers the problem of super-resolution. The goal is to resolve a Dirac distribution from knowledge of its discrete, low-pass, Fourier measurements. Classically, such problems have been dealt with parameter estimation methods. Recently, it has been shown that convex-optimization based formulations facilitate a continuous time solution to the super-resolution problem. Here we treat super-resolution from low-pass measurements in Phase Space. The Phase Space transformation parametrically generalizes a number of well known unitary mappings such as the Fractional Fourier, Fresnel, Laplace and Fourier transforms. Consequently, our work provides a general super-resolution strategy which is backward compatible with the usual Fourier domain result. We consider low-pass measurements of Dirac distributions in Phase Space and show that the super-resolution problem can be cast as Total Variation minimization. Remarkably, even though are setting is quite general, the bounds on the minimum separation distance of Dirac distributions is comparable to existing methods.

In this paper, we consider the problem of Sparse Linear Operator identification which is also linked with the topic of Sparse Deconvolution. In its abstract form, the problem can be stated as follows: Given a well behaved probing function, is it possible to identify a Sparse Linear Operator from its response to the function? We present a constructive solution to this problem. Furthermore, our approach is devoid of any sparsity inducing penalty term and explores the idea of parametric modeling. Consequently, our algorithm is non-iterative by design and circumvents tuning of any regularization parameter. Our approach is computationally efficient when compared the ℓ0/ℓ1-norm regularized counterparts. Our work addresses a problem of industrial significance: decomposition of mixed-pixels in Time-of-Flight/Range imaging. In this case, each pixel records range measurements from multiple contributing depths and the goal is to isolate each depth. Practical experiments corroborate our theoretical set-up and establish the efficiency of our approach, that is, speed-up in processing with lesser mean squared error. We also derive Cramér-Rao Bounds for performance characterization.

Several computer vision algorithms require a sequence of photographs taken in different illumination conditions, which has spurred development in the area of illumination multiplexing. Various techniques for optimizing the multiplexing process already exist, but are geared toward regular or high speed cameras. Such cameras are fast, but code on the order of milliseconds. In this paper we propose a fusion of two popular contexts, time of flight range cameras and illumination multiplexing. Time of flight cameras are a low cost, consumer-oriented technology capable of acquiring range maps at 30 frames per second. Such cameras have a natural connection to conventional illumination multiplexing strategies as both paradigms rely on the capture of multiple shots and synchronized illumination. While previous work on illumination multiplexing has exploited coding at millisecond intervals, we repurpose sensors that are ordinarily used in time of flight imaging to demultiplex via nanosecond coding strategies.

We investigate the problem of identifying the position of a viewer inside a room of planar mirrors with unknown geometry in conjunction with the room's shape parameters. We consider the observations to consist of angularly resolved depth measurements of a single scene point that is being observed via many multi-bounce interactions with the specular room geometry. Applications of this problem statement include areas such as calibration, acoustic echo cancelation and time-of-flight imaging. We theoretically analyze the problem and derive sufficient conditions for a combination of convex room geometry, observer, and scene point to be reconstruct able. The resulting constructive algorithm is exponential in nature and, therefore, not directly applicable to practical scenarios. To counter the situation, we propose theoretically devised geometric constraints that enable an efficient pruning of the solution space and develop a heuristic randomized search algorithm that uses these constraints to obtain an effective solution. We demonstrate the effectiveness of our algorithm on extensive simulations as well as in a challenging real-world calibration scenario.

Time of flight cameras produce real-time range maps at a relatively low cost using continuous wave amplitude modulation and demodulation. However, they are geared to measure range (or phase) for a single reflected bounce of light and suffer from systematic errors due to multipath interference. We re-purpose the conventional time of flight device for a new goal: to recover per-pixel sparse time profiles expressed as a sequence of impulses. With this modification, we show that we can not only address multipath interference but also enable new applications such as recovering depth of near-transparent surfaces, looking through diffusers and creating time-profile movies of sweeping light. Our key idea is to formulate the forward amplitude modulated light propagation as a convolution with custom codes, record samples by introducing a simple sequence of electronic time delays, and perform sparse deconvolution to recover sequences of Diracs that correspond to multipath returns. Applications to computer vision include ranging of near-transparent objects and subsurface imaging through diffusers. Our low cost prototype may lead to new insights regarding forward and inverse problems in light transport.

Sampling theory for continuous time signals which have a bandlimited representation in fractional Fourier transform (FrFT) domain-a transformation which generalizes the conventional Fourier transform-has blossomed in the recent past. The mechanistic principles behind Shannon's sampling theorem for fractional bandlimited (or fractional Fourier bandlimited) signals are the same as for the Fourier domain case i.e. sampling (and reconstruction) in FrFT domain can be seen as an orthogonal projection of a signal onto a subspace of fractional bandlimited signals. As neat as this extension of Shannon's framework is, it inherits the same fundamental limitation that is prevalent in the Fourier regime-what happens if the signals have singularities in the time domain (or the signal has a nonbandlimited spectrum)? In this paper, we propose a uniform sampling and reconstruction scheme for a class of signals which are nonbandlimited in FrFT sense. Specifically, we assume that samples of a smoothed version of a periodic stream of Diracs (which is sparse in time-domain) are accessible. In its parametric form, this signal has a finite number of degrees of freedom per unit time. Based on the representation of this signal in FrFT domain, we derive conditions under which exact recovery of parameters of the signal is possible. Knowledge of these parameters leads to exact reconstruction of the original signal.

Fractional delay filters (FDFs) play an important role in certain areas of digital signal processing and communication engineering, where it is desirable to generate delays that are of the order of a fraction of the sampling period. In this paper, we advocate the use of generalized cardinal exponential splines-a class of compactly supported functions that is much richer than the existing B-spline family-for designing precision FDFs. One advantage of using generalized cardinal exponential splines is that it provides ready access to several spline families and other kernels which could be used for FDF design. The B-spline and Lagrange interpolator based FDFs are a special case of our proposition. We also discuss a design example and show that it is possible to design filters that have lower interpolation errors as compared to its B-spline counterparts.