Blue-noise stacked error diffusion (BSED) is a high-quality multi-level halftoning (multitoning) algorithm based on the blue-noise dithering model. This algorithm transforms a multitoning task into multiple error diffusion halftoning tasks, each producing a sub-halftone with inter-layer dependencies. These sub-halftones are stacked together to form the final visually pleasing multitone with blue-noise characteristics. In this research, the significant potential for parallelizing this multitoning algorithm inspires the design of a novel parallel raster image processor (NPRIP) for highly efficient execution, capable of producing an output pixel every six clock cycles at a clock frequency of 200 MHz. A hardware prototype based on the Xilinx Zynq SoC implements our raster image processor design on the FPGA within the SoC. The prototype is also capable of a conventional software implementation of the BSED algorithm running on the ARM CPU within the SoC. Output images from the prototype are successfully validated in both the spatial and frequency domains. Benchmark results demonstrate a consistent speedup exceeding$30\times $with FPGA co-processing compared to conventional sequential execution on the ARM CPU, reaching a maximum processing speed of 3.963 megapixels per second.

Structured light illumination is an active 3D scanning technique based on projecting and capturing a set of striped patterns and measuring the warping of the patterns as they reflect off a target object's surface. As designed, each pixel in the camera sees exactly one pixel from the projector; however, there are multi-path situations where a camera pixel sees light from multiple projector positions. In the case of bimodal multi-path, the camera pixel receives light from exactly two positions, which occurs along a step edge where the edge slices through a pixel which, therefore, sees both a foreground and background surface. In this paper, we present a general mathematical model to address this bimodal multi-path issue in a phase-shifting or so-called phase-measuring-profilometry scanner to measure the constructive and destructive interference between the two light paths, and by taking advantage of this interference, separate the paths and make two decoupled depth measurements. We validate our algorithm with both simulations and a number of challenging real-world scenarios, significantly outperforming the state-of-the-art methods.

Structured light illumination is a 3D scanning technique based on projecting a series of striped patterns and reconstructing depth based on the observed warping of the pattern across the target surface. Extensively studied, real-time performance is of paramount importance in many applications. In this Letter, we build on prior researches with epipolar geometry to further simplify the computations of 3D point clouds, with the symmetrical epipolar features derived by extending epipolar geometry over normalized calibration matrices of the camera and projector. Experiments show that the proposed two new processes are of the same accuracy over the normalized calibration matrices with substantially fewer calculations.

Three-dimensional scanning by means of structured light illumination is an active imaging technique involving projecting and capturing a series of striped patterns and then using the observed warping of stripes to reconstruct the target object’s surface through triangulating each pixel in the camera to a unique projector coordinate corresponding to a particular feature in the projected patterns. The undesirable phenomenon of multi-path occurs when a camera pixel simultaneously sees features from multiple projector coordinates. Bimodal multi-path is a particularly common situation found along step edges, where the camera pixel sees both a foreground and background surface. Generalized from bimodal multi-path, this paper examines the phenomenon of sparse or N-modal multi-path as a more general case, where the camera pixel sees no fewer than two reflective surfaces, resulting in decoding errors. Using fringe projection profilometry, our proposed solution is to treat each camera pixel as an underdetermined linear system of equations and to find the sparsest (least number of paths) solution by taking an application-specific Bayesian learning approach. We validate this algorithm with both simulations and a number of challenging real-world scenarios, demonstrating that it outperforms state-of-the-art techniques.

Compressive multispectral imaging systems comprise a new generation of spectral imagers that capture coded projections of a scene where spectral data cubes are reconstructed computationally. Separately, time-of-flight (ToF) cameras obtain 2D range images where each pixel records the distance from the camera sensor to the target surface. The demand for these imaging modalities is rapidly increasing, and thus, there is strong interest in developing new image sensors that can simultaneously acquire multispectral-color-and-depth imagery (MS+D) using a single aperture. Work in this path has been mainly developed via RGB+D imaging. However, in RGB+D, the multispectral image is limited to three spectral channels, and the imaging system often relies on two image sensors. We recently proposed a compressive MS+D imaging device that used a digital-micromirror-device, requiring a bulky double imaging-and-relay path. To overcome the bulkiness and other difficulties of our previous imaging system, this work presents a more-compact MS+D imaging device with snapshot capabilities. It provides better spectral sensing, relying on a static color-coded-aperture (CCA) and a ToF sensor. To guarantee good quality in the recovery, we develop an optimization method for CCA based-on blue-noise-multitoning, solved via the direct-binary-search algorithm. A testbed-setup is reported along with simulated and real experiments that demonstrate the MS+D capabilities of the proposed system over static and dynamic scenes.

Structured light illumination is an active 3D scanning technique based on projecting/capturing a set of striped patterns and measuring the warping of the patterns as they reflect off a target object's surface. As designed, each pixel in the camera sees exactly one pixel from the projector; however, there are multi-path situations when the scanned surface has a complicated geometry with step edges and other discontinuities in depth or where the target surface has specularities that reflect light away from the camera. These situations are generally referred to multi-path where a camera pixel sees light from multiple projector positions. In the case of bimodal multi-path, the camera pixel receives light from exactly two positions which occurs along a step edge where the edge slices through a pixel so that the pixel sees both a foreground and background surface. In this paper, we present a general mathematical model to address the bimodal multi-path issue in a phase-measuring-profilometry scanner to measure the constructive and destructive interference between the two light paths, and by taking advantage of this interesting cue, separate the paths and make two decoupled phase measurements. We validate our algorithm with a number of challenging real-world scenarios, outperforming the state-of-the-art method.

This paper introduces the concept of QR images, an automatic method to embed QR codes into color images with bounded probability of detection error. These embeddings are compatible with standard decoding applications and can be applied to any color image with full area coverage. The QR information bits are encoded into the luminance values of the image, taking advantage of the immunity of QR readers against local luminance disturbances. To mitigate the visual distortion of the QR image, the algorithm utilizes halftoning masks for the selection of modified pixels and nonlinear programming techniques to locally optimize luminance levels. A tractable model for the probability of error is developed and models of the human visual system are considered in the quality metric used to optimize the luminance levels of the QR image. To minimize the processing time, the optimization techniques proposed to consider the mechanics of a common binarization method and are designed to be amenable for parallel implementations. Experimental results show the graceful degradation of the decoding rate and the perceptual quality as a function the embedding parameters. A visual comparison between the proposed and existing methods is presented.

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.

High dynamic range imaging (HDRI) methods in computational photography address situations where the dynamic range of a scene exceeds what can be captured by an image sensor in a single exposure. HDRI techniques have also been used to construct radiance maps in measurement applications; unfortunately, the design and evaluation of HDRI algorithms for use in these applications have received little attention. In this paper, we develop a novel HDRI technique based on pixel-by-pixel Kalman filtering and evaluate its performance using objective metrics that this paper also introduces. In the presented experiments, this new technique achieves as much as 9.4-dB improvement in signal-to-noise ratio and can achieve as much as a 29% improvement in radiometric accuracy over a classic method.

Phase shifting structured light illumination for range sensing involves projecting a set of grating patterns where accuracy is determined, in part, by the number of stripes. However, high pattern frequencies introduce ambiguities during phase unwrapping. This paper proposes a process for embedding a period cue into the projected pattern set without reducing the signal-to-noise ratio. As a result, each period of the high frequency signal can be identified. The proposed method can unwrap high frequency phase and achieve high measurement precision without increasing the pattern number. Therefore, the proposed method can significantly benefit real-time applications. The method is verified by theoretical and experimental analysis using prototype system built to achieve 120 fps at 640 × 480 resolution.

A new algorithm to build blue noise masks using centroidal Voronoi tessellations (CVT) and a variant of Lloyd's Algorithm is presented. The algorithm takes advantage of the optimality properties of CVTs and through a modified version of Lloyd's algorithm, achieves optimization of the stacked binary patterns that build the mask. A new ordering for binary pattern design is presented, as well as a new metric that allows the creation of quality profiles of the masks. The masks generated by this method are used to halftone sample images, and quality profiles are created. It is shown that CVT masks outperform masks created by DBS and VaC according to the new metric defined and thorough visual inspection of halftoned images.

To solve the problems associated with conventional 2-D fingerprint scanners such as skin deformation and print smearing, in this paper we introduce a noncontact fingerprint scanner employing structured light illumination to generate high-resolution albedo images as well as 3-D ridge scans. The question to be answered in this research is whether or not ridge depth information improves the quality and matching capability of acquired fingerprints? For evaluation of this question, we use the National Institute of Standards and Technology fingerprint image quality metrics. These metrics require the 3-D prints to be flattened. We present a complete and detailed flattening algorithm based upon unfolding an elastic tube fit to the peaks and valleys of ridges identified within the scan. Further improvement of the flattened print is achieved through the incorporation of ridge information extracted from the albedo image with the depth and albedo ridge information fused together according to local scan quality. Our study compares image quality between the flattened 3-D prints and ink rolled prints. Most significantly, the matching performance of 3-D flattened to 3-D flattened prints is evaluated based on ridge depth only, albedo only, and depth with albedo fusion.

The introduction of the blue-noise spectra-high-frequency white noise with minimal energy at low frequencies-has had a profound impact on digital halftoning for binary display devices, such as inkjet printers, because it represents an optimal distribution of black and white pixels producing the illusion of a given shade of gray. The blue-noise model, however, does not directly translate to printing with multiple ink intensities. New multilevel printing and display technologies require the development of corresponding quantization algorithms for continuous tone images, namely multitoning. In order to define an optimal distribution of multitone pixels, this paper develops the theory and design of multitone, blue-noise dithering. Here, arbitrary multitone dot patterns are modeled as a layered superposition of stack-constrained binary patterns. Multitone blue-noise exhibits minimum energy at low frequencies and a staircase-like, ascending, spectral pattern at higher frequencies. The optimum spectral profile is described by a set of principal frequencies and amplitudes whose calculation requires the definition of a spectral coherence structure governing the interaction between patterns of dots of different intensities. Efficient algorithms for the generation of multitone, blue-noise dither patterns are also introduced.

The problem of multitoning emerges as the result of advances on printing technology that allow the reproduction of different shades of gray applying techniques like multiple inks, different ink concentration or various dot sizes. The end result is an improvement in the quality of the reproduction. While some algorithms used for binary halftoning have been extended to deal with multitoning with a certain success, there is room for further improvement in the quality of the resulting images. Faheem, Arce, and Lau proposed gray level separation as a way to improve the quality of multitone images generated through error diffusion. In this paper a new gray level separation scheme is proposed whose objective is to ensure that black, gray and white pixels are printed in blue-noise patterns. The new scheme is applied to error diffusion as well as to the well known DBS algorithm. The results in both cases show an improvement in the texture of the printed patterns.

In this paper, we closely scrutinize the spatial and spectral properties of aperiodic halftoning schemes on rectangular and hexagonal sampling grids. Traditionally, hexagonal sampling grids have been shunned due to their inability to preserve the high-frequency components of blue-noise dither patterns at gray-levels near one-half, but as will be shown, only through the introduction of diagonal correlations between dots can even rectangular sampling grids preserve these frequencies. And by allowing the sampling grid to constrain the placement of dots, a particular algorithm may introduce visual artifacts just as disturbing as excess energy below the principal frequency. If, instead, the algorithm maintains radial symmetry by introducing a minimum degree of clustering, then that algorithm can maintain its grid defiance illusion fundamental to the spirit of the blue-noise model. As such, this paper shows that hexagonal grids are preferrable because they can support gray-levels near one-half with less required clustering of minority pixels and a higher principal frequency. Along with a thorough Fourier analysis of blue-noise dither patterns on both rectangular and hexagonal sampling grids, this paper also demonstrates the construction of a blue-noise dither array for hexagonal grids.

In this paper, we introduce two novel techniques for digital color halftoning with green-noise--stochastic dither patterns generated by homogeneously distributing minority pixel clusters. The first technique employs error diffusion with output-dependent feedback where, unlike monochrome image halftoning, an interference term is added such that the overlapping of pixels of different colors can be regulated for increased color control. The second technique uses a green-noise mask, a dither array designed to create green-noise halftone patterns, which has been constructed to also regulate the overlapping of different colored pixels. As is the case with monochrome image halftoning, both techniques are tunable, allowing for large clusters in printers with high dot-gain characteristics, and small clusters in printers with low dot-gain characteristics.

We introduce the concept of green noise-the mid-frequency component of white noise-and its advantages over blue noise for digital halftoning. Unlike blue noise, which creates the illusion of continuous tone by spreading the minority pixels of a binary dither pattern as homogeneously as possible, green noise forms minority pixel clusters which are themselves distributed as homogeneously as possible. By clustering pixels, green noise patterns are less susceptible to image degradation from printer distortions such as dot-overlap (the overlapping of a printed dot with its nearest neighbors), and by adjusting the average number of pixels per cluster, green noise patterns are tunable to specific printer characteristics. Using both spectral and spatial statistics, are establish models for ideal green noise patterns.

In this paper, we introduce the concept of green noise-the multifrequency component of white noise-and its advantages over blue noise for digital halftoning. Unlike blue-noise dither patterns, which are composed exclusively of isolated pixels, green-noise dither patterns are composed of pixel-clusters making them less susceptible to image degradation from nonideal printing artifacts such as dot-gain. Although they are not the only techniques which generate clustered halftones, error-diffusion with output-dependent feedback and variations based on filter weight perturbation are shown to be good generators of green noise, thereby allowing for tunable coarseness. Using statistics developed for blue noise, we closely examine the spectral content of resulting dither patterns. We introduce two spatial-domain statistics for analyzing the spatial arrangement of pixels in aperiodic dither patterns, because green noise patterns may be anisotropic, and therefore spectral statistics based on radial averages may be inappropriate for the study of these patterns.

In this paper we introduce a general framework for edge preserving filters, derived from the powerful class of M-estimators. First, we show that under very general assumptions, any location estimator generates an edge preserving filter if we approximate the estimate by one of the input samples. Based on this premise, we propose the family of S-estimators or S-filters, as a selection-type class of filters arising from a computationally tractable "selectification" of location M-estimators. S-filters inherit the richness of the theory underlying the M-estimators framework, providing a very flexible family of robust estimators with edge preservation capabilities. Several properties of S-filters are studied. Sufficient and necessary conditions are given for an S-filter to present edge enhancing capabilities, and several novel filters within this framework are introduced and illustrated. Data, figures and source code utilized in this paper are available at http://www.ee.udel.edu/signals/robust/.

Median based filters have gained wide-spread use because of their ability to preserve edges and suppress impulses. In this paper, we introduce the closest-to-mean (CTM) filter, which outputs the input sample closest to the sample mean. The CTM filtering framework offers lower computational complexity and better performance in near Gaussian environments than median filters. The formulation of the CTM filter is derived from the theory of S-filters, which form a class of generalized selection-type filters with the features of edge preservation and impulse suppression. S-filters can play a significant role in image processing, where edge and detail preservation are of paramount importance. We compare the performance of CTM, median, and mean filters in the smoothing of edges and impulses immersed in Gaussian noise. A sufficient condition for a signal to be a root of the CTM filter is included.

Donoho and Johnstone (1992) first introduced wavelet shrinkage as a denoising technique for signals embedded in Gaussian noise, but due to the linearity of wavelet decomposition, wavelet shrinkage is ineffective in non-Gaussian noise which exhibits outliers. We evaluate two schemes which have been developed to extend the denoising capabilities of wavelet shrinkage to signals corrupted by non-Gaussian noise. The first scheme introduced by Bruce et al. (see Proceedings SPIE Conference, 1994) smoother-cleaner wavelets integrates median filters into the wavelet decomposition. The second scheme, introduced by the authors, replaces the linear filters of wavelet decomposition with order statistic based Chameleon filters. We also show that a straight forward extension of these schemes to images does not offer the same effectiveness in denoising as they do with one dimensional signals.