This paper proposes a method for fusing data acquired by a ToF camera and a stereo pair based on a model for depth measurement by ToF cameras which accounts also for depth discontinuity artifacts due to the mixed pixel effect. Such model is exploited within both a ML and a MAP-MRF frameworks for ToF and stereo data fusion. The proposed MAP-MRF framework is characterized by site-dependent range values, a rather important feature since it can be used both to improve the accuracy and to decrease the computational complexity of standard MAP-MRF approaches. This paper, in order to optimize the site dependent global cost function characteristic of the proposed MAP-MRF approach, also introduces an extension to Loopy Belief Propagation which can be used in other contexts. Experimental data validate the proposed ToF measurements model and the effectiveness of the proposed fusion techniques.

Novel real-time depth acquisition devices, like Microsoft Kinect, allow a very fast and simple acquisition of 3D views. These sensors have been used in many applications but their employment for 3D scanning purposes is a very challenging task due to the limited accuracy and reliability of their data. In this paper we present a 3D reconstruction pipeline explicitly targeted to the Kinect. The proposed scheme aims at obtaining a reliable reconstruction that is not affected by the limiting issues of these cameras and is at the same time simple and fast in order to allow to use the Kinect sensor as an handheld scanner. In order to achieve these targets a novel algorithm for the extraction of salient points exploiting both depth and color data is firstly used. The extracted points are then used within a modified version of the ICP algorithm that exploits both geometry and color distances to precisely align the views produced by the Kinect even when the geometry information is not sufficient to constrain the registration. Experimental results show how the proposed approach is able to produce reliable 3D reconstructions from the Kinect data.

This paper explores the issues connected to the transmission of three dimensional scenes over unreliable networks such as the wireless ones. It analyzes the effect of the loss of compressed data packets in a typical image-based rendering scenario, where a set of compressed images together with the corresponding depth maps are transmitted and used to generate the views required from the user at client side. The different impact on the rendered views of the geometry and texture packets is analyzed in detail, taking into account also the position of the lost packets and the warping operation. Finally we will discuss how to exploit this results in the design of an efficient network protocol for the transmission of 3D models.

We consider an interactive client-server application for remote browsing of 3D scenes. The information about texture and geometry is available at server side in the form of scalably compressed images and depthmaps, corresponding to a multitude of original image views. Image and depth components are both open to augmentation as more content becomes available. During the interactive browsing experience, the server allocates the available bandwidth between the delivery of new elements from the various original view bit-streams and new elements from the original geometry bit-streams. We propose a rate-distortion criterion to decide the best transmission policy for the server, since the best solution is not always to send the nearest original view image to the one which the client is rendering. We also outline how the JPIP standard for interactive transmission of JPEG2000 images can be exploited for remote exploration of 3D scenes

This paper presents a new and effective method for estimating two-dimensional affine transformations and its application to image registration. The method is based on matching polar curves obtained from the radial projections of the image energies, defined as the squared magnitudes of their Fourier transforms. Such matching is formulated as a simple minimization problem whose optimal solution is found with the Levenberg-Marquardt algorithm. The analysis of affine transformations in the frequency domain exploits the well-known property whereby the translational displacement in this domain can be factored out and separately estimated through phase correlation after the four remaining degrees of freedom of the affine warping have been determined. Another important contribution of this paper, emphasized through one example of image mosaicking and one example of remote sensing image registration, consists in showing that affine motion can be accurately estimated by applying our algorithm to the shapes of macrofeatures extracted from the images to register. The excellent performance of the algorithm is also shown through a synthetic example of motion estimation and its comparison with another standard registration technique.

We consider an interactive browsing environment, with greedy optimization of a current view, conditioned on the availability of previously transmitted information for other (possibly nearby) views, and subject to a transmission budget constraint. Texture information is available at a server in the form of scalably compressed images, corresponding to a multitude of original image views. Surface geometry is also represented at the server in a scalable fashion. At any point in the interactive browsing experience, the server must decide how to allocate transmission resources between the delivery of new elements from the various original view bit-streams and new elements from the geometry bit-stream. The proposed framework may be interpreted as a greedy strategy for non-linear approximation of the plenoptic function, since it considers both view sampling and rate-distortion criteria. We particularly elaborate upon a novel geometry- and distortion-sensitive strategy for blending the information available from different views at the client.

This paper presents a new method for registering images related by 2-D affine transformations. The method is based on extracting macro-features from the images to register and matching the polar curves associated with their energies, defined as the squared Fourier transform magnitudes. Such matching is formulated as a simple minimization problem whose optimal solution if found with the Levenberg-Marquardt algorithm. The excellent performance of the algorithm is shown through a practical example of image mosaicking

This paper presents an approach for aspect-based retrieval of 3D models or circular views of real objects. The objects in our database are described by a set of images, which are taken varying the pose of a constant angle along a given plane. A set of parameters is computed for each image, taking into account general features like bounding boxes, contours, and mass distribution. The number of parameters of each image is then reduced using principal component analysis, obtaining a low dimensional feature space. We assume that the query is in the same form of objects in our database, and thus it can undergo the same feature extraction process. The system then retrieves the most similar objects computing the L/sub 1/ distance in the features space. Preliminary results are presented with a database of 100 different objects, consisting of 2000 different views.

A widespread use of three-dimensional (3-D) models in cultural heritage application requires low cost equipment and technically simple modeling procedures. In this context, methods for automatic 3-D modeling of textured objects will play a central role. Such methods need fully automatic techniques for 3-D views registration and for the removal of texture artifacts. This paper proposes a contribution in this direction based on image processing approaches. The proposed procedure is very robust and simple. It does not require special equipment or skill in order to make textured 3-D models. The results of this paper, originally conceived to address the costs issues of cultural heritage modeling, can be profitably exploited also in other modeling applications.

This work introduces an original method for registering pairs of 3D views consisting of range data sets which operates in the frequency domain. The Fourier transform allows the decoupling of the estimate of the rotation parameters from the estimate of the translation parameters, our algorithm exploits this well-known property by suggesting a three-step procedure. The rotation parameters are estimated by the first two steps through convenient representations and projections of the Fourier transforms' magnitudes and the translational displacement is recovered by the third step by means of a standard phase correlation technique after compensating one of the two views for rotation. The performance of the algorithm, which is well-suited for unsupervised registration, is clearly assessed through extensive testing with several objects and shows that good and robust estimates of 3D rigid motion are achievable. Our algorithm can be used as a prealignment tool for more accurate space-domain registration techniques, like the ICP algorithm.

Chaotic modulation is a way to transmit digital information by means of chaotic signals instead of sinusoids. Currently proposed schemes, obtained from traditional transmission systems by replacing sinusoids with chaotic signals, perform poorly when compared to traditional modulation schemes. In this paper we presented an original scheme of chaotic modulation/demodulation. Theoretical analysis and experimental results show that the performance of chaotic modulation can be superior to traditional modulation.

Exporting methods and tools used in industrial areas, where the profits can justify high modeling costs, may only be a starting point for cultural heritage modeling, but not a sustainable solution in view of the economics of this application. A way to lower the costs is to automate the modeling procedure. This work presents a robust procedure for the automatic alignment of pairs of 3D views, the first problem encountered in the modeling pipeline. The results of this paper, originally conceived to address the costs issue of cultural heritage modeling, can be profitably exploited also in other modeling applications.

Spurious range data pose a serious visualization problem with active light systems, as they can totally deform the acquired 3-D shapes. This work presents a technique for their detection based on geometrical considerations. The proposed method does not need extra hardware and can be used effectively for the provision of post-processing noise removal. The method has been extensively tested on real data. Its performance is discussed and demonstrated by examples.

This work examines the practical issues concerning the realization of switched-capacitors video-rate finite-impulse response filters based on standard CMOS technology. The capacitor spread/sensitivity tradeoff is explored for a number of architectural solutions and an architecture suitable to the challenges of the video field is identified. The nonidealities of switches and operational amplifiers, which must be accounted for at video-rate, are analyzed and practical solutions to prevent performance deterioration are proposed. The implementation of a concrete design case is discussed and presented as a demonstration of the technological solutions proposed.

This paper presents an original coarse-to-fine technique for creating mosaics of color images related by projective transformations. The actual projective transformation is first approximated with an affine transformation whose parameters are estimated by means of an algorithm operating in the frequency domain. The projective transformation is then computed with an algorithm based on feature matching; the computational complexity of this operation is substantially reduced by the preliminary affine approximation which reduces the search area for corresponding points. A practical example of mosaicing of two color images of a planar fresco is reported and discussed. Some experimental results are also reported in order to assess the practical performance of our technique.

This paper presents an original two‐step procedure for estimating projective transformations between pairs of images: first, the transformation between the images is approximated as an affine transformation; second, this estimate is refined into that of a projective transformation. This strategy for matching projective views is computationally very efficient. The proposed method can be applied both to mosaicking of high resolution images of planar textured objects (e.g., frescoes and paintings), with subpixel accuracy, and to construction of panoramic images. Practical examples of mosaicking of cultural heritage imagery obtained by using the presented procedure are discussed in the paper.

Free-form 3-D surfaces registration is a fundamental problem in 3-D imaging, typically approached by extensions or variations of the ICP algorithm. This work presents a new frequency domain technique for 3-D view registration, totally different form any other techniques for 3-D motion estimation also, based on the Fourier transform. The proposed method can give a non-feature-based method for unsupervised registration of 3-D views. The obtained results are useful "per se" in applications targeted to visual quality or can serve as good starting point for the ICP algorithm when a higher precision is needed.

An extension of the well-known authoring tool Macromedia DIRECTOR is presented. It is designed to facilitate the development of multimedia virtual visits to sites such as museums, art expositions, industrial and architectural exhibitions. The extension is written in the internal DIRECTOR language Lingo. It has been used to develop the virtual visits "Da Padovanino a Tiepolo" and "EPOC", and has given excellent results.

This article extends in three ways a frequency domain technique for estimating planar rotations introduced by Lucchese et al. (see Proc. of ISCAS'96, Atlanta, Georgia, vol.2, p.774-7, 1996). It refines the technique algorithmically, since it improves the precision by one or two orders of magnitude, depending upon the noise levels. It presents a stochastic version of that method and demonstrates analytically its robustness at very low signal-to-noise ratios (SNR). Finally, it extends the method of Lucchese et al. to the case of joint planar rotations and translations.

This article presents an original algorithm operating in the frequency domain for estimating the affine transformation between a pair of images. The affine matrix is estimated by the relationships between companion stretched slices of the Fourier transforms magnitudes of the two images. The translational vector is estimated by phase-correlation after compensating for the contribution of the affine matrix. This approach is "global" in the sense that it uses the whole images information and it does not rest upon features extraction and matching. It can be efficiently implemented via FFT and it is suited to unsupervised estimation of affine transformations. Experimental evidence of the effectiveness and robustness of the proposed method is reported.

Symmetric signals are defined by subsets of their support which depend on the specific symmetry. Depending on the symmetry, the cardinality of these subsets may be much smaller than that of the supports. The DFT of the symmetric signals enjoys symmetries related to the input symmetry, by which the DFT is defined by subsets the same as those defining the input signal. In principle, the computation of the DFT of symmetric signal can only use the input subset sufficient for defining the signal. In practice, all known MD FFT algorithms can only take advantage of even and odd parity's symmetries, and cannot exploit general signal symmetries for reducing the computation. This work presents a new MD FFT algorithm capable of saving computation in front of general signal's symmetries.

Switched-capacitor technology implemented by 2.4 /spl mu/m CMOS offers considerable reliability/cost advantages. Its limited clock-rate values are the traditional obstacle to its introduction in high frequency applications, such as video. This obstacle can be removed by an architectural scheme broadening the internal clock rate by an integer factor with respect to the input clock rate. The feasibility of this concept is demonstrated by the switched-capacitor realization of a color difference prefilter complying with Recommendation CCIR 601.>

SDTV/HDTV conversion has several applications of current interest, such as pyramidal coding for simultaneous SDTV and HDTV output, HDTV display of SDTV signals, and SDTV display of HDTV signals. The effectiveness of multistage filters in this application is established. The design of the filters to use in these configurations is a delicate issue, since it has to balance several, and sometimes conflicting, requirements. The design objectives depend on the specific application, and are ultimately targeted to the visual rendition of the system. These objectives do not fit traditional design specs and call for provisions of visual significance in both the frequency domain and the spatio-temporal domain. This work reports on the findings of some systematic research on this field.>

The practice of superimposing the fields of a frame is applied in various fields, for example, thermographic and biomedical imaging. The pictures obtained in this way, which are termed composite frames, are severely degraded if the scene's objects are not perfectly still. The restoration of composite frames affected by motion-induced blurring requires the ability to estimate the field displacement from composite frames. The frequency domain analysis of composite frames proposed in the paper suggests a displacement estimation technique of a phase-correlation type that can be applied to composite frames.

This work presents an original two-stages MD FFT algorithm where in the first stage the signal is processed by multiplier-free butterflies in such a way that at the second stage the computation only needs 1D FFT's. The proposed method is more efficient than any other MD FFT algorithm known to the authors.>

Examines the distinctive features of 2D diamond-shaped filters, a class of filters quite important for video interpolation/decimation. The geometrical interpretation of linear programming is proposed as an easy tool for understanding the characteristics of M-D FIR filters minimax optimal and the effects of the constraints most common in video application. The relationships between minimax error, filter orders and bandwidth of diamond-shaped FIR filters are investigated, extending results typical of 1D minimax filter design. The effect of constraining one of the filter orders to a pre-assigned value (provision quite useful if the filter order under consideration determines the number of frame-stores) is studied extensively. In light of the reported characteristics of the diamond-shaped FIR filters their multistage implementation becomes extremely attractive. The advantages and the potential of multi-stage implementation of diamond-shaped filters are discussed.>

Granular camera noise is per se objectionable in high-quality TV and it is a source of bothersome visual artifacts with digitally coded video sequences. The first and second order statistical characteristics of this noise are investigated in this work. The results quantify intuitively expected notions, such as the relative unimportance of chrominance noise with respect to the luminance noise and the spatially colored nature of the noise. Noise reduction techniques can use, as general guidance rules, the indications of the presented analysis.>

The frequency domain analysis of constant velocity motion has proven useful in a number of different contexts ranging from television signal processing to computer vision. This work considers the frequency domain analysis of a generalization of such simple dynamics, which includes the effect of acceleration and its time derivatives (of any order). In the frequency domain the contribution due to the object's shape and the contribution due to translation terms are clearly separated. Motion information in the frequency domain is given by functions characterized by a family of differential equations. Such differential equations are introduced in the work and their solutions are exemplified in order to compute the frequency domain motion contributions. For simplicity; the results are presented for the case of translations, but they apply equally well to rotations.>

Translations with piecewise cubic trajectories are studied in the frequency domain. This class of motion has as an important subcase: cubic spline trajectories. Translations with trajectories depending on time with general polynomial law are preliminarily considered, and a general theorem concerning this type of motion is introduced. The application of this theorem to the case of cubic time dependence and the consideration of finite-duration effects lead to the solution of the piecewise cubic trajectory case. The results, which are remarkably different from those concerning constant velocity translations, clearly indicate the importance of the role of velocity and time duration. In this respect, they confirm the validity of constant velocity motion as a first-order model for frequency domain analysis of motion.>

Subsampling offers an effective and simple way to implement a data compression technique. Its use in the video context is rather attractive, as its application to current video broadcasting schemes, such as MUSE and HD-MAC, clearly shows. A sensible exploitation of subsampling's potential requires the systematic evaluation of the image degradation effects due to the specific subsampling lattice. Such a possibility is equivalent to the determination of all the sublattices of a given index within a lattice, a problem that can be solved, as the work shows, on the basis of the Hermite normal-form theorem. The practical use of the result in the subsampling context is exemplified for a data compression case.>

The work extends multistage implementation of sampling structure conversion to the multidimensional (M-D) case. The issues arising in this task are usefully addressed on the basis of lattice theory. Numerical data supporting the advantages of multistage sampling conversion are presented, and the case of format conversion from the 4/3 to the 16/9 aspect ratio is examined as a study case. The main indication of the present work is that multistage implementation, in the case of systems for sampling structure conversion of video signals, may improve the system characteristics and visual rendition.>

Automatic key-section recognition is approached by a two-step procedure. The first step is a screening based on arc/straight-segment classification, which associates to each key section a morphological pattern. The second step measures the string distances by means of nonlinear elastic matching among the key-sections possessing the same morphological description. Arc/straight-segment classification, a typical geometrical form description problem, is studied by two techniques, one based on Euclidean geometry notions and the other based on discrete geometry. The results achieved by the second technique appear superior. The performance of the recognition procedure on key-section databases of practical significance are very satisfactory.>

To eliminate multipliers in the hardware implementation, filters with the coefficients that are given by the sum or difference of two power-of-two terms are designed. For a minimax criterion, a global optimization procedure known as simulated annealing has been used to determine the coefficients. Significant improvement in the filter performance is gained over methods that simply round the infinite precision solution. The algorithm also allows the maximum value of the coefficient wordlength beyond which filter performance does not improve to be found. A number of design examples involving typical video specifications, such as deinterlacing filters and luminance/chrominance separation filters, are reported. The method proves to be very effective, yet the designs are within the dimensional limits of the filters used in current industrial applications.>

The authors introduce an algorithm for simultaneous magnitude and group-delay design of FIR (finite impulse response) filters, a multiple criterion optimization problem for which no characterization theory is presently available. The algorithm, derived from differential correction, has global convergence. An experimental comparison with a method for complex polynomial approximation, a problem whose mathematical characterization is known, yields very useful and consistent results. The authors compare the results of two filter design formulations which are conceptually similar but formally different, and they present data potentially useful for the mathematical characterization of simultaneous polynomial approximation in magnitude and group-delay.>

Frequency domain techniques are presented for the evaluation of the autocorrelation in the case of multidimensional sequences. The analysis confirms that the use of the proposed frequency domain techniques leads to significant computation time savings, extremely valuable in practical contexts.>

The authors focus on some peculiarities of 2-D group delay equalization by means of cascades of 2*2 sections, a 2-D equalization technique that has lately received considerable attention. It appears that the group delay performance obtained by just one or two 2*2 section equalizers cannot be significantly improved by using equalizers with a greater number of sections. As a consequence, this type of 2-D group delay equalizer can seldom reach the group delay error performance of its 1-D counterpart. The work shows how the characteristics and the possibilities of 2-D group delay equalization markedly differ from those of the 1-D case.>

The authors propose an extension of the differential correction algorithm for simultaneous design of the real and imaginary part of IIR (infinite impulse response) transfer functions in minimax norm. The method has global convergence, and it can be proved to deliver stable IIR filters. Practical implementations of the algorithm compare very favorably with existing design tools. Actual operation is examined by means of filter design examples.>

In this paper we investigate for the first time the applicability of the concepts and algorithms of Multiple Criterion Optimization to the problem of optimally trading-off magnitude and phase in IIR digital filters design. Particular attention is paid to the mathematical formulation of the simultaneous magnitude and phase design problem. Three different design criteria are introduced. The special modifications of a standard algorithm [9] necessary for filter design are discussed. Examples of what can be expected from magnitude-phase trade-offs are discussed. The results of this work is expected to be practical CAD tools for designing simultaneous magnitude and phase optimal filters and theoretical information about the optimal trade-offs between magnitude and phase.