We propose a learning-based method for snapshot hyper-spectral (HS) imaging of deep 3D scenes. The method combines computational HS imaging and extended depth of field (EDoF) imaging capabilities in a single framework, resulting in novel EDoF-HS camera designs. The camera system incorporates a diffractive optical element at the aperture position, a CFA in front of the sensor and a residual dense network at the post-processing stage. These optical and neural components are jointly optimized through end-to-end learning procedure. We demonstrate high quality HS image reconstructions for scenes as deep as 4 diopters.

We present a learning model for reconstructing near-eye dense light field (LF) from a sparse set of multi-perspective views. The model integrates a fully-convolutional neural network and a model of the retinal image formation process optically connecting the pupil, retina and neural domains. Considering the problem of$9\times 9$near-eye LF reconstruction from the available five images, four at the corner viewpoints and one in the middle, we investigate the implications of using different loss functions in the learning process in terms of reconstruction qualities at different domains. Despite the utilized simplified retinal image formation model, the simulations reveal instructive results. In particular, combining the LF loss and the retinal focal stack loss is shown to improve the reconstruction quality of actual LF at the pupil plane, facilitating learning better features. On the other hand, concerning the retinal image quality, the model trained based on the same combination of losses is also demonstrated to produce better retinal images especially for non-Lambertian scenes, i.e., when there is monocular parallax, compared to model trained based on only retinal focal stack loss.

We present a computational accommodation-invariant near-eye display, which relies on imaging with coherent light and utilizes static optics together with convolutional neural network-based preprocessing. The network and the display optics are co-optimized to obtain a depth-invariant display point spread function, and thus relieve the conflict between accommodation and ocular vergence cues that typically exists in conventional near-eye displays. We demonstrate through simulations that the computational near-eye display designed based on the proposed approach can deliver sharp images within a depth range of 3 diopters for an effective aperture (eyepiece) size of 10 mm. Thus, it provides a competitive alternative to the existing accommodation-invariant displays.

We address the problem of efficient content-adaptive superpixel segmentation. Instead of adapting the size and/or amount of superpixels to the image content, we propose a warpingtransform that makes the image content more suitable to be segmented into regular superpixels. Regular superpixels in the warped image induce content-adaptive superpixels in the original image with improved segmentation accuracy. To efficiently compute the warping transform, we develop an iterative coarse-to-fine optimization procedure and employ a parallelization strategy allowing for a speedy GPU-based implementation. This solution works as a simple ‘add-on’ framework over an underlying segmentation algorithm and requires no additional parameters. Compared to the state-of-the-art methods, our approach provides competitive quality results and achieves a better time-accuracy trade-off. We further demonstrate the effectiveness of our method with an application to disparity estimation.

Depth of field is an important factor of imaging systems that highly affects the quality of the acquired spatial information. Extended depth of field (EDoF) imaging is a challenging ill-posed problem and has been extensively addressed in the literature. We propose a computational imaging approach for EDoF, where we employ wavefront coding via a diffractive optical element (DOE) and we achieve deblurring through a convolutional neural network. Thanks to the end-to-end differentiable modeling of optical image formation and computational post-processing, we jointly optimize the optical design, i.e., DOE, and the deblurring through standard gradient descent methods. Based on the properties of the underlying refractive lens and the desired EDoF range, we provide an analytical expression for the search space of the DOE, which is instrumental in the convergence of the end-to-end network. We achieve superior EDoF imaging performance compared to the state of the art, where we demonstrate results with minimal artifacts in various scenarios, including deep 3D scenes and broadband imaging.

We present an accommodation-invariant computational neareye display based on the extended depth of field imaging. The eyepiece of the display consists of a diffractive optical element (DOE) that is used in tandem with a conventional refractive lens. The DOE is co-designed with the pre-processing convolutional neural network, which is analogous to the post-processing deblurring networks in image capture. We demonstrate through simulations that such system achieves accommodation-invariant imaging within 2 Diopters depth range without significantly sacrificing spatial resolution.

We address the problem of 3D scene rendering on near-eye integral imaging displays and evaluation of different rendering methods in terms of human perception. We compare three rendering techniques in terms of perceived spatial resolution at different focused depths, simulating the display in virtual environment and representing the eye through a thin-lens camera model.

Shearlet Transform (ST) has been instrumental for the Densely-Sampled Light Field (DSLF) reconstruction, as it sparsifies the underlying Epipolar-Plane Images (EPIs). The sought sparsification is implemented through an iterative regularization, which tends to be slow because of the time spent on domain transformations for dozens of iterations. To overcome this limitation, this letter proposes a novel self-supervised DSLF reconstruction method, CycleST, which employs ST and cycle consistency. Specifically, CycleST is composed of an encoder-decoder network and a residual learning strategy that restore the shearlet coefficients of densely-sampled EPIs using EPI-reconstruction and cycle-consistency losses. CycleST is a self-supervised approach that can be trained solely on Sparsely-Sampled Light Fields (SSLFs) with small disparity ranges (≤8 pixels). Experimental results of DSLF reconstruction on SSLFs with large disparity ranges (16 - 32 pixels) demonstrate the effectiveness and efficiency of the proposed CycleST method. Furthermore, CycleST achieves ~ 9x speedup over ST, at least.

Light field (LF) acquisition devices capture spatial and angular information of a scene. In contrast with traditional cameras, the additional angular information enables novel postprocessing applications, such as 3D scene reconstruction, the ability to refocus at different depth planes, and synthetic aperture. In this paper, we present a novel compression scheme for LF data captured using multiple traditional cameras. The input LF views were divided into two groups: key views and decimated views. The key views were compressed using the multi-view extension of high-efficiency video coding (MV-HEVC) scheme, and decimated views were predicted using the shearlet-transform-based prediction (STBP) scheme. Additionally, the residual information of predicted views was also encoded and sent along with the coded stream of key views. The proposed scheme was evaluated over a benchmark multi-camera based LF datasets, demonstrating that incorporating the residual information into the compression scheme increased the overall peak signal to noise ratio (PSNR) by 2 dB. The proposed compression scheme performed significantly better at low bit rates compared to anchor schemes, which have a better level of compression efficiency in high bit-rate scenarios. The sensitivity of the human vision system towards compression artifacts, specifically at low bit rates, favors the proposed compression scheme over anchor schemes.

We present a fast and accurate method for dense depth reconstruction, which is specifically tailored to process sparse, wide-baseline light field data captured with camera arrays. In our method, the source images are over-segmented into non-overlapping compact superpixels. We model superpixel as planar patches in the image space and use them as basic primitives for depth estimation. Such superpixel-based representation yields desired reduction in both memory and computation requirements while preserving image geometry with respect to the object contours. The initial depth maps, obtained by plane-sweeping independently for each view, are jointly refined via iterative belief-propagation-like optimization in superpixel domain. During the optimization, smoothness between the neighboring superpixels and geometric consistency between the views are enforced. To ensure rapid information propagation into textureless and occluded regions, together with the immediate superpixel neighbors, candidates from larger neighborhoods are sampled. Additionally, in order to make full use of the parallel graphics hardware a synchronous message update schedule is employed allowing to process all the superpixels of all the images at once. This way, the distribution of the scene geometry becomes distinctive already after the first iterations, facilitating stability and fast convergence of the refinement procedure. We demonstrate that a few refinement iterations result in globally consistent dense depth maps even in the presence of wide textureless regions and occlusions. The experiments show that while the depth reconstruction takes about a second per full high-definition view, the accuracy of the obtained depth maps is comparable with the state-of-the-art results, which otherwise require much longer processing time.

We present a learning-based optimization framework for depth of field extension, combining rigorous modeling of coded aperture imaging system and convolutional neural network based deblurring. The coded mask discretization is defined for desired depth range using wave optics based imaging model. Such approach significantly decreases the number of parameters to be optimized and increases the convergence speed of the network. We verify the proposed algorithm in different scenarios achieving superior or comparable performance with respect to existing methods.

We propose simple and efficient method that produces content-adaptive superpixels, i.e. smaller segments in content-dense areas and larger segments in content-sparse areas. Previous adaptive methods distribute superpixels over the image according to image content. In contrast, we transform the image itself to redistribute the content density uniformly across the image area. This transformation is guided by a significance map, which characterizes the `importance' of each pixel. Arbitrary superpixel algorithm can be utilized to segment the transformed image into regular superpixels, providing a suitable representation for subsequent tasks. Regular superpixels in the transformed image induce content-adaptive superpixels in the original image facilitating the improved segmentation accuracy.

Shearlet Transform (ST) is one of the most effective methods for Densely-Sampled Light Field (DSLF) reconstruction from a Sparsely-Sampled Light Field (SSLF). However, ST requires a precise disparity estimation of the SSLF. To this end, in this paper a state-of-the-art optical flow method, i.e. PWC-Net, is employed to estimate bidirectional disparity maps between neighboring views in the SSLF. Moreover, to take full advantage of optical flow and ST for DSLF reconstruction, a novel learning-based method, referred to as Flow-Assisted Shearlet Transform (FAST), is proposed in this paper. Specifically, FAST consists of two deep convolutional neural networks, i.e. disparity refinement network and view synthesis network, which fully leverage the disparity information to synthesize novel views via warping and blending and to improve the novel view synthesis performance of ST. Experimental results demonstrate the superiority of the proposed FAST method over the other state-of-the-art DSLF reconstruction methods on nine challenging real-world SSLF sub-datasets with large disparity ranges (up to 26 pixels).

Shearlet Transform (ST) is one of the most effective algorithms for the Densely-Sampled Light Field (DSLF) reconstruction from a Sparsely-Sampled Light Field (SSLF) with a large disparity range. However, ST requires a precise estimation of the disparity range of the SSLF in order to design a shearlet system with decent scales and to pre-shear the sparsely-sampled Epipolar-Plane Images (EPIs) of the SSLF. To overcome this limitation, a novel coarse-to-fine DSLF reconstruction method, referred to as Mask-Accelerated Shearlet Transform (MAST), is proposed in this paper. Specifically, a state-of-the-art learning-based optical flow method, FlowNet2, is employed to estimate the disparities of a SSLF. The estimated disparities are then utilized to roughly estimate the densely-sampled EPIs for the sparsely-sampled EPIs of the SSLF. Finally, an elaborately-designed soft mask for a coarsely-inpainted EPI is exploited to perform an iterative refinement on this EPI. Experimental results on nine challenging horizontal-parallax real-world SSLF datasets with large disparity ranges (up to 35 pixels) demonstrate the effectiveness and efficiency of the proposed method over the other state-of-the-art approaches.

Light field acquisition technologies capture angular and spatial information of the scene. The spatial and angular information enables various post processing applications, e.g. 3D scene reconstruction, refocusing, synthetic aperture etc at the expense of an increased data size. In this paper, we present a novel prediction tool for compression of light field data acquired with multiple camera system. The captured light field (LF) can be described using two plane parametrization as, L(u, v, s, t), where (u, v) represents each view image plane coordinates and (s, t) represents the coordinates of the capturing plane. In the proposed scheme, the captured LF is uniformly decimated by a factor d in both directions (in s and t coordinates), resulting in a sparse set of views also referred to as key views. The key views are converted into a pseudo video sequence and compressed using high efficiency video coding (HEVC). The shearlet transform based reconstruction approach, presented in [1], is used at the decoder side to predict the decimated views with the help of the key views. Four LF images (Truck, Bunny from Stanford dataset, Set2 and Set9 from High Density Camera Array dataset) are used in the experiments. Input LF views are converted into a pseudo video sequence and compressed with HEVC to serve as anchor. Rate distortion analysis shows the average PSNR gain of 0.98 dB over the anchor scheme. Moreover, in low bit-rates, the compression efficiency of the proposed scheme is higher compared to the anchor and on the other hand the performance of the anchor is better in high bit-rates. Different compression response of the proposed and anchor scheme is a consequence of their utilization of input information. In the high bit-rate scenario, high quality residual information enables the anchor to achieve efficient compression. On the contrary, the shearlet transform relies on key views to predict the decimated views without incorporating residual information. Hence, it has inherit reconstruction error. In the low bit-rate scenario, the bit budget of the proposed compression scheme allows the encoder to achieve high quality for the key views. The HEVC anchor scheme distributes the same bit budget among all the input LF views that results in degradation of the overall visual quality. The sensitivity of human vision system toward compression artifacts in low-bit-rate cases favours the proposed compression scheme over the anchor scheme.

We propose an improved depth compression scheme which relies on depth decimation guided by super-pixel segmentation of the aligned color data. We modify the latter to ensure border congruency of segmenation refinement levels. Furthermore, a modification of our multi-modal regularized reconstruction is presented. We address also the problem of possible misalignments between color and depth maps. Such misalignments produce edge outliers which mislead the error optimization in the coding process. We propose an efficient encoding scheme of such outliers in so called “yieldflow” protocol. We compare our new and imporved method against a number of state-of-art approaches and demonstrate that it performs favorably especially in the low bit rate region.

In this article we develop an image based rendering technique based on light field reconstruction from a limited set of perspective views acquired by cameras. Our approach utilizes sparse representation of epipolar-plane images (EPI) in shearlet transform domain. The shearlet transform has been specifically modified to handle the straight lines characteristic for EPI. The devised iterative regularization algorithm based on adaptive thresholding provides high-quality reconstruction results for relatively big disparities between neighboring views. The generated densely sampled light field of a given 3D scene is thus suitable for all applications which require light field reconstruction. The proposed algorithm compares favorably against state of the art depth image based rendering techniques and shows superior performance specifically in reconstructing scenes containing semi-transparent objects.

In this work, we deal with the problem of denoising 3D scene range measurements acquired by Time-of-flight (ToF) range sensors and composed in the form of 2D image-like depth maps. We address the specific case of ToF low-sensing environment (LSE). Such environment is set by low-light sensing conditions, low-power hardware requirements, and low-reflectivity scenes. We demonstrate that data captured by a device in such mode can be effectively post-processed in order to reach the same measurement accuracy as if the device was working in normal operating mode. In order to achieve this, we first present an elaborated analysis of noise properties of ToF data sensed in LSE and verify the derived noise models by empirical measurements. Then, we develop a related novel non-local denoising approach working in complex domain and demonstrate its superiority against the state of the art for data acquired by an off-the-shelf ToF device.

Holographic stereograms (HSs) constitute one of the most widely used types of computer-generated holograms. The scene information required to calculate the HSs can be acquired by conventional digital cameras. It is, however, usually required that the scene should be captured from dense set of view points. Therefore, relieving this requirement is critical in the sense of easing the capture process. In this paper, in the capture stage of holographic stereograms, we employ our previously presented light field reconstruction algorithm [1], where we utilize sparse representation of light fields in the shearlet domain and reconstruct dense light fields from their highly under-sampled versions. The simulation results demonstrate that we can relieve the dense view sampling requirement of HSs, e.g. by as high as 8 × 8 sub-sampling factor, and still keep the perceived image quality of holographic reconstructions at satisfactory levels. This enables, for example, replacing the scanning camera setups with the more convenient multi-camera arrangements.

"View-plus-depth" is a popular 3D image representation format, in which the color 2D image is augmented with a gray-scale image representing the scene depth map aligned with the color pixels. In this paper, we propose a novel depth map compression method aimed at finding an optimal spatial depth scale and down-sampling (sparsifying) the depth image over it. The down-sampled depth image is then compressed by a combination of a new arithmetic and a predictive coder. Our approach is motivated by the current achievements in multi-sensor 3D scene sensing where low-resolution depth map captured by time-of-flight sensors is successfully up-sampled and aligned with high resolution RGB images. In our approach, color image segmentation in terms of super-pixels is used for finding the optimal depth scale and corresponding down-sampling. In contrast to other segmentation methods, it results in an isotropic and balanced low-resolution depth image, which is easily compressible. A bilateral regularizer is used for reconstructing the original-size depth map out of the low-resolution one and for splitting and predictive coding of segments with high reconstruction error. The scheme compares favorably with other methods for depth map compression.

Plane sweeping is an increasingly popular algorithm for generating depth estimates from multiview images. It avoids image rectification, and can align the matching process with slanted surfaces, improving accuracy and robustness. However, the size of the search space increases significantly when different surface orientations are considered. We present an efficient way to perform plane sweeping without individually computing reprojection and similarity metrics on image pixels for all cameras, all orientations and all distances. The procedure truly excels when the amount of views is increased and scales efficiently with the number of different plane orientations. It relies on approximation to generate the costs, but the differences are shown to be small. In practice, it provides results equivalent to conventional matching but faster, making it suitable for applying in many existing implementations.

In this paper we propose a method for reconstructing a densely sampled light field from a given sparse set of perspective views from rectified cameras without an explicit estimation of the scene depth. The desired intermediate views are synthesized by inpainting of epipolar-plane images, utilizing their sparsity in the shearlet domain. For the purpose of shearlet-domain representation, compactly supported shearlets have been constructed using different directional filters for different scales in an attempt to provide better directional selectivity at lower scales. The reconstruction procedure with shearlet-domain sparsity condition is implemented through an iterative thresholding algorithm. The performance of the method is quantified by tests on synthetic and real visual data and compared favorably against depth-image based rendering.

Light-field (LF) 3D displays require vast amount of views representing the original scene when using pure light-ray interpolation to convert multi-camera content to display-specific LF representation. Synthetic and real multi-camera setups are both used to feed these displays with image-based data, however the layout, number, frustum, and resolution of these cameras are mostly suboptimal. Storage and transmission of LF data is an issue, especially considering that some of the captured / rendered pixels are left unused while generating the final image. LF displays can have significantly different requirements for camera setups due to differences in Field of View (FOV), angular resolution and spatial resolution. An analysis of typical camera setups and LF display setups, and the typical patterns in pixel usage resulting from the combination of these setups are presented. Based on this analysis, an optimization method for virtual camera setups is proposed. As virtual cameras have wide range of adjustment possibilities, highly optimized setups for specific displays can be achieved.

Effective spatial resolution of projection-based 3D light-field (LF) displays is an important quantity, which is informative about the capabilities of the display to recreate views in space and is important for content creation. We propose a subjective experiment to measure the spatial resolution of LF displays and compare it to our objective measurement technique. The subjective experiment determines the limit of visibility on the screen as perceived by viewers. The test involves subjects determining the direction of patterns that resemble tumbling E eye test charts. These results are checked against the LF display resolution determined by objective means. The objective measurement models the display as a signal-processing channel. It characterizes the display throughput in terms of passband, quantified by spatial resolution measurements in multiple directions. We also explore the effect of viewing angle and motion parallax on the spatial resolution.

Some of the best performing local stereo-matching approaches use cross-bilateral filters for proper cost aggregation. The recent attempts have been directed toward efficient approximations of such filter aimed at higher speed. In this paper, we suggest a simple yet efficient coarse-to-fine cost volume aggregation scheme, which employs pyramidal decomposition of the cost volume followed by edge-avoiding reconstruction and aggregation. The scheme substantially reduces the computational complexity while providing fair quality of the estimated disparity maps compared to other approximated bilateral filtering schemes. In fact, the speed of the proposed technique is comparable with the speed of fixed kernel aggregation implemented through integral images.

We investigate possible improvements that can be achieved in depth estimation by merging coded apertures and stereo cameras. We analyze several stereo camera setups which are equipped with different sets of coded apertures to explore such possibilities. The demonstrated results of this analysis are encouraging in the sense that coded apertures can provide valuable complementary information to stereo vision based depth estimation in some cases. In addition to that, we take advantage of stereo camera arrangement to have a single shot multiple coded aperture system. We show that with this system, it is possible to extract depth information robustly, by utilizing the inherent relation between the disparity and defocus cues, even for scene regions which are problematic for stereo matching.

We propose a non-local de-noising approach aimed at filtering range data sensed by Photonic Mixer Device sensors. We address specifically the case of poor sensing environment when the reflected signal amplitude is low. In our approach, signal components of phase-delay and amplitude of the sensed signal are regarded as components of a complex-valued variable and processed together in a single step. This imposes better filter adaptivity and similarity weighting. The complex-domain filtering provides additional feedback in the form of improved noise-level confidence, which can be utilized in iterative de-noising schemes. Pre-filtering of individual components is proposed to suppress structural artifacts. Our approach compares favorably with state of the art approaches.

In this paper, we propose a practical approach for robust rectification of stereo camera setups without use of calibration pattern. Our solution simplifies the process to a non-general case of rectification to avoid explicit use of Fundamental Matrix estimation. The solution shows better or comparable robustness than some of recent solutions, but for much lower computational cost and code complexity.

The paper aims at characterizing the role of camera capture process in depth quality estimation by stereo-matching methods. An evaluation software application has been developed which integrates a modular image processing pipeline (IPP) in to the depth estimation process. The application allows for simulating the presence of camera-specific processing artifacts and their influence on the captured stereo imagery in typical imaging scenarios. Furthermore, it allows for benchmarking and optimizing both the depth estimation module and camera capture settings for a jointly-optimal performance.

In this paper, we present a complete framework of an end-to-end error resilient transmission of 3D video over DVB-H and provide an analysis of transmission parameters. We perform the analysis for various layering, protection strategy and prediction structure using different contents and different channel conditions.

In this paper, we present a novel stereoscopic video quality assessment method based on 3D-DCT transform. In our approach, similar blocks from left and right views of stereoscopic video frames are found by block-matching, grouped into 3D stack and then analyzed by 3D-DCT. Comparison between reference and distorted images are made in terms of MSE calculated within the 3D-DCT domain and modified to reflect the contrast sensitive function and luminance masking. We validate our quality assessment method using test videos annotated with results from subjective tests. The results show that the proposed algorithm outperforms current popular metrics over a wide range of distortion levels.

This paper aims at providing an overview of the core technologies enabling the delivery of 3-D Media to next-generation mobile devices. To succeed in the design of the corresponding system, a profound knowledge about the human visual system and the visual cues that form the perception of depth, combined with understanding of the user requirements for designing user experience for mobile 3-D media, are required. These aspects are addressed first and related with the critical parts of the generic system within a novel user-centered research framework. Next-generation mobile devices are characterized through their portable 3-D displays, as those are considered critical for enabling a genuine 3-D experience on mobiles. Quality of 3-D content is emphasized as the most important factor for the adoption of the new technology. Quality is characterized through the most typical, 3-D-specific visual artifacts on portable 3-D displays and through subjective tests addressing the acceptance and satisfaction of different 3-D video representation, coding, and transmission methods. An emphasis is put on 3-D video broadcast over digital video broadcasting-handheld (DVB-H) in order to illustrate the importance of the joint source-channel optimization of 3-D video for its efficient compression and robust transmission over error-prone channels. The comparative results obtained identify the best coding and transmission approaches and enlighten the interaction between video quality and depth perception along with the influence of the context of media use. Finally, the paper speculates on the role and place of 3-D multimedia mobile devices in the future internet continuum involving the users in cocreation and refining of rich 3-D media content.

In this paper, we present a complete framework of an end-to-end error resilient transmission of 3D video over Digital Video Broadcasting - Handheld (DVB-H) and provide an extensive analysis of coding and transmission parameters. We perform the analysis for different coding and error resilience schemes using different contents coded at different bitrate levels. Throughout the experiments, we investigate the effects of video content type, video bitrate, coding method and unequal protection level for different channel conditions. The results show that Multi-view Coding (MVC) coding outperforms Simulcast and distribution of available bitrate between video quality and Forward Error Correction (FEC) protection is an important factor in different channel conditions.

In this work, we present a multiple description coding (MDC) scheme for reliable transmission of compressed three dimensional (3-D) meshes. It trades off reconstruction quality for error resilience to provide the best expected reconstruction of 3-D mesh at the decoder side. The proposed scheme is based on multiresolution geometry compression achieved by using wavelet transform and modified SPIHT algorithm. The trees of wavelet coefficients are divided into sets. Each description contains the coarsest level mesh and a number of tree sets coded with different rates. The original 3-D geometry can be reconstructed with acceptable quality from any received description. More descriptions provide better reconstruction quality. The proposed algorithm provides flexible number of descriptions and is optimized for varying packet loss rates (PLR) and channel bandwidth.

Diffraction and holography are fertile areas for application of signal theory and processing. Recent work on 3DTV displays has posed particularly challenging signal processing problems. Various procedures to compute Rayleigh-Sommerfeld, Fresnel and Fraunhofer diffraction exist in the literature. Diffraction between parallel planes and tilted planes can be efficiently computed. Discretization and quantization of diffraction fields yield interesting theoretical and practical results, and allow efficient schemes compared to commonly used Nyquist sampling. The literature on computer-generated holography provides a good resource for holographic 3DTV related issues. Fast algorithms to compute Fourier, Walsh-Hadamard, fractional Fourier, linear canonical, Fresnel, and wavelet transforms, as well as optimization-based techniques such as best orthogonal basis, matching pursuit, basis pursuit etc., are especially relevant signal processing techniques for wave propagation, diffraction, holography, and related problems. Atomic decompositions, multiresolution techniques, Gabor functions, and Wigner distributions are among the signal processing techniques which have or may be applied to problems in optics. Research aimed at solving such problems at the intersection of wave optics and signal processing promises not only to facilitate the development of 3DTV systems, but also to contribute to fundamental advances in optics and signal processing theory.

Research efforts on 3DTV technology have been strengthened worldwide recently, covering the whole media processing chain from capture to display. Different 3DTV systems rely on different 3D scene representations that integrate various types of data. Efficient coding of these data is crucial for the success of 3DTV. Compression of pixel-type data including stereo video, multiview video, and associated depth or disparity maps extends available principles of classical video coding. Powerful algorithms and open international standards for multiview video coding and coding of video plus depth data are available and under development, which will provide the basis for introduction of various 3DTV systems and services in the near future. Compression of 3D mesh models has also reached a high level of maturity. For static geometry, a variety of powerful algorithms are available to efficiently compress vertices and connectivity. Compression of dynamic 3D geometry is currently a more active field of research. Temporal prediction is an important mechanism to remove redundancy from animated 3D mesh sequences. Error resilience is important for transmission of data over error prone channels, and multiple description coding (MDC) is a suitable way to protect data. MDC of still images and 2D video has already been widely studied, whereas multiview video and 3D meshes have been addressed only recently. Intellectual property protection of 3D data by watermarking is a pioneering research area as well. The 3D watermarking methods in the literature are classified into three groups, considering the dimensions of the main components of scene representations and the resulting components after applying the algorithm. In general, 3DTV coding technology is maturating. Systems and services may enter the market in the near future. However, the research area is relatively young compared to coding of other types of media. Therefore, there is still a lot of room for improvement and new development of algorithms.

In this paper we propose a computationally efficient method for image super-resolution reconstruction. We concentrate on pure translational motion and shift-invariant blur. This is the case of interlaced sampling where each low resolution image is uniformly sampled but the overall sampling is non-uniform with respect to the high resolution grid. The reconstruction problem is considered in a multi-resolution framework using separable B-spline wavelets as basis. This allows performing the reconstruction on a uniform higher resolution grid by digital filtering. A computationally efficient structure-the transposed modified Farrow structure is used to achieve a near least squares solution without using any matrix inversions or iterations. The reconstruction can also be performed at non-dyadic scales. Our method minimizes the aliasing distortions and gives results comparable with other least squares techniques.

Two important signal processing problems in the display side of a holographic 3DTV are the computation of the diffraction field of a 3D object from its abstract representation, and determination of the best display configuration to synthesize some intended light distribution. To solve the former problem, we worked on the computation of 1D diffraction patterns from discrete data distributed over 2D space. The problem is solved using matrix pseudo-inversion which dominates the computational complexity. Then, the light field synthesis problem by a deflectable mirror array device (DMAD) is posed as a constrained linear optimization problem. The formulation makes direct application of common optimization algorithms quite easy. The simulations indicate that developed methods are promising.

We present a method based on the matching pursuits algorithm for the extraction of time-frequency features that can be used for classification of various abnormal heartbeats. Further, we investigate the usefulness of independent component analysis for extracting additional spatial features from multichannel electrocardiographic recordings. The performance of these two different sets of features is assessed using the 48 recordings of the MIT-BIH arrhythmia database.

This paper introduces an image decimation technique based on the use of a near least-squares criterion that makes a proper compromise between the L/sub 2/ and l/sub 2/ norm minimization cases. The theory of orthogonal projections is related to the derivation of a computationally efficient decimation structure possessing good antialiasing properties. It is shown how this structure can be realized by the transposed Farrow structure when using piece-wise polynomial basis functions. It is shown, by means of examples, that with a considerably lower computational complexity the proposed structure provides practically the same quality for the restored images as the best existing structures.

A method for electromyogram (EMG) artifact suppression in electrocardiogram (ECG) recordings is presented. In an attempt to improve the traditional compromise between efficient EMG artifact suppression and preservation of the ECG waveform, the method combines a dynamic approximation filtering working in the QRS complexes areas, with a transform domain denoising of the segments outside them. The switching between the two procedures is controlled by the ECG signal slew rate. The results obtained show a virtual preservation of the QRS amplitudes and a considerable reduction of the EMG artifact.

An edge-preserving method for image resizing (decimation and interpolation) is proposed. The decimation is considered as an orthogonal projection with respect to the chosen interpolation basis. The latter one is formed in a spline-like manner as a linear combination of B-splines of different degrees. This combination is optimized in such a way that the small image details are preserved. Considering the strongest edges as step edges, a segmentation procedure preceding the decimation is proposed. It leads to resized images with clearly outlined borders.

A new two-stage algorithm for electrocardiographic (EGG) signal denoising has been proposed. It combines wavelet shrinkage with Wiener filtering in the translation-invariant wavelet domain. A time-frequency dependent thresholding has been proposed and grounded for obtaining a more adequate signal estimate in the first stage of the algorithm. It is related to ECG signal morphology and hence outperforms other thresholding approaches in this area. The experiments carried out on pathological and normal ECGs have shown better algorithm capabilities in comparison with other thresholding algorithms while suppressing parasite electromyographic (EMG) signals (the noise) and preserving diagnostically important ECG signal features.