Novel view synthesis in 360° scenes from extremely sparse input views is essential for applications like virtual reality and augmented reality. This paper presents a novel framework for novel view synthesis in extremely sparse-view cases. As typical structure-from-motion methods are unable to estimate camera poses in extremely sparse-view cases, we apply DUSt3R to estimate camera poses and generate a dense point cloud. Using the poses of estimated cameras, we densely sample additional views from the upper hemisphere space of the scenes, from which we render synthetic images together with the point cloud. Training 3D Gaussian Splatting model on a combination of reference images from sparse views and densely sampled synthetic images allows a larger scene coverage in 3D space, addressing the overfitting challenge due to the limited input in sparse-view cases. Retraining a diffusion-based image enhancement model on our created dataset, we further improve the quality of the point-cloud-rendered images by removing artifacts. We compare our framework with benchmark methods in cases of only four input views, demonstrating significant improvement in novel view synthesis under extremely sparse-view conditions for 360° scenes. The source code is available at https: //github.com/angchen-dev/hemiSparseGS.

The recently introduced 3D Gaussian Splatting and subsequent methods have achieved significantly reduced inference times for novel view synthesis. To reduce this rendering time even further, in this paper we propose four improvements which are fully compatible with the high-level Gaussian Splatting formulation and can thus be incorporated into most methods based on this paradigm. Most notably, we alter the way Gaussians are duplicated across tiles by allowing for non-square axis-aligned Gaussian bounding boxes whose sizes take into account the Gaussian's opacity information. Our experiments demonstrate that we can decrease the 3D Gaussian Splatting rendering times by up to a factor of almost 4.

Post-training quantization (PTQ) is an effective solution for deploying deep neural networks on edge devices with limited resources. PTQ is especially attractive because it does not require access to the entire original training dataset on the promise of being able to use a much smaller calibration dataset. However, many existing PTQ methods still require a sufficiently large calibration dataset (e.g., more than 1000 images) to achieve satisfactory model accuracy. In this paper, we present a novel post-training quantization method that estimates quantization parameters using a Bayesian Maximum A Posterior (MAP) estimator. By modeling the uncertainty of quantization operations, we formulate the neural network quantization as a Bayesian inference problem. In our method, we first employ probabilistic programming techniques to optimize quantization parameters by maximizing the posterior of quantization step sizes. In addition, we introduce a Minimum Description Length (MDL) prior that favors low quantization bit widths and a validation procedure, which enhances PTQ performance when learning from small calibration datasets. Comprehensive evaluations demonstrate that the proposed method can improve the PTQ performance using a minimal calibration dataset of just 64 images, and achieve nearly state-of-the-art PTQ performance. Furthermore, the proposed method shows strong generalization ability when calibrated on different data sources and tested across diverse data.

Hyperspectral images (HSIs) play a pivotal role in fields, such as medical diagnosis and agriculture. However, it often contends with significant noise stemming from narrowband spectral filtering. Existing denoising techniques have their limitations: model-driven methods rely on manual priors and hyperparameters, while learning-based methods struggle to discern intrinsic noise patterns, as they require paired images with specific example noise for training, fail to capture critical noise distribution information, leading to unrobust denoising results. This work addresses the issue by presenting a degradation-noise-aware unfolding network (DNA-Net). Unlike training directly with the simulated noise, DNA-Net initially models general sparse and Gaussian noise through statistic distributions. It then explicitly represents image priors with a customized spectral transformer. The model is subsequently unfolded into an end-to-end (E2E) network, with hyperparameters adaptively estimated from noisy HSI and degradation models, effectively regulating each iteration. Furthermore, a novel U-shaped local-nonlocal–spectral transformer (U-LNSA) is introduced, simultaneously capturing spectral correlations, local features, and nonlocal dependencies. The integration of U-LNSA into DNA-Net establishes the first Transformer-based deep unfolding method for HSI denoising. Experimental results on synthetic and real noise validate DNA-Net’s superior performance over state-of-the-art (SOTA) methods. Moreover, the DNA-Net, trained exclusively on mixed Gaussian noise and impulse noise, demonstrates the ability to generalize to unseen noise present in real images. Code and models will be released at:https://github.com/NavyZeng/DNA-Net.

Hyperspectral images (HSIs) have extensive applications in various fields such as medicine, agriculture, and industry. Nevertheless, acquiring high signal-to-noise ratio HSI poses a challenge due to narrow-band spectral filtering. Consequently, the importance of HSI denoising is substantial, especially for snapshot hyperspectral imaging technology. While most previous HSI denoising methods are supervised, creating supervised training datasets for the diverse scenes, hyperspectral cameras, and scan parameters is impractical. In this work, we present Diff-Unmix, a self-supervised denoising method for HSI using diffusion denoising generative models. Specifically, Diff-Unmix addresses the challenge of recovering noise-degraded HSI through a fusion of Spectral Unmixing and conditional abundance generation. Firstly, it employs a learnable block-based spectral unmixing strategy, complemented by a pure transformer-based backbone. Then, we introduce a self-supervised generative diffusion network to enhance abundance maps from the spectral unmixing block. This network reconstructs noise-free Unmixing probability distributions, effectively mitigating noise-induced degradations within these components. Finally, the reconstructed HSI is reconstructed through unmixing reconstruction by blending the diffusion-adjusted abundance map with the spectral endmembers. Experimental results on both simulated and real-world noisy datasets show that Diff-Unmix achieves state-of-the-art performance.

When using a GNSS/INS sensor to provide poses to combine LiDAR scans over time, knowing the extrinsic calibration between this GNSS/INS sensor and the LiDAR device is essential. In this work we present a new calibration method that directly optimises an entropy-based loss function, starting from a decent, but suboptimal calibration, e.g. obtained using simple measurement tools such as a ruler and a protractor. Our method does not require placing any markers, nor imposes restrictions on the types of scenes. In particular, we do not rely on plane detection and planar features. Our experiments demonstrate that we can compensate for initial rotational errors up to the order of 4 degrees.

In this article, we propose a novel bilayer low-rankness measure and two models based on it to recover a low-rank (LR) tensor. The global low rankness of underlying tensor is first encoded by LR matrix factorizations (MFs) to the all-mode matricizations, which can exploit multiorientational spectral low rankness. Presumably, the factor matrices of all-mode decomposition are LR, since local low-rankness property exists in within-mode correlation. In the decomposed subspace, to describe the refined local LR structures of factor/subspace, a new low-rankness insight of subspace: a double nuclear norm scheme is designed to explore the so-called second-layer low rankness. By simultaneously representing the bilayer low rankness of the all modes of the underlying tensor, the proposed methods aim to model multiorientational correlations for arbitrary N -way ( N ≥ 3 ) tensors. A block successive upper-bound minimization (BSUM) algorithm is designed to solve the optimization problem. Subsequence convergence of our algorithms can be established, and the iterates generated by our algorithms converge to the coordinatewise minimizers in some mild conditions. Experiments on several types of public datasets show that our algorithm can recover a variety of LR tensors from significantly fewer samples than its counterparts.

With the recent development of the joint classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) data, deep learning methods have achieved promising performance owing to their locally sematic feature extracting ability. Nonetheless, the limited receptive field restricted the convolutional neural networks (CNNs) to represent global contextual and sequential attributes, while visual image transformers (VITs) lose local semantic information. Focusing on these issues, we propose a fractional Fourier image transformer (FrIT) as a backbone network to extract both global and local contexts effectively. In the proposed FrIT framework, HSI and LiDAR data are first fused at the pixel level, and both multisource feature and HSI feature extractors are utilized to capture local contexts. Then, a plug-and-play image transformer FrIT is explored for global contextual and sequential feature extraction. Unlike the attention-based representations in classic VIT, FrIT is capable of speeding up the transformer architectures massively and learning valuable contextual information effectively and efficiently. More significantly, to reduce redundancy and loss of information from shallow to deep layers, FrIT is devised to connect contextual features in multiple fractional domains. Five HSI and LiDAR scenes including one newly labeled benchmark are utilized for extensive experiments, showing improvement over both CNNs and VITs.

In this paper we propose a computationally efficient neural network for high dynamic range fusion and tone mapping, for application in perception systems of autonomous vehicles. The proposed approach fuses two consecutive, differently exposed images into a single output with good exposure in all regions, in a standard dynamic range. Motion is compensated based on fast optical flow estimation, and subsequently by including an error mask as an input to the network to indicate the remaining artifact-prone regions. This is an efficient way for the network to learn to reduce the ghosting artifacts without increasing computational complexity. Unlike the conventional approach, we train the network on versatile traffic data, and evaluate the performance based on object detection quality metrics, rather than for visual quality. The performance was compared to a similarly complex representative method from literature. We achieved improved performance in challenging light conditions due to the robustness of our method in variable traffic conditions.

Radar has become a key sensor in many advanced driver assistance systems (ADAS). Due to its excellent range and Doppler resolution, low cost, and robustness against environmental conditions, it is considered an attractive sensor for detecting and tracking vulnerable road users (VRUs). In this paper, we provide a theoretical analysis of the accuracy of radar based VRU detection and tracking systems, thereby focusing on a number of specific scenarios such as early detection and tracking of VRUs, a VRU crossing the road at constant speed in front of a vehicle, a VRU moving parallel to a vehicle, etc. More specifically, we derive the Cramer-Rao lower bound (CRLB) for position and velocity estimation of a moving target based on a sequence of noisy range, azimuth, and Doppler measurements taken from a moving ego-vehicle equipped with one or multiple radars. Not only does the CRLB serve as a benchmark to evaluate the performance of any practical tracking algorithm, it also allows to gain practical insights regarding the impact of the radar setup and configuration on the tracking accuracy in different realistic scenarios. Furthermore, we show that the generalized least-squares estimator (GLSE) achieves excellent performance when few measurements are available, and propose a novel active sensing (AS) application based on the CRLB where the radar configuration is optimized on the fly to improve either tracking accuracy or computational efficiency.

Low-rank tensor completion has been widely used in computer vision and machine learning. This paper develops a novel multimodal core tensor factorization (MCTF) method combined with a tensor low-rankness measure and a better nonconvex relaxation form of this measure (NC-MCTF). The proposed models encode low-rank insights for general tensors provided by Tucker and T-SVD and thus are expected to simultaneously model spectral low-rankness in multiple orientations and accurately restore the data of intrinsic low-rank structure based on few observed entries. Furthermore, we study the MCTF and NC-MCTF regularization minimization problem and design an effective block successive upper-bound minimization (BSUM) algorithm to solve them. Theoretically, we prove that the iterates generated by the proposed models converge to the set of coordinatewise minimizers. This efficient solver can extend MCTF to various tasks such as tensor completion. A series of experiments including hyperspectral image (HSI), video and MRI completion confirm the superior performance of the proposed method.

Recent success in the field of single image super-resolution (SISR) is achieved by optimizing deep convolutional neural networks (CNNs) in the image space with the L1 or L2 loss. However, when trained with these loss functions, models usually fail to recover sharp edges present in the high-resolution (HR) images for the reason that the model tends to give a statistical average of potential HR solutions. During our research, we observe that gradient maps of images generated by the models trained with the L1 or L2 loss have significantly lower variance than the gradient maps of the original high-resolution images. In this work, we propose to alleviate the above issue by introducing a structure-enhancing loss function, coined Gradient Variance (GV) loss, and generate textures with perceptual-pleasant details. Specifically, during the training of the model, we extract patches from the gradient maps of the target and generated output, calculate the variance of each patch and form variance maps for these two images. Further, we minimize the distance between the computed variance maps to enforce the model to produce high variance gradient maps that will lead to the generation of high-resolution images with sharper edges. Experimental results show that the GV loss can significantly improve both Structure Similarity (SSIM) and peak signal-to-noise ratio (PSNR) performance of existing image super-resolution (SR) deep learning models.

To solve the limitation of labeled samples in hyperspectral image (HSI) classification, cross-scene learning methods are developed recently. However, the disparity caused by environmental variation between HSI scenes is still a challenge. As a supplement, light detection and ranging (LiDAR) data provides elevation and spatial information regardless the variations. In this paper, we propose a multisource cross-scene classification method using fractional fusion and spatial-spectral domain adaptation to reduce disparity between scenes. The spatial information of HSI is preserved by fractional differential masks (FrDM) firstly. Then the LiDAR data is utilized for spectral alignment of HSI. The utilization of LiDAR data reduces the pixel-level disparity between scenes. At last, a spatial-spectral domain adaptation network is proposed for feature extraction and classification. Experimental results on HSI and LiDAR scenes show 5% improvements in overall accuracy compared with state-of-the-art methods.

Focusing on joint classification of Hyperspectral image (HSI) and Light detection and ranging (LiDAR) data, a fractional Fourier image transformer (FrIT) is proposed as a backbone network in this paper. In the proposed FrIT, HSI and LiDAR data are firstly fused at pixel-level. Both multi-source and HSI feature extractors are utilized to capture local contexts. Then, a plug-and-play image transformer FrIT is explored for global contexts and sequential feature extraction. Unlike the attention-based representations in classic visual image transformer (VIT), FrIT is capable of speeding up the transformer architectures massively. To reduce the information loss from shallow to deep layers, FrIT is devised to connect contextual features in multiple fractional domains. At last, to evaluate the performance of FrIT, a new HSI and LiDAR benchmark is provided for extensive experiments, on which the proposed FrIT gains an improvement of 3% over state-of-the-art methods.

Pansharpening refers to a spatial–spectral contexts fusion procedure to produce high-quality multispectral (MS) images by retaining the fine spatial resolution of the panchromatic (PAN) images and the high spectral content of the MS images. This letter presents a novel end-to-end dual-branch deep learning-based fusion framework, exploiting the network to extract spatial and spectral contexts progressively in two separate branches level by level. For each level contexts extraction layer, a dual-branch weighted attentive fusion module is integrated to boost the important contexts aggregation and details injection while suppressing unimportant ones. Experimental results on two real datasets show that our method outperforms state-of-the-art methods in both objective metrics and image quality by visual appearance.

Pansharpening is an image fusion procedure, which aims to produce a high spatial resolution multispectral image by combining a low spatial resolution multispectral image and a high spatial resolution panchromatic image. The most popular and successful paradigm for pansharpening is the framework known as detail injection, while it cannot fully exploit complex and non-linear complementary features of both images. In this paper, we propose a detail injection model inspired deep fusion network for pansharpening (DIM-FuNet). Firstly, by treating pansharpening as a complicated and non-linear details learning and injection problem, we establish a unified optimizing detail-injection model with triple detail fidelity terms: 1) a band-dependent spatial detail fidelity term, 2) a local detail fidelity term and 3) a complicated details synthesis term. Secondly, the model is optimized via the iterative gradient descent and unfolded into a deep convolutional neural network. Subsequently, the unrolling network has triple branches, in which, a point-wise convolutional sub-network, a depth-wise convolutional sub-network are corresponding to the former two detail constrained terms, and an adaptive weighted reconstruction module with a fusion sub-network to aggregate details of two branches and synthesis the final complicated details. Finally, the deep unrolling network is trained in end-to-end manners. Different from traditional deep fusion networks, the architecture design of DIM-FuNet is guided by the optimizing model and thus promotes better interpretability. Experimental results on reduced and full-resolution demonstrate the effectiveness of the proposed DIM-FuNet which achieves the best performance compared with the state-of-the-art pansharpening method.

Remote sensing using multisensor platforms has been systematically applied for monitoring and optimizing human activities. Several advanced techniques have been developed to enhance and extract the spatially and spectrally semantic information in the hyperspectral image (HSI) and light detection and ranging (LiDAR) data processing and analysis. However, an abundance of redundant information and sometimes a lack of discriminative features reduce the efficiency and effectiveness of multisource classification methods. This article proposes a fractional Gabor convolutional network (FGCN), focusing on efficient feature fusion and comprehensive feature extraction. First, the proposed FGCN uses Octave convolution layers to perform multisource information fusion and preserve discriminative information. Second, fractional Gabor convolutional (FGC) layers are proposed to extract multiscale, multidirectional, and semantic change features. The completeness and discrimination of the multisource features using different FGC kernels are improved, which yield robust feature extraction against semantic changes. Finally, the fractional Gabor feature and spectral feature are combined with two weighting factors which can be learned during the network training. Experimental results and comparisons with state-of-the-art multisource classification methods indicate the effectiveness of the proposed FGCN. With the FGCN, we can obtain an 89.90% overall accuracy on the challenging Muufl Gulfport (MUUFL) data set, with an improvement of 3% over state-of-the-art methods.

Matching aerial images might be challenging when they contain a large number of repetitive patterns. In this paper, we propose a feature-matching method that exploits the use of Affine Oriented FAST and Rotated BRIEF (AORB) as key-point detector and feature descriptor and not accurate GPS (Global Position System) data to achieve a reliable feature matching of nadir UAV images that contain a large number of repetitive patterns. The proposed method assumes that the set of correct matches between two images only differ in a 2D translation. Experimental results show that the proposed method is able to correctly match pairs of very challenging images containing a large number of repetitive patterns.

Radar has been employed as a key component of perception modules for more than two decades. However, radar image labeling requires expert knowledge. At the same time, it is much more time-consuming than for general RGB images, which impedes further developments of radar. In order to alleviate the high-cost annotation problem in radar datasets, we present a novel, semi-supervised deep learning method based on the spatio-temporal consistency. This way we explore the potential of unlabeled radar frames to enhance performance. We utilize the consecutive radar frames from different timeline directions to encourage the model to learn the target motion. Moreover, the proposed self-weighted mechanism avoids over-fitting on certain predominant targets, by exploiting the supervised classification loss dynamically. We evaluate the proposed method on semantic segmentation and Vulnerable Road Users (VRUs) detection problems. The quantitative results compare favourably to the state-of-the-art and demonstrate the effectiveness of the proposed concepts. The ablation studies also show the effectiveness of the proposed components.

Hyperspectral image super-resolution by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI) aims at reconstructing high resolution spatial-spectral information of the scene. Existing methods mostly based on spectral unmixing and sparse representation are often developed from a low-level vision task perspective, they cannot sufficiently make use of the spatial and spectral priors available from higher-level analysis. To this issue, this paper proposes a novel HSI super-resolution method that fully considers the spatial/spectral subspace low-rank relationships between available HR-MSI/LR-HSI and latent HSI. Specifically, it relies on a new subspace clustering method named "structured sparse low-rank representation" (SSLRR), to represent the data samples as linear combinations of the bases in a given dictionary, where the sparse structure is induced by low-rank factorization for the affinity matrix. Then we exploit the proposed SSLRR model to learn the SSLRR along spatial/spectral domain from the MSI/HSI inputs. By using the learned spatial and spectral low-rank structures, we formulate the proposed HSI super-resolution model as a variational optimization problem, which can be readily solved by the ADMM algorithm. Compared with state-of-the-art hyperspectral super-resolution methods, the proposed method shows better performance on three benchmark datasets in terms of both visual and quantitative evaluation.

Depth information is useful for many applications in the real world such as view synthesis, and 3D reconstruction. However, the resolution of the depth map is much lower than the resolution of the color image. Therefore, super-resolution on depth maps has drawn a lot of attention and achieved great success with machine learning-based methods in recent years. However, the prior work is still limited to training a specific model for each integer scale factor and only works with a set of few integer scale factors. Moreover, the depth map collected by the depth sensor can also have a lot of depth missing values and depth error along the edge and corners of observed objects. In this work, we propose a novel method to perform both super-resolution by an arbitrary scale factor and inpainting to fill in the missing depth values on the depth map. The experimental results on Middlebury RGBD datasets show the effectiveness of our proposed approach.

Internet-of-Things (IoT) technologies incorporate a large number of different sensing devices and communication technologies to collect a large amount of data for various applications. Smart cities employ IoT infrastructures to build services useful for the administration of the city and the citizens. In this article, we present an IoT pipeline for acquisition, processing, and visualization of air pollution data over the city of Antwerp, Belgium. Our system employs IoT devices mounted on vehicles as well as static reference stations to measure a variety of city parameters, such as humidity, temperature, and air pollution. Mobile measurements cover a larger area compared to static stations; however, there is a tradeoff between temporal and spatial resolution. We address this problem as a matrix completion on graphs problem and rely on variational graph autoencoders to propose a deep learning solution for the estimation of the unknown air pollution values. Our model is extended to capture the correlation among different air pollutants, leading to improved estimation. We conduct experiments at different spatial and temporal resolution and compare with state-of-the-art methods to show the efficiency of our approach. The observed and estimated air pollution values can be accessed by interested users through a Web visualization tool designed to provide an air pollution map of the city of Antwerp.

Earth observation using multisensor data is drawing increasing attention. Fusing remotely sensed hyperspectral imagery and light detection and ranging (LiDAR) data helps to increase application performance. In this article, joint classification of hyperspectral imagery and LiDAR data is investigated using an effective hierarchical random walk network (HRWN). In the proposed HRWN, a dual-tunnel convolutional neural network (CNN) architecture is first developed to capture spectral and spatial features. A pixelwise affinity branch is proposed to capture the relationships between classes with different elevation information from LiDAR data and confirm the spatial contrast of classification. Then in the designed hierarchical random walk layer, the predicted distribution of dual-tunnel CNN serves as global prior while pixelwise affinity reflects the local similarity of pixel pairs, which enforce spatial consistency in the deeper layers of networks. Finally, a classification map is obtained by calculating the probability distribution. Experimental results validated with three real multisensor remote sensing data demonstrate that the proposed HRWN significantly outperforms other state-of-the-art methods. For example, the two branches CNN classifier achieves an accuracy of 88.91% on the University of Houston campus data set, while the proposed HRWN classifier obtains an accuracy of 93.61%, resulting in an improvement of approximately 5%.

Inferring air quality from a limited number of observations is an essential task for monitoring and controlling air pollution. Existing inference methods typically use low spatial resolution data collected by fixed monitoring stations and infer the concentration of air pollutants using additional types of data, e.g., meteorological and traffic information. In this work, we focus on street-level air quality inference by utilizing data collected by mobile stations. We formulate air quality inference in this setting as a graph-based matrix completion problem and propose a novel variational model based on graph convolutional autoencoders. Our model captures effectively the spatio-temporal correlation of the measurements and does not depend on the availability of additional information apart from the street-network topology. Experiments on a real air quality dataset, collected with mobile stations, shows that the proposed model outperforms state-of-the-art approaches.

Early detection of banana disease can limit the spread of disease, as well as reduce the treatment costs. However, the disease symptoms are so unapparent in the earlier stage that makes the labeled samples acquisition difficult and expensive. Meanwhile, it is much easier to obtain labeled samples at the late stage where the disease symptoms are obvious. In this paper, we exploit machine learning methods to use labeled samples from the late stage to train the model, then detect the banana disease in the earlier stage. Morphological openings and closings are utilized to extract the spectral-spatial features from banana leaves at both earlier and late stages, initial experimental results demonstrate significant improvements over using only spectral information.

In this work, we propose a semi-supervised method for classification of polarimetric synthetic aperture radar (PolSAR) images. In the proposed method, a 2-level mixture model is constructed by associating each component density with a unique Wishart mixture model (instead of a single Wishart distribution as that in the conventional Wishart mixture model). This modeling scheme facilitates the accurate description of data for the categories, each of which includes multiple subcategories. The learning algorithm for the proposed model is developed based on variational inference and all the update equations are obtained in closed form. In the learning algorithm, the spatial interdependencies are incorporated by imposing a Markov random field prior on the indicator variable to alleviate the speckle effect on the classification results. The experimental results demonstrate the improved performance of the proposed method compared with the unsupervised version and supervised version of the proposed model as well as an existing method for semi-supervised classification.

This paper proposes an unsupervised classification method for multilook polarimetric synthetic aperture radar (Pol-SAR) data. The proposed method simultaneously deals with the heterogeneity and incorporates the local correlation in PolSAR images. Specifically, within the probabilistic framework of the Dirichlet process mixture model (DPMM), an observed PolSAR data point is described by the multiplication of a Wishartdistributed component and a class-dependent random variable (i.e., the textual variable). This modeling scheme leads to the proposed textured DPMM (tDPMM), which possesses more flexibility in characterizing PolSAR data in heterogeneous areas and from high-resolution images due to the introduction of the classdependent texture variable. The proposed tDPMM is learned by solving an optimization problem to achieve its Bayesian inference. With the knowledge of this optimization-based learning, the local correlation is incorporated through the pairwise constraint, which integrates an appropriate penalty term into the objective function so as to encourage the neighboring pixels to fall into the same category and to alleviate the "salt-and-pepper" classification appearance.We develop the learning algorithm with all the closed-form updates. The performance of the proposed method is evaluated with both low-resolution and high-resolution PolSAR images, which involve homogeneous, heterogeneous, and extremely heterogeneous areas. The experimental results reveal that the class-dependent texture variable is beneficial to PolSAR image classification and the pairwise constraint can effectively incorporate the local correlation in PolSAR images.

We present Quasar, a new programming framework that handles many complex aspects in the design of smart vision systems on embedded platforms, such as parallelization, data flow management, scheduling and load balancing. Quasar, as a high-level programming language, is nearly hardware-agnostic, has a low barrier of entry and is therefore well suited for algorithm design and rapid prototyping. Through several benchmarks and application use cases we demonstrate that programs written in Quasar have a performance that is on a par with (or better than) hand-tuned CUDA and OpenACC code while the development requires much less time and is future-proof.

Pedestrian detection has achieved a remarkable progress in recent years, but challenges remain especially when occlusion happens.Intuitively, occluded pedestrian samples contain some characteristic occlusion appearance features that can help to improve detection.However, we have observed that most existing approaches intentionally avoid using samples of occluded pedestrians during the training stage.This is because such samples will introduce unreliable information, which affects the learning of model parameters and thus results in dramatic performance decline.In this paper, we propose a new framework for pedestrian detection.The proposed method exploits the use of occluded pedestrian samples to learn more robust features for discriminating pedestrians, and enables better performances on pedestrian detection, especially for the occluded pedestrians (which always happens in many real applications).Compared to some recent detectors on Caltech Pedestrian dataset, with our proposed method, detection miss rate for occluded pedestrians are significantly reduced.

The high dimensionality of hyperspectral images, the limited availability of ground-truth data as well as the low spatial resolution (causing pixels to contain mixtures of materials) hinder hyperspectral image classification. In this work we propose a novel hyperspectral classification method where we combine the outcome of spectral unmixing with the outcome of a supervised classifier. In particular, we consider fractional abundances obtained from a Sparse Unmixing method along with posterior probabilities acquired from a Multinomial Logistic Regression classifier. Both sources of information are fused using a Markov Random Field framework. We conducted experiments on publicly available real hyperspectral images: Indian Pines and University of Pavia using a very limited number of training samples. Our results indicate that the proposed decision fusion approach significantly improves the classification result over using the individual sources and outperforms the state of the art methods.

Early detection of banana disease can limit the spread of disease, as well as reduce the treatment costs. Current methods focus on either manually interpretation or calculation of spectral indices (e.g., the normalized difference vegetation index). In this paper, we exploit the fusion of close range hyperspectral (HS) image and high-resolution (HR) visible RGB image for potential disease detection in banana leaves. Our approach applies the joint bilateral filter to transfer the textural structures of HR RGB image to low-resolution HS image and obtain an enhanced HS image. Initial experimental results on Musa acuminata (banana) leaf images demonstrate the efficiency of our fusion approach, with significant improvements over either single data source or some conventional methods.

High Dynamic Range (HDR) displays can show images with higher color contrast levels and peak luminosities than the common Low Dynamic Range (LDR) displays. However, most existing video content is recorded and/or graded in LDR format. To show this LDR content on HDR displays, a dynamic range expansion by using an Inverse Tone Mapped Operator (iTMO) is required. In addition to requiring human intervention for tuning, most of the iTMOs don't consider artistic intentions inherent to the HDR domain. Furthermore, the quality of their results decays with peak brightness above 1000 nits. In this paper, we propose a fully-automatic inverse tone mapping operator based on mid-level mapping. This allows expanding LDR images into HDR with peak brightness over 1000 nits, preserving the artistic intentions inherent to the HDR domain. We assessed our results using full-reference objective quality metrics as HDR- VDP-2.2 and DRIM. Experimental results demonstrate that our proposed method outperforms the current state of the art.

This paper addresses the unsupervised classification problems for multilook Polarimetric synthetic aperture radar (PolSAR) images by proposing a patch-level spatially variant Wishart mixture model (SVWMM) with double constraints. We construct this model by jointly modeling the pixels in a patch (rather than an individual pixel) so as to effectively capture the local correlation in the PolSAR images. More importantly, a responsibility parameter is introduced to the proposed model, providing not only the possibility to represent the importance of different pixels within a patch but also the additional flexibility for incorporating the spatial information. As such, double constraints are further imposed by simultaneously utilizing the similarities of the neighboring pixels, respectively, defined on two different parameter spaces (i.e., the hyperparameter in the posterior distribution of mixing coefficients and the responsibility parameter). Furthermore, the variational inference algorithm is developed to achieve effective learning of the proposed SVWMM with the closed-form updates, facilitating the automatic determination of the cluster number. Experimental results on several PolSAR data sets from both airborne and spaceborne sensors demonstrate that the proposed method is effective and it enables better performances on unsupervised classification than the conventional methods.

A linear, or 1D, camera is a type of camera that sweeps a linear sensor array over the scene, rather than capturing the scene using a single impression on a 2D sensor array. They are often used in satellite imagery, industrial inspection, or hyperspectral imaging. In satellite imaging calibration is often done through a collection of ground points for which the 3D locations are known. In other applications, e.g. hyperspectral imaging, such known points are not available and annotating many different points is onerous. Hence we will use a checkerboard for calibration. The state-of-the-art method for linear camera calibration with a checkerboard becomes unstable when the checkerboards are parallel to the image plane. Our proposed method1yields more accurate camera calibrations without suffering from this shortcoming.

The added value of multiple data sources on tree species mapping has been widely analyzed. In particular, fusion of hyperspectral (HS) and LiDAR sensors for forest applications is a very hot topic. In this paper, we exploit the use of multi-scale features to fuse HS and LiDAR data for tree species mapping. Hyperspectral data is obtained from the APEX sensor with 286 spectral bands. LiDAR data has been acquired with a TopoSys sensor Harrier 56 at full waveform. We generate multi-scale features on both HS and LiDAR data, by considering the diameter and the height layer of different tree species. Experimental results on a forested area in Belgium demonstrate the effectiveness of using multi-scale features for fusion of HS image and LiDAR data both visually and quantitatively.

Synthetic aperture radar (SAR) data are becoming more and more accessible and have been widely used in many applications. To effectively and efficiently represent multiple SAR images, we propose the mixture of Itakura-Saito (IS) divergence for non-negative matrix factorization (NMF) to perform the dimension reduction. Our proposed method incorporates the unit-mean Gamma mixture model into the NMF to model the multiplicative noise. To obtain the closed-form update equations as much as possible, we approximate the log-likelihood function with its lower bound. Finally, we apply Expectation-Maximization (EM) algorithm to estimate the parameters, resulting in the closed-form multiplicative update rules for the two matrix factors. Experimental results on real SAR dataset demonstrate the effectiveness of the proposed method and its applicability to post applications (e.g., classification) with improved performances over the conventional dimension reduction methods.

We present a novel technique for fast and accurate reconstruction of depth images from 3D point clouds acquired in urban and rural driving environments. Our approach focuses entirely on the sparse distance and reflectance measurements generated by a LiDAR sensor. The main contribution of this paper is a combined segmentation and upsampling technique that preserves the important semantical structure of the scene. Data from the point cloud is segmented and projected onto a virtual camera image where a series of image processing steps are applied in order to reconstruct a fully sampled depth image. We achieve this by means of a multilateral filter that is guided into regions of distinct objects in the segmented point cloud. Thus, the gains of the proposed approach are two-fold: measurement noise in the original data is suppressed and missing depth values are reconstructed to arbitrary resolution. Objective evaluation in an automotive application shows state-of-the-art accuracy of our reconstructed depth images. Finally, we show the qualitative value of our images by training and evaluating a RGBD pedestrian detection system. By reinforcing the RGB pixels with our reconstructed depth values in the learning stage, a significant increase in detection rates can be realized while the model complexity remains comparable to the baseline.

High Dynamic Ranges (HDR) displays can show images with higher color contrast levels and peak luminosities than the commonly used Low Dynamic Range (LDR) displays. Although HDR displays are still expensive, they are reaching the consumer market in the coming years. Unfortunately, most video content is recorded and/or graded in LDR format. Typically, dynamic range expansion by using an Inverse Tone Mapped Operator (iTMO) is required to show LDR content in HDR displays. The most common type of artifact derived from dynamic range expansion is false contouring, which negatively affects the overall image quality. In this paper, we propose a new fast iterative false-contour removal method for inverse tone mapped HDR content. We consider the false-contour removal as a signal reconstruction problem, and we solve it using an iterative Projection Onto Convex Sets (POCS) minimization algorithm. Unlike most other false-contour removal techniques, we define reconstruction constraints taking into account the iTMO used. Experimental results demonstrate the effectiveness of the proposed method to remove false contours while preserving details in the image. In order speed-up the execution time, the proposed method was implemented to run on a GPU. We were able to show that it can be used to remove false contours in real-time from an inverse tone mapped High-definition HDR video sequences at 24 fps.

This letter introduces a new spectral-spatial classification method for hyperspectral images. A multiscale superpixel segmentation is first used to model the distribution of classes based on spatial information. In this context, the original hyperspectral image is integrated with segmentation maps via a feature fusion process in different scales such that the pixel-level data can be represented by multiscale superpixel-level (MSP) data sets. Then, a subspace-based support vector machine (SVMsub) is adopted to obtain the classification maps with multiscale inputs. Finally, the classification result is achieved via a decision fusion process. The resulting method, called MSP-SVMsub, makes use of the spatial and spectral coherences, and contributes to better feature characterization. Experimental results based on two real hyperspectral data sets indicate that the MSP-SVMsub exhibits good performance compared with other related methods.

Hyperspectral images are high dimensional while the available number of training samples can be very low. For very small training sizes, classical supervised classification strategies may fail. In this work we propose an alternative, semi-supervised approach which is based on sparse unmixing. In this method, all training samples are gathered in a dictionary and serve as possible endmembers. Unmixing then reveals the relative contributions of the different training samples to an unlabeled sample. Since standard unmixing strategies as the Fully Constrained Linear Spectral Unmixing (FCLSU) typically assume only one endmember per class, we investigate the use of sparse unmixing. In this work, we apply the SunSAL algorithm. We show that this method outperforms SVM classification in the case of extremely small training sizes of only a few samples per class.

Recent advances in airborne light detection and ranging (LiDAR) technology allow us to rapid measure the topographical information over large areas. LiDAR remote sensed data has been widely used in many applications, e.g. forest management, urban planning, disaster predictions, etc. However, extracting useful information from LiDAR data remains challenging, especially in the urban remote sensing, where many objects have the same elevation and are connected, such as road and parking lots, trees and buildings. In this work, we present a new method to extract geometric and textural information from LiDAR data by using attribute filters with partial reconstruction. The proposed method can separate the connected objects and better model the geometric and textural information than traditional connected filters (e.g. attribute filters). Experimental results on LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using original LiDAR data or attribute profiles computed by traditional attribute filters, with the proposed method, overall classification accuracies were improved by 35% and 12%, respectively.

Hyperspectral and LiDAR data, can provide plentiful information about the objects on the Earths surface. However there are some shortages for each of them, where hyperspectral sensor is easily influenced by cloud and difficult to distinguish different objects contained same materials, LiDAR cannot discriminate different objects which are similar in altitude. Fusion of these multi-source data for reliable classification attracts increasing interests but remains challenging. In this paper, we propose a new framework to fuse multi-source data for classification. The proposed method contains three main works: 1) cloud shadows extraction; 2) feature fusion of spectral and spatial information extracted from hyperspectral image, elevation information extracted from LiDAR data; 3) decision fusion of cloud and non-cloud regions. Experimental results on real HSI and LiDAR data demonstrate effectiveness of the proposed method both visually and quantitatively.

Intersection detection is a critical aspect for both route planning and path optimization. In literature, intersections are detected indirectly using the road users' turning behaviors. This paper proposes a novel approach to detect intersections directly using their definition of connecting road segments. We first detect the Longest Common Sub-Sequences (LCSS) between each pair of GPS traces using dynamic programming approach. Second, we partition the longest nonconsecutive subsequences into consecutive substrings. The starting and ending points of each common substring are connecting points where two GPS traces split to different directions after they share a series of common locations. At last, we estimate Kernel Density (KD) of the connecting points and find the local maximas on the density map as intersections. Experimental results show our proposed method outperforms the state-of-the-art work with a high accuracy for intersection detection.

In this paper we present a novel approach to quickly obtain detailed 3D reconstructions of large scale environments. The method is based on the consecutive registration of 3D point clouds generated by modern lidar scanners such as the Velodyne HDL-32e or HDL-64e. The main contribution of this work is that the proposed system specifically deals with the problem of sparsity and inhomogeneity of the point clouds typically produced by these scanners. More specifically, we combine the simplicity of the traditional iterative closest point (ICP) algorithm with the analysis of the underlying surface of each point in a local neighbourhood. The algorithm was evaluated on our own collected dataset captured with accurate ground truth. The experiments demonstrate that the system is producing highly detailed 3D maps at the speed of 10 sensor frames per second.

The most well known and widely used method for comparing two homogeneous color samples is the CIEDE2000 color difference formula because of its strong agreement with human perception. However, the formula is unreliable when applied over images and its spatial extensions have shown little improvement compared with the original formula. Hence, researchers have proposed many methods intending to measure color differences (CDs) in natural scene color images. However, these existing methods have not yet been rigorously compared. Therefore, in this work we review and evaluate CD measures with the purpose of answering the question to what extent do state-of-the-art CD measures agree with human perception of CDs in images? To answer the question, we have reviewed and evaluated eight state-of-the-art CD measures on a public image quality database. We found that the CIEDE2000, its spatial extension and the just noticeable CD measure perform well in computing CDs in images distorted by black level shift and color quantization algorithms (correlation higher than 0.8). However, none of the tested CD measures perform well on identifying CDs for the variety of color related distortions tested in this work, e.g., most of the tested CD measures showed a correlation lower than 0.65.

Dimensionality reduction (DR) is an important and helpful preprocessing step for hyperspectral image (HSI) classification. Recently, sparse graph embedding (SGE) has been widely used in the DR of HSIs. SGE explores the sparsity of the HSI data and can achieve good results. However, in most cases, locality is more important than sparsity when learning the features of the data. In this letter, we propose an extended SGE method: the weighted sparse graph based DR (WSGDR) method for HSIs. WSGDR explicitly encourages the sparse coding to be local and pays more attention to those training pixels that are more similar to the test pixel in representing the test pixel. Furthermore, WSGDR can offer data-adaptive neighborhoods, which results in the proposed method being more robust to noise. The proposed method was tested on two widely used HSI data sets, and the results suggest that WSGDR obtains sparser representation results. Furthermore, the experimental results also confirm the superiority of the proposed WSGDR method over the other state-of-the-art DR methods.

In this letter, we present an efficient image denoising method combining quadtree-based nonlocal means (NLM) and locally adaptive principal component analysis. It exploits nonlocal multiscale self-similarity better, by creating sub-patches of different sizes using quadtree decomposition on each patch. To achieve spatially uniform denoising, we propose a local noise variance estimator combined with denoiser based on locally adaptive principal component analysis. Experimental results demonstrate that our proposed method achieves very competitive denoising performance compared with state-of-the-art denoising methods, even obtaining better visual perception at high noise levels.

Extended attribute profiles (EAPs) have been widely used for the classification of high-resolution hyperspectral images. EAPs are obtained by computing a sequence of attribute operators. Attribute filters (AFs) are connected operators, so they can modify an image by only merging its flat zones. These filters are effective when dealing with very high resolution images since they preserve the geometrical characteristics of the regions that are not removed from the image. However, AFs, being connected filters, suffer the problem of “leakage” (i.e., regions related to different structures in the image that happen to be connected by spurious links will be considered as a single object). Objects expected to disappear at a certain threshold remain present when they are connected with other objects in the image. The attributes of small objects will be mixed with their larger connected objects. In this paper, we propose a novel framework for morphological AFs with partial reconstruction and extend it to the classification of high-resolution hyperspectral images. The ultimate goal of the proposed framework is to be able to extract spatial features which better model the attributes of different objects in the remote sensed imagery, which enables better performances on classification. An important characteristic of the presented approach is that it is very robust to the ranges of rescaled principal components, as well as the selection of attribute values. Our experimental results, conducted using a variety of hyperspectral images, indicate that the proposed framework for AFs with partial reconstruction provides state-of-the-art classification results. Compared to the methods using only single EAP and stacking all EAPs computed by existing attribute opening and closing together, the proposed framework benefits significant improvements in overall classification accuracy.

This paper proposes a generic methodology for semi-automatic generation of reliable position annotations for evaluating multi-camera people-trackers on large video datasets. Most of the annotation data is computed automatically, by estimating a consensus tracking result from multiple existing trackers and people detectors and classifying it as either reliable or not. A small subset of the data, composed of tracks with insufficient reliability is verified by a human using a simple binary decision task, a process faster than marking the correct person position. The proposed framework is generic and can handle additional trackers. We present results on a dataset of approximately 6 hours captured by 4 cameras, featuring a person in a holiday flat, performing activities such as walking, cooking, eating, cleaning, and watching TV. When aiming for a tracking accuracy of 60cm, 80% of all video frames are automatically annotated. The annotations for the remaining 20% of the frames were added after human verification of an automatically selected subset of data. This involved about 2.4 hours of manual labour. According to a subsequent comprehensive visual inspection to judge the annotation procedure, we found 99% of the automatically annotated frames to be correct. We provide guidelines on how to apply the proposed methodology to new datasets. We also provide an exploratory study for the multi-target case, applied on existing and new benchmark video sequences.

Multi-camera triangulation of feature points based on a minimisation of the overall ℓ2reprojection error can get stuck in suboptimal local minima or require slow global optimisation. For this reason, researchers have proposed optimising the ℓ∞norm of the ℓ2single view reprojection errors, which avoids the problem of local minima entirely. In this paper we present a novel method for ℓ∞triangulation that minimizes the ℓ∞norm of the ℓ∞reprojection errors: this apparently small difference leads to a much faster but equally accurate solution which is related to the MLE under the assumption of uniform noise. The proposed method adopts a new optimisation strategy based on solving simple quadratic equations. This stands in contrast with the fastest existing methods, which solve a sequence of more complex auxiliary Linear Programming or Second Order Cone Problems. The proposed algorithm performs well: for triangulation, it achieves the same accuracy as existing techniques while executing faster and being straightforward to implement.

In two-view stereo matching, the disparity of occluded pixels cannot accurately be estimated directly: it needs to be inferred through, e.g., regularisation. When capturing scenes using a plenoptic camera or a camera dolly on a track, more than two input images are available, and - contrary to the two-view case - pixels in the central view will only very rarely be occluded in all of the other views. By explicitly handling occlusions, we can limit the depth estimation of pixel p to only use those cameras that actually observe p. We do this by extending variational stereo matching to multiple views, and by explicitly handling occlusion on a view-by-view basis. Resulting depth maps are illustrated to be sharper and less noisy than typical recent techniques working on light fields.

Non-local self-similarity based image denoising depends strictly on similarity measure. The denoising performance is determined based on the ability to reliably find sufficient number of similar patches. In this paper, we propose a rotation invariant similarity measure to fully exploit the image non-local self-similarity. Instead of using image patches, we employ local frequency descriptors, that are rotation invariant and robust to noise, to measure the similarity. Thus, both translational and rotational similarity can be handled even at high noise level. The comparative experimental results show that the proposed method is effective as a rotation invariant similarity measure, and it can consistently improve the performance of non-local means algorithm to achieve better denoising results.

This paper proposes a semi-supervised graph-based fusion framework to couple dimensionality reduction and the fusion of multi-sensor data for classification. First, morphological features are used to model the elevation and spatial information contained in both LiDAR data and on the first few principal components (PCs) of the original hyperspectral (HS) image. Then, we fuse the features by projecting the spectral, spatial and elevation features onto a lower subspace through our proposed semi-supervised fusion graph. Experimental results on fusion of HS and LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using single data source or unsupervised graph fusion, with the proposed method, overall classification accuracies were improved by 9% and 4%, respectively.

Nowadays, we have diverse sensor technologies and image processing algorithms that allow one to measure different aspects of objects on the Earth [e.g., spectral characteristics in hyperspectral images (HSIs), height in light detection and ranging (LiDAR) data, and geometry in image processing technologies, such as morphological profiles (MPs)]. It is clear that no single technology can be sufficient for a reliable classification, but combining many of them can lead to problems such as the curse of dimensionality, excessive computation time, and so on. Applying feature reduction techniques on all the features together is not good either, because it does not take into account the differences in structure of the feature spaces. Decision fusion, on the other hand, has difficulties with modeling correlations between the different data sources. In this letter, we propose a generalized graph-based fusion method to couple dimension reduction and feature fusion of the spectral information (of the original HSI) and MPs (built on both HS and LiDAR data). In the proposed method, the edges of the fusion graph are weighted by the distance between the stacked feature points. This yields a clear improvement over an older approach with binary edges in the fusion graph. Experimental results on real HSI and LiDAR data demonstrate effectiveness of the proposed method both visually and quantitatively.

In this letter, we propose a novel approach for improving Random Forest (RF) in hyperspectral image classification. The proposed approach combines the ensemble of features and the semisupervised feature extraction (SSFE) technique. The main contribution of our approach is to construct an ensemble of RF classifiers. In this way, the feature space is divided into several disjoint feature subspaces. Then, the feature subspaces induced by the SSFE technique are used as the input space to an RF classifier. This method is compared with a regular RF and an RF with the reduced features by the SSFE on two real hyperspectral data sets, showing an improved performance in ill-posed, poor-posed, and well-posed conditions. An additional study shows that the proposed method is less sensitive to the parameters.

The current multi-camera systems have not studied the problem of person tracking under low resolution constraints. In this paper, we propose a low resolution sensor network for person tracking. The network is composed of cameras with a resolution of 30×30 pixels. The multi-camera system is used to evaluate probability occupancy mapping and maximum likelihood trackers against ground truth collected by ultra-wideband (UWB) testbed. Performance evaluation is performed on two video sequences of 30 minutes. The experimental results show that maximum likelihood estimation based tracker outperforms the state-of-the-art on low resolution cameras.

In image and video processing research, rapid prototyping and testing of different variations of an algorithm is quite essential (e.g., to find out if a given algorithm can solve a given problem or work in real-time). In the past decade, the computational performance of graphical processing units (GPUs) has improved significantly, where speed-up factors of 10×-50× compared to single-threaded CPU execution are not uncommon. However, GPU programming is challenging, requiring a significant programming expertise and moreover, most existing programming approaches are not well suited for rapid prototyping. In this Show & Tell session, we present a new programming framework, aimed at making the bridge between high-level program specification and low-level implementation and optimization on heterogeneous computation devices. The goal is that the programmer is relieved from (most) implementation issues and can focus on the specification and improvement of the algorithms. We present a new prototype domain-specific programming language (in the first place aimed at image and video processing) that provides a uniform programming approach toward different hardware devices, a run-time environment to manage and communicate with the heterogeneous devices and an integrated development environment (IDE). The IDE provides various useful image processing-related debugging and profiling features.

This paper proposes a method to combine feature fusion and decision fusion together for multi-sensor data classification. First, morphological features which contain elevation and spatial information, are generated on both LiDAR data and the first few principal components (PCs) of original hyper-spectral (HS) image. We got the fused features by projecting the spectral (original HS image), spatial and elevation features onto a lower subspace through a graph-based feature fusion method. Then, we got four classification maps by using spectral features, spatial features, elevation features and the graph fused features individually as input of SVM classifier. The final classification map was obtained by fusing the four classification maps through the weighted majority voting. Experimental results on fusion of HS and LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using single data source or only feature fusion, with the proposed method, overall classification accuracies were improved by 10% and 2%, respectively.

Real-time tracking of people has many applications in computer vision, especially in the domain of surveillance. Typically, a network of cameras is used to solve this task. However, real-time tracking remains challenging due to frequent occlusions and environmental changes. Besides, multicamera applications often require a trade-off between accuracy and communication load within a camera network. In this article, we present a real-time distributed multicamera tracking system for the analysis of people in a meeting room. One contribution of the article is that we provide a scalable solution using smart cameras. The system is scalable because it requires a very small communication bandwidth and only light-weight processing on a “fusion center” which produces final tracking results. The fusion center can thus be cheap and can be duplicated to increase reliability. In the proposed decentralized system all low level video processing is performed on smart cameras. The smart cameras transmit a compact high-level description of moving people to the fusion center, which fuses this data using a Bayesian approach. A second contribution in our system is that the camera-based processing takes feedback from the fusion center about the most recent locations and motion states of tracked people into account. Based on this feedback and background subtraction results, the smart cameras generate a best hypothesis for each person. We evaluate the performance (in terms of precision and accuracy) of the tracker in indoor and meeting scenarios where individuals are often occluded by other people and/or furniture. Experimental results are presented based on the tracking of up to 4 people in a meeting room of 9 m by 5 m using 6 cameras. In about two hours of data, our method has only 0.3 losses per minute and can typically measure the position with an accuracy of 21 cm. We compare our approach to state-of-the-art methods and show that our system performs at least as good as other methods. However, our system is capable to run in real-time and therefore produces instantaneous results.

Tracking persons with multiple cameras with overlapping fields of view instead of with one camera leads to more robust decisions. However, operating multiple cameras instead of one requires more processing power and communication bandwidth, which are limited resources in practical networks. When the fields of view of different cameras overlap, not all cameras are equally needed for localizing a tracking target. When only a selected set of cameras do processing and transmit data to track the target, a substantial saving of resources is achieved. The recent introduction of smart cameras with on-board image processing and communication hardware makes such a distributed implementation of tracking feasible. We present a novel framework for selecting cameras to track people in a distributed smart camera network that is based on generalized information-theory. By quantifying the contribution of one or more cameras to the tracking task, the limited network resources can be allocated appropriately, such that the best possible tracking performance is achieved. With the proposed method, we dynamically assign a subset of all available cameras to each target and track it in difficult circumstances of occlusions and limited fields of view with the same accuracy as when using all cameras.

In this paper we give new insights into quantitative microwave tomography with robust Huber regularizer and Gauss-Newton optimization. Firstly, we validate this approach for the first time on real electromagnetic measurements. Secondly, we extend the framework with a modified Huber function, which behaves like TV regularization. This is interesting for reconstructing piece-wise constant permittivities that appear in non-destructive testing of installations and other man-made objects.

Wavelet-based demosaicing techniques have the advantage of being computationally relatively fast, while having a reconstruction performance that is similar to state-of-the-art techniques. Because the demosaicing rules are linear, it is fairly simple to integrate denoising into the demosaicing. In this paper, we present a method that performs joint denoising and demosaicing, using a Gaussian Scale Mixture (GSM) prior model, thereby modeling the local edge direction as a hidden variable. The results indicate that this technique offers a better reconstruction performance (in PSNR sense and visually) than sequential demosaicing and denoising. On a recent GPU, our algorithm takes 3.5 s for reconstructing a 12 megapixel RAW digital camera image.

This paper presents a two-stage denoising method for hyper-spectral image (HSI) by combining kernel principal component analysis (KPCA) and total variation (TV). In the first stage, we use KPCA denoising to reduce spectrally uncorre-lated noise. In the second stage, the information content is largely separated from the remaining noise by means of principal component analysis (PCA). The remaining noise is then efficiently removed by fast primal-dual TV denoising in low-energy PCA channels. Experimental results on simulated and real HSIs are very encouraging.

This paper proposes a complete tracking system that is capable of long-term, real-time hand tracking with unsupervised initialization and error recovery. Initialization is steered by a three-stage hand detector, combining spatial and temporal information. Hand hypotheses are generated by a random forest detector in the first stage, whereas a simple linear classifier eliminates false positive detections. Resulting detections are tracked by particle filters that gather temporal statistics in order to make a final decision. The detector is scale and rotation invariant, and can detect hands in any pose in unconstrained environments. The resulting discriminative confidence map is combined with a generative particle filter based observation model to enable robust, long-term hand tracking in real-time. The proposed solution is evaluated using several challenging, publicly available datasets, and is shown to clearly outperform other state of the art object tracking methods.

Optical flow can greatly improve the robustness of visual tracking algorithms. While dense optical flow algorithms have various applications, they can not be used for real-time solutions without resorting to GPU calculations. Furthermore, most optical flow algorithms fail in challenging lighting environments due to the violation of the brightness constraint. We propose a simple but effective iterative regularisation scheme for real-time, sparse optical flow algorithms, that is shown to be robust to sudden illumination changes and can handle large displacements. The algorithm proves to outperform well known techniques in real life video sequences, while being much faster to calculate. Our solution increases the robustness of a real-time particle filter based tracking application, consuming only a fraction of the available CPU power. Furthermore, a new and realistic optical flow dataset with annotated ground truth is created and made freely available for research purposes.

In this letter, we present a new FCM-based method for spatially coherent and noise-robust image segmentation. Our contribution is twofold: 1) the spatial information of local image features is integrated into both the similarity measure and the membership function to compensate for the effect of noise; and 2) an anisotropic neighborhood, based on phase congruency features, is introduced to allow more accurate segmentation without image smoothing. The segmentation results, for both synthetic and real images, demonstrate that our method efficiently preserves the homogeneity of the regions and is more robust to noise than related FCM-based methods.

We propose a novel semisupervised local discriminant analysis method for feature extraction in hyperspectral remote sensing imagery, with improved performance in both ill-posed and poor-posed conditions. The proposed method combines unsupervised methods (local linear feature extraction methods and supervised method (linear discriminant analysis) in a novel framework without any free parameters. The underlying idea is to design an optimal projection matrix, which preserves the local neighborhood information inferred from unlabeled samples, while simultaneously maximizing the class discrimination of the data inferred from the labeled samples. Experimental results on four real hyperspectral images demonstrate that the proposed method compares favorably with conventional feature extraction methods.

In the field of image denoising, the non-local means (NLMS) filter is a conceptually simple, yet powerful technique. This filter exploits non-local, i.e. spatially repetitive, structure in natural images to estimate noise-free structure. In contrast, a wide variety of image restoration problems have been solved exploiting local smoothness of natural images, e.g. by enforcing sparsity of images when subjected to a multi-resolution transform. In this paper we introduce the prior knowledge of non-local repetitiveness of image structures into a broad multi-resolution image restoration framework. The proposed framework allows the power of the NLMS filter, supplemented by multi-resolution sparsity, to be extended for a wide variety of image restoration problems, such as demosaicing, deconvolution, reconstruction from insufficient measurements,... in a conceptually simple way.

In statistical theory, the Huber function yields robust estimations reducing the effect of outliers. In this paper, we employ the Huber function as regularization in a challenging inverse problem: quantitative microwave imaging. Quantitative microwave tomography aims at estimating the permittivity profile of a scattering object based on measured scattered fields, which is a nonlinear, ill-posed inverse problem. The results on 3D data sets are encouraging: the reconstruction error is reduced and the permittivity profile can be estimated from fewer measurements compared to state-of-the art inversion procedures.

Nowadays, an important problem in multi-camera systems is how to select the camera with the best frontal view of a person in order to visualize to an observer. Therefore, we present a minimum score based criterion for best view selection, based on face recognition with geometric features. In this approach, faces are represented with Curve Edge Maps (CEMs), which are collections of polynomial curves with a convex region. Face recognition is performed by matching face CEMs driven by histograms of intensities and histograms of relative positions. The resulting face recognition scores are employed as quality-of-view measures. They indicate whether or not persons are seen by cameras in frontal view. Experiments show that the method is robust and efficient when selecting the best view in a multi-camera system. Furthermore, our method outperforms view selection based on face detection only.

Intensity based image registration methods are necessary in situations where feature point based methods fail, e.g multispectral images or images with repetitive textures. However the higher robustness comes at the cost of being slower than the feature point alternative. This problem can be alleviated by subsampling the pixels used to calculate the intensity matching criterion. Unfortunately there are no guidelines for the amount of allowed subsampling. In this paper we present a model for a lower bound on the number of pixel samples in image sets to keep a certain confidence about the correctness of the result, using the variance on the estimated mutual information value. With this model we can speed up the intensity based registration while remaining confident about getting correct results.

In this paper we present an efficient way to both compute and extract salient information from trace transform signatures to perform object identification tasks. We also present a feature selection analysis of the classical trace-transform functionals, which reveals that most of them retrieve redundant information causing misleading similarity measurements. In order to overcome this problem, we propose a set of functionals based on Laguerre polynomials that return orthonormal signatures between these functionals. In this way, each signature provides salient and non-correlated information that contributes to the description of an image object. The proposed functionals were tested considering a vehicle identification problem, outperforming the classical trace transform functionals in terms of computational complexity and identification rate.

In this paper, we present a first-order primal-dual algorithm for tackling the joint demosaicking and deconvolution problem. The proposed algorithm exploits the sparsity of both discrete gradient (TV) and shearlet coefficients as prior knowledge. In order to deal with this sparsity across the color channels, we first decorrelate the signals in color space before sparsifying them spatially, resulting in a separable transform. We demonstrate that this approach yields better results than employing group sparsity strategies. We propose to update the decorrelation operator during the image reconstruction, this approach will result in a significant improvement in PSNR. By relaxing the sparsity of the chrominance signals, we obtain both better objective and subjective image quality compared to other state-of-the-art demosaicking and deconvolution algorithms. Also, color artifacts due to demosaicking are suppressed very well.

In this paper, we propose a novel global Markov Random Field based image inpainting method with context-aware label selection. Context is determined based on the texture and color features in fixed image regions and is used to distinguish areas of similar content to which the search for candidate patches is limited. Furthermore, we introduce a novel optimization approach, as an alternative to priority belief propagation framework, which further reduces the number of candidates and performs efficient inference to obtain final inpainting result. Experimental results show improvement over related state-of-the-art methods. Moreover, global optimization is significantly accelerated with the proposed inference approach.

This paper proposes a robust real-time hand tracking approach by combining a discriminative random forest classifier with generative color based cues using a particle filter. The proposed detector is scale and rotation invariant and is able to overcome ambiguities and local maxima in the color based likelihood function in real-time. A new hand tracking dataset with manually annotated groundtruths is created and made freely available for research purposes. Thorough evaluation shows the robustness and advantages of our proposal compared to other state of the art object tracking methods.

Particle filters are often used for tracking objects within a scene. As the prediction model of a particle filter is often implemented using basic movement predictions such as random walk, constant velocity or acceleration, these models will usually be incorrect. Therefore, this paper proposes a new approach, based on a Canonical Correlation Analysis (CCA) tracking method which provides an object specific motion model. This model is used to construct a proposal distribution of the prediction model which predicts new states, increasing the robustness of the particle filter. Results confirm an increase in accuracy compared to state-of-the-art methods.

In this paper, we present a new method for High Dynamic Range (HDR) reconstruction based on a set of multiple photographs with different exposure times. While most existing techniques take a deterministic approach by assuming that the acquired low dynamic range (LDR) images are noise-free, we explicitly model the photon arrival process by assuming sensor data corrupted by Poisson noise. Taking the noise characteristics of the sensor data into account leads to a more robust way to estimate the non-parametric camera response function (CRF) compared to existing techniques. To further improve the HDR reconstruction, we adopt the split-Bregman framework and use Total Variation for regularization. Experimental results on real camera images and ground-truth data show the effectiveness of the proposed approach.

The topic of registering multi- and hyperspectral imagery is well studied in literature. However when the registration must be done between multispectral images and vector data, the literature is more limited. In this paper we focus on registering aerial images in the thermal (IR) band, and vector data delineating houses and other man-made structures in the same region. This differs from classical registration because first of all, the vector data is not in the same format as the thermal image (i.e. vectors versus pixels), although it can be rasterized initially. Secondly, feature points fail to match between the two representations while intensity based methods also have problems due to the large changes between the rasterized vector and thermal images. We present a method based on a modified version of mutual information that outperforms existing methods for these specific inputs.

In the remote sensing domain data from many different sources are often available. Each of these data sources are characterized by their own sensor- and platform-specific properties, i.e. spectral range, or spatial and spectral resolution. In this paper we consider a low spatial, but high spectral resolution satellite image, together with its high spatial resolution RGB color image, e.g. obtained by UAV. Spatial features are extracted from the color image by combining the three color bands R, G and B, ordering these color vectors, and presenting color mathematical morphological profiles accordingly. This way the spatial information contained in the correlation between the different bands is completely taken into account and thus also totally preserved in the feature extraction. In a classification experiment these color morphological profiles are combined with the spectral features of the hyperspectral image, and we show that the spatial characterization of the color image is improved.

GPS by itself is not dependable in urban environments, due to signal reception issues such as multi-path effects or occlusion. Other sensor data is required to keep track of the vehicle in absence of a reliable GPS signal. We propose a new method to use a single on-board consumer-grade camera for vehicle motion estimation. The method is based on the tracking of ground plane features, taking into account the uncertainty on their backprojection as well as the uncertainty on the vehicle motion. A Hough-like parameter space vote is employed to extract motion parameters from the uncertainty models. The method is easy to calibrate and designed to be robust to outliers and bad feature quality. Experimental results show good accuracy and high reliability, with a positional estimate within 2 metres for a 400 metre elapsed distance.

Parallel Imaging MRI (pMRI) and Compressive Sensing (CS) are two reconstruction techniques that have recently been applied to increase MRI performance. In this paper we demonstrate that a combined analysis of the pMRI and CS problems leads to a conceptually simple, yet effective technique that outperforms independent approaches to both reconstruction problems. We argue that the proposed technique is also naturally resilient to noise, due to its relation to the MAP image denoising formulation. A modified Basis Pursuit (BP) formulation of the CS-MRI problem allows it to handle the pMRI problem at the same time. We also present an exact solution to this BP problem, using the split Bregman technique, with discrete shearlet transform (DST) regularization. The DST is an excellent choice for natural image applications, due to its optimal sparsity property. Results show that this Compressive Parallel Sensing (COMPASS) reconstruction algorithm outperforms more traditional MRI reconstruction algorithms in both pMRI and CS experiments.

Image processing methods have gained wide acceptance in the field of experimental mechanics for the measurement of full field displacements and strains in materials that undergo mechanical stress. Over the years various block based methods have been developed focusing on pixel and sub-pixel accurate displacement estimation, using as input data speckle images of the material before and after the deformation process. As high resolution imaging systems become more affordable, the limitations of block-based approaches become apparent because of their inherent inability to create high density motion fields while keeping the errors in the calculated motion vectors to a minimum due to the aperture problem. In this paper a novel low-level partitioning method is presented which adaptively divides the image into cells, according to the underlying speckle structure with the purpose of minimizing motion estimation errors. The method starts by determining relevant 'central' areas for speckles over a certain minimum size and subsequently builds the cells around these areas according to spatial distances and image structure. The results show clear improvements in motion accuracy compared to the regular block approach and provide a first step in the accurate isolation of various features in the images such as holes and cracks.

Fuzzy clustering techniques have been widely used in automated image segmentation. However, since the standard fuzzy c-means (FCM) clustering algorithm does not consider any spatial information, it is highly sensitive to noise. In this paper, we present an extension of the FCM algorithm to overcome this drawback, by incorporating spatial neighborhood information into a new similarity measure. We consider that spatial information depends on the relative location and features of the neighboring pixels. The performance of the proposed algorithm is tested on synthetic and real images with different noise levels. Experimental quantitative and qualitative segmentation results show that the proposed method is effective, more robust to noise and preserves the homogeneity of the regions better than other FCM-based methods.

This work presents a novel parallel technique to implement stack morphological filters for image processing. The method relies on applying the image bitwise decomposition to manipulate the grayscale image at a bit-plane level, while simple logical operations and Positive Boolean Functions (PBF's) are executed in parallel to derive the transformed bit-planes. The relationship between the bitwise and threshold decomposition is closely investigated and analysed, which lead us to derive an algorithm whose control flow is full binary encoded. Furthermore, the algorithm exhibits an interesting performance, which depends on the image histogram thanks to its hierarchical processing and the study of the relationship among binary decompositions.

A fast iterative Kernel Principal Component Analysis (KPCA) is proposed to extract features from hyperspectral images. The proposed method is a kernel version of the Candid Covariance-Free Incremental Principal Component Analysis, which solves the eigenvectors through iteration. Without performing eigen decomposition on Gram matrix, our method can reduce the space complexity and time complexity greatly. Experimental results were validated in comparison with the standard KPCA and linear version methods.

In this paper, we derive a novel robust image alignment technique that performs joint geometric and photometric registration in the total least square sense. The main idea is to use the total least square metrics instead of the ordinary least square metrics, which is commonly used in the literature. While the OLS model indicates that the target image may contain noise and the reference image should be noise-free, this puts a severe limitation on practical registration problems. By introducing the TLS model, which allows perturbations in both images, we can obtain mutually consistent parameters. Experimental results show that our method is indeed much more consistent and accurate in presence of noise compared to existing registration algorithms.

In this paper, a new wavelet based image fusion method is proposed. In this method, the blur levels of the edge points are estimated for every slice in the stack of images. Then from corresponding edge points in different slices, the sharpest one is brought to the final image and others are eliminated. The intensities of non-edge pixels are assigned by the slice of its nearest neighbor edge. Results are promising and outperform other methods in most cases of the tested methods.

Active contours or snakes are widely used for segmentation and tracking. We propose a new active contour model, which converges reliably even when the initialization is far from the object of interest. The proposed segmentation technique uses an external energy function where the energy slowly decreases in the vicinity of an edge. This new energy function is calculated using an efficient dual scan line algorithm. The proposed energy function is tested on computational speed, its effect on the convergence speed of the active contour and the segmentation result. The proposed method gets similar segmentation results as the gradient vector flow active contours, but the energy function needs much less time to calculate.

Our surroundings change all the time. Applications that require 3D models of a changing terrain, such as urban planning, are becoming ever more demanding with respect to the cost to create them and the accuracy of the result. A novel, cheap and fast solution for this problem is given by a UAV to take aerial images of the terrain in question, in combination with structure from motion algorithms to create a 3D model from those aerial images. However the question remains whether these on-the-fly 3D maps can match the accuracy of classical surveyor based models, which require more time to create. In this paper we investigate this question, and find that under certain conditions the accuracy of the UAV based model matches the accuracy of surveyor generated measurements.

In this paper we present a more efficient approach to the problem of finding the connected components in binary images. In conventional connected components algorithms, the main data structure to compute and store the connected components is the region label image. We replace the region label image with a singly-linked list of line segments (or runs) for each region. This enables us to design a very fast and memory efficient connected components algorithm. Most conventional algorithms require (at least) two raster scans. Those that only need one raster scan, require irregular and unbounded image access. The proposed algorithm is a single pass regular access algorithm and only requires access to the three most recently processed image lines at any given time. Experimental results demonstrate that our algorithm is considerably faster than the fastest conventional algorithm. Additionally, our novel region coding data structure uses much less memory in typical cases than the traditional region label image. Even in worst case situations the processing time of our algorithm is linear with the number of pixels in an image.

In the deep sub-micron domain wires consume more power than transistors. Power Gating for Wires is a form of bus segmentation that alleviates the power loss from on-chip interconnects, by switching off the supply voltage from inactive drivers, cycle by instruction-cycle. The success of Power Gating for Wires depends much on control: the gain from segmentation can conceivably be undone by control costs. Yet during design exploration, the data required for statistical analysis are not available. A theory of efficient control for Power Gating for Wires and a design framework, determining the balance of cost factors, at an early stage, are both needed. In this paper, we formulate a theory of Useful State Analysis to obtain minimal-redundancy encoding of control information. We establish two figures of merit, based on network topology: Intrinsic Sectioning Gain and Useful Encoding Efficiency. They quantify the power loss reduction achievable, and the success of Useful State Analysis in keeping control costs low. We propose a design pattern for the operation of a control plane, wherein the costs of control can be identified. From use cases, we find that architectures can have an Intrinsic Sectioning Gain of 50% and more. Useful Encoding Efficiency is found to be in a range of 44-80% for some common multipath architectures. Although ultimately, the limits of feasibility to control Power Gating for Wires must be decided by means of statistical analysis, we find Useful State Analysis is applicable to networks with tens of terminals, and that our method of control scales well with increasing network size and complexity.

The embryonic vertebrate heart starts pumping long before the formation of valves and chambers. Zebrafish embryonic hearts are morphologically comparable to human embryonic hearts in early stage. Since they are optically transparent, they are of obvious scientific use to study the beating heart. This paper proposes a fast and reliable algorithm based on tracking of linked geometric features to derive the motion of the heart walls and the velocity in the fluid flow. In contrast to active contour methods, we model the heart walls with parabola segments. This is both simpler and accurate enough to model heart walls. An original contribution in this paper is that we can track the heart walls and the blood cells using the same method and provide consistent estimates of both. Moreover, due to the pumping mechanism of the heart, the tracking of the heart walls is used to determine the region of interest for the tracking of the blood cells. This relation decreases the false matches of blood cells by more than 50%, which results in a much improved estimation of the velocity in the fluid flow.

The dual-tree complex wavelet transform (DT-CWT) uses approximate Hilbert transform pairs of wavelet, which requires that the filters of each tree of the dual-tree structure should be delayed approximately one half sample from each other. However, the filters of the first (finest) scale of the transform do not obey this condition, resulting in a poor directional selectivity for the first scale. In this paper, we describe a design technique for first-scale infinite impulse response (IIR) wavelet filters, that solves this problem. Results demonstrate that a much better directional selectivity is obtained, which indicates a performance improvement for many applications that use the DT-CWT where the preservation of high-frequency information is important.

In high dynamic range (HDR) imaging, multiple photographs with different exposure times are combined into a radiance map, which reflects the radiance in real-life scenes. This involves recovering the response function of the imaging process. The technique proposed by Debevec and Malik is a well-known HDR image synthesis algorithm, but the computational complexity is relatively high, which limits the possible image size and the reconstruction quality. In this paper we present an improved joint optimization technique for estimating the camera response function (CRF) and the radiance map and a new sequential two-step optimization technique, which first estimates the CRF and then reconstructs the radiance map, resulting in better visual results and a higher SNR in remarkably less computation time.

Current clinical practice is increasingly moving in the direction of volumetric imaging. However, model observers for 3D images have been little explored so far. This study is investigating the task of detecting 2D signals in multi-slice simulated image data. We propose a novel design of a multi-slice model observer. To evaluate it, we compare three different model designs of the channelized Hotelling observer (CHO), two multi-slice and one single-slice model. The multi-slice models are built as a sequence of a 2D CHO and 1D HO, where the CHO is used to calculate a vector of metrics for each slice in the planar view and the HO is used to calculate the final scalar statistic of the model. The single-slice model is a 2D CHO applied on the location of the lesion. Our results show that the multi-slice models outperform the single-slice one, and here the new model surpasses the existing one.

Fourier descriptors (FD's) are widely used shape descriptors. By first warping the scanning speed of the contour before calculating the FD's, the shape approximation can be improved. This approach has never been properly tested on real applications. In this paper we compare these new shape descriptors to the regular FD's. A database of over 400 leaf shapes is approximated using these shape descriptors and compared to the approximation based on FD's. The error of the approximation was measured with the Hausdorff distance and modified Hausdorff distance, resulting for both criteria in an average improvement of more than 10% over approximation based on FD's.

Due to weight constraints, UAVs often carry cameras with lenses that create distortions in the image. For practical applications this distortion should be removed with a proper calibration procedure, without spending too much extra time. Existing methods require costly manual interaction when the grid is not fully visible, or when not all points can be extracted. In this paper we present an algorithm to perform the calibration without any user interaction whatsoever, which works under almost all possible conditions. The only inputs are a number of pictures of a checkerboard, taken with the camera. We extract the corners from the chessboard pictures, and set up a world coordinate grid that is robust to missing corner points, occlusion and deformations. We automatically omit the pictures that are too close to another picture, to avoid giving too much weight to often viewed areas. Finally we optimize the result by iteratively removing outlier pictures from the set.

Integrating video coding and denoising is a novel processing paradigm, bringing mutual benefits to both video processing tools. In this paper, we propose a novel video denoising approach of which the main idea is reusing motion estimation resources from the video coding module for video denoising. In most cases, the motion fields produced by real-time video codecs cannot be directly employed in video denoising, since they, as opposed to noise filters, tolerate errors in the motion field. In order to solve this problem, we propose a novel motion-field filtering step that refines the accuracy of the motion estimates to a degree that is required for denoising. Additionally, a novel temporal filter is proposed that is robust against errors in the estimated motion field. Numerical results demonstrate that the proposed denoising scheme is of low-complexity and compares favorably to the state-of-the-art video denoising methods.

Images, captured with digital imaging devices, often contain noise. In literature, many algorithms exist for the removal of white uncorrelated noise, but they usually fail when applied to images with correlated noise. In this paper, we design a new denoising method for the removal of correlated noise, by modeling the significance of the noise-free wavelet coefficients in a local window using a new significance measure that defines the "signal of interest" and that is applicable to correlated noise. We combine the intrascale model with a hidden Markov tree model to capture the interscale dependencies between the wavelet coefficients. We propose a denoising method based on the combined model and a less redundant wavelet transform. We present results that show that the new method performs as well as the state-of-the-art wavelet-based methods, while having a lower computational complexity.

We propose a new statistical model for image restoration in which neighborhoods of wavelet subbands are modeled by a discrete mixture of linear projected Gaussian Scale Mixtures (MPGSM). In each projection, a lower dimensional approximation of the local neighborhood is obtained, thereby modeling the strongest correlations in that neighborhood. The model is a generalization of the recently developed Mixture of GSM (MGSM) model, that offers a significant improvement both in PSNR and visually compared to the current state-of-the-art wavelet techniques. However, the computation cost is very high which hampers its use for practical purposes. We present a fast EM algorithm that takes advantage of the projection bases to speed up the algorithm. The results show that, when projecting on a fixed data-independent basis, even computational advantages with a limited loss of PSNR can be obtained with respect to the BLS-GSM denoising method, while data-dependent bases of Principle Components offer a higher denoising performance, both visually and in PSNR compared to the current wavelet-based state-of-the-art denoising methods.

In video communication over lossy packet networks (e.g., the Internet), packet loss errors can severely damage the transmitted video. The damaged video can largely be repaired with passive error concealment, where neighboring information is used to estimate missing information. We address the problem of passive error concealment for wavelet coded data with dispersive packetization. The reported techniques of this kind have many problems and usually fail in the reconstruction of high-frequency content. This paper presents a novel locally adaptive error concealment method for subband coded I-frames based on an inhomogeneous Gaussian Markov random field model. We estimate the parameters of this model from a local context of each lost coefficient, and we interpolate the lost coefficients accordingly. The results demonstrate a significant improvement over the reported related methods both in terms of objective performance measures and visually. The biggest improvement of the proposed method compared to the state-of-the-art in the field is the correct reconstruction of high-frequency information such as textures and edges.

Digital image processing techniques are becoming a popular way of determining strains and full-field displacements in experimental mechanics due to advancements in image processing techniques and also because of the non intrusive way in which these measurements are done compared to traditional sensor based methods. This paper presents a polar component filtering technique for the image displacement fields in which two filters are used: a Kalman filter for temporal smoothing of the motion vector angles and a subsequent adaptive spatial filter for filtering both previously processed angles and amplitudes of the vectors.

In image denoising applications, noise is often correlated and the noise energy and correlation structure may even vary with the position in the image. Existing noise reduction and estimation methods are usually designed for stationary white Gaussian noise and generally work less efficient in this case because of the noise model mismatch. In this paper, we propose an EM algorithm for the estimation of spatially variant (nonstationary) correlated image noise in the wavelet domain. In particular, we study additive white Gaussian noise filtered by a space-variant linear filter. This general noise model is applicable to a wide variety of practical situations, including noise in Computed Tomography (CT). Results demonstrate the effectiveness of the proposed solution and its robustness to signal structures.

In this paper we present a novel method for detection of individual C.Elegans worms in population images in presence of overlapping. First, in a pre-processing phase the worms skeletons are obtained by morphological skeleton operation after image binarization and filling small holes. Then, after pruning the small branches of the skeletons, the skeletons are splited into several branches from the pixels with more than two neighbors. Angle of each branch side is calculated in the next stage and the neighbor branches with angle difference less than a predefined threshold are merged. Finally, a simple post-processing based on stastical analysis of worms' length on their widths is used in order to increase the automatic efficiency of the method. We have applied our method to a database of 147 isolated and overlapped worms and obtained 81.43% accuracy.

Very high spatial resolution satellite images allow to identify individual man-made objects. However, automatic extraction of these objects is still very difficult, especially in urban areas. Spectral information is insufficient to separate the different man-made object classes. Therefore, there is an increased interest in incorporating shape and contextual information in the classification process. Object-based approaches provide a straightforward method to incorporate both shape and contextual information. However, these approaches require a segmentation of the image, which is a very difficult and sensitive task, especially in urban areas. Recently some attempts have been made to incorporate shape information on a pixel basis. In this paper we further develop this approach and propose a method to also derive contextual information on a pixel basis. The per-pixel features developed in this paper contain information about the distance from the object the pixel belongs to to the nearest shadow object. Clearly, this information can be helpful to identify buildings, which are generally accompanied by shadow. However, the proposed method can also be used to describe other contextual information.

Within a camera network, the contribution of a camera to the observation of a scene depends on its viewpoint and on the scene configuration. This is a dynamic property, as the scene content is subject to change over time. An automatic selection of a subset of cameras that significantly contributes to the desired observation of a scene can be of great value for the reduction of the amount of transmitted or stored image data. In this work, we propose low data rate schemes to select from a vision network a subset of cameras that provides a good frontal observation of the persons in the scene and allows for the best approximation of their 3D shape. We also investigate to what degree low data rates trade off quality of reconstructed 3D shapes.

This letter presents a novel locally adaptive error concealment method for subband coded images. For each lost low-frequency coefficient, we estimate the optimal interpolation weights from its neighborhood. The calculation of the interpolation weights is optimized in the mean squared error sense, and it takes into account the errors that would arise by horizontally and vertically interpolating the available neighbors of the lost coefficient. Compared to methods of similar complexity, the proposed scheme estimates the lost coefficients more accurately: on average, the PSNR is increased by up to 4.5 dB. The reconstructed images also look better, and our method is fast and of low complexity.

Meter to submeter resolution satellite images have generated new interests in extracting man-made structures in the urban area. However, classification accuracies for such purposes are far from satisfactory. Spectral characteristics of urban land cover classes are so similar that they cannot be separated using only spectral information. As a result, there is an increased interest in incorporating geometrical information. One possible approach is the use of morphological profiles (MPs). In this paper, we introduce two improvements on the use of MPs. Current approaches use disk-shaped structuring elements (SEs) to derive an MP. This profile contains information about the minimum dimension of objects. In this paper, we extend this approach by using linear SEs. This results in a profile containing information about the maximum object dimension. We show that the addition of the line-based MP gives a substantial improvement of the classification result. A second improvement is achieved by using ldquopartial morphological reconstructionrdquo instead of the normal morphological reconstruction. Morphological reconstruction is commonly used to better preserve the shape of objects. However, we show that this leads to ldquoover-reconstructionrdquo in typical remote sensing images and a decreased classification performance. With ldquopartial reconstruction,rdquo we are able to overcome this problem and still preserve the shape of objects.

In image communication over lossy packet networks (e.g., cell phone communication), packet loss errors lead to damaged images. Damaged images can be repaired with passive error concealment methods, which use neighboring coefficient or pixel values to estimate the missing ones. Neighboring image data should, thus, be spread over different packets. This paper presents a novel robust packetization method for the transmission of image content in lossy packet networks. We first define novel criteria for a good packetization. Based on these properties, we propose a cost function for packetization masks. We then use stochastic optimization to calculate optimal packetization masks. We test our packetization technique on both wavelet coding and DCT coding. Compared to other packetization techniques, we are able to achieve the same or better mean quality of the reconstructed images but with less fluctuation in quality, which is important for the viewer experience. In this way, we significantly increase the worst case quality, especially for high packet loss rates. This leads to visually more pleasing images in case of a passive reconstruction.

In this paper, we introduce the idea of using motion estimation resources from a video codec for video denoising. This is not straightforward because the motion estimators aimed for video compression and coding, tolerate errors in the estimated motion field and hence are not directly applicable to video denoising. To solve this problem, we propose a novel motion field filtering step that refines the accuracy of the motion estimates to a degree that is required for denoising. We illustrate the use of the proposed motion estimation method within a wavelet-based video denoising scheme. The resulting video denoising method is of low-complexity and receives comparable results with respect to the latest video denoising methods.

In some video coding applications, it is desirable to reduce the complexity of the video encoder at the expense of a more complex decoder. Distributed video (DV) coding is a new paradigm that aims to achieve this. To allocate a proper number of bits to each frame, most DV coding algorithms use a feedback channel (FBC). However, in some cases, a FBC does not exist. In this paper, we therefore propose a rate allocation (RA) algorithm for pixel-domain distributed video coders without FBC. Our algorithm estimates at the encoder the number of bits for every frame without significantly increasing the encoder complexity. Experimental results show that our RA algorithm delivers satisfactory estimates of the adequate encoding rate, especially for sequences with little motion.

Limited depth of field is an important problem in microscopy imaging. 3D objects are often thicker than the depth of field of the microscope, which means that it is optically impossible to make one single sharp image of them. Instead, different images in which each time a different area of the object is in focus have to be fused together. In this work, we propose a curvelet-based image fusion method that is frequency-adaptive. Because of the high directional sensitivity of the curvelet transform (and consequentially, its extreme sparseness), the average performance gain of the new method over state-of-the-art methods is high.

In this paper we present a novel method for reconstructing low-resolution text images. Unlike other conventional interpolation methods, the unknown pixel value is not estimated based on its local surrounding neighbourhood, but on the whole text image. In particularly, we exploit the repetitive behaviour of the characters. A great advantage of our proposed approach is that we have more information at our disposal, which leads to a better reconstruction of the interpolated image. Results show the effectiveness of our proposed method and its superiority at very large magnifications to traditional interpolation methods.

We develop a new vector-based shrinkage rule, based on the concept of "signal of interest", for the removal of correlated noise. The multivariate Bessel K Form density is used for modeling the spatial correlations between complex wavelet coefficients. The interscale dependencies between the coefficients are captured using a Hidden Markov Tree model. The combined spatial and interscale model gives improvements over recently proposed Hidden Markov Models for white noise. The results show that correlated noise is suppressed well while image details are being preserved.

This paper presents a novel passive error concealment method for wavelet coded images. The proposed method is a locally adaptive directional interpolation approach, where the interpolation weights are estimated based on the available local context. For each lost low frequency coefficient, we estimate the optimal interpolation weights based on the errors that would arise by horizontally and vertically interpolating the available neighbors of the lost coefficient. Compared to older methods of similar complexity, the proposed scheme estimates the lost coefficients much better: on average, the PSNR is increased with up to 0.6 dB. The results also indicate improvements over the best available state-of-the-art techniques. The reconstructed images also look better. As our method is fast and of low complexity, it is widely usable.

In packet switched networks such as the Internet, packets may get lost during transmission due to, e.g., network congestion. This leads to a quality degradation of the original signal. As video communication is a bandwidth consuming application, the original data are first compressed. This compression step increases the impact of information loss even more. In wavelet based image and video coding, the low frequency data is the most important. Loss of low frequency coefficients results in annoying black holes in the received images and video. This effect can be countered by post processing error concealment: a lost coefficient is estimated from its neighboring coefficients. In this paper we present a locally adaptive interpolation method for the lost low frequency coefficients. For each lost low frequency coefficient, we estimate the optimal interpolation direction (horizontal or vertical) using novel error measures. In this way, we preserve the edges in the reconstructed image much better. Compared to older techniques of similar complexity, our scheme reconstructs images with the same or better quality. This is reflected in the visual as well as in the numerical results: there is an increase of up to 4.4 dB compared to bilinear concealment. The proposed scheme is fast and simple, which makes it suitable for real-time applications.

In this paper, we present a graph-based approach for mining geospatial data. The system uses error-tolerant graph matching to find correspondences between the detected image features and the geospatial vector data. Spatial relations between objects are used to find a reliable object-to-object mapping. Graph matching is used as a flexible query mechanism to answer the spatial query. A condition based on the expected graph error has been presented which allows determining the bounds of error tolerance and, in this way, characterizes the relevancy of a query solution. We show that the number of null labels is an important measure to determine relevancy. To be able to correctly interpret the matching results in terms of relevancy, the derived bounds of error tolerance are essential

We propose an efficient and accurate wavelet based noise estimation method for white Gaussian noise in video sequences. The proposed method analyzes the distribution of spatial and temporal gradients in the video sequence in order to estimate the noise variance. The estimate is derived from the most frequent gradient in the two distributions and is compensated for the errors due to the spatio-temporal image sequence content, by a novel correction function. The main application of the proposed algorithm is for the estimation of the stationary Gaussian noise in wavelet based video processing, for which we show that the proposed method is more accurate than other state-of-the-art noise estimation techniques and less sensitive to varying spatio-temporal content and noise level. Furthermore, we adapt the algorithm for local noise estimation and test its performance.

This paper reports an information-theoretic analysis of the inter-scale, inter-orientation and inter-location dependencies that exist between curvelet coefficients. We show that the marginal statistics of these coefficients can be accurately modeled using generalized Gaussian density functions. Though generally decorrelated, we find that curvelets exhibit unusually high dependencies in intra-band local micro-neighborhoods, of a magnitude not found for instance in classical wavelets. Furthermore, dependencies are subject to and decrease with increasing orientation and location differences. Finally, we conclude that intra-band coefficient dependencies are stronger than either their inter-scale or inter-direction counterparts.

We develop a low-complexity overcomplete wavelet domain method for denoising digital images corrupted with non-stationary white additive Gaussian noise. The noise level for each pixel is estimated from a local window around that pixel. We use a shrinkage function that adapts itself to the noise level and to the spatially changing statistics of the image. Experiments show that this noise model has good results for different non-stationary noise sources. Finally, we extend our method for denoising images corrupted with signal-dependent noise.

While printing is basically a binary process (put ink or not), the impression of grayvalues and colors can be obtained with patterns of small ink dots. The characteristics of and interactions between these dots have an important impact on the resulting print. Traditional neighbourhood look-up table based print models are practically limited to a 3x3 neighbourhood due to the exponential growth of the look-up table size. In this paper we propose novel techniques to reduce the size of the look-up table, which is valuable on it own, but also makes it achievable to employ neighbourhoods covering more than 3x3 pixels. Furthermore, we observed experimentally a favourable trade-off between modelling error and dimensionality reduction.

In this paper we present a novel method for interpolating images and we introduce the concept of non-local interpolation. Unlike other conventional interpolation methods, the estimation of the unknown pixel values is not only based on its local surrounding neighbourhood, but on the whole image (non-locally). In particularly, we exploit the repetitive character of the image. A great advantage of our proposed approach is that we have more information at our disposal, which leads to better estimates of the unknown pixel values. Results show the effectiveness of non-local interpolation and its superiority at very large magnifications to other interpolation methods.

We propose a novel, Bayesian formulation of the edge-stopping (diffusivity) function in a nonlinear diffusion scheme in terms of edge probability under a marginal prior on noise-free gradient. This formulation differs from the existing probabilistic diffusion approaches that give stochastic formulations for the conductivity but not for the diffusivity function of the gradient. In particular, we impose a Laplacian prior for the ideal gradient, but the proposed formulation is general and can be used with other marginal distributions. We also make links to related works that treat correspondences between nonlinear diffusion and wavelet shrinkage.

We propose an efficient and accurate wavelet-based noise estimation method for white Gaussian noise in video sequences. The proposed method analyzes the distribution of spatial and temporal gradients in the video sequence in order to estimate the noise variance. The estimate is derived from the most frequent gradient in the two distributions and is compensated for the errors due to the spatio-temporal image sequence content, by a novel correction function. The spatial and temporal gradients are determined from the finest scale of the spatial and temporal wavelet transform, respectively. The main application of the noise estimation algorithm is in wavelet-based video processing. The results show that the proposed method is more accurate than other state-of-the-art noise estimation techniques and less sensitive to varying spatio-temporal content and noise level.

This paper proposes a novel video denoising method based on nondecimated wavelet band filtering. In the proposed method, motion estimation and adaptive recursive temporal filtering are performed in a closed loop, followed by an intra-frame spatially adaptive filter. All processing occurs in the wavelet domain. The paper introduces new wavelet-based motion reliability measures. We make a difference between motion reliability per orientation and reliability per wavelet band. These two reliability measures are employed in different stages of the proposed denoising scheme. The reliability per orientation (horizontal and vertical) measure is used in the proposed motion estimation scheme while the reliability of the estimated motion vectors (MVs) per wavelet band is utilized for subsequent adaptive temporal and spatial filtering. We propose a novel cost function for motion estimation which takes into account the spatial orientation of image structures and their motion matching values. Our motion estimation approach is a novel wavelet-domain three-step scheme, where the refinement of MVs in each step is determined based on the proposed motion reliabilities per orientation. The temporal filtering is performed separately in each wavelet band along the estimated motion trajectory and the parameters of the temporal filter depend on the motion reliabilities per wavelet band. The final spatial filtering step employs an adaptive smoothing of wavelet coefficients that yields a stronger filtering at the positions where the temporal filter was less effective. The results on various grayscale sequences demonstrate that the proposed filter outperforms several state-of-the-art filters visually (as judged by a small test panel) as well as in terms of peak signal-to-noise ratio

We develop three novel wavelet domain denoising methods for subband-adaptive, spatially-adaptive and multivalued image denoising. The core of our approach is the estimation of the probability that a given coefficient contains a significant noise-free component, which we call "signal of interest." In this respect, we analyze cases where the probability of signal presence is 1) fixed per subband, 2) conditioned on a local spatial context, and 3) conditioned on information from multiple image bands. All the probabilities are estimated assuming a generalized Laplacian prior for noise-free subband data and additive white Gaussian noise. The results demonstrate that the new subband-adaptive shrinkage function outperforms Bayesian thresholding approaches in terms of mean-squared error. The spatially adaptive version of the proposed method yields better results than the existing spatially adaptive ones of similar and higher complexity. The performance on color and on multispectral images is superior with respect to recent multiband wavelet thresholding.

In this paper we present a method for interpolating images that also preserves sharp edge information. We concentrate on tackling blurred edges by mapping level curves of the image. Level curves or isophotes are spatial curves with constant intensity. The mapping of these intensities can be seen as a local contrast enhancement problem, therefore we can use contrast enhancement techniques coupled with additional constraints for the interpolation problem. A great advantage of this approach is that the shape of the level set contours is preserved and no explicit edge detection is needed here. Results show an improvement in visual quality: edges are sharper and ringing effects are removed.

We analyze if the presence of corners in very high resolution (VHR) satellite images can give us an indication on the type of structure present in a scene (man-made versus natural structures). Two the corner detectors are validated in this respect: Harris and SUSAN. The detection performance is evaluated over a spectrum of spatial resolutions for current and future VHR systems (from 2 meters to 17 centimeters). The ground truth of this study consists of annotated image extracts containing different types of man-made structures, in which the relevant corners have been identified.

With the advent of very high resolution satellite images, such as IKONOS, the question of how we can incorporate textural information in classifying and segmenting different regions has become of great interest, in this paper we compare the power of classifying regions based on using color information alone to using texture and color texture information. We use a 2D and 3D extension of the co-occurrence matrix and the features derived from them. In the latter case the effect of color space reduction is also evaluated. We found that although color features perform best in the easy classification tasks, very high classification rates are obtained using color texture features and the fragmentation degree in the classified areas is smaller.

We examine the use of road detection in VHR satellite images to automate the process of quality assessment of digital road network data. An important aspect is the emphasis on accuracy and reliability of the system. Although road detection has been studied for more than a decade, it is often difficult to assess what performance can be expected over datasets other than the images published. We propose a system to train a road detector based on image examples of typical roads. The system calculates the optimal detection parameters and estimates the performance over the dataset in terms of detection rate and degree of fragmentation. The methodology relies on error propagation and image statistics, and is generic in nature. By showing image examples, variations due to shadow, activity on the road, weather conditions etc. can be taken into account when estimating the expected performance.

The huge amount of incoming synthetic aperture radar (SAR) data nowadays demands the need for automatic image registration. Due the presence of speckle noise and the huge size of SAR images, registering SAR images is more difficult than traditional image registration methods. In This work we present an automatic hierarchical registration method, based on polar block matching, that can handle the registration of SAR images efficiently. Results show us that our proposed registration method outperforms the manual registration and that our method is very robust against speckle noise. An additional advantage of our registration method is that there is no need for a despeckling preprocessing step.

With the advent of very high resolution satellite images, such as IKONOS, the question of how we can incorporate textural information in classifying and segmenting different regions has become of great interest. In this work we compare the power of classifying regions based on using color information alone to using texture and color texture information. We use a 2D and 3D extension of the co-occurrence matrix and the features derived from them. In the latter case the effect of color space reduction is also evaluated. We found that although color features perform best in the easy classification tasks, very high classification rates are obtained using color texture features and the fragmentation degree in the classified areas is smaller.

In this paper we point out that the general principle "do not compute what you do not need to compute" can be applied easily and successfully within a MPEG audio decoding strategy. More specifically, we will discuss the problem of eliminating costly computation cycles being wasted at processing useless zero-valued data. Hence, the title: "do not zero-pute". At first, this may all sound somewhat obvious or trivial. Indeed, this can be true in many cases, but experience gathered in various teaching related projects during several academic years has also lead us to believe the opposite. Moreover, a survey of the existing literature quickly reveals that the approach discussed below has not been investigated and documented properly. Although we will only illustrate our optimization approach by discussing the MPEG-audio layer II decoding process in detail, we hope the reader will be able to apply, extend, and implement the basic principles presented here within many other applications.

The aim of this paper is to present a methodology to generate a partition of an image and a hierarchical region merging scheme to improve the meaningfulness of the segmentation, by reducing excessive object fragmentation. The segmentation method is based on the watershed transform applied to the image gradient magnitude. Prior to the actual segmentation, the image is smoothed to decrease the amount of detail detected by the watershed transform. To further improve the segmentation result, we use an iterative region merging process that uses a graph to represent the image partitions. In this process the most similar pair of adjacent regions is sequentially merged according to a predefined similarity metric. We investigate the use of a combined region merging criterion that takes into account both the intensity similarity and the contrast at the boundary of two adjacent regions. Results obtained illustrate the good combined performance of this segmentation and merging methods and the usefulness of the combined similarity function.

Image-scrambling schemes are designed to render the image content unintelligible. Wyner has proposed an elegant one-dimensional (1-D) scrambling scheme without bandwidth expansion, making use of the discrete prolate spheroidal sequences (DPSS). The DPSS are optimal regarding their energy concentration in a given frequency subband. In this paper, we propose the two-dimensional (2-D) extension and application of this algorithm. We discuss new possibilities introduced by the 2-D approach. We also include experimental results.

This paper proposes a new family of bivariate, nonseparable splines, called hex-splines, especially designed for hexagonal lattices. The starting point of the construction is the indicator function of the Voronoi cell, which is used to define in a natural way the first-order hex-spline. Higher order hex-splines are obtained by successive convolutions. A mathematical analysis of this new bivariate spline family is presented. In particular, we derive a closed form for a hex-spline of arbitrary order. We also discuss important properties, such as their Fourier transform and the fact they form a Riesz basis. We also highlight the approximation order. For conventional rectangular lattices, hex-splines revert to classical separable tensor-product B-splines. Finally, some prototypical applications and experimental results demonstrate the usefulness of hex-splines for handling hexagonally sampled data.

We develop a new filter which combines spatially adaptive noise filtering in the wavelet domain and temporal filtering in the signal domain. For spatial filtering, we propose a new wavelet shrinkage method, which estimates how probable it is that a wavelet coefficient represents a "signal of interest" given its value, given the locally averaged coefficient magnitude and given the global subband statistics. The temporal filter combines a motion detector and recursive time-averaging. The results show that this combination outperforms single resolution spatio-temporal filters in terms of quantitative performance measures as well as in terms of visual quality. Even though our current implementation of the new filter does not allow real-time processing, we believe that its optimized software implementation could be used for real- or near real-time filtering.

Motion and other changes in video sequences can be detected by analyzing the differences between grey levels of successive frames. The simplest method to do this, is to subtract the grey values of corresponding pixels of two consecutive frames. This is a pixel based technique: if there is a difference, the pixel in question is considered to be on or near to a moving object. This assumption is clearly not always correct because changes in grey scale may also be caused by noise. In this paper, we present a technique for change detection that has low noise-sensitivity and that has a low complexity. The technique combines recursive temporal filtering with local neighborhood conditioning.

A new fuzzy filter is presented for the noise reduction of images corrupted with additive noise. The filter consists of two stages. The first stage computes a fuzzy derivative for eight different directions. The second stage uses these fuzzy derivatives to perform fuzzy smoothing by weighting the contributions of neighboring pixel values. Both stages are based on fuzzy rules which make use of membership functions. The filter can be applied iteratively to effectively reduce heavy noise. In particular, the shape of the membership functions is adapted according to the remaining noise level after each iteration, making use of the distribution of the homogeneity in the image. A statistical model for the noise distribution can be incorporated to relate the homogeneity to the adaptation scheme of the membership functions. Experimental results are obtained to show the feasibility of the proposed approach. These results are also compared to other filters by numerical measures and visual inspection.

In this paper, we propose a robust wavelet domain method for noise filtering in medical images. The proposed method adapts itself to various types of image noise as well as to the preference of the medical expert; a single parameter can be used to balance the preservation of (expert-dependent) relevant details against the degree of noise reduction. The algorithm exploits generally valid knowledge about the correlation of significant image features across the resolution scales to perform a preliminary coefficient classification. This preliminary coefficient classification is used to empirically estimate the statistical distributions of the coefficients that represent useful image features on the one hand and mainly noise on the other. The adaptation to the spatial context in the image is achieved by using a wavelet domain indicator of the local spatial activity. The proposed method is of low complexity, both in its implementation and execution time. The results demonstrate its usefulness for noise suppression in medical ultrasound and magnetic resonance imaging. In these applications, the proposed method clearly outperforms single-resolution spatially adaptive algorithms, in terms of quantitative performance measures as well as in terms of visual quality of the images.

Resampling techniques are commonly required in digital image processing systems. Many times the classical interpolation functions are used, i.e., nearest-neighbour interpolation and bilinear interpolation, which are prone to the introduction of undesirable artifacts due to aliasing such as moire patterns. This paper presents a novel approach which minimizes the loss of information, in a least-squares sense, while resampling between orthogonal and hexagonal lattices. Making use of an extension of 2D splines to hexagonal lattices, the proper reconstruction function is derived. Experimental results for a printing application demonstrate the feasibility of the proposed method and are compared against the classical techniques.

In the digital prepress workflow, images are represented in the CMYK colour space. Lossy image compression alleviates the need for high storage and bandwidth capacities, resulting from the high spatial and tonal resolution. After the image has been printed on paper, the introduced visual quality loss should not be noticeable to a human observer. Since visual image quality depends on the compression algorithm both quantitatively and qualitatively, and since no visual image quality models incorporating the end‐to‐end image reproduction process are satisfactory, an experimental comparison is the only viable way to quantify subjective image quality. This paper presents the results from an intensive psychovisual study based on a two‐alternative forced‐choice approach involving 164 people, with expert and non‐expert observers distinguished. The primary goal is to evaluate two previously published adaptations of JPEG to CMYK images, and to determine a visually lossless compression ratio threshold for typical printing applications. The improvements are based on tonal decorrelation and overlapping block transforms. Results on three typical prepress test images indicate that the proposed adaptations are useful and that for the investigated printing configuration, compression ratios up to 20 can be used safely.

The optimal one-dimensional (1-D) window, an index-limited sequence with maximum energy concentration in a finite frequency interval, is related to a particular discrete prolate spheroidal sequence. This letter presents the N-dimensional (N-D) extension, i.e., the N-D window with limited support and maximum energy concentration in a general nonseparable N-D passband. These windows can be applied to multidimensional filter design and the design of optimal convolution functions. We show the three-dimensional (3-D) optimal window based on a rhombic dodecahedron as a passband region.

This paper presents a new wavelet-based image denoising method, which extends a "geometrical" Bayesian framework. The new method combines three criteria for distinguishing supposedly useful coefficients from noise: coefficient magnitudes, their evolution across scales and spatial clustering of large coefficients near image edges. These three criteria are combined in a Bayesian framework. The spatial clustering properties are expressed in a prior model. The statistical properties concerning coefficient magnitudes and their evolution across scales are expressed in a joint conditional model. The three main novelties with respect to related approaches are (1) the interscale-ratios of wavelet coefficients are statistically characterized and different local criteria for distinguishing useful coefficients from noise are evaluated, (2) a joint conditional model is introduced, and (3) a novel anisotropic Markov random field prior model is proposed. The results demonstrate an improved denoising performance over related earlier techniques.

In this paper we give an overview of classical and fuzzy-classical filters for image noise reduction. Together with our overview (2001) of fuzzy filters, this paper can be seen as a preparation to our comparative study of classical and fuzzy filters for image noise reduction.

In this paper we give an overview of existing fuzzy filters for image noise reduction. The paper is a sequel to our overview of classical and fuzzy-classical filters (2001), and can be seen as a preparation to our comparative study of classical and fuzzy filters for image noise reduction. We discuss the ideas behind and the construction of the following fuzzy filters: the "fuzzy inference ruled by else-action" filters, the "fuzzy control based" filters, and the GOA filter. Our goal is to give a consistent overview of these fuzzy filters, and to clearly show the mutual differences between them.

In this paper we present the results of a comparative study of classical and fuzzy filters for image noise reduction. The discussed fuzzy filters are classified, and their performance is compared with classical filters and evaluated by numerical and visual experiments.

This paper describes a novel global-to-local method for the adaptive enhancement and unsupervised segmentation of brain tissues in MRI (magnetic resonance imaging) images. Three brain tissues are of interest: CSF (cerebrospinal fluid), GM (gray matter), WM (white matter). Firstly, we de-noise the image using wavelet thresholding, and segment the image with minimum error thresholding. Both the thresholdings are global-wise. Subsequently, we combine locally adaptive weighted median and weighted average filters with FCM (fuzzy C-means) clustering to achieve a local-wise segmentation. The performance of the proposed method is quantitatively validated by four indices with respect to a MRI brain phantom.

We propose an efficient and fast wavelet based technique for speckle removal from SAR images. It relies on realistic distributions of the wavelet coefficients which represent mainly speckle noise on the one hand and those that represent the useful signal corrupted by speckle on the other. We propose analytic models for these distributions, and compute their parameters automatically from a given SAR image. The resulting algorithm strongly suppresses speckle, while preserving image details and sharpness.

20 to 50 percent of neonates with very low birth weight (VLBW:<1500 g) suffer from white matter damage (leukomalacia). Nowadays the diagnosis of WMD is still solely dependent on the visual interpretation by an expert. A need for a (semi-) computerized way of segmenting the affected regions, in order to make quantitative measurements as an aid to the subjective diagnosis, is felt. Applying active contours for this purpose, is a classical approach. The performance of active contours for this purpose, however, is heavily deteriorated by the presence of speckle noise. In this paper a new filter, taking into account local statistics in the image, is proposed; it removes a significant amount of speckle noise in the healthy parts, while it makes the areas affected by WMD more uniform, thus severely improving the performance of the active contour. The results show that applying an active contour after the proposed technique yields a segmentation much closer to that of an expert.

Summary form only given. An evaluation is made of different approaches to removing interframe redundancies. The test images we used are the MRI and CT images available from the Visual Human Project. Firstly, we found that linear predictive techniques are not able to provide any compression improvement. Even an optimal linear predictor (least-squares optimization), which holds pixels from the current and the previous frames, does not outperform the very simple 2D lossless JPEG predictor number 7. However nonlinear techniques, such as context-modeling do yield better results. In a simple experiment, lossless JPEG was estimated to code intraframe prediction errors using an interframe context. The context for coding the current pixel is formed by the magnitude of the prediction error of the corresponding pixel in the previous frame. Note that this prediction error comes from the intraframe prediction step in the previous frame. Using this interframe context-modeling approach, a reduction of the bit rate of up to 1 bit per pixel is obtainable. It is obvious that the context serves as an edge-detector, albeit a very simple and straightforward one. A new technique is proposed based on the 2D lossless image compressor JPEG-LS. The existing context-modeling scheme is extended to also catch the interframe redundancies. The interframe context is very similar to the abovementioned one, however the sign of the prediction error in the previous frame is also used.

The visual interpretation of ultrasound brain images is a proven method to detect the white matter damage at an early stage. A problem, common to all medical ultrasound images, is the presence of speckle noise, which not only complicates the visual interpretation of images, but also quantitative measurements. This paper proposes a new filter that removes a significant amount of speckle noise from ultrasound images, while preserving details very well. The filter is based on a new cleaning technique that operates on the detail images of a wavelet decomposition. The paper illustrates that the proposed technique has some advantages over other popular techniques, i.e., the ones proposed by Lee (1980), Frost (1982), Malfait and Roose (1997), and Sattar et al. (1997).

We propose a new wavelet-based method for image denoising that applies the Bayesian framework, using prior knowledge about the spatial clustering of the wavelet coefficients. Local spatial interactions of the wavelet coefficients are modeled by adopting a Markov random field model. An iterative updating technique known as iterated conditional modes (ICM) is applied to estimate the binary masks containing the positions of those wavelet coefficients that represent the useful signal in each subband. For each wavelet coefficient a shrinkage factor is determined, depending on its magnitude and on the local spatial neighbourhood in the estimated mask. We derive analytically a closed form expression for this shrinkage factor.

The interlaced video scan format suffers from major flaws such as visual artifacts and unsuitability for devices such as LCD displays, video printers, computers which require or prefer the progressive scan format. De-interlacing algorithms convert a video signal from the interlaced scan format to the progressive scan format. Next to simple spatial interpolation, two categories of de-interlacing techniques are available: motion adaptive and motion compensating methods. The latter have more potential to produce better results but are also more complex since they require accurate subpixel motion estimation. This paper presents a motion compensating de-interlacing technique based on the Irani and Peleg (1991) superresolution algorithm. Appropriate weighting terms are introduced to take into account the interlaced scanning format. Two operational modes are derived: a single iteration mode for real time video de-interlacing, and a multiple iteration mode for enhanced still image generation.

In this paper, we propose a novel automatic method based on fuzzy modeling of knowledge to segment brain structures in MRI (magnetic resonance imaging) images. The segmentation is achieved by the region-wise classification using GAs (genetic algorithms), followed by voxel-wise refinement using parallel region growing. To improve the accuracy of the labeling, we introduce a fuzzy model of ROI (regions of interest) by analogy with the electrostatic potential distribution, to represent more appropriately knowledge of shape, distance and reaction between structures, and to estimate more reliably the statistical moments. This modeling is also used in the design of the fitness function of GAs, and the criteria of region growing. The performance of our proposed method is quantitatively validated by 4 indexes with respect to manually segmented images.

CMYK color images are used extensively in prepress applications. When compressing those color images one has to deal with four different color channels. Usually compression algorithms only take into account the spatial redundancy that is present in the image data. This approach does not yield an optimal data reduction since there also exists a high correlation between the different colors in natural images. This paper shows that a significant gain in data reduction can be achieved by exploiting this color redundancy. Some popular transform coders, including DCT‐based JPEG and the SPIHT wavelet coder, were used for reducing the spatial redundancy. The performance of the algorithms was evaluated using a quality criterion based on human perception like the mean CIEL*a*b*ΔE error.

This paper shows that an n x 1 integer vector can be exactly recovered from its Hadamard transform coefficients, even when 0.5 n log(2)(n) of the (less significant) bits of these coefficients are removed. The paper introduces a fast "lossless" dequantization algorithm for this purpose. To investigate the usefulness of the procedure in data compression, the paper investigates an embedded block image coding technique called the "LHAD" based on the algorithm. The results show that lossless compression ratios close to the state of the art can be achieved, but that techniques such as CALIC and S+P still perform better.

We propose a novel imaging system useful in dermatology, more precisely, for the follow-up of patients with an increased risk of skin cancer. The system consists of a Pentium PC equipped with an RGB frame grabber, a three-chip charge coupled devices (CCD) camera controlled by the serial port and equipped with a zoom lens and a halogen annular light source. Calibration of the imaging system provides a way to transform the acquired images, which are defined in an unknown color space, to a standard, well-defined color space called sRGB. sRGB has a known relation to the CIE1 XYZ and CIE L*a*b* colorimetric spaces. These CIE color spaces are based on the human vision, and they allow the computation of a color difference metric called CIE deltaE*ab, which is proportional to the color difference, as seen by a human observer. Several types of polynomial RGB to sRGB transforms will be tried, including some optimized in perceptually uniform color spaces. The use of a standard and well-defined color space also allows meaningful exchange of images, e.g., in teledermatology. The calibration procedure is based on 24 patches with known color properties, and it takes about 5 minutes to perform. It results in a number of settings called a profile that remains valid for tens of hours of operation. Such a profile is checked before acquiring images using just one color patch, and is adjusted on the fly to compensate for short-term drift in the response of the imaging system. Precision or reproducibility of subsequent color measurements is very good with (deltaE*ab) = 0.3 and deltaE*ab < 1.2. Accuracy compared with spectrophotometric measurements is fair with (deltaE*ab) = 6.2 and deltaE*ab < 13.3.

Deinterlacing algorithms are used to convert an interlaced video sequence to the progressive scan format. Interlaced video exposes artifacts like line flicker and line crawling and is unsuitable for progressive media. Picture quality can be improved significantly when a proper deinterfacing algorithm is applied. The paper presents a motion adaptive technique based on a fuzzy motion detector. Preliminary experiments show results using this approach.

This paper presents experimental results that demonstrate that the weakly separable polynomial orthonormal transform outperforms the shape adaptive discrete cosine transform (SADCT) and the previously introduced improved SADCT with /spl Delta/DC correction at the expense of a nonprohibitive increase in the number of computations. Some other improvements to SADCT-like schemes are also suggested.

Summary form only given. In the pre-press industry color images have both a high spatial and a high color resolution. Such images require a considerable amount of storage space and impose long transmission times. Data compression is desired to reduce these storage and transmission problems. Because of the high quality requirements in the pre-press industry only lossless compression is acceptable. Most existing lossless compression schemes operate on gray-scale images. In this case the color components of color images must be compressed independently. However, higher compression ratios can be achieved by exploiting inter-color redundancies. We present a new technique for exploiting inter-component redundancies. The technique is based on a modified Karhunen-Loeve transform (KLT) scheme in combination with a novel quantization scheme that guarantees losslessness. The KLT decorrelates the color components. It is recomputed on a block by block basis and is therefore spatially adaptive. Spatial redundancies are removed using predictive techniques (lossless JPEG predictor no. 7 and the CALIC-predictor). The data which remains after the (spatial and color) decorrelation should be entropy-coded, but in the current implementation of our scheme only the entropy of the remaining data is computed. Note that in each block some block-dependent information must be sent, such as entropy-coder initialization information and KLT-descriptors (i.e., the rotation angles of the orthogonal KLT-matrix).

An error in the theoretical derivations leading up to the main equation of the above correspondence by Prost et al. is pointed out, and corrections are presented. Due to the error, the experimentally obtained "optimal" parameters are really only suboptimal.

This paper addresses the efficient high-compression coding of palettized color images. The most common methods for the lossy compression of color images rely on independent block oriented transform coding of the three or four color components. These techniques do not make use of the high redundancy of the color components and introduce some very undesirable errors at high compression, in particular block distortion. We present an efficient and original technique to code color images with a small number of colors. This is an important class of images in multimedia applications. The technique codes the according luminance image using an existing segmented image coding method for monochrome images. The color information is independently represented in a bit map. The method does not rely on the commonly used color separation and shows a far better subjective image quality than JPEG at high compression.

This paper presents a lossless block-transform coder based on the Hadamard transform that achieves a compression ratio which is as good as that of the current lossless JPEG-standard and about 10 to 20% lower than the currently best state-of-the-art technique Calic. The main advantage of the proposed technique is that it produces an embedded data stream which may be truncated to achieve lossy compression. Preliminary results obtained on the "lena" image show that, at compression ratios up to 10, the proposed scheme (somewhat surprisingly) produces a better image quality (both visually and in terms of rms error) than lossy JPEG. This means that the proposed coder performs well over a wide range of compression ratios and is therefore a useful alternative for the lossy and lossless JPEG-standards in nearly-lossless coding or when lossless and lossy decoding must be supported simultaneously. Finally, the transform part of the proposed scheme requires fewer computations than lossy JPEG (and only slightly more than lossless JPEG).

This paper presents a new method for compactly representing contours. The new method is a generalization of the Fourier descriptor (FD) method. Like the FD method it approximates a complex-valued parameter representation of the contour by a trigonometric polynomial (TP). In the FD method, the TP is the best least mean squares approximation of the parameter representation. However, the mean squared error is not a good a criterion for judging the accuracy of a contour approximation. The paper proposes a better criterion and a method for computing the corresponding optimal TP. It shows that computing this optimal TP is equivalent to fitting a so-called warped TP to the parameter representation in the least mean squares sense. The paper presents preliminary results which show that the new method produces more accurate contour approximations than FD approximations of the same degree.

A segmented image coding algorithm for video sequences is presented. The first frame in the data is always encoded in an intraframe mode, while the rest of the frames in the data are encoded in an interframe mode. The interframe encoding is based on (1) block motion vector estimation and coding, (2) segmentation of the prediction error image and classification of the regions in the foreground/background, (3) contour simplification and coding, and (4) texture approximation by a linear combination of weakly separable base functions and coefficient coding. The contour simplification leads to an average reduction of 30% in the number of bits needed for the contour coding, the system can be adjusted at different bit-rates by parameter tuning, the simulation results are of high quality in terms of PSNR and they show that our coding approach is particularly promising for very low bit-rate applications.

Compression of medical images is of great importance in the implementation of systems such as PACS (Picture Archival and Communication System) and teleradiology. Significant compression is only achievable by lossy methods, though distortion must be kept low to preserve all the diagnostic information. The JPEG lossy compression standard has some shortcomings, in particular block distortion. We investigated two block-based DCT transform techniques which tend to suppress the block effect, the DCT/DST coder and the LOT coder, and compared them with JPEG and an ordinary DCT coder for different types of medical images. The results show that the LOT coder performs best at all compression ratios for all kinds of investigated medical images. At very high image quality, JPEG is preferable for it requires almost the same bit rate as the LOT and is computationally less demanding.

In segmented image coding (SIC) images are segmented and the texture of a segment is expanded as a weighted sum of polynomial base functions, orthogonal on that segment. At very low bit rates, SIC produces images of better subjective quality than standard techniques such as JPEG. The computational requirements of traditional SIC are huge since the number of operations for computing b orthogonal base functions increases faster than quadratically with b. This paper presents a new class of orthogonal bases which can be computed extremely quickly because they are weakly separable (WS). The number of operations needed for generating them grows nearly linearly with b. Typically, WS bases can be generated 8 to 30 times faster than their traditional counterparts. A coding example shows that their use does not affect subjective image quality.>

This paper presents an adaptive image coding method which uses an image dependent orthogonal transform. The method is a generalization of a one-dimensional coding scheme which represents signals as linear combinations of signal-dependent time-warped (TW) orthogonal polynomials. This paper briefly summarizes the theory of time-warped signal coding; next, it describes the new image compression method. Basically, this method transforms all of the image rows and columns with different, but not completely independent bases. By adapting the bases to the image, high quality coding is achieved even when retaining only a small number of transform coefficients. Also, the overhead involved in coding the bases is very small. This paper shows that at net bit rates of about 0.3 bpp, images compressed by the new method are sharper and less distorted by ringing effects than those produced by JPEG or the full-image DCT. Block distortion, which is an important problem in JPEG at 0.3 bpp cannot occur in the new method, since it transforms the full image instead of blocks.>

In region-oriented transform coding, an image is divided into several nonrectangular regions. The image intensity in each of these regions is represented as a weighted sum of orthogonal base functions. These base functions depend on the shape of the image region and are obtained by orthogonalizing a nonorthogonal starting base. This can be done by using the Gramm-Schmidt procedure. In that case, if base functions of degree up to N are required, a number of O(N/sup 4/) multiplications and additions must be computed for each pixel in the image region. This paper presents a new orthogonalization algorithm, in which the complexity is reduced from O(N/sup 4/) to O(N/sup 3/). The algorithm applies in the important cases, where the starting base consists of polynomials or sinewaves.>