Abstract In the context of Earth observation, the trade-off between spatial, spectral, and temporal resolution often limits the versatility of remote sensing images in many important applications. In response, this paper introduces a novel deep learning diffusion model, specifically tailored to improve the spatial resolution of the optical products acquired by the Sentinel-3 (S3) satellite. Our framework employs a diffusion probabilistic model, benefiting from the higher spatial resolution of the Sentinel-2 satellite during training via a new multi-modal loss formulation. This ensures consistency with the original S3 images while enhancing the spatial details. Two distinct conditional low-resolution encoders were experimented with, providing insights into their respective contributions to the diffusion process. The efficacy of the proposed model is demonstrated through extensive ablation studies and comparisons with state-of-the-art methods, using both synthetic and real S3 products. The findings indicate that our model successfully improves spatial resolution while maintaining the integrity of the spectral information, contributing to the field of remote sensing single-image super-resolution.

Motivated by the increasing demand for robust segmentation in unlabeled remote sensing data, we propose DAM-Former, a novel UDA model that fuses high-resolution multimodal imagery with multi-temporal multispectral data. Current UDA approaches in remote sensing rarely exploit the complementary strengths of spatial and temporal features. To address this gap, our framework integrates two interconnected branches: a transformer-based network for high-resolution multimodal data and a lightweight convolutional network with temporal attention for multi-temporal imagery. To improve segmentation accuracy and lower noise, the extracted features are robustly combined through a deep temporal fusion module and a new mixed loss with an ensemble pseudo-label strategy. Extensive experiments and an ablation study on the FLAIR-2 dataset demonstrate that DAM-Former outperforms state-of-the-art methods, marking the first in-depth study of temporal information fusion in UDA segmentation for remote sensing data.

Earth data collection from satellites and aircraft has exponentially grown, but a substantial portion of it remains unlabeled. This has prompted the remote sensing community to explore effective methods for leveraging unlabeled data. In our prior investigation [1], we evaluated various deep semi-supervised learning algorithms on two very high-resolution (VHR) optical datasets (UCM [2] and AID [3]). Notably, the CoMatch [4] algorithm demonstrated the highest accuracy, motivating further exploration. This letter extends our earlier work by integrating the established Class-Aware Contrastive Semi-Supervised Learning framework (Comatch+CCSSL) [5] into CoMatch and introducing a new triplet metric learning loss (CoMatch+Triplet). CoMatch+Triplet excelled with 93.2% accuracy on UCM, while CoMatch led with 92.19% on AID. The addition of the triplet loss can produce a clearer separation of the samples from different classes in the embedding space at very early learning stages, being able to learn faster and getting maximum performance with few iterations. The exploration of diverse semi and self-supervised training methodologies presented in this work sheds light on the strengths and limitations of these approaches, enhancing our understanding of their applicability in remote sensing applications.

The increasing availability of remote sensing data allows dealing with spatial-spectral limitations by means of pan-sharpening methods. However, fusing inter-sensor data poses important challenges, in terms of resolution differences, sensor-dependent deformations and ground-truth data availability, that demand more accurate pan-sharpening solutions. In response, this paper proposes a novel deep learning-based pan-sharpening model which is termed as the double-U network for self-supervised pan-sharpening (W-NetPan). In more details, the proposed architecture adopts an innovative W-shape that integrates two U-Net segments which sequentially work for spatially matching and fusing inter-sensor multi-modal data. In this way, a synergic effect is produced where the first segment resolves inter-sensor deviations while stimulating the second one to achieve a more accurate data fusion. Additionally, a joint loss formulation is proposed for effectively training the proposed model without external data supervision. The experimental comparison, conducted over four coupled Sentinel-2 and Sentinel-3 datasets, reveals the advantages of W-NetPan with respect to several of the most important state-of-the-art pan-sharpening methods available in the literature. The codes related to this paper will be available at https://github.com/rufernan/WNetPan.

Image registration is an essential task in image processing, where the final objective is to geometrically align two or more images. In remote sensing, this process allows comparing, fusing, or analyzing data, especially when multimodal images are used. In addition, multimodal image registration becomes fairly challenging when the images have a significant difference in scale and resolution, together with local small image deformations. For this purpose, this letter presents a novel optical flow (OF)-based image registration network, named the FloU-Net, which tries to further exploit intersensor synergies by means of deep learning. The proposed method is able to extract spatial information from resolution differences and through a U-Net backbone generate an OF field estimation to accurately register small local deformations of multimodal images in a self-supervised fashion. For instance, the registration between Sentinel-2 (S2) and Sentinel-3 (S3) optical data is not trivial, as there are considerable spectral–spatial differences among their sensors. In this case, the higher spatial resolution of S2 results in S2 data being a convenient reference to spatially improve S3 products, as well as those of the forthcoming Fluorescence Explorer (FLEX) mission, since image registration is the initial requirement to obtain higher data processing level products. To validate our method, we compare the proposed FloU-Net with other state-of-the-art techniques using 21 coupled S2/S3 optical images from different locations of interest across Europe. The comparison is performed through different performance measures. Results show that the proposed FloU-Net can outperform the compared methods. The code and dataset are available inhttps://github.com/ibanezfd/FloU-Net.

Continuous wave Time of Flight cameras obtain depth images by emitting a modulated continuous light wave and measuring the delay of the received signal. One of the main sources of error is shot noise. In this letter we analyze and provide insights about the effect of shot noise when obtaining the phase delay with a new architecture, based on differentiating the phase integrations in the analog domain, and using any number of points in the calculation of the Discrete Fourier Transform (DFT). One of the main findings is that the error depends on the integration time regardless the number of points in the DFT used to calculate the phase offset. Simulated experiments are provided which support the proposed theoretical analysis.

Remote sensing is also benefiting from the quick development of deep learning algorithms for image analysis and classification tasks. In this paper, we evaluate the classification performance of a well-known Convolutional Neural Network (CNN) models, such as ResNet50, using a transfer learning approach. We compare the performance when using vector-features acquired from general purpose data, such as the ImageNet [1], versus remote sensing data like BigEarthNet [2], UCMerced [3], RESISC45 [4] and So2Sat [5]. The results show that the model pre-trained on RESISC-45 data achieved the highest accuracy when classifying the Eurosat [6] testing dataset. This was followed by the model pre-trained on Imagenet with 95.94% and BigEarthNet with 95.93%. When presented with diverse remote sensing data, the classification improved in regards to large quantities of general-purpose data. The experiments carried out also show, that multi modal (co-registered synthetic aperture radar and multispectral) did not increase the classification rate with respect to using only multispectral data. The source codes of this work are available for reproducible research at https://github.com/itzahs/CNN-RS.

The availability of geo-referenced coupled data of dif-ferent platforms is essential to train remote sensing (RS) multi-modal classification and bio-phyiscal parameter esti-mation learning methods. To properly develop a general-izing model different scenes and topographies are required. For this purpose, different multi-modal datasets have been published for the last years. Nevertheless, to our knowl-edge there is not any dataset composed of Sentinel-2 (S2) and Sentinel-3 (S3) geo-referenced images. In this paper we present SEN23, a dataset composed of 100 complete multi-spectral S2 and S3 paired images of different locations along Europe from the 2021 summer. The coupled images were obtained with a time difference of three or less days, containing less than a 1 % of cloud coverage and have a resolution difference of × 15. SEN23E is expected to help with the development of new multi-spectral, multi-resolution and multi-modal models for complex tasks which need con-text and complete images. SEN23E will be available at https://github.com/ibanezdf/SEN23E.

Sentinel missions provide widespread opportunities of exploiting inter-sensor synergies to improve the operational monitoring of terrestrial photosynthetic activity and canopy structural variations using vegetation indices (VI). In this context, continuous and consistent temporal data are logically required to rapidly detect vegetation changes across sensors. Nonetheless, the existing temporal limitations inherent to satellite orbits, cloud occlusions, data degradation, and many other factors may severely constrain the availability of data involving multiple satellites. In response, this letter proposes a novel deep 3-D convolutional regression network (3CRN) for temporally enhancing Sentinel-3 (S3) VI by taking advantage of inter-sensor Sentinel-2 (S2) observations. Unlike existing regression and deep learning-based methods, the proposed approach allows convolutional kernels to slide across the temporal dimension to exploit not only the higher spatial resolution of the S2 instrument but also its own temporal evolution to better estimate time-resolved VI in S3. To validate the proposed approach, we built a database made of multiple day-synchronized S2 and S3 operational products from a study area in Extremadura (Spain). The conducted experimental comparison, including multiple state-of-the-art regression and deep learning models, shows the statistically significant advantages of the presented framework. The codes of this work will be made available athttps://github.com/rufernan/3CRN.

Deep learning has certainly become the dominant trend in hyper-spectral (HS) remote sensing image classification owing to its excellent capabilities to extract highly discriminating spatial-spectral features. In this context, transformer networks have recently shown prominent results in distinguishing even the most subtle spectral differences because of their potential to characterize sequential spectral data. Nonetheless, many complexities affecting HS remote sensing data (e.g. atmospheric effects, thermal noise, quantization noise, etc.) may severely undermine such potential since no mode of relieving noisy feature patterns has still been developed within transformer networks. To address the problem, this paper presents a novel masked auto-encoding spectral-spatial transformer (MAEST), which gathers two different collaborative branches: (i) a reconstruction path, which dynamically uncovers the most robust encoding features based on a masking auto-encoding strategy; and (ii) a classification path, which embeds these features onto a transformer network to classify the data focusing on the features that better reconstruct the input. Unlike other existing models, this novel design pursues to learn refined transformer features considering the aforementioned complexities of the HS remote sensing image domain. The experimental comparison, including several state-of-the-art methods and benchmark datasets, shows the superior results obtained by MAEST. The codes of this paper will be available at https://github.com/ibanezfd/MAEST.

With the increasing availability of Sentinel-2 (S2) and Sentinel-3 (S3) data, developing higher-level data products becomes a very attractive option to relieve the spatial limitations of the Ocean and Land Colour Instrument (OLCI) of S3. In this context, this paper investigates the suitability of super-resolving operational OLCI products using the Multi-Spectral Instrument (MSI) of S2 as an offline spatial reference. Specifically, the proposed approach assembles a multi -spectral data fusion scheme together with a convolutinal neural network (CNN) mapping function to project the OLCI sensor onto its corresponding spatial reference which is synthetically generated by the OLCI/MSI fusion. In this way, the trained model is able to super-resolve operational OLCI products under demand without the need of using MSI data. The experimental part of the work shows the suitability of the proposed approach in the context of the Copernicus programme.

Image registration is an essential task in image processing, where the final objective is to align geometrically two or more images. In Remote Sensing this process allows to compare, fusion or analyse data. For this purpose, different methods and techniques have been proposed. In this paper a selection of image registration methods has been compared performing inter-sensor registration between Sentinel-2 (S2) and Sentinel-3 (S3) operational data. Registration between S2 and S3 data is not trivial, as there are considerable spectral-spatial differences among them. Nevertheless, the resolution difference results in S2 products being a convenient reference to improve S3 products spatially. The experimentation has been done using four sample pairs of S2 and S3 operational data and representatives of the main registration algorithms used in the last years. Performance measures and results are shown and discussed to check the accuracy and quality of the selected methods.

Unsupervised hashing methods have attracted considerable attention in large-scale remote sensing (RS) image retrieval, due to their capability for massive data processing with significantly reduced storage and computation. Although existing unsupervised hashing methods are suitable for operational applications, they exhibit limitations when accurately modeling the complex semantic content present in RS images using binary codes (in an unsupervised manner). To address this problem, in this letter, we introduce a novel unsupervised hashing method that takes advantage of the generative nature of probabilistic topic models to encapsulate the hidden semantic patterns of the data into the final binary representation. Specifically, we introduce a new probabilistic latent semantic hashing (pLSH) model to effectively learn the hash codes using three main steps: 1) data grouping, where the input RS archive is clustered into several groups; 2) topic computation, where the pLSH model is used to uncover highly descriptive hidden patterns from each group; and 3) hash code generation, where the data probability distributions are thresholded to generate the final binary codes. Our experimental results, obtained on two benchmark archives, reveal that the proposed method significantly outperforms state-of-the-art unsupervised hashing methods.

The global coverage of Sentinel-2 provides widespread opportunities for accurately mapping and monitoring key crops in emerging countries, like in the case of Nepal's rice production. While previous studies based on other satellites show some important spatial and temporal limitations, the use of operational Sentinel-2 data still remains unexplored in this regard. As a result, this work investigates the viability of using the Sentinel-2 instrument for a precise rice crop classification in Nepal. Initially, we define a dataset made of multi-temporal Sentinel-2 data from the Terai region of Nepal. Then, we conduct several classification experiments to provide empirical evidences about the suitability of different classification models when identifying rice crops in developing countries, where only limited ground-truth data could be available. The experiments reveal the suitability of using Sentinel-2 for accurately mapping rice crops in Nepal with a CNN-based classification model.

The recent availability of operational data from the Sentinel-2 and Sentinel-3 missions provides widespread opportunities to generate diverse high-level remote sensing products. However, the synergies between both multi-spectral instruments are often difficult to exploit from an operational perspective. Standard pansharpening algorithms may encounter important disadvantages due to the limited intersensor data availability in actual production environments. Moreover, the lack of a real high-resolution ground-truth for super-resolution techniques may affect the radiometric quality of the final result. In this scenario, this work investigates the viability of using the Multi-Spectral Instrument of Sentinel-2 for super-resolving data products acquired by the Ocean and Land Colour Instrument of Sentinel-3. Specifically, we define an inter-sensor image enhancement framework which combines a PCA-based component substitution pansharpening scheme with a CNN-based spatial enhancing super-resolution mapping. The conducted experiments reveal the suitability of the proposed approach for generating Level-4 data products within the Copernicus programme context.

This letter presents a novel hyperspectral endmember extraction approach that integrates a tensor-based decomposition scheme with a probabilistic framework in order to take advantage of both technologies when uncovering the signatures of pure spectral constituents in the scene. On the one hand, statistical unmixing models are generally able to provide accurate endmember estimates by means of rather complex optimization algorithms. On the other hand, tensor decomposition techniques are very effective factorization tools which are often constrained by the lack of physical interpretation within the remote sensing field. In this context, this letter develops a new hybrid endmember extraction approach based on the decomposition of the probabilistic tensor moments of the hyperspectral data. Initially, the input image reflectance values are modeled as a collection of multinomial distributions provided by a family of Dirichlet generalized functions. Then, the unmixing process is effectively conducted by the tensor decomposition of the third-order probabilistic tensor moments of the multivariate data. Our experiments, conducted over four hyperspectral data sets, reveal that the proposed approach is able to provide efficient and competitive results when compared to different state-of-the-art endmember extraction methods.

This letter presents a novel intersensor vegetation estimation framework, which aims at combining Sentinel-2 (S2) spatial resolution with Sentinel-3 (S3) spectral characteristics in order to generate fused vegetation maps. On the one hand, the multispectral instrument (MSI), carried by S2, provides high spatial resolution images. On the other hand, the Ocean and Land Color Instrument (OLCI), one of the instruments of S3, captures the Earth's surface at a substantially coarser spatial resolution but using smaller spectral bandwidths, which makes the OLCI data more convenient to highlight specific spectral features and motivates the development of synergetic fusion products. In this scenario, the approach presented here takes advantage of the proposed constrained probabilistic latent semantic analysis (CpLSA) model to produce intersensor vegetation estimations, which aim at synergically exploiting MSI's spatial resolution and OLCI's spectral characteristics. Initially, CpLSA is used to uncover the MSI reflectance patterns, which are able to represent the OLCI-derived vegetation. Then, the original MSI data are projected onto this higher abstraction-level representation space in order to generate a high-resolution version of the vegetation captured in the OLCI domain. Our experimental comparison, conducted using four data sets, three different regression algorithms, and two vegetation indices, reveals that the proposed framework is able to provide a competitive advantage in terms of quantitative and qualitative vegetation estimation results.

This letter presents a novel intersensor registration framework specially designed to register Sentinel-3 (S3) operational data using the Sentinel-2 (S2) instrument as a reference. The substantially higher resolution of the Multispectral Instrument (MSI), on-board S2, with respect to the Ocean and Land Color Instrument (OLCI), carried by S3, makes the former sensor a suitable spatial reference to finely adjust OLCI products. Nonetheless, the important spectral-spatial differences between both instruments may constrain traditional registration mechanisms to effectively align data of such different nature. In this context, the proposed registration scheme advocates the use of a topic model-based embedding approach to conduct the intersensor registration task within a common multispectral semantic space, where the input imagery is represented according to their corresponding spectral feature patterns instead of the low-level attributes. Thus, the OLCI products can be effectively registered to the MSI reference data by aligning those hidden patterns that fundamentally express the same visual concepts across the sensors. The experiments, conducted over four different S2 and S3 operational data collections, reveal that the proposed approach provides performance advantages over six different intersensor registration counterparts.

Convolutional neural networks (CNNs) have recently exhibited an excellent performance in hyperspectral image classification tasks. However, the straightforward CNN-based network architecture still finds obstacles when effectively exploiting the relationships between hyperspectral imaging (HSI) features in the spectral-spatial domain, which is a key factor to deal with the high level of complexity present in remotely sensed HSI data. Despite the fact that deeper architectures try to mitigate these limitations, they also find challenges with the convergence of the network parameters, which eventually limit the classification performance under highly demanding scenarios. In this paper, we propose a new CNN architecture based on spectral-spatial capsule networks in order to achieve a highly accurate classification of HSIs while significantly reducing the network design complexity. Specifically, based on Hinton's capsule networks, we develop a CNN model extension that redefines the concept of capsule units to become spectral-spatial units specialized in classifying remotely sensed HSI data. The proposed model is composed by several building blocks, called spectral-spatial capsules, which are able to learn HSI spectral-spatial features considering their corresponding spatial positions in the scene, their associated spectral signatures, and also their possible transformations. Our experiments, conducted using five well-known HSI data sets and several state-of-the-art classification methods, reveal that our HSI classification approach based on spectral-spatial capsules is able to provide competitive advantages in terms of both classification accuracy and computational time.

Convolutional neural networks (CNNs) exhibit good performance in image processing tasks, pointing themselves as the current state-of-the-art of deep learning methods. However, the intrinsic complexity of remotely sensed hyperspectral images still limits the performance of many CNN models. The high dimensionality of the HSI data, together with the underlying redundancy and noise, often makes the standard CNN approaches unable to generalize discriminative spectral-spatial features. Moreover, deeper CNN architectures also find challenges when additional layers are added, which hampers the network convergence and produces low classification accuracies. In order to mitigate these issues, this paper presents a new deep CNN architecture specially designed for the HSI data. Our new model pursues to improve the spectral-spatial features uncovered by the convolutional filters of the network. Specifically, the proposed residual-based approach gradually increases the feature map dimension at all convolutional layers, grouped in pyramidal bottleneck residual blocks, in order to involve more locations as the network depth increases while balancing the workload among all units, preserving the time complexity per layer. It can be seen as a pyramid, where the deeper the blocks, the more feature maps can be extracted. Therefore, the diversity of high-level spectral-spatial attributes can be gradually increased across layers to enhance the performance of the proposed network with the HSI data. Our experiments, conducted using four well-known HSI data sets and 10 different classification techniques, reveal that our newly developed HSI pyramidal residual model is able to provide competitive advantages (in terms of both classification accuracy and computational time) over the state-of-the-art HSI classification methods.

The relatively recent availability of operational products from Sentinel-2 and Sentinel-3 missions gives widespread opportunities to combine data collected from different sensors in order to provide products of a higher processing level. Nonetheless, the availability of these products may be affected by multiple factors, such as cloud occlusions, band saturation, geolocation errors or even misaligned detectors. All these anomalies affecting remote sensing data may eventually limit the accessibility to fused products because some of the required information may become partially unavailable for specific areas of interest. In this scenario, the work presented here aims at analyzing the effectiveness of several state-of-the-art regression models in order to restore Sentinel-3 products with partial anomalies from Sentinel-2 integral data. In particular this work investigates three regression methods, two linear-regression method and a non-linear artificial neural networks based method. Obtained results prove that the nonlinear approach and linear RIDGE method are able to carry out a good estimation of S3 from S2 data.

Probabilistic topic models have recently shown a great potential in the remote sensing image fusion field, which is particularly helpful in land-cover categorization tasks. This letter first studies the application of probabilistic latent semantic analysis (pLSA) and latent Dirichlet allocation to remote sensing synthetic aperture radar (SAR) and multispectral imaging (MSI) unsupervised land-cover categorization. Then, a novel pLSA-based image fusion approach is presented, which pursues to uncover multimodal feature patterns from SAR and MSI data in order to effectively fuse and categorize Sentinel-1 and Sentinel-2 remotely sensed data. Experiments conducted over two different data sets reveal the advantages of the proposed approach for unsupervised land-cover categorization tasks.

This paper presents a novel approach for spectral unmixing of remotely sensed hyperspectral data. It exploits probabilistic latent topics in order to take advantage of the semantics pervading the latent topic space when identifying spectral signatures and estimating fractional abundances from hyperspectral images. Despite the contrasted potential of topic models to uncover image semantics, they have been merely used in hyperspectral unmixing as a straightforward data decomposition process. This limits their actual capabilities to provide semantic representations of the spectral data. The proposed model, called dual-depth sparse probabilistic latent semantic analysis (DEpLSA), makes use of two different levels of topics to exploit the semantic patterns extracted from the initial spectral space in order to relieve the ill-posed nature of the unmixing problem. In other words, DEpLSA defines a first level of deep topics to capture the semantic representations of the spectra, and a second level of restricted topics to estimate endmembers and abundances over this semantic space. An experimental comparison in conducted using the two standard topic models and the seven state-of-the-art unmixing methods available in the literature. Our experiments, conducted using four different hyperspectral images, reveal that the proposed approach is able to provide competitive advantages over available unmixing approaches.

Super-resolution (SR) brings an excellent opportunity to improve a wide range of different remote sensing applications. SR techniques are concerned about increasing the image resolution while providing finer spatial details than those captured by the original acquisition instrument. Therefore, SR techniques are particularly useful to cope with the increasing demand remote sensing imaging applications requiring fine spatial resolution. Even though different machine learning paradigms have been successfully applied in SR, more research is required to improve the SR process without the need of external high-resolution (HR) training examples. This paper proposes a new convolutional generator model to super-resolve low-resolution (LR) remote sensing data from an unsupervised perspective. That is, the proposed generative network is able to initially learn relationships between the LR and HR domains throughout several convolutional, downsampling, batch normalization, and activation layers. Then, the data are symmetrically projected to the target resolution while guaranteeing a reconstruction constraint over the LR input image. An experimental comparison is conducted using 12 different unsupervised SR methods over different test images. Our experiments reveal the potential of the proposed approach to improve the resolution of remote sensing imagery.

Multidimensional optical imaging systems for information processing and visualization technologies have numerous applications in fields such as manufacturing, medical sciences, entertainment, robotics, surveillance, and defense. Among different three-dimensional (3-D) imaging methods, integral imaging is a promising multiperspective sensing and display technique. Compared with other 3-D imaging techniques, integral imaging can capture a scene using an incoherent light source and generate real 3-D images for observation without any special viewing devices. This review paper describes passive multidimensional imaging systems combined with different integral imaging configurations. One example is the integral-imaging-based multidimensional optical sensing and imaging systems (MOSIS), which can be used for 3-D visualization, seeing through obscurations, material inspection, and object recognition from microscales to long range imaging. This system utilizes many degrees of freedom such as time and space multiplexing, depth information, polarimetric, temporal, photon flux and multispectral information based on integral imaging to record and reconstruct the multidimensionally integrated scene. Image fusion may be used to integrate the multidimensional images obtained by polarimetric sensors, multispectral cameras, and various multiplexing techniques. The multidimensional images contain substantially more information compared with two-dimensional (2-D) images or conventional 3-D images. In addition, we present recent progress and applications of 3-D integral imaging including human gesture recognition in the time domain, depth estimation, mid-wave-infrared photon counting, 3-D polarimetric imaging for object shape and material identification, dynamic integral imaging implemented with liquid-crystal devices, and 3-D endoscopy for healthcare applications.

Over the last years, three-dimensional (3-D) imaging has been applied to human action and gesture recognition, usually in the form of depth maps from RGB-D sensors. An alternative which has not been explored is 3-D integral imaging, aside from a recent preliminary study which shows that it can be an effective sensory modality with some advantages over the conventional monocular imaging. Since integral imaging has also been shown to be a powerful tool in other visual tasks (e.g., object reconstruction and recognition) under challenging conditions (e.g., low illumination, occlusions), and its passive long-range operation brings benefits over active close-range devices, a natural question is whether these advantages also hold for gesture recognition. Furthermore, occlusions are present in many real-world scenarios in gesture recognition, but it is an elusive problem which has scarcely been addressed. As far as we know, this letter analyzes for the first time the potential of integral imaging for gesture recognition under occlusions, by comparing it to monocular imaging and to RGB-D sensory data. Empirical results corroborates the benefits of 3-D integral imaging for gesture recognition, mainly under occlusions.

This paper presents an overview of research work and some novel strategies and results on using data fusion in 3-D imaging when using multiple information sources. We examine a variety of approaches and applications such as 3-D imaging integrated with polarimetric and multispectral imaging, low levels of photon flux for photon-counting 3-D imaging, and image fusion in both multiwavelength 3-D digital holography and 3-D integral imaging. Results demonstrate the benefits data fusion provides for different purposes, including visualization enhancement under different conditions, and 3-D reconstruction quality improvement.

Scattering from a leaf responds differently at different wavelengths to changes in leaf properties such as pigment concentrations, chemical constituents, internal structure, and leaf-surface properties. Radiation scattered by leaves and exiting the vegetation canopy toward the sensor is affected by canopy structure. The concept of canopy spectral invariants is used to decompose multiangular hyperspectral Compact High Resolution Imaging Spectroradiometer–PROBA surface reflectances over agricultural crops during peak growth season into structural and optical components. The former, called the directional area scattering factor, is determined by the canopy geometrical properties and varies with crop type. The latter is a function of the leaf scattering properties and more directly related to the leaf interior. For dense crops, the decomposition technique does not require the use of canopy radiation models, prior knowledge, or ancillary information regarding the leaf scattering properties and thus provides a powerful means to remove canopy structural influences in hyperspectral remote sensing of leaf biochemical constituents. Our results also suggest that leaf-surface characteristics can increase canopy scattering spectra. This may decrease the ability to remotely sense leaf biochemistry.

This letter presents a spectral-spatial pixel characterization method for hyperspectral images. The characterization is based on textural features obtained using Gabor filters over a selected set of spectral bands. This scheme aims at improving land-use classification results, decreasing significantly the number of spectral bands needed in order to reduce the dimensionality of the task owing to an adequate description of the spatial characteristics of the image. This allows requiring less data and avoiding the curse of dimensionality. Very promising results are obtained which are similar to or better than previous classification results provided by other spectral-spatial methods but here also reducing the complexity using a reduced number of spectral bands.

In this paper, the results of a semi-supervised approach based on the Expectation-Maximisation algorithm for model-based clustering are presented. We show in this work that, if the appropriate generative model is chosen, the classification accuracy on clustering for image segmentation can be significantly improved by the combination of a reduced set of labelled data and a large set of unlabelled data. This technique has been tested on real images as well as on medical images from a dermatology application. The preliminary results are quite promising. Not only the unsupervised accuracies have been improved as expected but the segmentation results obtained are considerably better than the results obtained by other powerful and well-known unsupervised image segmentation techniques.

A problem when working with optical satellite or airborne images is the need to compensate for changes in the illumination conditions at the time of acquisition. This is particularly critical when working with time series of data. Atmospheric correction strategies based on radiative transfer codes may provide a rigorous solution but it may not be the best solution for situations where a huge amount of hyperspectral images may need to be processed and computational time is a critical factor. The GMES (¿Global Monitoring for Environment and Security¿) initiative has promoted the creation of a new generation of satellites (the SENTINEL series) with ¿ultra-high resolution¿ and ¿superspectral imaging¿ capabilities. Therefore, there is an urgent need to quickly and reliably compensate for changes in the atmospheric transmittance and varying solar illumination conditions. In this paper three different forms of affine transformation models (general, particular and diagonal) are considered as candidates for rapid compensation of illumination variations. They are tested on a series of simulated multispectral images of Top-Of-Atmosphere (TOA) radiance, where the surface is a synthetic scene of a test site in Spain called Barrax, where reference data for validation is available. The results indicate that in 2 of the more moderate Sun positions, for all the Visibilities tested, the particular affine method is better than the other 2. The results also indicate that the proposed methodology is satisfactory for practical normalization of varying illumination and atmospheric conditions in remotely sensed images required for operational or time critical applications.

In this contribution a feature selection method in semi-supervised problems is proposed. This method selects variables using a feature clustering strategy, using a combination of supervised and unsupervised feature distance measure, which is based on Conditional Mutual Information and Conditional Entropy. Real databases were analyzed with different ratios between labelled and unlabelled samples in the training set, showing the satisfactory behaviour of the proposed approach.

Hyperspectral imaging involves large amounts of information. This paper presents a technique for dimensionality reduction to deal with hyperspectral images. The proposed method is based on a hierarchical clustering structure to group bands to minimize the intracluster variance and maximize the intercluster variance. This aim is pursued using information measures, such as distances based on mutual information or Kullback–Leibler divergence, in order to reduce data redundancy and nonuseful information among image bands. Experimental results include a comparison among some relevant and recent methods for hyperspectral band selection using no labeled information, showing their performance with regard to pixel image classification tasks. The technique that is presented has a stable behavior for different image data sets and a noticeable accuracy, mainly when selecting small sets of bands.

In this paper, a band selection technique for hyperspectral image data is proposed. Supervised feature extraction techniques allow a reduction of the dimensionality to extract relevant features through a labeled training set. This implies an analysis of the existing class distributions, which usually means, in the case of hyperspectral imaging, a large number of samples, making the labeling process difficult. A possible alternative could be the use of information measures, which are the basis of the proposed method. The present approach basically behaves as an unsupervised feature selection criterion, to obtain the relevant spectral bands from a set of sample images. The relations of information content between spectral bands are analyzed, leading to the proposed technique based on the minimization of the dependent information between spectral bands, while trying to maximize the conditional entropies of the selected bands.

This article presents a new framework for the motion segmentation and estimation task on sequences of two gray images without a priori information of the number of moving regions present in the sequence. The proposed algorithm uses temporal information, by using an accurate Generalized Least-Squares motion estimation process, and spatial information, by using an iterative region-growing algorithm that classifies regions of pixels into the different motion models present in the sequence. The initial regions of pixels are obtained from a given gray-level segmentation process. The performance of the algorithm is tested on synthetic and real images with multiple objects undergoing different types of motion. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 577–590, 2005.

Intensive fruit and vegetable sorting is a common task in productive regions. In order to meet the market standards, produce is classified according to quality levels that depend on maturity degree, weight, size, density, skin defects, etc. Probably the most important of these tasks involve automatic visual inspection. A distributed and scalable system for sorting automation is presented, that addresses all aspects of the quality classification mentioned above. The main characteristics of the system are: it can control from 1 to 10 conveyor belts; its maximum performance is 15 fruits per second per belt; apart from weight sensors, it combines infrared, color and ultraviolet images; some fruit defect modules are available to take account of the presence of a given defect.

This paper presents a motion estimation and segmentation algorithm based on multiple parametric model estimation that determines the a priori unknown number of motion models present in the data. The algorithm applies a quasi-simultaneous parametric model fitting method based on a general least square fitting. Some experiments are showed in order to demonstrate the results obtained using the proposed algorithm.

This paper describes a method for tracking regions in image sequences. Regions segmented from each frame by a motion segmentation technique are matched by using a relaxation procedure. Matching is based on measuring the similarity of the regions from the current frame and a list of regions corresponding to objects. A Kalman filter is used in order to estimate motion parameters. This filter uses a kinematic model which considers varying acceleration. This assumption allows the system to model the movement when objects are approaching the camera. The tracking method presented here has been successfully applied to traffic monitoring tasks, where it connects to other two computer vision based modules: motion segmentation and temporal integration.

This paper presents an algorithm for segmenting and tracking moving objects in a scene. Temporal information provided by a region tracking strategy is integrated for improving frame to frame motion segmentation. The method has been applied to a traffic monitoring system and it provides facilities such as estimating trajectories of vehicles, detecting stopped vehicles and estimating the mean velocity of the traffic.

This paper describes an algorithm for oblique decision tree induction. The approach is based on the construction of a Voronoi diagram over the set of points representing patterns in a d-dimensional feature space. The procedure basically consists of a decomposition of the feature space into convex regions with samples from just one class.

A novel approach for dominant point detection in chain-coded contours is presented. Classical operations, such as computing a measurement of the curvature from the (x, y) co-ordinates of the contour points, finding curvature maxima, etc., are substituted by a neural network that traverses the contour, and gives a measurement of the relevance of every point. Further and straight-forward processing of the network output provides the dominant points. Two translations of the Freeman chain-code are presented, that easily provide the network input from the chain link values. Results with real and test images are presented that show the feasibility of the proposed algorithm. The simulation of the neural computations on a sequential machine makes the execution time of this algorithm of the same order as that of existing algorithms, but the cost can be significantly reduced by executing these computations on a neural-oriented hardware.

A new algorithm for dominant point detection in chain-coded contours is presented. The algorithm directly operates on the chain-code link values. No computation of the (x,y) co-ordinates of the contour points is done, nor any classical computation of the curvature or its derivative. Instead, a dynamic neural network traverses the contour giving a measurement of the relevance of each point, further and simple processing provides the dominant points. The network is trained with the result that a classical dominant point detection algorithm gives for the training contours, and using as training set a number of contours extracted from natural images. Results with real and test images are presented that show the reliability of the proposed algorithm. Since this algorithm is based on applying a neural network to the contour, it significantly reduces the execution time of existing dominant point detection algorithms. Computational time measurements are presented.