Improving the spatial resolution of hyperspectral images (HSIs) has traditionally been an important topic in the field of remote sensing. Many approaches have been proposed based on various theories, including component substitution, multiresolution analysis, spectral unmixing, Bayesian probability, and tensor representation. However, these methods have some common disadvantages such that their performance degrades dramatically as the up-scale ratio increases, and they have little concern for the per-pixel radiometric accuracy of the sharpened image. Moreover, many learning-based methods have been proposed through decades of innovations, but most of them require a large set of training pairs, which is unpractical for many real problems. To solve these problems, we propose a stable hyperspectral sharpening method based on the Laplacian pyramid and the generative convolutional neural network (CNN), which achieves superior radiometric accuracy of the sharpened data in different up-scale ratios based on one single input pair. First, with a low-resolution HSI (LR-HSI) and high-resolution multispectral image (HR-MSI) pair, the preliminary high-resolution HSI (HR-HSI) is calculated via linear regression. Then, the high-frequency details of the preliminary HR-HSI are estimated via the subtraction between it and the CNN-generated-blurry version. By injecting the details to the output of the generative CNN with the LR-HSI as input, the final HR-HSI is obtained. Nine different state-of-the-art sharpening methods are chosen as our baselines, and three different datasets with different scene content are tested. Furthermore, the target detection method, the adaptive coherence estimator (ACE), is conducted on the reconstructed HR-HSI to evaluate the per-pixel radiometric accuracy. The results demonstrate that the proposed method has the best and the most stable performance in terms of spectral and spatial accuracies.

The convenience of cell-phone cameras has made them one of the most common ways by which people document their lives, whether it is everyday pleasures or celebrations. With thousands of images, it might prove to be a daunting task to organize them by hand. When applying automated algorithms to help us, we would like to have both images that are dear to us but are also of good quality. In this paper we explore the performance of the MobileNet CNN architecture, and the different design (inputs size, and layer depth) choices, in their ability in solving various aesthetic inference task: binary classification, regression, image cropping. We show that the baseline MobileNet architecture achieves near state-of-the-art results for binary classification on the AVA dataset while being more than 10 times smaller and compute efficient. We further show that these models, when trained for fine-grained aesthetics inference, achieve better cropping performance than other aestheticsbased croppers.

The ability to rank images based on their appearance finds many real-world applications such as image retrieval or image album creation. Despite the recent dominance of deep learning methods in computer vision which often result in superior performance, they are not always the methods of choice because they lack interpretability. In this work, we investigate the possibility of improving image aesthetic inference of convolutional neural networks with hand-designed features that rely on domain expertise in various fields. We perform a comparison of hand-crafted feature sets in their ability to predict fine-grained aesthetics scores on two image aesthetics datasets. We observe that even feature sets published earlier are able to compete with more recently published algorithms and, by combining the algorithms together, one can obtain a significant improvement in predicting image aesthetics. By using a tree-based learner, we perform feature elimination to understand the best performing features overall and across different image categories. Only roughly 15 % and 8 % of the features are needed to achieve full performance in predicting a fine-grained aesthetic score and binary classification respectively. By combining hand-crafted features with meta-features that predict the quality of an image based on CNN features, the model performs better than a baseline VGG16 model. One can, however, achieve more significant improvement in both aesthetics score prediction and binary classification by fusing the hand-crafted features and the penultimate layer activations. Our experiments indicate an improvement up to 2.2 % achieving current state-of-the-art binary classification accuracy on the AVA dataset when the hand-designed features are fused with activation from VGG16 and ResNet50 networks.

The Schroedinger Eigenmaps (SE) embedding has been previously introduced and applied to spectral target detection problems in hyperspectral imagery (HSI). The proposed SE-based detection approach combines the spectral and spatial connectivity of target-like pixels into the Schroedinger operator by using a “knowledge propagation” scheme. Likewise, it has been noted the impact that the local data structure modeling has over the SE-based detector. This local data structure is modeled by using an adjacency graph, i.e, set of vertices and edges, that in most of the applications is a k nearest neighbor (knn) graph. In this paper, the impact of k in the graph-building process is analyzed and assessed from a target detection point of view. Two ways to estimate the parameter k in the graph construction are considered and their assessment is performed by using two HSI data sets and the Receiver Optimal Curve (ROC) as a detection performance metric.

The past decade has seen the emergence of many hyperspectral image (HSI) analysis algorithms based on graph theory and derived manifold coordinates. The performance of these algorithms is inextricably tied to the graphical model constructed from the spectral data, i.e., the community structure of the spectral data must be well represented to extract meaningful information. This paper provides a survey of many spectral graph construction techniques currently used by the hyperspectral community and discusses their advantages and disadvantages for hyperspectral analyses. A focus is provided on techniques influenced by spectral density from which the concept of community structure arises. Two inherently density-weighted graph construction techniques from the data mining literature, shared nearest neighbor (NN) and mutual proximity, are also introduced and compared as they have not been previously employed in HSI analyses. Density-based edge allocation is demonstrated to produce more uniform NN lists than nondensity-based techniques by demonstrating an increase in the number of intracluster edges and improved k-NN classification performance. Imposing the mutuality constraint to symmetrify an adjacency matrix is demonstrated to be beneficial in most circumstances, especially in rural (less cluttered) scenes. Surprisingly, many complex edgereweighting techniques are shown to slightly degrade NN list characteristics. An analysis suggests this condition is possibly attributable to the validity of characterizing spectral density by a single variable representing data scale. As such, these complex edge-reweighting techniques may need to be modified to increase their effectiveness, or simply not be used.

This paper reports the initial investigation of the use of cube-satellites (cubesats) for image-based, three-dimensional (3D) reconstruction. Cubesats are emerging as a low-cost, inexpensive, and quickly deployable alternative to their larger satellite predecessors but their functionality is limited by the payload and bus electronics that can fit in the minimal volume. This paper addresses the impact of the spatial resolution limitation of state-of-the-art cubesat imagers on 3D reconstruction, which is assessed with building height and surface normal measurements. For the nadir ground-sampled distance (GSD) range of 0.25 to 2 m, reconstruction results yielded building height estimates that varied by approximately two meters and surface normal estimates with an error ranging from 1-29 degrees depending on the complexity of the surface.

As part of a study of the ecology of the ancient Mesoamerican Zapotex state, endmember abundance maps of a large portion of the state of Oaxaca, Mexico, were generated using Hyperion imagery and a spatially adaptive spectral unmixing approach. However, the ecological diversity of Oaxaca makes it difficult to identify the materials the endmembers correspond to without ground-truth data. Therefore, during December 2011 ground-truth data and samples were collected from three different valleys in Oaxaca (Tlacolula, Yanhuitlan, and Yucuita). A total of 24 different soil and plant samples and one terra cotta tile sample were collected from these valleys. These samples include white caliché rocks and soil, grey basalt volcanic rocks, red iron-stained soils, yellow iron-stained soils, and several different senescent plant samples. The reflectance spectra of these samples were acquired through laboratory measurements using an Analytical Spectral Device FieldSpec Pro covering 400-2500nm. Concurrently to the laboratory measurements, endmember abundance maps were generated using a spatially-adaptive spectral unmixing algorithm and Hyperion hyperspectral imagery of the Tlacolula Valey. A comparison of the Hyperion endmembers to the laboratory sample reflectance measurements is performed to determine which of the Hyperion endmemberss result from a single type and which from a linear mixture of multiple materials. It is shown that of the eight derived endmembers, seven appear to be mixtures of multiple materials.

We propose a novel unsupervised segmentation method that efficiently exploits spectral intensity, gradient and textural information in remotely sensed imagery. Our approach begins in a multi-band gradient detection scheme of the input scene, utilized to determine the spectral intensity variations across it. The obtained gradient map is employed in an iterative region growth procedure that originates in pixel locations of gradual/low intensity variations and concludes in sites of abrupt intensity transitions, to yield an initial region map. This map is combined with co-occurrence matrices based textural descriptors using a multivariate region merging procedure that fuses regions with similar characteristics to yield the final segmentation. Our approach was tested on several multi/hyperspectral data sets with favorable results.

Spectral unmixing is a common task in hyperspectral data analysis. In order to sufficiently spectrally unmix the data, three key steps must be accomplished: Estimate the number of endmembers (EMs), identify the EMs, and then unmix the data. Several different statistical and geometrical approaches have been developed for all steps of the unmixing process. However, many of these methods rely on using the full image to estimate the number and extract the EMs from the background data. In this paper, spectral unmixing is accomplished using a spatially adaptive approach. Linear unmixing is performed per pixel with EMs identified at the local level, but global abundance maps are created by clustering the locally determined EMs into common groups. Results show that the unmixing residual error of each pixel's spectrum from real data, estimated from the spatially adaptive methodology, is reduced when compared to a global scale EM estimation and linear unmixing methodology. The component algorithms of the new spatially adaptive approach, which complete the three key unmixing steps, can be interchanged while maintaining spatial information, making this new methodology modular. A final advantage of the spatially adaptive spectral unmixing methodology is the user-defined spatial scale size.

Target detection in reflective hyperspectral imagery generally involves the application of a spectral matched filter on a per-pixel basis to create an image of the target likelihood of occupying each pixel. Stochastic (or unstructured) target detection techniques require the user to define an estimate of the background mean and covariance from which to separate out the desired targets in the image. Typically, scene-wide statistics are used, although it is simple to show that this methodology does not produce sufficiently multivariate normal backgrounds, nor does it necessarily represent the best suppression of likely false alarms. This technique can be improved on by segmentation methods that selectively choose which pixels best represent the background for a particular test pixel and / or target spectrum. Here, several spatial and spectral segmentation techniques are presented and improved target detection performance over scene-wide statistics is shown for a common target in two data sets with different scene content. Results are presented in the form of average false alarm rates and a chi-squared goodness of fit measure of the background multivariate normality. Improvements are possible using segmentation methods over global estimation of background mean and covariance. However, the best method of background characterization depends strongly on the spatial and spectral characteristics of the target of interest and scene content.