The problem of robust matrix completion-the recovery of a low-rank matrix and a sparse matrix from a sampling of their superposition-has been addressed extensively in prior literature. Yet, much of this work has focused exclusively on the case in which the matrix sampling is done at random, as this scenario is amenable to theoretical analysis. In contrast, sampling with an arbitrary deterministic pattern is often more accommodating to hardware implementation; consequently, the problem of robust matrix completion under deterministic sampling is considered. To this end, a restricted approximate isometry property is proposed and used, along with a modified golfing scheme and a slightly strengthened incoherence condition, to prove that the latent low-rank and sparse matrices are uniquely recoverable via convex optimization with asymptotically high probability, providing the first exact-recovery theory for robust matrix completion with arbitrary deterministic sampling. A corresponding convex-optimization algorithm, driven by a traditional nuclear norm, is developed and then subsequently generalized by substituting a convolutional nuclear norm in order to cover a broader range of application scenarios. Empirical experiments on synthetic data verify the proposed theory while a battery of results on real-world images demonstrate the practical efficacy of the generalized algorithm for robust matrix recovery.

Hyperspectral super-resolution is commonly accomplished by the fusing of a hyperspectral imaging of low spatial resolution with a multispectral image of high spatial resolution, and many tensor-based approaches to this task have been recently proposed. Yet, it is assumed in such tensor-based methods that the spatial-blurring operation that creates the observed hyperspectral image from the desired super-resolved image is separable into independent horizontal and vertical blurring. Recent work has argued that such separable spatial degradation is ill-equipped to model the operation of real sensors which may exhibit, for example, anisotropic blurring. To accommodate this fact, a generalized tensor formulation based on a Kronecker decomposition is proposed to handle any general spatial-degradation matrix, including those that are not separable as previously assumed. Analysis of the generalized formulation reveals conditions under which exact recovery of the desired super-resolved image is guaranteed, and a practical algorithm for such recovery, driven by a blockwise-group-sparsity regularization, is proposed. Extensive experimental results demonstrate that the proposed generalized tensor approach outperforms not only traditional matrix-based techniques but also state-of-the-art tensor-based methods; the gains with respect to the latter are especially significant in cases of anisotropic spatial blurring.

Convolutional networks have been widely used for the classification of hyperspectral images; however, such networks are notorious for their large number of trainable parameters and high computational complexity. Additionally, traditional convolution-based methods are typically implemented as a simple cascade of a number of convolutions using a single-scale convolution kernel. In contrast, a lightweight multiscale convolutional network is proposed, capitalizing on feature extraction at multiple scales in parallel branches followed by feature fusion. In this approach, 2D depthwise convolution is used instead of conventional convolution in order to reduce network complexity without sacrificing classification accuracy. Furthermore, multiscale channel attention is also employed to selectively exploit discriminative capability across various channels. To do so, multiple 1D convolutions with varying kernel sizes provide channel attention at multiple scales, again with the goal of minimizing network complexity. Experimental results reveal that the proposed network not only outperforms other competing lightweight classifiers in terms of classification accuracy but also exhibits a lower number of parameters as well as significantly less computational cost.

While deep learning has been widely employed for the classification of hyperspectral imagery, many scenarios arise in practice in which too few labeled samples exist to effectively train the networks. Few-shot learning has been recently used to deploy classifiers trained on source-domain datasets comprising a large number of labeled samples to datasets from a target domain with only few labeled samples. However, most techniques in this vein effectively assume that the source and target domains possess the same data distribution, whereas the distributions between the two domains often differ widely in practice. Adversarial domain adaption driven by prototype classifiers deployed independently in the source and target domains is proposed to handle such differing source and target distributions, while an attention-based feature extractor with residual skip connections is developed in order to weight spectral bands according to their importance to the hyperspectral classification task. Experimental results demonstrate improved performance for the proposed few-shot-learning framework relative to both fully-supervised classifiers as well as other few-shot techniques.

Deep learning is being increasingly employed for hyperspectral classification, although such use is often predicated on the availability of a sufficiently large set of labeled samples for training. To improve classification performance under a limited training-set size, a semi-supervised network with end-to-end local–global active learning (AL) based on graph convolutional networks (GCNs) is proposed. The proposed AL extracts both global as well as local graph-based features to gauge the discriminative information in unlabeled samples, while semi-supervised classification expands the training set of a fully supervised classifier by attaching pseudo-labels to high-confidence unlabeled samples. Experimental results demonstrate that the proposed network outperforms not only other approaches to semi-supervised classification but also several existing fully supervised methods. The source code of this method can be found athttps://github.com/XtaoS/semi-LG-AGCN.

Low-rank subspace representations have been observed to be well-suited to hyperspectral imagery, which tends to have a global structure composed of a small number of ground-cover signatures, and additional graph-based regularization can further incorporate local information. However, in the context of unsupervised classification, existing approaches typically limit consideration to simple graphs built on spectral information alone. In contrast, a hypergraph-based low-rank subspace clustering is proposed to capture a more complex manifold structure. In addition, basing the hypergraph on a superpixel segmentation of the image exploits structure that is meaningful both spatially as well as spectrally. The experimental results reveal performance for the proposed superpixel-hypergraph approach superior to that of competing techniques representative of several prominent classes of unsupervised classification for hyperspectral imagery.

Tensor-based fusion that couples the high spatial resolution of a multispectral image (MSI) to the high spectral resolution of a hyperspectral image (HSI) is considered. The fusion problem is first formulated mathematically as a convex optimization of a tensor trace norm imposing low-rank spatially as well as spectrally, with an alternating-directions optimization featuring linearization providing the solution. Although prior tensor-based fusion approaches typically resort to tensor decomposition, the proposed algorithm exploits ideas from the field of tensor completion to directly impose a low-rank property spatially and spectrally while avoiding the computationally complex patch clustering and dictionary learning common to competing fusion techniques. Additionally, small modifications to the basic optimization permit a fusion process robust to missing hyperspectral values such as those that can result from dead stripes in real hyperspectral sensors. The experimental evaluations on both synthetic imagery as well as real imagery demonstrate that the resulting low-rank tensor-approximation (LRTA) fusion algorithm preserves both spatial details and texture, yielding significantly improved image quality when compared to other state-of-the-art fusion methods as well as effective restoration under conditions of missing stripes within the HSI.

Although support vector classifiers for hyperspectral imagery traditionally exploit spectral information alone, there has been increasing interest in spatial-spectral classifiers that incorporate spatial context due to the potential for significant performance improvement over spectral-only approaches. Accordingly, a new approach for spatial-spectral classification is introduced which incorporates spatial information into a prior hyperspectral classifier driven by functional data analysis (FDA) applied to continuous spectral functions. FDA permits functional properties-such as the smoothness inherent to spectral signatures-to inform hyperspectral classification. The proposed spatial FDA (SFDA) incorporates an additional spatial coherency factor that attempts to ensure that each pixel is represented with a spectral curve that is similar to those of its nearest spatial neighbors. Experimental results demonstrate that the proposed SFDA coupled with a support vector classifier yields results superior to other state-of-the-art spatial-spectral techniques for hyperspectral classification.

Pushbroom acquisition of hyperspectral imagery is prone to striping artifacts in the along-track direction. A hyperspectral destriping algorithm is proposed such that the subbands of a 3-D wavelet transform most affected by pushbroom stripes-namely, those with spatially vertical orientation-are the exclusive focus of destriping. The proposed method features an iterative image decomposition composed of a low-rank model for the stripes coupled with a group-sparse prior on the wavelet coefficients of the subbands in question. While low-rank stripe models have been widely used in the past, they typically have been deployed in conjunction with a total-variation prior on the image that is prone to oversmoothing and residual stripe artifacts. On the other hand, the proposed group-sparse prior not only captures the well-known sparse nature of wavelet coefficients but also capitalizes on their vertical clustering in the subbands in question. In addition, while many prior destriping methods are wavelet-based, they employ 2-D transforms band by band. In contrast, the proposed 3-D wavelet transform provides a greater concentration of stripe information into fewer wavelet coefficients, leading to more effective destriping. Experimental results on both synthetically striped imagery as well as real striped imagery from an actual hyperspectral sensor demonstrate superior image quality for the proposed method as compared with other state-of-the-art methods.

The traditional use of support-vector machines for hyperspectral imagery exploits spectral information alone; however, classifiers that incorporate spatial context have witnessed increasing interest due to their potential for significant improvement over spectral-only approaches. A new paradigm for spatial-spectral support-vector classification is introduced in which spatial context is included into the logistic regression commonly used with support-vector classifiers to provide a probabilistic output. In experimental results, the proposed approach is compared to methods representative of two prominent families of spatial-spectral support-vector classifiers-composite kernels and postprocessing regularization-and it is observed that the proposed approach provides superior classification accuracy.

Dimensionality reduction based on random projections is investigated in the context of spectral unmixing of hyperspectral imagery with aims toward unmixing accuracy and computational efficiency. To this end, both Hadamard-based random projections-which significantly reduce computational costs with respect to more traditional Gaussian-driven projections-as well as a fast singular value decomposition deployed within a random-projection space are considered. Experimental results reveal that the methods based on Hadamard random projections offer abundance-estimation performance superior to other methods in conjunction with significantly reduced computational complexity.

Summary form only given. Malware is an ever-increasing threat to personal, corporate, and government computing systems alike. Particularly in the corporate and government sectors, the attribution of malware - including the identification of the authorship of malware as well as potentially the malefactor responsible for an attack - is of growing interest. Such malware attribution is often enabled by the fact that malware authors build on the work of others through the use of generators, libraries, and borrowed code. Determining malware phylogeny - the evolutionary history of and the derivative relations between malware - is consequently an endeavor of increasing importance, with a growing focus on the dynamic analysis of malware which involves executing a malware sample and determining the actions it takes after some period of operation. In most cases, such dynamic analysis occurs in a virtual machine, or "sandbox," in order to confine the malware to an environment in which it can do no harm to real systems. In sandbox-driven dynamic analysis of malware, a virtual machine is typically run starting from some known, malware-free baseline state. The malware is injected into the virtual machine, and the machine is allowed to run for some period of time during which the malware presumably activates. The machine is then suspended, and the current machine memory is dumped to disk. The process may then be repeated for other malware samples, each time starting from the baseline state. Stored in raw form on the disk, the dumped memory file is the same size as the virtual-machine memory, for virtual machines running modern operating systems, such memory would likely be no less than 512 MB but could be up to several GBs. If the corresponding memory dumps are to be retained for repeated analysis - as is likely to be required in order to determine a phylogeny for a large database of malware samples - lossless compression of the memory dumps is necessarily to prevent explosive disk usage. For example, the VirusShare project maintains a database of over 19 million malware samples, running these in a virtual machine with 512 MB of memory would require of 9 petabytes (PB) of storage to retain the memory dumps. In this paper, we develop a scheme for the lossless compression of memory dumps resulting from the repeated execution of malware samples in a virtual-machine sandbox. Rather than compress each memory dump individually, we capitalize on the fact that memory dumps stem from a known baseline virtual-machine state and code with respect to this baseline memory. Additionally, to further improve compression efficiency, we exploit the fact that a significant portion of the difference between the baseline memory and that of the currently running machine is the result of the loading of known executable programs and shared libraries. Consequently, we propose delta coding to compress the current virtual-machine memory dump by coding its differences with respect to a predicted memory image, with the latter formed by duplicating the loading of the executables and libraries into the baseline memory, resulting in a significant improvement in compression performance over straightforward delta coding alone. In experimental results for a body of malware samples, the proposed approach outperformed the widely used xdelta3 delta coder by approximately 20% and the popular generic gzip coder by 79%.

The rich spectral information in hyperspectral imagery gives rise to huge storage and transmission costs. Dimensionality reduction aims to reduce the space complexity in hyperspectral imagery by projecting data into a low-dimensional subspace. There has been an increasing interest in dimensionality reduction driven by random projections due to its data-independent representation as well as desirable qualities such as the preservation of important information and low computational costs. The performance of a random projection derived from a Hadamard-Walsh matrix is investigated, with experimental results demonstrating classification performance superior to other random dimensionality-reduction methods when deployed in conjunction with a composite-kernel support vector machine that exploits both spatial and spectral information for the classification of hyperspectral imagery.

While data-dependent dimensionality reduction has dominated in many applications of hyperspectral imagery, there is increasing interest in data-independent strategies - such as random projections - due to their promise for reduced computational complexity as well as their demonstrated ability to preserve application-important information. Such random-projection-based dimensionality reduction is investigated in the specific context of supervised hyperspectral classification. Both Hadamard- and Gaussian-based random projections are considered, applied alone as well as incorporated into a fast approximate singular value decomposition (SVD). Experimental results reveal that the proposed Hadamard-based random projection with the fast SVD (FSVD) offers a computationally attractive alternative to not only traditional SVD but also Gaussian-based FSVD for dimensionality reduction in hyperspectral classification.

Nonnegative matrix factorization (NMF), which can lead to nonsubtractive parts-based representation, has been demonstrated to be effective for dimensionality reduction of hyperspectral imagery (HSI). However, existing NMF methods applied to HSI use only a single spectral feature and do not take into consideration spatial information, such as texture or morphological features, while it has been widely acknowledged that exploiting multiple features can improve performance. Consequently, a variant of orthogonal NMF, which can not only achieve a nonnegative factorization but also exploit the complementary information that arises among heterogeneous features, is proposed for hyperspectral dimensionality reduction. The proposed method, which couples orthogonal NMF with a previous multiple-features-combining algorithm, yields a discriminative low-dimensional feature representation that matches the intuition that parts should sum to produce a whole. An efficient multiplicative updating procedure is derived, and its local convergence is guaranteed theoretically. Experimental results on two hyperspectral data sets demonstrate the effectiveness of the proposed method.

There is increasing interest in driving supervised classification of hyperspectral imagery by a support vector machine using a composite kernel employing both spectral and spatial features. While the spectral signature of the current hyper-spectral pixel is often used directly to supply the spectral feature, a statistic - such as the mean - calculated across a spatial window surrounding the pixel is typically employed as a spatial feature. In contrast, a nearest-neighbor spatial feature is proposed in which the nearest neighbors in Euclidean distance to the current pixel are used to calculate the spatial feature. It is argued that the proposed nearest-neighbor spatial feature is more likely to incorporate relevant, same-class neighbor pixels than window-based features for which borders between coherent single-class regions may give rise to misclassification. Experimental results illustrate the performance advantage of the proposed nearest-neighbor framework at supervised hyperspectral classification in comparison to several competing benchmark algorithms that also employ kernel-based support vector machines.

Most existing architectures for the compressive acquisition of hyperspectral imagery - which perform dimensionality reduction simultaneously with image acquisition - have focused on framing designs which require the entire spatial extent of the image be available at once to the sensor. On the other hand, hyperspectral imagery in remote-sensing applications is frequently acquired with a pushbroom or whiskbroom sensing paradigm which - incorporating line-based or pixel-based scanning, respectively - exploits the motion inherent in an airborne or satellite-borne sensing platform to acquire the image. Such pushbroom and whiskbroom sensing architectures are proposed for the compressive acquisition of hyperspectral imagery. Additionally, the necessity of employing multiple sensor arrays in order to sense a broad spectrum, including the infrared regime, is considered.

Multiple views of a scene - obtained via different sensing modalities - have the potential to significantly enhance image analysis for remote sensing and other applications. This benefit is expected to be significant if the multiple views are providing independent, yet useful, information about the underlying classes in a scene. To exploit such multi-sensor information, a compressive-projection approach to the fusion of multi-sensor imagery is proposed. It is argued that that random projections yield subspaces that preserve the discriminative nature of multi-sensor datasets with profound implications in a practical scenario wherein compressive measurements can directly facilitate data fusion without the need for complicated subspace-learning approaches. A case study fusing experimental hyperspectral and LiDAR data demonstrates that statistical learning in the compressive-measurement domain is not only feasible, but also provides a natural framework for sensor fusion without the need for explicit reconstruction from compressive measurements.

The Gaussian mixture model is a well-known classification tool that captures non-Gaussian statistics of multivariate data. However, the impractically large size of the resulting parameter space has hindered widespread adoption of Gaussian mixture models for hyperspectral imagery. To counter this parameter-space issue, dimensionality reduction targeting the preservation of multimodal structures is proposed. Specifically, locality-preserving nonnegative matrix factorization, as well as local Fisher's discriminant analysis, is deployed as preprocessing to reduce the dimensionality of data for the Gaussian-mixture-model classifier, while preserving multimodal structures within the data. In addition, the pixel-wise classification results from the Gaussian mixture model are combined with spatial-context information resulting from a Markov random field. Experimental results demonstrate that the proposed classification system significantly outperforms other approaches even under limited training data.

The same high dimensionality of hyperspectral imagery that facilitates detection of subtle differences in spectral response due to differing chemical composition also hinders the deployment of traditional statistical pattern-classification procedures, particularly when relatively few training samples are available. Traditional approaches to addressing this issue, which typically employ dimensionality reduction based on either projection or feature selection, are at best suboptimal for hyperspectral classification tasks. A divide-and-conquer algorithm is proposed to exploit the high correlation between successive spectral bands and the resulting block-diagonal correlation structure to partition the hyperspectral space into approximately independent subspaces. Subsequently, dimensionality reduction based on a graph-theoretic locality-preserving discriminant analysis is combined with classification driven by Gaussian mixture models independently in each subspace. The locality-preserving discriminant analysis preserves the potentially multimodal statistical structure of the data, which the Gaussian mixture model classifier learns in the reduced-dimensional subspace. Experimental results demonstrate that the proposed system significantly outperforms traditional classification approaches, even when few training samples are employed.

In this letter, a fusion-classification system is proposed to alleviate ill-conditioned distributions in hyperspectral image classification. A windowed 3-D discrete wavelet transform is first combined with a feature grouping-a wavelet-coefficient correlation matrix (WCM)-to extract and select spectral-spatial features from the hyperspectral image dataset. The adjacent wavelet-coefficient subspaces (from the WCM) are intelligently grouped such that correlated coefficients are assigned to the same group. Afterwards, a multiclassifier decision-fusion approach is employed for the final classification. The performance of the proposed classification system is assessed with various classifiers, including maximum-likelihood estimation, Gaussian mixture models, and support vector machines. Experimental results show that with the proposed fusion system, independent of the classifier adopted, the proposed classification system substantially outperforms the popular single-classifier classification paradigm under small-sample-size conditions and noisy environments.

The one-against-one (OAO) strategy is commonly employed with classifiers-such as support vector machines-which inherently provide binary two-class classification in order to handle multiple classes. This OAO strategy is introduced for the classification of hyperspectral imagery using discriminant analysis within kernel-induced feature spaces, producing a pair of algorithms-kernel discriminant analysis and kernel local Fisher discriminant analysis-for dimensionality reduction, which are followed by a quadratic Gaussian maximum-likelihood-estimation classifier. In the proposed approach, a multiclass problem is broken down into all possible binary classifiers, and various decision-fusion rules are considered for merging results from this classifier ensemble. Experimental results using several hyperspectral data sets demonstrate the benefits of the proposed approach-in addition to improved classification performance, the resulting classifier framework requires reduced memory for estimating kernel matrices.

Reconstruction of hyperspectral imagery from spectral random projections is considered. Specifically, multiple predictions drawn for a pixel vector of interest are made from spatially neighboring pixel vectors within an initial non-predicted reconstruction. A two-phase hypothesis-generation procedure based on partitioning and merging of spectral bands according to the correlation coefficients between bands is proposed to fine-tune the hypotheses. The resulting prediction is used to generate a residual in the projection domain. This residual being typically more compressible than the original pixel vector leads to improved reconstruction quality. To appropriately weight the hypothesis predictions, a distance-weighted Tikhonov regularization to an ill-posed least-squares optimization is proposed. Experimental results demonstrate that the proposed reconstruction significantly outperforms alternative strategies not employing multihypothesis prediction.

A classifier that couples nearest-subspace classification with a distance-weighted Tikhonov regularization is proposed for hyperspectral imagery. The resulting nearest-regularized-subspace classifier seeks an approximation of each testing sample via a linear combination of training samples within each class. The class label is then derived according to the class which best approximates the test sample. The distance-weighted Tikhonov regularization is then modified by measuring distance within a locality-preserving lower-dimensional subspace. Furthermore, a competitive process among the classes is proposed to simplify parameter tuning. Classification results for several hyperspectral image data sets demonstrate superior performance of the proposed approach when compared to other, more traditional classification techniques.

Sparsity-preserving graph construction is investigated for the dimensionality reduction of hyperspectral imagery. In particular, a sparse graph-based discriminant analysis is proposed when labeled samples are available. By forcing the projection to be along the direction where a sample is clustered with within-class samples that best represented it, the discriminative power can be enhanced. The proposed method has no requirement on the number of labeled samples as in traditional linear discriminant analysis, and it can be solved by a simple generalized eigenproblem. The quality of the dimensionality reduction is evaluated by a support vector machine with a composite spatial-spectral kernel. Experimental results demonstrate that the proposed sparse graph-based discriminant analysis can yield superior classification performance with much lower dimensionality as compared to performance on the original data or on data transformed with other dimensionality-reduction approaches.

Compressed-sensing reconstruction using motion estimation and compensation for dynamic MRI data is proposed. Reconstruction is driven from a residual in the k-space domain between the current-frame measurements and a corresponding motion-compensated prediction. Due to the periodicity commonly exhibited in dynamic MRI, a telescopic motion search through the entire group of pictures is used to determine the best match for the block-based motion estimation. Experimental comparisons demonstrate improved performance as compared to existing dynamic-MRI reconstructions, both those with and without motion compensation.

Linear discriminant analysis (LDA) is a popular approach for dimensionality reduction for pattern classification; however, its performance is often degraded when samples are too few, particularly when the dimensionality of the input feature space is excessively high. The classic solution to the small-sample-size problem is to implement LDA in a principal component (PC) subspace, i.e., a strategy known as subspace LDA. This latter approach is extended by coupling LDA and noise-adjusted HSI analysis in order to provide noise-robust feature extraction and classification of high-dimensional data. An extension of the proposed approach in a kernel-induced space is also studied. The resulting noise-adjusted subspace discriminant analysis is evaluated using hyperspectral imagery, with experimental results demonstrating that the proposed approach provides not only superior classification performance, as compared with traditional methods, but also effective dimensionality reduction for classification even in the presence of noise.

Compressive-projection principal component analysis (CPPCA) has been developed to provide reconstruction from random projections of hyperspectral pixels and then subsequently extended by coupling it with classification such that the resulting class-dependent CPPCA yielded improved reconstruction performance. This letter provides an even greater integration of spatial and spectral information to further improve reconstruction performance. Specifically, instead of a pixel-based modulo partitioning employed by the original CPPCA sender, this work proposes an alternative block-based modulo partitioning, which preserves local spatial coherence; spatial segmentation is combined with the pixel-wise classification results using a majority voting rule at the receiver. Experimental results demonstrate not only improved reconstruction performance but also better detection of anomalies, as compared with previous approaches.

There is increasing interest in dimensionality reduction through random projections due in part to the emerging paradigm of compressed sensing. It is anticipated that signal acquisition with random projections will decrease signal-sensing costs significantly; moreover, it has been demonstrated that both supervised and unsupervised statistical learning algorithms work reliably within randomly projected subspaces. Capitalizing on this latter development, several class-dependent strategies are proposed for the reconstruction of hyperspectral imagery from random projections. In this approach, each hyperspectral pixel is first classified into one of several pixel groups using either a conventional supervised classifier or an unsupervised clustering algorithm. After the grouping procedure, a suitable reconstruction method, such as compressive projection principal component analysis, is employed independently within each group. Experimental results confirm that such class-dependent reconstruction, which employs statistics pertinent to each class as opposed to the global statistics estimated over the entire data set, results in more accurate reconstructions of hyperspectral pixels from random projections.

In lossy compression such as PCA+JPEG2000 for hyperspectral imagery, the bitrate is usually not fixed, resulting in various rate-distortion performance. In this paper, we propose an operational approach to determine the approximately optimal bitrate to be used to preserve both the majority of the information in the dataset as well as the anomalous pixels. The classification results using the reconstructed data after compression with this bitrate are comparable to those using the original data without compression; meanwhile, detection accuracy can be 100% if all the anomalies are pre-removed before compression.

Feature extraction based on nonnegative matrix factorization is considered for hyperspectral image classification. One shortcoming of most remote-sensing data is low spatial resolution, which causes a pixel to be mixed with several pure spectral signatures, or endmembers. To counter this effect, locality-preserving nonnegative matrix factorization is employed in order to extract an endmembers-based feature representation as well as to preserve the intrinsic geometric structure of hyperspectral data. Subsequently, a Gaussian mixture model classifier is employed in the induced-feature subspace. Experimental results demonstrate that the proposed classification system significantly outperforms traditional approaches even in instances of limited training data and severe pixel mixing.

Locality-preserving projection as well as local Fisher discriminant analysis is applied for dimensionality reduction of hyperspectral imagery based on both spatial and spectral information. These techniques preserve the local geometric structure of hyperspectral data into a low-dimensional subspace wherein a Gaussian-mixture-model classifier is then considered. In the proposed classification system, local spatial information—which is expected to be more multimodal than strictly spectral features—is used. Results with experimental hyperspectral data demonstrate that this system outperforms traditional classification approaches.

Hyperspectral imagery typically provides a wealth of information captured in a wide range of the electromagnetic spectrum for each pixel in the image; however, when used in statistical pattern-classification tasks, the resulting high-dimensional feature spaces often tend to result in ill-conditioned formulations. Popular dimensionality-reduction techniques such as principal component analysis, linear discriminant analysis, and their variants typically assume a Gaussian distribution. The quadratic maximum-likelihood classifier commonly employed for hyperspectral analysis also assumes single-Gaussian class-conditional distributions. Departing from this single-Gaussian assumption, a classification paradigm designed to exploit the rich statistical structure of the data is proposed. The proposed framework employs local Fisher's discriminant analysis to reduce the dimensionality of the data while preserving its multimodal structure, while a subsequent Gaussian mixture model or support vector machine provides effective classification of the reduced-dimension multimodal data. Experimental results on several different multiple-class hyperspectral-classification tasks demonstrate that the proposed approach significantly outperforms several traditional alternatives.

Hyperspectral imagery comprises high-dimensional reflectance vectors representing the spectral response over a wide range of wavelengths per pixel in the image. The resulting high-dimensional feature spaces often result in statistically ill-conditioned class-conditional distributions. Conventional methods for alleviating this problem typically employ dimensionality reduction such as linear discriminant analysis along with single-classifier systems, yet these methods are suboptimal and lack noise robustness. In contrast, a divide-and-conquer approach is proposed to address the high dimensionality of hyperspectral data for effective and noise-robust classification. Central to the proposed framework is a redundant wavelet transform for representing the data in a feature space amenable to noise-robust multiscale analysis as well as a multiclassifier and decision-fusion system for classification and target recognition in high-dimensional spaces under small-sample-size conditions. The proposed partitioning of this feature space assigns a collection of all coefficients across all scales at a particular spectral wavelength to a dedicated classifier. It is demonstrated that such a partitioning of the feature space for a multiclassifier system yields superior noise performance for classification tasks. Additionally, validation studies with experimental hyperspectral data show that the proposed system significantly outperforms conventional denoising and classification approaches.

Compressed-sensing methodology typically employs random projections simultaneously with signal acquisition to accomplish dimensionality reduction within a sensor device. The effect of such random projections on the preservation of anomalous data is investigated. The popular RX anomaly detector is derived for the case in which global anomalies are to be identified directly in the random-projection domain, and it is determined via both random simulation, as well as empirical observation that strongly anomalous vectors are likely to be identifiable by the projection-domain RX detector even in low-dimensional projections. Finally, a reconstruction procedure for hyperspectral imagery is developed wherein projection-domain anomaly detection is employed to partition the data set, permitting anomaly and normal pixel classes to be separately reconstructed in order to improve the representation of the anomaly pixels.

A simple block-based compressed-sensing reconstruction for still images is adapted to video. Incorporating reconstruction from a residual arising from motion estimation and compensation, the proposed technique alternatively reconstructs frames of the video sequence and their corresponding motion fields in an iterative fashion. Experimental results reveal that the proposed technique achieves significantly higher quality than a straightforward reconstruction that applies a still-image reconstruction independently frame by frame, a 3D reconstruction that exploits temporal correlation between frames merely in the form of a motion-agnostic 3D transform, and a similar, yet non-iterative, motion-compensated residual reconstruction.

The compressed-sensing recovery of video sequences driven by multihypothesis predictions is considered. Specifically, multihypothesis predictions of the current frame are used to generate a residual in the domain of the compressed-sensing random projections. This residual being typically more compressible than the original frame leads to improved reconstruction quality. To appropriately weight the hypothesis predictions, a Tikhonov regularization to an ill-posed least-squares optimization is proposed. This method is shown to outperform both recovery of the frame independently of the others as well as recovery based on single-hypothesis prediction.

Compressive-projection principal component analysis reconstructs vectors from random projections by recovering an approximation to the principal eigenvectors of the principal-component transform. A heuristic for the number of eigenvectors to approximate is developed to provide consistency with the Johnson-Lindenstrauss lemma and the restricted isometry property from compressed-sensing theory. The resulting heuristic is driven by only quantities known at the reconstruction side of the system. The heuristic is evaluated empirically for hyperspectral imagery and is demonstrated to provide near-optimal reconstruction quality.

Random projection for dimensionality reduction of hyperspectral imagery with a goal of target detection is investigated. Random projection is attractive in this task because it is data independent and computationally more efficient than other widely-used dimensionality-reduction methods, such as principal component analysis or the maximum-noise-fraction transform. Experimental results reveal that dimensionality reduction based on random projections yields improved target detection after decision fusion across multiple instances of the projections. Parallel implementation using a graphics processing unit is also investigated.

Linear discriminant analysis (LDA) has been widely applied for hyperspectral image (HSI) analysis as a popular method for feature extraction and dimensionality reduction. Linear methods such as LDA work well for unimodal Gaussian class-conditional distributions. However, when data samples between classes are nonlinearly separated in the input space, linear methods such as LDA are expected to fail. The kernel discriminant analysis (KDA) attempts to address this issue by mapping data in the input space onto a subspace such that Fisher's ratio in an intermediate (higher-dimensional) kernel-induced space is maximized. In recent studies with HSI data, KDA has been shown to outperform LDA, particularly when the data distributions are non-Gaussian and multimodal, such as when pixels represent target classes severely mixed with background classes. In this letter, a modified KDA algorithm, i.e., kernel local Fisher discriminant analysis (KLFDA), is studied for HSI analysis. Unlike KDA, KLFDA imposes an additional constraint on the mapping-it ensures that neighboring points in the input space stay close-by in the projected subspace and vice versa. Classification experiments with a challenging HSI task demonstrate that this approach outperforms current state-of-the-art HSI-classification methods.

The compression of multitemporal hyperspectral imagery is considered, wherein the encoder uses a reference image to effectuate temporal decorrelation for the coding of the current image. Both linear prediction and a spectral concatenation of images are explored to this end. Experimental results demonstrate that, when there are few changes between two images, the gain in rate-distortion performance is achieved over the independent coding of the current image. In addition, a strategy that explicitly removes salient temporal changes and stores them losslessly in the bitstream is proposed, and it is observed that this change-removal process results in a slight decrease in the rate-distortion performance with the benefit of perfect representation of the changed pixels.

Recent years have seen significant interest in the paradigm of compressed sensing (CS) which permits, under certain conditions, signals to be sampled at sub-Nyquist rates via linear projection onto a random basis while still enabling exact reconstruction of the original signal. As applied to 2D images, however, CS faces several challenges including a computationally expensive reconstruction process and huge memory required to store the random sampling operator. Recently, several fast algorithms have been developed for CS reconstruction, while the latter challenge was addressed by Gan using a block-based sampling operation as well as projection-based Landweber iterations to accomplish fast CS reconstruction while simultaneously imposing smoothing with the goal of improving the reconstructed-image quality by eliminating blocking artifacts. In this technique, smoothing is achieved by interleaving Wiener filtering with the Landweber iterations, a process facilitated by the relative simple implementation of the Landweber algorithm. In this work, we adopt Gan's basic framework of block-based CS sampling of images coupled with iterative projection-based reconstruction with smoothing. Our contribution lies in that we cast the reconstruction in the domain of recent transforms that feature a highly directional decomposition. These transforms---specifically, contourlets and complex-valued dual-tree wavelets---have shown promise to overcome deficiencies of widely-used wavelet transforms in several application areas. In their application to iterative projection-based CS recovery, we adapt bivariate shrinkage to their directional decomposition structure to provide sparsity-enforcing thresholding, while a Wiener-filter step encourages smoothness of the result. In experimental simulations, we find that the proposed CS reconstruction based on directional transforms outperforms equivalent reconstruction using common wavelet and cosine transforms. Additionally, the proposed technique usually matches or exceeds the quality of total-variation (TV) reconstruction, a popular approach to CS recovery for images whose gradient-based operation also promotes smoothing but runs several orders of magnitude slower than our proposed algorithm.

Compressed sensing is applied to multiview image sets and inter-image disparity compensation is incorporated into image reconstruction in order to take advantage of the high degree of inter-image correlation common to multiview scenarios. Instead of recovering images in the set independently from one another, two neighboring images are used to calculate a prediction of a target image, and the difference between the original measurements and the compressed-sensing projection of the prediction is then reconstructed as a residual and added back to the prediction in an iterated fashion. The proposed method shows large gains in performance over straightforward, independent compressed-sensing recovery. Additionally, projection and recovery are block-based to significantly reduce computation time.

In a multiview-imaging setting, image-acquisition costs could be substantially diminished if some of the cameras operate at a reduced quality. Compressed sensing is proposed to effectuate such a reduction in image quality wherein certain images are acquired with random measurements at a reduced sampling rate via projection onto a random basis of lower dimension. To recover such projected images, compressed-sensing recovery incorporating disparity compensation is employed. Based on a recent compressed-sensing recovery algorithm for images that couples an iterative projection-based reconstruction with a smoothing step, the proposed algorithm drives image recovery using the projection-domain residual between the random measurements of the image in question and a disparity-based prediction created from adjacent, high-quality images. Experimental results reveal that the disparity-based reconstruction significantly outperforms direct reconstruction using simply the random measurements of the image alone.

In this paper, we investigate the use of random-projection-based dimensionality reduction for hyperspectral endmember extraction. It is data-independent and computationally more efficient than other widely used dimensionality reduction methods, such as principal component analysis and maximum noise fraction transform. Based on the preliminary result, random-projection-based dimensionality reduction is capable of providing better endmembers after effective decision fusion.

Compressed sensing is applied to multiview image sets and the high degree of correlation between views is exploited to enhance recovery performance over straightforward independent view recovery. This gain in performance is obtained by recovering the difference between a set of acquired measurements and the projection of a prediction of the signal they represent. The recovered difference is then added back to the prediction, and the prediction and recovery procedure is repeated in an iterated fashion for each of the views in the multiview image set. The recovered multiview image set is then used as an initialization to repeat the entire process again to form a multistage refinement. Experimental results reveal substantial performance gains from the multistage reconstruction.

An analysis is presented that extends existing Rayleigh-Ritz theory to the special case of highly eccentric distributions. Specifically, a bound on the angle between the first Ritz vector and the orthonormal projection of the first eigenvector is developed for the case of a random projection onto a lower-dimensional subspace. It is shown that this bound is expected to be small if the eigenvalues are widely separated, i.e., if the data distribution is highly eccentric. This analysis verifies the validity of a fundamental approximation behind compressive projection principal component analysis,a technique proposed previously to recover from random projections not only the coefficients associated with principal component analysis but also an approximation to the principal-component transform basis itself.

Block-based random image sampling is coupled with a projection-driven compressed-sensing recovery that encourages sparsity in the domain of directional transforms simultaneously with a smooth reconstructed image. Both contourlets as well as complex-valued dual-tree wavelets are considered for their highly directional representation, while bivariate shrinkage is adapted to their multiscale decomposition structure to provide the requisite sparsity constraint. Smoothing is achieved via a Wiener filter incorporated into iterative projected Landweber compressed-sensing recovery, yielding fast reconstruction. The proposed approach yields images with quality that matches or exceeds that produced by a popular, yet computationally expensive, technique which minimizes total variation. Additionally, reconstruction quality is substantially superior to that from several prominent pursuits-based algorithms that do not include any smoothing.

High-dimensional data such as hyperspectral imagery is traditionally acquired in full dimensionality before being reduced in dimension prior to processing. Conventional dimensionality reduction on-board remote devices is often prohibitive due to limited computational resources; on the other hand, integrating random projections directly into signal acquisition offers alternative dimensionality reduction without sender-side computational cost. Effective receiver-side reconstruction from such random projections has been demonstrated previously using compressive-projection principal component analysis (CPPCA). While this prior work has focused on squared-error quality measures, the present work reports experimental results illustrating preservation of statistical class separation and anomaly-detection performance for CPPCA reconstruction following random-projection-based dimensionality reduction.

The performance of the recent JPEG2000 Part 10 standard, known as JP3D, is evaluated for the lossy and lossless compression of hyperspectral imagery. Experimental results using a Karhunen-Loève transform (KLT) for spectral decorrelation and a 2D wavelet transform for spatial decorrelation compare the performance of JP3D against 2D JPEG2000 as specified by Part 2 of the standard. JP3D is used with both the 2D arithmetic-coding contexts as specified in the JP3D standard as well as non-standard experimental 3D contexts. Results reveal that, while for lossless coding, JP3D very slightly surpasses the performance of JPEG2000 Part 2, for lossy coding, JP3D fails to match the rate-distortion performance of the 2D Part-2 coder.

Principal component analysis (PCA) is widely used for spectral decorrelation in the JPEG2000 compression of hyperspectral imagery. However, due to the data-dependent nature of principal components, the principal component transform matrix is stored in the JPEG2000 bitstream, constituting an overhead that is often negligible if the spatial size of the image is large. However, in parallel compression in which the data set is partitioned to multiple independent processing nodes, the overhead may no longer remain negligible. It is shown that a segmented approach to PCA can greatly mitigate the detrimental effects of transform-matrix overhead and can outperform wavelet-based decorrelation which entails no such overhead.

Reflectance data are often preferred to radiance data in applications of multispectral and hyperspectral imagery in which subtle spectral features are analyzed. In such applications, atmospheric correction, the process which provides radiance-to-reflectance conversion, plays a prominent role in the data-distribution and archiving pipeline. Lossy compression, often in the form of the JPEG2000 standard, will also likely factor into the distribution and archiving data flow. The relative position of data compression with respect to atmospheric correction is considered and evaluated with experimental results on both multispectral and hyperspectral imagery, and recommendations on an appropriate order for compression in the data-flow chain are made.

Principal component analysis (PCA) is often central to dimensionality reduction and compression in many applications, yet its data-dependent nature as a transform computed via expensive eigendecomposition often hinders its use in severely resource-constrained settings such as satellite-borne sensors. A process is presented that effectively shifts the computational burden of PCA from the resource-constrained encoder to a presumably more capable base-station decoder. The proposed approach, compressive-projection PCA (CPPCA), is driven by projections at the sensor onto lower-dimensional subspaces chosen at random, while the CPPCA decoder, given only these random projections, recovers not only the coefficients associated with the PCA transform, but also an approximation to the PCA transform basis itself. An analysis is presented that extends existing Rayleigh-Ritz theory to the special case of highly eccentric distributions; this analysis in turn motivates a reconstruction process at the CPPCA decoder that consists of a novel eigenvector reconstruction based on a convex-set optimization driven by Ritz vectors within the projected subspaces. As such, CPPCA constitutes a fundamental departure from traditional PCA in that it permits its excellent dimensionality-reduction and compression performance to be realized in an light-encoder/heavy-decoder system architecture. In experimental results, CPPCA outperforms a multiple-vector variant of compressed sensing for the reconstruction of hyperspectral data.

A method is proposed for the compression of hyperspectral signature vectors on severely resource-constrained encoding platforms. The proposed technique, compressive-projection principal component analysis, recovers from random projections not only transform coefficients but also an approximation to the principal-component basis, effectively shifting the computational burden of principal component analysis from the encoder to the decoder. In its use of random projections, the proposed method resembles compressed sensing but differs in that simple linear reconstruction suffices for coefficient recovery. Existing results from perturbation theory are invoked to argue for the robustness under quantization of the eigenvector-recovery process central to the proposed technique, and experimental results demonstrate a significant rate-distortion performance advantage over compressed sensing using a variety of popular bases.

We propose a new lossy compression algorithm for hyperspectral images, which is based on spectral principal component analysis (PCA), followed by JPEG2000 (JP2K). The approach employs an anomaly-removal model in the compression process to preserve anomalous pixels. Results on two different hyperspectral image scenes show that the new algorithm not only provides good post-compression anomaly-detection performance but also improves rate-distortion performance.

The high dimensionality of hyperspectral imagery challenges image processing and analysis. It has been shown that hyperspectral compression can be achieved by principal component analysis (PCA) for spectral decorrelation followed by the JPEG2000-based coding. This approach, referred to as PCA+JPEG2000, provides superior rate-distortion performance and can preserve useful data information. However, its main disadvantage is high computational complexity in the PCA process which entails the calculation of the data covariance matrix and its eigenvectors. Parallel processing is an appropriate approach to relieve the computation burden of such a PCA-based compression. In this paper, several parallel PCA implementations are proposed and their processing speed and resulting compression performance are investigated.

In this paper, we propose several improvements to the 3D-tarp coder for the lossy compression of hyperspectral imagery. Specific ameliorations include use of principal component analysis instead of a wavelet transform for spectral decorrelation, use of the quincunx wavelet transform instead of the traditional dyadic decomposition in the spatial direction, and spectral partitioning with skipping of insignificant zeros. Experimental results reveal that the enhanced coder achieves improved rate-distortion performance.

Lossy compression of hyperspectral imagery is considered, with special emphasis on the preservation of anomalous pixels. In the proposed scheme, anomalous pixels are extracted before compression and replaced with interpolation from surrounding nonanomalous pixels. The image is then coded using principal component analysis for spectral decorrelation followed by JPEG2000. The anomalous pixels do not participate in this lossy compression and are rather transmitted separately in a lossless fashion. Upon decoding, the anomalous pixels are inserted back into the image. Experimental results demonstrate that the proposed scheme improves not only anomaly detection performed subsequent to decoding but also the rate-distortion performance of the lossy-compression process.

An embedded lossy-to-lossless coder for hyperspectral images is presented. The proposed coder couples a reversible integer-valued Karhunen-Loeve transform with an extension into 3-D of the tarp-based coding with classification for embedding (TCE) algorithm that was originally developed for lossy coding of 2-D images. The resulting coder obtains lossy-to-lossless operation while closely matching the lossy performance of JPEG2000. Additionally, for lossless compression, it consistently outperforms not only JPEG2000 but, often, several prominent purely lossless methods.

In this paper presents a video-coding system that does not perform explicit ME/MC but instead relies on the DDWT to isolate moving signal features. To counteract the redundancy of the DDWT, a noise-shaping process increases the sparsity of the transform coefficients, resulting in a high degree of spatiotemporally coherent regions of insignificant coefficients. The transform coefficients are coded with binary set-partitioning using k-d trees (BISK) in an algorithm (DDWT-BISK) that exploits both the within-subband spatiotemporal coherency as well as cross-subband correlation to achieve efficient coding. Whereas a prior DDWT-based coder (DDWTVC2) exploits correlation as it exists across subbands in the DDWT, our DDWT-BISK coder additionally exploits coherent regions of insignificant coefficients that occur within subbands, a coherence that must necessarily be substantial due to the sparsity imposed by the noise-shaping process.

The geometric features of images, such as edges, are difficult to represent. When a redundant transform is used for their extraction, the compression challenge is even more difficult. In this paper we present a new rate-distortion optimization algorithm based on graph theory that can encode efficiently the coefficients of a critically sampled, non-orthogonal or even redundant transform, like the contourlet decomposition. The basic idea is to construct a specialized graph such that its minimum cut minimizes the energy functional. We propose to apply this technique for rate-distortion Lagrangian optimization in subband image coding. The method yields good compression results compared to the state-of-art JPEG2000 codec, as well as a general improvement in visual quality.

The complex dual-tree discrete wavelet transform is explored for the coding of hyperspectral imagery using a coder that has previously demonstrated efficient video-coding performance. A noise-shaping process increases the sparsity of the redundant transform-coefficient set, resulting in a high degree of regional coherency within the coefficient subbands. This coherency, as well as correlation across subbands, is exploited by a coding algorithm that performs set-partitioning using k-d trees. Prior experiments have shown that the proposed set-partitioning algorithm outperforms state-of-the-art JPEG2000 when coding video. However, experimental results indicate that the same coder fails to show similar gains when coding hyperspectral data, suggesting that hyperspectral data does not have properties that can be exploited by the increased directionality of the dual-tree transform.

Several linear transforms with constructions more general than that of principal component analysis are considered for spectral decorrelation in the compression of hyperspectral imagery. Specifically, orthogonal nonnegative matrix factorization, generalized principal component analysis, and principal component analysis coupled with explicit segmentation based on spectral angle mapping are considered. These spectral- decorrelation techniques are employed in conjunction with wavelet-based spatial decorrelation for hyperspectral compression using a 3D version of the well-known SPIHT algorithm. A shape-adaptive wavelet transform and shape-adaptive SPIHT coder are used in the case of the latter two spectral-decorrelation techniques which segment the hyperspectral dataset into multiple distinct pixel classes. Experimental results reveal that, despite their general formulation, the proposed techniques fail to offer spectral-decorrelation performance superior to that of traditional principal component analysis.

Principal component analysis (PCA) is deployed in JPEG2000 to provide spectral decorrelation as well as spectral dimensionality reduction. The proposed scheme is evaluated in terms of rate-distortion performance as well as in terms of information preservation in an anomaly-detection task. Additionally, the proposed scheme is compared to the common approach of JPEG2000 coupled with a wavelet transform for spectral decorrelation. Experimental results reveal that, not only does the proposed PCA-based coder yield rate-distortion and information-preservation performance superior to that of the wavelet-based coder, the best PCA performance occurs when a reduced number of PCs are retained and coded. A linear model to estimate the optimal number of PCs to use in such dimensionality reduction is proposed

A video-coding system that exploits the motion-selective characteristics of the 3-D complex dual-tree discrete wavelet transform is presented. The proposed system does not perform explicit motion compensation but instead relies on the dual-tree transform to isolate moving features. Although the dual-tree transform is redundant, a noise-shaping process increases the sparsity of the transform coefficients, resulting in a high degree of spatiotemporally coherent regions of insignificant coefficients. The transform coefficients are coded with binary set-partitioning using k-d trees in an algorithm that exploits both within-subband spatiotemporal coherency as well as cross-subband correlation to achieve efficient coding. Experimental results demonstrate that the proposed system outperforms other coders that also do not perform explicit motion estimation or compensation.

A joint source-channel coding scheme for transmission of video over flat Rayleigh fading channels is described. The coding scheme consists of a spatiotemporal motion-compensated wavelet decomposition, a vector quantization of the coefficients through maximum-diversity lattices, and a linear labeling which minimizes simultaneously the source and channel distortion. Modulation diversity via rotated constellations produces the maximum-diversity lattices which increase robustness to channel fading without the addition of redundancy. Experimental results compare the proposed system to a prominent scalable video coder protected by more traditional convolutional codes, and superior performance is observed for high levels of channel noise.

An analysis is presented that examines multihypothesis motion-compensated video coding using a redundant wavelet transform to produce multiple predictions that are diverse in transform phase. In such redundant-wavelet multihypothesis, the corresponding multiple-phase inverse transform implicitly combines the phase-diverse predictions into a single spatial-domain prediction. The performance advantage of this approach is investigated analytically, invoking the fact that the multiple-phase inverse involves a projection that significantly reduces the power of the noise not captured by the motion model. The analysis predicts that, under the assumption of a simple translational motion model, redundant-wavelet multihypothesis is capable of up to a 7-dB reduction in prediction-error variance over an equivalent single-phase, single-hypothesis approach. Experimental results support the performance advantage for real motion-compensation residuals.

Tarp filtering, an image coder with a simple implementation, is coupled with a block-based discrete cosine transform equipped with pre- and postfiltering. The prefilter reduces intra-block and inter-block correlation of the block-based coefficients, resulting in coefficients that are less correlated and thereby more suitable to tarp filtering. Experimental results show that the proposed coder achieves a significant improvement in rate-distortion performance as compared to the corresponding tarp coder in its original wavelet-based formulation for images with highly detailed content. A similar gain over JPEG2000 is seen for these same images, while, for images that are mostly smooth, the proposed coder performs comparably to JPEG2000

A simple and computationally lightweight video coder employing shape-adaptive, embedded intraframe coding and wavelet-domain conditional replenishment is proposed. Robustness to packet losses arises from packetization of the embedded bitstream with unequal error protection which is assigned to the packets with a fast, locally optimal procedure. Experimental results reveal that, when compared to H.264/AVC configured for low-complexity, error-resilient operation, not only does the proposed coder usually produce substantially superior rate-distortion performance as packet losses increase, it also achieves a significantly faster encoding speed.

A scalable video coder consisting of motion-compensated temporal filtering coupled with structured vector quantization plus a linear mapping of quantizer indexes that minimizes simultaneously source and channel distortions is presented. The linear index assignment takes the form of either a direct, uncoded mapping or a coded mapping via Reed-Muller codes. Experimental results compare the proposed system to a similar scheme using unstructured vector quantization as well as to a prominent scalable video coder protected by more traditional convolutional codes. The proposed system consistently outperforms the other two schemes by a significant margin for very noisy channel conditions.

Multihypothesis with phase diversity is introduced into motion-compensated temporal filtering by deploying the latter in the domain of a spatially redundant wavelet transform. The centerpiece of this redundant-wavelet approach to multihypothesis temporal filtering is a multiple-phase inverse transform that involves an implicit projection significantly reducing noise not captured by the motion model of the temporal filtering. The primary contribution of the work is a derivation that establishes analytically the advantage of the redundant-wavelet approach as compared to equivalent temporal filtering taking place in the spatial domain. For practical implementation, a regular triangle mesh is used to track motion between frames, and an affine transform between mesh triangles implements motion compensation within a lifting-based temporal transform. Experimental results reveal that the incorporation of phase-diversity multihypothesis into motion-compensated temporal filtering improves rate-distortion performance, and state-of-the-art scalable performance is observed.

Multihypothesis motion compensation has been widely used in video coding with previous attention focused on techniques employing predictions that are diverse spatially or temporally. In this paper, the multihypothesis concept is extended into the transform domain by using a redundant wavelet transform to produce multiple predictions that are diverse in transform phase. The corresponding multiple-phase inverse transform implicitly combines the phase-diverse predictions into a single spatial-domain prediction for motion compensation. The performance advantage of this redundant-wavelet-multihypothesis approach is investigated analytically, invoking the fact that the multiple-phase inverse involves a projection that significantly reduces the power of a dense-motion residual modeled as additive noise. The analysis shows that redundant-wavelet multihypothesis is capable of up to a 7-dB reduction in prediction-residual variance over an equivalent single-phase, single-hypothesis approach. Experimental results substantiate the performance advantage for a block-based implementation.

Motion compensation with redundant-wavelet multihypothesis, in which multiple predictions that are diverse in transform phase contribute to a single motion estimate, is deployed into the fully scalable MC-EZBC video coder. The bidirectional motion-compensated temporal-filtering process of MC-EZBC is adapted to the redundant-wavelet domain, wherein transform redundancy is exploited to generate a phase-diverse multihypothesis prediction of the true temporal filtering. Noise not captured by the motion model is substantially reduced, leading to greater coding efficiency. In experimental results, the proposed system exhibits substantial gains in rate-distortion performance over the original MC-EZBC coder for sequences with fast or complex motion.

Joint source-channel coding of scalable video using motion-compensated temporal filtering is considered. The proposed coding scheme consists of a structured vector quantizer based on lattice constellations and a linear index assignment, which minimizes simultaneously the channel and source distortions. Both uncoded linear index assignment as well as partially coded linear index assignment via Reed-Muller codes are considered. The proposed system is compared to an unstructured quantizer with minimax index assignment. Simulation results indicate that, for a Gaussian channel, the structured-codebook scheme is very robust, maintaining near-noiseless performance even when the channel is very noisy. Additionally, the proposed structured-quantizer scheme outperforms its unstructured counterpart when channel noise levels are high.

An algorithm is presented which implements image watermarking in the domain of an overcomplete, or redundant, wavelet transform. This algorithm expands on a previous method which employs a traditional, critically sampled wavelet transform coupled with perceptually-based watermark casting and optimal Neyman-Pearson detection. Specifically, in the proposed method, the redundancy inherent in the transform facilitates detection of perceptually salient texture local to a given spatial location and guides the placement of watermarking energy so as to minimize the impact on perceptual image quality. Additionally, the optimal detection strategy is adjusted to account for the overcomplete ness of the transform. The performance of the proposed technique is compared to that of its critically sampled counterpart and greater robustness under attack by compression is observed.

In using JPEG2000 for the coding of multiple-component, or multiband, images such as hyperspectral imagery, one must consider spectral decorrelation and rate allocation between image components, issues that concern the design of the JPEG2000 encoder and are, consequently, outside the scope of the JPEG2000 standard. Spectral decorrelation via a wavelet transform, as well as three alternative strategies for extending to multiple components the optimal codeblock-bitstream-truncation process widely used for spatial rate allocation in JPEG2000 coding of single-component imagery, are considered. Results indicate that the strategy of simultaneously truncating all code-block bitstreams from all codeblocks from all image components coupled with wavelet-based spectral decorrelation significantly outperforms the other techniques considered in terms of not only rate-distortion performance but also accuracy of unsupervised classification.

The behavior under additive noise of the redundant discrete wavelet transform (RDWT), which is a frame expansion that is essentially an undecimated discrete wavelet transform, is studied. Known prior results in the form of inequalities bound distortion energy in the original signal domain from additive noise in frame-expansion coefficients. In this letter, a precise relationship between RDWT-domain and original-signal-domain distortion for additive white noise in the RDWT domain is derived.

The binary set splitting with k-d trees (BISK) algorithm is introduced. An embedded wavelet-based image coder based on the popular bitplane-coding paradigm, BISK is designed specifically for the coding of image objects with arbitrary shape. While other similar algorithms employ quadtree-based set partitioning to code significance-map information, BISK uses a simpler and more flexible, binary decomposition via k-d trees. Additionally, aggressive discarding of transparent regions is implemented by shrinking sets to the bounding box of their constituent opaque coefficients before further partitioning. Empirical results indicate that the proposed BISK coder consistently yields efficient performance when compared to a variety of other shape-adaptive coders.

An embedded wavelet-based coder for the shape-adaptive coding of ocean-temperature data is described. The proposed coder, 3D binary set splitting with k-d trees (3D-BISK), is based upon the popular bitplane-coding paradigm and is specifically designed for shape-adaptive coding. Other similar coding methods use octree-based set partitioning; however, 3D-BISK employs a simpler set decomposition based on k-d trees which makes it more flexible when considering shape-adaptive coding. The performance of 3D-BISK is compared to prominent shape-adaptive coders and superior performance is demonstrated for a variety of ocean-temperature datasets.

An embedded wavelet-based coder for the compression of hyperspectral imagery is described. The proposed coder, three-dimensional (3-D) tarp, employs an explicit estimate of the probability of coefficient significance to drive a nonadaptive arithmetic coder, resulting in a simple implementation suited to vectorized acceleration in single-instruction-multiple-data (SIMD) hardware. The proposed 3-D tarp coder is compared to other prominent coders for the compression of hyperspectral imagery, and state-of-the-art rate-distortion performance is observed.

Geoscience applications often produce sizable datasets that are vector-valued and increasingly in need of compression algorithms to reduce storage and transmission burdens, particularly when the data are time-varying. In this paper, several advanced interframe-compression techniques are extended from the traditional realm of natural video to the coding of time-varying vector fields. Although similar to natural video in some respects, time-varying vector-field sequences often possess complex temporal evolution of vector-valued features that are important to the analytic quality of the data yet defy the simple motion models widely employed for natural video. To improve coding performance, motion compensation with reduced resolution is proposed such that motion compensation is applied only at low spatial resolution, while high-resolution information, for which the motion model fails, is intraframe coded with no temporal decorrelation. In empirical results on datasets of ocean-surface winds, this reduced-resolution motion-compensation technique results in significant performance improvement and greater feature preservation.

Multihypothesis motion compensation is extended into the transform domain by using a redundant wavelet transform to produce multiple predictions that are diverse in transform phase. The corresponding inverse transform implicitly combines the multihypothesis predictions into a single spatial-domain prediction for motion compensation such that no side information is needed to describe the combination weights. Additionally, we use a hierarchical search to tailor the motion-vector field to individual phases. Substantial gains in rate-distortion performance are obtained in comparison to an equivalent system using single-phase prediction.

An embedded wavelet coder is proposed for arbitrarily shaped image objects. Contrary to the popular approach of considering transparent regions surrounding image objects as permanently insignificant, the proposed coder imposes no assumptions on the significance state of unknown transparent regions. Instead, the proposed shape-adaptive tarp coder employs a simple algorithm called tarp filtering to produce a probability estimate of coefficient significance which propagates unchanged across transparent regions. Experimental results show that the shape-adaptive tarp coder offers coding performance superior to a popular zerotree-based technique and equivalent to a popular method based on context conditioning.

A video coder is presented that combines mesh-based motion-compensated temporal filtering, phase-diversity multihypothesis motion compensation, and an embedded 3D wavelet-coefficient coder. The key contribution of this work is the introduction of the phase-diversity multihypothesis paradigm into motion-compensated temporal filtering, which is achieved by deploying temporal filtering in the domain of a spatially redundant wavelet transform. A regular triangle mesh is used to track motion between frames, and an affine transform between mesh triangles implements motion compensation within a lifting-based temporal transform. Experimental results reveal that the incorporation of pulse-diversity multihypothesis into mesh-based motion-compensated temporal filtering significantly improves the rate-distortion performance of the 3D video coder.

An embedded wavelet-based coder for the compression of hyperspectral imagery is described. The proposed coder, 3D tarp, employs an explicit estimate of the probability of coefficient significance to drive a nonadaptive arithmetic coder, resulting in a simple implementation suited to resourcelimited on-board processing. The performance of the proposed 3D tarp coder is compared to that of other prominent coders for the compression of hyperspectral imagery, and state-of-the-art performance is observed.

Vector wavelet transforms for vector-valued fields can be implemented directly from multiwavelets; however, existing multiwavelets offer surprisingly poor performance for transforms in vector-valued signal-processing applications. In this paper, the reason for this performance failure is identified, and a remedy is proposed. A multiwavelet design criterion, omnidirectional balancing, is introduced to extend to vector transforms the balancing philosophy previously proposed for multiwavelet-based scalar-signal expansion. Additionally, a family of symmetric-antisymmetric multiwavelets is designed according to the omnidirectional-balancing criterion. In empirical results for a vector-field compression system, it is observed that the performance of vector wavelet transforms derived from these omnidirectionally-balanced symmetric-antisymmetric multiwavelets is far superior to that of transforms implemented via other multiwavelets.

Many applications in a variety of scientific domains produce datasets that consist of a data field lying on a sampling grid that may not be uniformly spaced. However, progressive access for visualization, exploration, and communication of these datasets is a critical issue, and wavelet-based embedded coding an attractive solution given its prior success in realms such as image coding. As grid information may compose a significant portion, or even a majority, of that of the overall dataset, coding the grid as initial overhead is often impractical. Approaches for the joint embedded coding of data and grid are proposed using first-generation wavelet transforms so as not to require prior grid knowledge for transform inversion. In one proposed technique, two independently generated embedded codings, one for data and one for grid, are interleaved. As an alternative, an embedded vector-valued coder, in which data and grid are combined into a single vector-valued field, is considered. Experimental results are reported that favor the former approach over the latter.

A technique is presented that incorporates an irregular triangle mesh into wavelet-domain motion-estimation and motion-compensation using a shift-invariant redundant wavelet transform. Triangle vertices are identified by a simple correlation operator locating image edges in the wavelet subbands, while motion compensation takes place through an affine transformation mapping triangles from one frame to the next. The motion-compensated residual is downsampled to a non-redundant form which is then coded using any wavelet-based still-image coder. Experimental results indicate that the combined approach outperforms either technique applied separately,, in addition, the proposed method outperforms a variety of motion-estimation and motion-compensation approaches operating in both the spatial and wavelet domains.

Spread-spectrum watermarking, in which random noise is added to transform coefficients and detected with a correlation operator has become a preferred paradigm for many watermarking applications. This paper analyzes the performance of such a watermarking system when the underlying transform is a tight frame rather than a traditional orthonormal expansion. The analysis indicates that a tight frame offers no inherent performance advantage over an orthonormal transform in the watermark-detection process despite the well known ability of redundant transforms to accommodate greater amounts of added noise for a given distortion.

An image-adaptive watermarking technique based upon a redundant wavelet transform is proposed. The redundant transform provides an overcomplete representation of the image which facilitates the identification of significant image features via a simple correlation operation across scales. Although the watermarking algorithm is image adaptive, it is not necessary for the original image to be available for successful detection of the watermark. The performance and robustness of the proposed technique is tested by applying common image-processing operations such as filtering, requantization, and JPEG compression. A quantitative measure is proposed to objectify performance; under this measure, the proposed technique outperforms a wavelet scheme based on the usual critically sampled DWT.

We describe the wavelets around land masses (WAVAL) system for the embedded coding of three-dimensional (3-D) oceanographic images. These images differ from those arising in other applications in that valid data exists only at grid points corresponding to sea. Grid points that cover land or lie beyond the bathymetry have no associated data. For these images, the WAVAL system employs a 3-D lifting wavelet transform tailored specifically to the potentially sparse nature of the data by processing only the valid sea data points between land masses. We introduce successive-approximation runlength (SARL) coding, an embedded-coding procedure that adds successive-approximation properties to the well known stack-run (SR) algorithm. SARL is employed to code wavelet coefficients resulting from the 3-D transform in the WAVAL system. However, it is a general technique applicable to other coding tasks in which embedded coding is desired but for which zerotree techniques are impractical. Experimental results show that the WAVAL system achieves substantial improvement in rate-distortion performance over the technique currently used by the US Navy for compression of oceanographic imagery.

Summary form only given. We describe the QccPack software package, an open-source collection of library routines and utility programs for quantization, compression, and coding of data. QccPack is written to expedite data compression research and development by providing general and reliable implementations of common compression techniques. Functionality of the current release includes entropy coding, scalar quantization, vector quantization, adaptive vector quantization, wavelet transforms and subband coding, error-correcting codes, image processing support, and general vector-mathematics, matrix-mathematics, file-I/O, and error-message routines. All QccPack functionality is accessible via library calls; additionally, many utility programs provide command-line access. Although primary development efforts have concentrated on the Red Hat Linux i386 platform, it should be a straightforward procedure to build QccPack on other varieties of Linux as well as non-Linux UNIX-based systems.

We describe an algorithm for the embedded coding of 3D oceanographic images. These images differ from those arising in other applications in that valid data exist only at grid points corresponding to the sea; grid points that cover land or lie beyond the bathymetry have no associated data. For these images, we employ a 3D lifting wavelet transform tailored specifically to the potentially sparse nature of the data by processing only the valid sea data points in between land masses. In addition, we introduce successive-approximation runlength (SARL) coding, an embedded-coding procedure which adds successive-approximation properties to the well known stack-run (SR) algorithm. SARL is a general technique applicable to the oceanographic images considered here as well as to other coding tasks in which embedded coding is desired but for which zerotree-techniques are impractical.

A new system for intraframe coding of video is described. This system combines zerotrees of vectors of wavelet coefficients and the generalized-threshold-replenishment (GTR) technique for adaptive vector quantization (AVQ). A data structure, the vector zerotree (VZT), is introduced to identify trees of insignificant vectors, i.e., those vectors of the wavelet coefficients in a dyadic subband decomposition that are to be coded as zero. GTR coders are then applied to each subband to efficiently code the significant vectors by way of adapting to their changing statistics. Both VZT generation and GTR coding are based upon minimization of criteria involving both rate and distortion. In addition, the perceptual performance is improved by invoking simple, perceptually motivated weighting in both the VZT and the GTR coders. Our experimental findings indicate that the described VZTGTR system handles dramatic changes in image statistics, such as those due to a scene change, more efficiently than wavelet-based techniques employing nonadaptive scalar quantizers.

Summary form only given. We present a new algorithm for intraframe coding of video which combines zerotrees of vectors of wavelet coefficients and the generalized-threshold-replenishment (GTR) technique for adaptive vector quantization (AVQ). A data structure, the vector zerotree (VZT), is introduced to identify trees of insignificant vectors, i.e., those vectors of wavelet coefficients in a dyadic subband decomposition that are to be coded as zero. GTR coders are then applied to each subband to efficiently code the significant vectors by way of adapting to their changing statistics. Both VZT generation anti GTR coding are based upon minimization of criteria involving both rate and distortion. In addition, perceptual performance is improved by invoking simple, perceptually motivated weighting in both the VZT and the GTR coders.

A new system for intraframe coding of video is described. This system combines zerotrees of vectors of wavelet coefficients and the generalized-threshold-replenishment (GTR) technique for adaptive vector quantization (AVQ). A data structure, the vector zerotree (VZT), is introduced to identify trees of insignificant vectors, i.e., those vectors of wavelet coefficients in a dyadic subband decimalisation that are to be coded as zero. The GTR coders are then applied to each subband to efficiently code the significant vectors by way of adapting to their changing statistics. Both VZT generation and GTR coding are based upon minimization of criteria involving both rate and distortion. In addition, perceptual performance is improved by invoking simple, perceptually motivated weighting in both the VZT and the GTR coders. Our experimental findings indicate that the described VZTGTR system handles dramatic changes in image statistics, such as those due to a scene change more efficiently than a scalar zerotree technique employing a nonadaptive scalar quantizer.

In this paper, we describe a new adaptive-vector-quantization (AVQ) algorithm designed for the coding of non-stationary sources. This new algorithm, generalized threshold replenishment (GTR), differs from prior AVQ algorithms in that it features an explicit, online consideration of both rate and distortion. Because of its online nature, GTR is more amenable to real-time hardware and software implementation than are many prior AVQ algorithms that rely on traditional batch training methods. Additionally, as rate-distortion cost criteria are used in both the determination of nearest-neighbor codewords and the decision to update the codebook, GTR achieves rate-distortion performance superior to that of other AVQ algorithms, particularly for low-rate coding. Results are presented that illustrate low-rate performance surpassing that of other AVQ algorithms for the coding of both an image sequence and an artificial non-stationary random process. For the image sequence, it is shown that (1) most AVQ algorithms achieve distortion much lower than that of nonadaptive VQ for the same rate (about 1.5 b/pixel), and (2) GTR achieves performance substantially superior to that of the other AVQ algorithms for low-rate coding, being the only algorithm to achieve a rate below 1.0 b/pixel.

In this paper, we describe a new adaptive vector quantization (AVQ) algorithm designed for the coding of nonstationary sources. This new algorithm, generalized threshold replenishment (GTR), differs from prior AVQ algorithms in that it features an explicit, online consideration of both rate and distortion. Rate-distortion cost criteria are used in both the determination of nearest-neighbor codewords and the decision to update the codebook. Results presented indicate that, for the coding of an image sequence, (1) most AVQ algorithms achieve distortion much lower than that of nonadaptive VQ for the same rate (about 1.5 bits/pixel), and (2) the GTR algorithm achieves rate-distortion performance substantially superior to that of the prior AVQ algorithms for low-rate coding, being the only algorithm to achieve a rate below 1.0 bits/pixel.

We describe a new adaptive vector quantization (AVQ) algorithm designed for the coding of nonstationary sources. This new algorithm, generalized threshold replenishment (GTR), differs from prior AVQ algorithms in that it features an explicit, online consideration of both rate and distortion. Rate-distortion cost criteria are used in the determination of nearest-neighbor codewords and as well as in the decision to update the codebook. Results presented indicate that, for the coding of an image sequence: (1) most AVQ algorithms achieve distortion much lower than that of nonadaptive VQ for the same rate (about 1.5 bits/pixel), and (2) the GTR algorithm achieves rate-distortion performance substantially superior to that of other AVQ algorithms for low-rate coding, being the only algorithm to achieve a rate below 1.0 bits/pixel.

This paper describes hardware that has been built to compress video in real time using full-search vector quantization (VQ). This architecture implements a differential-vector-quantization (DVQ) algorithm and features a special-purpose digital associative memory, the VAMPIRE chip, which has been fabricated in 2 /spl mu/m CMOS. We describe the DVQ algorithm, its adaptations for sampled NTSC composite-color video, and details of its hardware implementation. We conclude by presenting both numerical results and images drawn from real-time operation of the DVQ hardware.>

Describes hardware that has been built to compress video in real time using full-search vector quantization (VQ). This architecture implements a differential-vector-quantization (DVQ) algorithm and features a special-purpose digital associative memory, the VAMPIRE chip, which has been fabricated in 2 /spl mu/m CMOS. The authors describe the DVQ algorithm, its adaptations for sampled NTSC composite-color video, and details of its hardware implementation. They conclude by presenting images drawn from real-time operation of the DVQ hardware.>

The authors demonstrate the use of a differential vector quantization (DVQ) architecture for the coding of digital images. An artificial neural network is used to develop entropy-biased codebooks which yield substantial data compression without entropy coding and are very robust with respect to transmission channel errors. Two methods are presented for variable bit-rate coding using the described DVQ algorithm. In the first method, both the encoder and the decoder have multiple codebooks of different sizes. In the second, variable bit-rates are achieved by using subsets of one fixed codebook. The performance of these approaches is compared, under conditions of error-free and error-prone channels. Results show that this coding technique yields pictures of excellent visual quality at moderate compression rate.>

The paper describes a differential-vector-quantization (DVQ) algorithm that has been developed for realtime compression of digital video images. The authors discuss a number of experiments performed to insure that the algorithm is suitable for hardware implementation. They show that the DVQ algorithm performs comparably to scalar differential-pulsecode-modulation (DPCM) methods in terms of image quality and compression. They also show that the DVQ algorithm is much more robust in the presence of channel noise than alternatives employing variable-length encoding.>

Abstract An efficient method for simulating instruction sets is described. The method allows for compiled instruction set simulation using the macro expansion capabilities found in many languages. Additionally, we show how the semantics of the C case statement allows instruction branching to be incorporated in an efficient manner. The method is compared with conventional interpreted techniques and is shown to offer considerable performance benefits.