We propose UniPose+, a unified framework for 2D and 3D human pose estimation in images and videos. The UniPose+ architecture leverages multi-scale feature representations to increase the effectiveness of backbone feature extractors, with no significant increase in network size and no postprocessing. Current pose estimation methods heavily rely on statistical postprocessing or predefined anchor poses for joint localization. The UniPose+ framework incorporates contextual information across scales and joint localization with Gaussian heatmap modulation at the decoder output to estimate 2D and 3D human pose in a single stage with state-of-the-art accuracy, without relying on predefined anchor poses. The multi-scale representations allowed by the waterfall module in the UniPose+ framework leverage the efficiency of progressive filtering in the cascade architecture, while maintaining multi-scale fields-of-view comparable to spatial pyramid configurations. Our results on multiple datasets demonstrate that UniPose+, with a HRNet, ResNet or SENet backbone and waterfall module, is a robust and efficient architecture for single person 2D and 3D pose estimation in single images and videos.

Video captioning is a challenging task as it needs to accurately transform visual understanding into natural language description. To date, state-of-the-art methods inadequately model global-local vision representation for sentence generation, leaving plenty of room for improvement. In this work, we approach the video captioning task from a new perspective and propose a GLR framework, namely a global-local representation granularity. Our GLR demonstrates three advantages over the prior efforts. First, we propose a simple solution, which exploits extensive vision representations from different video ranges to improve linguistic expression. Second, we devise a novel global-local encoder, which encodes different video representations including long-range, short-range and local-keyframe, to produce rich semantic vocabulary for obtaining a descriptive granularity of video contents across frames. Finally, we introduce the progressive training strategy which can effectively organize feature learning to incur optimal captioning behavior. Evaluated on the MSR-VTT and MSVD dataset, we outperform recent state-of-the-art methods including a well-tuned SA-LSTM baseline by a significant margin, with shorter training schedules. Because of its simplicity and efficacy, we hope that our GLR could serve as a strong baseline for many video understanding tasks besides video captioning. Code will be available.

We propose UniPose, a unified framework for human pose estimation, based on our “Waterfall” Atrous Spatial Pooling architecture, that achieves state-of-art-results on several pose estimation metrics. UniPose incorporates contextual segmentation and joint localization to estimate the human pose in a single stage, with high accuracy, without relying on statistical postprocessing methods. The Waterfall module in UniPose leverages the efficiency of progressive filtering in the cascade architecture, while maintaining multi-scale fields-of-view comparable to spatial pyramid configurations. Additionally, our method is extended to UniPose-LSTM for multi-frame processing and achieves state-of-the-art results for temporal pose estimation in Video. Our results on multiple datasets demonstrate that UniPose, with a ResNet backbone and Waterfall module, is a robust and efficient architecture for pose estimation obtaining state-of-the-art results in single person pose detection for both single images and videos.

Synthetic Aperture Radar (SAR) imagery has diverse applications in land and marine surveillance. Unlike electro-optical (EO) systems, these systems are not affected by weather conditions and can be used in the day and night times. With the growing importance of SAR imagery, it would be desirable if models trained on widely available EO datasets can also be used for SAR images. In this work, we consider transfer learning to leverage deep features from a network trained on an EO ships dataset and generate predictions on SAR imagery. Furthermore, by exploring the network activations in the form of class-activation maps (CAMs), we visualize the transfer learning process to SAR imagery and gain insight on how a deep network interprets a new modality.

We propose a data-driven approach for deep convolutional neural network compression that achieves high accuracy with high throughput and low memory requirements. Current network compression methods either find a low-rank factorization of the features that requires more memory or select only a subset of features by pruning entire filter channels. We propose the Cascaded Projection (CaP) compression method that projects the output and input filter channels of successive layers to a unified low dimensional space based on a low-rank projection. We optimize the projection to minimize classification loss and the difference between the next layer's features in the compressed and uncompressed networks. To solve this non-convex optimization problem we propose a new optimization method of a proxy matrix using backpropagation and Stochastic Gradient Descent (SGD) with geometric constraints. Our cascaded projection approach leads to improvements in all critical areas of network compression: high accuracy, low memory consumption, low parameter count and high processing speed. The proposed CaP method demonstrates state of the art results compressing VGG16 and ResNet networks with over 4X reduction in the number of computations and excellent performance in top-5 accuracy on the ImageNet dataset before and after fine-tuning.

In this letter, we propose a fast Grassmann manifold optimization method for L1-norm based principal component analysis (GM-L1-PCA). Our approach is a two-step iterative costminimization and manifold retraction technique that efficiently finds all principal components simultaneously. We perform complexity analysis and show that GM-L1-PCA achieves a significant reduction in processing time while obtaining comparable or better results to current state-of-the-art L1-PCA methods. We further demonstrate the improvement of GM-L1-PCA technique over L2-PCA on a dataset of facial imagery corrupted with outlying data points. Our experiments show that GM-L1-PCA is computationally more efficient and produces results with lower reprojection error than previous methods. Furthermore, the processing time of our approach is relatively independent of dataset size and well suited for various big-data problems commonly encountered today.

The enormous success and popularity of deep convolutional neural networks for object detection has prompted their deployment in various real world applications. However, their performance in the presence of hardware faults or damage that could occur in the field has not been studied. This paper explores the resiliency of three popular network architectures for object detection, AlexNet, VGG and ResNet50, when subjected to various degrees of failures. We introduce failures in a deep network by dropping a percentage of weights at each layer. We then assess the effects of these failures on classification performance. Finally, we determine the ability of the network to self-heal and recover its performance by retraining its healthy portions after partial damage. To our knowledge, this is the first time that this type of study has been conducted.

Image captioning has demonstrated models that are capable of generating plausible text given input images or videos. Further, recent work in image generation has shown significant improvements in image quality when text is used as a prior. Our work ties these concepts together by creating an architecture that can enable bidirectional generation of images and text. We call this network Multi-Modal Vector Representation (MMVR). Along with MMVR, we propose two improvements to the text conditioned image generation. Firstly, a n-gram metric based cost function is introduced that generalizes the caption with respect to the image. Secondly, multiple semantically similar sentences are shown to help in generating better images. Qualitative and quantitative evaluations demonstrate that MMVR improves upon existing text conditioned image generation results by over 20%, while integrating visual and text modalities.

Recent work has demonstrated that neural embedding from multiple modalities can be utilized to focus the results of generative adversarial networks. However, little work has been done towards developing a procedure to combine vectors from different modalities for the purpose of reconstructing input. Generally, embeddings from different modalities are concatenated to create a larger input vector. In this paper, we propose learning a Common Vector Space (CVS) where similar inputs from different modalities cluster together. We develop a framework to analyze the extent of reconstruction and robustness offered by CVS. We apply the CVS for the purpose of annotating, generating and captioning images on MS-COCO. We show that CVS is on par with techniques used for multiple modality embeddings while offering more flexibility as the number of modalities increases.

Visual odometry is a challenging task related to simultaneous localization and mapping that aims to generate a map traveled from a visual data stream. Based on one or two cameras, motion is estimated from features and pixel differences between frames. Because of the frame rate of the cameras, there are generally small, incremental changes between subsequent frames where optical flow can be assumed to be proportional to the physical distance moved by an egocentric reference, such as a camera on a vehicle. In this paper, a visual odometry system called Flowdometry is proposed based on optical flow and deep learning. Optical flow images are used as input to a convolutional neural network, which calculates a rotation and displacement for each image pixel. The displacements and rotations are applied incrementally to construct a map of where the camera has traveled. The proposed system is trained and tested on the KITTI visual odometry dataset, and accuracy is measured by the difference in distances between ground truth and predicted driving trajectories. Different convolutional neural network architecture configurations are tested for accuracy, and then results are compared to other state-of-the-art monocular odometry systems using the same dataset. The average translation error from the Flowdometry system is 10.77% and the average rotation error is 0.0623 degrees per meter. The total execution time of the system per optical flow frame is 0.633 seconds, which offers a 23.796x speedup over state-of-the-art methods using deep learning.

Time-frequency (TF) representations have been widely used over the past decade to characterize the non-stationary content of signals in the joint time and frequency domain. Although a number of effective TF analysis methods based on wavelet or Gabor transform have been developed, these methods use pre-determined basis functions and still require feature extraction methods to reduce redundancy and preserve important TF information related to the application of interest. This paper explores a novel TF feature extraction algorithm using a modified dictionary learning approach. The proposed algorithm is developed to modify learned dictionaries and derive TF features unique to each class. It emulates the way joint dictionary learning algorithms use common dictionaries to promote discrimination between the data from different classes, thereby allowing for an improved analysis of complex and multitask data. The proposed method indicated a significant performance in identification of the discriminant vs. common structures of the TF data.

We present a biologically motivated manifold learning framework for image set classification inspired by Independent Component Analysis for Grassmann manifolds. A Grassmann manifold is a collection of linear subspaces, such that each subspace is mapped on a single point on the manifold. We propose constructing Grassmann subspaces using Independent Component Analysis for robustness and improved class separation. The independent components capture spatially local information similar to Gabor-like filters within each subspace resulting in better classification accuracy. We further utilize linear discriminant analysis or sparse representation classification on the Grassmann manifold to achieve robust classification performance. We demonstrate the efficacy of our approach for image set classification on face and object recognition datasets.

Presents a listing of the new Associate Editors for this issue of the publication.

Classification on Grassmann manifolds has found application in computer vision problems because it yields improved accuracy and fast computation times. Grassmann manifolds map subspaces to single points, which involves solving for a unit vector representation that is obtained using principal component analysis (PCA). However, PCA may suffer from the presence of outliers due to noise and occlusions often encountered in unconstrained settings. We address this problem by introducing L1-Grassmann manifolds where L1-PCA is used for subspace generation during the mapping process. We utilize a new approach to L1-PCA and demonstrate the effectiveness of L1-Grassmann manifolds for robust face recognition. Results using the Yale face database and the ORL database of faces show that L1-Grassmann manifolds outperform traditional L1-Grassmann manifolds for face recognition and are more robust to noise and occlusions.

Support Vector Machines (SVM) are effective classification engines used in a large number of applications that stand to benefit from acceleration. OpenCL is a software platform specification for parallel programming that supports heterogeneous computing on a wide range of devices including GPUs, FPGAs, and multicore CPUs. In this paper, we present an accelerated implementation of SVM using a heterogeneous computing system programmed using OpenCL. The popular LIBSVM, an open source implementation of SVM, is used as the basis for our system, which allows the presented work to be integrated seamlessly into existing environments. The proposed framework is evaluated in terms of speed and accuracy for training and classification. Testing was based on two GPUs, the NVIDIA GTX 480 and Tesla K20, and compared to the serial implementation based on the Intel i5 Quad Core, 3 GHz, CPU. We find that SVM training is accelerated by a factor ranging from 9 to 22, and SVM classification is accelerated by a factor of up to 12.

With the introduction of low cost depth sensors, gesture recognition systems for computer interface control are becoming a reality and require accurate recognition in real time. In this paper, we introduce a salient feature for gesture recognition called active difference signature, obtained by robust processing of depth maps and kinematic joint information. This feature is classified with variants of semi-supervised Linear extension of Graph Embedding and sparse representations to demonstrate advanced gesture control using depth cameras. Our active difference signature technique delivers highly accurate gesture recognition that is invariant to subject distance to camera, subject size, subject location within the camera field of view, and speed at which the subject performs the gestures.

The parsimonious nature of sparse representations has been successfully exploited for the development of highly accurate classifiers for various scientific applications. Despite the successes of Sparse Representation techniques, a large number of dictionary atoms as well as the high dimensionality of the data can make these classifiers computationally demanding. Furthermore, sparse classifiers are subject to the adverse effects of a phenomenon known as coefficient contamination, where, for example, variations in pose may affect identity and expression recognition. We analyze the interaction between dimensionality reduction and sparse representations, and propose a technique, called Linear extension of Graph Embedding K-means-based Singular Value Decomposition (LGE-KSVD) to address both issues of computational intensity and coefficient contamination. In particular, the LGE-KSVD utilizes variants of the LGE to optimize the K-SVD, an iterative technique for small yet over complete dictionary learning. The dimensionality reduction matrix, sparse representation dictionary, sparse coefficients, and sparsity-based classifier are jointly learned through the LGE-KSVD. The atom optimization process is redefined to allow variable support using graph embedding techniques and produce a more flexible and elegant dictionary learning algorithm. Results are presented on a wide variety of facial and activity recognition problems that demonstrate the robustness of the proposed method.

Active Shape Models (ASMs) have been widely used in facial feature representation and related applications. In this paper, we exploit the availability of registered depth and color information from a low resolution camera sensor to improve Active Shape Model fitting accuracy and efficiency. Two independent Active Shape Model profile models are constructed based on the gradient profiles of depth and color intensity information. These two channels are fused using full or partial depth information. The algorithm determines the optimal point localization using the color/depth profiles during the update step of the ASM fitting process. Results show improvements in accuracy over standard methods using color intensity or depth information alone.

The field of sparse representations has found applications in a variety of computer vision and scientific fields. Although sparse representations were initially considered for reconstruction, they have been successfully adapted for classification. In this paper, we demonstrate the application of sparse representations in matching salient keypoint descriptors for image alignment, registration, or stitching. Our method initially builds a dictionary from keypoints in a reference image. Then keypoints associated with one or more secondary images are sparsely represented using the reference dictionary. The sparse coefficient signatures are analyzed to determine if there is a matched pair and identify the reference keypoint of the match. Top keypoint matches are used to construct the homography transformation for image registration. The usefulness of our methodology is demonstrated across several types of imagery showing robust performance while delivering state-of-the-art image alignment and registration.

Action recognition is an important research area that is particularly challenging when dealing with view independent and unconstrained human motion. While progress has been made in developing pose-dependent action classification systems, the introduction of affordable 3D sensors has opened up opportunities for action classification with depth data. In this paper, we propose an efficient 3D descriptor combining radial distance measures on 2D video sequences with 3D joint tracking on depth data for action classification through Manifold Learning using supervised Locality Preserving Projections (sLPP). We find that the application of radial distances on depth data is effective at classifying actions and when combined with 3D joint tracking the action classification performance improves. We applied our method on the Microsoft Research 3D Dataset (MSR3D) and obtained good classification accuracy on all 20 unique 3D actions. Activity recognition rates were as high as 98.95% on subsets of 3D actions.

Facial expression research on unconstrained spontaneous expressions has benefited from recent advances in feature extraction, dimensionality reduction, and classification techniques. While the facial action coding relies on temporal predictors, state-of-the-art facial expression recognition techniques have been slow to adapt to temporal methods. Further, despite strong evidence of sparsity in the visual cortex, few approaches to facial understanding utilize sparse representations. This paper proposes using a temporal detector based upon facial dynamics, a static expression detector based on sparse representations, and an advanced temporal-sparse fused emotion estimator. Our approach leverages techniques from both computer vision and human brain research to produce a state-of-the-art emotion estimator on unconstrained faces in natural conditions.

Research on facial expression recognition has steadily been moving from analysis of deliberative frontal expressions to analysis of unconstrained spontaneous expressions. This shift has spawned complex 3D models and computationally expensive geometric methods that prevent usage on resource constrained platforms such as smart phones. This paper presents manifold learning techniques for accurate multi-view facial expression on low resolution 2D images. Our results indicate that mixed class local pose and expression manifold methods perform better than global expression techniques and work just as well as fusing together results from multiple manifolds.

Tracking an object without any prior information regarding its appearance is a challenging problem. Modern tracking algorithms treat tracking as a binary classification problem between the object class and the background class. The binary classifier can be learned offline, if a specific object model is available, or online, if there is no prior information about the object's appearance. In this paper, we propose the use of online distance metric learning in combination with nearest neighbor classification for object tracking. We assume that the previous appearances of the object and the background are clustered so that a nearest neighbor classifier can be used to distinguish between the new appearance of the object and the appearance of the background. In order to support the classification, we employ a distance metric learning (DML) algorithm that learns to separate the object from the background. We utilize the first few frames to build an initial model of the object and the background and subsequently update the model at every frame during the course of tracking, so that changes in the appearance of the object and the background are incorporated into the model. Furthermore, instead of using only the previous frame as the object's model, we utilize a collection of previous appearances encoded in a template library to estimate the similarity under variations in appearance. In addition to the utilization of the online DML algorithm for learning the object/background model, we propose a novel feature representation of image patches. This representation is based on the extraction of scale invariant features over a regular grid coupled with dimensionality reduction using random projections. This type of representation is both robust, capitalizing on the reproducibility of the scale invariant features, and fast, performing the tracking on a reduced dimensional space. The proposed tracking algorithm was tested under challenging conditions and achieved state-of-the art performance.

This paper presents robust facial pose estimation techniques based on the underlying low dimensional manifolds embedded in facial images of varying pose. In our approach, facial feature points of training faces are converted to a low dimensional projection space to form a smooth manifold surface. Subsequent faces are automatically detected, facial feature points are extracted and mapped onto the low dimensional projection surface, where regression models robustly estimate pose. The benefit of using facial feature points for manifold learning over raw facial images is demonstrated by a variety of experiments. Linear, nonlinear, unsupervised and supervised methods are considered including Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Locally Linear Embedding (LLE), Isomap, unsupervised Locality Preserving Projections (LPP), and supervised LPP (SLPP).

In this paper, we propose the combination of manifold learning and distance metric learning for the generation of a representation that is both discriminative and informative, and we demonstrate that this approach is effective for face recognition. Initial dimensionality reduction is achieved using random projections, a computationally efficient and data independent linear transformation. Distance metric learning is then applied to increase the separation between classes and improve the accuracy of nearest neighbor classification. Finally, a manifold learning method is used to generate a mapping between the randomly projected data and a low dimensional manifold. Face recognition results suggest that the combination of distance metric learning and manifold learning can increase performance. Furthermore, random projections can be applied as an initial step without significantly affecting the classification accuracy.

Face detection is a key component in numerous computer vision applications. Most face detection algorithms achieve real-time performance by some form of dimensionality reduction of the input data, such as Principal Component Analysis. In this paper, we are exploring the emerging method of Random Projections (RP), a data independent linear projection method, for dimensionality reduction in the context of face detection. The benefits of using random projections include computational efficiency that can be obtained by implementing matrix multiplications with a small number of integer additions or subtractions. The computational savings are of great significance in resource constrained environments, such as wireless video sensor networks. Experimental results suggest that RP can achieve performance that is comparable to that obtained with traditional dimensionality reduction techniques for face detection using support vector machines.

This paper presents a robust approach to performing facial pose estimation by examining the behavior of key facial features over a wide range of poses. Such methods are useful in intelligent vision systems for entertainment, human computer interaction, and security. In our approach, faces of varying pose are automatically detected, eyes and mouth are located and an active shape model is superimposed. A facial pose estimator is developed using predictor models based on the position, size, and symmetry of facial features. By modeling these predictors over pose positions with varying yaw and pitch, excellent results are obtained without the need for complex computationally intensive methods.

The multiple description coding framework is well-suited for the transmission of images in the presence of transmission errors. However, it has been mainly explored for idealized channels, where packets are either received correctly or not at all. In more realistic scenarios, bitstreams are affected by bit-level noise and error correcting codes must be employed. In the proposed scheme, the input image is decomposed into wavelet coefficients that are encoded using lattice vector quantization for the generation of multiple descriptions. Low-density parity-check (LDPC) codes are used for the protection of each stream, while joint decoding exploits the correlation between the multiple descriptions. The proposed joint source-channel decoding scheme can provide robust transmission of images by introducing a controlled amount of redundancy that can be used for error recovery. Results are presented for image transmission over binary symmetric and Gaussian channels.

This paper presents a new method for adaptive multilevel median (AML3D) filtering of image sequences that can be used for impulse noise suppression in movies that have been converted from film to digital format. The proposed AML3D filter operates on three consecutive frames using an adaptive window and is designed to provide good image quality at modest computational cost. In addition, we present a benchmark comparison of methods used for median filtering of image sequences. Testing includes both grayscale and color images that are synthetically degraded by various levels of impulse noise. Results obtained for timing performance and image quality illustrate that the AML3D filter provides the highest PSNR improvement, while maintaining a reasonable throughput.

We present the design and implementation of a flexible hardware architecture used to perform the 2-D Discrete Wavelet Transform (DWT). A generic VHDL IP core is developed that is configurable to perform the forward and inverse 2-D DWT on an image using a user-defined biorthogonal wavelet. The DWT is performed using the hardware-efficient Lifting Scheme method versus the traditional convolution-based methods minimizing memory requirements, eliminating the need for expensive floating-point arithmetic, and increasing speed. This DWT core is synthesized to a Xilinx Virtex FPGA and configured to perform the Cohen-Dabuechies-Feaveau 5/3 and 9/7 DWTs, which are used in the JPEG2000 still image compression standard.

The MPEG-7 video compression standard allows for multimedia metadata description. By providing robust descriptors and an effective framework for storing them, MPEG-7 is designed to provide a means of navigation through audio-visual content. In particular, MPEG-7 provides a region based shape descriptor, the Angular Radial Transform (ART), for use in image and video annotation and retrieval. For this work, an FPGA based ART extractor was designed and simulated for a Xilinx Virtex-E XCV300e in order to provide a speedup over software based extraction. The design created is capable of processing over 69,4400 pixels a minute. This design utilizes 99% of the FPGA's logical resources and operates at a clock rate of 25 MHz.

Watermarking schemes that have been developed for continuous tone images cannot be directly applied to halftone images. Many of these watermarking methods require characteristics implicit in continuous tone images but absent from halftones. With this in mind, it seems reasonable to have a robust watermarking technique specific to halftones, equipped to work in the binary image domain. The paper reviews existing techniques and suggests improvements to increase their performance and overcome their limitations. The first set of post-halftone watermarking methods work on existing halftones. Data hiding cell parity (DHCP) works in the parity domain instead of on individual pixels. Data hiding mask toggling (DHMT) works by encoding two bits in the 2/spl times/2 neighborhood of a pseudorandom location. Dispersed pseudorandom generator (DPRG), on the other hand, is a preprocessing method that takes place before halftoning. DPRG achieves better visual results. Using the modified peak signal-to-noise ratio (MPSNR) metric, the proposed techniques outperform existing methods by up to 5-15%, depending on the image type and method considered.

This paper presents an efficient algorithm for learning a Bayesian belief network (BBN) structure from a database, as well as providing a comparison between two BBN structure fitness functions. A Bayesian belief network is a directed acyclic graph representing conditional expectations. In this paper, we propose a two-phase algorithm. The first phase uses asymptotically correct structure learning for efficient search space exploration, while the second phase uses greedy model selection for accurate search space exploration. The minimum description length (MDL) structure fitness function is also compared with the database given model probability (DGM) fitness function in the second phase. The model selection algorithms are applied to the ALARM network to provide a comparison for the accuracy of the techniques.

In this paper, algorithms for automatic albuming of consumer photographs are described. Specifically, two core algorithms namely event clustering and screening of low-quality images, are introduced and their performance is evaluated. Event clustering and image quality screening have many applications including albuming services, image management and organization, and digital photofinishing. These are difficult tasks because there is, in general, none (or very limited) contextual information about picture content, and the final interpretation could be subjective. A novel event-clustering algorithm is created to automatically segment pictures into events and subevents for albuming, based on date/time metadata information, as well as color content of the pictures. A block-based color histogram correlation technique is developed for image content comparison of general consumer pictures. A new quality-screening algorithm is developed based on object quality measures, to detect problematic images caused by underexposure, low contrast, and camera defocus or movement.

The JPEG-LS algorithm is one of the recently designated standards for lossless compression of grayscale and color images. Simulation results for lossless and near lossless compression of various image types are presented in order to explore the algorithm's effectiveness for a number of applications. A hardware implementation using VHDL is proposed and the schematic of a JPEG-LS codec, that is capable of standard-compliant lossless and near lossless encoding and decoding, has been generated using the Synopsys synthesis tool. Hardware implementation of the proposed solution on an FPGA allows for real-time processing of large image volumes.

Scene categorization to indoor vs outdoor may be approached by using low-level features for inferring high-level information about the image. Low-level features such as color and texture have been used extensively in image understanding research, however, they cannot solve the problem completely. We propose the use of a Bayesian network for integrating knowledge from low-level and semantic features for indoor vs outdoor classification of images. Using ground truth data for sky and grass detection, we demonstrate that the classification performance can be significantly improved when semantic features are employed in the classification process.

A two-stage texture segmentation approach is proposed where an initial segmentation map is obtained through unsupervised clustering of multiresolution simultaneous autoregressive (MRSAR) features and is followed by self-supervised or bootstrapped classification of wavelet features. The self-supervised stage is based on a segmentation confidence map, where the regions of "high confidence" and "low confidence" are identified on the MRSAR segmentation result using multilevel morphological erosion. The second-stage wavelet classifier is trained from the "high-confidence" samples and is used to reclassify only the "low-confidence" pixels. The final reclassification is based on rules that combine minimum distance and spatial constraints. Additionally, an improved coefficient feature normalization procedure is used during the classification process of both stages. The proposed two-stage approach leverages on the advantages of both MRSAR and wavelet features, and incorporates an adaptive neighborhood-based spatial constraint. Experimental results show that the misclassification error can be significantly reduced compared to morphological cleaning operations alone.

Belief networks, such as Bayes nets, have emerged as an effective knowledge representation and inference engine in artificial intelligence and expert systems research. Their effectiveness is due to the ability to explicitly integrate domain knowledge in the network structure and to reduce a joint probability distribution to conditionally independence relationships. Current research in content-based image processing and analysis is largely limited to low-level feature extraction and classification. The ability to extract both low-level and semantic features and perform knowledge integration of different types of features would be very useful. We present a general knowledge integration framework that incorporates Bayes networks and has been used in two applications involving semantic understanding of consumer photographs. The first application aims at detecting main photographic subjects in an image and the second aims at selecting the most appealing image in an event. With these diverse examples, we demonstrate that effective inference engines can be built according to specific domain knowledge and available training data to solve inherently uncertain vision problems.

A comparative study of lossless compression algorithms is presented. The following algorithms are considered: UNIX compress, gzip, LZW, CCITT Group 3 and Group 4, JBIG, old lossless JPEG, JPEG-LS based on LOCO, CALIC, FELICS, S+P Transform, and PNG. In cases where the algorithm under consideration may only be applied to binary data, the bit planes of the gray scale image are separated, with and without Gray encoding, and the compression is applied to individual bit planes. Testing is done using a set of document images obtained by gray scale scanning of prints of the eight standard CCITT images and a set of nine gray scale pictorial images. The results show that the highest compression is obtained using the CALIC and JPEG-LS algorithms.

A novel formulation of image processing operations in the wavelet domain is presented which directly associates multiresolution with multichannel image processing. The formation of the multiresolution image is expressed as an operator on the image domain that transforms block-circulant structures into partially-block-circulant structures. The proposed implementation relaxes the stationarity and space-invariance assumptions in the image domain and introduces new operator structures for the implementation of single-channel algorithms which take advantage of the correlation structure in the wavelet domain. Based on this structure, the authors discuss the estimation of the power spectrum in the wavelet domain. Image restoration examples using the linear minimum mean square error filter show significant improvement achieved by the proposed approach over the conventional discrete Fourier transform (DFT) implementation.>

The estimation of the point spread function (PSF) for blur identification, often a necessary first step in the restoration of real images, method is presented. The PSF estimate is chosen from a collection of candidate PSFs, which may be constructed using a parametric model or from experimental measurements. The PSF estimate is selected to provide the best match between the restoration residual power spectrum and its expected value, derived under the assumption that the candidate PSF is equal to the true PSF. Several distance measures were studied to determine which one provides the best match. The a priori knowledge required is the noise variance and the original image spectrum. The estimation of these statistics is discussed, and the sensitivity of the method to the estimates is examined analytically and by simulations. The method successfully identified blurs in both synthetically and optically blurred images.

Point spread function (PSF) models derived from physical optics provide a more accurate representation of real blurs than simpler models based on geometrical optics. However, the physical PSF models do not always result in a significantly better restoration, due to the coarse sampling of the recording device and insufficiently high signal-to-noise ratio (SNR) levels. Low recording resolutions result in aliasing errors in the PSF and suboptimal restorations. A high-resolution representation of the PSF where aliasing errors are minimized is used to obtain improved restorations. The SNR is the parameter which ultimately limits the restoration quality and determines the need for an accurate PSF model. As a rule of thumb, the geometrical PSF can be used in place of the physical PSF without significant loss in restoration quality when the SNR is less than 30 dB.

A new blur identification method is proposed which estimates the point spread function (PSF) parameters by comparing the power spectrum of the Wiener restoration residual with its expected value, derived under the assumption that the parameter values tested are correct. The parameter values for which there is the closest match between the residual statistics are chosen as the estimates of the true parameters. The method has successfully identified blur parameters in both synthetically and in-camera blurred images.>