This paper proposes an innovative framework for fast image registration of UAV surveillance video frames by fusing the data from a GPS receiver high-frequency IMU sensor (Piccolo autopilot) and a feature-domain registration method through a non-linear filter. The high-frequency imprecise data from the Piccolo autopilot is refined by the low-frequency precise data from our feature-domain based random M least squares (RMLS) method. The projective transformation model is chosen to achieve high precision. The state and measurement models are non-linear to approximate the real-world imaging dynamics. A periodic hybrid particle filter (PHPF), composed of extended Kalman filter (EKF) and unscented Kalman filter (UKF), is proposed to minimize running time while maintaining accuracy. Both the efficiency and effectiveness of the proposed algorithm will be evaluated through our experiments.

This paper proposes an efficient and robust algorithm to recover a panorama from poorly-obtained UAV video frames contaminated with significant noise. In this algorithm, the eigen-space based neighborhood region will be introduced with our novel feature-based random M least-squares (RMLS) registration technique. Meanwhile, the corresponding similarity regions will be assigned weights according to the relativity between these neighboring regions. Next, Bayesian multi-frame sampling will be implemented utilizing the homography estimated by the frame registration. Finally, the sub-region in each frame which is applicable to the multi-frame sampling will be stitched utilizing multi-resolution blending.

Two main challenges lie in tracking the partially occluded targets in a high-similarity background: 1) similar intensities increase the difficulty of discriminating targets from the background, and 2) occlusion (illumination and shape) decreases the relativity of targets to templates. In this paper, a novel eigenspace-based hybrid particle filter tracking method combined with online non-local appearance model is proposed to track the objects under highly similar environment with occlusions. By on-line training of the templates through non-local methods to generate the active appearance model, it is more likely find the maximum-likelihood distribution correctly. The projective transformation is utilized to cover all of the possible motion factors between the templates. The extended and unscented Kalman filters are switched to update the particles according to the linearity of the motion parameters. The experiment results show the effectiveness of our algorithm while dealing with occluded target in a high-similarity background.

This paper explains and implements our efficient multi-frame super-resolution mosaicking algorithm. In this algorithm, feature points between images are matched using SIFT, and then random M-least squares is used to estimate the homography between frames. Next, separate frames are registered and the overlapping region is extracted. A generative model is then adopted and combined with maximum a posteriori estimation to construct the underdetermined sparse linear system. To solve the ill-posed large-scale inverse system, we derive and implement a new hybrid regularization (bilateral total variance Hubert) method. Cross validation is utilized to estimate the derivative of the blur kernel as well as the regularization parameter. Super-resolution is then applied to the individual sub-frames from the overlapping region. Finally, multi-band blending is used to stitch these resolution-enhanced frames to form the final image. The whole process is semi-real time (roughly 30 seconds for 35 frames) and the effectiveness of our algorithm is validated by applying it to real and synthetic UAV video frames.

Polymerase chain reaction (PCR) and gel electrophoresis are two widely used techniques for genetic studies that require the bench scientist to perform many tedious manual steps. Advances in automation are making these techniques more accessible, but detection and image analysis still remain labor-intensive. Although several commercial software packages are now available, DNA image analysis still requires some intervention by the user, and thus a certain level of image processing expertise. To allow researchers to speed up their analyses and obtain more repeatable results, we present a fully automated image analysis system for DNA or protein studies with high accuracy. The proposed system is based mainly on four steps: automatic thresholding, shifting, filtering, and processing. The automatic thresholding that is used to equalize the gray values of the gel electrophoreses image background is one of the key and novel operations in this algorithm. An enhancement is also used to improve poor quality images that have faint DNA bands. Experimental results show that the proposed method eliminates defects due to noise for good and average quality gel electrophoresis images, while it also improves the appearance of poor quality images.

In several recent movie blockbusters, including Enemy of the State and Patriot Games, scientists magnify digital satellite imagery to determine the identities of people captured on surveillance video. These individuals appear rather blocky at the coarsest resolution scales due to undersampling by the image sensor array, but they become remarkably clear after zooming in on a particular region-of-interest. Obviously, real-world video enhancement algorithms are simply not capable of calculating the perfect results produced in Hollywood. However, additional detail can be extracted from an image sequence by integrating several adjacent digital video frames to compute a high-resolution video still. Extensive prior research has been conducted on the Bayesian super-resolution enhancement of visible light image sequences. For night vision applications, super-resolution enhancement must be applied to image sequences with frames acquired in the infrared region of the electromagnetic spectrum. Most of these applications are DoD-related (military), and they include the enhancement of infrared reconnaissance and surveillance imagery, as well as the enhancement of infrared imagery for automated target tracking and real time multisensor fusion. Several computational simulations have been performed, which visually depict how well the proposed technique enhances short night vision image sequences consisting of two-to-five frames This technique is capable of increasing resolution by factors up to 16/spl times/, although video stills appear somewhat blocky at higher magnification factors.

A subpixel-resolution image registration algorithm based on the nonlinear projective transformation model is proposed to account for camera translation, rotation, zoom, pan, and tilt. Typically, parameter estimation techniques for transformation models require the user to manually select feature points between the images undergoing registration. In this research, block matching is used to automatically select correlated feature point pairs between two images, and these features are used to calculate an iterative least squares solution for the projective transformation parameters. Since block matching is capable of estimating accurate translation motion vectors only in discontinuous edge regions, inaccurate feature point pairs are statistically eliminated prior to computing the least squares parameter estimate. Convergence of the projective transformation model estimation algorithm is generally achieved in several iterations. After subpixel-resolution image registration, a high-resolution video still may be computed by integrating the registered pixels from a short sequence of low-resolution image sequence frames.

The Bayesian high-resolution video stills (HRVS) algorithm has been embedded within the MPEG video compression standard, such that an MPEG-1 intra-coded frame (I-picture) can be enhanced by temporally integrating the neighboring motion-compensated P- and B-pictures. Super-resolution enhancement is achievable if objects within the image sequence move with subpixel increments; thus, the Bayesian HRVS algorithm requires subpixel-resolution motion fields estimated with respect to the I-picture. Half-pel resolution motion fields are provided by the MPEG encoder in the compressed bit stream. These motion vectors are used as initial conditions to a reduced-search block matching algorithm which calculates higher-resolution motion fields with acceptable accuracy. The computation time required to estimate the subpixel motion fields in the Bayesian HRVS algorithm is reduced by a factor of approximately 30 when lower-resolution motion vectors are already available from the MPEG decoder, with little degradation in video still quality.

Super-resolution enhancement algorithms are used to estimate a high-resolution video still (HRVS) from several low-resolution frames, provided that objects within the digital image sequence move with subpixel increments. A Bayesian multi_frame enhancement algorithm is presented to compute an HRVS using the spatial information present within each frame as well as the temporal information present due to object motion between frames. However, the required subpixel-resolution motion vectors must be estimated from low-resolution and noisy video frames, resulting in an inaccurate motion field which can adversely impact the quality of the enhanced image. Several subpixel motion estimation techniques are incorporated into the Bayesian multiframe enhancement algorithm to determine their efficacy in the presence of global data transformations between frames (i.e., camera pan, rotation, tilt, and zoom) and independent object motion. Visual and quantitative comparisons of the resulting high-resolution video stills computed from two video frames and the corresponding estimated motion fields show that the eight-parameter projective motion model is appropriate for global scene changes, while block matching and Horn–Schunck optical flow estimation each have their own advantages and disadvantages when used to estimate independent object motion.

Multiframe resolution enhancement methods are used to estimate a high-resolution video still (HRVS) from several low-resolution image sequence frames, provided that objects within the video sequence move with subpixel increments. Estimating accurate subpixel-resolution motion vectors is a challenging, albeit critically important component of super-resolution enhancement algorithms. A Bayesian motion estimation technique is proposed which models the motion field with a discontinuity-preserving prior. The method is related to Horn-Schunck (1981) optical flow estimation, except that the discontinuity-preserving prior can allow abrupt changes within the motion field without the use of line processes. Simulations compare the high-resolution video stills which result from using the subpixel motion vectors calculated by block matching and the proposed Bayesian motion estimation technique.

The human visual system appears to be capable of temporally integrating information in a video sequence in such a way that the perceived spatial resolution of a sequence appears much higher than the spatial resolution of an individual frame. While the mechanisms in the human visual system that do this are unknown, the effect is not too surprising given that temporally adjacent frames in a video sequence contain slightly different, but unique, information. This paper addresses the use of both the spatial and temporal information present in a short image sequence to create a single high-resolution video frame. A novel observation model based on motion compensated subsampling is proposed for a video sequence. Since the reconstruction problem is ill-posed, Bayesian restoration with a discontinuity-preserving prior image model is used to extract a high-resolution video still given a short low-resolution sequence. Estimates computed from a low-resolution image sequence containing a subpixel camera pan show dramatic visual and quantitative improvements over bilinear, cubic B-spline, and Bayesian single frame interpolations. Visual and quantitative improvements are also shown for an image sequence containing objects moving with independent trajectories. Finally, the video frame extraction algorithm is used for the motion-compensated scan conversion of interlaced video data, with a visual comparison to the resolution enhancement obtained from progressively scanned frames.

The human visual system seems to be capable of temporally integrating information in a video sequence in such a way that the perceived spatial resolution of a sequence appears much higher than the spatial resolution of an individual frame. This paper addresses how to utilize both the spatial and temporal information present in an image sequence to create a high-resolution video still. A novel observation model based on motion compensated subsampling is proposed for a video sequence. Since the reconstruction problem is ill-posed, Bayesian restoration with an edge-preserving prior image model is used to extract a high-resolution video frame from a low-resolution sequence. Estimates computed from an image sequence containing a camera pan show dramatic improvement over bilinear, cubic B-spline, and Bayesian single frame interpolations. Improved definition is also shown for a video sequence containing objects moving with independent trajectories.

Multispectral images consist of multiple channels, each containing data acquired from a different band within the frequency spectrum. Since most objects emit or reflect energy over a large spectral bandwidth, there usually exists a significant correlation between channels. Due to often harsh imaging environments, the acquired data may be degraded by both blur and noise. Simply applying a monochromatic restoration algorithm to each frequency band ignores the cross-channel correlation present within a multispectral image. A Gibbs prior is proposed for multispectral data modeled as a Markov random field, containing both spatial and spectral cliques. Spatial components of the model use a nonlinear operator to preserve discontinuities within each frequency band, while spectral components incorporate nonstationary cross-channel correlations. The multispectral model is used in a Bayesian algorithm for the restoration of color images, in which the resulting nonlinear estimates are shown to be quantitatively and visually superior to linear estimates generated by multichannel Wiener and least squares restoration.

Maximum a posteriori (MAP) estimation using a robust signal model is capable of extracting a signal from additive noise, preserving edges in the signal estimate. A digital network can compute the estimate numerically using the gradient descent algorithm, or an analog network can be designed which provides the estimate as a steady-state output. An ASIC designed to perform robust signal processing has been manufactured, using the Huber-Markov random field (HMRF) signal model. This robust model is dependent upon two tunable parameters, which may be adjusted for a particular signal due to the design of novel analog circuit blocks.>

The application of a monochromatic restoration algorithm to each channel within a multispectral image does not result in an estimate which takes into account cross-channel correlation. A non-Gaussian prior model is proposed for multispectral images, using the Gibbs distribution. The density accounts for both spatial (within channel) and spectral (between channel) information. Spatial components use a nonlinear operator to preserve discontinuities within each channel, while spectral components incorporate cross-channel information in the model. The prior density is used in a maximum a posteriori (MAP) estimation algorithm for the restoration of color images. The resulting nonlinear estimates are shown to be quantitatively superior to linear estimates produced by multichannel Wiener and least squares restoration, which implicitly use Gaussian priors.>

For signals containing discontinuities, the usual assumptions of Gauss-Markov distributed signal sources do not hold. To preserve edges, non-Gaussian prior models have been developed for use in Bayesian restoration. These models are generally dependent upon two parameters, one controlling the size of reconstructed discontinuities, and the other controlling data smoothing. The authors propose a maximum likelihood technique for automatically estimating these parameters, resulting in the optimization of an expression dependent upon the prior model partition function. An exact expression is derived for the 1D signal model partition function, while an approximation is proposed for the 2D image model partition function. Parameters estimated from degraded signals result in high quality restorations.>

Accurate image expansion is important in many areas of image analysis. Common methods of expansion, such as linear and spline techniques, tend to smooth the image data at edge regions. This paper introduces a method for nonlinear image expansion which preserves the discontinuities of the original image, producing an expanded image with improved definition. The maximum a posteriori (MAP) estimation techniques that are proposed for noise-free and noisy images result in the optimization of convex functionals. The expanded images produced from these methods will be shown to be aesthetically and quantitatively superior to images expanded by the standard methods of replication, linear interpolation, and cubic B-spline expansion.

A method for nonlinear image expansion is introduced. The method preserves the discontinuities of the original image, therefore producing an expanded image with improved definition. The maximum a posteriori (MAP) estimation technique that is proposed results in the constrained optimization of a convex functional. The expanded image produced from this method will be shown to be aesthetically and quantitatively superior to images expanded by the standard methods of replication, linear interpolation, and cubic B-spline expansion.>