Video stabilization is an important technique in digital cameras. Its impact increases rapidly with the rising popularity of handheld cameras and cameras mounted on moving platforms (e.g., cars). Stabilization of two images can be viewed as an image registration problem. However, to ensure the visual quality of the whole video, video stabilization has a particular emphasis on the accuracy and robustness over long image sequences. In this paper, we propose a novel technique for video stabilization based on the particle filtering framework. We extend the traditional use of particle filters in object tracking to tracking of the projected affine model of the camera motions. We rely on the inverse of the resulting image transform to obtain a stable video sequence. The correspondence between scale-invariant feature transform points is used to obtain a crude estimate of the projected camera motion. We subsequently postprocess the crude estimate with particle filters to obtain a smooth estimate. It is shown both theoretically and experimentally that particle filtering can reduce the error variance compared to estimation without particle filtering. The superior performance of our algorithm over other methods for video stabilization is demonstrated through computer simulated experiments.

A method is introduced to track the object's motion and estimate its pose from multiple cameras. Firstly pose estimation from one camera is explained. We show that pose estimation from the corresponding feature points can be formed as a solution to Sylvester's equation. Furthermore, we develop a distributed solution, which indicates that pose estimation from multiple cameras can be obtained from the linear combination of results from each single camera. We relay on the results from other cameras to improve the estimate of the first one, and vice versa. Finally, the computer simulation experiments demonstrate the superior performance of our algorithm in robustness.

In this paper, we propose a novel technique for estimating focused image sequences captured by an out-of-focus camera. The basic concept used in the proposed algorithm employs multiple images with different blurs from the video sequence to estimate the blur model and construct a more robust method for focused image estimation. This algorithm can be used to correct focusing errors in the video capture process. It could potentially be used to replace the expensive apparatus required for auto-focus adjustments by miniature engines in many camera devices. We demonstrate the successful performance of our approach to focused image estimation through computer simulation experiments using various blur models.

A generalized class of normalized distance functions called Q-Metrics is described in this paper. The Q- Metrics approach relies on a unique functional, using a single bounded parameter λ, which characterizes the conventional distance functions in a normalized per-unit metric space. In addition to this coverage property of the proposed model, a distinguishing and extremely attractive characteristic of the Q-Metric function is its low computational complexity. We present a formal mathematical proof that Q-Metrics satisfy the standard metric axioms. A novel artificial neural network is completely defined and constructed using Q-Metrics. This new network is shown to outperform a conventional feed forward back propagation network with the same size when tested on real data sets.

A novel class of highly adaptive nonlinear digital aggregation connectives called Q-aggregates is described in this paper. The Q-aggregate approach relies on a unique control parameter lambda to characterize the three conventional fuzzy intersection, union, and averaging aggregation operators. In addition to this universal coverage property of the proposed model, a distinguishing and extremely interesting characteristic of the Q-aggregate connective is that even for a fixed value of the control parameter lambda, a given operator can behave as "more than one" family of the conventional connectives depending on the input values. We present the Q-aggregate in application to a real-valued signal processing task, with an optimization algorithm, so that the parameters of the operators can be tuned automatically. The Q-aggregate operator is tested on electrocardiogram (EKG) data. The experiments show that the proposed model can be used to map input signals to their corresponding target signals through learning.

An advanced approach for adaptive nonlinear digital data processing is described in this article. Three primal computational structures referred to as Q-Measures, Q-Metrics, and Q-Aggregates are introduced and utilized in unison as highly adaptive data analysis handlers. The proposed approach relies on universal functionals using few parameters to characterize dynamic system behaviors in broad ranges of unconventional measure, metric, and aggregation spaces. We present this unique approach in application to real-valued signal processing tasks, with suitable optimization algorithms, so that the parameters of the proposed models can be tuned automatically. The new approach is tested on real data sets and the experiments show promising results.

Most existing multiple object tracking techniques suffer from the well-known “multi-object occlusion” problem and/or immense computational cost due to the use of highdimensional joint state representation. In this paper, we present a novel distributed Bayesian framework using multiple collaborative cameras for robust and efficient multiple object tracking with significant and persistent occlusion. When the objects are in close proximity or present multi-object occlusions in a particular camera view, camera collaboration between different views is activated in order to handle the multi-object occlusion problem. Moreover, we propose to model the camera collaboration likelihood density by using epipolar geometry with particle filter implementation. The performance of our approach has been demonstrated on real-world video data.

Radial basis function artificial neural networks using different types of nodes are described in this paper. Two new functions referred to as Q-metrics and Q-aggregates are utilized together to provide highly nonlinear and adaptive node operations. Q-metric distance functions are used in place of the conventional distance functions of the radial basis functions in the hidden layer of the network. Q-aggregate operators are used in the output layer to summarize the information delivered by the hidden layer. Separate classification and regression networks relying on these functions that use simple mathematical formation to characterize dynamic system behaviors in broad ranges of unconventional metric and aggregation spaces are proposed here. We present this approach in application to real-valued signal processing tasks, with suitable optimization algorithms, so that the parameters of the models can be tuned automatically. The new approach has been used to solve real-world problems and it shows promising results.

In this paper, we introduce a method to jointly track the object motion and estimate pose within the framework of particle filtering. We focus on direct estimation of the 3D pose from a 2D image sequence. Scale-Invariant Feature Transform (SIFT) is used to extract feature points in the images. We show that pose estimation from the corresponding feature points can be formed as a solution to Sylvester's equation. We rely on a solution to Sylvester's equation based on the Kronecker product method to solve the equation and determine the pose state. We demonstrate that the classical Singular Value Decomposition (SVD) approach to pose estimation provides a solution to Sylvester's equation in 3D-3D pose estimation. The proposed approach to the solution of Sylvester's equation is therefore equivalent to the classical SVD method for 3D-3D pose estimation, yet it can also be used for pose estimation from 2D image sequences. Finally, we rely on computer simulation experiments to demonstrate the performance of our algorithm on video sequences.

This paper presents a method which avoids the common practice of using a complex joint state-space representation and performing tedious joint data association for multiple object tracking applications. Instead, we propose a distributed Bayesian formulation using multiple interactive trackers that requires much lower complexity for real-time tracking applications. When the objects' observations do not interact with each other, our approach performs as multiple independent trackers. However, when the objects' observations exhibit interaction, defined as close proximity or partial and complete occlusion, we extend the conventional Bayesian tracking framework by modeling such interaction in terms of potential functions. The proposed“magnetic-inertia”model represents the cumulative effect of virtual physical forces that objects undergo while interacting with each other. It implicitly handles the“error merge”and“object labeling”problems and thus solves the difficult object occlusion and data association problems in an innovative way. Our preliminary simulations have demonstrated that the proposed approach is far superior to other methods in both robustness and speed.

A highly adaptive nonlinear digital filtering technique called the Q-Filter is described. The Q-Filter is defined as a Choquet integral with respect to a Q-Measure over a finite window of observations. In addition to robustness, the advantage of the Q-Filter is that it can behave as a combination of different filters, so that a single Q-Filter can be used instead of applying an expensive sequence of conventional filtering operations. In this paper we present the Q-Filter in application to a real-valued signal processing task, with a regression algorithm, so that the parameters of the filter can be tuned automatically. This algorithm also resolves the signal scaling and shifting issues associated with the direct filtering operation. The Q-Filter model is tested on acoustic heartbeat sound data. The experiments show that the proposed model can be used to map input signals to their corresponding target signals through learning.

Particle filters have been introduced as a powerful tool to estimate the posterior density of nonlinear systems. These filters are also capable of processing data online as required in many practical applications. In this paper, we propose a novel technique for video stabilization based on the particle filtering framework. Scale-invariant feature points are extracted to form a rough estimate which is used to model the importance density. We use a constant-velocity Kalman filter model to estimate intentional camera movement. We also prove that the particle filtering estimate will lower the error variance. The superior performance and robustness of our algorithm is demonstrated by computer simulations.

In this paper, we present a novel decentralized Bayesian framework using multiple collaborative cameras for robust and efficient multiple object tracking with significant and persistent occlusion. This approach avoids the common practice of using a complex joint state representation and a centralized processor for multiple camera tracking. When the objects are in close proximity or present multi-object occlusions in a particular camera view, camera collaboration between different views is activated in order to handle the multi-object occlusion problem. Specifically, we propose to model the camera collaboration likelihood density by using epipolar geometry with particle filter implementation. The performance of our approach has been demonstrated on both synthetic and real-world video data

This paper breaks with the common practice of using a joint state space representation and performing the joint data association in multi-object tracking. Instead, we present an interactively distributed framework with linear complexity for real-time applications. When objects do not interact on each other our approach performs like multiple independent trackers. When, the objects are in close proximity or present occlusions, we propose a magnetic-inertia potential model to handle the "error merge" and "labeling" problems in a particle filtering framework. Specifically we propose to model the interactive likelihood densities by a "gravitation" and "magnetic" repulsion scheme and relax the common first-order Markov chain assumption by using an "inertia" Markov chain. Our model represents the cumulative effect of virtual physical forces that objects undergo while interacting with others. It implicitly handles the "error merge" and "labeling" problems and thus solves the difficult object occlusion and data association problems using an innovative scheme. Our preliminary work has demonstrated that the proposed approach is far superior to existing methods not only in robustness but also in speed.

In this paper, we introduce a new class of fuzzy measures, referred to as q-measures, based on a modification of the definition of the well-known Sugeno /spl lambda/-measure. Our proposed definition of the q-measures not only includes the /spl lambda/-measure as a special case, but also preserves all desirable properties and avoids some of the limitations of the conventional /spl lambda/-measure. The q-measure approach provides a more flexible and powerful method for constructing various fuzzy measures. We provide an iterative algorithm for constructing an interesting sequence of q-measures and analytically prove its convergence as a distinguishing characteristic of the proposed formulation.

In this paper, we present the theoretical framework for the generalization of classical hidden Markov models using fuzzy measures and fuzzy integrals. The main characteristic of the generalization is the relaxation of the usual additivity constraint of probability measures. Fuzzy integrals are defined with respect to fuzzy measures, whose key property is monotonicity with respect to set inclusion. This property is far weaker than the usual additivity property of probability measures. As a result of the new formulation, the statistical independence assumption of the classical hidden Markov models is relaxed. Two attractive properties of this generalization are: the generalized hidden Markov model reduces to the classical hidden Markov model if we used the Choquet fuzzy integral and probability measures; and the establishment of a relation between the generalized hidden Markov model and the classical nonstationary hidden Markov model in which the transitional parameters vary with time.

For part I see ibid. vol.8, no. 1 (2000). This paper presents an application of the generalized hidden Markov models to handwritten word recognition. The system represents a word image as an ordered list of observation vectors by encoding features computed from each column in the given word image. Word models are formed by concatenating the state chains of the constituent character hidden Markov models. The novel work presented includes the preprocessing, feature extraction, and the application of the generalized hidden Markov models to handwritten word recognition. Methods for training the classical and generalized (fuzzy) models are described. Experiments were performed on a standard data set of handwritten word images obtained from the US Post Office mail stream, which contains real-word samples of different styles and qualities.

An off-line handwritten word recognition system is described. Images of handwritten words are matched to lexicons of candidate strings. A word image is segmented into primitives. The best match between sequences of unions of primitives and a lexicon string is found using dynamic programming. Neural networks assign match scores between characters and segments. Two particularly unique features are that neural networks assign confidence that pairs of segments are compatible with character confidence assignments and that this confidence is integrated into the dynamic programming. Experimental results are provided on data from the U.S. Postal Service.

A lexicon-based, handwritten word recognition system combining segmentation-free and segmentation-based techniques is described. The segmentation-free technique constructs a continuous density hidden Markov model for each lexicon string. The segmentation-based technique uses dynamic programming to match word images and strings. The combination module uses differences in classifier capabilities to achieve significantly better performance.

Experiments comparing neural networks trained with crisp and fuzzy desired outputs are described. A handwritten word recognition algorithm using the neural networks for character level confidence assignment was tested on images of words taken from the United States Postal Service mailstream. The fuzzy outputs were defined using a fuzzy k-nearest neighbor algorithm. The crisp networks slightly outperformed the fuzzy networks at the character level but the fuzzy networks outperformed the crisp networks at the word level. This empirical result is interpreted as an example of the principle of least commitment.>