Presentation attack detection (PAD) is a critical component in secure face authentication. We present a PAD algorithm to distinguish face spoofs generated by a photograph of a subject from live images. Our method uses an image decomposition network to extract albedo and normal. The domain gap between the real and spoof face images leads to easily identifiable differences, especially between the re-covered albedo maps. We enhance this domain gap by retraining existing methods using supervised contrastive loss. We present empirical and theoretical analysis that demonstrates that contrast and lighting effects can play a significant role in PAD; these show up particularly in the recovered albedo. Finally, we demonstrate that by combining all of these methods we achieve state-of-the-art results on both intra-dataset testing for CelebA-Spoof, OULU, CASIA-SURF datasets and inter-dataset setting on SiW, CASIA-MFSD, Replay-Attack and MSU-MFSD datasets.

In this paper, we present a non parametric modeling for phase maps of interferometric SAR. Cosine and Sine projections maps are generated from the SAR phase map, and each of them are individually modeled by fitting 2D basis functions. The coefficients of these basis functions describe a "smoothed" version of the original phase map. Several applications can be derived from this noise filtered phase map: better phase unwrapping and SAR image compression are to of the applications that we will be discussing in the paper. The approach can be extended to other imaging domains that involve large maps of directional or phase data, such as modeling of phase MRI images, modeling of wind directions in meteorological data, etc.

For applications such as behavior recognition it is important to maintain the identity of multiple targets, while tracking them in the presence of splits and merges, or occlusion of the targets by background obstacles. Here we propose an algorithm to handle multiple splits and merges of objects based on dynamic programming and a new geometric shape matching measure. We then cooperatively combine Kalman filter-based motion and shape tracking with the efficient and novel geometric shape matching algorithm. The system is fully automatic and requires no manual input of any kind for initialization of tracking. The target track initialization problem is formulated as computation of shortest paths in a directed and attributed graph using Dijkstra's shortest path algorithm. This scheme correctly initializes multiple target tracks for tracking even in the presence of clutter and segmentation errors which may occur in detecting a target. We present results on a large number of real world image sequences, where upto 17 objects have been tracked simultaneously in real-time, despite clutter, splits, and merges in measurements of objects. The complete tracking system including segmentation of moving objects works at 25 Hz on 352times288 pixel color image sequences on a 2.8-GHz Pentium-4 workstation

This paper proposes a method for progressive lossy-to-lossless compression of four-dimensional (4-D) medical images (sequences of volumetric images over time) by using a combination of three-dimensional (3-D) integer wavelet transform (IWT) and 3-D motion compensation. A 3-D extension of the set-partitioning in hierarchical trees (SPIHT) algorithm is employed for coding the wavelet coefficients. To effectively exploit the redundancy between consecutive 3-D images, the concepts of key and residual frames from video coding is used. A fast 3-D cube matching algorithm is employed to do motion estimation. The key and the residual volumes are then coded using 3-D IWT and the modified 3-D SPIHT. The experimental results presented in this paper show that our proposed compression scheme achieves better lossy and lossless compression performance on 4-D medical images when compared with JPEG-2000 and volumetric compression based on 3-D SPIHT.

Video-based surveillance systems have a wide range of applications for traffic monitoring, as they provide more information as compared to other sensors. In this paper, we present a rule-based framework for behavior and activity detection in traffic videos obtained from stationary video cameras. Moving targets are segmented from the images and tracked in real time. These are classified into different categories using a novel Bayesian network approach, which makes use of image features and image-sequence-based tracking results for robust classification. Tracking and classification results are used in a programmed context to analyze behavior. For behavior recognition, two types of interactions have mainly been considered. One is interaction between two or more mobile targets in the field of view (FoV) of the camera. The other is interaction between targets and stationary objects in the environment. The framework is based on two types of a priori information: 1) the contextual information of the camera's FoV, in terms of the different stationary objects in the scene and 2) sets of predefined behavior scenarios, which need to be analyzed in different contexts. The system can recognize behavior from videos and give a lexical output of the detected behavior. It also is capable of handling uncertainties that arise due to errors in visual signal processing. We demonstrate successful behavior recognition results for pedestrian-vehicle interaction and vehicle-checkpost interactions.

Audio-visual speech recognition employing both acoustic and visual speech information is a novel extension of acoustic speech recognition and it significantly improves the recognition accuracy in noisy environments. Although various audio-visual speech-recognition systems have been developed, a rigorous and detailed comparison of the potential geometric visual features from speakers' faces is essential. Thus, in this paper the geometric visual features are compared and analyzed rigorously for their importance in audio-visual speech recognition. Experimental results show that among the geometric visual features analyzed, lip vertical aperture is the most relevant; and the visual feature vector formed by vertical and horizontal lip apertures and the first-order derivative of the lip corner angle leads to the best recognition results. Speech signals are modeled by hidden Markov models (HMMs) and using the optimized HMMs and geometric visual features the accuracy of acoustic-only, visual-only, and audio-visual speech recognition methods are compared. The audio-visual speech recognition scheme has a much improved recognition accuracy compared to acoustic-only and visual-only speech recognition especially at high noise levels. The experimental results showed that a set of as few as three labial geometric features are sufficient to improve the recognition rate by as much as 20% (from 62%, with acoustic-only information, to 82%, with audio-visual information at a signal-to-noise ratio of 0 dB).

Over the past years, video communication has emerged as one of the most popular form of communication. Developing the technology for merging the real and virtual image streams and 3D models for better communication is an active research topic. In this paper, we propose a novel method of real time augmentation of 3D objects in 3D scenes using trifocal tensors. The augmentation is robust to arbitrary camera motion. The proposed technique does not require camera calibration, use of fiducials and prior knowledge of structural model of the scene. Also, using background-foreground segmentation, the occlusion issue in augmented reality is addressed. We show several results of the successful working of our algorithm in real life situations. The technique works on a real time video from a USB camera, creative Webcam III on a P IV 1.6 GHz system without any special hardware support.

Independent component analysis (ICA) has found a wide range of applications in signal processing and multimedia, ranging from speech cleaning to face recognition. This paper presents a non-parametric approach to the ICA problem that is robust towards outlier effects. The algorithm, for the first time in the field of ICA, adopts an intuitive and direct approach, focusing on the very definition of independence itself; i.e. the joint probability density function (pdf) of independent sources is factorial over the marginal distributions. This is contrary to traditional independent component analysis (ICA) algorithms, which achieve the objective by attempting to fulfill necessary conditions (but not sufficient) for independence. For example, the Jade algorithm attempts to approximate independence by minimizing higher order statistics. In the proposed algorithm, kernel density estimation is employed to provide a good approximation of the distributions that are required to be estimated. This estimation technique is inherently robust towards outlier effects. The application of kernel density estimation also enables the algorithm to be free from the assumptions of source distributions. Experimental results show that the algorithm is able to perform separation of sources in the presence of outliers, whereas existing algorithms like Jade and Info max break down under such conditions. The results have also shown that the proposed non-parametric approach is generally source distribution independent. In addition, it is able to separate non-Gaussian zero-kurtotic signals unlike the traditional ICA algorithms like Jade and Infomax.

Emergency response requires strategic assessment of risks, decisions, and communications that are time critical while requiring teams of individuals to have fast access to large volumes of complex information and technologies that enable tightly coordinated work. The access to this information by crisis management teams in emergency operations centers can be facilitated through various human-computer interfaces. Unfortunately, these interfaces are hard to use, require extensive training, and often impede rather than support teamwork. Dialogue-enabled devices, based on natural, multimodal interfaces, have the potential of making a variety of information technology tools accessible during crisis management. This paper establishes the importance of multimodal interfaces in various aspects of crisis management and explores many issues in realizing successful speech-gesture driven, dialogue-enabled interfaces for crisis management. This paper is organized in five parts. The first part discusses the needs of crisis management that can be potentially met by the development of appropriate interfaces. The second part discusses the issues related to the design and development of multimodal interfaces in the context of crisis management. The third part discusses the state of the art in both the theories and practices involving these human-computer interfaces. In particular, it describes the evolution and implementation details of two representative systems, Crisis Management (XISM) and Dialog Assisted Visual Environment for Geoinformation (DAVE/spl I.bar/G). The fourth part speculates on the short-term and long-term research directions that will help addressing the outstanding challenges in interfaces that support dialogue and collaboration. Finally, the fifth part concludes the paper.

Generation and transmission of complex animation sequences can benefit substantially from the availability of tools for handling large amounts of data associated with dynamic three-dimensional (3-D) models. Previous works in 3-D dynamic compression consider only the simplest situation where the connectivity changes do not occur with time. In this paper, we present an approach for compressing 3-D dynamic models in which both the vertex data and the connectivity data can change with time. Using our framework, 3-D animation sequences generated using commercial graphics tools or dynamic range data captured using range scanners can be compressed significantly. We use 3-D registration to identify the changes in the vertex data and the connectivity of the 3-D geometry between successive frames. Next, the interframe motion is encoded using affine motion parameters and the differential pulse coded modulation (DPCM) predictor. Our work is the first to exploit the temporal coherence in the connectivity data between frames and presents a detailed encoding scheme for 3-D dynamic data. We also discuss the issue of inserting I-frames in the compressed data for better performance. We show that our algorithm has a far superior performance when compared with existing techniques for both vertex compression and connectivity compression of 3-D dynamic datasets.

In this paper, we present the registration of disparate spatial data. To be specific, we consider the registration of digital terrain elevation data (DTED) to National High Altitude Photography (NHAP). Initially, the DTED is shaded to form a synthetic image, and our registration process maps point in the shaded image to points in the NHAP. For the purpose of comparison, we propose two distinct techniques for matching. The first method is a semi-autonomous. It requires two pairs of user defined matched points to estimate an initial transform as starting point in the search for the best fitting transform using Nelder-Mead Simplex Method. The second method, being more novel in nature, attempts to eliminate the need for any user intervention and registers the two data autonomously by employing the Multi Objective Evolutionary Algorithm (MOEA) toolbox. Both methods worked well in estimating the best fitting affine transform to register the image and elevation data, and the MOEA based autonomous technique outperforms the much simpler single objective based semi autonomous technique.

Presents a solution to a particular curve (surface) fitting problem and demonstrate its application in modeling objects from monocular image sequences. The curve-fitting algorithm is based on a modified nonparametric regression method, which forms the core contribution of this work. This method is far more effective compared to standard estimation techniques, such as the maximum likelihood estimation method, and can take into account the discontinuities present in the curve. Next, the theoretical results of this 1D curve estimation technique ate extended significantly for an object modeling problem. The input to the algorithm is a monocular image sequence of an object undergoing rigid motion. By using the affine camera projection geometry and a given choice of an image frame pair in the sequence, we adopt the KvD (Koenderink and van Doorn, 1991) model to express the depth at each point on the object as a function of the unknown out-of-plane rotation, and some measurable quantities computed directly from the optical flow. This is repeated for multiple image pairs (keeping one fixed image frame which we formally call the base image and choosing another frame from the sequence). The depth map is next estimated from these equations using the modified nonparametric regression analysis. We conducted experiments on various image sequences to verify the effectiveness of the technique. The results obtained using our curve-fitting technique can be refined further by hierarchical techniques, as well as by nonlinear optimization techniques in structure from motion.

We discuss a type of hidden Markov model (HMM) based on fuzzy sets and fuzzy integral theory which generalizes the classical stochastic HMM. The Choquet integral is used as a fuzzy integral which relaxes one of the two independence assumptions that we had with the classical HMM. We apply this new model to speech recognition and compare the performance with the classical HMM. In this research, the main innovation is that this new generalized fuzzy HMM is applied for the first time to speech recognition. Due to the fuzziness of the model, an interesting gain can be observed in terms of a lower computation time.

The use of soft computing for virtual object implementation is a novel technique that has received a lot of attention in the last few years. This field is very useful especially for applications related to computer games and virtual environments. One of the application areas is the use of real time lip tracking information for the implementation of the virtual lips in animated characters. The increased computing power of computers has made it possible to do real time lip tracking. This real time lip tracking information can be used to implement and control a virtual lip. On the other hand, the use of soft computing to represent the real time lip parameters enables us to have a more robust and flexible system which can compensate for the potential errors of lip tracking. In this paper we present a system which generates real time lip information for soft computing based virtual animated lips.

In this paper, we present a object modeling technique based on volume interaction using silhouettes of objects placed far away from the camera. The method is flexible and easy to use. By using a generic calibration matrix, we surpass the stringent requirements of accurate camera calibration inherent in volume intersection algorithms. This widens the scope of utility of this technique even to the laymen without significant back ground in computer vision. In addition, we propose a modified intersection test to determine inclusion/ exclusion of projected cubes which is fast, accurate and does not require any additional sophisticated hardware. We successfully conducted experimental test on synthetic objects and real images taken through video input. The technique is highly suitable for modeling objects for virtual environments, Internet shopping, e-commerce, etc. and requires easily available, Ubiquitous hardware.

Bimodal speech recognition is a robust technique for automated speech analysis, and has received a lot of attention in the last few decades. In this paper, we analyze the effect of the HMM models on the performance of the bimodal speech recognizer, present a comparative analysis of the different HMM models that can be used in bimodal speech recognition, and finally propose a novel model, which has been experimentally verified to perform better than others. One of the unique characteristic of our HMM model is the novel fusion strategy of the acoustic and the visual features, that takes into account the different sampling rates of these two signals. Compared to audio only, the bimodal speech recognition scheme has a much more improved recognition accuracy, especially in presence of noise.

Audio-visual speech recognition is a novel extension of acoustic speech recognition and has received a lot of attention in the last few decades. The main motivation behind bimodal speech recognition is the bimodal characteristics of speech perception and production systems of human beings. The effect of the modeling parameters of hidden Markov models (HMM) on the recognition accuracy of the bimodal speech recognizer is analyzed, a comparative analysis of the different HMMs that can be used in bimodal speech recognition is presented, and finally a novel model, which has been experimentally verified to perform better than others is proposed. Also, the geometric visual features are compared and analyzed for their importance in bimodal speech recognition. One of the unique characteristics of our bimodal speech recognition system is the novel fusion strategy of the acoustic and the visual features, which takes into account the different sampling rates of these two signals. Compared to acoustic only, the audio-visual speech recognition scheme has a much more improved recognition accuracy, especially in the presence of noise.

We present the theory of modified nonparametric regression for estimating the 3D face structure of a human from a monocular image sequence. In the preprocessing stage, the face region is segmented from the background using both color and motion information, by using a hierarchical block motion estimation method. By using the affine camera projection geometry, and a given choice of an image frame pair in the sequence, we adopt the KvD model to express the depth at each point on the face region as a function of the unknown out-of-plane rotation, and some measurable quantities computed directly from the optical flow. This is repeated for multiple image pairs (keeping one fixed image frame which we formally call the "base" image, and choosing another frame from the sequence). The true depth map is next estimated from these equations using a modified nonparametric regression technique, and this forms the core contribution of this paper. We conducted experiments on various image sequences to verify the effectiveness of the technique, and propose to extend it for photorealistic modeling of arbitrary (non-face) objects from image sequences.

We present a real time system for detecting people and interpreting some of their postures. The system runs reliably on different people and under different lighting conditions. It uses an inexpensive PC desktop camera to acquire color monocular video sequences. Two fundamental problems of such a CCD camera for background-foreground segmentation are analyzed and a solution to them has been proposed. The system locates humans by extracting their profile using a physics based approach, which is robust in the presence of noise. Next it uses a pattern recognition algorithm to detect and interpret the posture of a human. The system can be used as a peripheral user interface, in pervasive computing applications, surveillance, monitoring people activities in an indoor environment, visual computer games, smart screen savers etc. We have already used this system for several real-world interactive video game applications.

We present a new camera projection model, which is intermediate between the affine camera model and the pin hole projection model. It is modeled as a perspective projection of 3D points into an arbitrary plane, followed by an affine transform of these projected points. We observe that the reprojection of a point into a novel image can be achieved uniquely provided that we have located a set of five reference points over four images (of which three are input images, and the fourth is the novel image). Also, the reprojection theory does not assume that the input images are captured from cameras with identical internal calibration parameters. Thus, we apply our technique to two different domains: 1) generation of novel images from a stereo pair; and 2) generation of virtual walk-through sequence with a monocular image sequence as input.

We present an algorithm to generate new views of a human face, starting with at least two other views of the face. In a typical face recognition system, the task of comparison becomes easier if the faces have similar orientation with respect to the camera. The affine coordinate based reprojection algorithm presented enables us to do that. Dense point matches between the two input faces of the same individual are computed using an affine coordinate based reprojection framework. This is followed by the reprojection of one of these to faces to the target face once the user has matched four feature points across two input face images and the target face image.

We present an eigenvalue or spectral representation for CAD models to be used in conjunction with the more traditional attributed graph based representation of these models. The eigenvalues provide a gross description of the structure of the objects, and help to divide a large modelbase into structurally homogeneous partitions. Models in each partition are next hierarchically organized according to the algorithm presented in Sengupta and Boyer (1995). In recognition, gross features computed from a hypothesized object in a range image are used to prune the modelbase by selecting a few "favorable" partitions in which the correct object model is likely to lie. The partitioning experiments presented here are for real range images using a modelbase of 125 CAD objects with planar, cylindrical, and spherical surfaces.

Presents a hierarchically structured approach to organizing large structural modelbases using an information theoretic criterion. Objects (patterns) are modeled in the form of random parametric structural descriptions (RPSDs), an extension of the parametric structural description graph-theoretic formalism. Objects in scenes are modeled as parametric structural descriptions (PSDs). The organization process is driven by pair-wise dissimilarity values between RPSDs. The authors also introduce the node pointer lists, which are computed offline during modelbase organization. During recognition, the only exponential matching process involved is between the scene PSD and the RPSD at the root of the organized tree. Using the organized hierarchy along with the node pointer lists, the remaining work simplifies to a series of inexpensive linear tests at the subsequent levels of the tree search. The authors develop the theory and present three modelbases: 50 objects built from real image data, 100 CAD models, and 1000 synthetic abstract models.>

A hierarchically structured approach to organizing large structural model bases using an information theoretic criterion is presented. Objects (patterns) are modeled in the form of random parametric structural descriptions (RPSDs), an extension of the parametric structural description graph-theoretic formalism. Hierarchically clustering the RPSDs reduces the computational work to O(log N). The node pointers allow a mapping between the observation and a stored representation at one level, and the mapping to all potential models at all subsequent levels is reduced to mere tests, eliminating the exponential search for the best interprimitive mapping function for each stored candidate pattern.>