We present a novel learning-based image restoration and enhancement technique for improving character recognition performance of OCR products for degraded documents or documents/text captured with mobile devices such as camera-phones. The proposed technique is language independent and can simultaneously increase the effective resolution and restore broken characters with artifacts due to image capturing device such as a low quality/low resolution camera, or due to previous pre-processing such as extracting text region from the document image. The proposed technique develops a predictive relationship between high-resolution training images and their low-resolution/degraded counterparts, and exploits this relationship in a probabilistic scheme to generate a high resolution image from a low quality, low-resolution text image. We present a fast and scalable implementation of the proposed character restoration algorithm to improve the text recognition for document/text images captured by mobile phones. Experimental results demonstrate that the system effectively increases OCR performance for documents captured by mobile imaging devices, from levels of 50% to levels of over 80% for non-latin document/scene text images at 120dpi.

In this paper we present a scalable and high performance near-duplicate image search method. The proposed algorithm follows the common paradigm of computing local features around repeatable scale invariant interest points. Unlike existing methods, much shorter hashes are used (40 bits). By leveraging on the shortness of the hashes, a novel high performance search algorithm is introduced which analyzes the reliability of each bit of a hash and performs content adaptive hash lookups by adaptively adjusting the "range" of each hash bit based on reliability. Matched features are post-processed to determine the final match results. We experimentally show that the algorithm can detect cropped, resized, print-scanned and re-encoded images and pieces from images among thousands of images. The proposed algorithm can search for a 200x200 piece of image in a database of 2,250 images with size 2400x4000 in 0.020 seconds on 2.5GHz Intel Core 2.

A personal computer(pc)-based low-cost visual system that can detect and extract text regions in visual signs in the scene and recognize them for location awareness is described. It employs fast image enhancement and segmentation methods based on symmetric neighborhood filter and hierarchical-connected component analysis to extract written information on signboards, which appears in the scene. Our approach allows us to remove the undesired segments by analyzing their features, removing the perspective distortion and correcting the orientation for better detection using signboard boundary lines and corners, extract the words and characters. As the signboard may not contain only text but also may contain nontext illustrations, the systems classifies each text region inside the signboard as "text-only" and "sign-only" using support vector machine (SVM) based classifiers. Experimental results show the robustness of our region-based detection algorithm over pixel-based algorithms.

We describe a real-time computer vision system and algorithms that extracts customer behavior information by detecting and tracking multiple people as they wait and watch advertisements on a billboard or a new product promotion at a stand. Our system segments each frame into foreground regions which contains multiple people. Foreground regions are further segmented into individuals using a temporal segmentation of foreground and motion cues and global shape constraints on 2D Silhouettes. A 2D dynamic appearance templates is used to track people. The system can provide online customer information, such as, number of people currently watching the billboard, their gender, and offline customer data, such as, how long each people looked at the billboard. Experimental results demonstrate robustness and real-time performance of the algorithm.

We describe a scene text extraction system for handheld devices to provide enhanced information perception services to the user. It uses a color camera attached to a personal digital assistant as an input device to capture scene images from the real world and it employs image enhancement and segmentation methods to extract written information from the scene, convert them to text information and show them to the user so that he/she can see both the real world and information together. We implemented a prototype application: an automatic sign/text language translation for foreign travelers, where people can use the system whenever they want to see text or signs in their own language where they are originally written in a foreign language in the scene.

We describe a monocular real-time computer vision system that identifies shopping groups by detecting and tracking multiple people as they wait in a checkout line or service counter. Our system segments each frame into foreground regions which contains multiple people. Foreground regions are further segmented into individuals using a temporal segmentation of foreground and motion cues. Once a person is detected, an appearance model based on color and edge density in conjunction with a mean-shift tracker is used to recover the person's trajectory. People are grouped together as a shopping group by analyzing interbody distances. The system also monitors the cashier's activities to determine when shopping transactions start and end. Experimental results demonstrate the robustness and real-time performance of the algorithm.

We describe fast background scene modeling and maintenance techniques for real time visual surveillance system for tracking people in an outdoor environment. It operates on monocular gray scale video imagery or on video imagery from an infrared camera. The system learns and models background scene statistically to detect foreground objects, even when the background is not completely stationary (e.g. motion of tree branches) using shape and motion cues. Also, a background maintenance model is proposed for preventing false positives, such as, illumination changes (the sun being blocked by clouds causing changes in brightness), or false negative, such as, physical changes (person detection while he is getting out of the parked car). Experimental results demonstrate robustness and real-time performance of the algorithm.

We describe an appearance-based human body model for tracking multiple people when they are interaction with each others causing significant occlusion amongst them, or when they re-enter the scene. The proposed model allows real time surveillance systems understand "who is who" after multiple people interactions, partial or total occlusions, or when a person "reappeal" in the scene. It combines the grayscale textural appearance and expected shape information together in a 2D dynamic template. Experimental results demonstrates the robustness and real time performance of the proposed model.

Applications such as video gaming, virtual reality, multimodal user interfaces and videoconferencing, require systems that can locate and track persons in a room through a combination of visual and audio cues, enhance the sound that they produce, and perform identification. We describe the development of a particular multimodal sensor fusion system that is portable, runs in real time and achieves these objectives. The system employs novel algorithms for acoustical source location, video-based person tracking and overall system control, which are also described.

W/sup 4/ is a real time visual surveillance system for detecting and tracking multiple people and monitoring their activities in an outdoor environment. It operates on monocular gray-scale video imagery, or on video imagery from an infrared camera. W/sup 4/ employs a combination of shape analysis and tracking to locate people and their parts (head, hands, feet, torso) and to create models of people's appearance so that they can be tracked through interactions such as occlusions. It can determine whether a foreground region contains multiple people and can segment the region into its constituent people and track them. W/sup 4/ can also determine whether people are carrying objects, and can segment objects from their silhouettes, and construct appearance models for them so they can be identified in subsequent frames. W/sup 4/ can recognize events between people and objects, such as depositing an object, exchanging bags, or removing an object. It runs at 25 Hz for 320/spl times/240 resolution images on a 400 MHz dual-Pentium II PC.

We described a video-rate surveillance algorithm to detect and track people from a stationary camera, and to determine if they are carrying objects or moving unencumbered. The contribution of the paper is the shape analysis algorithm that both determines if a person is carrying an object and segments the object from the person so that it can be tracked, e.g., during an exchange of objects between two people. As the object is segmented an appearance model of the object is constructed. The method combines periodic motion estimation with static symmetry analysis of the silhouettes of a person in each frame of the sequence. Experimental results demonstrate robustness and real-time performance of the proposed algorithm.

Ghost is a real time system for estimating human body posture and detecting body parts in monochromatic imagery. It constructs a silhouette based body model to determine the location of the body parts while people are in generic postures. It combines hierarchical body pose estimation, a convex hull analysis of the silhouette, and a partial mapping from the body parts to the silhouette segments using a distance transform method that does not violate the topology of the human body. Experimental results demonstrate robustness and real-time performance of the proposed algorithm.