We describe the effective use of online learning to enhance the conversational capabilities of a concierge robot that we have been developing over the last two years. The robot was designed to interact naturally with visitors and uses a speech recognition system in conjunction with a natural language classifier. The online learning component monitors interactions and collects explicit and implicit user feedback from a conversation and feeds it back to the classifier in the form of new class instances and adjusted threshold values for triggering the classes. In addition, it enables a trusted master to teach it new question-answer pairs via question-answer paraphrasing, and solicits help with maintaining question-answer-class relationships when needed, obviating the need for explicit programming. The system has been completely implemented and demonstrated using the SoftBank Robotics humanoid robots Pepper and NAO, and the telepresence robot known as Double from Double Robotics.

Iris recognition has gained popularity due to factors such as its perceived high accuracy, significant usability advantages attributed to its non-contact acquisition method, and the availability of low cost sensors due to improvements in technology. However, non-contact biometrics authentication systems are vulnerable to different types of attacks than contact-type biometrics, such as fingerprints, for which there are a number of simple techniques to guard against attacks. In particular, the fashion industry has developed designer contact lenses with patterns that range from a simple change in eye color to the imposition of stars or other festive decorations. As these lenses are readily available and can be personalized at a very affordable price, their use in thwarting or spoofing iris-based authentication systems becomes plausible. Given the high security nature of many of these systems, there is a urgent need for a some countermeasure to this type of attack. In this paper, we describe a novel method to detect the presence of fake iris patterns, such as designer contact lenses, during the image acquisition stage to further enhance the basic security value of iris biometrics. Exploiting the anatomy and geometry of the human eye, we present a structured light projection method to detect the presence of artificial items obscuring the real iris. The detection principle has been verified using an inexpensive experimental setup consisting of a miniature projector and an offset camera. We also describe a novel algorithm to process the acquired images to find patterned contact lenses, and measure its performance using data collected with our apparatus. We argue that the addition of the proposed system and algorithm to existing iris biometrics based authentication systems will significantly improve their security.

Management and monitoring of data centers is a growing field of interest, with much current research, and the emergence of a variety of commercial products aiming to improve performance, resource utilization and energy efficiency of the computing infrastructure. Despite the large body of work on optimizing data center operations, few studies actually focus on discovering and tracking the physical layout of assets in these centers. Such asset tracking is a prerequisite to faithfully performing administration and any form of optimization that relies on physical layout characteristics.

In this study, we propose a novel biometric signature for human identification based on anatomically unique structures of the left ventricle of the heart. An algorithm is developed that analyzes the 3 primary anatomical structures of the left ventricle: the endocardium, myocardium, and papillary muscles. Comparisons of these analyses between probe and gallery images produces a similarity score that is used as the basis of the biometric. The performance of the algorithm is tested on a cohort of 10 de-identified subjects imaged by Cardiac MRI. Perfect matching between individuals is obtained with good separation between the genuine and impostor classes. In summary, this study demonstrates using anatomy of the left ventricle of the human heart for the purposes of a biometric signature.

We describe an inexpensive autonomous robot capable of navigating previously unseen data centers and monitoring key metrics such as air temperature. The robot provides real-time navigation and sensor data to commercial IBM software, thereby enabling real-time generation of the data center layout, a thermal map and other visualizations of energy dynamics. Once it has mapped a data center, the robot can efficiently monitor it for hot spots and other anomalies using intelligent sampling. We demonstrate the robot's effectiveness via experimental studies from two production data centers.

We describe a system that controls whether the headlights of a vehicle are in the highbeam or lowbeam state based on input from a forward looking video camera. The core of the system relies on conventional computer vision techniques, albeit with a sophisticated spot finder front-end. Despite this architecture we are able to use an automated supervised learning technique to tune the system to yield high performance. Using a customer-imposed metric we present both in-car and off-line results from our system along with several competitors, and investigate the system's performance under different weather conditions.

We will demonstrate a robot for data center energy management, in action, on a simulated data center floor. We shall highlight the robot's navigation, tile and obstacle classification, event scheduling and preemption capabilities, along with its ability to discover charging docks, and successfully dock with extreme precision. We shall also show simulations on real data center layouts evincing navigational efficiency gains obtained by our latest heuristic enhancements.

Many modern retail stores have self-checkout stations where customers can ring up their own orders without the assistance of any store personnel. To promote customer honesty these systems often weigh each item as it is placed in the bag to confirm that it has the expected mass for the product scanned. In our system we augment this basic check with an assessment of the item's visual appearance to further ensure that the correct code has been entered.

We present a face reconstruction approach for revocable face matching. The proposed approach generates photometrically valid cancelable face images by following the image formation process. Given a face image, the approach estimates facial albedo followed by a subject-specific key based photometric deformation to generate a cancelable face image. The proposed approach allows for using any available face matcher to perform verification or recognition in the transformed domain, a capability missing from most existing works on cancelable face matching. Experiments are performed to evaluate the performance, privacy and cancelable aspects of the face images reconstructed using the approach. Results obtained are very promising and make a strong case for such backward compatible cancelable face representations that can seamlessly make use of advancements in automatic face recognition research.

Cancelable biometric systems are gaining in popularity for use in person authentication for applications where the privacy and security of biometric templates are important considerations. A variety of approaches have been proposed in the literature. In this work, we have chosen two (a registration based and a registration free) techniques and performed a comparative study focusing on template representation size, useful dataset coverage, system accuracy and transform strength. Results show that both systems have their own advantages that are suited for use in specific applications.

A person only has two irises - if his pattern is stolen he quickly runs out of alternatives. Thus methods that protect the true iris pattern need to be adopted in practical biometric applications. In particular, it is desirable to have a system that can generate a new unique pattern if the one being used is lost, or generate different unique patterns for different applications to prevent cross-matching. For backwards compatibility, these patterns should look like plausible irises so they can be handled with the same processing tools. However, they should also non-invertibly hide the true biometric so it is never exposed, or even stored. In this paper four such ldquocancelablerdquo biometric methods are proposed that work with conventional iris recognition systems, either at the unwrapped image level or at the binary iris code level.

Pervasive sensor based systems are transforming Information Technology systems from being transactional in nature to being observational in nature. Observational systems are inherently distributed and capture information at a much finer grain of space and time. Enabling and building such systems also poses many technology challenges, extracting information from sensor signals, indexing and searching sensor meta-data, data mining and scalability. In this paper we use S3: the IBM smart surveillance system as an example of an observational system to explore several of these issues through real world deployment examples.

We present a system for automatically detecting driver's speech in the automobile domain using visual-only information extracted from the driver's mouth region. The work is motivated by the desire to eliminate manual push-to-talk activation of the speech recognition engine in newly designed voice interfaces in the typically noisy car environment, aiming at reducing driver cognitive load and increasing naturalness of the interaction. The proposed system uses a camera mounted on the rearview mirror to monitor the driver, detect face boundaries and facial features, and finally employ lip motion clues to recognize visual speech activity. In particular, the designed algorithm has very low computational cost, which allows real-time implementation on currently available inexpensive embedded platforms, as described in the paper. Experiments are also reported on a small multi-speaker database collected in moving automobiles, that demonstrate promising accuracy.

Biometrics-based authentication systems offer obvious usability advantages over traditional password and token-based authentication schemes. However, biometrics raises several privacy concerns. A biometric is permanently associated with a user and cannot be changed. Hence, if a biometric identifier is compromised, it is lost forever and possibly for every application where the biometric is used. Moreover, if the same biometric is used in multiple applications, a user can potentially be tracked from one application to the next by cross-matching biometric databases. In this paper, we demonstrate several methods to generate multiple cancelable identifiers from fingerprint images to overcome these problems. In essence, a user can be given as many biometric identifiers as needed by issuing a new transformation "key." The identifiers can be cancelled and replaced when compromised. We empirically compare the performance of several algorithms such as Cartesian, polar, and surface folding transformations of the minutiae positions. It is demonstrated through multiple experiments that we can achieve revocability and prevent cross-matching of biometric databases. It is also shown that the transforms are noninvertible by demonstrating that it is computationally as hard to recover the original biometric identifier from a transformed version as by randomly guessing. Based on these empirical results and a theoretical analysis we conclude that feature-level cancelable biometric construction is practicable in large biometric deployments.

With increased concern for physical security, video surveillance is becoming an important business area. Similar camera-based system can also be used in such diverse applications as retail-store shopper motion analysis and casino behavioral policy monitoring. There are two aspects of video surveillance that require significant computing power: image analysis for detecting objects, and video compression for digital storage. The new STI CELL broadband engine (CBE) processor is an appealing platform for such applications because it incorporates 8 separate high-speed processing cores with an aggregate performance of 256Gflops. Moreover, this chip is the heart of the new Sony Playstation 3 and can be expected to be relatively inexpensive due to the high volume of production. In this paper we show how object detection and compression can be implemented on the CBE, discuss the difficulties encountered in porting the code, and provide performance results demonstrating significant speed-up

As smart surveillance technology becomes a critical component in security infrastructures, the system architecture assumes a critical importance. This paper considers the example of smart surveillance in an airport environment. We start with a threat model for airports and use this to derive the security requirements. These requirements are used to motivate an open-standards based architecture for surveillance. We discuss the critical aspects of this architecture and its implementation in the IBM S3 smart surveillance system. Demo results from a pilot deployment in Hawthorne, NY are presented.

Much progress has been achieved during the past two decades in audio-visual automatic speech recognition (AVASR). However, challenges persist that hinder AVASR deployment in practical situations, most notably, robust and fast extraction of visual speech features. We review our efforts in overcoming this problem, based on an appearance-based visual feature representation of the speaker's mouth region. We cover three topics in particular. Firstly, we discuss AVASR in realistic, visually challenging domains, where lighting, background, and head-pose vary significantly. To enhance visual-front-end robustness in such environments, we employ an improved statistical-based face detection algorithm that significantly outperforms our baseline scheme. However, visual-only recognition remains inferior to visually "clean" (studio-like) data, thus demonstrating the importance of accurate mouth region extraction. We then consider a wearable audio-visual sensor to capture the mouth region directly, thus eliminating face detection. Its use improves visual-only recognition, even over full-face videos recorded in the studio-like environment. Finally, we address the speed issue in visual feature extraction, by discussing our real-time AVASR prototype implementation. The reported progress demonstrates the feasibility of practical AVASR.

The detection and tracking of people lie at the heart of many current and near-future applications of computer vision. We describe a background subtraction system designed to detect moving objects in a wide variety of conditions, and a second system to detect objects moving in front of moving backgrounds. Detected foreground regions are tracked with a tracking system which can initiate real-time alarms and generate a smart surveillance index which can be searched to find interesting events in stored video

There are a number of sensor possibilities for mobile robots. Unfortunately many of these are relatively expensive (e.g., laser scanners) or only provide sparse information (e.g., sonar rings). As an alternative, vision-based navigation is very attractive because cameras are cheap these days and computer power is plentiful. The trick is to figure out how to get valuable information out of at least some fraction of the copious pixel stream. In this paper we demonstrate how environmental landmarks can be visually extracted and tracked in order to estimate the rotation of a mobile robot. This method is superior to odometry (wheel turn counting) because it would work with a wide range of environments and robot configurations. In particular, we have applied this method to a very simple motorized base in order to get it to drive in straight lines. As expected, this works far better than ballistic control. We present quantitative results of several experiments to bolster this conclusion.

We present a prototype for the automatic recognition of audio-visual speech, developed to augment the IBM ViaVoice/spl trade/ speech recognition system. Frontal face, full frame video is captured through a USB 2.0 interface by means of an inexpensive PC camera, and processed to obtain appearance-based visual features. Subsequently, these are combined with audio features, synchronously extracted from the acoustic signal, using a simple discriminant feature fusion technique. On the average, the required computations utilize approximately 67% of a Pentium/spl trade/ 4, 1.8 GHz processor, leaving the remaining resources available to hidden Markov model based speech recognition. Real-time performance is there- fore achieved for small-vocabulary tasks, such as connected-digit recognition. In the paper, we discuss the prototype architecture based on the ViaVoice engine, the basic algorithms employed, and their necessary modifications to ensure real-time performance and causality of the visual front end processing. We benchmark the resulting system performance on stored videos against prior research experiments, and we report a close match between the two.

Biometrics-based authentication and identification systems have to handle images acquired in noisy and hostile environments. The signal quality is assessed to decide if there is sufficient signal strength to process further. Poor quality signals require "enhancement" before further processing of the input signal. Often enhancement implies creating a more visibly pleasing image. However, biometrics signals need to improve the image quality for machine processability. This means that the enhancement algorithm should have some weak model about the sample (image) formation process. Enhancement is then some type of "normalization" or "beautification". We present a weak model-based image enhancement algorithm for fingerprint images. The results of the proposed algorithm are presented in terms of the improvements in the overall system performance measured in terms of a receiver operating characteristics curve.

With the use of inkless scanners as input devices for acquiring fingerprints of a person, the digital image of the finger is restricted to the area in contact with the sensor The conventional method of fingerprint image acquisition involves obtaining a nail-to-nail image of the finger known as the rolled fingerprint impression. We present a method of constructing a rolled fingerprint from an image sequence of partial fingerprints using a live-scan fingerprint imager.

The authors present an automatic product 1D system ("VeggieVision"), intended to ease the produce checkout process. The system consists of an integrated scale and imaging system with a user-friendly interface. When a produce item is placed on the scale, an image is taken. A variety of features, color, texture (shape, density), are then extracted. These features are compared to stored "signatures" which were obtained by prior system training (either on-line or off-line). Depending on the certainty of the classification, the final decision is made either by the system or by a human from a number of choices selected by the system. Over 95% of the time, the correct produce classification is in the top four choices.

Describes a three-layer architecture, SSS, for robot control. It combines a servo-control layer, a subsumption layer, and a symbolic layer in a way that allows the advantages of each technique to be fully exploited. The key to this synergy is the interface between the individual subsystems. The design of situation recognizers that bridge the gap between the servo and subsumption layers, and event detectors that link the subsumption layers and symbolic layers are discussed. The development of such a combined system is illustrated by a fully implemented indoor navigation example. The resulting robot was able to automatically map office building environments, and smoothly navigate through them at the rapid speed of 2.6 feet per second.>

Multiagent control systems for robots are considered. A robot is described that is based on treating the existing behavioral agents as simply an enhanced effector command language and designing a higher-level control structure that switches them on and off. It is shown how supervisory control can be added to such a reactive multiagent system.>

The author presents a working, implemented controller for an actual mobile robot arm. The goal of the system is to locate and retrieve empty soda cans in an unstructured environment using a variety of local sensors. The controller, however, is not a centralized sequential program, but rather a collection of 15 independent behaviors. Each of these behaviors contains some grain of expertise concerning the collection task and cooperates with the others to accomplish its goal. These behaviors run concurrently, in real time, on a set of eight loosely coupled on-board 8-bit microprocessors. The author describes the methodology used to decompose the collection task and discusses the types of implicit spatial representation and reasoning used by the system.>