Distinguishing between identical twins is the Holy Grail in face recognition because of the great similarity between the faces of a pair of twins. Most existing face recognition systems choose to simply ignore it. However, as the population of twins increases quickly, such an “ostrich strategy” is no longer acceptable. The biometric systems that overlook the twins problem are presenting a serious security hole. Inspired by recent advances in motion-based face recognition techniques, we propose to use facial motion to address the twins problem. We collect a twins facial expression database and conduct a series of experiments in two assumed scenarios: the Social Party Scenario and the Access Control Scenario. The experimental results show that facial motion ourperforms facial appearance in distinguishing between twins. Based on this finding, we propose a two-stage cascaded General Access Control System, which combines facial appearance with facial motion. The experimental results show that, compared with an appearance-based face recognition system, this cascaded system is much more secure against an “evil-twin” imposter attack, while performing as good for normal population.

In document digitization through camera-based systems, simple imaging setups often produce geometric distortions in the resultant 2D images because of the non-planar geometric shapes of certain documents such as thick bound books, rolled, folded or crumpled materials, etc. Previous works have demonstrated that arbitrary warped documents can be successfully restored by flattening a 3D scan of the document. These approaches use physically-based or relaxation-based techniques in their flattening process. While this has been demonstrated to be effective in rectifying the image content and improving OCR, these previous approaches have several limitations in terms of speed and stability. In this paper, we propose a distance-based penalty metric to replace the mass-spring model and introduce additional bending resistance and drag forces to improve the efficiency of the existing approaches. The use of Verlet integration and special plane collision handling schemes also help to achieve better stability without sacrificing efficiency. Experiments on various document images captured from books, brochures and historical documents with arbitrary warpings have demonstrated large improvements over the existing approaches in terms of stability and efficiency.

In this paper, we describe a 2-pass iterative scheme to solve the general partial differential equation (PDE) related to the Shape-from-Shading (SFS) problem under both distant and close point light sources. In particular, we discuss its applications in restoring warped document images that often appear in the daily snapshots. The proposed method consists of two steps. First the image irradiance equation is formulated as a static Hamilton-Jacobi (HJ) equation and solved using a fast sweeping strategy with Lax-Friedrichs Hamiltonian. However, abrupt errors may arise when applying to real document images due to noises in the approximated shading image. To reduce the noise sensitivity, a minimization method thus follows to smooth out the abrupt ridges in the initial result and produce a better reconstruction. Experiments on synthetic surfaces show promising results comparing to the ground truth data. Moreover, a general framework is developed, which demonstrates that the SFS method can help to remove both geometric and photometric distortions in warped document images for better visual appearance and higher recognition rate.

This paper presents a restoration framework for correcting both photometric and geometric distortions in camera-based images of non-planar shaped documents to facilitate human perception and machine recognition. The photometric distortions, usually perceived as shading artifacts, are corrected by separating the shading image from the reflectance image through digital inpainting and surface fitting techniques. Meanwhile, the extracted shading image is also used to recover the document's surface shape through a Shape-from-Shading (SFS) method with a generic formulation of the image irradiance under arbitrary illumination conditions. The recovered surface shape is then employed to correct the geometric distortions through a physically-based flattening process. Results on real document images demonstrate the performance of each sub-task and the functionality of the whole framework. OCR results on restored images also show great improvements over the original distorted images.

Shading distortions are often perceived in geometrically distorted document images due to the change of surface normal with respect to the illumination direction. Such distortions are undesirable because they hamper OCR performance tremendously even when the geometric distortions are corrected. In this paper, we propose an effective method that removes shading distortions in images of documents with various geometric shapes based on the notion of intrinsic images. We first try to derive the shading image using an inpainting technique with an automatic mask generation routine and then apply a surface fitting procedure with radial basis functions to remove pepper noises in the inpainted image and return a smooth shading image. Once the shading image is extracted, the reflectance image can be obtained automatically. Experiments on a wide range of distorted document images demonstrate a robust performance. Moreover, we also show its potential applications to the restoration of historical handwritten documents.

With the pervasive use of handheld digital devices such as camera phones and PDAs, people have started to capture images as a way of recording information. However, due to the non-planar geometric shapes of some documents such as thick bound book pages, rolled scripts, etc, the captured images are often warped. This causes problems for many document analysis and retrieval tasks. This paper proposes a dewarping approach that first reconstructs the surface using a shape-from-shading (SFS) technique and then flattens the warped surface using a physically-based deformable model. The SFS method is based on the viscosity framework and considers perspective projection under oblique light source. The recovered surface is represented as a triangular mesh and is restored to its planar state through a numerical integration process. We have tested the proposed approach on both synthetic and real images. The synthetic images are mainly used to evaluate the SFS method and the results are compared with those of a prior SFS method. The reconstructed surfaces of real document images are also compared with the actual shapes captured using a 3D scanner and the restored images are evaluated with those obtained from range-scanned surface shapes

Scanning a document page from a thick bound volume often results in two kinds of distortions in the scanned image, i.e., shade along the "spine" of the book and warping in the shade area. In this paper, we propose an efficient restoration method based on the discovery of the 3D shape of a book surface from the shading information in a scanned document image. From a technical point of view, this shape from shading (SFS) problem in real-world environments is characterized by 1) a proximal and moving light source, 2) Lambertian reflection, 3) nonuniform albedo distribution, and 4) document skew. Taking all these factors into account, we first build practical models (consisting of a 3D geometric model and a 3D optical model) for the practical scanning conditions to reconstruct the 3D shape of the book surface. We next restore the scanned document image using this shape based on deshading and dewarping models. Finally, we evaluate the restoration results by comparing our estimated surface shape with the real shape as well as the OCR performance on original and restored document images. The results show that the geometric and photometric distortions are mostly removed and the OCR results are improved markedly.

When one scans a document page from a thick bound volume, the curvature of the page to be scanned results in two kinds of distortion in the scanned document images: i) shade along the 'spine' of the book; and ii) warping in the shade area. In this paper, we propose an efficient restoration method based on the discovery of the 3D shape of a book surface from the shading information in a scanned document image. We first build practical models namely a 3D geometric model and a 3D optical model for the practical scanning conditions to reconstruct the 3D shape of book surface. We next restore the scanned document image using this shape based on de-shading and de-warping models. Finally, we evaluate the restoration results by comparing the OCR (optical character recognition) performance on the original and restored document images. The experiments show that the geometric and photometric distortions are mostly removed and the OCR results are improved markedly.

We present a framework for acquiring and restoring images of warped documents to their original planar shapes. Image capturing of warped documents often results in warped images. Most digital restoration approaches make use of 2D image processing methods which are dependent on the content of the images. Some recent work attempts to remove this constraint by working directly on the 3D shape of the warped documents. In our framework, we improve on this recent approach to make it more efficient and stable. First we capture the warped surface representation through a set of active lighting from a laser range scanner onto the document surface. This gives us an accurate 3D representation of the document shape. Next by using a stick constraint and a stable numerical integrator, we can digitally flatten the 3D model to a planar shape. This simple approach avoids potential instability problem and is fast. Restoration results for a few types of warped documents are demonstrated and a significant improvement over the original images can be observed.

This paper investigates the problem of blurring caused by motion during image capture of text documents. Motion blurring prevents proper optical character recognition of the document text contents. One area of such applications is to deblur name card images obtained from handheld cameras. In this paper, a complete motion deblurring procedure for document images has been proposed. The method handles both uniform linear motion blur and uniform acceleration motion blur. Experiments on synthetic and real-life blurred images prove the feasibility and reliability of this algorithm provided that the motion is not too irregular. The restoration procedure consumes only small amount of computation time.

Recently, high resolution digital cameras have made the digitization process more flexible and convenient than traditional scanning technology. Therefore, document image analysis techniques need to be extended to handle images captured using ordinary hand-held cameras under an uncontrolled environment. In this paper, we propose an image restoration technique that corrects various warping distortions such as rolling curl, binding curl and fold distortions in camera images of geometrically distorted documents. Our goal is to restore these arbitrarily warped document images to a planar shape so as to facilitate OCR. Our method uses spline interpolation technique based on a ruled surface model constructed from text lines in the 2D document image. Comparisons on OCR performance using Microsoft imaging software with two existing methods show an encouraging improvement in terms of recall and precision. The OCR precision on the corrected images using our method is improved up to 21.8% comparing to original images with a resolution of 3 mega pixels.

Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval