We present a method for using human describable face attributes to perform face identification in criminal investigations. To enable this approach, a set of 46 facial attributes were carefully defined with the goal of capturing all describable and persistent facial features. Using crowd sourced labor, a large corpus of face images were manually annotated with the proposed attributes. In turn, we train an automated attribute extraction algorithm to encode target repositories with the attribute information. Attribute extraction is performed using localized face components to improve the extraction accuracy. Experiments are conducted to compare the use of attribute feature information, derived from crowd workers, to face sketch information, drawn by expert artists. In addition to removing the dependence on expert artists, the proposed method complements sketchbased face recognition by allowing investigators to immediately search face repositories without the time delay that is incurred due to sketch generation.

Facial composites are widely used by law enforcement agencies to assist in the identification and apprehension of suspects involved in criminal activities. These composites, generated from witness descriptions, are posted in public places and media with the hope that some viewers will provide tips about the identity of the suspect. This method of identifying suspects is slow, tedious, and may not lead to the timely apprehension of a suspect. Hence, there is a need for a method that can automatically and efficiently match facial composites to large police mugshot databases. Because of this requirement, facial composite recognition is an important topic for biometrics researchers. While substantial progress has been made in nonforensic facial composite (or viewed composite) recognition over the past decade, very little work has been done using operational composites relevant to law enforcement agencies. Furthermore, no facial composite to mugshot matching systems have been documented that are readily deployable as standalone software. Thus, the contributions of this paper include: 1) an exploration of composite recognition use cases involving multiple forms of facial composites; 2) the FaceSketchID System, a scalable, and operationally deployable software system that achieves state-of-the-art matching accuracy on facial composites using two algorithms (holistic and component based); and 3) a study of the effects of training data on algorithm performance. We present experimental results using a large mugshot gallery that is representative of a law enforcement agency’s mugshot database. All results are compared against three state-of-the-art commercial-off-the-shelf face recognition systems.