In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.

We address the problem of shoulder-surfing attacks on authentication schemes by proposing IllusionPIN (IPIN), a PIN-based authentication method that operates on touchscreen devices. IPIN uses the technique of hybrid images to blend two keypads with different digit orderings in such a way, that the user who is close to the device is seeing one keypad to enter her PIN, while the attacker who is looking at the device from a bigger distance is seeing only the other keypad. The user's keypad is shuffled in every authentication attempt, since the attacker may memorize the spatial arrangement of the pressed digits. To reason about the security of IPIN, we developed an algorithm which is based on human visual perception and estimates the minimum distance from which an observer is unable to interpret the keypad of the user. We tested our estimations with 84 simulated shoulder-surfing attacks from 21 different people. None of the attacks was successful against our estimations. In addition, we estimated the minimum distance from which a camera is unable to capture the visual information from the keypad of the user. Based on our analysis, it seems practically almost impossible for a surveillance camera to capture the PIN of a smartphone user when IPIN is in use.