Handwritten signature verification is a challenging task because signatures of a writer may be skillfully imitated by a forger. As skilled forgeries are generally difficult to acquire for training, in this paper, we propose a deep learning-based dynamic signature verification framework, SynSig2Vec, to address the skilled forgery attack without training with any skilled forgeries. Specifically, SynSig2Vec consists of a novel learning-by-synthesis method for training and a 1D convolutional neural network model, called Sig2Vec, for signature representation extraction. The learning-by-synthesis method first applies the Sigma Lognormal model to synthesize signatures with different distortion levels for genuine template signatures, and then learns to rank these synthesized samples in a learnable representation space based on average precision optimization. The representation space is achieved by the proposed Sig2Vec model, which is designed to extract fixed-length representations from dynamic signatures of arbitrary lengths. Through this training method, the Sig2Vec model can extract extremely effective signature representations for verification. Our SynSig2Vec framework requires only genuine signatures for training, yet achieves state-of-the-art performance on the largest dynamic signature database to date, DeepSignDB, in both skilled forgery and random forgery scenarios. Source codes of SynSig2Vec will be available at https://github.com/LaiSongxuan/SynSig2Vec.

This article presents SVC-onGoing1, an on-going competition for on-line signature verification where researchers can easily benchmark their systems against the state of the art in an open common platform using large-scale public databases, such as DeepSignDB2 and SVC2021_EvalDB3, and standard experimental protocols. SVC-onGoing is based on the ICDAR 2021 Competition on On-Line Signature Verification (SVC 2021), which has been extended to allow participants anytime. The goal of SVC-onGoing is to evaluate the limits of on-line signature verification systems on popular scenarios (office/mobile) and writing inputs (stylus/finger) through large-scale public databases. Three different tasks are considered in the competition, simulating realistic scenarios as both random and skilled forgeries are simultaneously considered on each task. The results obtained in SVC-onGoing prove the high potential of deep learning methods in comparison with traditional methods. In particular, the best signature verification system has obtained Equal Error Rate (EER) values of 3.33% (Task 1), 7.41% (Task 2), and 6.04% (Task 3). Future studies in the field should be oriented to improve the performance of signature verification systems on the challenging mobile scenarios of SVC-onGoing in which several mobile devices and the finger are used during the signature acquisition.

Dynamic time warping (DTW) is a popular technique for sequence alignment, and is the de facto standard for dynamic signature verification. In this paper, we go a significant step further to enhance DTW with the capability of deep representation learning, and propose an end-to-end trainable Deep soft-DTW (DsDTW) model for dynamic signature verification. Specifically, we design a convolutional recurrent adaptive network (CRAN) to process dynamic signatures, and utilize it to provide robust and discriminative local representations as inputs for DTW. As DTW is not fully differentiable with regard to its inputs, we introduce its smoothed formulation, soft-DTW, and incorporate the soft-DTW distances of signature pairs into the loss function for optimization. Because soft-DTW is differentiable, the proposed DsDTW is end-to-end trainable, and achieves an elegant integration of CRAN deep learning model and traditional DTW mechanism. Our method achieves state-of-the-art performance on several public benchmarks, and has won first place in the ICDAR 2021 competition for online signature verification. Source codes of DsDTW is available at https://github.com/KAKAFEI123/DsDTW.

An open research problem in automatic signature verification is the skilled forgery attacks. However, the skilled forgeries are very difficult to acquire for representation learning. To tackle this issue, this paper proposes to learn dynamic signature representations through ranking synthesized signatures. First, a neuromotor inspired signature synthesis method is proposed to synthesize signatures with different distortion levels for any template signature. Then, given the templates, we construct a lightweight one-dimensional convolutional network to learn to rank the synthesized samples, and directly optimize the average precision of the ranking to exploit relative and fine-grained signature similarities. Finally, after training, fixed-length representations can be extracted from dynamic signatures of variable lengths for verification. One highlight of our method is that it requires neither skilled nor random forgeries for training, yet it surpasses the state-of-the-art by a large margin on two public benchmarks.

Handwritten mathematical expression recognition (HMER) is an important research direction in handwriting recognition. The performance of HMER suffers from the two-dimensional structure of mathematical expressions (MEs). To address this issue, in this paper, we propose a high-performance HMER model with scale augmentation and drop attention. Specifically, tackling ME with unstable scale in both horizontal and vertical directions, scale augmentation improves the performance of the model on MEs of various scales. An attention-based encoder-decoder network is used for extracting features and generating predictions. In addition, drop attention is proposed to further improve performance when the attention distribution of the decoder is not precise. Compared with previous methods, our method achieves state-of-the-art performance on two public datasets of CROHME 2014 and CROHME 2016.

Offline signature verification is a challenging task, where the scarcity of the signature data per writer makes it a few-shot problem. We found that previous deep metric learning based methods, whether in pairs or triplets, are unaware of intra-writer variations and have low training efficiency because only point-to-point (P2P) distances are considered. To address this issue, we present a novel point-to-set (P2S) metric for offline signature verification in this paper. By dividing a training batch into a support set and a query set, our optimization goal is to pull each query to its belonging support set. To further strengthen the P2S metric, a hard mining scheme and a margin strategy are introduced. Experiments conducted on three datasets show the effectiveness of our proposed method.

Offline writer identification plays an important role in forensic document examination and historical document analysis. Today, challenges still exist in historical writer identification (WI), where documents may present very complex handwriting styles. In this paper, we propose novel techniques for a detailed description and accurate identification of handwriting in historical documents. Because handwriting contours are one of the most salient components to characterize one's handwriting style, a novel pathlet feature is proposed to describe their rich properties beyond slant and curvature in a principled way; these properties can be exploited in a VLAD-like encoding framework for fine-grained handwriting description. Besides the pathlet feature, we extract unidirectional SIFT feature to describe handwriting corners and junctions. To effectively encode the pathlet and SIFT features, a novel encoding method, named bagged VLAD, is further proposed to address the problem that a large codebook sparsely spreads out the data points and leads to a degraded performance, allowing a much larger codebook for improved encoding performance. Our proposed method achieves state-of-the-art performance on ICDAR2017 Historical-WI database and ICDAR2019 HDRC-IR database, and has won the first place in ICDAR2019 HDRC-IR competition.