In this paper, we introduce HoughToRadon Transform layer, a novel layer designed to improve the speed of neural networks incorporated with Hough Transform to solve semantic image segmentation problems. By placing it after a Hough Transform layer, ’inner’ convolutions receive modified feature maps with new beneficial properties, such as a smaller area of processed images and parameter space linearity by angle and shift. These properties were not presented in Hough Transform alone. Furthermore, HoughToRadon Transform layer allows us to adjust the size of intermediate feature maps using two new parameters, thus allowing us to balance the speed and quality of the resulting neural network. Our experiments on the open MIDV-500 dataset show that this new approach leads to time savings in document segmentation tasks and achieves state-of-the-art 97.7% accuracy, outperforming HoughEncoder with larger computational complexity.

In this work, we present the auto-clustering method which can be used for pattern recognition tasks and applied to the training of a metric convolutional neural network. The main idea is that the algorithm creates clusters consisting of classes similar from a network’s point of view. The usage of clusters allows the network to pay more attention to classes that are hard to differ. This method improves the generation of pairs during the training process, which is a current problem because the optimal generation of data significantly affects the quality of training. The algorithm works in parallel with the training process and is fully automatic. To evaluate this method we chose the Korean alphabet with the corresponding PHD08 dataset and compared our auto-clustering with random-mining, hard-mining, distance-based mining. Opensource framework Tesseract OCR 4.0.0 was also considered to evaluate the baseline.

Local patch descriptors are used in many computer vision tasks. Over the past decades, many methods of descriptor extraction were proposed. In recent years researchers have started to train descriptors via Convolutional Neural Networks (CNNs) which have shown their advantages in many other computer vision fields. However, the resulting descriptors are usually represented as a long real-valued vector. That leads to high computational complexity and memory usage in real applications with a large amount of data being processed. To deal with that problem binary local descriptors were designed, but they still have a large size. In this paper, we propose a method of discrete low-dimensional local descriptor creation with lightweight CNN. We show that for small-sized descriptors the quality drops significantly during simple binarization compared to floating-point ones. The experiments on HPatches dataset [1] demonstrate that our descretization approach dramatically outperforms the naive binarization for the compact descriptors.

In this paper, we propose a HoughEncoder neural network architecture for the semantic image segmentation task. The main feature of the proposed architecture is that it contains layers calculating direct and transposed integral operators, namely Fast Hough Transform. These layers split deep fully convolutional architecture into three blocks. Therefore, the neural network inherits a possibility to make a decision in every point using integral features along different lines. It is important, that by doing this we do not increase the complexity of the neural network in terms of the number of trainable parameters. Our experiments on the publicly available datasets MIDV-500 and MIDV-2019 (both train and test) show that the suggested modification greatly increases quality. HoughEncoder outperforms UNet which shows state-of-the-art results in many semantic image segmentation tasks even while it has a one hundred times fewer parameters.

In the field of document analysis and recognition using mobile devices for capturing, and the field of object recognition in a video stream, it is important to be able to combine the information received from different frames, since the quality of text recognition depends on the effectiveness of collecting the maximal amount of information about the target object. This paper examines and compares the effectiveness of two different combination approaches, namely pre-combination of images before recognition and the combination of recognition results. The combination methods are briefly described. The quality of the combined results obtained using different methods was measured and compared on the MIDV-500 dataset. The results show that the approach with a combination of text strings recognition results is more effective in comparison with the preliminary combination of images. It can be concluded that simple image stacking with projective alignment does not allow to achieve a comparable recognition results combination quality, and thus in order to include the information about per-frame changes of the text images more sophisticated image combination algorithms need to be employed.

In this paper, we propose a new method to detect monospaced font in text line images. Although many authors address more complex problems of text recognition or font recognition, this problem is still challenging when dealing with camera-captured images of identity documents. However, such a font characteristic can be useful in document authentication. These images usually contain complex backgrounds and various distortions. Our approach is based on a segmentation neural network and Fourier Transform for detecting “strong” periodic components in the segmentor output. The experimental results show that the combination of neural network and Fourier Transform deals with the task of monospaced font detection more effectively than the same Fourier analysis applied to the results of an image processing method for segmentation. The main advantage of the neural network is that its output does not depend on background, font and characters characteristics directly.

Despite the significant success in the field of text recognition, complex and unsolved problems still exist in this field. In recent years, the recognition accuracy of the English language has greatly increased, while the problem of recognition of hieroglyphs has received much less attention. Hieroglyph recognition or image recognition with Korean, Japanese or Chinese characters have differences from the traditional text recognition task. This article discusses the main differences between hieroglyph languages and the Latin alphabet in the context of image recognition. A light-weight method for recognizing images of the hieroglyphs is proposed and tested on a public dataset of Korean hieroglyph images. Despite the existing solutions, the proposed method is suitable for mobile devices. Its recognition accuracy is better than the accuracy of the open-source OCR framework. The presented method of training embedded net bases on the similarities in the recognition data.

Due to a noticeable expansion of document recognition applicability, there is a high demand for recognition on mobile devices. A mobile camera, unlike a scanner, cannot always ensure the absence of various image distortions, therefore the task of improving the recognition precision is relevant. The advantage of mobile devices over scanners is the ability to use video stream input, which allows to get multiple images of a recognized document. Despite this, not enough attention is currently paid to the issue of combining recognition results obtained from different frames when using video stream input. In this paper we propose a weighted text string recognition results combination method and weighting criteria, and provide experimental data for verifying their validity and effectiveness. Based on the obtained results, it is concluded that the use of such weighted combination is appropriate for improving the quality of the video stream recognition result.

The paper presents an algorithm for document image recognition robust to projective distortions. This algorithm is based on a similarity metric, which is learned using Siamese architecture. The idea of training Siamese networks is to build a function of converting the image into a space where a distance function corresponding to a pre-defined metric approximates the similarity between objects of initial space. During learning the loss function tries to minimize the distance between pairs of object from the same class and maximize it between the ones from different classes. A convolutional network is used for mapping initial space to the target one. This network lets to construct a feature vector in target space for each class. Classification of objects is performed using the mapping function and finding the nearest feature vector. The proposed algorithm achieved recognition quality comparable to classifying convolutional network on an open dataset of document images MIDV-500 [1]. Another important advantage of this method is the possibility of one-shot learning that is also shown in the paper.

This paper proposes an improvement for an existing and widely spread approach of panorama stitching for images of planar objects. The proposed method is based on projective transformations graph adjustment. Evaluation is presented on a heterogeneous dataset which contains images of Earth’s and Mars’s surfaces, images taken using a microscope, as well as handwritten and printed text documents. Quality enhancement of panorama stitching method is illustrated on this dataset and shows more than twofold reduction in the accumulated computation error of projective transformations.

Textual blocks rectification or slant correction is an important stage of document image processing in OCR systems. This paper considers existing methods and introduces an approach for the construction of such algorithms based on Fast Hough Transform analysis. A quality measurement technique is proposed and obtained results are shown for both printed and handwritten textual blocks processing as a part of an industrial system of identity documents recognition on mobile devices.

The important part of the system of a planar rectangular object analysis is the localization: the estimation of projective transform from template image of an object to its photograph. The system also includes such subsystems as the selection and recognition of text fields, the usage of contexts etc. In this paper three localization algorithms are described. All algorithms use feature points and two of them also analyze near-horizontal and near- vertical lines on the photograph. The algorithms and their combinations are tested on a dataset of real document photographs. Also the method of localization quality estimation is proposed that allows configuring the localization subsystem independently of the other subsystems quality.

In this paper, we present a new technique of space symbol detection in the task of monospaced text recognition for credit cardholder name as an example. In the considered case, standard methods fail or have low quality because of background complexity and variability of symbol colors. Suggested method is based on the usage of two conjoined symbol images as an input for artificial neural network. This provides background visual context for the recognizer, which helps it to distinguish between symbols and spaces.