Synthetic datasets constructed from formal languages allow fine-grained examination of the learning and generalization capabilities of machine learning systems for sequence classification. This article presents a new benchmark for machine learning systems on sequence classification called MLRegTest, which contains training, development, and test sets from 1,800 regular languages. Different kinds of formal languages represent different kinds of long-distance dependencies, and correctly identifying long-distance dependencies in sequences is a known challenge for ML systems to generalize successfully. MLRegTest organizes its languages according to their logical complexity (monadic second order, first order, propositional, or restricted propositional) and the kind of logical literals (string, tier-string, subsequence, or combinations thereof). The logical complexity and choice of literal provides a systematic way to understand different kinds of long-distance dependencies in regular languages, and therefore to understand the capacities of different ML systems to learn such long-distance dependencies. Finally, the performance of different neural networks (simple RNN, LSTM, GRU, transformer) on MLRegTest is examined. The main conclusion is that performance depends significantly on the kind of test set, the class of language, and the neural network architecture.

Long Short-Term Memory recurrent neural network (LSTM) is widely used and known to capture informative long-term syntactic dependencies.However, how such information are reflected in its internal vectors for natural text has not yet been sufficiently investigated.We analyze them by learning a language model where syntactic structures are implicitly given.We empirically show that the context update vectors, i.e. outputs of internal gates, are approximately quantized to binary or ternary values to help the language model to count the depth of nesting accurately, as Suzgun et al. (2019) recently showed for synthetic Dyck languages.For some dimensions in the context vector, we show that their activations are highly correlated with the depth of phrase structures, such as VP and NP.Moreover, with an L 1 regularization, we also found that it can be accurately predicted whether a word is inside a phrase structure or not from a small number of components of the context vector.Even for the case of learning from raw text, context vectors are still shown to correlate well with the phrase structures.Finally, we show that natural clusters of the functional words and the parts of speech that trigger phrases are represented in a small but principal subspace of the context-update vector of LSTM.

We propose a modification of generative adversarial networks (GANs) that generate illustrations of human figures from given poses represented by stick figures. In recent years, while various methods that generate images of characters using GANs have been proposed, it is not yet possible for users to freely designate poses of human figures. When generating an image of a character, the pose of the character takes is an important component of its composition. Thus it is necessary fora user who wants to create an illustration to be able to specify the pose easily. We collected a set of illustrations of human figures from the internet, and for each illustration, a simple line drawing that specifies the pose was drawn manually. We constructed a GAN that takes a line drawing as its input and creates an illustration of a person in a pose that matches the line drawing. These networks are learned using the data set we prepared. In this paper, we propose a new network architecture. After constructing two networks both of which have almost the same structure as pix2pix, which is a variant model of GANs, we stack up those networks based on the idea of stack GAN. The experimental results show that, from stick figures representing common poses such as a standing pose, our methods was able to successfully generate images of characters. However, in the case of stick figures having rare poses that were not in the dataset, such as figures raising a hand or lying down, the generated images were blurred and not of a high-quality but still had the desired shapes. By expanding the dataset to include various poses, it is possible to generate diverse poses more precisely.

In recent years, it has become possible to perform text classification with high accuracy by using convolutional neural networks (CNNs). Zhang et al. decomposed words into characters and classified texts using a CNN with relatively deep layers to obtain excellent classification results. However, it is often difficult to prepare a sufficient number of labeled samples for solving real-world text-classification problems. One method for handling this problem is transfer learning, which uses a network tuned for an arbitrary task as the initial network for a target task. While transfer learning is known to be effective for image recognition, for tasks in natural language processing, such as document classification, it has not yet been shown for what types of data and to what extent transfer learning is effective. In this paper, we first introduce a character-level CNN adopting the structure of a residual network to construct a network with deeper layers for Japanese text classification. We then demonstrate that we can improve classification accuracy by performing transfer learning between two particular datasets. Additionally, we propose an approach to evaluate the effectiveness of transfer learning and use it to evaluate our model.

Once we had tried to propose an unbreakable CAPTCHA and we reached a result that limitation of time is effect to prevent computers from recognizing characters accurately while computers can finally recognize all text-based CAPTCHA in unlimited time. One of the existing usual ways to prevent computers from recognizing characters is distortion, and adding noise is also effective for the prevention. However, these kinds of prevention also make recognition of characters by human beings difficult. As a solution of the problems, an effective text-based CAPTCHA algorithm with amodal completion was proposed by our team. Our CAPTCHA causes computers a large amount of calculation costs while amodal completion helps human beings to recognize characters momentarily. Our CAPTCHA has evolved with aftereffects and combinations of complementary colors. We evaluated our CAPTCHA with deep learning which is attracting the most attention since deep learning is faster and more accurate than existing methods for recognition with computers. In this paper, we add jagged lines to edges of characters since edges are one of the most important parts for recognition in deep learning. In this paper, we also evaluate that how much the jagged lines decrease recognition of human beings and how much they prevent computers from the recognition. We confirm the effects of our method to deep learning.