Automatic few-shot font generation aims to solve a well-defined, real-world problem because manual font designs are expensive and sensitive to the expertise of designers. Existing methods learn to disentangle style and content elements by developing a universal style representation for each font style. However, this approach limits the model in representing diverse local styles because it is unsuitable for complicated letter systems. For example, Chinese characters consist of a varying number of components (often called "radical") with a highly complex structure. In this paper, we propose a novel font generation method that learns localized styles, namely component-wise style representations, instead of universal styles. The proposed style representations enable synthesizing complex local details in text designs. However, learning component-wise styles solely from a few reference glyphs is infeasible when a target script has a large number of components, for example, over 200 for Chinese. To reduce the number of required reference glyphs, we represent component-wise styles by a product of component and style factors inspired by low-rank matrix factorization. Owing to the combination of strong representation and a compact factorization strategy, our method shows remarkably better few-shot font generation results (with only eight reference glyphs) than other state-of-the-art methods. Moreover, strong locality supervision was not utilized, such as the location of each component, skeleton, or strokes. The source code is available at https://github.com/clovaai/lffont.

Geewook Kim, Hodong Lee, Daehee Kim, Haeji Jung, Sanghee Park, Yoonsik Kim, Sangdoo Yun, Taeho Kil, Bado Lee, Seunghyun Park. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

Automatic few-shot font generation is a practical and widely studied problem because manual designs are expensive and sensitive to the expertise of designers. Existing few-shot font generation methods aim to learn to disentangle the style and content element from a few reference glyphs, and mainly focus on a universal style representation for each font style. However, such approach limits the model in representing diverse local styles, and thus makes it unsuitable to the most complicated letter system, e.g., Chinese, whose characters consist of a varying number of components (often called ``radical'') with a highly complex structure. In this paper, we propose a novel font generation method by learning localized styles, namely component-wise style representations, instead of universal styles. The proposed style representations enable us to synthesize complex local details in text designs. However, learning component-wise styles solely from reference glyphs is infeasible in the few-shot font generation scenario, when a target script has a large number of components, e.g., over 200 for Chinese. To reduce the number of reference glyphs, we simplify component-wise styles by a product of component factor and style factor, inspired by low-rank matrix factorization. Thanks to the combination of strong representation and a compact factorization strategy, our method shows remarkably better few-shot font generation results (with only 8 reference glyph images) than other state-of-the-arts, without utilizing strong locality supervision, e.g., location of each component, skeleton, or strokes. The source code is available at https://github.com/clovaai/lffont.

A few-shot font generation (FFG) method has to satisfy two objectives: the generated images should preserve the underlying global structure of the target character and present the diverse local reference style. Existing FFG methods aim to disentangle content and style either by extracting a universal representation style or extracting multiple component-wise style representations. However, previous methods either fail to capture diverse local styles or cannot be generalized to a character with unseen components, e.g., unseen language systems. To mitigate the issues, we propose a novel FFG method, named Multiple Localized Experts Few-shot Font Generation Network (MX-Font). MX-Font extracts multiple style features not explicitly conditioned on component labels, but automatically by multiple experts to represent different local concepts, e.g., left-side sub-glyph. Owing to the multiple experts, MX-Font can capture diverse local concepts and show the generalizability to unseen languages. During training, we utilize component labels as weak supervision to guide each expert to be specialized for different local concepts. We formulate the component assign problem to each expert as the graph matching problem, and solve it by the Hungarian algorithm. We also employ the independence loss and the content-style adversarial loss to impose the content-style disentanglement. In our experiments, MX-Font outperforms previous state-of-the-art FFG methods in the Chinese generation and cross-lingual, e.g., Chinese to Korean, generation. Source code is available at https://github.com/clovaai/mxfont.

Scene text detection methods based on neural networks have emerged recently and have shown promising results. Previous methods trained with rigid word-level bounding boxes exhibit limitations in representing the text region in an arbitrary shape. In this paper, we propose a new scene text detection method to effectively detect text area by exploring each character and affinity between characters. To overcome the lack of individual character level annotations, our proposed framework exploits both the given character-level annotations for synthetic images and the estimated character-level ground-truths for real images acquired by the learned interim model. In order to estimate affinity between characters, the network is trained with the newly proposed representation for affinity. Extensive experiments on six benchmarks, including the TotalText and CTW-1500 datasets which contain highly curved texts in natural images, demonstrate that our character-level text detection significantly outperforms the state-of-the-art detectors. According to the results, our proposed method guarantees high flexibility in detecting complicated scene text images, such as arbitrarily-oriented, curved, or deformed texts.

Abstract—We describe a “molecular ” evolutionary algorithm that can be implemented in DNA computing in vitro to learn the recently-proposed hypernetwork model of cognitive memory. The molecular learning process is designed to make it possible to perform wet-lab experiments using DNA molecules and bio-lab tools. We present the bio-experimental protocols for selection, amplification and mutation operators for evolving hypernetworks. We analyze the convergence properties of the molecular evolutionary algorithms on simulated DNA computers. The performance of the algorithms is demonstrated on the task of simulating the cognitive process of learning a language model from a drama corpus to identify the style of an unknown drama. We also discuss other applications of the molecular evolutionary algorithms. In addition to their feasibility in DNA computing, which opens a new horizon of in vitro evolutionary computing, the molecular evolutionary algorithm provides unique properties that are distinguished from conventional evolutionary algorithms and makes a new addition to the arsenal of tools in evolutionary computation. Keywords-molecular evolutionary algorithms, molecular evolutionary learning; DNA computing; hypernetwork; cognitive memory simualtion I.

We describe an evolutionary method for learning concepts of objects from multimodal data. The proposed method uses a population code (hypernetwork representation), i.e. a col lection of codewords (hyperedges) and associated weights, which is adapted by evolutionary computation based on observations of positive and negative examples. The goal of evolution is to find the best compositions and weights of hyperedges to estimate the underlying distribution of the target concepts. We discuss the relationship of this method with estimation of distribution algorithms (EDAs), classifier systems, and ensemble learning methods. We evaluate the method on a suite of image/text benchmarks. The experimental results demonstrate that the evolutionary process successfully discovers salient codewords representing multi-modal feature combinations for describing and distinguishing different concepts. We also analyze how the complexity of the population code evolves as learning proceeds.

Exploring micro RNA (miRNA) and mRNA regulatory interactions may give new insights into diverse biological phenomena. While elucidating complex miRNA-mRNA interactions has been studied with experimental and computational approaches, it is still difficult to infer miRNA-mRNA regulatory modules. Here we present a novel method for identifying functional miRNA-mRNA modules from heterogeneous expression data. The proposed approach is layered hypernetworks consisting of two layers which are the layer of modality-dependent hypernetworks and of an integrating hypernetwork. The layered hypernetwork model is suitable for detecting relationships between heterogeneous modalities. Applied to the analysis of miRNA and mRNA expression profiles on multiple human cancers, the proposed model identifies oncogenic miRNA-mRNA regulatory modules. The experimental results show that our method provides a competitive performance to support vector machines, and outperforms other standard machine learning algorithms. The biological significance of the discovered miRNA-mRNA modules were validated by literature reviews.