Few-shot font generation has garnered significant attention due to its wide range of applications. The mainstream methods are based on the idea of the style and content disentangled representation learning and can be mainly categorized into two kinds of methods according to the prior used, i.e., the deep prior and glyph prior. However, the prior information used in existing methods mainly focuses on static spatial information and ignores dynamic temporal information symbolizing the internal correlation of characters, which results in stroke misalignment and poor performance on the generation of glyph articulations. To address these issues, we propose a novel few-shot font generation model by learning stroke-order dynamics via sequential awareness, where both the static spatial stroke information and dynamic temporal stroke-order information are incorporated into the generation. By leveraging these kinds of stroke information, the issues of stroke misalignment and poor articulation generation can be significantly alleviated. We conduct extensive experiments over 150 fonts, which show the superiority of the proposed model compared to state-of-the-art, and good generalization performance for the cross-lingual font generation.

Font design is a very resource-intensive endeavor, especially for intricate fonts. The task of few-shot font generation (FFG) has attracted great interest recently. This method captures style from a limited set of reference glyphs and then transfers it to other characters to generate diverse style fonts. Existing FFG methods mainly revolve around learning font content or style. However, these methods often only learn content or style or lack the ability to represent style and content, resulting in poor font quality. To address these issues, we introduce CLIP-Font—a novel few-shot font generation model. CLIP-Font uses font text semantics for self-supervision to guide font generation at the content level, and uses attention-based contrast learning at the style level to capture the representation capabilities of the font fine-grained style enhancement model. Experimental results on various datasets demonstrate the effectiveness of our method, surpassing the performance of existing FFG techniques.

The few-shot font generation (FFG) task aims to create a new font library using only a small number of reference samples. The predominated methods for this task are mainly based on the style-content disentangled representation learning. Existing style-content disentangling based few-shot font generation models are mainly devoted to the extraction of better content and style features by leveraging extra prior information such as strokes and skeletons, or introducing auxiliary networks while ignoring the aggregation scheme of style and content features. To address this issue, we propose a novel few-shot font generation method called SCA-Font by introducing an effective style-content feature aggregation module (SCAM), where the content features from the source characters and the style features from the target reference characters are effectively aggregated by a novel neural network. Experimental results on a dataset of 35 font styles collected by ourselves demonstrate that the proposed SCA-Font model outperforms state-of-the-art models in both quantitative results and the quality of generated characters. We also verify the effect of the number of shots for the proposed model. Numerical experiment results show that six shots of reference characters are preferred to achieve the best performance of the proposed model.

Chinese calligraphy generation is an important problem involved in many applications. Existing methods generally regard it as an image-to-image translation problem. However, existing models meet the challenge of poor generation performance for the Chinese calligraphy generation due to the lack of effective guided information. In this paper, we propose a novel model called CRA-GAN for the Chinese calligraphy generation through utilizing the contours of calligraphy characters as certain guided information and introducing a region-aware attention module to capture their content regions, motivated by the observation that the contour and content region provide certain delicate characteristics on the calligraphy style and content, respectively. Noticing that there is usually a large glyph difference between the source and target fonts, resulting in the degradation of model performance, we borrow the idea of adaptive pre-deformation from the literature to address this issue. A series of experiments are conducted to show the effectiveness of the suggested contour and region-aware attention, as well as the used adaptive pre-deformation operation. The outperformance of the proposed model over the state-of-the-art models is also demonstrated through extensive quantitative and qualitative comparisons over nine Chinese calligraphic font datasets.

Referring expressions are commonly used when referring to a specific target in people's daily dialogue. In this paper, we develop a novel task of audio-visual grounding referring expression for robotic manipulation. The robot leverages both the audio and visual information to understand the referring expression in the given manipulation instruction and the corresponding manipulations are implemented. To solve the proposed task, an audio-visual framework is proposed for visual localization and sound recognition. We have also established a dataset which contains visual data, auditory data and manipulation instructions for evaluation. Finally, extensive experiments are conducted both offline and online to verify the effectiveness of the proposed audio-visual framework. And it is demonstrated that the robot performs better with the audio-visual data than with only the visual data.

The recent extension of a macroscopic fundamental diagram (MFD) into a bi-modal MFD (or 3D-MFD) provides the relationship among the total network circulating flows and the accumulations of private vehicles and public buses. 3D-MFD reveals the contribution of large occupancy vehicles such as buses in improving urban transportation efficiency. A lot of bi-modal traffic management techniques are introduced based on 3D-MFD to improve the urban traffic efficiency without using detailed origin-destination (OD) information. However, similar to MFD, 3D-MFD is also highly affected by the heterogeneity of a road network. In order to form 3D-MFDs with low scatter to be utilized for further bi-modal traffic management, this paper proposes a partition method to cluster road links into several homogeneous regions for a bi-modal urban network. It is comprised of three layers named as initial partition, merging, and boundary adjusting. At the initial partition layer, Seeded Region Growing (SRG) and the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) are integrated to obtain a number of subregions. A modified Genetic Algorithm (GA) is developed to merge the subregions into larger regions at the merging layer. Then, boundary adjusting is performed by changing the region to which a boundary is clustered to optimize the result. Multi-sensor data collected from Shenzhen in China are utilized to verify the effectiveness of the proposed partition method.

End-to-end image compression using trained deep networks as encoding/decoding models has been developed substantially in the recent years. Previous work is limited in using a single encoding/decoding model, whereas we explore the usage of multiple encoding/decoding models as an ensemble. We propose several methods to obtain multiple models. First, we adopt the boosting strategy to train multiple networks with diversity as an ensemble. Second, we train an ensemble of multiple probability distribution models to reduce the distribution gap for efficient entropy coding. Third, we present a geometric transform-based self-ensemble method. The multiple models can be regarded as the multiple coding modes, similar to those in non-deep video coding schemes. We further adopt block-level model/mode selection at the encoder side to pursue rate-distortion optimization, where we use hierarchical block partitioning to improve the adaptation ability. Compared with single-model end-to-end compression, our proposed method improves the compression efficiency significantly, leading to 21% BD-rate reduction on the Kodak dataset, without increasing the decoding complexity. On the other hand, when keeping the same compression efficiency, our method can use much simplified decoding models, where the floating-point operations are reduced by 70%.

Virtual reality (VR) has been a hot topic in both research and industry, calling for more efficient compression of panoramic videos. Currently, panoramic video is played as if it is spherical, but such video is actually mapped to planar video, e.g. using equirectangular projection, before compression and transmission. The projection causes deformation and thus makes the traditional translational motion model not efficient. In this paper, we propose a new motion model based on spherical coordinates transform to compensate for the deformation in panoramic videos. Our model requires no additional motion vector but rather derives pixel-wise 2D motion vectors from a block-level 3D motion vector. Our experimental results show the significant bits saving achieved by the new model, which leads to as high as 8.0% BD-rate on the test sequences.

The ability to design thermostable proteins offers enormous potential for the development of novel protein bioreagents. In this work, a combined computational and experimental method was developed to increase the T m of the flavin mononucleotide based fluorescent protein Bacillus Subtilis YtvA LOV domain by 31 Celsius, thus extending its applicability in thermophilic systems. Briefly, the method includes five steps, the single mutant computer screening to identify thermostable mutant candidates, the experimental evaluation to confirm the positive selections, the computational redesign around the thermostable mutation regions, the experimental reevaluation and finally the multiple mutations combination. The adopted method is simple and effective, can be applied to other important proteins where other methods have difficulties, and therefore provides a new tool to improve protein thermostability.