Recently, state-of-the-art text-to-image generation models, such as FLUX and Ideogram 2.0, have made significant progress in sentence-level visual text rendering. In this paper, we focus on the more challenging scenarios of article-level visual text rendering and address a novel task of generating high-quality business content, including infographics and slides, based on user provided article-level descriptive prompts and ultra-dense layouts. The fundamental challenges are twofold: significantly longer context lengths and the scarcity of high-quality business content data.In contrast to most previous works that focus on a limited number of sub-regions and sentence-level prompts, ensuring precise adherence to ultra-dense layouts with tens or even hundreds of sub-regions in business content is far more challenging. We make two key technical contributions: (i) the construction of scalable, high-quality business content dataset, i.e., Infographics-650K, equipped with ultra-dense layouts and prompts by implementing a layer-wise retrieval-augmented infographic generation scheme; and (ii) a layout-guided cross attention scheme, which injects tens of region-wise prompts into a set of cropped region latent space according to the ultra-dense layouts, and refine each sub-regions flexibly during inference using a layout conditional CFG. We demonstrate the strong results of our system compared to previous SOTA systems such as FLUX and SD3 on our BizEval prompt set. Additionally, we conduct thorough ablation experiments to verify the effectiveness of each component. We hope our constructed Infographics-650K and BizEval can encourage the broader community to advance the progress of business content generation.

Pre-trained models such as BERT have achieved remarkable results in text matching tasks. However, existing models still suffer from the challenge of capturing local subtle differences when modeling complex semantic matching relationships. In this work, we find that the integration of local syntax awareness and global semantics is crucial for text matching. Meanwhile, we propose the Local and Global Syntax Graph Calibration (LG-SGC) module, which can explore local syntactic and global semantic information for the matching task. Specifically, we first introduce an auxiliary task inside BERT to capture local subtle grammatical differences. Then, we retain the original attention operation to capture global matching features. Finally, we design an information fusion layer to effectively combine local and global information to deepen the understanding of the matching task. We conduct extensive experiments on 10 benchmarks, and LG-SGC significantly outperforms previous models.

Instruction tuned large language models (LLMs), such as ChatGPT, demonstrate remarkable performance in a wide range of tasks.Despite numerous recent studies that examine the performance of instruction-tuned LLMs on various NLP benchmarks, there remains a lack of comprehensive investigation into their ability to understand cross-lingual sociopragmatic meaning (SM), i.e., meaning embedded within social and interactive contexts.This deficiency arises partly from SM not being adequately represented in any of the existing benchmarks.To address this gap, we present SPARROW, an extensive multilingual benchmark specifically designed for SM understanding.SPARROW comprises 169 datasets covering 13 task types across six primary categories (e.g., anti-social language detection, emotion recognition).SPAR-ROW datasets encompass 64 different languages originating from 12 language families representing 16 writing scripts.We evaluate the performance of various multilingual pretrained language models (e.g., mT5) and instruction-tuned LLMs (e.g., BLOOMZ, ChatGPT) on SPARROW through fine-tuning, zero-shot, and/or few-shot learning.Our comprehensive analysis reveals that existing opensource instruction tuned LLMs still struggle to understand SM across various languages, performing close to a random baseline in some cases.We also find that although Chat-GPT outperforms many LLMs, it still falls behind task-specific finetuned models with a gap of 12.19 SPARROW score.Our benchmark is available