Unlike general text summarization, Query-focused summarization (QFS) is severely limited by insufficient datasets, forcing previous research to transform datasets from other tasks into QFS format for data augmentation. However, this approach has resulted in two problems: the task and traintest gaps. To alleviate these gaps, we propose QFS-CL, a novel in-place data augmentation framework equipped with contrastive learning. Firstly, we design diverse prompts for ChatGPT to paraphrase the original QFS data into high-quality/low-quality document-summary pairs, filling the task gap. Then, instead of directly incorporating the augmented data into the training set, we train the QFS baseline model in a contrastive learning scheme. Specifically, our approach encourages the model to imitate high-quality pairs and distinguish itself from low-quality pairs, enabling the model to learn how to acquire reliable information and avoid extracting invalid information. Our method achieves state-of-the-art performance on Debatepedia and DUC datasets in ROUGE scores, GPT-4, and human evaluations.

In general text generation, models are typically trained on ground truth tokens, where erroneous tokens are not considered during generation. However, during the inference process, erroneous tokens are inevitably generated, and such errors accumulate in the autoregressive generation, leading to exposure bias. To address this problem directly, we propose a token replacement strategy with a sequential decay rate to simulate inference scenarios during training. Firstly, during the training process, we randomly replace tokens in the target text with a certain probability, simulating situations where erroneous tokens may be generated during inference. Then, we simulate inference scenarios by performing probability-decayed replacements from start to end. Since earlier generated tokens can impact the generation of subsequent tokens, the preceding tokens exert a stronger influence. Our method improves the base models’ performance on image captioning, text summarization, and dialogue datasets, achieving state-of-the-art performance on the query-focused summarization dataset.

ChatGPT is instruct-tuned to generate general and human-expected content to align with human preference through Reinforcement Learning from Human Feedback (RLHF), meanwhile resulting in generated responses not salient enough. Therefore, in this case, ChatGPT may fail to satisfy domain requirements in zero-shot settings, leading to poor ROUGE scores. Inspired by the In-Context Learning (ICL) and retelling ability of ChatGPT, this paper proposes PADS, a Pipeline for Assisting ChatGPT in Domain Summarization. PADS consists of a retriever to retrieve similar examples from corpora and a rank model to rerank the multiple candidate summaries generated by ChatGPT. Specifically, given an inference document, we first retrieve an in-context demonstration via the retriever. Then, we require ChatGPT to generate k candidate summaries for the inference document at a time under the guidance of the retrieved demonstration. Finally, the rank model independently scores the k candidate summaries according to their quality and selects the optimal one. We extensively explore dense and sparse retrieval methods to select effective demonstrations for reference and efficiently train the rank model to reflect the quality of candidate summaries for each given summarized document. Additionally, PADS contains merely 400M trainable parameters originating from the rank model and we merely collect 2.5k data to train it. We evaluate PADS on five datasets from different domains, and the result indicates that each module in PADS is committed to effectively guiding ChatGPT to generate salient summaries fitting different domain requirements. Specifically, in the popular summarization dataset Gigaword, PADS achieves over +8 gain on ROUGE-L, compared with the naive ChatGPT in the zero-shot setting.1

More and more end-to-end text spotting methods based on Transformer architecture have demonstrated superior performance. These methods utilize a bipartite graph matching algorithm to perform one-to-one optimal matching between predicted objects and actual objects. However, the instability of bipartite graph matching can lead to inconsistent optimization targets, thereby affecting the training performance of the model. Existing literature applies denoising training to solve the problem of bipartite graph matching instability in object detection tasks. Unfortunately, this denoising training method cannot be directly applied to text spotting tasks, as these tasks need to perform irregular shape detection tasks and more complex text recognition tasks than classification. To address this issue, we propose a novel denoising training method (DNTextSpotter) for arbitrary-shaped text spotting. Specifically, we decompose the queries of the denoising part into noised positional and noised content queries. We use the four Bezier control points of the Bezier center curve to generate the noised positional queries. For the noised content queries, considering that the output of the text in a fixed positional order is not conducive to aligning position with content, we employ a masked character sliding method to initialize noised content queries, thereby assisting in the alignment of text content and position. Additionally, to improve the model's perception of the background, we further utilize an additional loss function for background characters classification in the denoising training part. DNTextSpotter outperforms state-of-the-art methods on four benchmarks-Total-Text, SCUT-CTW1500, ICDAR15, and Inverse-Text-most notably achieving an 11.3% improvement over the best approach on Inverse-Text.