This paper explores the task: Chinese spelling correction (CSC), from a fine-grained perspec- tive by recognizing that existing evaluations lack nuanced typology for the spelling errors. This deficiency can create a misleading impres- sion of model performance, incurring an “in- visible” bottleneck hindering the advancement of CSC research. In this paper, we first cate- gorize spelling errors into six types and con- duct a fine-grained evaluation across a wide variety of models, including BERT-based mod- els and LLMs. Thus, we are able to pinpoint the underlying weaknesses of existing state-of- the-art models - utilizing contextual clues and handling co-existence of multiple typos, asso- ciated to contextual errors and multi-typo er- rors. However, these errors occur infrequently in conventional training corpus. Therefore, we introduce new error generation methods to aug- ment their occurrence, which can be leveraged to enhance the training of CSC models. We hope this work could provide fresh insight for future CSC research.

Recent methods address Chinese Spelling Correction (CSC) with either BERT-based models or large language models (LLMs) independently.However, both of them face challenges.BERT-based models are efficient for this task but struggle with limited generalizability to error patterns, thus failing in opendomain CSC.LLMs are advantageous in their extensive knowledge but fall into low efficiency in character-level editing.To address this dilemma, we propose Automatic Corrector Iteration (ACI), a novel model collaboration pipeline to iteratively optimize a BERT-based corrector.This pipeline is free of human annotation, by leveraging the knowledge and reasoning ability of an LLM verifier to provide useful signals for the corrector.Experimental results demonstrate that our pipeline consistently improves the model performance across iterations and significantly outperforms existing data augmentation methods, achieving comparable performance with human annotation.

This paper studies Chinese Spelling Correction (CSC), which aims to detect and correct potential spelling errors in a given sentence. Current state-of-the-art methods regard CSC as a sequence tagging task and fine-tune BERT-based models on sentence pairs. However, we note a critical flaw in the process of tagging one character to another, that the correction is excessively conditioned on the error. This is opposite from human mindset, where individuals rephrase the complete sentence based on its semantics, rather than solely on the error patterns memorized before. Such a counter-intuitive learning process results in the bottleneck of generalizability and transferability of machine spelling correction. To address this, we propose Rephrasing Language Modeling (ReLM), where the model is trained to rephrase the entire sentence by infilling additional slots, instead of character-to-character tagging. This novel training paradigm achieves the new state-of-theart results across fine-tuned and zero-shot CSC benchmarks, outperforming previous counterparts by a large margin. Our method also learns transferable language representation when CSC is jointly trained with other tasks.

Beyond the great cognitive powers showcased by language models, it is crucial to scrutinize whether their reasoning capabilities stem from strong generalization or merely exposure to relevant data.As opposed to constructing increasingly complex logic, this paper probes into the boolean logic, the root capability of a logical reasoner.We find that any pre-trained language models even including large language models only behave like a random selector in the face of multi-nested boolean logic, a task that humans can handle with ease.To empower language models with this fundamental capability, this paper proposes a new self-supervised learning method Curriculum Logical Reasoning (CLR), where we augment the training data with nested boolean logic chain step-by-step, and program the training from simpler logical patterns gradually to harder ones.This new training paradigm allows language models to effectively generalize to much harder and longer-hop logic, which can hardly be learned through naive training.Furthermore, we show that boolean logic is a great foundation for improving the subsequent general logical tasks 1 .