Researchers have explored different ways to improve large language models (LLMs)' capabilities via dummy token insertion in contexts.However, existing works focus solely on the dummy tokens themselves, but fail to leverage the inherent sentence-level structure of natural language.This is a critical oversight, as LLMs acquire linguistic capabilities through exposure to human-generated texts, which are inherently structured at the sentence level.Motivated by this gap, we propose an approach that inserts delimiters at sentence boundaries in LLM inputs, which not only integrates dummy tokens into the context, but also facilitates LLMs with sentence-by-sentence processing behavior during reasoning.Two concrete methods: (1).In-context learning and (2).Supervised finetuning are experimented using 7B models to 600B Deepseek-V3.Our results demonstrate consistent improvements across various tasks, with notable gains of up to 7.7% on GSM8k and 12.5% on DROP.Furthermore, the fine-tuned LLMs can incorporate sentence awareness evidenced by their internal representations.Our work establishes a simple yet effective technique 1 for enhancing LLM's capabilities, offering promising directions for cognitive-inspired LLM enhancement paradigm.

Human and model-generated texts can be distinguished by examining the magnitude of likelihood in language.However, it is becoming increasingly difficult as language model's capabilities of generating human-like texts keep evolving.This study provides a new perspective by using the relative likelihood values instead of absolute ones, and extracting useful features from the spectrum-view of likelihood for the human-model text detection task.We propose a detection procedure with two classification methods, supervised and heuristic-based, respectively, which results in competitive performances with previous zero-shot detection methods and a new state-of-the-art on short-text detection.Our method can also reveal subtle differences between human and model languages, which find theoretical roots in psycholinguistics studies.