Audio chaptering, the task of segmenting longform audio into coherent sections, is increasingly important for navigating podcasts, lectures, and videos.Despite its relevance, research remains limited and text-based, leaving key questions unresolved about leveraging audio information, handling ASR errors, and transcript-free evaluation.We address these gaps through three contributions: (1) a systematic comparison between text-based models with acoustic features, a novel audio-only architecture (AudioSeg) operating on learned audio representations, and multimodal LLMs;(2) empirical analysis of factors affecting performance, including transcript quality, acoustic features, duration, and speaker composition; and (3) formalized evaluation protocols contrasting transcript-dependent text-space protocols with transcript-invariant time-space protocols.Our experiments on YTSeg reveal that AudioSeg substantially outperforms text-based approaches, pauses provide the largest acoustic gains, and MLLMs remain limited by context length and weak instruction following, yet MLLMs are promising on shorter audio. 1

Fabian Retkowski, Maike Züfle, Andreas Sudmann, Dinah Pfau, Shinji Watanabe, Jan Niehues, Alexander Waibel. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Text segmentation is a fundamental task in natural language processing, where documents are split into contiguous sections.However, prior research in this area has been constrained by limited datasets, which are either small in scale, synthesized, or only contain well-structured documents.In this paper, we address these limitations by introducing a novel benchmark YTSEG focusing on spoken content that is inherently more unstructured and both topically and structurally diverse.As part of this work, we introduce an efficient hierarchical segmentation model MiniSeg, that outperforms state-ofthe-art baselines.Lastly, we expand the notion of text segmentation to a more practical "smart chaptering" task that involves the segmentation of unstructured content, the generation of meaningful segment titles, and a potential real-time application of the models.