Semantic Overlap Summarization (SOS) is a multi-document summarization task focused on extracting the common information shared across alternative narratives which is a capability that is critical for trustworthy generation in domains such as news, law, and healthcare.We benchmark popular Large Language Models (LLMs) on SOS and introduce PrivacyPolicy-Pairs (3P), a new dataset of 135 high-quality samples from privacy policy documents, which complements existing resources and broadens domain coverage.Using the TELeR prompting taxonomy, we evaluate nearly one million LLM-generated summaries across two SOS datasets and conduct human evaluation on a curated subset.Our analysis reveals strong prompt sensitivity, identifies which automatic metrics align most closely with human judgments, and provides new baselines for future SOS research 1 .