Multi-view clustering typically leverages the consistency and complementarity among views to partition different samples. However, existing deep learning-based methods often face the dilemma between selecting complementary information and capturing essential details: 1) Capturing complementary semantics among views may introduce label-irrelevant redundant information. 2) Only extracting consistent semantic information will cause information loss, hindering the clarity in downstream tasks. To address these issues, we propose a novel method from the perspective of meta-learning to learn clustering-friendly representations with minimal redundancy. Specifically, we train an information compressor to guide the model in describing the original samples as compact as possible with minimal information, thus learning the key semantics with minimized redundancy. Meta-learning bi-level optimization promotes the nested optimization of feature embedding and information compressor. Meanwhile, a semantic puzzle mechanism complements the semantic fragments by exploiting the relationships between low-level features, resulting in a consensus representation with strong discriminative power. We conducted extensive experiments on datasets with various sizes to validate the effectiveness of our model, demonstrating significant performance improvements over several state-of-the-art methods.