Few-shot knowledge graph completion (FKGC) aims to enhance knowledge reasoning for newly emerging relations using only a limited number of supporting triples. Given the inherent growth and evolution of knowledge graphs (KGs), FKGC faces greater challenges in continual learning (CL) scenarios, as models tend to rely more heavily on historical knowledge in few-shot settings. Existing studies have mainly focused on preserving old knowledge to mitigate catastrophic forgetting. However, these strategies often overlook that not all knowledge is worth retaining. Some prior knowledge may conflict with newly acquired knowledge, either structurally or semantically, distorting entity representations and relation patterns and weakening few-shot inference for new relations. To bridge this research gap, we propose a novel FKGC framework named CSKU, which achieves dynamic knowledge adaptation by selectively unlearning structural and semantic conflicts. Specifically, to preserve entity representation consistency, we design a graph-structured dynamic progressive unlearning mechanism that adaptively mitigates structural conflicts during training. Meanwhile, we introduce a semantic-guided selective unlearning mechanism that dynamically attenuates the impact of conflicting relations. Experimental results on multiple real-world FKGC datasets demonstrate that CSKU significantly outperforms state-of-the-art baselines in both forgetting mitigation and few-shot completion, validating the effectiveness of the proposed selective knowledge unlearning strategy. Our code and datasets are publicly available. https://anonymous.4open.science/r/CSKU.