Recent years have witnessed significant advancements in Artificial Intelligence (AI), particularly with the rise of Deep Neural Networks fueled by large datasets and increased model complexity. However, the demand for substantial computational resources poses challenges in centralized data scenarios. Edge Intelligence (EI), combining Edge Computing and AI, emerges as a transformative solution for decentralized learning, crucial in the era of IoT proliferation. While Federated Learning (FL) has been a prominent paradigm in decentralized learning, its limitations have prompted researchers to explore alternative solutions using Knowledge Distillation (KD) as a basis. The purpose of this Ph.D. research is to explore KD as a new paradigm for decentralized learning, contribute to enhancing its performance, and study the trade-off between FL and KD in terms of efficiency and effectiveness to identify best practices and insights in EI environments.