Intelligent fault diagnosis of mechanical equipment is crucial to ensure reliable operation. However, cloud-based fault diagnosis methods often encounter challenges such as time delays and data loss. Therefore, edge computing-based fault diagnosis has emerged as a promising alternative. However, the limited hardware resources of edge devices in the Industrial Internet of Things (IoT) pose significant challenges in striking a balance between diagnostic capabilities and operational efficiency. This paper introduces a novel lightweight intelligent fault diagnosis method, which is tailored for IoT edge computing scenarios. Optimal weights are trained on cloud computing and inference is performed on edge computing to ensure timely diagnosis. Based on adaptive knowledge distillation, fault knowledge is transferred from a cloud-based deep neural network model (teacher model) to an edge-based lightweight model (student model). By dynamically adjusting the distillation temperature, the student model effectively acquires and deeply understands the knowledge representation from the teacher model. Additionally, we explore practical considerations and potential challenges in the application of the proposed approach. Verification experiments were conducted on two experimental devices, and the NVIDIA Jetson Xavier NX suite was selected as the edge computing platform. The proposed method exhibited significant enhancements in diagnostic accuracy, demonstrating an average improvement of 10.7% compared to existing methods. In lightweight tests, our method achieved an average 25.5% increase in inference speed compared to current approaches. Furthermore, our method reduced memory usage by 96.58% compared to the teacher model, concurrently boosting processing speed by a factor of 8.79.