Self-knowledge distillation, abbreviated as SKD, exhibits greater computational efficiency than traditional knowledge distillation (KD) because it learns from its own predictions rather than from a pretrained teacher. Existing SKD methods diversify knowledge through auxiliary branches, data augmentation, historical models, and label smoothing. However, previous methods primarily extract knowledge from a single-source teacher, overlooking the diversity and complementarity of various types of teacher knowledge in model learning, thereby limiting performance improvements. In response to this challenge, we propose a pioneering paradigm termed multisource teacher collaboration for self-knowledge distillation (MSTCS-KD), which integrates knowledge from diverse types of teachers to complementarily enhance the model's learning capability. We start by adding lightweight auxiliary branches with different structures in the shallow layers to build the student network, while also incorporating a teacher-guided attention mechanism to support adaptive learning. Then, we perform collaborative distillation by combining "heterogeneous knowledge" from the primary network's deepest layers with "homogeneous knowledge" from the student's outputs on augmented samples. This complementary distillation approach improves the model's ability to learn features, generalize, and enhance trainability. Extensive experiments demonstrate that our method outperforms other state-of-the-art SKD methods across various network architectures and datasets.

Embedded real-time systems impose rigorous timing constraints, where the failure to complete critical tasks within prescribed deadlines can lead to system crashes and catastrophic errors. Control latency, encompassing I/O and interrupt latency, significantly impacts system performance. Previous studies have primarily concentrated on architectural design to meet timing requirements or optimize for performance enhancement. Although the Ti programmable real-time unit (PRU) addresses both timing requirements and control latency, it remains a proprietary commercial chip. This article introduces a deterministic response architecture called Sophon, founded on the open and freely available reduced instruction set computer five (RISC-V). The essential part of this architecture is a tiny and flexible Sophon core that has fixed instruction latency. We propose an enhanced instruction set architecture (ISA) extension interface (EEI) capable of transmitting up to 32 operands in a single instruction, facilitating the development of domain-specific applications. In addition, we have devised two custom instructions to minimize control latency. The Sophon core requires a minimum of 28.6k gate equivalents. Experimental results demonstrate that the Sophon architecture eliminates execution time deviations while preserving low control latency. The highest achievable general purpose I/O (GPIO) flipping frequency is half of the core frequency, and the fastest interrupt latency is three clock cycles.