Knowledge Distillation (KD) transfers knowledge from large models to small models and has recently achieved remarkable success. However, the reliability of existing KD methods in real-world applications, especially under distribution shift, remains underexplored. Distribution shift refers to the data distribution drifts between the training and testing phases, and this can adversely affect the efficacy of KD. In this paper, we propose a unified and systematic framework ShiftKD to benchmark KD against two general distributional shifts: diversity and correlation shift. The evaluation benchmark covers more than 30 methods from algorithmic, data-driven, and optimization perspectives for five benchmark datasets. Our development of ShiftKD conducts extensive experiments and reveals strengths and limitations of current SOTA KD methods. More importantly, we thoroughly analyze key factors in student model training process, including data augmentation, pruning methods, optimizers, and evaluation metrics. We believe ShiftKD could serve as an effective benchmark for assessing KD in real-world scenarios, thus driving the development of more robust KD methods in response to evolving demands. The code will be made available upon publication.

Quaternion adaptive filters (QAFs) are extensively used in processing three- or four-dimensional signals effectively. However, their performance can significantly deteriorate or even diverge when system inputs and outputs are contaminated by complex noises. Therefore, this letter addresses the issue of parameter estimation in the quaternion errors-in-variables (QEIV) in asymmetric noise. First, a novel robust criterion, called improved quaternion minimum error entropy criterion with fiducial points (IQMEEF), is constructed. Then, a minimum total quaternion error entropy algorithm with fiducial points (MTQEEF) is proposed by integrating the IQMEEF criterion with the total least squares (TLS) method, leveraging stochastic gradient and quaternion generalized Hamilton-real (GHR) calculus theory. Finally, simulations validate the superior performance of MTQEEF in the QEIV model under asymmetric noise environments.

In many practical applications, it is difficult or impossible to obtain the exact solution of the mathematical model due to the limitations of solving methods and the complexity of the neural network itself. A natural problem is given as follows: does the exact solution of quaternion-valued neural networks (QVNNs) exist when successively improved approximate solutions can be obtained? Fortunately, the Hyers-Ulam stability happens to be one of the important means to deal with this problem. In this article, the issue of Hyers-Ulam stability of QVNNs with time-varying delays is addressed. First, inspired by the Hyers-Ulam stability of general functional equations, the concept of the Hyers-Ulam stability of QVNNs is proposed along with the QVNNs model. Then, by utilizing the successive approximation method, both delay-dependent and delay-independent Hyers-Ulam stability criteria are obtained to ensure the Hyers-Ulam stability of the QVNNs considered. Finally, a simulation example is given to verify the effectiveness of the derived results.

This paper addresses the issue of state estimation for the quaternion-valued neural networks (QVNNs) with leakage, discrete, and distributed delays by employing the Lyapunov stability theory and the quaternion matrix theory. The criteria are developed in two forms of quaternion-valued linear matrix inequalities (LMIs) and complex-valued LMIs for guaranteeing the existence and stability of state estimators of the delayed QVNNs. Two numerical examples are provided to illustrate the effectiveness of the obtained results.

This paper addresses the problem of stability for continuous-time and discrete-time quaternion-valued neural networks (QVNNs) with linear threshold neurons. Applying the semidiscretization technique to the continuous-time QVNNs, the discrete-time analogs are obtained, which preserve the dynamical characteristics of their continuous-time counterparts. Via the plural decomposition method of quaternion, homeomorphic mapping theorem, as well as Lyapunov theorem, some sufficient conditions on the existence, uniqueness, and global asymptotical stability of the equilibrium point are derived for the continuous-time QVNNs and their discrete-time analogs, respectively. Furthermore, a uniform sufficient condition on the existence, uniqueness, and global asymptotical stability of the equilibrium point is obtained for both continuous-time QVNNs and their discrete-time version. Finally, two numerical examples are provided to substantiate the effectiveness of the proposed results.

This paper addresses the multistability issue for quaternion-valued neural networks (QVNNs) with time delays. By using the inequality technique, sufficient conditions are proposed for the boundedness and the global attractivity of delayed QVNNs. Based on the geometrical properties of the activation functions, several criteria are obtained to ensure the existence of equilibrium points, of which are locally stable. Two numerical examples are provided to illustrate the effectiveness of the obtained results.

This paper addresses the problem of designing associative memories based on quaternion-valued neural networks (QVNNs). A system designing procedure for QVNNs is developed by employing quaternion matrix decomposition, and a given set of states can be assigned as the equilibrium points of the designed QVNNs. Moreover, some sufficient conditions for the asymptotic stability of the equilibrium points are obtained via Lyapunov's direct method. Numerical simulations manifest that the constructed QVNNs work efficiently on storing and retrieving blurred gray-scale and true color images.