The Karhunen-Loève transform (KLT), as a component of block scalar quanti-zation, is optimal among linear orthonormal transforms and allows to obtain the smallest value of mean squared error (MSE) for a given rate of data representation. In this paper we propose a novel and robust optimization scheme designed for arti-ficial neural networks that implies possibly minimal constraints and allows to obtain the KLT up to the permutation of basis vectors. The proposed scheme involves two optimization criteria: (i) minimization of the MSE of signal reconstruction and (ii) minimization of the entropy related criterion, see Fig. 1(a).