Neural networks are widely used for image–related tasks but typically demand considerable computing power. Once a network has been trained, however, its memory‑ and compute‑footprint can be reduced by compression. In this work, we focus on compression through tensorization and low‑rank representations. Whereas classical approaches search for a low‑rank approximation by minimizing an isotropic norm such as the Frobenius norm in weight‑space, we use data‑informed norms that measure the error in function space. Concretely, we minimize the change in the layer’s output distribution, which can be expressed as $\lVert (W - \widetilde{W}) \Sigma^{1/2}\rVert_F$ where $\Sigma^{1/2}$ is the square root of the covariance matrix of the layer’s input and $W$, $\widetilde{W}$ are the original and compressed weights. We propose new alternating least square algorithms for the two most common tensor decompositions (Tucker‑2 and CPD) that directly optimize the new norm. Unlike conventional compression pipelines, which almost always require post‑compression fine‑tuning, our data‑informed approach often achieves competitive accuracy without any fine‑tuning. We further show that the same covariance‑based norm can be transferred from one dataset to another with only a minor accuracy drop, enabling compression even when the original training dataset is unavailable. Experiments on several CNN architectures (ResNet‑18/50, and GoogLeNet) and datasets (ImageNet, FGVC‑Aircraft, Cifar10, and Cifar100) confirm the advantages of the proposed method.

Neural networks (NN) often need to be lightweight and fast for practical deployment. Various NN compression techniques, including tensor decomposition, aim to achieve this goal. Tensor decomposition involves computing a decomposition of the weight tensor of a layer and replacing it with smaller layers using the decomposed weights. Subsequently, fine-tuning is performed to regain lost accuracy. However, while NNs are known to exhibit robustness issues, tensor decomposition for compression has primarily been evaluated on accuracy. In this study, we investigate the impact of tensor decomposition on the robustness of large CNN (Convolutional Neural Network) models. Through multiple experiments on different models trained on ImageNet, we demonstrate that tensor decomposition preserves model robustness. Furthermore, we observe that the choice of fine-tuning learning rate plays a crucial role in determining robustness. A high learning rate may enhance accuracy but significantly compromises robustness. Conversely, a low learning rate can effectively restore model robustness, albeit with a smaller accuracy improvement. These findings offer a practical approach to preserving model robustness without resorting to adversarial learning, thus eliminating the need for additional knowledge about the defense methods used in the original model.

We propose a new framework for tensor decomposition based on trace invariants, which are particular cases of tensor networks. In general, tensor networks are diagrams/graphs that specify a way to "multiply" a collection of tensors together to produce another tensor, matrix or scalar. The particularity of trace invariants is that the operation of multiplying copies of a certain input tensor that produces a scalar obeys specific symmetry constraints. In other words, the scalar resulting from this multiplication is invariant under some specific transformations of the involved tensor. We focus our study on the O(N)-invariant graphs, i.e. invariant under orthogonal transformations of the input tensor. The proposed approach is novel and versatile since it allows to address different theoretical and practical aspects of both CANDECOMP/PARAFAC (CP) and Tucker decomposition models. In particular we obtain several results: (i) we generalize the computational limit of Tensor PCA (a rank-one tensor decomposition) to the case of a tensor with axes of different dimensions (ii) we introduce new algorithms for both decomposition models (iii) we obtain theoretical guarantees for these algorithms and (iv) we show improvements with respect to state of the art on synthetic and real data which also highlights a promising potential for practical applications.

The main objective is to generate kinematic models for the head and neck movements. The motivation comes from our study of individuals with quadriplegia and the need to design rehabilitation aiding devices such as robots and teletheses that can be controlled by head-neck movements. It is then necessary to develop mathematical models for the head and neck movements. Two identification methods have been applied to study the kinematics of head-neck movements of able-body as well as neck-injured subjects. In particular, sagittal plane movements are well modeled by a planar two-revolute-joint linkage. In fact, the motion in joint space seems to indicate that sagittal plane movements may be classified as a single DOF motion. Finally, a spatial three-revolute-joint system has been employed to model 3D head-neck movements.