Efficiency and security issues are significant considerations in the transmission of information. The joint compression and encryption method is an effective way to improve both issues. In this paper, a novel chaos system named logistic coupled sine and exponential function map (LSEM) and a joint JPEG compression and encryption scheme are proposed. Unlike existing JPEG image schemes, a scanning permutation before the differential pulse code modulation (DPCM) is proposed, containing two scanning modes, which can achieve good scrambling performance while reserving file space for subsequent encryption. In addition, an inter-group cross-permutation on random groups of DC coefficients is designed to permute the DC coefficients. The DC coefficients are grouped according to random length and position, and subsequently subjected to inter-group cross-permutation based on random indexes generated with the proposed chaotic system. For AC coefficients, an inter-block permutation method is proposed to change ZRV (zero-run length, value of a non-zero quantized AC coefficient) pairs quantities, which can effectively alter the corresponding block features and the histogram distribution. Experimental results demonstrate that the proposed scheme is reliable in protecting JPEG images while suppressing file size growth and maintaining format compatibility. Notably, the scanning permutation can reserve an average of 0.055% and 0.162% file size for the test images under scanning modes 1 and 2, respectively. The NPCR and UACI results for sensitivity analysis are close to the ideal values. Besides, the change rate of block features is no less than 90% under different quality factors.

Masked autoencoders have become popular training paradigms for self-supervised visual representation learning. These models randomly mask a portion of the input and reconstruct the masked portion according to assigned target representations. In this paper, we show that a careful choice of the target representation is unnecessary for learning good visual representation since different targets tend to derive similarly behaved models. Driven by this observation, we propose a multi-stage masked distillation pipeline and use a randomly initialized model as the teacher, enabling us to effectively train high-capacity models without any effort to carefully design the target representation. On various downstream tasks, the proposed method to perform masked knowledge distillation with bootstrapped teachers (dbot) outperforms previous self-supervised methods by nontrivial margins. We hope our findings, as well as the proposed method, could motivate people to rethink the roles of target representations in pre-training masked autoencoders.

Visual pre-training with large-scale real-world data has made great progress in recent years, showing great potential in robot learning with pixel observations. However, the recipes of visual pre-training for robot manipulation tasks are yet to be built. In this paper, we thoroughly investigate the effects of visual pre-training strategies on robot manipulation tasks from three fundamental perspectives: pre-training datasets, model architectures and training methods. Several significant experimental findings are provided that are beneficial for robot learning. Further, we propose a visual pre-training scheme for robot manipulation termed Vi-PRoM, which combines self-supervised learning and supervised learning. Concretely, the former employs contrastive learning to acquire underlying patterns from large-scale unlabeled data, while the latter aims learning visual semantics and temporal dynamics. Extensive experiments on robot manipulations in various simulation environments and the real robot demonstrate the superiority of the proposed scheme. Videos and more details can be found on https://explore-pretrain-robot.github.io.