Knowledge distillation is widely used to improve generalization in practice, yet its theoretical understanding remains elusive. In the standard distillation setting, a teacher model provides soft predictions to guide the training of a student model. We model teacher and student training as coupled stochastic processes and introduce a distillation divergence, defined as the Kullback-Leibler divergence between these two stochastic kernels. Within this framework, we derive two generalization bounds for the student model relative to the teacher's generalization gap: an upper bound under a sub-Gaussian assumption via algorithmic stability, and a lower bound under a central condition with sharper dependence on the distillation divergence. We further develop a loss-sharpness-aware bound with an explicit tightness regime, showing that the teacher's local flatness can strictly tighten the bound. Additionally, in a linear Gaussian case study, the distillation divergence admits an interpretable decomposition into bias, variance, and rank-bottleneck costs, yielding practical guidance for distillation design.

Recently, with the rapid development of deep learning, the collaborative classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) image has become a research hotspot in remote sensing (RS) technology. However, existing methods either only consider complementary learning of spatial-domain information, or do not take into account the intrinsic dependencies between pixels and overlook the importance difference of pixels. In this letter, we propose a Mamba-Wavelet Cross-Modal Fusion Network with Graph Pooling (MW-CMFNet) for HSI and LiDAR joint classification. First, a Two-Branch Feature Extraction (TBFE) is used to extract spatial and spectral features. Then, in order to dig deeper into the complementary information of different modalities and fully fuse them under the guidance of frequency-domain information, a Mamba-Wavelet Cross-Modal Feature Fusion (MW-CMFF) Module is devised, it aims to utilize Mamba’s outstanding long-range modeling ability to learn complementary information in the spatial and frequency domains, Finally, the Graph Pooling module is designed to sense the intrinsic dependencies of neighbouring pixels and explore the importance difference of pixels, rather than assigning the same weight to different pixels. Experiments on the Houston2013 and Trento datasets show that the MW-CMFNet achieves higher classification accuracy compared to other state-of-the-art methods.

Recent studies have found that compared to single-modal data, the joint classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) multimodal data can utilize their complementary information to further improve the accuracy of land-cover classification. However, due to the significant differences between multimodal data, the complementarity among them is difficult to be fully exploited and utilized, and the features after fusion are not refined and optimized, which limits the further improvement of land-cover classification accuracy. To alleviate these issues, a novel Mamba Cross-Modal Information Fusion Self-Distillation (Mb-CMIFSD) model is designed. Specifically, Mb-CMIFSD first uses conventional convolutional neural networks (CNN) to transform each patch into a token sequence. Second, a Mamba Cross Modal Information Fusion (MCMIF) module is developed to combine cross-modal attention with bidirectional Mamba mechanism, which can better explore the complementarity of multimodal remote sensing (RS) data and obtain more discriminative multimodal fusion features. Finally, a Prototype Constrained Self-Distillation (PCSD) module is designed to utilize the constructed prototype orthogonal regularization knowledge distillation function to further refine cross-modal fusion features, thereby enhancing the robustness and adaptability of feature extraction. The experimental results on three benchmark HSI and LiDAR datasets show that the designed Mb-CMIFSD model has higher classification accuracy compared to other state-of-the-art methods, and the ablation experiments also confirm the positive effect of the designed two key modules.

Transformer-based object detection models usually adopt an encoding-decoding architecture that mainly combines self-attention (SA) and multilayer perceptron (MLP). Although this architecture does not require nonmaximum suppression (NMS) and can really achieve end-to-end object detection, it also suffers from the disadvantage of insufficient multiscale object perception in the image, which leads to low accuracy in detecting small objects. Focusing on these issues, a new full-scale bidirectional interactive attention (FSBDIA) mechanism is constructed, thereby a novel hierarchical full-scale interactive transformer (HFSI-TF) model is designed for object detection in remote sensing image (RSI). First, in order to enhance the multiscale perception ability of the model, the FSBDIA mechanism is designed under the guidance of full-scale information. Then, based on FSBDIA, a hierarchical HFSI-TF encoder is constructed to interactively fuse multilayer feature maps layer by layer, thereby obtaining multiscale encoded features of RSI. Finally, a mixed cross attention (MCA) mechanism is also constructed, and an iterative decoding architecture is designed based on it to improve the accuracy of small object detection. Comparative experiments based on two benchmark datasets (i.e., DIOR and HRSC2016) show that the designed HFSI-TF model can effectively improve the accuracy of object detection in RSI, and the model we designed has superior performance compared to other state-of-the-art methods.

China had conducted some of the most stringent public health measures to control the spread of successive SARS-CoV-2 variants. However, the effectiveness of these measures and their impacts on the associated disease burden have rarely been quantitatively assessed at the national level. To address this gap, we developed a stochastic age-stratified metapopulation model that incorporates testing, contact tracing and isolation, based on 419 million travel movements among 366 Chinese cities. The study period for this model began from September 2022. The COVID-19 disease burden was evaluated, considering 8 types of underlying health conditions in the Chinese population. We identified the marginal effects between the testing speed and reduction in the epidemic duration. The findings suggest that assuming a vaccine coverage of 89%, the Omicron-like wave could be suppressed by 3-day interval population-level testing (PLT), while it would become endemic with 4-day interval PLT, and without testing, it would result in an epidemic. PLT conducted every 3 days would not only eliminate infections but also keep hospital bed occupancy at less than 29.46% (95% CI, 22.73-38.68%) of capacity for respiratory illness and ICU bed occupancy at less than 58.94% (95% CI, 45.70-76.90%) during an outbreak. Furthermore, the underlying health conditions would lead to an extra 2.35 (95% CI, 1.89-2.92) million hospital admissions and 0.16 (95% CI, 0.13-0.2) million ICU admissions. Our study provides insights into health preparedness to balance the disease burden and sustainability for a country with a population of billions.