ABSTRACT As large language models (LLMs) continue to advance, they have demonstrated remarkable performance across a wide range of tasks. However, their application in highly specialized scientific domains—such as quantum physics—remains limited, particularly in educational settings where precision and domain expertise are crucial. Given the abstract nature and mathematical rigor of quantum theory, delivering accurate AI‐assisted support in this field is both technically demanding and educationally valuable. To address these challenges, we propose a retrieval‐augmented generation (RAG) framework that incorporates a domain‐specific knowledge base constructed from authoritative quantum physics textbooks. Our approach introduces customized strategies at both the preretrieval and postretrieval stages to enhance the relevance of retrieved content, mitigate hallucinations, and improve factual accuracy and contextual alignment. This tighter integration between retrieval and generation substantially improves the system's ability to produce reliable and pedagogically meaningful responses. To enable rigorous and systematic evaluation, we present PhysicBench—the first comprehensive benchmark specifically designed for quantum physics question answering in RAG systems. Unlike existing datasets, which are either scarce or ill‐suited to this domain, PhysicBench supports fine‐grained evaluation across key components of the RAG pipeline, including document retrieval, ranking effectiveness, and answer generation accuracy. It fills a critical gap in current resources and provides detailed metrics for performance analysis at each stage. Experimental results show that our proposed system significantly outperforms LLM‐only baselines, highlighting the effectiveness of domain‐aware RAG approaches. Overall, this work contributes both a robust technical solution and a much‐needed benchmark to advance research in AI‐powered education and intelligent question answering within the field of quantum physics.

Knowledge distillation (KD) is a widely used approach for BERT compression, where a larger BERT model serves as a teacher to transfer knowledge to a smaller student model.Prior works have found that distilling a larger BERT with superior performance may degrade student's performance than a smaller BERT.In this paper, we investigate the limitations of existing KD methods for larger BERT models.Through Canonical Correlation Analysis, we identify that these methods fail to fully exploit the potential advantages of larger teachers.To address this, we propose an improved distillation approach that effectively enhances knowledge transfer.Comprehensive experiments demonstrate the effectiveness of our method in enabling larger BERT models to distill knowledge more efficiently.

Knowledge Distillation (KD) has evolved into a practical technology for transferring knowledge from a well-performing model (teacher) to a weak model (student). A counter-intuitive phenomenon known as capacity mismatch has been identified, wherein KD performance may not be good when a better teacher instructs the student. Various preliminary methods have been proposed to alleviate capacity mismatch, but a unifying explanation for its cause remains lacking. In this paper, we propose a unifying analytical framework to pinpoint the core of capacity mismatch based on calibration. Through extensive analytical experiments, we observe a positive correlation between the calibration of the teacher model and the KD performance with original KD methods. As this correlation arises due to the sensitivity of metrics (e.g., KL divergence) to calibration, we recommend employing measurements insensitive to calibration such as ranking-based loss. Our experiments demonstrate that ranking-based loss can effectively replace KL divergence, aiding large models with poor calibration to teach better.

Binary hashing is an effective approach for content-based image retrieval, and learning binary codes with neural networks has attracted increasing attention in recent years. However, the training of hashing neural networks is difficult due to the binary constraint on hash codes. In addition, neural networks are easily affected by input data with small perturbations. Therefore, a sensitive binary hashing autoencoder (SBHA) is proposed to handle these challenges by introducing stochastic sensitivity for image retrieval. SBHA extracts meaningful features from original inputs and maps them onto a binary space to obtain binary hash codes directly. Different from ordinary autoencoders, SBHA is trained by minimizing the reconstruction error, the stochastic sensitive error, and the binary constraint error simultaneously. SBHA reduces output sensitivity to unseen samples with small perturbations from training samples by minimizing the stochastic sensitive error, which helps to learn more robust features. Moreover, SBHA is trained with a binary constraint and outputs binary codes directly. To tackle the difficulty of optimization with the binary constraint, we train the SBHA with alternating optimization. Experimental results on three benchmark datasets show that SBHA is competitive and significantly outperforms state-of-the-art methods for binary hashing.

The knowledge of a well-trained deep neural network (a.k.a. the "teacher") is valuable for learning similar tasks. Knowledge distillation extracts knowledge from the teacher and integrates it with the target model (a.k.a. the "student"), which expands the student's knowledge and improves its learning efficacy. Instead of enforcing the teacher to work on the same task as the student, we borrow the knowledge from a teacher trained from a general label space - in this "Generalized Knowledge Distillation (GKD)," the classes of the teacher and the student may be the same, completely different, or partially overlapped. We claim that the comparison ability between instances acts as an essential factor threading knowledge across tasks, and propose the RElationship FacIlitated Local cLassifiEr Distillation (stance-label confidence between models, ReFilled requires the teacher to reweight the hard tuples pushed forward by the student and then matches the similarity comparison levels between instances. An embedding-induced classifier based on the teacher model supervises the student's classification confidence and adaptively emphasizes the most related supervision from the teacher. ReFilled demonstrates strong discriminative ability when the classes of the teacher vary from the same to a fully non-overlapped set w.r.t. the student. It also achieves state-of-the-art performance on standard knowledge distillation, one-step incremental learning, and few-shot learning tasks.

Few-Shot Learning (FSL) is essential for visual recognition. Many methods tackle this challenging problem via learning an embedding function from seen classes and transfer it to unseen classes with a few labeled instances. Researchers recently found it beneficial to incorporate task-specific feature adaptation into FSL models, which produces the most representative features for each task. However, these methods ignore the diversity of classes and apply a global transformation to the task. In this paper, we propose Global and Local Feature Adaptor (GLoFA), a unifying framework that tailors the instance representation to specific tasks by global and local feature adaptors. We claim that class-specific local transformation helps to improve the representation ability of feature adaptor. Global masks tend to capture sketchy patterns, while local masks focus on detailed characteristics. A strategy to measure the relationship between instances adaptively based on the characteristics of both tasks and classes endow GLoFA with the ability to handle mix-grained tasks. GLoFA outperforms other methods on a heterogeneous task distribution and achieves competitive results on benchmark datasets.

This paper proposes a path correction method for surgical robotic systems performing needle handoff manipulations as part of autonomous execution of surgical suturing. During handoff motions, the position and orientation of the needle is subject to perturbations from the idealized planned pose due to uncertainties in camera-robot calibration and needle localization. If, after a perturbation, the system needs to perform subsequent needle regrasp(s), but the robot still follows the originally planned trajectory out of the path planner [1], it has a lower chance of gripping the needle properly. In order to accommodate these unpredictable needle pose perturbations, the proposed path correction method works locally to direct the needle from the wrong pose to the original pose by partial replanning of the robot motion. The reliability of the proposed method is evaluated with three sets of experiments in a simulation environment.

Meta-learning can extract an inductive bias from previous learning experience and assist the training of new tasks. It is often realized through optimizing a meta-model with the evaluation loss of task-specific solvers. Most existing algorithms sample non-overlapping $\mathit{support}$ sets and $\mathit{query}$ sets to train and evaluate the solvers respectively due to simplicity ($\mathcal{S}$/$\mathcal{Q}$ protocol). Different from $\mathcal{S}$/$\mathcal{Q}$ protocol, we can also evaluate a task-specific solver by comparing it to a target model $\mathcal{T}$, which is the optimal model for this task or a model that behaves well enough on this task ($\mathcal{S}$/$\mathcal{T}$ protocol). Although being short of research, $\mathcal{S}$/$\mathcal{T}$ protocol has unique advantages such as offering more informative supervision, but it is computationally expensive. This paper looks into this special evaluation method and takes a step towards putting it into practice. We find that with a small ratio of tasks armed with target models, classic meta-learning algorithms can be improved a lot without consuming many resources. We empirically verify the effectiveness of $\mathcal{S}$/$\mathcal{T}$ protocol in a typical application of meta-learning, $\mathit{i.e.}$, few-shot learning. In detail, after constructing target models by fine-tuning the pre-trained network on those hard tasks, we match the task-specific solvers and target models via knowledge distillation.

The discriminative knowledge from a high-capacity deep neural network (a.k.a. the "teacher") could be distilled to facilitate the learning efficacy of a shallow counterpart (a.k.a. the "student"). This paper deals with a general scenario reusing the knowledge from a cross-task teacher --- two models are targeting non-overlapping label spaces. We emphasize that the comparison ability between instances acts as an essential factor threading knowledge across domains, and propose the RElationship FacIlitated Local cLassifiEr Distillation (ReFilled) approach, which decomposes the knowledge distillation flow into branches for embedding and the top-layer classifier. In particular, different from reconciling the instance-label confidence between models, ReFilled requires the teacher to reweight the hard triplets push forwarded by the student so that the similarity comparison levels between instances are matched. A local embedding-induced classifier from the teacher further supervises the student's classification confidence. ReFilled demonstrates its effectiveness when reusing cross-task models, and also achieves state-of-the-art performance on the standard knowledge distillation benchmarks. The code of the paper can be accessed at https://github.com/njulus/ReFilled.

Identifying anomalies in multi-view data is a difficult task due to the complicated data characteristics of anomalies. Specifically, there are two types of anomalies in multi-view data–anomalies that have inconsistent features across multiple views and anomalies that are consistently anomalous in each view. Existing multi-view anomaly detection approaches have some issues, e.g., they assume multiple views of a normal instance share consistent and normal clustering structures while anomaly exhibits anomalous clustering characteristics across multiple views. When there are no clusters in data, it is difficult for existing approaches to detect anomalies. Besides, existing approaches construct a profile of normal instances, then identify instances that do not conform to the normal profile as anomalies. The objective is formulated to profile normal instances, but not to estimate the set of normal instances, which results in sub-optimal detectors. In addition, the model trained to profile normal instances uses the entire dataset including anomalies. However, anomalies could undermine the model, i.e., the model is not robust to anomalies. To address these issues, we propose the nearest neighborbased MUlti-View Anomaly Detection (MUVAD) approach. Specifically, we first propose an anomaly measurement criterion and utilize this criterion to formulate the objective of MUVAD to estimate the set of normal instances explicitly. We further develop two concrete relaxations for implementing the MUVAD as MUVAD-QPR and MUVAD-FSR. Experimental results validate the superiority of the proposed MUVAD approaches.

Deep neural networks are becoming increasingly popular in Internet of Things (IoT) applications. Their capabilities of fusing multiple sensor inputs and extracting temporal relationships can enhance intelligence in a wide range of applications. However, one key problem is the missing of adaptation to heterogeneous on-device sensors. These low-end sensors on IoT devices possess different accuracies, granularities, and amounts of information, whose sensing qualities are heterogeneous and vary over time. The existing deep learning frameworks for IoT applications usually treat every sensor input equally over time or increase model capacity in an ad-hoc manner, lacking the ability to identify and exploit the sensor heterogeneities. In this work, we propose SADeepSense, a deep learning framework that can automatically balance the contributions of multiple sensor inputs over time by exploiting their sensing qualities. SADeepSense makes two key contributions. First, SADeepSense employs the self-attention mechanism to learn the correlations among different sensors over time with no additional supervision. The correlations are then applied to infer the sensing qualities and to reassign model concentrations in multiple sensors over time. Second, instead of directly learning the sensing qualities and contributions, SADeepSense generates the residual concentrations that are deviated from the equal contributions, which helps to stabilize the training process. We demonstrate the effectiveness of SADeepSense with two representative IoT sensing tasks: heterogeneous human activity recognition with motion sensors and gesture recognition with the wireless signal. SADeepSense consistently outperforms the state-of-the-art methods by a clear margin. In addition, we show that SADeepSense only imposes little additional resource-consumption burden on embedded devices compared to the corresponding state-of-the-art framework.

This paper addresses text detection utilizing weighted discrete cosine transform (DCT) coefficients as a discrimination statistic. The sum of absolute value of DCT coefficients in each block is considered as an energy statistic for detection. Linear discriminant analysis (LDA) is conducted to calculate the optimal threshold. Different types of weights are employed to strengthen discrimination statistic, which include uniform, binary, linear and quadratic weights. The evaluation of weighted algorithms is conducted in the receiver operating characteristics (ROC) space. The ROC curves show that there is a tradeoff between true positive rate (TPR) and false positive rate (FPR) for all weight configurations. In terms of maximizing the separation between two distributions, experimental results show that the quadratic weighted energy achieves the best recall and precision.