Knowledge distillation (KD) enables a smaller 'student' model to mimic a larger 'teacher' model by transferring knowledge from the teacher's output or features. However, most KD methods treat all samples uniformly, overlooking the varying learning value of each sample and thereby limiting effectiveness. In this paper, we propose Entropy- based Adaptive Knowledge Distillation (EA-KD), a simple yet effective plug-and-play KD method that prioritizes learning from valuable samples. EA-KD quantifies each sample's learning value by strategically combining the entropy of the teacher and student output, then dynamically reweights the distillation loss to place greater emphasis on high-entropy samples. Extensive experiments across diverse KD frameworks and tasks-including image classification, object detection, and large language model (LLM) distillation-demonstrate that EA-KD consistently enhances performance, achieving state-of-the-art results with negligible computational cost. Our code is available at https://github.com/cpsu00/EA-KD.

Motion mode (M-mode) echocardiography is essential for measuring cardiac dimension and ejection fraction. However, the current diagnosis is time-consuming and suffers from diagnosis accuracy variance. This work resorts to building an automatic scheme through well-designed and well-trained deep learning to conquer the situation. That is, we proposed RAMEM, an automatic scheme of real-time M-mode echocardiography, which contributes three aspects to address the challenges: 1) provide MEIS, the first dataset of M-mode echocardiograms, to enable consistent results and support developing an automatic scheme; For detecting objects accurately in echocardiograms, it requires big receptive field for covering long-range diastole to systole cycle. However, the limited receptive field in the typical backbone of convolutional neural networks (CNN) and the losing information risk in non-local block (NL) equipped CNN risk the accuracy requirement. Therefore, we 2) propose panel attention embedding with updated UPANets V2, a convolutional backbone network, in a real-time instance segmentation (RIS) scheme for boosting big object detection performance; 3) introduce AMEM, an efficient algorithm of automatic M-mode echocardiography measurement, for automatic diagnosis; The experimental results show that RAMEM surpasses existing RIS schemes (CNNs with NL & Transformers as the backbone) in PASCAL 2012 SBD and human performances in MEIS.

In deep learning, optimization plays a vital role. By focusing on image classification, this work investigates the pros and cons of the widely used optimizers and proposes a new optimizer: Perturbed Unit Gradient Descent (PUGD) algorithm with extending normalized gradient operation in tensor within perturbation to update in unit space. Via a set of experiments and analyses, we show that PUGD is locally bounded updating, which means the updating from time to time is controlled. On the other hand, PUGD can push models to a flat minimum, where the error remains approximately constant, not only because of the nature of avoiding stationary points in gradient normalization but also by scanning sharpness in a unit ball. From a series of rigorous experiments, PUGD helps models to gain a state-of-the-art Top-1 accuracy in Tiny ImageNet and competitive performances in CIFAR- {10, 100}. We open-source our code at link: https://github.com/hanktseng131415go/PUGD.

Abstract Multi‐focus image fusion technology solves the problem of limited depth of field of the optical lens. It can extract different focus parts under the same target to synthesize a full‐focus image. This paper proposes an unsupervised dense network for multi‐focus image fusion. In the network, a multi‐scale feature extraction module is employed to extract the spatial details of source images from different scales, and a convolutional block attention module is used to select the useful deep features, and a residual module is used to effectively optimize the performance of the network. By introducing these three modules, the proposed network can effectively extract the shallow and deep features of the source images. Besides, Gaussian‐based Sum‐Modified‐Laplacian (GSML) is used to calculate the activity level of the feature map to generate a decision map. The performance of the proposed method is analyzed from two aspects: visual quality and objective metrics. Experimental results show that compared with nine image fusion methods, the performance of this algorithm is better.

Deep convolutional neural networks have respectively achieved significant success in image super-resolution and colorization. The DNN has a strong capability to generate high quality images. Both colorization and super-resolution (SR) can be regarded as an independent pixel mapping problem, and this work combines these two visual problems into an integrated task. In this work, we propose an end-to-end model for accomplishing single satellite image colorization and SR simultaneously. Our model comprises two phases: features extraction network and recovery network. First, the residual receptive field block structure is introduced in features extraction network to learn better feature representations for image colorization and SR. Residual Receptive Field Block(RRFB) is improved by expanding the receptive field and enhancing the context connection from inception model. Second, the extracted features are transformed to a color high-resolution image by a recovery architecture. In this work, U-net is employed as the key structure of the recovery architecture. Besides, the squeeze-and-excitation blocks and complex residual blocks are incorporated into the proposed model to increase the reconstruction performance. To verify the performance, our method is compared with the state-of-the-art methods of SR and colorization. The experiments show that proposed method can get competitive in visual effect and evaluation index compared with the existing methods. In the end, the panchromatic dataset is also used to validate our model, and a good color high-resolution image can be obtained by giving a gray and low-resolution panchromatic image.

In fuzzy set theory, the distance and entropy measure of intuitionistic fuzzy sets (IFSs) have received extensive concern because of the capability for handling imprecise or uncertain problems. However, most of the existing modeling methods for distance and entropy measure are imperfect in teams of intelligibility and performance. In this work, we proposed a new geometric modeling method that can be simultaneously used for distance and fuzzy entropy modeling of IFSs. We used rigorously mathematical derivation to prove that the proposed distance and fuzzy entropy measures satisfy the properties of the definitions. In the experiments, we applied the proposed distance and fuzzy entropy measure into pattern recognition, medical diagnosis, and multi-attribute decision making to examine the usability of the two measures in practical situations.

Abstract With air pollution having become a global concern, scientists are committed to working on its amelioration. In the field of air pollution prediction, there have been good results in experimental research so far, but few studies have integrated weather forecast information and the properties of air pollution drift. In this work, we propose a novel wind‐sensitive attention mechanism with a long short‐term memory (LSTM) neural network model to predict the air pollution ‐ PM2.5 concentrations by considering the influence of wind direction and speed on the changes of spatial–temporal PM2.5 concentrations in neighbouring areas. Preliminary predictions for PM2.5 are then made by an LSTM neural network regarding neighbouring pollution; these predictions are “paid attention to” and we finally apply an ensemble learning method based on eXtreme Gradient Boosting (XGBoost) to combine the preliminary predictions with weather forecasting to make second phase predictions of PM2.5. The experiment is conducted using PM2.5 data and weather forecast data. Our results illustrate that the proposed method is superior to other methods in predicting PM2.5 concentrations, including multi‐layer perceptron, support vector regression, LSTM neural network, and extreme gradient boosting algorithm.

This paper presents a similarity-based fuzzy learning approach with a two-layer optimization scheme to make fuzzy systems more compact and accuracy. Two ways to improve fuzzy learning algorithms are considered in this paper, including the pruning strategy for simplifying the structure of fuzzy systems and the optimization scheme for parameters optimization. So far as the pruning strategy is concerned, the purpose aims at refining the fuzzy rule base by the similarity analysis of fuzzy sets, fuzzy numbers, fuzzy membership functions or fuzzy rules. Through the similarity analysis, the complete rules can be probably kept by decreasing the redundant rules in the rule base of fuzzy systems. Moreover, the optimization scheme can be regarded as a two-layer parameters optimization in the entire work, because the parameters of the initial fuzzy model have been fine tuning by two phases gradation on layer for discovering a better local minimum.

In the real world, it is extremely difficult to avoid errors; for instance, a doctor may misdiagnose patients. In other words, databases are never free from data entry or other related errors, and many kinds of mistakes are unavoidable in real world data sets. In existing approaches to pattern recognition, handling noisy data in the learning process always produces better generalization performance than if the noise were ignored. In this article, a novel and adaptive weighting mechanism for noise learning tasks is proposed, especially for boosting learning approaches, preventing the algorithm from concentrating on unreasonably noisy learning samples. Several experiments on UC Irvine Machine Learning Repository and a facial expression data set demonstrate the effectiveness of our method.

This article presents a clustering-based approach to fuzzy system identification. In order to construct an effective initial fuzzy model, this article tries to present a modular method to identify fuzzy systems based on a hybrid clustering-based technique. Moreover, the determination of the proper number of clusters and the appropriate location of clusters are one of primary considerations on constructing an effective initial fuzzy model. Due to the above reasons, a hybrid clustering algorithm concerning input, output, generalization and specialization has hence been introduced in this article. Further, the proposed clustering technique, three-part input-output clustering algorithm, integrates a variety of clustering features simultaneously, including the advantages of input clustering, output clustering, flat clustering, and hierarchical clustering, to effectively perform the identification of clustering problem.

There are many methods trying to do relational database estimations with a highly estimated accuracy rate by constructing a fuzzy learning algorithm automatically. However, there exists a conflict between the degree of the interpretability and the accuracy of the approximation in a general fuzzy system. Thus, how to make the best compromise between the accuracy of the approximation and the degree of the interpretability is a significant study of the subject. In order to achieve the best compromise, this article attempts to propose a simple fuzzy learning algorithm to get a positive result in the relational database estimation on the real world database system, including partition determination, automatic membership function, and rule generation, and system approximation.