As a critical component in the Internet of Things (IoT) ecosystem, unmanned aerial vehicles (UAVs) play a pivotal role in low-altitude economy applications, where flight safety is essential for reliable and efficient operations. Although significant progress has been made in UAV intelligent fault diagnosis, it is challenging to effectively address the noise issues in complex flight environments, especially when sample noise and label noise coexist. To this end, this paper first investigates the sample-label noise coupling fault diagnosis (SLNFD) problem in UAVs, and proposes a novel two-stage framework called Learnable Wavelet Packet Transform and Scale Graph Label Enhancement (LWPT-SGLE). Specifically, the LWPT subnetwork integrates wavelet packet transform and deep learning models, and a convolutional layer is employed to replace the traditional filter component to adaptively learn denoising parameters for efficient noisy sample reconstruction. The SGLE subnetwork is designed for label noise, which extracts multi-scale data embeddings, identifies clean samples using a small loss criterion, and applies graph embedding learning to relabel the remaining samples. Comprehensive experiments on the RflyMAD dataset demonstrate the superiority of the proposed framework, achieving outstanding denoising performance, with a reconstructed signalR² reaching 0.9974, and outperforming existing methods by at least 7% when handling label noise.

Accurate load prediction ensures edge cluster stability, which is crucial for resource allocation and task scheduling. However, varied service levels in multi-service clusters cause uneven resource distribution, overloading some devices. To address this issue, this paper introduces a multiservice, multitask transformer framework for edge load prediction, referred to as MSM-TFL. This framework applies dual encoders to capture variations in service features over time and leverages the Continuous Time Screening (CTS) method to calculate adjacent time points or identify the most similar time points as inputs to the dual encoders using the Similarity Search Using Service Features (SSUSF) method. In tests conducted on 30 server clusters with five loads, MSM-TFL-Continue outperforms the SOTA approach in terms of prediction errors for 1-, 5-, and 15-minute loads, total CPU time, and user CPU time. Compared with iTransformer, MSM-TFL-Continue reduces the prediction error in terms of the MAE, RSE, RMSE, and MAPE by 77.29%, 70.06%, 73.90%, and 77.47%, respectively, when a 5-minute load is used as an example. The prediction errors in terms of the MAE, RSE, RMSE, and MAPE are decreased by 74.46%, 64.75%, 69.31%, and 72.45%, respectively, by applying MSM-TFL-Similarity.

Recently, deep learning has achieved remarkable breakthroughs in hyperspectral image (HSI) classification tasks, particularly with methods based on convolutional neural networks (CNNs) and transformers. However, these methods have several limitations: 1) the limited receptive field inherent in the convolutional layer greatly hampers capturing feature contextual information on a large scale and 2) transformers cannot establish strong local relationships, making it challenging to characterize complex dependencies between distant pixels and different bands in HSIs. Moreover, as the network complexity increases, so does the number of network parameters. We propose a novel network called the spectral-spatial convolutional fusion long-range dependence transformer network (LRDTN) for HSI classification to address these challenges. LRDTN comprises three key components: dynamic-dependent convolutional (DDC) module, the multiscale enhanced fusion (MsEF) module, and the local–global perception transformer (LGPT). Specifically, the DDC dynamically models local features, while the MsEF integrates information from different scales to capture contextual relationships in HSI features effectively. Additionally, the ability to mine and utilize HSI local-global features and complex long-range dependencies is enhanced by the proposed transformer variant, LGPT. Ultimately, through the ingeniously designed structure of the LRDTN, the model effectively maintains its performance while reducing the number of network parameters. Extensive experiments conducted on four typical HSI datasets, including urban areas, agricultural areas, and swamps, demonstrate the superiority of LRDTN over other state-of-the-art networks. The code is available athttps://github.com/ybyangjing/LRDTN.

Zijun Yao, Chengjiang Li, Tiansi Dong, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Yichi Zhang, Zelin Dai. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Entity alignment typically suffers from the issues of structural heterogeneity and limited seed alignments.In this paper, we propose a novel Multi-channel Graph Neural Network model (MuGNN) to learn alignment-oriented knowledge graph (KG) embeddings by robustly encoding two KGs via multiple channels.Each channel encodes KGs via different relation weighting schemes with respect to self-attention towards KG completion and cross-KG attention for pruning exclusive entities respectively, which are further combined via pooling techniques.Moreover, we also infer and transfer rule knowledge for completing two KGs consistently.MuGNN is expected to reconcile the structural differences of two KGs, and thus make better use of seed alignments.Extensive experiments on five publicly available datasets demonstrate our superior performance (5% Hits@1 up on average).

Chengjiang Li, Yixin Cao, Lei Hou, Jiaxin Shi, Juanzi Li, Tat-Seng Chua. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Joint representation learning of words and entities benefits many NLP tasks, but has not been well explored in cross-lingual settings.In this paper, we propose a novel method for joint representation learning of cross-lingual words and entities.It captures mutually complementary knowledge, and enables cross-lingual inferences among knowledge bases and texts.Our method does not require parallel corpora, and automatically generates comparable data via distant supervision using multi-lingual knowledge bases.We utilize two types of regularizers to align cross-lingual words and entities, and design knowledge attention and crosslingual attention to further reduce noises.We conducted a series of experiments on three tasks: word translation, entity relatedness, and cross-lingual entity linking.The results, both qualitatively and quantitatively, demonstrate the significance of our method.1 https://en.wikipedia.org/wiki/Help: Interlanguage_links• We did qualitative analysis to have an intuitive impression of our embeddings, and quantitative analysis in three tasks: word translation, entity relatedness, and crosslingual entity linking.Experiment results show that our method demonstrates significant improvements in all three tasks.

What prerequisite knowledge should students achieve a level of mastery before moving forward to learn subsequent coursewares?We study the extent to which the prerequisite relation between knowledge concepts in Massive Open Online Courses (MOOCs) can be inferred automatically.In particular, what kinds of information can be leveraged to uncover the potential prerequisite relation between knowledge concepts.We first propose a representation learning-based method for learning latent representations of course concepts, and then investigate how different features capture the prerequisite relations between concepts.Our experiments on three datasets form Coursera show that the proposed method achieves significant improvements (+5.9-48.0%by F1-score) comparing with existing methods.

To enhance the expression ability of distributional word representation learning model, many researchers tend to induce word senses through clustering, and learn multiple embedding vectors for each word, namely multi-prototype word embedding model. However, most related work ignores the relatedness among word senses which actually plays an important role. In this paper, we propose a novel approach to capture word sense relatedness in multi-prototype word embedding model. Particularly, we differentiate the original sense and extended senses of a word by introducing their global occurrence information and model their relatedness through the local textual context information. Based on the idea of fuzzy clustering, we introduce a random process to integrate these two types of senses and design two non-parametric methods for word sense induction. To make our model more scalable and efficient, we use an online joint learning framework extended from the Skip-gram model. The experimental results demonstrate that our model outperforms both conventional single-prototype embedding models and other multi-prototype embedding models, and achieves more stable performance when trained on smaller data.

Massive Open Online Courses (MOOCs), offering a new way to study online, are revolutionizing education. One challenging issue in MOOCs is how to design effective and fine-grained course concepts such that students with different backgrounds can grasp the essence of the course. In this paper, we conduct a systematic investigation of the problem of course concept extraction for MOOCs. We propose to learn latent representations for candidate concepts via an embedding-based method. Moreover, we develop a graph-based propagation algorithm to rank the candidate concepts based on the learned representations. We evaluate the proposed method using different courses from XuetangX and Coursera. Experimental results show that our method significantly outperforms all the alternative methods (+0.013-0.318 in terms of R-precision; p<<0.01, t-test).