Sequential knowledge editing techniques aim to continuously update knowledge in large language models at low cost, preventing models from generating outdated or incorrect information. However, existing sequential editing methods suffer from a significant decline in editing success rates after long-term editing. Through theoretical analysis and experiments, our findings reveal that as the number of edits increases, the model's output increasingly deviates from the desired target, leading to a drop in editing success rates. We refer to this issue as the superimposed noise accumulation problem. Our further analysis demonstrates that the problem is related to the erroneous activation of irrelevant knowledge and conflicts between activated knowledge. Based on this analysis, a method named DeltaEdit is proposed that reduces conflicts between knowledge through dynamic orthogonal constraint strategies. Experiments show that DeltaEdit significantly reduces superimposed noise, achieving a 16.8% improvement in editing performance over the strongest baseline.

Data augmentation through linearly interpolating inputs and modeling targets of random samples has significantly improved predictive performance.However, data augmentation based on linear interpolation generates semantically cluttered and ambiguous text, resulting in ineffective augmentation.To address these issues, in this paper, we propose a novel data augmentation approach called Attention-based Smooth Data Augmentation (ASMix).ASMix accepts the smoothed embeddings of pairwise data predicted by a masked language model (MLM) instead of one-hot embeddings, which makes the inputs more informative and context-rich.We employ the attention mechanism to select discriminative and more-attentioned parts of text hidden representations and mix up the parts containing key semantics in the hidden representations of pairwise data through a multi-token replacement strategy to augment the data of the minority class, which greatly reduces redundant information in the representations that hurts the performance of the model.On several public imbalanced text classification benchmarks, ASMix outperforms state-of-the-art data augmentation methods.In minority classes, the performance improvement of ASMix is particularly prominent.

Hierarchical Text Classification (HTC), which aims to predict text labels organized in hierarchical space, is a significant task lacking in investigation in natural language processing. Existing methods usually encode the entire hierarchical structure and fail to construct a robust label-dependent model, making it hard to make accurate predictions on sparse lower-level labels and achieving low Macro-F1. In this paper, we explore the level dependency and path dependency of the label hierarchy in a generative way for building the knowledge of upper-level labels of current path into lower-level ones, and thus propose a novel PAAM-HiA-T5 model for HTC: a hierarchy-aware T5 model with path-adaptive attention mechanism. Specifically, we generate a multi-level sequential label structure to exploit hierarchical dependency across different levels with Breadth-First Search (BFS) and T5 model. To further improve label dependency prediction within each path, we then propose an original path-adaptive attention mechanism (PAAM) to lead the model to adaptively focus on the path where the currently generated label is located, shielding the noise from other paths. Comprehensive experiments on three benchmark datasets show that PAAM-HiA-T5 greatly outperforms all state-of-the-art HTC approaches especially in Macro-F1.

Short-term road traffic speed prediction is a long-standing topic in the area of Intelligent Transportation System. Apparently, effective prediction of the traffic speed on the road can not only provide timely details for the navigation system concerned and help the drivers to make better path selection, but also greatly improve the road supervision efficiency of the traffic department. At present, some researches on speed prediction based on GPS data, by adding weather and other auxiliary information, using graph convolutional neural network to capture the temporal and spatial characteristics, have achieved excellent results. In this paper, the problem of short-term traffic speed prediction based on GPS positioning data is further studied. For the processing of time series, we innovatively introduce Dynamic Time Warping algorithm into the problem and propose a Long Short-Term Memory with Dynamic Time Warping (D-LSTM) model. D-LSTM model, which integrates Dynamic Time Warping algorithm, can fine-tune the time feature, thus adjusting the current data distribution to be close to the historical data. More importantly, the fine-tuned data can still get a distinct improvement without special treatment of holidays. Meanwhile, considering that the data under different feature distributions have different effects on the prediction results, attention mechanism is also introduced in the model. Our experiments show that our proposed model D-LSTM performs better than other basic models in many kinds of traffic speed prediction problems with different time intervals, and especially significant in the traffic speed prediction on weekends.

The pre-training models such as BERT have achieved great results in various natural language processing problems. However, a large number of parameters need significant amounts of memory and the consumption of inference time, which makes it difficult to deploy them on edge devices. In this work, we propose a knowledge distillation method LRC-BERT based on contrastive learning to fit the output of the intermediate layer from the angular distance aspect, which is not considered by the existing distillation methods. Furthermore, we introduce a gradient perturbation-based training architecture in the training phase to increase the robustness of LRC-BERT, which is the first attempt in knowledge distillation. Additionally, in order to better capture the distribution characteristics of the intermediate layer, we design a two-stage training method for the total distillation loss. Finally, by verifying 8 datasets on the General Language Understanding Evaluation (GLUE) benchmark, the performance of the proposed LRC-BERT exceeds the existing state-of-the-art methods, which proves the effectiveness of our method.

For better user satisfaction and business effectiveness, more and more attention has been paid to the sequence-based recommendation system, which is used to infer the evolution of users’ dynamic preferences, and recent studies have noticed that the evolution of users’ preferences can be better understood from the implicit and explicit feedback sequences. However, most of the existing recommendation techniques do not consider the noise contained in implicit feedback, which will lead to the biased representation of user interest and a suboptimal recommendation performance. Meanwhile, the existing methods utilize item sequence for capturing the evolution of user interest. The performance of these methods is limited by the length of the sequence, and can not effectively model the long-term interest in a long period of time. Based on this observation, we propose a novel CTR model named denoising user-aware memory network (DUMN). Specifically, the framework: (i) proposes a feature purification module based on orthogonal mapping, which use the representation of explicit feedback to purify the representation of implicit feedback, and effectively denoise the implicit feedback; (ii) designs a user memory network to model the long-term interests in a fine-grained way by improving the memory network, which is ignored by the existing methods; and (iii) develops a preference-aware interactive representation component to fuse the long-term and short-term interests of users based on gating to understand the evolution of unbiased preferences of users. Extensive experiments on two real e-commerce user behavior datasets show that DUMN has a significant improvement over the state-of-the-art baselines.

Conventional interactive machine translation typically requires a human translator to validate every generated target word, even though most of them are correct in the advanced neural machine translation (NMT) scenario. Previous studies have exploited confidence approaches to address the intensive human involvement issue, which request human guidance only for a few number of words with low confidences. However, such approaches do not take the history of human involvement into account, and optimize the models only for the translation quality while ignoring the cost of human involvement. In response to these pitfalls, we propose a novel interactive NMT model, which explicitly accounts the history of human involvements and particularly is optimized towards two objectives corresponding to the translation quality and the cost of human involvement, respectively. Specifically, the model jointly predicts a target word and a decision on whether to request human guidance, which is based on both the partial translation and the history of human involvements. Since there is no explicit signals on the decisions of requesting human guidance in the bilingual corpus, we optimize the model with the reinforcement learning technique which enables our model to accurately predict when to request human guidance. Simulated and real experiments show that the proposed model can achieve higher translation quality with similar or less human involvement over the confidence-based baseline.

It is fundamental to detect seismic events reliably and efficiently when processing continuous waveform data recorded by seismic stations. Recently, convolutional neural network (CNN) based detecting methods have been proposed for seismic events detection and obtained great success in this area, where the learning of seismic event detecting network of all seismic stations is considered as one learning task and numerous labeled data need to be collected for training the detecting network. However, they tend to ignore the differences between seismic stations caused by geographic position. Moreover, due to a few seismic activities and high cost of manual data labeling, in some areas, the labeled data for seismic event detecting tasks is insufficient. Under this condition, these methods always encounter over-fitting problem leading to bad detection performance. In this paper, we propose a multi-task based framework based on convolutional neural network for accurate seismic event detection under the condition of insufficient labeled data. Specifically, we first cluster the seismic stations into several station clusters and treat the learning of seismic event detecting network of every station cluster as a learning task, and then we propose a deep multi-task network named detectMTIA among multiple tasks. Experimental results on a real-world seismic dataset with nine stations demonstrate the effectiveness of the proposed framework, especially when the labeled data is insufficient.

Neural machine translation (NMT) heavily relies on parallel bilingual data for training. Since large-scale, high-quality parallel corpora are usually costly to collect, it is appealing to exploit monolingual corpora to improve NMT. Inspired by the law of total probability, which connects the probability of a given target-side monolingual sentence to the conditional probability of translating from a source sentence to the target one, we propose to explicitly exploit this connection to learn from and regularize the training of NMT models using monolingual data. The key technical challenge of this approach is that there are exponentially many source sentences for a target monolingual sentence while computing the sum of the conditional probability given each possible source sentence. We address this challenge by leveraging the dual translation model (target-to-source translation) to sample several mostly likely source-side sentences and avoid enumerating all possible candidate source sentences. That is, we transfer the knowledge contained in the dual model to boost the training of the primal model (source-to-target translation), and we call such an approach dual transfer learning. Experiment results on English-French and German-English tasks demonstrate that dual transfer learning achieves significant improvement over several strong baselines and obtains new state-of-the-art results.

Zero-shot learning deals with the problem when the training domain and the test domain have different class sets of image instances. To tackle the problem of some classes in the test data never appeared in the training set, a most popular approach is to map both images and classes in a common space under the embedding based framework. Nevertheless, most embedding based models suffered from the semantic loss problem. Furthermore, the expressive power is limited by representing classes and images as mere points. To tackle these problems, in this paper, we propose an Energy-Based Zero-shot Learning model (EBZL) to encode the association between class attributes and input images for zero-shot learning. EBZL is composed of two parts. The first part is a variational autoencoder that reduces the input dimension of images with representative hidden representations. By feeding the hidden representations as the input of the second part, the second part works as the energy function part based on the deep Boltzmann machine. Specifically, we adapt tradition deep Boltzmann machine to a supervised setting without changing its property as an undirected probabilistic graphic model, which helps to preserve semantic integrity and circumvents semantic loss problem. We further utilize variational inference techniques and mean-field approximation to reduce time complexity in model training process. Finally, extensive experimental results on several real-world datasets clearly show the effectiveness of our proposed method.

This paper focuses on the problem of finding a few representatives for a given dataset, which have both representation and discrimination ability. To solve this problem, we propose a novel algorithm, called Structure Sparsity based Discriminative Representative Selection (SSDRS), to find a representative subset of data points. The selected representative subset keeps the representation ability based on sparse representation models assuming that each data point can be expressed as a linear combination of those representatives. Meanwhile, we employ the Fisher discrimination criterion to make the coefficient matrix possess small within-class scatter but big between-class scatter, which leads to the discriminant ability of representatives. Since such a selected subset is representative and discriminative, it can be used to properly describe the entire dataset and achieve a good classification performance simultaneously. Experimental results in terms of video summarization and image classification indicate that our proposed algorithm outperforms the state-of-the-art methods.

Clipping Web pages, namely extracting the informative clips (areas) from Web pages, has many applications, such as Web printing and e-reading on small handheld devices. Although many existing methods attempt to address this task, most of them can either work only on certain types of Web pages (e.g., news- and blog-like web pages), or perform semi-automatically where extra user efforts are required in adjusting the outputs. The problem of clipping any types of Web pages accurately in a totally automatic way remains pretty much open. To this end in this study we harness the wisdom of the crowds to provide accurate recommendation of informative clips on any given Web pages. Specifically, we leverage the knowledge on how previous users clip similar Web pages, and this knowledge repository can be represented as a transaction database where each transaction contains the clips selected by a user on a certain Web page. Then, we formulate a new pattern mining problem, mining top-1 qualified pattern, on transaction database for this recommendation. Here, the recommendation considers not only the pattern support but also the pattern occupancy (proposed in this work). High support requires that patterns appear frequently in the database, while high occupancy requires that patterns occupy a large portion of the transactions they appear in. Thus, it leads to both precise and complete recommendation. Additionally, we explore the properties on occupancy to further prune the search space for high-efficient pattern mining. Finally, we show the effectiveness of the proposed algorithm on a human-labeled ground truth dataset consisting of 2000 web pages from 100 major Web sites, and demonstrate its efficiency on large synthetic datasets.

Today's storage systems and database systems are highly complex and configurable, which makes storage management intricate and costly. One critical aspect of storage management, particularly in large storage infrastructures (e.g. cloud storage), is to determine which application data sets to store on which devices. With a mechanism which has the ability to predict the performance of the storage device for any given workload, administrator could automate this process. Therefore, storage device performance prediction has become a critical aspect of self-managed storage systems. To this end, we propose a general smoothing hybrid model (namely SRT-SVR) which combines regression tree (RT) and support vector regression (SVR) to accurately model storage device performance. With this new method, the advantages of the two techniques (i.e. RT and SVR) are completely amalgamated to obtain a more accurate and efficient model without compromising prediction time. In addition, we propose a new workload characterization method which can describe request more accurately. Experiments show that SRT-SVR method and the characterization method used in the storage device modeling can produce more accurate and stable predictions than RT and SVR.