Multi-label text classification (MLTC) assigns multiple labels to a sentence, with the key challenge being capturing label correlations. Existing models prioritize leveraging correlations but often overlook overfitting, while plug-and-play regularization methods fail to preserve correlations effectively. In this paper, we distinguish two types of label correlations: explicit co-occurring correlations and implicit semantic correlations, and propose regularizations on prototypical label embeddings for correlation preservation. Specifically, we first generate the prototypical embedding of multiple co-occurred labels as an intermediate. We then apply a prototypical regularization on the distance between the sentence embedding and corresponding prototypical embedding to alleviate the over-alignment issue caused by binary cross entropy loss and facilitate explicit correlation preservation. We finally extend the vanilla Mixup, which solely mixes multi-hot labels, on prototypical embedding mixing to promote implicit correlation preservation. Empirical studies show the effectiveness of our regularization methods.

Cross-lingual learning aims to transfer knowledge from one natural language to another. Zero-shot cross-lingual named entity recognition (NER) tasks are to train an NER model on source languages and to identify named entities in other languages. Existing knowledge distillation-based models in a teacher-student manner leverage the unlabeled samples from the target languages and show their superiority in this setting. However, the valuable similarity information between tokens in the target language is ignored. And the teacher model trained solely on the source language generates low-quality pseudo-labels. These two facts impact the performance of cross-lingual NER. To improve the reliability of the teacher model, in this study, we first introduce one extra simple binary classification teacher model by similarity learning to measure if the inputs are from the same class. We note that this binary classification auxiliary task is easier, and the two teachers simultaneously supervise the student model for better performance. Furthermore, given such a stronger student model, we propose a progressive knowledge distillation framework that extensively fine-tunes the teacher model on the target-language pseudo-labels generated by the student model. Empirical studies on three datasets across seven different languages show that our presented model outperforms state-of-the-art methods.

The accumulation of time-series data and the absence of labels make time-series Anomaly Detection (AD) a self- supervised deep learning task. Single-normality-assumption- based methods, which reveal only a certain aspect of the whole normality, are incapable of tasks involved with a large number of anomalies. Specifically, Contrastive Learning (CL) methods distance negative pairs, many of which consist of both normal samples, thus reducing the AD performance. Existing multi-normality-assumption-based methods are usually two-staged, firstly pre-training through certain tasks whose target may differ from AD, limiting their performance. To overcome the shortcomings, a deep Contrastive One-Class Anomaly detection method of time series (COCA) is proposed by authors, following the normality assumptions of CL and one-class classification. It treats the original and reconstructed representations as the positive pair of negative-sample-free CL, namely “sequence contrast”. Next, invariance terms and variance terms compose a contrastive one-class loss function in which the loss of the assumptions is optimized by invariance terms simultaneously and the “hypersphere collapse” is prevented by variance terms. In addition, extensive experiments on two real- world time-series datasets show the superior performance of the proposed method achieves state-of-the-art. *The full version of the paper can be accessed at https://arxiv.org/abs/2207.01472

Cross-lingual named entity recognition task is one of the critical problems for evaluating the potential transfer learning techniques on low resource languages.Knowledge distillation using pre-trained multilingual language models between source and target languages have shown their superiority in transfer.However, existing cross-lingual distillation models merely consider the potential transferability between two identical single tasks across both domains.Other possible auxiliary tasks to improve the learning performance have not been fully investigated.In this study, based on the knowledge distillation framework and multitask learning, we introduce the similarity metric model as an auxiliary task to improve the cross-lingual NER performance on the target domain.Specifically, an entity recognizer and a similarity evaluator are first trained in parallel as two teachers from the source domain.Then, two tasks in the student model are supervised by these teachers simultaneously.Empirical studies on the three datasets across 7 different languages confirm the effectiveness of the proposed model.

Overfitting is a common problem when there is insufficient data to train deep neural networks in machine learning tasks.Data augmentation regularization methods such as Dropout, Mixup, and their enhanced variants, are effective and prevalent, and achieve promising performance to overcome overfitting.However, in text learning, most of the existing regularization approaches merely adopt ideas from computer vision without considering the importance of dimensionality in natural language processing.In this paper, we argue that the property is essential to overcome overfitting in text learning.Accordingly, we present a saliency map informed textual data augmentation and regularization framework, which combines Dropout and Mixup, namely DropMix, to mitigate the overfitting problem in text learning.In addition, we design a procedure that drops and patches fine grained shapes of the saliency map under the DropMix framework to enhance regularization.Empirical studies confirm the effectiveness of the proposed approach on 12 text classification tasks.

Time series augmentation is an essential technique in training deep learning models for time series, especially achieving remarkable results in tackling the overfitting problems. However, existing methods fail to specifically protect discriminative features that contribute significantly to the classification results, so these features may be destroyed during the augmentation process. This leads to lower fidelity of augmented time series, which ultimately interferes with classification decisions during inference. To address this issue, we propose an adaptive shapelets preservation approach, named ASP. First, we exploit the saliency map to detect shapelets on the original time series that contain discriminative features. Second, we preserve them during augmentation and assign a proprietary label to each time series. It improves the fidelity of augmented time series and the confidence of their labels, thereby avoiding the risk of interfering with classification decisions. Experimental results on 128 datasets of the UCR2018 archive show that our method ASP outperforms that without augmentation on 98 datasets, and helps the classifier achieve the average accuracy improvement from 71.26% to 75.46%, which is far better than the state-of-the-art approaches.

In the future, vehicles will be equipped with increasingly advanced interactive intelligent electronic devices, which will induce drivers to conduct secondary tasks, thereby leading to distractions. Therefore, the detection and early warning of driver distraction are essential for improving driving safety and pose an important challenge in intelligent transportation systems. Previous studies used traditional machine learning and deep learning transfer models, which have the disadvantages of complicated and time-consuming manual feature engineering, strong subjectivity, and weak generalization performance. In this paper, we propose a fine-grained detection method for driver distraction based on neural architecture search. First, we design an automatic construction algorithm for deep convolutional neural networks based on neural architecture search, which automatically searches for the optimal deep convolutional neural network architecture without human involvement. In addition, we fuse driver-related multisource perception information, use an automatically constructed deep convolutional neural network to extract high-dimensional mapping features, and implement fine-grained detection of various types of driver distraction states. The results on a large-scale multimodal driver distraction dataset demonstrate that our method can efficiently search an optimal deep convolutional neural network, which can quickly converge, and can accurately detect the considered types of driver distraction states, the average detection accuracy reaches 99.7796%; moreover, it has satisfactory robustness.

It is difficult to distinguish complicated distribution characteristics of objects, which limits the performance of two-stage detectors in the field of optical remote sensing images object detection. In this paper, we propose a deep adaptive proposal network (DAPNet), which is a new category prior network (CPN) on the basis of the existing faster region convolutional neural network (Faster RCNN) architecture. The adaptive candidate boxes for each image is obtained by combining the candidate regions and the object number, which are generated by the fine-region proposal network (F-RPN) and the CPN respectively. These adaptive candidate boxes can satisfy the detection tasks in sparse and dense scenes. A set of experimental results verify the superiority of the proposed approach.

With the development of industrial automation and intelligence deepening, industrial robots' installations increase rapidly. The traditional robot operation and management methods are no longer suitable for current requirements. Cloud computing and big data technology together are deemed to form a promising and possible solution to this challenge. From this point of view, this paper proposes a novel industrial robotics cloud platform architecture, and application orchestration and containerized software appliance technology-enabled cloud-edge collaboration mechanism is designed therein. Two exemplary and real-world case studies, supervisory control and data acquisition, and industrial robot predictive maintenance, are further conducted to demonstrate our architecture's generality and versatility.

Objective: The installation of EHR systems can disrupt operations at clinical practice sites, but also lead to improvements in information availability. We examined how the installation of an ambulatory EHR at OB/GYN practices and its subsequent interface with an inpatient perinatal EHR affected providers' satisfaction with the transmission of clinical information and patients' ratings of their care experience. Methods: We collected data on provider satisfaction through 4 survey rounds during the phased implementation of the EHR. Data on patient satisfaction were drawn from Press Ganey surveys issued by the healthcare network through a standard process. Using multivariable models, we determined how provider satisfaction with information transmission and patient satisfaction with their care experience changed as the EHR system allowed greater information flow between OB/GYN practices and the hospital. Results: Outpatient OB/GYN providers became more satisfied with their access to information from the inpatient perinatal triage unit once system capabilities included automatic data flow from triage back to the OB/GYN offices. Yet physicians were generally less satisfied with how the EHR affected their work processes than other clinical and non-clinical staff. Patient satisfaction dropped after initial EHR installation, and we find no evidence of increased satisfaction linked to system integration. Conclusions: Dissatisfaction of providers with an EHR system and difficulties incorporating EHR technology into patient care may negatively impact patient satisfaction. Care must be taken during EHR implementations to maintain good communication with patients while satisfying documentation requirements.

With the development of Service-Oriented technologies, the amount of Web services grows rapidly. QoS-Aware Web service recommendation can help service users to design more efficient service-oriented systems. However, existing methods assume the QoS information for service users are all known and accurate, but in real case, there are always many missing QoS values in history records, which increase the difficulty of the missing QoS value prediction. By considering the user-service-time three dimension context information, we study a Temporal QoS-Aware Web Service Prediction Framework which aims to recommend best candidates to service user's requirements and meanwhile improve the QoS prediction accuracy. One major challenge is that how to deal with the high dimension, sparse QoS value data. Tensor which is known as multi-way array provides a natural representation for such QoS value data. Therefore, we formalize this problem as a tensor factorization model and propose a Tucker Decomposition (TD) algorithm which is able to deal with the triadic relations of user-service-time model. Extensive experiments are conducted based on our real-world QoS dataset collected on Planet-Lab, comprised of service invocation response-time values from 408 users on 5,473 Web services at 56 time periods. Comprehensive empirical studies demonstrate that our approach is more accuracy than other approaches and achieves 100X to 1000X memory space reduction.

With the rapid growth of Web Service in the past decade, the issue of QoS-aware Web service recommendation is becoming more and more critical. Since the Web service QoS information collection work requires much time and effort, and is sometimes even impractical, the service QoS value is usually missing. There are some work to predict the missing QoS value using traditional collaborative filtering methods based on user-service static model. However, the QoS value is highly related to the invocation context (e.g., QoS value are various at different time). By considering the third dynamic context information, a Temporal QoS-aware Web Service Recommendation Framework is presented to predict missing QoS value under various temporal context. Further, we formalize this problem as a generalized tensor factorization model and propose a Non-negative Tensor Factorization (NTF) algorithm which is able to deal with the triadic relations of user-service-time model. Extensive experiments are conducted based on our real-world Web service QoS dataset collected on Planet-Lab, which is comprised of service invocation response-time and throughput value from 343 users on 5817 Web services at 32 time periods. The comprehensive experimental analysis shows that our approach achieves better prediction accuracy than other approaches.

The increasing availability of socially shared media with tags annotated makes it vital for retrieval approaches to precisely detect web content topic semantic and better understand user interest. Most existing methodologies process the queries merely considering user posted keywords and retrieve media labeled with tags that are similar to query words, while ignoring users implicit interests and preferences. This fact stimulates us to develop preference discovering models to reveal the users' latent intents. In this paper, we study the problem of finding user preference and interest from folksonomy corpus and propose a preference-topic model that exploits probabilistic graphical model and Gibbs sampling algorithm to infer the user interested latent semantic topics. The experimental results show that, with the help of the proposed model, preference topics of the web content creators can be effectively discovered. In addition, two exemplified applications are discussed briefly.

This paper presents an global social travel system to assist tourists in their itinerary planning, tour navigation, and travel knowledge sharing. In particular, firstly, we propose an efficient and flexible itinerary planning algorithm for organizing itineraries. Secondly, we design an intelligent tour path planning and navigation system by mining patterns from trajectories and geo-photos shared by other tourists. Thirdly, we provide a framework to monitor the status and events that may affect the predefined itineraries. Finally, the social media module enhances the opportunities of experience (itinerary, trajectory, and travelogue) sharing and helps travelers make better decisions. Concrete demonstrations of our system are also provided in this paper to show the flexibility and applicability of our system.

Dynamic Web service composition is challenging problem and has been intensively investigated in recent years. Nevertheless, most of existing approaches do not provide satisfactory composition results, especially when confronted with a large scale of services. In this paper, we present a novel algorithm called HRLPLA for composing Web services. The algorithm considers functional properties and QoS properties simultaneously. By using hierarchical reinforcement learning, it can deal with large scales of services and generate efficient service compositions. Moreover, the algorithm is suitable for composing Web services in dynamic environment, as reinforcement learning his highly adaptive. We conducted experimental study to verify the effectiveness and efficiency of our method in dynamic service composition.

This paper introduces the concept of temporal logic of actions (short for TLA), with which we can formally specify the behavior of a service, and compose Web services. The approach is demonstrated by an example. A services composition algorithm is presented.