Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. Though one could lower the complexity of Transformers by inducing the sparsity in point-wise self-attentions for LTTF, the limited information utilization prohibits the model from exploring the complex dependencies comprehensively. To this end, we propose an efficient Transformer-based model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects: (i) an encoder-decoder architecture incorporating a linear complexity without sacrificing information utilization is proposed on top of sliding-window attention and Stationary and Instant Recurrent Network (SIRN); (ii) a module derived from the normalizing flow is devised to further improve the information utilization by inferring the outputs with the latent variables in SIRN directly; (iii) the inter-series correlation and temporal dynamics in time-series data are modeled explicitly to fuel the downstream self-attention mechanism. Extensive experiments on seven real-world datasets demonstrate that Conformer outperforms the state-of-the-art methods on LTTF and generates reliable prediction results with uncertainty quantification.

Dynamic systems are often monitored with multivariate time series where each dimension represents a local component measured through a (virtual) sensor. Performing accurate diagnostic for dynamic systems while simultaneously taking into account their similarities/distinctions, is a non-trivial task. To this end, we develop an adaptive regularization approach to learning sparse connectivity structures in complex dynamic systems. The learned connectivity networks shed lights on the structural compositions of the system and hence can serve as highly informative inputs for various machine learning tasks such as classification. In particular, we focus on high-dimensional and semi-supervised learning scenarios and present a joint learning approach to recover system-wise connectivity patterns by adaptively constructing a shared, sparsity-inducing regularization template across all systems. The shared template can be physically interpreted and used as a modeling template for analyzing new systems. Moreover, our approach has the flexibility to incorporate supervising information such as must-links and cannot-links for constructing regularization templates. Overall, our approach, named sparse adaptive regularization (SAR), can extract structure-related connectivity features efficiently and effectively, and result in significant improvements for machine learning tasks in dynamic systems. We benchmark our approach against the state-of-the-art methods with real-world data. Our results demonstrate the superiority of our approach.

Predicting treatment medicines is a key task in many intelligent healthcare systems. Prediction of treatment medicines can assist doctors in making informed prescription decisions for patients according to their Electronic Health Records (EHRs). However, predicting treatment medicines is a challenging task due to the following reasons: (1) heterogeneous nature of EHR data that typically includes laboratory results, treatment records, disease conditions, and demographic information; (2) complex correlations among EHR sequences, including inter-correlations between sequences and temporal intra-correlations within each sequence; (3) temporal dynamics of these correlations changing with disease progression. In this paper, we predict treatment medicines for patients with dual adaptive sequential networks (DASNet). Specifically, DASNet is designed with three components. First, a decomposed adaptive long short-term memory network (DA-LSTM) is designed to capture the intra- and inter-correlations in multiple heterogeneous temporal sequences. Then, we develop an attentive meta learning network (AT-MetaNet) to learn dynamic weight parameters for DA-LSTM, thus enabling it to model various correlation structures. Finally, we employ an attentive fusion network (AT-FuNet) to incorporate historical information and collectively fuse representation embeddings of heterogeneous data to predict treatment medicines. Our results on the public MIMIC-III dataset covering 11 medical conditions demonstrate that the proposed end-to-end model can achieve the state-of-the-art prediction performance while providing clinically useful insights.

Numerous state-of-the-art recommendation frameworks employ deep neural networks in Collaborative Filtering (CF). In this paper, we propose a cross feature fusion neural network (CFFNN) for the enhancement of CF. Existing studies overlook either user preferences for various item features or the relationship between item features and user features. To solve this problem, we construct a cross feature fusion network to enable the fusion of user features and item features as well as a self-attention network to determine users’ preferences for items. Specifically, we design a feature extraction layer with multiple MLP (Multilayer Perceptrons) modules to extract both user features and item features. Then, we introduce a cross feature fusion mechanism for an accurate determination of the relationship between different user-item interactions. The features of users and items are crossly embedded and then fed into a prediction network. The attention mechanism enables the model to focus on more effective features. The effectiveness of CFFNN model is demonstrated through extensive experiments on four real-world datasets. The experimental results indicate that CFFNN significantly outperforms the existing state-of-the-art models, with a relative improvement of 3.0 to 12.1 percent on hit ratio (HR) and normalized discounted cumulative gain (NDCG) compared with the baselines.

Graph Neural Networks (GNNs) are prone to adversarial attacks and discriminatory biases. The cutting-edge studies usually adopt a perturbation-invariant consistency regularization strategy without considering the inherent prediction uncertainties, which can lead to unsatisfactory overconfidence for incorrect prediction under intent graph topology or node features attacks. Besides, operating on the complete graph structure is biased towards global level graph noise and brings severe computational issues. In this work, we develop a multi-view confidence-calibrated framework, called MCCNIFTY, for unified fair and stable graph representation learning. At its core is a multi-view uncertainty-aware node embedding learning module derived from evidential theory, including an intra-view evidence calibration, an inter-view evidence fusion, and an uncertainty-aware message passing process in a GNN architecture, which simultaneously optimizes for counterfactual fairness and stability at the sub-graph level. Experimental results on three real-world datasets demonstrate that our method is capable of adequately capturing inherent uncertainties while improving the fairness and stability via subgraph-induced multiview confidence calibration.

The COVID-19 pandemic forces schools to move teaching online and stimulates the development of online tutoring platforms.Although online tutoring platforms provide students the access to learning materials and tools anytime and anywhere, the quality of studies is impeded by the fact that students learn by watching videos, which lacks interactions between teachers and students.Such dilemma prevents us from respectively understanding and improving the online learning patterns and efficiency of students.To achieve this goal, we need to solve three challenges: (1) How can we quantify the study quality of online learning?(2) How can we design an appropriate data structure to describe online study behaviors?(3) How can we model the online study behaviors to better mine online study patterns?To address the challenges, we first propose a new measurement to quantify the online study quality from the perspective of study engagement.We then define a study behavior sequence to describe online study behaviors.The study behavior at each timestamp is an event of a video lecture watching behavior type, such as, watching, dragging forward and dragging backward.Moreover, we develop a neural hawkes process framework (namely EduHawkes ) for online study behavior modeling.The EduHawkes is a novel hierarchical encode-decode architecture with simultaneously optimizing the study behavior prediction task (event-level) and the study quality prediction task (course-level).In the experiments, we apply EduHawkes to the applications of study quality prediction and flippant student identification in order to demonstrate the improved performances of our proposed method on modeling online study behaviors.

Social recommendation tasks exploit social connections to enhance recommendation performance. To fully utilize each user's first-order and high-order neighborhood preferences, recent approaches incorporate influence diffusion process for better user preference modeling. Despite the superior performance of these models, they either neglect the latent individual interests hidden in the user-item interactions or rely on computationally expensive graph attention models to uncover the item-induced sub-relations, which essentially determine the influence propagation passages. Considering the sparse substructures are derived from original social network, we name them as partial relationships between users. We argue such relationships can be directly modeled such that both personal interests and shared interests can propagate along a few channels (or dimensions) of latent users' embeddings. To this end, we propose a partial relationship aware influence diffusion structure via a computationally efficient multi-channel encoding scheme. Specifically, the encoding scheme first simplifies graph attention operation based on a channel-wise sparsity assumption, and then adds an InfluenceNorm function to maintain such sparsity. Moreover, ChannelNorm is designed to alleviate the oversmoothing problem in graph neural network models. Extensive experiments on two benchmark datasets show that our method is comparable to state-of-the-art graph attention-based social recommendation models while capturing user interests according to partial relationships more efficiently.

Partial correlation-based connectivity networks can describe the direct connectivity between features while avoiding spurious effects, and hence they can be implemented in diagnosing complex dynamic multivariate systems. However, existing studies mainly focus on single systems that are ill-equipped for incremental learning. Moreover, related methods estimate temporal connectivity network by imposing only sparse regularization without integrating pattern priors (e.g., inter-system shared pattern and intra-system intrinsic pattern), which have been proven effective in limiting noise interference. To this end, we develop an adaptive connectivity estimation model that incorporates prior patterns, namely Sparse Adaptive Meta-Learning Connectivity Network (SAMCN). Specifically, our model extends ideas of the gradient-based meta-learning to capture inter-system shared prior information by generating fast adaptive initialization parameters for the connectivity matrix. Then, a sparse variational autoencoder is proposed to generate a weight matrix for sparse regularization penalty in reweighted LASSO, which helps extract intra-system intrinsic patterns (local manifold structure). Experimental results on both synthetic data and real-world datasets demonstrate that our method is capable of adequately capturing the aforementioned pattern priors. Further, experiments from corresponding classification tasks validate the strength of the prior pattern-aware features connectivity network in resulting in better classification performance.

In this article, we address the multiple-choice machine comprehension (MC) problem in natural language processing. Existing approaches for MC are usually designed for general cases; however, we specially develop a novel method for solving the multiple-choice MC problem. We take the inspiration generative adversarial networks (GANs) and first propose an adversarial framework for multiple-choice oriented MC, named McGAN . Specifically, our approach is designed as a GAN-based method that unifies both generative and discriminative MC models. Working together, the generative model focuses on predicting relevant answer given a passage (text) and a question; the discriminative model focuses on predicting their relevancy given an answer-passage-question set. Based on the competition via adversarial training in a minimize-maximize game, the proposed method takes advantages from both models. To evaluate the performance, we test our McGAN model on three well-known datasets for multiple-choice MC. Our results show that McGAN can achieve a significant increase in accuracy compared to existing models based on all three datasets, and it consistently outperforms all tested baselines, including state-of-the-art techniques.

In this paper, we develop an efficient parallel heuristic method to solve the global optimization problem associated with the ridesharing system. Based on the carefully formalized problem and objective function, we fully utilize the heuristic characteristics of the algorithm for handling the real-life constraints in ridesharing. Following the principles of simulated annealing, our method is adaptive in handling the matching and route optimization tasks. We develop an efficient parallel scheme with simulated annealing, named PCSA, for solving the global optimization problem for ridesharing. Our algorithm is capable of efficiently addressing the potential of ridesharing by exploiting the mobility information of the ride requests. Based on extensive experiments on large real-world data, we validate the performance of our parallel heuristic algorithm. Our results confirm the effectiveness and efficiency of the proposed method and its superiority over all other benchmarks.

Finding a right doctor with suitable expertise that meets one's health needs is important yet challenging. In this paper, we study the problem of finding high-rated doctors for a specific disease using imbalanced and heterogeneous healthcare experience rating data. We develop a data analytical framework, namely Dr. Right!, which incorporates the so-called network-textual embeddings, together with data-imbalance-aware mixture multi-classification models to rate doctors per specific disease. First, Dr. Right! collects the comments and rating records from patients for doctors on specific diseases from an online hospital and constructs a doctor-patient-disease network, where every edge weight is a pairwise average rating (experience score) among doctors, patients, and diseases. Then, Dr. Right! learns the embeddings of patient experiences from textual comments using the Word2Vec, as well as the embeddings of doctors and diseases from the doctor-patient-disease network via the Node2Vec. The two types of embeddings are fused to represent a doctor-patient pair. With the embedding representations of doctor-patient pairs, Dr. Right! learns an adaptively-weighted mixture multi-classification model to map a doctor-disease pair to an experience rating score, while addressing the challenges of data imbalance and group heterogeneity. Finally, extensive experimental results demonstrate the enhanced performances of Dr. Right! for predicting the disease-specific experience scores of doctors.

Following the recent advances of artificial intelligence, financial text mining has gained new potential to benefit theoretical research with practice impacts. An essential research question for financial text mining is how to accurately identify the actual financial opinions (e.g., bullish or bearish) behind words in plain text. Traditional methods mainly consider this task as a text classification problem with solutions based on machine learning algorithms. However, most of them rely heavily on the hand-crafted features extracted from the text. Indeed, a critical issue along this line is that the latent global and local contexts of the financial opinions usually cannot be fully captured. To this end, we propose a context-aware deep embedding network for financial text mining, named CADEN, by jointly encoding the global and local contextual information. Especially, we capture and include an attitude-aware user embedding to enhance the performance of our model. We validate our method with extensive experiments based on a real-world dataset and several state-of-the-art baselines for investor sentiment recognition. Our results show a consistently superior performance of our approach for identifying the financial opinions from texts of different formats.

Recent years have witnessed an opportunity for improving healthcare efficiency and quality by mining Electronic Medical Records (EMRs). This paper is aimed at developing a treatment engine, which learns from historical EMR data and provides a patient with next-period prescriptions based on disease conditions, laboratory results, and treatment records of the patient. Importantly, the engine takes consideration of both treatment records and physical examination sequences which are not only heterogeneous and temporal in nature but also often with different record frequencies and lengths. Moreover, the engine also combines static information (e.g., demographics) with the temporal sequences to provide personalized treatment prescriptions to patients. In this regard, a novel Long Short-Term Memory (LSTM) learning framework is proposed to model inter-correlations of different types of medical sequences by connections between hidden neurons. With this framework, we develop three multifaceted LSTM models: Fully Connected Heterogeneous LSTM, Partially Connected Heterogeneous LSTM, and Decomposed Heterogeneous LSTM. The experiments are conducted on two datasets: one is the public MIMIC-III ICU data, and the other comes from several Chinese hospitals. Experimental results reveal the effectiveness of the framework and the three models. The work is deemed important and meaningful for both academia and practitioners in the realm of medical treatment and prediction, as well as in other fields of applications where intelligent decision support becomes pervasive.

Parkinson's disease (PD) is a chronic disease that develops over years and varies dramatically in its clinical manifestations. A preferred strategy to resolve this heterogeneity and thus enable better prognosis and targeted therapies is to segment out more homogeneous patient sub-populations. However, it is challenging to evaluate the clinical similarities among patients because of the longitudinality and temporality of their records. To address this issue, we propose a deep model that directly learns patient similarity from longitudinal and multi-modal patient records with an Recurrent Neural Network (RNN) architecture, which learns the similarity between two longitudinal patient record sequences through dynamically matching temporal patterns in patient sequences. Evaluations on real world patient records demonstrate the promising utility and efficacy of the proposed architecture in personalized predictions.

Patent litigation not only covers legal and technical issues, it is also a key consideration for managers of high-technology (high-tech) companies when making strategic decisions. Patent litigation influences the market value of high-tech companies. However, this raises unique challenges. To this end, in this paper, we develop a novel recommendation framework to solve the problem of litigation risk prediction. We will introduce a specific type of patent-related litigation, that is, Section 337 investigations, which prohibit all acts of unfair competition, or any unfair trade practices, when exporting products to the United States. To build this recommendation framework, we collect and exploit a large amount of published information related to almost all Section 337 investigation cases. This study has two aims: (1) to predict the litigation risk in a specific industry category for high-tech companies and (2) to predict the litigation risk from competitors for high-tech companies. These aims can be achieved by mining historical investigation cases and related patents. Specifically, we propose two methods to meet the needs of both aims: a proximal slope one predictor and a time-aware predictor. Several factors are considered in the proposed methods, including the litigation risk if a company wants to enter a new market and the risk that a potential competitor would file a lawsuit against the new entrant. Comparative experiments using real-world data demonstrate that the proposed methods outperform several baselines with a significant margin.

Technology prospecting is a process to evaluate the potential business values of high tech companies from the technology perspective. In this paper, we provide a new view-angle to understand technology prospecting by studying the evolving distributions of technologies in the companies. Specifically, we first exploit topic models to learn technological context in the form of probabilistic distributions of assignees and locations from large-scale patent documents. Then, we develop a matching solution to measure the relationships between patent topics and the description documents of technology terms. In this way, we can obtain the distribution of technologies for each company. In addition, we are able to assess the technology prospecting of a company by a designed indicator, which allows to compare the levels of discrepancies between the emerging technology distributions available as Garner Hype Cycles and the distribution of technologies of the company. Finally, experimental results on real-world patent data show the effectiveness of our approach for technology prospecting.