Transformers based on Self-Attention (SA) mechanism have demonstrated unrivaled superiority in numerous areas. Compared to RNN-based networks, Transformers can learn the temporal dependency representation of an entire sequence in parallel, while efficiently dealing with long-range dependencies. However, the $\mathcal {O}(L^{2})$O(L2) ($L$L denotes the length of the sequence) computational complexity of the SA mechanism and the high memory usage make the construction cost of the Transformer-based model prohibitively expensive. To address these challenges, we propose a Transformer-like model, HPformer: Low-Parameter Transformer with Temporal Dependency Hierarchical Propagation. HPformer first chunks the sequence into $K$K ($K = \left\lceil \log {L} \right\rceil + 1$K=logL+1, $\left\lceil \cdot \right\rceil$· denotes ceiling operation) sequence segments, then leverages the hierarchical propagation mechanism with $\mathcal {O}(L)$O(L) computational complexity to learn the temporal dependencies between the segments and within the segments, and ultimately generates $K$K vectors as $Key$Key matrices. This reduces the complexity of the SA mechanism from $\mathcal {O}(L^{2})$O(L2) to $\mathcal {O}(L\log {L})$O(LlogL). In addition, we employ a strategy of sharing $Key$Key and $Value$Value matrices between layers to build the HPformer, thus reducing memory usage. Extensive experiments based on public health informatics benchmark and Long-Range Arena (LRA) benchmark have demonstrated that HPformer has advantages over Transformer-based models in terms of memory usage and efficiency.

Social insurance benefits qualification assessment is an important task to ensure that retirees enjoy their benefits according to the regulations. It also plays a key role in curbing social security frauds. In this paper, we report the deployment of the Intelligent Benefit Certification and Analysis (IBCA) platform, an AI-empowered platform for verifying the status of retirees to ensure proper dispursement of funds in Shandong province, China. Based on an improved Gated Recurrent Unit (GRU) neural network, IBCA aggregates missing value interpolation, temporal information, and global and local feature extraction to perform accurate retiree survival rate prediction. Based on the predicted results, a reliability assessment mechanism based on Variational Auto-Encoder (VAE) and Monte-Carlo Dropout (MC Dropout) is executed to perform reliability assessment. Deployed since November 2019, the IBCA platform has been adopted by 12 cities across the Shandong province, handling over 50 terabytes of data. It has empowered human resources and social services, civil affairs, and health care institutions to collaboratively provide high-quality public services. Under the IBCA platform, the efficiency of resources utilization as well as the accuracy of benefit qualification assessment have been significantly improved. It has helped Dareway Software Co. Ltd earn over RMB 50 million of revenue.

In recent years, there has been a lot of research on deep learning for survival prediction in EHR (Electronic Health Record). At present, EHR usually contains multivariate time series data with missing values. How to better predict mortality based on such data is what many studies are currently doing. Most of the mortality prediction methods based on deep learning only pay attention to missing value filling or adjusting the model structure to enhance the mortality prediction performance, but they do not combine these two aspects well. In this paper, we propose MTSSP (Multivariate Time Series for Survival Prediction), a new method that combines missing value filling and time series classification. It takes two representations of the loss pattern: the mask and the time interval, which are combined into the recurrent neural network for the interpolation of the missing value of patient characteristics. When the missing data values are interpolated, the model combines bidirectional RNN and one-dimensional CNN to jointly capture the patient's medical behavior from a global and local perspective to enhance the representation ability of data information, thereby improving the prediction accuracy of the model. In the end, we conducted our mortality prediction experiments on the real-world emergency MIMIC-III dataset and MIMIC-IV dataset. The experimental results demonstrate that the proposed approach has been shown to significantly outperform other approaches.

Accurate prediction of patients’ ICU transfer events is of great significance for improving ICU treatment efficiency. ICU transition prediction task based on Electronic Health Records (EHR) is a temporal mining task like many other health informatics mining tasks. In the EHR-based temporal mining task, existing approaches are usually unable to mine and exploit patterns used to improve model performance. This article proposes a network based on Interval Pattern-Aware, Multi-Scale Interval Pattern-Aware (MSIPA) network. MSIPA mines different interval patterns in temporal EHR data according to the short, medium, and long intervals. MSIPA utilizes the Scaled Dot-Product Attention mechanism to query the contexts corresponding to the three scale patterns. Furthermore, Transformer will use all three types of contextual information simultaneously for ICU transfer prediction. Extensive experiments on real-world data demonstrate that an MSIPA network outperforms state-of-the-art methods.

In China, with the continuous development of national health insurance policies, more and more people have joined the health insurance. How to accurately predict patients future medical treatment behavior becomes a hotspot issue. The biggest challenge in this issue is how to improve the prediction performance by modeling health insurance data with high-dimensional time characteristics. At present, most of the research is to solve this issue by using Recurrent Neural Networks (RNNs) to construct an overall prediction model for the medical visit sequences. However, RNNs can not effectively solve the long-term dependence, and RNNs ignores the importance of time interval of the medical visit sequence. Additionally, the global model may lose some important content to different groups. In order to solve these problems, we propose a Grouping and Global Attention based Time-aware Bidirectional Long Short-Term Memory (GGATB-LSTM) model to achieve medical treatment behavior prediction. The model first constructs a heterogeneous information network based on health insurance data, and uses a tensor CANDECOMP/PARAFAC decomposition method to achieve similarity grouping. In terms of group prediction, a global attention and time factor are introduced to extend the bidirectional LSTM. Finally, the proposed model is evaluated by using real dataset, and conclude that GGATB-LSTM is better than other methods.

Mining Electronic Health Records (EHRs) is of great significance to improve the efficiency and quality of medical services. In recent years, researchers have used deep recurrent neural networks to predict patients' next-period prescriptions. The main challenges of predicting next-period prescriptions are as follows: i) The latent interdependence information which can be utilized to enhance the representation of the data between heterogeneous features is dynamic. However, most existing approaches do not consider capturing and utilizing this information. ii) Conventional recurrent neural networks cannot store historical interdependent information between heterogeneous features and take advantage of it. To address these challenges, We propose a novel attention mechanism named Cross Attention (CA) that can capture the interdependence between two sequences. We further propose three types of recurrent neural networks that can capture and utilize current and historical latent interdependence between the two sequences. Extensive experiments on real-world data demonstrate that DSCA networks can capture the interdependence information between sequences and outperform state-of-the-art methods.

At present, software development is an important way to make our life more convenient and intelligent. With the development of software programming, we have accumulated a lot of expert experience and common sense of domain knowledge. How to effectively organize and reuse these high-quality knowledge has become an urgent problem because reusing high-quality expert knowledge can greatly improve the efficiency of solving problems, especially for the novices of programming. When we encounter the programming problems, the usual solutions to get the answers is to query the search engines or consult the senior developers. However, these solutions have the following limitations: 1) the information in the field of software development is relatively scattered, for example, the demanded information is distributed on different websites. The developers need to query the search engine several times to get the information that they want, which is unfriendly, especially for the novices; 2) there is no such a unified organization form for the information we retrieve, and we need to process it further to get the answers, which is inefficient. To address the above limitations, we propose to build a software development knowledge graph (SoftKG) through Wikipedia taxonomy. Specifically, we propose a framework to build SoftKG based on open source knowledge communities.