Healthcare Representation learning has been a key element to achieving state-of-the-art performance on healthcare prediction. Recent advances based Electronic Healthcare Records(EHRs) are mostly devoted to extracting temporal progression patterns with temporal model and their variants. Although these works have shown excellent performances in healthcare prediction, the unified temporal pattern may not be suitable for individuals in all healthcare conditions. Moreover, some studies ususally introduce complex Deep Neural Networks models and medical prior knowledge to get compact representation, causing great computational burden. In this paper, we propose a general health care representation model, named AEFNet. We only leverage three simple convolution operations and a set of up and down sampling to ensure performance and model complexity equally, which achieves adaptively extract distinct individual key feature in a light manner. AEFNet can shrink and refine highly suitable scale information adaptively and comletely. Breaking traditional fixed convolution scale or multi-scale, AEFNet achieves scale adaptively to extract the most significant information and context relationship. Finally, We validate our method on the public dataset MIMIC-III, and the evaluation results indicate that our method can significantly outperform other remarkable baseline models.

Electrocardiogram (ECG) is one of the critical diagnostic tools in healthcare. Various deep learning models, except Transformers, have been explored and applied to map ECG patterns to heart abnormalities. Transformer models have been adopted from natural language processing to computer vision with advanced features. Most recently, vision transformers show exceptional performances, even on moderate-scale datasets. However, naively applying vision transformers on electrocardiogram datasets leads to poor results. In this paper, we propose a novel network called Beat-aligned Transformer (BaT), a hierarchical Transformer that sufficiently exploits the cyclicity of ECG. We organize and treat an input ECG as multiple aligned beats instead of a single time series. In the BaT, shifted-window-based Transformer blocks (SW Block) are adopted to learn the representation for each beat, and aggregation blocks are designed to exchange information among the beat representations. Nested SW Blocks and aggregation blocks form a beat-aware hierarchical structure of BaT. In this way, the new data format and the BaT hierarchical structure boost Transformer performance on ECG classification. From the experiments on public ECG datasets, we observe BaT outperforms other Transformer-based models and achieves competitive performance compared with other state-of-the-art methods.

Deep neural networks (DNNs) provide the best of class solutions to many supervised tasks due to their powerful function fitting capabilities. However, it is challenging to handle data bias, such as label noise and class imbalance, when applying DNNs to solve real-world problems. Sample reweighting is a popular strategy to tackle data bias, which assigns higher weights to informative samples or samples with clean labels. However, conventional reweighting methods require prior knowledge of the distribution information of data bias, which is intractable in practice. In recent years, meta-learning-based methods have been proposed to learn to assign weights to training samples adaptively by using their online training loss or gradient directions. However, the latent bias distribution cannot be adequately characterized in an online fashion. The online loss distribution changes over the training procedure, making it even harder to perform the sample weight learning. In contrast to past methods, we propose a two-stage training strategy to tackle the above problems. In the first stage, the loss sequences of samples are collected. In the second stage, a subnet with convolutional layers is utilized to learn the mapping from offline sample loss sequence to sample weight adaptively. Guided by a small unbiased meta dataset, this subnet is optimized iteratively with the main classifier network in a meta-learning manner. Empirical results show that our method, called Meta Reweighting with Offline Loss Sequence (MROLS), outperforms state-of-the-art reweighting techniques on most benchmarks. Moreover, the weights of training samples learned via MROLS can be well utilized by other classifiers, which can directly enhance the standard training schema. Our source code is available at https://github.com/Neronjust2017/MROLS.