Compared to traditional Computed Tomography (CT) scans and floatation catheters, chest X-ray offers an efficient, safe and timely examination paradigm, with broader range of scenarios (including intensive care units), for the detection of Pulmonary Arterial Hypertension (PAH). However, it is difficult to learn the variable radiological features of PAH from X-rays due to its low resolution and low contrast. To address the above issues, we propose a diversified features learning framework to fully explore the PAH-related representation from chest X-ray. We first employ a Chest Feature Enhancement Attention (CFEA) module to enhance the initial feature representation. Then, we employ the Deep Temporal Anti-Interference Metric Learning (TAIML) module to fully explore the PAH-related features. We incorporate the information on the temporal evolution of patients’ conditions. Specifically, a patient x , after undergoing treatment, may exhibit two possible states: x + (ill) and x − (cured). Therefore, we can define the distance d ( x , x + ) as the intra-class structural distance, and the distance d ( x , x − ) as the inter-class safe distance. Unlike existing metric learning, we adopt a new strategy: we push positive samples towards negative samples, but ensure distance between them is no less than d ( x , x − ) , thereby enhancing intra-class diversity while maintaining discriminability. Meanwhile, we ensure that the distance between positive samples is greater than d ( x , x + ) , thereby preserving the intra-class structure. Through these two steps, we can learn a diversified but discriminative representation of PAH. Comprehensive experiments showed the our model achieved an impressive accuracy of 86.27 % and an AUC of 0.857 in identifying PAH patients. The code is available at https://github.com/zgfdmn/PAH .

Recently, a large number of channel attention blocks are proposed to boost the representational power of deep convolutional neural networks (CNNs). These approaches commonly learn the relationship between global contexts and attention activations by fully-connected layers or linear transformations. However, we empirically find that though many parameters are introduced, these attention blocks may not learn the relationship well. In this paper, we hypothesize that the relationship is predetermined. Based on this hypothesis, we propose a simple yet extremely efficient channel attention block, called Gaussian Context Transformer (GCT), which achieves contextual feature excitation using a Gaussian function that satisfies the presupposed relation-ship. According to whether the standard deviation of the Gaussian function is learnable, we develop two versions of GCT: GCT-B0 and GCT-B1. GCT-B0 is a parameter-free channel attention block by fixing the standard deviation. It directly maps global contexts to attention activations with-out learning. In contrast, GCT-B1 is a parameterized version, which adaptively learns the standard deviation to enhance the mapping ability. Extensive experiments on ImageNet and MS COCO benchmarks demonstrate that our GCTs lead to consistent improvements across various deep CNNs and detectors. Compared with a bank of state-of-the-art channel attention blocks, such as SE [17] and ECA [42], our GCTs are superior in effectiveness and efficiency.

A variety of computer vision tasks benefit significantly from increasingly powerful deep convolutional neural networks. However, the inherently local property of convolution operations prevents most existing models from capturing long-range feature interactions for improved performances. In this paper, we propose a novel module, called Spatially-Aware Context (SAC) block, to learn spatially-aware contexts by capturing multi-mode global contextual semantics for sophisticated long-range dependencies modeling. We enable customized non-local feature interactions for each spatial position through re-weighted global context fusion in a non-normalized way. SAC is very lightweight and can be easily plugged into popular backbone models. Extensive experiments on COCO, ImageNet, and HICO-DET benchmarks show that our SAC block achieves significant performance improvements over existing baseline architectures while with a negligible computational burden increase. The results also demonstrate the exceptional effectiveness and scalability of the proposed approach on capturing long-range dependencies for object detection, segmentation, and image classification, outperforming a bank of state-of-the-art attention blocks.

Squeeze-and-Excitation (SE) block presents a channel attention mechanism for modeling global context via explicitly capturing dependencies across channels. However, we are still far from understanding how the SE block works. In this work, we first revisit the SE block, and then present a detailed empirical study of the relationship between global context and attention distribution, based on which we propose a simple yet effective module, called Linear Context Transform (LCT) block. We divide all channels into different groups and normalize the globally aggregated context features within each channel group, reducing the disturbance from irrelevant channels. Through linear transform of the normalized context features, we model global context for each channel independently. The LCT block is extremely lightweight and easy to be plugged into different backbone models while with negligible parameters and computational burden increase. Extensive experiments show that the LCT block outperforms the SE block in image classification task on the ImageNet and object detection/segmentation on the COCO dataset with different backbone models. Moreover, LCT yields consistent performance gains over existing state-of-the-art detection architectures, e.g., 1.5∼1.7% APbbox and 1.0%∼1.2% APmask improvements on the COCO benchmark, irrespective of different baseline models of varied capacities. We hope our simple yet effective approach will shed some light on future research of attention-based models.