As an important part of healthcare service in hospitals, the emergency system is the most concentrated and complex department for severe patients. With artificial intelligence, the development of manual–machine interaction system effectively improves the efficiency of medical treatment for avoiding doctors from daily manual operations in emergencies, and it promotes emergency doctors to focus more on patients. The ultimate goal of emergency department information establishment aims to construct a complete clinical process with electronic, standard and optimal directions in emergency. It achieves information sharing and quality control among all steps in the process, and forms a complete solution plan for emergency, including: integration, information and networking of pre-hospital first aid, hospital triage, emergency reception, rescue, emergency observation and monitoring. This paper aims to propose and construct a highly integrated emergency clinical information system and focus on resolving the missing and demanding parts of hospital information chain, and the proposed system is available for standardizing and optimizing the clinical workflow in emergency. It realizes the integration, informatization, networking and mobility of the emergent workflow, and upgrades the efficiency and management level in emergency. For changing the conventional mode in clinics and optimizing the process of emergent treatment, it also develops an application that closely meets the actual demands of emergency department in hospitals. It also provides the statistical inquiry and analysis of daily business and quality indicators to manifest the medical performance and quality management for hospital emergency department in clinics.

Imbalanced small datasets are common scenarios in the field of machine learning for medical imaging, especially in real-world clinical applications. Many existing works focus on synthesize new images via data generation. However, generative methods cannot ensure reliability for medical images where categories cannot be easily distinguished, such as the pathologic complete response (pCR) evaluation via MRIs in cancer prognosis. Meanwhile, few-shot learning can deal with training with small datasets, but it depends on a balanced data distribution and a large number of image categories. In this paper, we propose an image similarity comparison classification network, referred to as Cross-Supervision Similarity Network (CSSN), using cross-supervision between class features and patch features. CSSN transforms the classification task into comparison task by calculating similarity scores at both patch and class scales, effectively training on imbalanced small datasets with limited categories. To balance the training difficulty of the two similarity branches, soft logarithmic supervision is used to construct soft labels between them. Through experiments on PCR-ISD, we observe significant performance improvements of 15% in F1 score, 6% in accuracy and 9 % in balanced accuracy over existing methods, indicating the superiority of our method in identifying minority classes and enhancing classification capabilities. Extensive experiments on three datasets and ablation experiments confirm the effectiveness and generalization ability of the proposed method. The source code is available at https://github.com/lxy-146/CSSN_TMI.

Discrete latent representation techniques, such as Vector Quantization (VQ) and Sparse Coding (SC), have demonstrated superior image reconstruction and generation quality compared to continuous representation methods in Variational Autoencoders (VAEs). However, existing approaches often treat the latent representations of an image independently in their discrete representation space, neglecting both the inherent structural information within each representation and the correlations among them. This oversight leads to coarse representations and suboptimal generated results. In this paper, we address these limitations by introducing correlations among and within the latent representations of individual images in the latent discrete space of VAEs using sparse coding. We impose two-dimensional structural information through adaptive thresholding, enhancing local structure in image representations while suppressing noise via parsimonious representation with a learned dictionary. Empirical studies on three real benchmark datasets, including a clinical Ultrasound dataset, BSDS500, and mini-Imagenet, demonstrate that our proposed model preserves fine-grained details in image reconstruction and significantly outperforms baseline models of SC-VAE and VQ-VAE across objective and subjective image quality metrics. Particularly noteworthy are the substantial performance improvements observed on the ultrasound dataset, where structure information is crucial. Specifically, we observe significant performance improvements of 7.68 % and 17.03 % in SSIM, 3.25 dB and 6.58 dB in PSNR, 0.15 and 0.24 in LPIPS, 45.38 and 84.05 in FID over SC-VAE and VQ-VAE, respectively, indicating the superiority of our method in terms of image reconstruction quality and fidelity.

Contemporary domain generalization methods have demonstrated effectiveness in aiding the generalized diagnosis of medical images with multi-source data by joint optimization. However, the centralized training paradigm employed by these approaches becomes infeasible when data are non-shared across domains due to the high privacy of medical data. Despite attempts by existing federated domain generalization methods to address this issue, the simultaneous attainment of strict privacy protection and a satisfactory level of generalization ability on out-of-distribution data remains a persistent challenge. In this paper, to tackle this challenging problem, we propose a novel approach called the Bilateral Proxy Framework (BPF). The BPF leverages the client-side proxies to facilitate the strict privacy-preserving communications with the server and ensure smoother and more stable convergences of local models through mutual distillation. Meanwhile, the server-side proxy adopts a distance-based strategy and a parameter moving average scheme, which enhances the stability and robustness of the global model, particularly by averting abrupt parameter changes that could result in fluctuations or overfitting. Through these advancements, our framework strives to enhance the generalization capability of the global model, enabling more accurate and reliable medical image diagnosis in federated settings. The effectiveness of our method is demonstrated with superior performance over state-of-the-arts on both simulated and real-world distribution medical image diagnosis tasks.

Fundus image registration holds significant importance in medical image analysis. This research addresses the challenges inherent in fundus image registration through the introduction of a novel framework, denoted as Reliable Cosine Matching via Neighborhood Consensus (RCM-NC). The framework aims to enhance the precision and reliability of automatic fundus image registration by refining both the pre-establishment of pairing relations and the removal of mismatches. Key innovations include the incorporation of a Nearest Neighbor Distance Ratio Extension (NNDRE) to broaden the spectrum of initial matches, while simultaneously preserving the accuracy of matching outcomes. Furthermore, Reliable Cosine Matching (RCM) strategies leverage neighborhood consensus to scrutinize and validate matching pairs, improving the overall robustness of the registration process. Empirical assessments on the public fundus image registration dataset demonstrate the superior performance of the proposed method compared to traditional and deep learning approaches.

Microvascular invasion (MVI) is an independent risk factor for postoperative recurrence of hepatocellular carcinoma (HCC). Preoperative MVI grading is beneficial for patients recovery and survival. However, preoperative MVI grading is primarily accomplished through magnetic resonance imaging (MRI), which is challenging due to the heterogeneity of tumors and the characteristics of MVI. In this paper, we propose a cascade network that extracts fine-grained information from multimodal MRI images to assist in accurate MVI grading. We extract fine-grained features from different modalities and integrate them using an attention-based module called Multimodal Fine-Grained generator (M-FG) to obtain finegrained features from multimodal MRIs. Extensive experiments show our MVI grading network achieved an accuracy of 0.77, up to 10% improvement compared to the comparative methods, which validates that our method effectively utilizes fine-grained features from different modalities and improves performance of MVI grading. The codes are available at https://github.com/lxy-146/FG_MVInet

To design a dialogue model and standard artificial programming interface (API), this paper designs an intelligent dialogue support for medical service systems and improving hospital intelligent serviceability, which combines patient pre-diagnosis, diagnosis, and post-diagnosis services with artificial intelligence depth. This is done via an intelligent man–machine dialogue support platform (MMDSP) suitable for medical services based on a multi-dimensional disease model and outpatient knowledge base with artificial intelligence. As a result of the intelligent service capability of the hospital service systems and platforms, patients’ medical experiences have significantly improved. The platform standardizes the multi-channel service process, improves patient service efficiency, and reduces the cost of human resources and business knowledge learning. In addition, the integration of intelligent multi-system service information provides data support for hospitals to serve patients accurately and has good application and promotion value.

Techniques of dictionary learning and sparse representation are popular in recent study on image denoising, including classic K-SVD and its variants. The extension of K-SVD to its deep structure learned in an end-to-end way shows the state-of-the-art denoising performance with a great computation efficiency. However, we notice that the current learning framework takes images patches as independent samples, which ignores the inherent correlation among the patches. In this paper, we propose a deep K-SVD denoising network with attention mechanism to enhance the correlation within and among the patches. We impose the two-dimensional correlation on the intermediate parameters during the sparse representation procedure to achieve more smoothing and local-structure enhanced image features. Extensive numerical experiments using public data are conducted. The results on two datasets show that the proposed network achieves an average improvement of 0.81dB in peak signal-to-noise ratio (PSNR), 1.66% in the structural similarity (SSIM) and more than 90% in the convergence rate comparing to its counterpart, which demonstrate the efficiency and the competitive performance of our proposed network.

Recently, there has been significant progress in medical image segmentation utilizing deep learning techniques. However, these achievements largely rely on the supposition that the source and target domain data are identically distributed, and the direct application of related methods without addressing the distribution shift results in dramatic degradation in realistic clinical environments. Current approaches concerning the distribution shift either require the target domain data in advance for adaptation, or focus only on the distribution shift across domains while ignoring the intra-domain data variation. This paper proposes a domain-aware dual attention network for the generalized medical image segmentation task on unseen target domains. To alleviate the severe distribution shift between the source and target domains, an Extrinsic Attention (EA) module is designed to learn image features with knowledge originating from multi-source domains. Moreover, an Intrinsic Attention (IA) module is also proposed to handle the intra-domain variation by individually modeling the pixel-region relations derived from an image. The EA and IA modules complement each other well in terms of modeling the extrinsic and intrinsic domain relationships, respectively. To validate the model effectiveness, comprehensive experiments are conducted on various benchmark datasets, including the prostate segmentation in magnetic resonance imaging (MRI) scans and the optic cup/disc segmentation in fundus images. The experimental results demonstrate that our proposed model effectively generalizes to unseen domains and exceeds the existing advanced approaches.

Despite the recent success of long-tailed object detection, almost all long-tailed object detectors are developed based on the two-stage paradigm. In practice, one-stage detectors are more prevalent in the industry because they have a simple and fast pipeline that is easy to deploy. However, in the long-tailed scenario, this line of work has not been explored so far. In this paper, we investigate whether one-stage detectors can perform well in this case. We discover the primary obstacle that prevents one-stage detectors from achieving excellent performance is: categories suffer from different degrees of positive-negative imbalance problems under the long-tailed data distribution. The conventional focal loss balances the training process with the same modulating factor for all categories, thus failing to handle the long-tailed problem. To address this issue, we propose the Equalized Focal Loss (EFL) that rebalances the loss contribution of positive and negative samples of different categories independently according to their imbalance degrees. Specifically, EFL adopts a category-relevant modulating factor which can be adjusted dynamically by the training status of different categories. Extensive experiments conducted on the challenging LVIS v1 benchmark demonstrate the effectiveness of our proposed method. With an end-to-end training pipeline, EFL achieves 29.2% in terms of overall AP and obtains significant performance improvements on rare categories, surpassing all existing state-of-the-art methods. The code is available at https: //github.com/ModelTC/EOD.

Dynamic scene deblurring is a challenging problem as it is difficult to be modeled mathematically. Benefiting from the deep convolutional neural networks, this problem has been significantly advanced by the end-to-end network architectures. However, the success of these methods is mainly due to simply stacking network layers. In addition, the methods based on the end-to-end network architectures usually estimate latent images in a regression way which does not preserve the structural details. In this paper, we propose an exemplar-based method to solve dynamic scene deblurring problem. To explore the properties of the exemplars, we propose a siamese encoder network and a shallow encoder network to respectively extract input features and exemplar features and then develop a rank module to explore useful features for better blur removing, where the rank modules are applied to the last three layers of encoder, respectively. The proposed method can be further extended to the way of multi-scale, which enables to recover more texture from the exemplar. Extensive experiments show that our method achieves significant improvements in both quantitative and qualitative evaluations.

The automatic and accurate segmentation of the prostate cancer from the multi-modal magnetic resonance images is of prime importance for the disease assessment and follow-up treatment plan. However, how to use the multi-modal image features more efficiently is still a challenging problem in the field of medical image segmentation. In this paper, we develop a cross-modal self-attention distillation network by fully exploiting the encoded information of the intermediate layers from different modalities, and the generated attention maps of different modalities enable the model to transfer significant and discriminative information that contains more details. Moreover, a novel spatial correlated feature fusion module is further employed for learning more complementary correlation and non-linear information of different modality images. We evaluate our model in five-fold cross-validation on 358 MRI images with biopsy confirmed. Without bells and whistles, our proposed network achieves state-of-the-art performance on extensive experiments.

The application of deep learning to medical image segmentation has been hampered due to the lack of abundant pixel-level annotated data. Few-shot Semantic Segmentation (FSS) is a promising strategy for breaking the deadlock. However, a high-performing FSS model still requires sufficient pixel-level annotated classes for training to avoid overfitting, which leads to its performance bottleneck in medical image segmentation due to the unmet need for annotations. Thus, semi-supervised FSS for medical images is accordingly proposed to utilize unlabeled data for further performance improvement. Nevertheless, existing semi-supervised FSS methods has two obvious defects: (1) neglecting the relationship between the labeled and unlabeled data; (2) using unlabeled data directly for end-to-end training leads to degenerated representation learning. To address these problems, we propose a novel semi-supervised FSS framework for medical image segmentation. The proposed framework employs Poisson learning for modeling data relationship and propagating supervision signals, and Spatial Consistency Calibration for encouraging the model to learn more coherent representations. In this process, unlabeled samples do not involve in end-to-end training, but provide supervisory information for query image segmentation through graph-based learning. We conduct extensive experiments on three medical image segmentation datasets (i.e. ISIC skin lesion segmentation, abdominal organs segmentation for MRI and abdominal organs segmentation for CT) to demonstrate the state-of-the-art performance and broad applicability of the proposed framework.

Automatic liver tumor detection can assist doctors to make effective treatments. However, how to utilize multi-modal images to improve detection performance is still challenging. Common solutions for using multi-modal images consist of early, inter-layer, and late fusion. They either do not fully consider the intermediate multi-modal feature interaction or have not put their focus on tumor detection. In this paper, we propose a novel multi-scale intermediate multi-modal fusion detection framework to achieve multi-modal liver tumor detection. Unlike early or late fusion, it maintains two branches of different modal information and introduces cross-modal feature interaction progressively, thus better leveraging the complementary information contained in multi-modalities. To further enhance the multi-modal context at all scales, we design a multi-modal enhanced feature pyramid. Extensive experiments on the collected liver tumor magnetic resonance imaging (MRI) dataset show that our framework outperforms other state-of-the-art detection approaches in the case of using multi-modal images.

Coarse-to-fine models and cascade segmentation architectures are widely adopted to solve the problem of large scale variations in medical image segmentation. However, those methods have two primary limitations: the first-stage segmentation becomes a performance bottleneck; the lack of overall differentiability makes the training process of two stages asynchronous and inconsistent. In this paper, we propose a differentiable two-stage network architecture to tackle these problems. In the first stage, a localization network (L-Net) locates Regions of Interest (RoIs) in a detection fashion; in the second stage, a segmentation network (S-Net) performs fine segmentation on the recalibrated RoIs; an RoI recalibration module between L-Net and S-Net eliminating the inconsistencies. Experimental results on the public dataset show that our method outperforms state-of-the-art coarse-to-fine models with comparable computation.

A common problem in econometrics, statistics, and machine learning is to estimate and make inference on functions that satisfy shape restrictions. For example, distribution functions are nondecreasing and range between zero and one, height growth charts are nondecreasing in age, and production functions are nondecreasing and quasi-concave in input quantities. We propose a method to enforce these restrictions ex post on generic unconstrained point and interval estimates of the target function by applying functional operators. The interval estimates could be either frequentist confidence bands or Bayesian credible regions. If an operator has reshaping, invariance, order-preserving, and distance-reducing properties, the shape-enforced point estimates are closer to the target function than the original point estimates and the shape-enforced interval estimates have greater coverage and shorter length than the original interval estimates. We show that these properties hold for six different operators that cover commonly used shape restrictions in practice: range, convexity, monotonicity, monotone convexity, quasi-convexity, and monotone quasi-convexity, with the latter two restrictions being of paramount importance. The main attractive property of the post-processing approach is that it works in conjunction with any generic initial point or interval estimate, obtained using any of parametric, semi-parametric or nonparametric learning methods, including recent methods that are able to exploit either smoothness, sparsity, or other forms of structured parsimony of target functions. The post-processed point and interval estimates automatically inherit and provably improve these properties in finite samples, while also enforcing qualitative shape restrictions brought by scientific reasoning. We illustrate the results with two empirical applications to the estimation of a height growth chart for infants in India and a production function for chemical firms in China.

The automatic segmentation of prostate cancer (PCa) from the multi-modal magnetic resonance imaging (MRI) is of prime importance for the initial staging and prognosis of patients. Nevertheless, the challenge of how to utilize multi-modal image features more efficiently still needs resolving, especially in the segmentation scenario. In this paper, we propose a crossmodal self-attention distillation network that can fully exploit the encoded information of the intermediate layers from different modalities, and the learned attention maps of different modalities are then transferred among modalities to provide significant spatial information with more details incorporated. Furthermore, we propose a novel spatial correlated feature fusion module that is able to learn more complementary correlation and nonlinear information from different modality images. To evaluate the effectiveness of the proposed approach, we conduct extensive experiments on the PCa MRI dataset, and the experiment results demonstrate that our proposed approach could achieve state-of-the-art performance.

Ultrasound images are often contaminated by speckle noise during the acquisition process, which influences the performance of subsequent applications. The paper introduces a nonconvex low-rank matrix approximation model for ultrasound images restoration, which integrates the weighted nuclear norm minimization (WNNM) and data fidelity term. WNNM can adaptively assign weights on different singular values to preserve more details in restored images. The fidelity term about ultrasound images do not be utilized in existing low-rank ultrasound denoising methods. This optimization question can effectively solved by alternating direction method of multipliers (ADMM). The experimental results on simulated images and real medical ultrasound images demonstrate the excellent performance of the proposed method compared with other four state-of-the-art methods.

Lung cancer has been the most prevalent cancer in the world and an effective way to diagnose the cancer at the early stage is to detect the pulmonary nodule by computer-aided system. However, the size of the pulmonary nodules varies and the one with small diameter is generally one of the most difficult cases to diagnose. Under this condition, traditional convolution network based nodule classification methods fail to achieve satisfied result due to the miss of tiny but vital features by the pooling operation. To tackle this problem, we propose a novel 3D spatial pyramid dilated convolution network to classify the malignancy of the pulmonary nodules. Instead of using the pooling layers, we utilize the 3D dilated convolution to capture and preserve more detailed characteristic information of the nodules. Moreover, a multiple receptive field fusion strategy is applied to extract the multi-scale features from the nodule CT images. Extensive experimental results show that our model achieves a better result with an accuracy of 88.6% which outperforms other state-of-the-art methods.

Recent advances in saliency detection have utilized deep learning to obtain high-level features to detect salient regions. These advances have demonstrated superior results over previous works that utilize hand-crafted low-level features for saliency detection. In this paper, we propose a new multilayer Convolutional Neural Network (CNN) model to learn high-level features for saliency detection. Compared to other methods, our method presents two merits. First, when performing features extraction, apart from the convolution and pooling step in our method, we add Restricted Boltzmann Machine (RBM) into the CNN framework to obtain more accurate features in intermediate step. Second, in order to deal with case of non-linear classification, we add the Deep Belief Network (DBN) classifier at the end of this model to classify the salient and non-salient regions. Quantitative and qualitative experiments on three benchmark datasets demonstrate that our method performs favorably against the state-of-the-art methods.

Cloud detection in multi-temporal optical remote sensing images is a significant task. In this paper, we proposed an efficient method to coarsely detect the cloud via Mean Shift Cloud Detection (MSCD) algorithm, which can work automatically without any reference images. Experimental results on Landsat-8 OLI dataset show the effectiveness of the proposed method.

We elicit from a fundamental definition of action low-level attributes that can reveal agency and intentionality. These descriptors are mainly trajectory-based, measuring sudden changes, temporal synchrony, and repetitiveness. The actionness map can be used to localize actions in a way that is generic across action and agent types. Furthermore, it also groups interacting regions into a useful unit of analysis, which is crucial for recognition of actions involving interactions. We then implement an actionness-driven pooling scheme to improve action recognition performance. Experimental results on three datasets show the advantages of our method on both action detection and action recognition comparing with other state-of-the-art methods.

Many consumer videos focus on and follow salient objects in a scene. Detecting such salient objects is thus of great interests to video analytics and search. Instead of detecting salient object in individual frames separately, we propose to detect and track salient object simultaneously by finding a spatio-temporal path of the highest saliency density in the video. As salient video objects usually appear in consecutive frames, leveraging the motion coherence of videos can detect salient object more robustly. Without any prior knowledge of the salient objects, our method can automatically detect the salient objects of different shapes and sizes, and is able to handle noisy saliency maps and moving cameras. Experimental results on two public datasets demonstrate the effectiveness of the proposed method on salient video object detection.

In spite of extensive studies on visual saliency, e.g., generating a saliency map from an image, less work has been addressed how to crop salient regions from saliency maps. We present a new approach to detect salient regions with maximum saliency density from videos. A branch-and-bound search algorithm is developed to find the global optimal solution efficiently. The proposed detection approach can automatically adapt to the shapes and motions of salient objects regardless of cluttered backgrounds. Moreover, by introducing an intermediate cropping window, video retargeting as an application of salient region detection gets optimized saliency coverage. Extensive experimental results validate the advantages of the proposed method.

Detection of salient objects in an image remains a challenging problem despite extensive studies in visual saliency, as the generated saliency map is usually noisy and incomplete. In this paper, we propose a new method to discover the salient object without prior knowledge on its shape and size. By searching the sub-image, i.e., a bounding box of maximum saliency density, the new formulation can automatically crop the salient objects of various sizes in spite of the cluttered background, and is capable to handle different types of saliency maps. A global optimal solution is obtained by the proposed density-based branch-and-bound search. The proposed method can apply to both images and videos. Experimental results on a public dataset of 5000 images show that our unsupervised detection approach is comparable to the state-of-the-art learning-based methods. Promising results are also observed in the salient object detection for videos with a good potential in video retargeting.