Multi-Object Tracking (MOT) of polyps in colonoscopy videos serves as a foundational component for intelligent medical workflows. In tracking-by-detection algorithms, managing the equilibrium among competing optimization objectives during data association presents a complex issue. Current approaches, including threshold-based weighted linear aggregation methodologies and hierarchical cascade matching schemes, demonstrate persistent limitations when deployed in endoscopic polyp monitoring scenarios characterized by vigorous polyp movements and complex morphological deformations. Thus, this study proposes a novel polyp tracking framework named MO2Tracker, which can simultaneously optimize multiple objectives during the data association phase. Specifically, MO2Tracker integrates three discriminative similarity cues in parallel: polyp coordinates, detection confidence scores, and appearance consistency, to establish associations between new detections and existing trajectories. To resolve this optimization challenge, we implement Pareto Front analysis via a multi-objective optimization algorithm, deriving Pareto Front solutions. Subsequently, we design two novel selection paradigms, knee-oriented selection and human-inspired selection, to filter optimal detector-trajectory pairings from the Pareto Front. Extensive experiments conducted on two public datasets and one private dataset validate the superior performance of MO2Tracker. On the public dataset SUN-SEG, MO2Tracker achieves improvements of +4.1% in Multi-object Tracking Accuracy (MOTA), +7.2% in Higher Order Tracking Accuracy(HOTA), and +5.8% in IDF1 Score.

Facial expression recognition (FER) in the wild suffers from ambiguous facial expressions, occlusions, and backgrounds, leading to inter-class similarities and intra-class variations. Most existing methods introduce maximization and orthogonality loss functions into convolutional neural networks (CNN) to restrict inter-class samples in a mini-batch. Such methods neglect intrinsic emotion correlations, resulting in over-separation and under-separation. Besides, CNNs have limitations in capturing global spatial information due to the inductive bias property. Thus, we propose a novel FER network to reveal intrinsic emotion correlations and learn global expression-related features for high-accurate FER in the wild. In this network, first, a correlation-biased orthogonality loss based on emotion correlations separates inter-class samples towards class distances. To decouple this loss from the mini-batch and alleviate the influence of ambiguous expressions, we use a conditional queue to maintain high-quality expressions. Second, implicit selective transformers are plugged into the intermediate stages of CNN to construct long-range relationships between local regions. Particularly, this module measures the importance of local regions implicitly to eliminate occlusions and backgrounds. Third, a cross-attention fusion module uses high-level features to guide the intermediate features to be class-specific and finally fuse them for emotion recognition. The experimental results on five in-the-wild datasets demonstrate the effectiveness and superiority of our network by showing clear performance improvements over other state-of-the-art FER methods. Codes are available at https://github.com/Gabrella/R-SFT.

Facial expression recognition (FER) in the wild is a more challenging topic than that under laboratory-controlled conditions. The major obstacles of FER in the wild are head pose variations, illumination changes, and different skin colors. To address these problems, we propose a framework named quaternion deformable local binary pattern (QDLBP)-Net for color FER in the wild. First, to eliminate the interferences of head pose variations, a pose-correction facial decomposition (PCFD) strategy is proposed to correct the head pose and decompose the facial image into five emotion-related regions. Then, to handle the problems of illumination changes and different skin colors, an effective feature descriptor named “QDLBP” is developed. QDLBP extracts color quaternion features from each emotional region, which not only computes the strength of emotional features, but also maintains the spectral correlation between color channels. Finally, a quaternion classification network (QC-Net) is proposed to classify the quaternion features from five emotional regions into seven basic expressions. The experimental results on three in-the-wild FER datasets and two nonfrontal pose variation datasets exhibit the effectiveness and superiority of QDLBP-Net by showing clear performance improvements over other state-of-the-art (SOTA) FER methods.

Accurate volumetric segmentation of brain tumors and tissues is beneficial for quantitative brain analysis and brain disease identification in multi-modal Magnetic Resonance (MR) images. Nevertheless, due to the complex relationship between modalities, 3D Fully Convolutional Networks (3D FCNs) using simple multi-modal fusion strategies hardly learn the complex and nonlinear complementary information between modalities. Meanwhile, the indiscriminative feature aggregation between low-level and high-level features easily causes volumetric feature misalignment in 3D FCNs. On the other hand, the 3D convolution operations of 3D FCNs are excellent at modeling local relations but typically inefficient at capturing global relations between distant regions in volumetric images. To tackle these issues, we propose an Aligned Cross-Modality Interaction Network (ACMINet) for segmenting the regions of brain tumors and tissues from MR images. In this network, the cross-modality feature interaction module is first designed to adaptively and efficiently fuse and refine multi-modal features. Secondly, the volumetric feature alignment module is developed for dynamically aligning low-level and high-level features by the learnable volumetric feature deformation field. Thirdly, we propose the volumetric dual interaction graph reasoning module for graph-based global context modeling in spatial and channel dimensions. Our proposed method is applied to brain glioma, vestibular schwannoma, and brain tissue segmentation tasks, and we performed extensive experiments on BraTS2018, BraTS2020, Vestibular Schwannoma, and iSeg-2017 datasets. Experimental results show that ACMINet achieves state-of-the-art segmentation performance on all four benchmark datasets and obtains the highest DSC score of hard-segmented enhanced tumor region on the validation leaderboard of the BraTS2020 challenge.

Brain tissue segmentation in multi-modal magnetic resonance (MR) images is significant for the clinical diagnosis of brain diseases. Due to blurred boundaries, low contrast, and intricate anatomical relationships between brain tissue regions, automatic brain tissue segmentation without prior knowledge is still challenging. This paper presents a novel 3D fully convolutional network (FCN) for brain tissue segmentation, called APRNet. In this network, we first propose a 3D anisotropic pyramidal convolutional reversible residual sequence (3DAPC-RRS) module to integrate the intra-slice information with the inter-slice information without significant memory consumption; secondly, we design a multi-modal cross-dimension attention (MCDA) module to automatically capture the effective information in each dimension of multi-modal images; then, we apply 3DAPC-RRS modules and MCDA modules to a 3D FCN with multiple encoded streams and one decoded stream for constituting the overall architecture of APRNet. We evaluated APRNet on two benchmark challenges, namely MRBrainS13 and iSeg-2017. The experimental results show that APRNet yields state-of-the-art segmentation results on both benchmark challenge datasets and achieves the best segmentation performance on the cerebrospinal fluid region. Compared with other methods, our proposed approach exploits the complementary information of different modalities to segment brain tissue regions in both adult and infant MR images, and it achieves the average Dice coefficient of 87.22% and 93.03% on the MRBrainS13 and iSeg-2017 testing data, respectively. The proposed method is beneficial for quantitative brain analysis in the clinical study, and our code is made publicly available.

The outbreak of Coronavirus Disease-2019 (COVID-19) has posed a threat to world health. With the increasing number of people infected, healthcare systems, especially those in developing countries, are bearing tremendous pressure. There is an urgent need for the diagnosis of COVID-19 and the prognosis of inpatients. To alleviate these problems, a data-driven medical assistance system is put forward in this article. Based on two real-world data sets in Wuhan, China, the proposed system integrates data from different sources with tools of machine learning (ML) to predict COVID-19 infected probability of suspected patients in their first visit, and then predict mortality of confirmed cases. Rather than choosing an interpretable algorithm, this system separates the explanations from ML models. It can do help to patient triaging and provide some useful advice for doctors.

Recently, the ear recognition system takes more increasingly interesting for many applications, especially, in immigration system, forensic, and surveillance applications. For face re-identification and image classification, metric learning has significantly improved machine learning accuracies by using K-Nearest Neighbor (KNN) and Support Vector Machine (SVM) classifiers. However, metric learning via SVM has not yet been investigated for the ear recognition problem. To achieve better generalization ability than the traditional previous classifiers, a novel framework for ear recognition is proposed based on learning distance metric (LDM) via SVM since the LDM and the directed acyclic graph SVM (DAGSVM) are two emerging techniques which perform outstanding in dealing with classification problems. This work considers metric learning for SVM by proposing a hybrid learning distance metric and directed acyclic graph SVM (LDM-DAGSVM) model for ear recognition system. Different from existing ear biometric methods, the proposed approach aims to learn a Mahalanobis distance metric via SVM to maximize the inter-class variations and minimize the intra-class variations, simultaneously. The experiments are conducted on complicated ear datasets and the results can achieve better performance compared with the state-of-the-art ear recognition methods. The proposed approach can get classification accuracy up to 98.79%, 98.70%, and 84.30% for AWE, AME and WPUT ear datasets, respectively.

In facial expression recognition (FER), convolutional neural networks (CNNs) have been shown great capability of learning features. In this paper, we propose a new CNN framework for FER in color images, which incorporates deformable Gabor filters into a quaternion CNN. Deformable Gabor filters reinforce the network's ability of extracting different orientations of facial wrinkles information. Quaternion CNNs have greater advantages over the regular CNNs in handling the coupling between color channels. The proposed deformable quaternion Gabor convolutional neural network (DQG-CNN) not only learns FER feature representation excellently, but also processes spectral correlation between color channels naturally. Moreover, it can also effectively reduce training complexity compared to other reference models. Experimental results on three benchmark color datasets Oulu-CASIA, MMI, and SFEW, demonstrate that the proposed DGQ-CNN outperforms other state-of-the-art methods clearly.

The multi‐atlas‐based methods are widely applied in the automatic labelling in magnetic resonance (MR) images. However, most multi‐atlas‐based methods require that all atlases be registered to the target image accurately to have a correct label propagation. In this study, the authors introduce the term spatial indexes and construct a hybrid atlas forest model to gather the labelling information from all atlases without propagating labels from every single atlas. Furthermore, a new automatic labelling method using the hybrid atlas forest model based on spatial indexes is proposed. In the proposed framework, an atlas is chosen arbitrarily as a reference image and the spatial indexes are constructed on this image space. Then, the samples are selected from all atlases in the dataset based on the spatial indexes to construct a samples pool. Finally, the hybrid atlas forest model will be trained on the samples pool and used to predict the labelling of the target. Experiments are conducted on two public datasets to evaluate the effectiveness of the proposed method. The experimental results show that the proposed method reduces the requirement of strong dependence on precise registration and improve the accuracy of labelling.

Early detection of lung cancer is an effective way to improve the survival rate of patients. It is a critical step to have accurate detection of lung nodules in computed tomography (CT) images for the diagnosis of lung cancer. However, due to the heterogeneity of the lung nodules and the complexity of the surrounding environment, it is a challenge to develop a robust nodule detection method. In this study, we propose a two-stage convolutional neural networks (TSCNN) for lung nodule detection. The first stage based on the improved U-Net segmentation network is to establish an initial detection of lung nodules. During this stage, in order to obtain a high recall rate without introducing excessive false positive nodules, we propose a new sampling strategy for training. Simultaneously, a two-phase prediction method is also proposed in this stage. The second stage in the TSCNN architecture based on the proposed dual pooling structure is built into three 3D-CNN classification networks for false positive reduction. Since the network training requires a significant amount of training data, we designed a random mask as the data augmentation method in this study. Furthermore, we have improved the generalization ability of the false positive reduction model by means of ensemble learning. We verified the proposed architecture on the LUNA dataset in our experiments, which showed that the proposed TSCNN architecture did obtain competitive detection performance.

To obtain a higher accuracy in the multi‐atlas patch‐based label fusion method, it is essential to have the accurate similarity measure of selected patches. In this study, the authors propose a new sparse patch‐based representation method using a local binary texture (LBT) in the atlas image and atlas label information for the multi‐atlas label fusion. In the proposed method, the intensity information in a patch is converted into a LBT which is then combined with the labels of corresponding patches from the atlas to form an atom of a dictionary. The initial labels of target images are estimated through a rough segmentation. The voxel in a patch to be labelled is also constructed as a vector similar to the atom. The voxel vector is then modelled as a sparse linear combination of the atoms in the dictionary. Experimental results on two MR brain data sets demonstrated that the proposed method is efficient in the segmentation which can achieve competitive performance compared with the state‐of‐the‐art methods.

Impulse noise detection is important to the restoration of color images contaminated by impulse noise in switching vector median filters. To increase detection accuracy, an effective color-impulse detector is presented. A new color distance metric based on quaternion theory is proposed. The proposed color distance metric is used to calculate the local density of a color pixel. A hard thresholding strategy is used to determine whether a color pixel is corrupted by impulse noise or not (i.e., an outlier). The noisy pixels detected will be restored by a weighted vector median filter, while the noise-free pixels remain unchanged. The experimental comparisons show that the proposed algorithm can obtain lower false and miss detection rate, and produces better performance in terms of peak signal-to-noise ratio and feature similarity measures, compared to other well-known color image filtering methods.

A structure-adaptive vector median filter (SAVMF) for removal of impulse noise from color images is presented in this paper. A color image is represented in quaternion form, and then quaternion Fourier transform is employed to detect the dominant orientation of the pattern in a local neighborhood. Based on the local orientation and its strength, the size, shape and orientation of the support window of vector median filter (VMF) can be adaptively computed, and thus structure-adaptive VMF is implemented. Experimental results exhibit the validity of the proposed method by showing clearly performance improvements both in noise removal and in detail preservation, compared to other VMF-based vector filters.

The multi‐Atlas patch‐based label fusion method (MAS‐PBM) has emerged as a promising technique for the magnetic resonance imaging (MRI) image segmentation. The state‐of‐the‐art MAS‐PBM approach measures the patch similarity between the target image and each atlas image using the features extracted from images intensity only. It is well known that each atlas consists of both MRI image and labelled image (which is also called the map). In other words, the map information is not used in calculating the similarity in the existing MAS‐PBM. To improve the segmentation result, the authors propose an enhanced MAS‐PBM in which the maps will be used for similarity measure. The first component of the proposed method is that an initial segmentation result (i.e. an appropriate map for the target) is obtained by using either the non‐local‐patch‐based label fusion method (NPBM) or the sparse patch‐based label fusion method (SPBM) based on the grey scales of patches. Then, the SPBM is applied again to obtain the finer segmentation based on the labels of patches. The authors called these two versions of the proposed fusion method as MAS‐PBM‐NPBM and MAS‐PBM‐SPBM. Experimental results show that more accurate segmentation results are achieved compared with those of the majority voting, NPBM, SPBM, STEPS and the hierarchical multi‐atlas label fusion with multi‐scale feature representation and label‐specific patch partition.

In developing countries, people's living pressure is increasing with the society's development by inefficient economic growth mode. Recently, the number of people who suffer from sudden cardiac death is progressively increasing, and cardiovascular disease (CVD) has become one great killer which threats the life and health of people. However, at present, we are confronted with one problem: when a patient has chest distress or chest pain, etc., he/she hurries to the hospital to go through electrocardiograph (ECG) examination but the abnormal ECG signal disappears. Therefore, the opportunity to timely capture the ECG status of a patient and make an accurate judgment is lost. Thus, extensive efforts have been made to design various systems for patient monitoring at anytime and anywhere, in order to have real-time records and analysis on vital signal of patents, so as to provide early detection before the occurrence of adverse effect. However, the mobility of the patient is limited in most existing healthcare systems. While sporting is beneficial to improve patient's health, designing a comfortable and durable healthcare system for facilitating patient's movement is a critical issue. This paper presents a novel comfortable and durable portable ECG monitoring system to have real-time monitoring and analysis on a moving user. In the meantime, its special low power and on-demand data collection design alleviates the problem that the current wearable ECG monitoring equipment could not be used for a long time due to the constraint of its battery life.

Estimating gradients of color images is important to many color image processing tasks. However, the research on color image gradients is limited. In this paper, a novel method of estimating color image gradients is presented. The proposed gradient mechanism is based on measuring the squared local contrast variation of a color image function, in which the chromatic variation is evaluated by quaternion representation. As an application in color image processing, we apply the proposed gradient operator to the traditional grayscale image Canny operator for color edge detection. The edge detection results indicate that the proposed color gradient operator is superior to other state-of-the-art color image gradient methods.

Graph-based segmentation has become a major trend in image segmentation. A key issue in graph-based segmentation is how to build the affinity matrix. Among the previous methods, many successful ones only compute the pairwise affinities between adjacent pixels and superpixels without considering the nonadjacent ones. Thus, they often obtain unsatisfactory results when foreground is cut into several nonadjacent parts by background or shadows. In this paper, we propose a full-range affinities learning method for graph-based segmentation. Our method computes the affinities both between adjacent pixels and nonadjacent pixels, which are inversely proportional to the shortest connectivity paths. The experimental results demonstrate the superiority of the proposed approach comparing with existing popular methods.

In pervasive computing environment, due to the mobility feature of mobile terminals, the mobile service needs to dynamically adapt execution behavior to the changing computing environment as mobile user moves. However, previous researches mainly focused on deploying a service adaption module on mobile terminals or local central server to support the adaptive execution of mobile services, which brings huge overhead to mobile terminals or can hardly meet user's personalized requirements. Therefore, we propose a cloud based framework, called CAMSPF, which includes three parts: RMC (resource management cloud), AMSPC (adaptive mobile service provision cloud), and MSM (mobile service middleware). The CAMSPF deploys the service resources in RMC for realizing efficient resource management and provision, and constructs a PMSAA (private mobile service adaption agent) for each mobile user in AMSPC in order to efficiently support personalized adaptive execution of mobile service. In addition, the MCM is a lightweight software installed on mobile terminals by which CAMSPF can collect user's realtime context and monitor service request from mobile user. Our prototype implementation of CAMSPF verifies that the adaptive execution of mobile services can be performed more efficiently than other traditional approaches, with lower energy consumption on mobile terminals.

In this paper, a new quaternion vector filter for removal of random impulse noise in color video sequences is presented. First, luminance distances and chromaticity differences that are represented in quaternion form are combined together to measure color distances between color pixels. Then, based on this new color distance mechanism, the samples along horizontal, vertical, and diagonal directions in current frame and the samples of adjacent frames on motion trajectory are used to detect whether each pixel is noisy or not. By analyzing the spatiotemporal order-statistic information about these directional samples, the video pixels are classified into noise free and noisy. Finally, 3-D weighted vector median filtering is performed on the pixels that are judged as noisy, and the other pixels remain unchanged. The experimental results show that the proposed algorithm significantly outperforms other state-of-the-art video denoising methods in terms of both objective measure and visual evaluation.

The authors studied current levels of computerization in dental clinics and the attitudes of dentists towards dental computerization in metropolises in China. A survey consisting of 22 questions was e-mailed or mailed to a random sample of 354 dentists. Of all respondents, 80.5% reported using a computer in their practice. The authors found that administrative tasks were the first to be computerized. A majority of respondents supported the statement that computerization is a benefit to patient care. The authors found that the computerization of dental clinics in Chinese metropolises is a few years behind that of western nations.

This paper examines an alternative approach to separating magnetic resonance imaging (MRI) intensity inhomogeneity from underlying tissue-intensity structure using a direct template-based paradigm. This permits the explicit spatial modeling of subtle intensity variations present in normal anatomy which may confound common retrospective correction techniques using criteria derived from a global intensity model. A fine-scale entropy driven spatial normalisation procedure is employed to map intensity distorted MR images to a tissue reference template. This allows a direct estimation of the relative bias field between template and subject MR images, from the ratio of their low-pass filtered intensity values. A tissue template for an aging individual is constructed and used to correct distortion in a set of data acquired as part of a study on dementia. A careful validation based on manual segmentation and correction of nine datasets with a range of anatomies and distortion levels is carried out. This reveals a consistent improvement in the removal of global intensity variation in terms of the agreement with a global manual bias estimate, and in the reduction in the coefficient of intensity variation in manually delineated regions of white matter.