Joint learning on diverse ultrasonic source scenarios presents a challenge in preserving stable gen-eralization due to the combination of heterogeneity of different sources and the inconsistency of joint learning features. Previous joint learning studies, which are not source-resilient frameworks, may not preserve stable generalization when trained on diverse source scenarios. Furthermore, the limited variations insingle-source data and the interference from ultrasound imaging, which are common in ultrasonic source scenarios, further decrease generalization. To address these problems, we pro posed a source-resilient joint learning framework consisting of three stages: 1) Source transforming, where our 1-to-N transformation unifies diverse source scenarios for source-resiliency. 2) Our feature enhancement modules model the source-resilient joint learning network, including a manifold-constraint normalization module (MCNM) for addressing heterogeneity by minimizing manifold-based loss, a task-consistent attention module (TCAM) shares the multi-scale features with self-attention to address inconsistency, and an adaptive feature-shifting module (AFSM) for feature-level augmentation to overcome single-source data.3) Our ultrasound-hybrid linear mapping (USmapping) cascades speckle randomization and mask-guiding Monge-Kantorovitch linear mapping to achieve ultrasonic style randomization for addressing the interference of ultrasonic data. Our framework was evaluated on eight ultrasound datasets from various scanners at multiple center sand surpassed previous comparable studies in both segmentation (DSCWAvgof 75.7%) and classification (AUROCWAvgof 68.8%) tasks. Our framework has the potential to serve as a general framework for enhancing the performance of joint learning under diverse ultrasonic source scenarios.

The accurate diagnosis of significant liver fibrosis ( ≥ F2) in patients with chronic liver disease (CLD) is critical, as ≥ F2 is a crucial factor that should be considered in selecting an antiviral therapy for these patients. This article proposes a handcrafted-feature-assisted deep convolutional neural network (HFA-DCNN) that helps radiologists automatically and accurately diagnose significant liver fibrosis from ultrasound (US) brightness (B)-mode images. The HFA-DCNN model has three main branches: one for automatic region of interest (ROI) segmentation in the US images, another for attention deep feature learning from the segmented ROI, and the third for handcrafted feature extraction. The attention deep learning features and handcrafted features are fused in the back end of the model to enable more accurate diagnosis of significant liver fibrosis. The usefulness and effectiveness of the proposed model were validated on a dataset built upon 321 CLD patients with liver fibrosis stages confirmed by pathological evaluations. In a fivefold cross validation (FFCV), the proposed model achieves accuracy, sensitivity, specificity, and area under the receiver-operating-characteristic (ROC) curve (AUC) values of 0.863 (95% confidence interval (CI) 0.820-0.899), 0.879 (95% CI 0.823-0.920), 0.872 (95% CI 0.800-0.925), and 0.925 (95% CI 0.891-0.952), which are significantly better than those obtained by the comparative methods. Given its excellent performance, the proposed HFA-DCNN model can serve as a promising tool for the noninvasive and accurate diagnosis of significant liver fibrosis in CLD patients.

Quantitative ultrasound (QUS), which attempts to extract quantitative features from the US radiofrequency (RF) or envelope data for tissue characterization, is becoming a promising technique for noninvasive assessments of liver fibrosis. However, the number of feature variables examined and finally used in the existing QUS methods is typically small, limiting the diagnostic performance. Therefore, this paper devises a new multiparametric QUS (MP-QUS) method which enables the extraction of a large number of feature variables from US RF signals and allows for the use of feature-engineering and machine-learning based algorithms for liver fibrosis assessment. In the MP-QUS, eighty-four feature variables were extracted from multiple QUS parametric maps derived from the RF signals and the envelope data. Afterwards, feature reduction and selection were performed in turn to remove the feature redundancy and identify the best combination of features in the reduced feature set. Finally, a variety of machine-learning algorithms were tested for fibrosis classification with the selected features, based on the results of which the optimal classifier was established. The performance of the proposed MP-QUS method for staging liver fibrosis was evaluated on an animal model, with histologic examination as the reference standard. The mean accuracy, sensitivity, specificity and area under the receiver-operating-characteristic curve achieved by MP-QUS are respectively 83.38%, 86.04%, 80.82%, and 0.891 for recognizing significant liver fibrosis, and 85.50%, 88.92%, 85.24%, and 0.924 for diagnosing liver cirrhosis. The proposed MP-QUS method paves a way for its future extension to assess liver fibrosis in human subjects.

This paper proposes a new algorithm for automatic estimation of muscle fiber orientation (MFO) in musculoskeletal ultrasound images, which is commonly used for both diagnosis and rehabilitation assessment of patients. The algorithm is based on a novel adaptive fading Bayesian Kalman filter (AF-BKF) and an automatic region of interest (ROI) extraction method. The ROI is first enhanced by the Gabor filter (GF) and extracted automatically using the revoting constrained Radon transform (RCRT) approach. The dominant MFO in the ROI is then detected by the RT and tracked by the proposed AF-BKF, which employs simplified Gaussian mixtures to approximate the non-Gaussian state densities and a new adaptive fading method to update the mixture parameters. An AF-BK smoother (AF-BKS) is also proposed by extending the AF-BKF using the concept of Rauch-Tung-Striebel smoother for further smoothing the fascicle orientations. The experimental results and comparisons show that: 1) the maximum segmentation error of the proposed RCRT is below nine pixels, which is sufficiently small for MFO tracking; 2) the accuracy of MFO gauged by RT in the ROI enhanced by the GF is comparable to that of using multiscale vessel enhancement filter-based method and better than those of local RT and revoting Hough transform approaches; and 3) the proposed AF-BKS algorithm outperforms the other tested approaches and achieves a performance close to those obtained by experienced operators (the overall covariance obtained by the AF-BKS is 3.19, which is rather close to that of the operators, 2.86). It, thus, serves as a valuable tool for automatic estimation of fascicle orientations and possibly for other applications in musculoskeletal ultrasound images.

This paper proposes a new multi-view human body tracking and model-based rendering system with a textured deformable human body model and explores its practical application to view transfer in image-based rendering (IBR). The proposed approach first reconstructs an initial 3-D model of the human subject offline using Kinect depth cameras or from a general 3-D model if such information is unavailable in the original data. The human pose of the subject is then tracked with multi-view videos using an annealed particle filter (APF)-based tracker with a new color-based likelihood function and a Bayesian-Kalman filter smoother. The previous captured model can then be deformed to the new position for model-based rendering. An immediate application of the proposed approach is to support fly-over effects for view transfer in IBR systems with limited cameras. It avoids the reconstruction of the complete dynamic 3-D model where a large number of cameras may be required. Moreover, for static background, the background can be rendered using IBR with precaptured depth maps to further enhance the user's experience. To reduce the artifacts during fly-over caused by tracking errors and model deformation, a novel morphing technique utilizing a new free-form deformation-based artifacts suppression (FFD-AS) method and other user interface design techniques are also proposed. It allows smooth transition between the original view and the model-rendered views. The performance of the proposed algorithm is evaluated using the publicly available HumanEva dataset and our captured RGB-D multi-view dataset. Experimental results show that the proposed APF-based tracker offered improved tracking performance compared with the conventional bidirectional silhouette likelihood criterion. The proposed morphing approach is also shown to be effective in mitigating the rendering artifacts during view transfer.

This paper presents a new hand gesture recognition algorithm based on canonical formed superpixel earth mover's distance (CF-SP-EMD). SP-EMD is a recently proposed distance metric designed for depth based hand gesture recognition, which shows promising performance. However, in real life, people may have their own habits while performing certain hand gestures. This will yield a variety of hand shapes with different finger poses as compared with the standard templates. Such variety may affect the accuracy of SP-EMD and hence will degrade its performance. In this paper, we propose a new distance metric CF-SP-EMD to alleviate the problem. We organize superpixels in canonical forms that can factor out nonstandard finger poses, resulting a well-structured fingerpose-neutral shape representation for hand gestures. Experimental results using public gesture datasets show that the proposed CF-SP-EMD can achieve better performance for hand gesture recognition, compared with the state-of-art algorithms.

This paper proposes a multi-view articulated human motion tracking approach with textured deformable mesh model. Firstly, a subject-specific mesh model is initialized by using linear blend skinning method. The model is then textured according to the multi-view image observations. We introduce a segmentation-based method to refine the appearance of the subject. With the textured mesh model, a color-based likelihood (CbL) is also proposed for human body tracking with Annealed Particle Filter (APF). Experiments in the paper show that the performance can be considerately improved by using CbL as the measurement for pose tracking.

This paper presents a new superpixel-based hand gesture recognition system based on a novel superpixel earth mover's distance metric, together with Kinect depth camera. The depth and skeleton information from Kinect are effectively utilized to produce markerless hand extraction. The hand shapes, corresponding textures and depths are represented in the form of superpixels, which effectively retain the overall shapes and color of the gestures to be recognized. Based on this representation, a novel distance metric, superpixel earth mover's distance (SP-EMD), is proposed to measure the dissimilarity between the hand gestures. This measurement is not only robust to distortion and articulation, but also invariant to scaling, translation and rotation with proper preprocessing. The effectiveness of the proposed distance metric and recognition algorithm are illustrated by extensive experiments with our own gesture dataset as well as two other public datasets. Simulation results show that the proposed system is able to achieve high mean accuracy and fast recognition speed. Its superiority is further demonstrated by comparisons with other conventional techniques and two real-life applications.

This paper proposes a new articulated human motion tracking and pose estimation algorithm using an improved silhouette extraction method with view adaptive fusion. It is developed around the baseline algorithm in HumanEva, which uses the Annealed Particle Filter (APF). Shadow detection and removal and a level-set method are employed to achieve better silhouette extraction. An adaptive view fusion approach is also proposed to improve the matching between the human 3D model and the observations. Experimental results show that the proposed approach has considerably better performance than the baseline algorithm in the HumanEva dataset, due to better shadow handling and data fusion of multiple views.