Deep learning in ultrasound (US) imaging aims to construct foundational models that accurately reflect the modality's unique characteristics. Nevertheless, the limited datasets and narrow task types have restricted this field in recent years. To address these challenges, we introduce US-MTD120 K, a multi-task ultrasound dataset with 120,354 real-world two-dimensional images. This dataset covers three standard plane recognition and two diagnostic tasks in ultrasound imaging, providing a rich basis for model training and evaluation. We detail the data collection, distribution, and labelling processes, ensuring a thorough understanding of the dataset's structure. Furthermore, we conduct extensive benchmark tests on 27 state-of-the-art methods from both supervised and self-supervised learning(SSL) perspectives. In the realm of supervised learning, we analyze the sensitivity of two main feature computation methods to ultrasound images at the representational level, highlighting that models which judiciously constrain global feature computation could potentially serve as a viable analytical approach for US image analysis. In the context of self-supervised learning, we delved into the modelling process of self-supervised learning models for medical images and proposed an improvement strategy, named MoCo-US, a solution that addresses the excessive reliance on pretext task design from the input side. It achieves competitive performance with minimal pretext task design and enhances other SSL methods simply.

The analysis of 3D meshes with deep learning has become prevalent in computer graphics. As an essential structure, hierarchical representation is critical for mesh pooling in multiscale analysis. Existing clustering-based mesh hierarchy construction methods involve nonlinear discretization optimization operations, making them nondifferential and challenging to embed in other trainable networks for learning. Inspired by deep superpixel learning methods in image processing, we extend them from 2D images to 3D meshes by proposing a novel differentiable chart-based segmentation method named geodesic differential supervertex (GDSV). The key to the GDSV method is to ensure that the geodesic position updates are differentiable while satisfying the constraint that the renewed supervertices lie on the manifold surface. To this end, in addition to using the differential SLIC clustering algorithm to update the nonpositional features of the supervertices, a reparameterization trick, the Gumbel-Softmax trick, is employed to renew the geodesic positions of the supervertices. Therefore, the geodesic position update problem is converted into a linear matrix multiplication issue. The GDSV method can be an independent module for chart-based segmentation tasks. Meanwhile, it can be combined with the front-end feature learning network and the back-end task-specific network as a plug-in-plug-out module for training; and be applied to tasks such as shape classification, part segmentation, and 3D scene understanding. Experimental results show the excellent performance of our proposed algorithm on a range of datasets.

The liver ultrasound standard planes (LUSP) have significant diagnostic significance during ultrasonic liver diagnosis. However, the location and acquisition of LUSP could be a time-consuming and complicated mission and requires the relevant operator to have comprehensive knowledge of ultrasound diagnosis. Therefore, this study puts forward an automatic classification approach for eight types of LUSP based on a pre-trained CNN(Convolutional Neural Network). With the comparison to classification methods on the basis of conventional hand-craft characteristics, the method proposed by us can automatically catch the appearance in LUSP and classify the LUSP. The proposed model is consisted of 13 convolutional layers with little 3×3 size kernels and three completely connected layers. To address the limitation of data, we adopt the transfer learning strategy, which pre-trains the weight of convolutional layers and fine-tune the weight of fully connected layers. These extensive experiments show that the accuracy of the suggested method reaches 92.31%, as well as the performance of the suggested means outperforms previous ways, which demonstrates the suitability and effectiveness of CNN to classify LUSP for clinical diagnosis.

In colposcopy-assisted diagnosis, the difference between the different lesion grades of colposcopic images is small, and the visual similarity is high. Therefore, it is a very challenging task to accurately diagnose cervical lesions through colposcopic images. This paper proposes a new risk assessment net of cervical lesions in colposcopic images (RACNet). The RACNet mainly consists of two parts. At first, the location and grade of lesions in different scales are detected through a multi-scale detection network. Then, these lesions are classified by designing a multi-branch convolutional neural network (CNN) to improve the performance of risk assessment. The RACNet was compared with the most advanced methods and colposcopists under the same condition. The experimental results show that the RACNet in this paper is superior to other methods, with an accuracy rate of 84.5%, which is 15% higher than the average level of colposcopists. It can provide clinicians with auxiliary diagnosis and reduce missed diagnosis and misdiagnosis.

We propose an effective method to boost the accuracy of multi-person pose estimation in images. Initially, the three-layer fractal network was constructed to regress multi-person joints location heatmap that can help to enhance an image region with receptive field and capture more joints local-contextual feature information, thereby producing keypoints heatmap intermediate prediction to optimize human body joints regression results. Subsequently, the hierarchical bi-directional inference algorithm was proposed to calculate the degree of relatedness (call it Kinship) for adjacent joints, and it combines the Kinship between adjacent joints with the spatial constraints, which we refer to as joints kinship pattern matching mechanism, to determine the best matched joints pair. We iterate the above-mentioned joints matching process layer by layer until all joints are assigned to a corresponding individual. Comprehensive experiments demonstrate that the proposed approach outperforms the state-of-the-art schemes and achieves about 1% and 0.6% increase in mAP on MPII multi-person subset and MSCOCO 2016 keypoints challenge.

Ocular recognition is expected to provide a higher flexibility in handling practical applications as oppose to the iris recognition, which only works for the ideal open-eye case. However, the accuracy of the recent efforts is still far from satisfactory at uncontrollable conditions, such as eye blinking which implies any poses of eyes. To address these issues, the skin texture, eyelids, and additional geometrical features are employed. In addition, to achieve higher accuracy, sequential forward floating selection is utilized to select the best feature combinations. Finally, the non-linear support vector machine is applied for identification purpose. Experimental results demonstrate that the proposed algorithm achieves the best accuracy for both open eye and blinking eye scenarios. As a result, it offers greater flexibility for the prospective subjects during recognition as well as higher reliability for security.

This paper presents an effective image retrieval method by combining high-level features from convolutional neural network (CNN) model and low-level features from dot-diffused block truncation coding (DDBTC). The low-level features, e.g., texture and color, are constructed by vector quantization -indexed histogram from DDBTC bitmap, maximum, and minimum quantizers. Conversely, high-level features from CNN can effectively capture human perception. With the fusion of the DDBTC and CNN features, the extended deep learning two-layer codebook features is generated using the proposed two-layer codebook, dimension reduction, and similarity reweighting to improve the overall retrieval rate. Two metrics, average precision rate and average recall rate (ARR), are employed to examine various data sets. As documented in the experimental results, the proposed schemes can achieve superior performance compared with the state-of-the-art methods with either low-or high-level features in terms of the retrieval rate. Thus, it can be a strong candidate for various image retrieval related applications.

Since a human face can be represented by a few feature points (FPs) with less redundant information, and calculated by a linear combination of a small number of prototypical faces, we propose a two-step 3D face reconstruction approach including FP depth estimation and shape deformation. The proposed approach can reconstruct a realistic 3D face from a 2D frontal face image. In the first step, a coupled dictionary learning method based on sparse representation is employed to explore the underlying mappings between 2D and 3D training FPs, and then the depth of the FPs is estimated. In the second step, a novel shape deformation method is proposed to reconstruct the 3D face by combining a small number of most relevant deformed faces by the estimated FPs. The proposed approach can explore the distributions of 2D and 3D faces and the underlying mappings between them well, because human faces are represented by low-dimensional FPs, and their distributions are described by sparse representations. Moreover, it is much more flexible since we can make any change in any step. Extensive experiments are conducted on BJUT_3D database, and the results validate the effectiveness of the proposed approach.

Visual object tracking is a challenging task because designing an effective and efficient appearance model is difficult. Current online tracking algorithms treat tracking as a classification task and use labeled samples to update appearance model. However, it is not clear to evaluate instance confidence belong to the object. In this paper, we propose a simple and efficient tracking algorithm with a deformable structure appearance. In our method, model updates with continuous labeled samples which are dense sampling. In order to improve the accuracy, we introduce a couple-layer regression model which prevents negative background from impacting on the model learning rather than traditional classification. The proposed DSR tracker runs in real-time and performs favorably against state-of-the-art trackers on various challenging sequences.