In the past decade, significant advancements have been made in utilizing deep learning for breast lesion segmentation. Recently, researchers have increasingly focused on harnessing the power of multiple modalities, recognizing its potential for enhancing segmentation performance. We observe that in clinical practice, many radiologists often rely on two types of ultrasound images, namely ultrasound (US) and contrast-enhanced ultrasound (CEUS) data for diagnosis. This motivates us to propose a multi-modal segmentation network, called as IMAN (Iterative Mutual-Aid Network), based on these two modalities. The architecture of IMAN adopts a novel hourglass shape, featuring two branches connected by an ‘X’ pathway. One branch is dedicated to processing CEUS data, while the other branch handles US data. Each branch generates segmentation results specific to its respective modality. The ’X’ pathway, realized by a margin mask generator module, serves as a bridge between these branches by forcing the segmentation results from one branch as additional input to the other. This head-to-tail pathway effectively facilitates mutual aid between the two modalities. In addition, we propose an iterative training policy during the training process to fully exploit the information from both US and CEUS data. Experimental results on a Breast-US-CEUS dataset comprising 169 samples demonstrate the effectiveness of IMAN, achieving Dice Similarity Coefficient of 83.96% and 81.16% for US images and CEUS videos, respectively. These scores surpass those obtained by many state-of-the-art segmentation methods. Furthermore, IMAN exhibits robust generalization capabilities across different segmentation structures.

The performance of deep learning models generally relies on large and high-quality labeled datasets. However, in medical domain, as labeling process is much more laborious and time-consuming, most medical datasets are much smaller compared with natural image datasets. To mitigate this weakness, recent researches in medical image analysis adopt semi-supervised learning methods, especially consistency regularization methods to learn from a large amount of unlabeled medical data. However, as these semi-supervised learning methods are originally designed for tasks of natural images, specific properties of medical domain are not fully investigated and utilized. In this paper, we present DK-Consistency, a domain knowledge guided consistency regularization method for semi-supervised breast cancer diagnosis in ultrasound images. In DK-Consistency, domain knowledge of medical doctors is first incorporated into the generation process of perturbed samples for each unlabeled image. Then consistency regularization is adopted to force the model to make consistent predictions for unlabeled images and their perturbed samples. Extensive experiments demonstrate that, by injecting domain knowledge, DK-Consistency significantly improves the diagnostic performance of breast cancer and outperforms many state-of the-art semi-supervised methods.

Numerous research efforts have been conducted to simulate the crowd movements, while relatively few of them are specifically focused on multihazard situations. In this paper, we propose a novel crowd simulation method by modeling the generation and contagion of panic emotion under multihazard circumstances. In order to depict the effect from hazards and other agents to crowd movement, we first classify hazards into different types (transient and persistent, concurrent and nonconcurrent, and static and dynamic) based on their inherent characteristics. Second, we introduce the concept of perilous field for each hazard and further transform the critical level of the field to its invoked-panic emotion. After that, we propose an emotional contagion model to simulate the evolving process of panic emotion caused by multiple hazards. Finally, we introduce an emotional reciprocal velocity obstacles (RVOs) model to simulate the crowd behaviors by augmenting the traditional RVO model with emotional contagion, which for the first time combines the emotional impact and local avoidance together. Our experimental results demonstrate that the overall approach is robust, can better generate realistic crowds and the panic emotion dynamics in a crowd. Furthermore, it is recommended that our method can be applied to various complex multihazard environments.