Robots powered by large language model (LLM) demonstrate significant research and application potential by effectively interpreting scene information to respond to human commands. However, when robots rely on static scene information during task execution, they face difficulties in adapting to changes in the environment, posing a major challenge for dynamic scene perception. To address the above issues, we propose an innovative interaction-driven approach to enhance robots' ability to perceive dynamic scene information. This approach consists of two contributions, the observation point selection module and the dynamic scene maintenance module. Specifically, first, the robot uses the 3D scene graph (3DSG) containing assets and objects to perceive static scene information through the LLM planner. Next, the best observation point for each asset is obtained through the observation point selection module. Then, with the help of the best observation point, the dynamic scene maintenance module interacts with the asset-related objects to dynamically update all the object node information related to the asset node. This approach enables robots to maintain dynamic scene information, enhancing their adaptability in unpredictable environments and improving task reliability. We evaluated our method using the iTHOR and RoboTHOR datasets within the AI2-THOR simulator and in real-world scenarios. Experimental results demonstrate that our method effectively and accurately maintains robots' perception of dynamic scene information.

Neural implicit representations have been explored to enhance visual SLAM algorithms, especially in providing high-fidelity dense map. Existing methods operate robustly in static scenes but struggle with the disruption caused by moving objects. In this paper we present NID-SLAM, which significantly improves the performance of neural SLAM in dynamic environments. We propose a new approach to enhance inaccurate regions in semantic masks, particularly in marginal areas. Utilizing the geometric information present in depth images, this method enables accurate removal of dynamic objects, thereby reducing the probability of camera drift. Additionally, we introduce a keyframe selection strategy for dynamic scenes, which enhances camera tracking robustness against large-scale objects and improves the efficiency of mapping. Experiments on publicly available RGB-D datasets demonstrate that our method outperforms competitive neural SLAM approaches in tracking accuracy and mapping quality in dynamic environments.

In the past decade, significant advancements have been made in utilizing deep learning for breast lesion segmentation. Recently, researchers have increasingly focused on harnessing the power of multiple modalities, recognizing its potential for enhancing segmentation performance. We observe that in clinical practice, many radiologists often rely on two types of ultrasound images, namely ultrasound (US) and contrast-enhanced ultrasound (CEUS) data for diagnosis. This motivates us to propose a multi-modal segmentation network, called as IMAN (Iterative Mutual-Aid Network), based on these two modalities. The architecture of IMAN adopts a novel hourglass shape, featuring two branches connected by an ‘X’ pathway. One branch is dedicated to processing CEUS data, while the other branch handles US data. Each branch generates segmentation results specific to its respective modality. The ’X’ pathway, realized by a margin mask generator module, serves as a bridge between these branches by forcing the segmentation results from one branch as additional input to the other. This head-to-tail pathway effectively facilitates mutual aid between the two modalities. In addition, we propose an iterative training policy during the training process to fully exploit the information from both US and CEUS data. Experimental results on a Breast-US-CEUS dataset comprising 169 samples demonstrate the effectiveness of IMAN, achieving Dice Similarity Coefficient of 83.96% and 81.16% for US images and CEUS videos, respectively. These scores surpass those obtained by many state-of-the-art segmentation methods. Furthermore, IMAN exhibits robust generalization capabilities across different segmentation structures.

Cephalometric tracing is widely used in orthodontic diagnosis and treatment planning. Since manual landmark lo-calization suffers from severe inter-observer and intra-observer inconsistency, a large number of efforts have been made by researchers to develop automatic localization methods. However, most of the existing methods are developed based on rules which sample uniformly from origin images rather than with the highest density in a focal point and ignore intermediate layers' results in their networks or their outputs' channels. To address the issue, this paper proposes a deep learning model based on multi-scale and multi-channel attention to identify landmarks. The channel attention network is first trained by multi-scale image patches cropped from 100 Cephalograms, and then enhanced by cross-layer connections to extract high-level features, finally involved in the collaboration with a three-layer MLP module to accurately locate coordinates. We conduct extensive evaluation on a real cephalometric X-ray data-set and a Non-public dataset, both achieve promising performance improvements especially in terms of high-precision detection.

In recent years, deep learning has been widely used in breast cancer diagnosis, and many high-performance models have emerged. However, most of the existing deep learning models are mainly based on static breast ultrasound (US) images. In actual diagnostic process, contrast-enhanced ultrasound (CEUS) is a commonly used technique by radiologists. Compared with static breast US images, CEUS videos can provide more detailed blood supply information of tumors, and therefore can help radiologists make a more accurate diagnosis. In this paper, we propose a novel diagnosis model based on CEUS videos. The backbone of the model is a 3D convolutional neural network. More specifically, we notice that radiologists generally follow two specific patterns when browsing CEUS videos. One pattern is that they focus on specific time slots, and the other is that they pay attention to the differences between the CEUS frames and the corresponding US images. To incorporate these two patterns into our deep learning model, we design a domain-knowledge-guided temporal attention module and a channel attention module. We validate our model on our Breast-CEUS dataset composed of 221 cases. The result shows that our model can achieve a sensitivity of 97.2% and an accuracy of 86.3%. In particular, the incorporation of domain knowledge leads to a 3.5% improvement in sensitivity and a 6.0% improvement in specificity. Finally, we also prove the validity of two domain knowledge modules in the 3D convolutional neural network (C3D) and the 3D ResNet (R3D).