Robots powered by large language model (LLM) demonstrate significant research and application potential by effectively interpreting scene information to respond to human commands. However, when robots rely on static scene information during task execution, they face difficulties in adapting to changes in the environment, posing a major challenge for dynamic scene perception. To address the above issues, we propose an innovative interaction-driven approach to enhance robots' ability to perceive dynamic scene information. This approach consists of two contributions, the observation point selection module and the dynamic scene maintenance module. Specifically, first, the robot uses the 3D scene graph (3DSG) containing assets and objects to perceive static scene information through the LLM planner. Next, the best observation point for each asset is obtained through the observation point selection module. Then, with the help of the best observation point, the dynamic scene maintenance module interacts with the asset-related objects to dynamically update all the object node information related to the asset node. This approach enables robots to maintain dynamic scene information, enhancing their adaptability in unpredictable environments and improving task reliability. We evaluated our method using the iTHOR and RoboTHOR datasets within the AI2-THOR simulator and in real-world scenarios. Experimental results demonstrate that our method effectively and accurately maintains robots' perception of dynamic scene information.

Neural implicit representations have been explored to enhance visual SLAM algorithms, especially in providing high-fidelity dense map. Existing methods operate robustly in static scenes but struggle with the disruption caused by moving objects. In this paper we present NID-SLAM, which significantly improves the performance of neural SLAM in dynamic environments. We propose a new approach to enhance inaccurate regions in semantic masks, particularly in marginal areas. Utilizing the geometric information present in depth images, this method enables accurate removal of dynamic objects, thereby reducing the probability of camera drift. Additionally, we introduce a keyframe selection strategy for dynamic scenes, which enhances camera tracking robustness against large-scale objects and improves the efficiency of mapping. Experiments on publicly available RGB-D datasets demonstrate that our method outperforms competitive neural SLAM approaches in tracking accuracy and mapping quality in dynamic environments.

Cephalometric tracing is widely used in orthodontic diagnosis and treatment planning. Since manual landmark lo-calization suffers from severe inter-observer and intra-observer inconsistency, a large number of efforts have been made by researchers to develop automatic localization methods. However, most of the existing methods are developed based on rules which sample uniformly from origin images rather than with the highest density in a focal point and ignore intermediate layers' results in their networks or their outputs' channels. To address the issue, this paper proposes a deep learning model based on multi-scale and multi-channel attention to identify landmarks. The channel attention network is first trained by multi-scale image patches cropped from 100 Cephalograms, and then enhanced by cross-layer connections to extract high-level features, finally involved in the collaboration with a three-layer MLP module to accurately locate coordinates. We conduct extensive evaluation on a real cephalometric X-ray data-set and a Non-public dataset, both achieve promising performance improvements especially in terms of high-precision detection.

Affordance detection is of great importance in robot operational tasks, due to its capability of helping robots effectively interact with objects. Many affordance detectors have been proposed, primarily based on two-stage object detection, significantly suffering from the slow detection speed. Hence, recent years have saw the popularity of one-stage affordance detectors based on encoder-decoder structures that adopt dilated convolutions to extract high-resolution feature maps. However, dilated convolutions on high resolution features tend to be computation and memory-intensive, greatly limiting the practicality of one-stage detectors. To address the issue, this paper proposes a novel convolution neural network (CNN) based encoder-decoder architecture, without the need of adopting dilated convolution. A repeated multi-scale feature-map-fusion network is introduced to produce high-resolution features, effectively improving the feature representation performance of the model. Besides, a semantic encode module is embedded to capture global semantic information and enhance category-relevant feature maps. Extensive experiments show that the proposed framework outperforms the start-of-art methods with only 1/2 of the computational cost, while maintaining the inference at the speed of 26ms per image, indicating the promising affordance-detection performance of our network on IIT-AFF dataset and UMD dataset.

Mobile edge computing (MEC) has been envisioned as a promising paradigm that could effectively enhance the computational capacity of wireless user devices (WUDs) and quality of experience of mobile applications. One of the most crucial issues of MEC is computation offloading, which decides how to offload WUDs’ tasks to edge severs for further intensive computation. Conventional mathematical programming-based offloading approaches could face troubles in dynamic MEC environments due to the time-varying channel conditions (caused primarily by WUD mobility). To address the problem, reinforcement learning (RL) based offloading approaches have been proposed, which develop offloading policies by mapping MEC states to offloading actions. However, these approaches could fail to converge in large-scale MEC due to the exponentially-growing state and action spaces. In this article, we propose a novel online computation offloading approach that could effectively reduce task latency and energy consumption in dynamic MEC with large-scale WUDs. First, a RL-based computation offloading and energy transmission algorithm is proposed to accelerate the learning process. Then, a joint optimization method is adopted to develop the allocating algorithm, which obtains near-optimal solutions for energy and computation resources allocation. Simulation results show that the proposed approach can converge efficiently and achieve significant performance improvements over baseline approaches.

Manga is a world popular comic form originated in Japan, which typically employs black-and-white stroke lines and geometric exaggeration to describe humans' appearances, poses, and actions. In this paper, we propose MangaGAN, the first method based on Generative Adversarial Network (GAN) for unpaired photo-to-manga translation. Inspired by the drawing process of experienced manga artists, MangaGAN generates geometric features and converts each facial region into the manga domain with a tailored multi-GANs architecture. For training MangaGAN, we collect a new data-set from a popular manga work with extensive features. To produce high-quality manga faces, we propose a structural smoothing loss to smooth stroke-lines and avoid noisy pixels, and a similarity preserving module to improve the similarity between domains of photo and manga. Extensive experiments show that MangaGAN can produce high-quality manga faces preserving both the facial similarity and manga style, and outperforms other reference methods.

The performance of deep learning models generally relies on large and high-quality labeled datasets. However, in medical domain, as labeling process is much more laborious and time-consuming, most medical datasets are much smaller compared with natural image datasets. To mitigate this weakness, recent researches in medical image analysis adopt semi-supervised learning methods, especially consistency regularization methods to learn from a large amount of unlabeled medical data. However, as these semi-supervised learning methods are originally designed for tasks of natural images, specific properties of medical domain are not fully investigated and utilized. In this paper, we present DK-Consistency, a domain knowledge guided consistency regularization method for semi-supervised breast cancer diagnosis in ultrasound images. In DK-Consistency, domain knowledge of medical doctors is first incorporated into the generation process of perturbed samples for each unlabeled image. Then consistency regularization is adopted to force the model to make consistent predictions for unlabeled images and their perturbed samples. Extensive experiments demonstrate that, by injecting domain knowledge, DK-Consistency significantly improves the diagnostic performance of breast cancer and outperforms many state-of the-art semi-supervised methods.

Quick Response (QR) code is one of the most worldwide used two-dimensional codes. Traditional QR codes appear as random collections of black-and-white modules that lack visual semantics and aesthetic elements, which inspires the recent works to beautify the appearances of QR codes. However, these works adopt fixed generation algorithms and therefore can only generate QR codes with a pre-defined style. In this paper, combining the Neural Style Transfer technique, we propose a novel end-to-end method, named ArtCoder, to generate the stylized QR codes that are personalized, diverse, attractive, and scanning-robust. To guarantee that the generated stylized QR codes are still scanning-robust, we propose a Sampling-Simulation layer, a module-based code loss, and a competition mechanism. The experimental results show that our stylized QR codes have high-quality in both the visual effect and the scanning-robustness, and they are able to support the real-world application.

Quick Response (QR) code is a popular form of matrix barcodes that are widely used to tag online links on print media (e.g., posters, leaflets, and books). However, standard QR codes typically appear as noise-like black/white squares (named modules) which seriously disrupt the attractiveness of their carriers. In this paper, we propose StyleCode-Net, a method to generate novel art-style QR codes which can better match the entire style of their carriers to improve the visual quality. For endowing QR codes with artistic elements, a big challenge is that the scanning-robustness must be preserved after transforming colors and textures. To address these issues, we propose a module-based deformable convolutional mechanism (MDCM) and a dynamic target mechanism (DTM) in StyleCode-Net. MDCM can extract the features of black and white modules of QR codes respectively. Then, the extracted features are fed to DTM to balance the scanning-robustness and the style representation. Extensive subjective and objective experiments show that our art-style QR codes have reached the state-of-the-art level in both visual quality and scanning-robustness, and these codes have the potential to replace standard QR codes in real-world applications.

In recent years, deep learning has been widely used in breast cancer diagnosis, and many high-performance models have emerged. However, most of the existing deep learning models are mainly based on static breast ultrasound (US) images. In actual diagnostic process, contrast-enhanced ultrasound (CEUS) is a commonly used technique by radiologists. Compared with static breast US images, CEUS videos can provide more detailed blood supply information of tumors, and therefore can help radiologists make a more accurate diagnosis. In this paper, we propose a novel diagnosis model based on CEUS videos. The backbone of the model is a 3D convolutional neural network. More specifically, we notice that radiologists generally follow two specific patterns when browsing CEUS videos. One pattern is that they focus on specific time slots, and the other is that they pay attention to the differences between the CEUS frames and the corresponding US images. To incorporate these two patterns into our deep learning model, we design a domain-knowledge-guided temporal attention module and a channel attention module. We validate our model on our Breast-CEUS dataset composed of 221 cases. The result shows that our model can achieve a sensitivity of 97.2% and an accuracy of 86.3%. In particular, the incorporation of domain knowledge leads to a 3.5% improvement in sensitivity and a 6.0% improvement in specificity. Finally, we also prove the validity of two domain knowledge modules in the 3D convolutional neural network (C3D) and the 3D ResNet (R3D).

Quick response (QR) codes are usually scanned in different environments, so they must be robust to variations in illumination, scale, coverage, and camera angles. Aesthetic QR codes improve the visual quality, but subtle changes in their appearance may cause scanning failure. In this article, a new method to generate scanning-robust aesthetic QR codes is proposed, which is based on a module-based scanning probability estimation model that can effectively balance the tradeoff between visual quality and scanning robustness. Our method locally adjusts the luminance of each module by estimating the probability of successful sampling. The approach adopts the hierarchical, coarse-to-fine strategy to enhance the visual quality of aesthetic QR codes, which sequentially generate the following three codes: a binary aesthetic QR code, a grayscale aesthetic QR code, and the final color aesthetic QR code. Our approach also can be used to create QR codes with different visual styles by adjusting some initialization parameters. User surveys and decoding experiments were adopted for evaluating our method compared with state-of-the-art algorithms, which indicates that the proposed approach has excellent performance in terms of both visual quality and scanning robustness.