Open-world 3D scene understanding is fundamentally challenging for vision and robotics, due to the constraints of closed-vocabulary supervision and static annotations. To address this, we propose a unified framework for Open-World 3D Scene Graph Generation with Retrieval-Augmented Reasoning, which enables generalizable and interactive 3D scene understanding. Our method integrates vision-language models with retrieval-based reasoning to support multimodal exploration and language-guided interaction. The framework comprises two key components: (1) a dynamic scene graph generation module that detects objects and infers semantic relationships without fixed label sets, and (2) a retrieval-augmented reasoning pipeline that encodes scene graphs into a vector database to support text/image-conditioned queries. We evaluate our method on 3DSSG and Replica benchmarks across four tasks—scene question answering, visual grounding, instance retrieval, and task planning—demonstrating robust generalization and superior performance in diverse environments. Our results highlight the effectiveness of combining open-vocabulary perception with retrieval-based reasoning for scalable 3D scene understanding.

Current adversarial attacks pose a serious threat to the robustness of visual-language models (VLMs), including vision-language pre-trained models (VLPMs) and multimodal large language models (MLLMs). Traditional adversarial attacks are example-specific and rely on specific datasets. This practice suffers from low transferability and additional computation cost, while universal adversarial perturbations (UAPs) offer example-agnostic solutions by generalizing across inputs. However, current UAP methods mainly target VLPMs, demonstrating limited transferability and effectiveness in MLLMs. To bridge this gap, we propose the Recursive Perturbation Attack (RPA), a novel black-box UAP method for both VLPMs and MLLMs. RPA employs a recursive perturbations strategy, utilizing token filtering and polynomial sampling methods to generate perturbations, thereby achieving incremental disruption and enhancing the transferability of the attack. To further enhance the effectiveness of the attack, RPA integrates a three-tier modality decoupling strategy, disentangling intra-modal, cross-modal, and fusion-modal features to effectively disrupt feature alignment and interactions. Extensive experiments validate that RPA achieves superior attack performance compared to existing UAP approaches. This work highlights new security concerns in multimodal AI systems and provides insights into the design of more robust models. Code is available at https://github.com/chilljudaoren/RPAttack.

With the rapid development of Large Language Models (LLMs), an increasing number of Large Visual-Language Models (LVLMs) have achieved unprecedented performance in response generation. Recent work shows that LVLMs are vulnerable to adversarial attacks. However, many existing methods tend to overfit to the source model by overemphasizing specific features, which compromises their transferability. Other approaches suffer from reduced attack effectiveness due to insufficient differentiation between features. In this paper, we propose a novel transfer-based black-box untargeted attack—Shared Adversarial Feature (SAF) dynamic attack. By exploring the feature extraction patterns of LVLMs, we identify the features shared among various models that are most susceptible to adversarial attacks and disrupt them. Moreover, due to the powerful attention mechanisms of LVLMs, they are still able to extract similar semantics from perturbed images, even when primary features are disrupted. We design a dynamic update strategy to address this challenge. Finally, from the perspective of SAF, we conduct an in-depth analysis of vulnerabilities in the vision encoder and projector within LVLMs and find that attacking the projector exhibits stronger transferability across heterogeneous model architectures. Extensive experiments show that our method exhibits superior attack performance compared to existing methods across different models, datasets, and tasks. The code will be publicly available after publication.

Despite the rapid development of Chinese vision-language models (VLMs), most existing Chinese vision-language (VL) datasets are constructed on Western-centric images from existing English VL datasets. The cultural bias in the images makes these datasets unsuitable for evaluating VLMs in Chinese culture. To remedy this issue, we present a new Chinese Vision-Language Understanding Evaluation (CVLUE) benchmark dataset, where the selection of object categories and images is entirely driven by Chinese native speakers, ensuring that the source images are representative of Chinese culture. The benchmark contains four distinct VL tasks ranging from image-text retrieval to visual question answering, visual grounding and visual dialogue. We present a detailed statistical analysis of CVLUE and provide a baseline performance analysis with several open-source multilingual VLMs on CVLUE and its English counterparts to reveal their performance gap between English and Chinese. Our in-depth category-level analysis reveals a lack of Chinese cultural knowledge in existing VLMs. We also find that fine-tuning on Chinese culture-related VL datasets effectively enhances VLMs' understanding of Chinese culture.

Understanding complex 3D environments is essential for many computer vision and robotic applications, especially in highly dynamic open-scene scenarios. The 3D scene graph plays an important role in the comprehension of 3D environments. However, most existing methods for 3D scene graph generation depend on pre-specified object and relationship classes (i.e., closed vocabulary) and labeled data for training, which restricts their effectiveness in the open-scene setting. To address this issue, we propose a novel Open-Scene Understanding-oriented 3D Scene Graph (OSU-3DSG) framework that can operate without labeled training data. The OSU-3DSG framework effectively extracts visual features from RGB-D image sequences and fuses them with camera pose estimates to create accurate 3D object maps. Then, by leveraging a pre-trained Vision Language Model (VLM), it generates relational triplets and constructs 3D scene graphs in a zero-shot manner. In particular, it excels at adaptively recognizing and interpreting object relationships, making it suitable for open-world applications. Finally, we perform extensive experiments on two open-world 3D datasets, namely 3DSSG and Replica, to evaluate the effectiveness and adaptability of the OSU-3DSG framework, demonstrating its potential to pave the way for the advancement of open-scene understanding. Our code and data are published at https://github.com/YuansuHao/OSU-3DSG.

Estimating spoken content from silent videos is crucial for applications in Assistive Technology (AT) and Augmented Reality (AR). However, accurately mapping lip movement sequences in videos to words poses significant challenges due to variability across sequences and the uneven distribution of information within each sequence. To tackle this, we introduce InfoSyncNet, a non-uniform sequence modeling network enhanced by tailored data augmentation techniques. Central to InfoSyncNet is a non-uniform quantization module positioned between the encoder and decoder, enabling dynamic adjustment to the network’s focus and effectively handling the natural inconsistencies in visual speech data. Additionally, multiple training strategies are incorporated to enhance the model’s capability to handle variations in lighting and the speaker’s orientation. Comprehensive experiments on the LRW and LRW1000 datasets confirm the superiority of InfoSyncNet, achieving new state-of-the-art accuracies of 92.0% and 60.7% Top-1 ACC1.

The transition from 5 G to beyond 5 G (B5G) networks is accelerating the development of ubiquitous and intelligent IoT applications. Semantic communication (SemCom) has emerged as a transformative paradigm that prioritizes task-relevant meaning over bit-level accuracy. However, deploying computationally intensive semantic models on resource-constrained end devices presents a major challenge. To address this issue, we propose FedSC, a new task-oriented semantic communication framework that enables end-edge collaborative learning by integrating federated learning with knowledge distillation. FedSC distributes model training between edge servers and end devices, where lightweight terminal models are collaboratively optimized with the assistance of more capable edge nodes, thus ensuring efficient and privacy-preserving semantic communication across heterogeneous environments. The framework adopts a hierarchical learning architecture in which the end and edge models cooperate via dynamic feature alignment, guided by the information bottleneck principle to balance semantic fidelity and compression. To further enhance efficiency and robustness, FedSC incorporates Gumbel-Softmax discretization for bandwidth-efficient semantic representation and introduces an SNR-adaptive inference mechanism to adapt to fluctuating wireless conditions. Experiments on Raspberry Pi clusters show that FedSC achieves improved performance in image classification and reconstruction tasks while significantly reducing communication overhead compared to existing baseline approaches.

Sonar imaging systems play a crucial role in several marine applications. However, complex underwater environment introduces scattering noise, significantly degrading sonar image quality and hindering performance for downstream tasks. Although several self-supervised denoising methods have emerged to address the lack of clean reference images, they often fail to effectively capture both local and global structural information, thus showing suboptimal performance on sonar images. To address these challenges, we propose NSC-SSNet, a Self-Supervised Network with Neighborhood Subsampling and Calibration constraints for sonar image denoising. In particular, NSC-SSNet adopts an end-to-end self-supervised framework that operates in the denoising and calibration stages. By leveraging neighborhood subsampling and calibration constraints, it effectively extracts latent features of clean images from noisy input. Moreover, it simultaneously captures local and global associations between pixels by incorporating additional terms in the loss function to improve image quality while denoising. Extensive experiments on real-world sonar image datasets demonstrate that NSC-SSNet outperforms existing self-supervised denoising methods in terms of both noise removal and quality enhancement.

Federated learning systems are susceptible to adversarial attacks. To combat this, we introduce a novel aggregator based on Huber loss minimization, and provide a comprehensive theoretical analysis. Under independent and identically distributed (i.i.d) assumption, our approach has several advantages compared to existing methods. Firstly, it has optimal dependence on epsilon, which stands for the ratio of attacked clients. Secondly, our approach does not need precise knowledge of epsilon. Thirdly, it allows different clients to have unequal data sizes. We then broaden our analysis to include non-i.i.d data, such that clients have slightly different distributions.

In order to improve performance and avoid overheating on mobile devices, precise thermal control with low overhead is crucial. To achieve this, we propose incorporating a performance monitoring counter (PMC)-based power model into thermal control, which enables a more accurate evaluation of the CPU’s power consumption. We demonstrate the plausibility of this approach using polynomial regression based on Moore’s Law. Additionally, we introduce a lightweight PMC sampling method that can collect multiple PMCs at once in the kernel space, reducing sampling overhead. By replacing the utilization-based model in the original the intelligent power allocation (IPA) with a PMC-based power model, we realize the PMC-based IPA governor can be ported to real mobile devices. After updating the thermal control governor in the Linux kernel, we perform tests on our PMC-based IPA using a mobile phone device. We compare it with Stepwise and IPA, which are commonly used in current mobile phone systems. We choose the CPU-intensive workbench, I/O-intensive workbench, and CPU and I/O-intensive hybrid workbench as workloads. The results show that PMC-based IPA effectively reduces energy consumption while improving performance. In particular, during the CPU and I/O-intensive hybrid experiment, where CPU-intensive and I/O-intensive tasks are executed alternately, PMC-based IPA reduces the running time by 10.0% and energy consumption by 16.6% compared to the original IPA. In order to verify the benefits of PMC-based IPA, mobile phone testing software AI Bench and Antutu are utilized. The results show that our scheme is able to control temperature more precisely than IPA and achieves a better score while consuming less energy, particularly during AI computing. These experiment results suggest that PMC-based IPA is valuable for practical use.

Accurate load forecasting of charging stations enable managers to reduce the drivers' waiting time and operating costs. But the existing works for spatial-temporal sequence forecasting usually assume the spatial-continuity of signals. However, the recharging scenario, in which the above assumptions are not valid due to the sparse spatial distribution of stations, need further research. To fill the gap, we present a Heterogeneous Components Fusion Network to model dual components sourced from the planned and the unplanned recharging events independently. For planned recharging component, we design a customized transformer to 'looks up' the reference 'memory' for the prediction. And we propose the time-variant graph to model highly dynamic unplanned events. Experiments conducted on a load reading dataset of 120 stations suggest that our model achieves better performance than a series of state-of-the-arts for spatial-temporal sequence prediction problem.

With the wide application of location-based social networks (LBSNs), point-of-interest (POI) recommendation has become one of the major services in LBSNs. The behaviors of users in LBSNs are mainly checking in POIs, and these checking-in behaviors are influenced by user's behavior habits and his/her friends. In social networks, social influence is often used to help businesses to attract more users. Each target user has a different influence on different POI in social networks. This paper selects the list of POIs with the greatest influence for recommending users. Our goals are to satisfy the target user's service need, and simultaneously to promote businesses' locations (POIs). This paper defines a POI recommendation problem for location promotion. Additionally, we use submodular properties to solve the optimization problem. At last, this paper conducted a comprehensive performance evaluation for our method using two real LBSN datasets. Experimental results show that our proposed method achieves significantly superior POI recommendations comparing with other state-of-the-art recommendation approaches in terms of location promotion.