Future Internet-of-Things (IoT) networks demand scalable, energy-efficient, and intelligent wireless nodes with integrated sensing and communication (ISAC). Conventional transmitter architectures rely on complex and power-hungry radio-frequency (RF) chains, limiting their suitability for dense IoT deployment. This work presents a programmable millimeter-wave information metasurface (mmWave-InfoMeta) transmitter enabling compact, low-cost, and reconfigurable operation for IoT applications. The mmWave-InfoMeta directly modulates information onto multiple carrier frequencies using programmable modulation schemes, allowing adaptive operation across diverse IoT environments. With its array characteristics, the metasurface-based transmitter supports dynamic switching between SISO and MIMO modes, enabling space and frequency division multiplexing, while also integrating real-time user localization without the need for additional hardware. Experiments demonstrate a closed-loop ISAC platform capable of adaptive beamforming for signal enhancement. The proposed mmWave-InfoMeta offers a lightweight and multifunctional transmitter solution for large-scale and intelligent IoT systems.

Understanding people's social interactions in complex real-world scenarios often relies on intricate mental reasoning. To truly understand how and why people interact with one another, we must infer the underlying mental states that give rise to the social interactions, i.e., Theory of Mind reasoning in multi-agent interactions. Additionally, social interactions are often multi-modal -- we can watch people's actions, hear their conversations, and/or read about their past behaviors. For AI systems to successfully and safely interact with people in real-world environments, they also need to understand people's mental states as well as their inferences about each other's mental states based on multi-modal information about their interactions. For this, we introduce MuMA-ToM, a Multi-modal Multi-Agent Theory of Mind benchmark. MuMA-ToM is the first multi-modal Theory of Mind benchmark that evaluates mental reasoning in embodied multi-agent interactions. In MuMA-ToM, we provide video and text descriptions of people's multi-modal behavior in realistic household environments. Based on the context, we then ask questions about people's goals, beliefs, and beliefs about others' goals. We validated MuMA-ToM in a human experiment and provided a human baseline. We also proposed a novel multi-modal, multi-agent ToM model, LIMP (Language model-based Inverse Multi-agent Planning). Our experimental results show that LIMP significantly outperforms state-of-the-art methods, including large multi-modal models (e.g., GPT-4o, Gemini-1.5 Pro) and a recent multi-modal ToM model, BIP-ALM.

Zicheng Zhang, Xiangyu Zhao, Xinyu Fang, Chunyi Li, Xiaohong Liu, Xiongkuo Min, Haodong Duan, Kai Chen, Guangtao Zhai. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Xiangyu Zhao, Shengyuan Ding, Zicheng Zhang, Haian Huang, Maosongcao Maosongcao, Jiaqi Wang, Weiyun Wang, Xinyu Fang, Wenhai Wang, Guangtao Zhai, Hua Yang, Haodong Duan, Kai Chen. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Large Multimodal Models (LMMs) have shown significant progress in various complex vision tasks with the solid linguistic and reasoning capacity inherited from large language models (LMMs). Low-rank adaptation (LoRA) offers a promising method to integrate external knowledge into LMMs, compensating for their limitations on domain-specific tasks. However, the existing LoRA model serving is excessively computationally expensive and causes extremely high latency. In this paper, we present an end-to-end solution that empowers diverse vision tasks and enriches vision applications with LoRA LMMs. Our system, VaLoRA, enables accurate and efficient vision tasks by 1) an accuracy-aware LoRA adapter generation approach that generates LoRA adapters rich in domain-specific knowledge to meet application-specific accuracy requirements, 2) an adaptive-tiling LoRA adapters batching operator that efficiently computes concurrent heterogeneous LoRA adapters, and 3) a flexible LoRA adapter orchestration mechanism that manages application requests and LoRA adapters to achieve the lowest average response latency. We prototype VaLoRA on five popular vision tasks on three LMMs. Experiment results reveal that VaLoRA improves 24-62% of the accuracy compared to the original LMMs and reduces 20-89% of the latency compared to the state-of-the-art LoRA model serving systems.

Space-time modulated metasurface (STMM)-based direction-of-arrival (DoA) estimation methods has emerged as a prominent research area in recent years, owing to its remarkable flexibility and low-complexity. However, the receiving antenna is typically positioned in the far-field region of the STMM, thereby having potential risks of direct wave interference (DWI). In this paper, we propose a DoA estimation method based on STMM in the code domain to mitigate the effects of DWI. The wideband DWI shares the same center frequency with the incident waves under estimation, leading to frequency spectrum overlap and making conventional frequency-domain methods ineffective. In contrast, through the design of pseudo-random modulation signals, our proposed method effectively suppresses specific types of wideband DWI within the code domain. As a result, the DoA can be accurately estimated using the interference-suppressed spectrum in the code domain. The novelty of the proposed method lies in its unique approach to DoA estimation within the code domain, along with its ability to effectively counter in-band DWI. To validate the proposed DoA estimation method, we design and fabricate a reconfigurable metasurface operating at the X-band. Experimental results show that the DoA estimation error remains below 2∘ even in the presence of a linear frequency modulation (LFM) DWI, demonstrating the effectiveness and accuracy of the proposed method.

The advent of large vision-language models (LVLMs) has spurred research into their applications in multi-modal contexts, particularly in video understanding. Traditional VideoQA benchmarks, despite providing quantitative metrics, often fail to encompass the full spectrum of video content and inadequately assess models' temporal comprehension. To address these limitations, we introduce MMBench-Video, a quantitative benchmark designed to rigorously evaluate LVLMs' proficiency in video understanding. MMBench-Video incorporates lengthy videos from YouTube and employs free-form questions, mirroring practical use cases. The benchmark is meticulously crafted to probe the models' temporal reasoning skills, with all questions human-annotated according to a carefully constructed ability taxonomy.We employ GPT-4 for automated assessment, demonstrating superior accuracy and robustness over earlier LLM-based evaluations. Utilizing MMBench-Video, we have conducted comprehensive evaluations that include both proprietary and open-source LVLMs for images and videos. MMBench-Video stands as a valuable resource for the research community, facilitating improved evaluation of LVLMs and catalyzing progress in the field of video understanding.

Vision Language Models (VLMs) demonstrate remarkable proficiency in addressing a wide array of visual questions, which requires strong perception and reasoning faculties. Assessing these two competencies independently is crucial for model refinement, despite the inherent difficulty due to the intertwined nature of seeing and reasoning in existing VLMs. To tackle this issue, we present Prism, an innovative framework designed to disentangle the perception and reasoning processes involved in visual question solving. Prism comprises two distinct stages: a perception stage that utilizes a VLM to extract and articulate visual information in textual form, and a reasoning stage that formulates responses based on the extracted visual information using a Large Language Model (LLM). This modular design enables the systematic comparison and assessment of both proprietary and open-source VLM for their perception and reasoning strengths. Our analytical framework provides several valuable insights, underscoring Prism's potential as a cost-effective solution for vision-language tasks. By combining a streamlined VLM focused on perception with a powerful LLM tailored for reasoning, Prism achieves superior results in general vision-language tasks while substantially cutting down on training and operational expenses. Quantitative evaluations show that Prism, when configured with a vanilla 2B LLaVA and freely accessible GPT-3.5, delivers performance on par with VLMs $10 \times$ larger on the rigorous multimodal benchmark MMStar.

An electromagnetic (EM) metasurface is a passive device capable of manipulating the reflected signal of a radar in both spatial and frequency domains. This paper presents a novel analytical design of EM scattering center model guided passive synthetic aperture radar (SAR) deception based on diverse frequency time-modulation. The objective of this design is to disguise real target as an intended deceptive target and conceal its original EM characteristics. To achieve this, we first extract the scattering center model of an intended deceptive target and express its SAR image analytically. The SAR image is determined by the number, amplitude and position of scattering center. Subsequently, we derive the SAR image of a metasurface modulated on fast and slow time scales with diverse frequencies, establishing the relation between modulation parameters and the resulting SAR image properties. By comparing and aligning the SAR images between scattering center model and the modulated metasurface with predetermined similarity threshold, we successfully accomplish scattering center model guided SAR deception. The simulated results demonstrate that the SAR image similarities of both scattering center model and modulated metasurface exceed 0.9, indicating the effectiveness of this method as an electronic countermeasure.

Images contain rich relational knowledge that can help machines understand the world. Existing methods on visual knowledge extraction often rely on the pre-defined format (e.g., sub-verb-obj tuples) or vocabulary (e.g., relation types), restricting the expressiveness of the extracted knowledge. In this work, we take a first exploration to a new paradigm of open visual knowledge extraction. To achieve this, we present OpenVik which consists of an open relational region detector to detect regions potentially containing relational knowledge and a visual knowledge generator that generates format-free knowledge by prompting the large multimodality model with the detected region of interest. We also explore two data enhancement techniques for diversifying the generated format-free visual knowledge. Extensive knowledge quality evaluations highlight the correctness and uniqueness of the extracted open visual knowledge by OpenVik. Moreover, integrating our extracted knowledge across various visual reasoning applications shows consistent improvements, indicating the real-world applicability of OpenVik.

Robot-centered cells play a more and more important role in the fields of incorporate automation and repetitive processing in order to increase productivity and to improve quality. In practice, it is always desired to achieve maximum/near-maximum throughput in a robotic cell. Sometimes, even a small improvement in throughput will be one of the highlighted objectives in robot scheduling, for instance, in the 3C industry of communication, computer and consumer electronics. In this paper, the scheduling problem in an n-workstation (n<;=8) m-stage robotic cell with parallel workstations is discussed by considering the constraints in real engineering practice, including free/non-free pick-up, allowed time window, and free/non-free process. A new method is proposed for optimal scheduling by means of optimally arranging each blocked cycles in every LCM cycle (Least Common Multiple), and assembling them in a proper order. This method is applied to a practical scheduling problem for a 7-workstation robotic cell. It is shown that the throughput is dramatically enhanced after optimization in the engineering scenario. Moreover, by comparing the resultant increase in revenue with the additional equipment costs, the result diagrams also provide managerial insights into links between throughput and cell layout flexibility.