Millimeter-wave (mmWave) radar offers significant potential for fine-grained sensing, yet transitioning from controlled laboratory environments to dynamic real-world applications remains challenging. Existing methods face a dichotomy: real-time point clouds sacrifice crucial signal details needed for sophisticated tasks, whereas information-rich raw data sensing imposes prohibitive transmission and computation overheads, often limiting analysis to offline settings and hindering real-time viability. To this end, we present CoSense, areal-timeedge-end collaborative sensing system built on commodity mmWave radar (end) and edge intelligence. We first introduce a novel dual-stream data acquisition mechanism via realizing radar driver-level interfaces, enabling simultaneous transmission of point clouds and raw data. To bridge the fidelity-latency trade-off, we implement an adaptive transmission strategy via firmware modifications, selectively forwarding raw data segments (corresponding to regions of interest identified in the point cloud) for detailed fine-grained analysis, while continuously delivering point clouds for low-latency coarse-grained sensing and control loops. Furthermore, we incorporate closed-loop feedback beamforming, dynamically steering the radar beam based on real-time tracking to counteract motion-induced misalignment and enhance signal fidelity. Extensive evaluations under dynamic conditions demonstrate that CoSense successfully achieves real-time fine-grained sensing with high fidelity and manageable overhead.

Open vocabulary object detection (OVOD) task aims to detect objects of novel categories beyond the base categories in the training set. To this end, the detector needs to access image-text pairs containing rich semantic information or the visual language pre-trained model (VLM) learned on them. Recent OVOD methods rely on knowledge distillation from VLMs. However, there are two main problems in current methods: (1) Current knowledge distillation frameworks fail to take advantage of the global category information of VLMs and thus fail to learn category-specific knowledge. (2) Due to the overfitting phenomenon of base categories during training, current OVOD networks generally have the problem of suppressing novel categories as background. To address these two problems, we propose a Category Aware Knowledge Extraction framework (CAKE), which consists of a Category-Specific Knowledge Distillation branch (CSKD) and a Category Generalization Region Proposal Network (CG-RPN). CSKD can more fully extract category-strong related information through category-specific distillation, and it is also conducive to filtering the exclusion problem between individuals of the same category; in this process, the model constructs a category-specific feature set to maintain high-quality category features. CG-RPN leverages the guidance of feature set to adjust the confidence scores of region proposals, thereby mining proposals that potentially contain novel categories of objects. Extensive experiments show that our method can plug and play well with many existing methods and significantly improve their detection performance. Moreover, our CAKE framework can reach the-state-of-the-art performance on OV-COCO and OV-LVIS datasets.

With the rise of intelligent systems like assisted driving and robotics, all-weather target identification and 3D localization systems have become crucial for reliable obstacle avoidance and navigation. However, vision-based methods struggle to provide accurate target locations under low light or bad weather. Radar-based solutions like mmWave radar and LiDAR are robust but hindered by high costs and challenges in recognizing target identities at scale. In this paper, we propose alow-cost, all-weather target identification and 3D localization systembased onLED-tags, which system can address the needs of intelligent systems for obstacle avoidance in complex environments. We explore the backscatter communication of LED devices and design adual-modal LED-Tag, which includes two features: a backscatter RF signal detectable by RF devices and visual light spot information detectable by cameras, both sharing the same ID. To enhance the limited backscatter capability, we propose amulti-branch parallel modelthat enhances the signal strength using beamforming synthesis and achannel adjustment mechanismto improve robustness in complex environments, ensuring accurate 3D localization. For multi-target identification, we design an LED-tag encoding system, assigning each tag a unique encoding sequence. Each target's identity can be recognized with our customizedID decoding method, which leverages prior information and time-domain sampling characteristics. Extensive experimental results show that the backscatter communication and target detection range of LED-tags can reach15m. Moreover, the system achieves anaverage localization error of 7.3cm within a 5m range, demonstrating the system's excellent performance in terms of practicality and accuracy.

Nowadays, object detection and 3D tracking have become key technologies for intelligent system or robot navigation to realize automatic obstacle avoidance and target detection, especially in low-light and night vision scenarios. In this paper, we explore the backscattering capability of LEDs and implement a multi-modal tag LED-tag to realize object detection and 3D tracking. Our basic idea is to utilize the fact that feeding modulation signals to an LED-tag can generate both RF and visual features. We fuse the depth of field information perceived from the RF domain and the pixel coordinates obtained from the visual domain to derive a 3D position by matching the decoded ID. In the RF domain, the depth of field is acquired through ultra-wideband channel measurements and estimated phase. In the visual domain, the pixel coordinate in the XOY coordinates can be extract from the image and mapped into 2D spatial coordinates. To address the limited backscatter capability of the LED-tag, we propose a multiple parallel branch model to increase backscatter paths for amplifying the LED-tag's backscattering intensity. Additionally, we propose a decoding ID scheme that utilizes a priori knowledge and repetitive samples to restore the IDs whose encoding frequency is higher than four times the sampling rate. We have implemented a prototype system and evaluated its performance in real-world environments. Extensive experimental results show that LED-tag can backscatter RF signals ranging up to 15m. Besides, the system achieves an average position error of 8cm within the range of 3m.