Monitoring Earth's evolving land covers requires methods capable of detecting changes across a wide range of categories and contexts. Existing change detection methods are hindered by their dependency on predefined classes, reducing their effectiveness in open-world applications. To address this issue, we introduce open-vocabulary change detection (OVCD), a novel task that bridges vision and language to detect changes across any category. Considering the lack of high-quality data and annotation, we propose two training-free frameworks, M-C-I and I-M-C, which leverage and integrate off-the-shelf foundation models for the OVCD task. The insight behind the M-C-I~framework is to discover all potential changes and then classify these changes, while the insight of I-M-C~framework is to identify all targets of interest and then determine whether their states have changed. Based on these two frameworks, we instantiate to obtain several methods, e.g., SAM-DINOv2-SegEarth-OV, Grounding-DINO-SAM2-DINO, etc. Extensive evaluations on 4 benchmark datasets demonstrate the superior generalization and robustness of our OVCD methods over existing supervised and unsupervised methods. To support continued exploration, we release DynamicEarth, a dedicated codebase designed to advance research and application of OVCD.

Current remote sensing semantic segmentation methods are mostly built on the close-set assumption, meaning that the model can only recognize pre-defined categories that exist in the training set. However, in practical Earth observation, there are countless new categories, and manual annotation is impractical. To address this challenge, we first attempt to introduce training-free1open-vocabulary semantic segmentation (OVSS) into the remote sensing context. However, due to the sensitivity of remote sensing images to low-resolution features, distorted target shapes and ill-fitting boundaries are exhibited in the prediction mask. To tackle these issues, we propose a simple and universal upsampler, i.e. SimFeatUp, to restore lost spatial information of deep features. Specifically, SimFeatUp only needs to learn from a few unlabeled images, and can upsample arbitrary remote sensing image features. Furthermore, based on the observation of the abnormal response

Mining multiple longest common subsequences (MLCS) from a set of sequences of three or more over a finite alphabet$\Sigma$(a classical NP-hard problem [1]) is an important task in many fields, e.g., bio-informatics, computational genomics, pattern recognition, information extraction, etc. Applications in these fields often involve generating very long sequences (length$\geq 10_{,}000)$, referred to as big sequences. However, both existing exact and approximate MLCS algorithms face severe challenges in handling big sequences due to the over-whelming size of their problem-solving graph model MLCS­-$DAG$(Directed Acyclic Graph), leading to the issue of memory explosion or extremely high time complexity.

We present Open-CD, a change detection toolbox that contains a rich set of change detection methods as well as related components and modules. The toolbox started from a series of open source general vision task tools, including OpenMMLab Toolkits, PyTorch Image Models (Timm), etc. It gradually evolves into a unified platform that covers many popular change detection methods and contemporary modules. It not only includes training and inference codes, but also provides some useful scripts for data analysis. We believe this toolbox is by far the most comprehensive change detection toolbox. In this report, we introduce the features, supported methods and applications of Open-CD. In addition, we also conduct a benchmarking study on different methods and components. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to re-implement existing methods and develop their own new change detectors. Code and models are available at https://github.com/likyoo/open-cd.

The detection of human objects can be crucial for various real-world applications, such as surveillance and autonomous driving. However, traditional vision-based approaches suffer from limitations such as low lighting conditions, occlusions, and privacy concerns. To address these challenges, we introduce mmYodar+, a novel mmWave-based automatic human detection system. Our system processes mmWave signals to generate a 3D point cloud, which is then transformed into a 2D radar image for easier visualization and analysis. To enhance human profiling, we filter the point cloud using biometric information and expand human-related points in the image based on radar angle resolution, incorporating color to improve the differentiation. Additionally, we employ a deep mutual learning (DML) framework, enabling efficient human detection using a lightweight DNN. Experimental results show that mmYodar+ achieves an average precision of 96.29% in various scenarios, including indoor and outdoor environments, various lighting conditions, and in the presence of occlusions. These results demonstrate the effectiveness of using mmWave radar signals for reliable and accurate human detection.

Change detection (CD) aims to identify pixels with semantic changes between images. However, annotating massive numbers of pixel-level images is labor-intensive and costly, especially for multitemporal images, which require pixel-wise comparisons by human experts. Considering the excellent performance of visual-language models (VLMs) for zero-shot, OV, etc., with prompt-based reasoning, it is promising to utilize VLMs to make better CD under limited labeled data. In this article, we propose a VLM guidance-based semi-supervised CD method, namely SemiCD-VL. The insight of SemiCD-VL is to synthesize free change labels using VLMs to provide additional supervision signals for unlabeled data. However, almost all current VLMs are designed for single-temporal images and cannot be directly applied to bi- or multitemporal images. Motivated by this, we first propose a VLM-based mixed change event generation (CEG) strategy to yield pseudo-labels for unlabeled CD data. Since the additional supervised signals provided by these VLM-driven pseudo-labels may conflict with the original pseudo-labels from the consistency regularization paradigm (e.g., FixMatch), we propose the dual projection head for de-entangling different signal sources. Further, we explicitly decouple the bitemporal images semantic representation through two auxiliary segmentation decoders, which are also guided by VLM. Finally, to make the model more adequately capture change representations, we introduce contrastive consistency regularization (CCR) by constructing feature-level contrastive loss in auxiliary branches. Extensive experiments show the advantage of SemiCD-VL. For instance, SemiCD-VL improves the FixMatch baseline by$+ 5.3~\text {IoU}^{c}$on WHU-CD and by$+ 2.4~\text {IoU}^{c}$on LEVIR-CD with 5% labels, and SemiCD-VL requires only 5%–10% of the labels to achieve performance similar to the supervised methods. In addition, our CEG strategy, in an unsupervised manner, can achieve performance far superior to state-of-the-art (SOTA) unsupervised CD methods (e.g., IoU improved from 18.8% to 46.3% on LEVIR-CD dataset). The code is available athttps://github.com/likyoo/SemiCD-VL.

Mining multiple longest common subsequences (MLCS) from a set of sequences of length three or more over a finite alphabet (a classical NP-hard problem) is an important task in many fields, e.g., bioinformatics, computational genomics, pattern recognition, information extraction, etc. Applications in these fields often involve generating very long sequences (length$\geqslant$10,000), referred to as big sequences. Despite efforts in improving the time and space complexities ofMLCSmining algorithms, both existing exact and approximate algorithms face challenges in handling big sequences due to the overwhelming size of their problem-solving graph modelMLCS-DAG(DirectedAcyclicGraph), leading to the issue of memory explosion or extremely high time complexity. To bridge the gap, this paper first proposes a new identification and deletion strategy for different classes of non-critical points in the mining ofMLCS, which are the points that do not contribute to theirMLCSs mining in theMLCS-DAG. It then proposes a newMLCSproblem-solving graph model, namely$DAG_{KP}$(a newMLCS-DAGcontaining onlyKeyPoints). A novel parallelMLCSalgorithm, calledKP-MLCS(KeyPoint basedMLCS), is also presented, which can mine and compress allMLCSs of big sequences effectively and efficiently. Extensive experiments on both synthetic and real-world biological sequences show that the proposed algorithmKP-MLCSdrastically outperforms the existing state-of-the-artMLCSalgorithms in terms of both efficiency and effectiveness.

Contactless mmWave-based human activity recognition (HAR) is essential for various applications, yet most existing approaches often assume consistent environments. Integrating domain adaptation offers a promising solution to this challenge. This prevailing paradigm works well when the source and target data are centralized on a single server while learning to adapt. However, in more universal and practical situations, such as personal health records, users’ biometric information, and financial issues, the raw data is typically protected by different privacy-preserving policies and is stored by multiple parties. Additionally, labeling RF signals in the target domain is a non-trivial and labor-intensive task for most end-users. To address these problems, this paper introduces FMDA, a federated multi-source domain adaptation framework for mmWave-based HAR. FMDA assesses the contribution of each source and performs weighted parameter aggregation for knowledge transfer. This facilitates unsupervised training of the target HAR model without requiring access to any source domain data. Moreover, the model is optimized by minimizing the generalization gaps between the source and target models, benefiting all participants during the learning process and enhancing overall performance. Extensive experiments demonstrate the effectiveness of FMDA. The results indicate that in the target domain, FMDA achieves comparable performance to supervised learning approaches, while also enhancing the efficacy of source domain models to varying degrees.

Falls pose a significant risk to the health and wellbeing of older adults, driving the development of various fall detection systems. Existing solutions have explored wearable and vision sensors, while non-invasive RF-based approaches have raised a growing interest due to their convenience and privacy considerations. Despite major advancements in RF-based passive estimation, current approaches still face challenges in handling complex real-world scenarios. They often lack the ability to generalize to new domains (i.e., people, position, environment), and struggle to accurately detect and localize a fallen person in the presence of unknown activities from nearby objects (e.g., pet animal and robot vacuum cleaner) or persons. To address these challenges, we present mm-Fall, a novel mmWave-based non-invasive fall detection system that utilizes Range-Angle (RA) energy maps to separate and localize multiple moving targets, and further accurately estimate their states. Unlike previous approaches, mm-Fall is capable of working with new domains and effectively distinguishing falls from non-fall motions that may appear similar. Additionally, it performs well in challenging conditions, such as poor lighting and occluded scenarios. Our design of mm-Fall is evaluated in 13 environments with over 16 individuals performing 24+ types of motions. The results demonstrate an impressive average recall of 0.969 and precision of 0.996 in detecting falls, whether involving single or multiple moving targets simultaneously. The code and dataset will be made publicly available.

RFID-based gesture recognition has gained considerable attention in recent years due to the cost-effectiveness of RFID tags and their advantages in preserving visual privacy, providing convenience to users. Existing RFID-based gesture recognition systems typically require users to collect a large amount of training data for each gesture class, and once a new class is introduced, the entire recognition model needs to be retrained. This greatly limits their scalability for new gestures. In this article, we propose FewSense, a practical RFID sensing system that achieves accurate gesture recognition with a small number of training samples. To provide sufficient training samples for FewSense, we introduce a virtual sample generation method to achieve data augmentation. Based on the augmented training data, FewSense enables few-shot gesture recognition. With the introduction of a fine-tuning mechanism, FewSense can easily adapt to changing gesture classes. Real-world experiments demonstrate that even with only seven training samples, FewSense achieves 90% recognition accuracy.

This paper introduces a pioneering framework for music representation learning, which harnesses knowledge graph embeddings to enrich genre classification. Leveraging metadata from publicly available datasets like FMA and OpenMIC-2018, the constructed knowledge graph delineates intricate relationships among genres, artists, and instruments, offering valuable insights for genre representation. Within this framework, we propose two models tailored for distinct genre classification scenarios: fixed-set genre classification and open-set genre classification. These models exploit the knowledge graph to unveil correlations among different genres and integrate this knowledge into the audio representation. Notably, our approach is the first to merge audio data with high-level knowledge for music genre classification. Experimental results demonstrate that our proposed methods outperform state-of-the-art approaches, achieving an average genre classification accuracy of 68.07% on the FMA-medium dataset and 42.4% for open-set classification on the FMA-large dataset.

With the popularity of LED infrastructure and the camera on smartphone, LED-Camera visible light communication (VLC) has become a realistic and promising technology. However, the existing LED-Camera VLC has limited throughput due to the sampling manner of camera. In this paper, by introducing a polarization dimension, we propose a hybrid modulation scheme with LED and polarization signals to boost throughput. Nevertheless, directly mixing LED and polarized signals may suffer from channel conflict. We exploit well-designed packet structure and Symmetric Return-to-Zero Inverted (SRZI) coding to overcome the conflict. In addition, in the demodulation of hybrid signal, we alleviate the noise of polarization on the LED signals by the polarization background subtraction. We further propose a pixel-free approach to correct the perspective distortion caused by the shift of view angle by adding polarizers around the liquid crystal array. We build a prototype of this hybrid modulation scheme using off-the-shelf optical components. We enhance the basic version (Zou et al. 2023) of preliminary work by analyzing the performance with FSK modulation. Extensive experimental results demonstrate that the hybrid modulation scheme can achieve reliable communication, achieving 13.4 kbps throughput, which is 400$\%$of the existing state-of-the-art LED-Camera VLC.

Rodent infestation is a great danger to human society, continuously threatening food safety and inducing disease spread. Existing methods to deal with rodent infestation are mainly based on passive bait traps and poisoning. These methods lack timeliness and effectiveness due to the missing of real-time detection. In this paper, we develop WiHunter, a new wireless sensing system to discover small objects (e.g., rat). Our idea is to exploit reflection signal effect of wireless channels induced by the movement of small objects around the receiver antenna. However, existing wireless sensing works usually employ customized or costly device-dependency Network Interface Cards(NIC), which are impractical to be densely deployed in reality. We implement WiHunter with several CSI-enabled standalone IoT nodes. We show how such devices enable moving small object detection via WiFi signal. The rationale behind this is 1) thanks to the widespread deployments of WiFi infrastructures and IoT devices, the WiFi signal covers almost every location of the corner, 2) the signal amplitude of each device is related to the small object near the receiver antenna. This ability gives us the opportunity to sense object as small as a rat. We implement WiHunter with ESP32 microcontroller on Espressif IoT Development Framework (both of them are cheap commodity off-the-shelf (COTS) devices) and design a practical small object intrusion detection system. Comprehensive and real-world experiments demonstrate that our system is effective in detecting the presence of small objects with an average accuracy of 92.1%.

In recent years, the next-POI recommendation has become a trending research topic in the field of trajectory data mining. For protection of user privacy, users’ complete GPS trajectories are difficult to obtain. The check-in information posted by users on social networks has become an important data source for Spatio-temporal Trajectory research. However, state-of-the-art methods neglect the social meaning and the information dissemination function of check-in behavior. The social meaning is an important reason why users are willing to post check-in on social networks, and the information dissemination function means, users can affect each other’s behavior by check-ins. The above characteristics of the check-in behavior make it different from the visiting behavior. We consider a new problem of predicting the next check-in behavior including the check-in time, the POI (point-of-interest) where the check-in is located, functional semantics of the POI, and so on. To solve the proposed problem, we build a multi-task learning model called DPMTM, and a pre-training module is designed to extract dynamic social semantics of check-in behaviors. Our results show that the DPMTM model works well in the check-in behavior problem.

Many big data applications produce a massive amount of high-dimensional, real-time, and evolving streaming data. Clustering such data streams with both effectiveness and efficiency are critical for these applications. Although there are well-known data stream clustering algorithms that are based on the popular online-offline framework, these algorithms still face some major challenges. Several critical questions are still not answer satisfactorily: How to perform dimensionality reduction effectively and efficiently in the online dynamic environment? How to enable the clustering algorithm to achieve complete real-time online processing? How to make algorithm parameters learn in a self-supervised or self-adaptive manner to cope with high-speed evolving streams? In this paper, we focus on tackling these challenges by proposing a fully online data stream clustering algorithm (called ESA-Stream) that can learn parameters online dynamically in a self-adaptive manner, speedup dimensionality reduction, and cluster data streams effectively and efficiently in an online and dynamic environment. Experiments on a wide range of synthetic and real-world data streams show that ESA-Stream outperforms state-of-the-art baselines considerably in both effectiveness and efficiency.

With ever-increasing data streams from various applications such as smart phones, network monitoring, Internet of Things (IoT), etc., unsupervised clustering of data streams has become an important problem for machine learning and big data analysis. As data streams are data-intensive, temporally ordered, and rapidly evolving, efficiently and effectively online clustering of data streams presents a challenging problem [1] .

U-Net has been considered as an outstanding deep learning neural network in medical image segmentation problems. The segmentation results of the U-Net based model, however, are always too conservative and smooth. MufiNet, a segmentation model using multiple U-Net chains (with multiple encoder-decoder branches), is proposed in this paper. It can fuse the receptive fields obtained from different scales. The convolution layer of 1 × 1 is introduced to add the residual connection to enhance the adaptability to the depth of the network. The multi-scale fusion module with residuals is combined with the U-Net chain architecture to retain more information flow paths, and the multi-scale context information is used to improve the performance and robustness of the segmented network. MufiNet model is extensively evaluated on three datasets in this paper, including two benchmark datasets (lung segmentation and skin cancer lesion segmentation) and cervical cancer dataset jointly constructed with a hospital. The experimental results show that MufiNet could yield better performance in medical image segmentation tasks than U-Net and LadderNet models.

With rapid development and popularity, smartphones have been of importance in our daily life. Despite of its convenience in communication and computing, smartphones also lead potential security threats to users. Existing methods on smartphones for protecting user's privacy mainly depend on password or fingerprint based authentication. Most smartphone passwords are very simple and easy to guess or crack, and fingerprinting requires extra hardware and hence increases the price of smartphones. In this paper, we present a smartphone-based identity authentication method based on user's lip motion characteristics, called SALM, which can be used as an additional authentication with password. SALM extracts the feature of lip movements as the authentication token, which is unique for each user. We implement SALM using off-the-shelf smartphones and evaluate its performance via extensive experiments. The results show that the overall accuracy of user authentication using SALM (without password) is higher than 96%.

Information in various applications is often expressed as character sequences over a finite alphabet (e.g., DNA or protein sequences). In Big Data era, the lengths and sizes of these sequences are growing explosively, leading to grand challenges for the classical NP-hard problem, namely searching for the Multiple Longest Common Subsequences (MLCS) from multiple sequences. In this paper, we first unveil the fact that the state-of-the-art MLCS algorithms are unable to be applied to long and large-scale sequences alignments. To overcome their defects and tackle the longer and large-scale or even big sequences alignments, based on the proposed novel problem-solving model and various strategies, e.g., parallel topological sorting, optimal calculating, reuse of intermediate results, subsection calculation and serialization, etc., we present a novel parallel MLCS algorithm. Exhaustive experiments on the datasets of both synthetic and real-world biological sequences demonstrate that both the time and space of the proposed algorithm are only linear in the number of dominants from aligned sequences, and the proposed algorithm significantly outperforms the state-of-the-art MLCS algorithms, being applicable to longer and large-scale sequences alignments.

Multi-user MIMO (MU-MIMO) is proposed in 802.11ac to achieve more than 3x faster than 802.11n. In the real world no-one gets close to theoretical speeds. The primary reason for this anomaly are the various overheads of channel access and channel state information (CSI) feedback. In order to achieve concurrent data transmission, (CSI) feedback from users is required. However, this overhead can easily overwhelm the actual channel time spent on data transmission in large-scale network. Moreover, due to spontaneous uplink traffic, which makes the problem even more challenging.

Activity recognition plays an important role in human-computer interactions. Recently, Channel State Information (CSI), known as a fine-grained information capturing the properties of WiFi signal propagation, has been widely used for activity recognition in a device-free pattern. Since CSI is much sensitive to ambient changes, CSI can be used as fingerprints as human activities. However, existing approaches require tremendous overhead in the model training and suffer from failures due to environmental interferences. In this paper, we propose HAR, a CSI based human activity recognition system. HAR investigates the CSI intra-correlation structure (termed as topics) of different human activities. We leverage an unsupervised machine learning method, namely topic model, to extract action characters. Compared to prior works, HAR only requests minor manual intervention, significantly reducing manpower costs in the model training. We implement HAR using commodity WiFi devices to evaluate its performance under different environment settings. The results show that the extracted features are stable to different devices and volunteers, facilitating HAR to achieving an average matching accuracy, i.e., > 90%.

Different from online shopping, in-store shopping has few ways to collect the customer behaviors before purchase. In this paper, we present the design and implementation of an on-site Customer Behavior IDentification system based on passive RFID tags, named CBID. By collecting and analyzing wireless signal features, CBID can detect and track tag movements and further infer corresponding customer behaviors. We model three main objectives of behavior identification by concrete problems and solve them using novel protocols and algorithms. The design innovations of this work include a Doppler effect based protocol to detect tag movements, an accurate Doppler frequency estimation algorithm, an image-based human count estimation protocol and a tag clustering algorithm using cosine similarity. We have implemented a prototype of CBID in which all components are built by off-the-shelf devices. We have deployed CBID in real environments and conducted extensive experiments to demonstrate the accuracy and efficiency of CBID in customer behavior identification.

Device-free object tracking provides a promising solution for many localization and tracking systems to monitor non-cooperative objects, such as intruders, which do not carry any transceiver. However, existing device-free solutions mainly use special sensors or active RFID tags, which are much more expensive compared to passive tags. In this paper, we propose a novel motion detection and tracking method using passive RFID tags, named Twins. The method leverages a newly observed phenomenon called critical state caused by interference among passive tags. We contribute to both theory and practice of this phenomenon by presenting a new interference model that precisely explains it and using extensive experiments to validate it. We design a practical Twins based intrusion detection system and implement a real prototype by commercial off-the-shelf RFID reader and tags. Experimental results show that Twins is effective in detecting the moving object, with very low location errors of 0.75 m in average (with a deployment spacing of 0.6 m).

Different from online shopping, in-store shopping has few ways to collect the customer behaviors before purchase. In this paper, we present the design and implementation of an on-site Customer Behavior Identification system based on passive RFID tags, named CBID. By collecting and analyzing wireless signal features, CBID can detect and track tag movements and further infer corresponding customer behaviors. We model three main objectives of behavior identification by concrete problems and solve them using novel protocols and algorithms. The design innovations of this work include a Doppler effect based protocol to detect tag movements, an accurate Doppler frequency estimation algorithm, a multi-RSS based tag localization protocol, and a tag clustering algorithm using cosine similarity. We have implemented a prototype of CBID in which all components are built by off-the-shelf devices. We have deployed CBID in real environments and conducted extensive experiments to demonstrate the accuracy and efficiency of CBID in customer behavior identification.

Although fingerprint based localization is promising for indoor applications, its accuracy still remains a huge challenge. Most of existing approaches rely on the Radio Signal Strength (RSS) to generate fingerprints. However, merely using RSS is unable to accurately localize objects since such an one-dimensional fingerprint will be seriously influenced by the interference and multi-path effect in the indoor environment. In this paper, we propose a new localization approach based on multidimensional Wi-Fi fingerprint. Instead of only using RSS to construct fingerprint, we employ RSS, transmitted power, and channel information to construct an integrated fingerprint. The extended fingerprint enables fine-grained localization and tracking services. We also deign a cosine similarity based matching algorithm and enhanced particle filter mechanism to achieve accurate localization and tracking. Extensive experiment and implementation results show that the new fingerprint and proposed algorithms can achieve an accuracy within two meters in 90% of testing points, while demonstrating a good adaptability to complex indoor environments.

Locating objects labeled with RFID tags is an important issue which should be addressed in many applications, such as warehouse management, goods management in supermarket and finding of lost objects. Some existing works use large numbers of reference tags which involve lots of manpower to deploy them. Others achieve high accuracy, but rely on sophisticated equipments which are hardly available in large scale to the industry. This work exploits the radiation pattern of existing directional panel antenna which is steerable and derives angle-of-arrival (AoA) information from the energy reflected by the target tag when the antenna is rotating. We use Commercial Off-The-Shelf (COTS) equipments and get median position accuracy of 29cm in our preliminary experiment.

In this paper, we develop novel physical-layer identification schemes for passive Radio Frequency IDentification (RFID) tags. Due to the collision among tags, existing RFID systems suffer from a low identification efficiency. In this paper, we propose to use the unique physical-layer information of tags as the identification basis. We design a batch identification scheme for passive tags. Our Scheme can fully utilize the collided signals to achieve efficient and trustworthy identification. Leveraging collided signals, we also propose an AoA-based spatial identification scheme for providing location service. Our scheme are seamlessly compatible with commercial off-the-shelf RFID devices. The initial result shows the feasibility of our proposals.

Although GPS is deemed as ubiquitous outdoor localization technology, we are still far from a similar technology for indoor environments. Though a number of techniques are proposed for indoor localization, they are separated efforts that are way from a real ubiquitous localization system. Our real-world experience from InSpace, a pervasive computing system with wireless devices to provide intelligent services to users, shows that locating mobile users remains very challenging due to various interfering factors. We analyze real traces of mobile phones carried by users and find that mobile users exhibit temporal-spatial stability and neighborhood relativity. Motivated by this observation, we develop a Mobile Boundary Localization approach, MBL, to exploit the associated information to locate mobile users. This localization approach uses different treatment in different conditions and lets each mobile phone try to estimate its possible location range. We have implemented and evaluated MBL by extensive real-world experiments in InSpace and simulations. The results demonstrate that MBL significantly outperforms state-of-the-art localization approaches with more accurate, efficient, and consistent performance.