CLIP has been widely adopted in affective computing for its strong vision-language representation capabilities. However, it fails to accurately distinguish visually similar yet label-distinct facial expressions. This limitation is rooted in CLIP's encoding paradigm and large-scale contrastive pretraining, which bias the model toward focusing primarily on globally salient visual features and aligning them with broad semantic concepts. Such alignment overlooks subtle facial variations and induces representational shortcuts, where emotionally distinct categories are projected into overlapping regions of the shared semantic space. This semantic entanglement severely compromises the model's ability to preserve emotional separability. We propose LES-CLIP, a Lightweight and Emotion-Sensitive framework that adapts CLIP for precise discrimination of similar emotions. LES-CLIP achieves fine-grained emotional sensitivity using only simple text prompts and facial images. It introduces three novel components: 1) an Emotion-Sensitive Adaptive Mixture-of-Experts, which pre-adapts representations for subtle expression discrimination; 2) a Prompt-Guided Emotion Discrimination module that activates CLIP's visual sensitivity to fine-grained facial cues; and 3) a LES hybrid loss that guides contrastive learning toward accurate emotion-label alignment. Extensive experiments demonstrate that LES-CLIP achieves state-of-the-art performance, reaching 70.18% on the 8-class AffectNet dataset. Moreover, it converges faster and requires significantly fewer parameters.

Contactless mmWave-based human activity recognition (HAR) is essential for various applications, yet most existing approaches often assume consistent environments. Integrating domain adaptation offers a promising solution to this challenge. This prevailing paradigm works well when the source and target data are centralized on a single server while learning to adapt. However, in more universal and practical situations, such as personal health records, users’ biometric information, and financial issues, the raw data is typically protected by different privacy-preserving policies and is stored by multiple parties. Additionally, labeling RF signals in the target domain is a non-trivial and labor-intensive task for most end-users. To address these problems, this paper introduces FMDA, a federated multi-source domain adaptation framework for mmWave-based HAR. FMDA assesses the contribution of each source and performs weighted parameter aggregation for knowledge transfer. This facilitates unsupervised training of the target HAR model without requiring access to any source domain data. Moreover, the model is optimized by minimizing the generalization gaps between the source and target models, benefiting all participants during the learning process and enhancing overall performance. Extensive experiments demonstrate the effectiveness of FMDA. The results indicate that in the target domain, FMDA achieves comparable performance to supervised learning approaches, while also enhancing the efficacy of source domain models to varying degrees.

Falls pose a significant risk to the health and wellbeing of older adults, driving the development of various fall detection systems. Existing solutions have explored wearable and vision sensors, while non-invasive RF-based approaches have raised a growing interest due to their convenience and privacy considerations. Despite major advancements in RF-based passive estimation, current approaches still face challenges in handling complex real-world scenarios. They often lack the ability to generalize to new domains (i.e., people, position, environment), and struggle to accurately detect and localize a fallen person in the presence of unknown activities from nearby objects (e.g., pet animal and robot vacuum cleaner) or persons. To address these challenges, we present mm-Fall, a novel mmWave-based non-invasive fall detection system that utilizes Range-Angle (RA) energy maps to separate and localize multiple moving targets, and further accurately estimate their states. Unlike previous approaches, mm-Fall is capable of working with new domains and effectively distinguishing falls from non-fall motions that may appear similar. Additionally, it performs well in challenging conditions, such as poor lighting and occluded scenarios. Our design of mm-Fall is evaluated in 13 environments with over 16 individuals performing 24+ types of motions. The results demonstrate an impressive average recall of 0.969 and precision of 0.996 in detecting falls, whether involving single or multiple moving targets simultaneously. The code and dataset will be made publicly available.

There have been increasing interests in exploring the sensing capabilities of RFID to enable numerous IoT applications, including object localization, trajectory tracking, and human behavior sensing. However, most existing methods rely on the signal measurement either in a low multipath environment, which is unlikely to exist in many practical situations, or with special devices, which increase the operating cost. This paper investigates the possibility of measuring ‘multi-path-free’ signal information in multipath-prevalent environments simply using a commodity RFID reader. The proposed solution, Clean Physical Information Extraction (CPIX), is universal, accurate, and compatible to standard protocols and devices. CPIX improves RFID sensing quality with near zero cost – it requires no extra device. We implement CPIX and study three major RFID sensing applications: tag localization, device calibration and human behavior sensing. CPIX reduces the localization error by 30% to 50% and achieves the MOST accurate localization by commodity readers compared to existing work. It also significantly improves the quality of device calibration and human behaviour sensing.

We study a new problem, refined localization, in this article. Refined localization calculates the location of an object in high precision, given that the object is in a relatively small region such as the surface of a table. Refined localization is useful in many cyber–physical systems such as industrial autonomous robots. Existing vision-based approaches suffer from several disadvantages, including good lighting conditions, line of sight, prelearning process, and high computation overhead. Also, vision-based approaches cannot differentiate objects with similar colors and shapes. This article presents a new refined localization system, called Trio, which uses passive radio frequency identification (RFID) tags for low cost and easy deployment. Trio utilizes RF interference for tag localization by modeling the equivalent circuits of coupled tags. We implement our prototype using commercial off-the-shelf RFID reader and tags. Extensive experiment results demonstrate that Trio effectively achieves high accuracy of refined localization, i.e., < 1 cm errors for several types of main stream tags.

The research on Wi-Fi sensing has been thriving over the past decade but the process has not been smooth. Three barriers always hamper the research: 1) unknown baseband design and its influence; 2) inadequate hardware; and 3) the lack of versatile and flexible measurement software. This article tries to eliminate these barriers through the following work.First, we present an in-depth study of the baseband design of the Qualcomm Atheros AR9300 (QCA9300) NIC. We identify a missing item of the existing channel state information (CSI) model, namely, the CSI distortion, and identify the baseband filter as its origin. We also propose a distortion removal method.Second, we reintroduce both the QCA9300 and software-defined radio (SDR) as powerful hardware for research. For the QCA9300, we unlock the arbitrary tuning of both the carrier frequency and bandwidth. For SDR, we develop a high-performance software implementation of the 802.11a/g/n/ac/ax baseband, allowing users to fully control the baseband and access the complete physical-layer information.Third, we release the PicoScenes software, which supports concurrent CSI measurement from multiple QCA9300, Intel Wireless Link (IWL5300), and SDR hardware. PicoScenes features rich low-level controls, packet injection, and software baseband implementation. It also allows users to develop their own measurement plugins.Finally, we report state-of-the-art results in the extensive evaluations of the PicoScenes system, such as the >2-GHz available spectrum on the QCA9300, concurrent CSI measurement, and up to 40 and 1 kHz CSI measurement rates achieved by the QCA9300 and SDR. PicoScenes is available athttps://ps.zpj.io.

Security over mobile Internet-of-Things (IoT) devices is critical due to the open nature of distributed wireless communication. To efficiently establish a secure connection between two communication parties, a fast mobile key extraction protocol, KEEP, is proposed. KEEP fastly generates similar bit sequences from two communication parties’ measurements of channel-state information (CSI) of different subcarriers. Then, a distributed “verification-recombination” mechanism is introduced to generate the same encryption key from bit sequences without the public-key authentication, digital signature, or key distribution center of the other party. We implemented real-world experiments using commercial off-the-shelf 802.11n devices to evaluate the performance of KEEP in various scenarios. Theoretical analysis and experimental verification show that KEEP is more secure, effective, and reliable than the state-of-the-art methods.

An important function of smart environments is the ubiquitous access of computing devices. In public areas such as hospitals, libraries, and airports, people may want to interact with nearby computing systems to get information, such as directions to a hospital room, locations of books, and flight departure/arrival information. Touch screen based displays and kiosks, which are commonly used today, may incur extra hardware cost or even possible germ and bacteria infection. This work provides a new solution: users can make queries and inputs by performing in-air handwriting to an array of passive RFID tags, named RFIPad. This input method does not require human hands to carry any device and hence is convenient for applications in public areas. Besides the mobile and contactless property, this system is a cost-efficient extension to current RFID systems: an existing reader can monitor multiple RFIPads while performing its regular applications such as identification and tracking. We implement a prototype of RFIPad using commercial off-the-shelf UHF RFID devices. Experimental results show that RFIPad achieves >91% accuracy in recognizing basic touch-screen operations and English letters.

Device-to-device communication is important to emerging mobile applications such as Internet of Things and mobile social networks. Authentication and key agreement among multiple legitimate devices is the important first step to build a secure communication channel. Existing solutions put the devices into physical proximity and use the common radio environment as a proof of identities and the common secret to agree on a same key. However they experience very slow secret bit generation rate and high errors, requiring several minutes to build a 256-bit key. In this work, we design and implement an authentication and key agreement protocol for mobile devices, called The Dancing Signals (TDS), being extremely fast and error-free. TDS uses channel state information (CSI) as the common secret among legitimate devices. It guarantees that only devices in a close physical proximity can agree on a key and any device outside a certain distance gets nothing about the key. Compared with existing solutions, TDS is very fast and robust, supports group key agreement, and can effectively defend against predictable channel attacks. We implement TDS using commodity off-the-shelf 802.11n devices and evaluate its performance via extensive experiments. Results show that TDS only takes a couple of seconds to make devices agree on a 256-bit secret key with high entropy.

Identifying the object that attracts human visual attention is an essential function for automatic services in smart environments. However, existing solutions can compute the gaze direction without providing the distance to the target. In addition, most of them rely on special devices or infrastructure support. This paper explores the possibility of using a smartphone to detect the visual attention of a user. By applying the proposed VADS system, acquiring the location of the intended object only requires one simple action: gazing at the intended object and holding up the smartphone so that the object as well as user's face can be simultaneously captured by the front and rear cameras. We extend the current advances of computer vision to develop efficient algorithms to obtain the distance between the camera and user, the user's gaze direction, and the object's direction from camera. The object's location can then be computed by solving a trigonometric problem. VADS has been prototyped on commercial off-the-shelf (COTS) devices. Extensive evaluation results show that VADS achieves low error (about 1.5° in angle and 0.15m in distance for objects within 12m) as well as short latency. We believe that VADS enables a large variety of applications in smart environments.

Multi-user MIMO (MU-MIMO) is proposed in 802.11ac to achieve more than 3x faster than 802.11n. In the real world no-one gets close to theoretical speeds. The primary reason for this anomaly are the various overheads of channel access and channel state information (CSI) feedback. In order to achieve concurrent data transmission, (CSI) feedback from users is required. However, this overhead can easily overwhelm the actual channel time spent on data transmission in large-scale network. Moreover, due to spontaneous uplink traffic, which makes the problem even more challenging.

Activity recognition plays an important role in human-computer interactions. Recently, Channel State Information (CSI), known as a fine-grained information capturing the properties of WiFi signal propagation, has been widely used for activity recognition in a device-free pattern. Since CSI is much sensitive to ambient changes, CSI can be used as fingerprints as human activities. However, existing approaches require tremendous overhead in the model training and suffer from failures due to environmental interferences. In this paper, we propose HAR, a CSI based human activity recognition system. HAR investigates the CSI intra-correlation structure (termed as topics) of different human activities. We leverage an unsupervised machine learning method, namely topic model, to extract action characters. Compared to prior works, HAR only requests minor manual intervention, significantly reducing manpower costs in the model training. We implement HAR using commodity WiFi devices to evaluate its performance under different environment settings. The results show that the extracted features are stable to different devices and volunteers, facilitating HAR to achieving an average matching accuracy, i.e., > 90%.

Efficient and accurate tracking of device-free objects is critical for anti-intrusion systems. Prior solutions for device-free object tracking are mainly based on costly sensing infrastructures, resulting in barriers to practical applications. In this paper, we propose an accurate and efficient motion detection system, named EMoD, to track device-free objects based on cheap passive RFID tags. EMoD is the first RFID system that can estimate the moving direction as well as the current location of a device-free object by measuring critical power variation sequences of passive tags. Compared with previous solutions, the unique advantage of EMoD, i.e., the capability to estimate moving directions, enables object tracking using a much sparser tag deployment. We contribute to both theory and practice of this phenomenon by presenting the interference model that precisely explains it and using extensive experiments to validate it. We design a practical EMoD based intrusion detection system and implement a prototype by commercial off-the-shelf (COTS) RFID reader and tags. The real-world experiments results show that EMoD is effective in tracking the trajectory of moving object in various environments.

Indoor localization is of importance for many applications. Crowdsourcing individual users' measurements can provide accurate localization without costly site-survey. However, crowdsourcing based approaches suffer from the cold start problem, in which at the beginning of system deployment, there are insufficient users to contribute their measurements, resulting in inaccurate and time-inefficient localization. In this paper, we propose a hybrid indoor localization method to solve such problem, called ACIL. We first employ the inertial navigation technique to localize some core positions or paths. To tackle the inaccuracy problem, we propose an effective method that utilizes the channel state information (CSI) of wireless signals for accurate distance estimation. This method is based on a new observation: there is a ripple-like fading pattern in wireless signals upon moving objects. Leveraging this observation, our system is capable of calculating the distance of human's movement and his/her direction. We also propose a graph-matching algorithm to setup the correlation between the trajectory and floor map. With those extra obtained location information, the impact of cold start issue will be significantly mitigated, while the LBS can be guaranteed with high localization accuracy. Extensive experiments show that the effectiveness in the human localization and movement detection. Extensive experiments validate the great performance of our protocol in case of various human locations and diverse channel conditions.

Although fingerprint based localization is promising for indoor applications, its accuracy still remains a huge challenge. Most of existing approaches rely on the Radio Signal Strength (RSS) to generate fingerprints. However, merely using RSS is unable to accurately localize objects since such an one-dimensional fingerprint will be seriously influenced by the interference and multi-path effect in the indoor environment. In this paper, we propose a new localization approach based on multidimensional Wi-Fi fingerprint. Instead of only using RSS to construct fingerprint, we employ RSS, transmitted power, and channel information to construct an integrated fingerprint. The extended fingerprint enables fine-grained localization and tracking services. We also deign a cosine similarity based matching algorithm and enhanced particle filter mechanism to achieve accurate localization and tracking. Extensive experiment and implementation results show that the new fingerprint and proposed algorithms can achieve an accuracy within two meters in 90% of testing points, while demonstrating a good adaptability to complex indoor environments.

Crowd counting, which count or accurately estimate the number of human beings within a region, is critical in many applications, such as guided tour, crowd control and marketing research and analysis. A crowd counting solution should be scalable and be minimally intrusive (i.e., device-free) to users. Image-based solutions are device-free, but cannot work well in a dim or dark environment. Non-image based solutions usually require every human being carrying device, and are inaccurate and unreliable in practice. In this paper, we present FCC, a device-Free Crowd Counting approach based on Channel State Information (CSI). Our design is motivated by our observation that CSI is highly sensitive to environment variation, like a frog eye. We theoretically discuss the relationship between the number of moving people and the variation of wireless channel state. A major challenge in our design of FCC is to find a stable monotonic function to characterize the relationship between the crowd number and various features of CSI. To this end, we propose a metric, the Percentage of nonzero Elements (PEM), in the dilated CSI Matrix. The monotonic relationship can be explicitly formulated by the Grey Verhulst Model, which is used for crowd counting without a labor-intensive site survey. We implement FCC using off-the-shelf IEEE 802.11n devices and evaluate its performance via extensive experiments in typical real-world scenarios. Our results demonstrate that FCC outperforms the state-of-art approaches with much better accuracy, scalability and reliability.

Device to device (D2D) communication is expected to become a promising technology of the next-generation wireless communication systems. Security issues have become technical barriers of D2D communication due to its “open-air” nature and lack of centralized control. Generating symmetric keys individually on different communication parties without key exchange or distribution is desirable but challenging. Recent work has proposed to extract keys from the measurement of physical layer random variations of a wireless channel, e.g., the channel state information (CSI) from orthogonal frequency-division multiplexing (OFDM). Existing CSI-based key extraction methods usually use the measurement results of individual subcarriers. However, our real world experiment results show that CSI measurements from near-by subcarriers have strong correlations and a generated key may have a large proportion of repeated bit segments. Hence attackers may crack the key in a relatively short time and hence reduce the security level of the generated keys. In this work, we propose a fast secret key extraction protocol, called KEEP. KEEP uses a validation-recombination mechanism to obtain consistent secret keys from CSI measurements of all subcarriers. It achieves high security level of the keys and fast key-generation rate. We implement KEEP using off-the-shelf 802.11n devices and evaluate its performance via extensive experiments. Both theoretical analysis and experimental results demonstrate that KEEP is safer and more effective than the state-of-the-art approaches.